Social scientists do important work that is often used to make real-world policy decisions. But how do we know that their conclusions are right? Peer review is one important way that we ensure that research employs solid data and analysis methods. Under the peer review system, academic journals require two to three experts in the field to approve an article before it is published. But peer reviewers are not regularly provided with the data used in the paper, nor are these data made available to the public. This means that an outside person cannot rerun the analyses to make sure that the conclusions are sound. In this case, we say that the data are not reproducible.

Because most social scientists have not been required to make our analyses reproducible, many of us have developed sloppy habits. For example, we might recode variables but not write down exactly how we went from the original variable to the new one. Or we might not keep records about a few incomplete cases that we dropped out of our analysis. This kind of shoddy record keeping does not usually compromise the research, nor does it arise from nefarious intentions. We usually do these things because we are in a hurry and we think we will remember what we did. And then, of course, the years pass by and we can no longer remember the details. And even if we could remember, our lack of record keeping makes it difficult (or impossible) for others to replicate what we did.

It’s an exciting moment to be starting in the social sciences because there is growing recognition that good science requires reproducibility. New practices have been developed to help researchers store their data and document how they conducted their analysis. Journals are starting to require that replication packages be provided with all published work. Graduate schools are making similar rules about the submission of dissertations. The Tier Protocol (which is introduced in this exercise) has been developed specifically for the social sciences and is an excellent way to ensure that your data meets current reproducibility standards. By learning this now, you will save yourself a lot of grief later.

To get started on this exercise,  click on the “Introduction to the Exercise” tab on the right hand side of this page.