Scientists learn about the world by collecting and analyzing data. Sometimes, they begin research with a clear idea and conduct tests that they had planned all along. Other times, their research takes unpredicted turns, resulting in unplanned discoveries. These modes of research are often called confirmatory research and exploratory research, respectively.
Many believe that confirmatory results should be more reliable than exploratory discoveries. This is because committing to an idea and a way of testing it in advance prevents researchers from capitalizing on chance when analyzing the data, especially if a certain outcome is desired. Such beliefs have informed research reforms, namely, the spread of preregistration. Preregistration refers to creating a time-stamped version of a researcher’s planned analyses before observing data. This practice delineates between confirmatory and exploratory findings.
However, it is unknown whether confirmatory findings actually are more reliable than exploratory discoveries. We seek to provide initial answers to this question through 1) a randomized experiment that will assign researchers to engage in confirmatory or exploratory research when investigating a particular question and 2) a more naturalistic observation of researchers conducting exploratory and confirmatory research in a large, previously unreleased dataset. These efforts will allow us to compare the two modes of research. Overall, the findings generated from these studies will inform scientific practices and help researchers to make better decisions in planning and interpreting research.
In addition, we will also examine how features of researchers influence, and are influenced by, this process. In particular, we will examine the intellectual humility of participating researchers, both as an outcome of engaging in preregistration and as a predictor of the reliability of their results.
