6.9 KiB
| title | chunk | source | category | tags | date_saved | instance |
|---|---|---|---|---|---|---|
| Preregistration (science) | 3/4 | https://en.wikipedia.org/wiki/Preregistration_(science) | reference | science, encyclopedia | 2026-05-05T03:50:00.803493+00:00 | kb-cron |
== Benefits == Several articles have outlined the rationale for preregistration (e.g., Lakens, 2019; Nosek et al., 2018; Wagenmakers et al., 2012). The primary goal of preregistration is to improve the transparency of reported hypothesis tests, which allows readers to evaluate the extent to which decisions during the data analysis were pre-planned (maintaining statistical error control) or data-driven (increasing the Type 1 or Type 2 error rate). Meta-scientific research has revealed additional benefits. Researchers indicate preregistering a study leads to a more carefully thought through research hypothesis, experimental design, and statistical analysis. In addition, preregistration has been shown to encourage better learning of Open Science concepts and students felt that they understood their dissertation and it improved the clarity of the manuscript writing, promoted rigour and were more likely to avoid questionable research practices. In addition, it becomes a tool that supervisors can use to shape students to combat any questionable research practices. A 2024 study in the Journal of Political Economy: Microeconomics preregistration in economics journals found that preregistration reduced p-hacking and publication bias if the preregistration was accompanied by a preanalysis plan, but not if the preregistration did not specify the planned analyses.
== Criticisms ==
=== Analytical Flexibility === Proponents of preregistration have argued that it is "a method to increase the credibility of published results" (Nosek & Lakens, 2014), that it "makes your science better by increasing the credibility of your results" (Centre for Open Science), and that it "improves the interpretability and credibility of research findings" (Nosek et al., 2018, p. 2605). This argument assumes that on average non-preregistered analyses are less "credible" and/or "interpretable" than preregistered analyses because researchers may opportunistically abuse flexibility in the data analysis to reduce the severity of the tests. However, critics have argued that preregistration is not necessary to take analytical flexibility into consideration: Some hypotheses allow more analytical flexibility than others (e.g., Auspurg & Brüderl, 2021), and researchers, reviewers, and readers can take these differences into account when evaluating research conclusions (Hitchcock & Sober, 2004, p. 7; Lakatos, 1968, pp. 375-376; Lash & Vandenbroucke, 2012, pp. 185-186; Szollosi & Donkin, 2021, pp. 2-3; Rubin, 2020, p. 378; Rubin & Donkin, 2024, p. 2035). As Popper explained, theories that allow a wider "range" of predictions in a study should be downgraded as being less "severely testable" (Popper, 2002, pp. 95, 108). Importantly, this Popperian assessment of testability can be made in the absence of preregistration (Rubin, 2025). It is also worth noting that researchers face a range of practical constraints that limit their ability to opportunistically abuse analytical flexibility. Specifically, they are constrained by analytical norms and conventions as well as the requirement to produce multiple, theoretically interesting, directionally consistent results that survive robustness checks and use conceptually consistent methods and analytical approaches across multiple studies in their research articles (Murayama et al., 2014, pp. 108-109; Wegener et al., 2024). However, this criticism itself has been criticized as "Authors who have raised this criticism on preregistration fail to provide any real-life examples of theories that sufficiently constrain how they can be tested, nor do they provide empirical support for their hypothesis that peers can identify systematic bias".
=== Circular Reasoning === Nosek et al. (2018) argued that preregistration is important because it provides a clear distinction between predictions and postdictions (post hoc explanations). Failing to make this distinction can lead to the fallacy of "circular reasoning––generating a hypothesis based on observing data, and then evaluating the validity of the hypothesis based on the same data" (Nosek et al., 2018, p. 2600). However, critics have argued that preregistration is not necessary to identify circular reasoning (Rubin & Donkin, 2024, p. 2025). Circular reasoning can be identified by analysing the logic of the reasoning per se without needing to knowing the timing of that reasoning (Popper, 1962, p. 288; Popper, 1983, p. 133; Popper, 2002, p. 274; for examples, see Kriegeskorte et al., 2009, p. 536).
=== Deterring Exploratory Analyses === Critics have noted that the idea that preregistration improves research credibility may deter researchers from undertaking non-preregistered exploratory analyses (Coffman & Niederle, 2015; see also Collins et al., 2021, Study 1). In response, preregistration advocates have stressed that a) exploratory analyses were rarely published to begin with, and b) that exploratory analyses are permitted in preregistered studies, and that the results of these analyses retain some value vis-a-vis hypothesis generation rather than hypothesis testing. Preregistration merely makes the distinction between confirmatory and exploratory research clearer (Nosek et al., 2018; Nosek & Lakens, 2014; Wagenmakers et al., 2012). Hence, although preregistration is supposed to reduce researcher degrees of freedom during the data analysis stage, it is also supposed to be "a plan, not a prison" (Dehaven, 2017). Deviations are sometimes improvements, and should be transparently reported so that others can evaluate the consequences of the deviation. However, critics have argued that treating preregistration as a plan, rather than a prison, blurs the distinction between confirmatory and exploratory research "because adjustable plans do not control the Type I error rate" (Rubin, 2025, p. 19; see also Navarro, 2020, p. 8), and research becomes "exploratory" when error rates are not controlled (Ditroilo et al., 2024, p. 1109).
=== The Distinction Between Confirmatory and Exploratory Research === Critics have also argued that the distinction between confirmatory and exploratory analyses is unclear and/or irrelevant (Devezer et al., 2020; Rubin, 2020; Szollosi & Donkin, 2019),. However, more recent work has provided a more principled definition of 'exploratory' and 'confirmatory' by arguing that "hypothesis tests are confirmatory when their error rates are controlled, and exploratory when the error rates are not controlled." which both clarifies the distinction, and demonstrates the relevance of the distinction for preregistration. As discussed above, however, this definition implies that research should be regarded as "exploratory" when preregistration is treated as "a plan, not a prison," because adjustable plans do not control error rates (Navarro, 2020, p. 8; Rubin, 2025).