4.8 KiB
| title | chunk | source | category | tags | date_saved | instance |
|---|---|---|---|---|---|---|
| Replication crisis | 3/15 | https://en.wikipedia.org/wiki/Replication_crisis | reference | science, encyclopedia | 2026-05-05T03:45:08.741659+00:00 | kb-cron |
Social priming failures: In the early 2010s, two direct replication attempts failed to reproduce results from social psychologist John Bargh's much-cited "elderly-walking" study (originally published in 1996). This experiment was part of a series of three studies that had been widely cited throughout the years, was regularly taught in university courses, and had inspired many conceptual replications. These replication failures triggered intense disagreement between replication researchers and the original authors. Notably, many of the conceptual replications of the original studies also failed to replicate in subsequent direct replications. Experiments on extrasensory perception: Social psychologist Daryl Bem conducted a series of experiments supposedly providing evidence for the controversial phenomenon of extrasensory perception. Bem faced substantial criticism of his study's methodology. Reanalysis of his data found no evidence for extrasensory perception. The experiment also failed to replicate in subsequent direct replications. According to Romero, what the community found particularly upsetting was that many of the flawed procedures and statistical tools used in Bem's studies were part of common research practice in psychology. Biomedical replication failures: Scientists from biotech companies Amgen and Bayer Healthcare reported alarmingly low replication rates (11–20%) of landmark findings in preclinical oncological research. P-hacking studies and questionable research practices: Since the late 2000s, a number of studies in metascience showed how commonly adopted practices in many scientific fields, such as exploiting the flexibility of the process of data collection and reporting, could greatly increase the probability of false positive results. These studies suggested how a significant proportion of published literature in several scientific fields could be nonreplicable research. This series of events generated a great deal of skepticism about the validity of existing research in light of widespread methodological flaws and failures to replicate findings. This led prominent scholars to declare a "crisis of confidence" in psychology and other fields, and the ensuing situation came to be known as the "replication crisis". Although the beginning of the replication crisis can be traced to the early 2010s, some authors point out that concerns about replicability and research practices in the social sciences had been expressed much earlier. Romero notes that authors voiced concerns about the lack of direct replications in psychological research in the late 1960s and early 1970s. He also writes that certain studies in the 1990s were already reporting that journal editors and reviewers are generally biased against publishing replication studies. In the social sciences, the blog Data Colada (whose three authors coined the term "p-hacking" in a 2014 paper) has been credited with contributing to the start of the replication crisis. University of Virginia professor and cognitive psychologist Barbara Spellman has written that many criticisms of research practices and concerns about replicability of research are not new. She reports that between the late 1950s and the 1990s, scholars were already expressing concerns about a possible crisis of replication, a suspiciously high rate of positive findings, questionable research practices, the effects of publication bias, issues with statistical power, and bad standards of reporting. Spellman also identifies reasons that the reiteration of these criticisms and concerns in recent years led to a full-blown crisis and challenges to the status quo. First, technological improvements facilitated conducting and disseminating replication studies, and analyzing large swaths of literature for systemic problems. Second, the research community's increasing size and diversity made the work of established members more easily scrutinized by other community members unfamiliar with them. According to Spellman, these factors, coupled with increasingly limited resources and misaligned incentives for doing scientific work, led to a crisis in psychology and other fields. According to Andrew Gelman, the works of Paul Meehl, Jacob Cohen, and Tversky and Kahneman in the 1960s-70s were early warnings of replication crisis. In discussing the origins of the problem, Kahneman himself noted historical precedents in subliminal perception and dissonance reduction replication failures. It had been repeatedly pointed out since 1962 that most psychological studies have low power (true positive rate), but low power persisted for 50 years, indicating a structural and persistent problem in psychological research.
== Prevalence ==