5.3 KiB
| title | chunk | source | category | tags | date_saved | instance |
|---|---|---|---|---|---|---|
| Decline effect | 2/3 | https://en.wikipedia.org/wiki/Decline_effect | reference | science, encyclopedia | 2026-05-05T03:36:35.033819+00:00 | kb-cron |
If the true effect size is small, but the initial study has low power (i.e., small sample size), then the null hypothesis will only be rejected if the effect estimate is far from zero, as illustrated in the figure. This means that subsequent replications, with larger sample sizes, will discover effect estimates that are closer to the true effect, which is closer to zero than the initial estimate.
=== Publication bias === Another reason may be the publication bias: scientists and scientific journals prefer to publish positive results of experiments and tests over null results, especially with new ideas. As a result, the journals may refuse to publish papers that do not prove that the idea works. Later, when an idea is accepted, journals may refuse to publish papers that support it.
=== Experimenter effect ===
In the debate that followed the original article, Lehrer answered some of the questions by claiming that scientific observations might be shaped by one's expectations and desires, sometimes even unconsciously, thus creating a bias towards the desired outcome. This is known as the experimenter effect. For example, in parapsychology, the "experimenter effect" is used to explain how an experimenter who does not believe in psi would discover no evidence for psi, while the same experiment would when performed by an experimenter who does believe in psi. A significant factor contributing to the decline effect can also be the sample size of the scientific research, since smaller sample size is very likely to give more extreme results, suggesting a significant breakthrough, but also a higher probability of an error. Typical examples of this effect are the opinion polls, where those including a larger number of people are closer to reality than those with a small pool of respondents. This suggestion would not appear to account for the observed decrease over time regardless of sample size. Researcher John Ioannidis offers some explanation. He states that early research is usually small and more prone to highly positive results supporting the original idea, including early confirmatory studies. Later, as larger studies are being made, they often show regression to the mean and a failure to repeat the early exaggerated results.
=== Genuine decline === A 2012 report by National Public Radio's show "On The Media" covered scientists who are exploring another option: that the act of observing the universe changes the universe, and that repeated measurement might actually be rendering earlier results invalid. In other words, antipsychotic drugs did work originally, but the more we measured their effectiveness, the more the laws governing those drugs changed so they ceased to be effective. Science fiction author Geoff Ryman explores this idea and its possible ramifications further in his 2012 short story What We Found, which won the Nebula Award for Best Novelette in 2012. Another reason for some decline effects may be that certain researchers tend to publish larger effect sizes than others. For example, alongside publication bias and sample size effects, the decline effect in ocean acidification effects on fish behavior was largely driven by outstanding effect sizes reported by two particular investigators from the same laboratory who are currently under investigation for potential scientific misconduct and data fabrication.
== Contesting views == Several commenters have contested Jonah Lehrer's view of the decline effect being a problematic side of the phenomenon, as presented in his New Yorker article. "The decline effect is troubling because it reminds us how difficult it is to prove anything. We like to pretend that our experiments define the truth for us. But that's often not the case. Just because an idea is true doesn't mean it can be proved. And just because an idea can be proved doesn't mean it's true. When the experiments are done, we still have to choose what to believe." Steven Novella also challenges Lehrer's view of the decline effect, arguing that Lehrer is concentrating on new discoveries on the cutting edge of scientific research and applying the conclusions to all areas of science. Novella points out that most of the examples used by Lehrer come from medicine, psychology and ecology, scientific fields most influenced by a complex human aspect and that there is not much evidence of the decline effect in other areas of science, such as physics. Another scientist, Paul Zachary Myers, is also contesting Lehrer's view on the decline effect being a surprising phenomenon in science, claiming that: "This isn't surprising at all. It's what we expect, and there are many very good reasons for the shift." Lehrer's statements about the difficulty of proving anything and publication bias find support from Jerry A. Coyne. Coyne holds that in the fields of genetics and evolutionary biology, almost no research is replicated and there is a premium motivation offered for publishing positive results of research studies. However, he also contests Lehrer's approach of applying conclusions on all fields of science, stating that in physics, chemistry or molecular biology, previous results are constantly repeated by others in order to progress in their own research.