Plate Nº 43 · recorded November 12, 2026
Reported finding
Even Trained Researchers Misread Basic Statistics, Review Finds
A review of 55 studies and over 15,000 participants finds researchers, statisticians and teachers routinely misinterpret p-values and statistical significance.
By Elena Vasquez3 min read690 words
In brief
- A systematic review analyzed 55 studies with over 15,000 participants, published in PLOS One.
- The reviewed studies span publications from 1963 to 2024.
- Misunderstandings were common even among statisticians and statistics teachers.
- A frequent error: treating the p-value as the probability the hypothesis is true or false.
- The review was carried out by researchers at Uppsala University and colleagues.
A systematic review of 55 studies involving more than 15,000 participants has found that even experienced researchers and people with extensive statistical training routinely misinterpret statistical results. Researchers at Uppsala University and colleagues published the work in PLOS One (DOI: 10.1371/journal.pone.0362071).
The finding matters because statistics underpin conclusions across science — whether a medicine works, or whether air pollution harms human health. If the people producing and reading those results misunderstand them, the conclusions can go wrong.
What did the review cover?
The analysis pulls together 55 studies published between 1963 and 2024. Most were questionnaire surveys. Participants had to interpret statistical results or judge whether statements about them were correct or incorrect. Other studies looked at how statistical results shape people's confidence in scientific conclusions.
The participants were not amateurs. They included students, researchers, health care professionals, statisticians and teachers of statistics.
Despite using different methods across six decades, the studies reached strikingly similar conclusions, said Per Lytsy, an affiliated researcher at the Department of Public Health and Caring Sciences, Uppsala University.
"What's striking is that, despite using different methods, the researchers have reached such similar conclusions: understanding of what can actually be inferred from statistical results is inadequate and misunderstandings are common," Lytsy said.
In short: the problem is widespread, persistent and does not disappear with training.
Which concepts get misunderstood?
The review focuses on the core tools of statistical inference — the process of drawing general conclusions from data. Three concepts sit at the center:
- P-values: numbers that describe how compatible data are with a hypothesis, under assumptions.
- Statistical significance: a threshold label based on the p-value.
- Confidence intervals: ranges intended to convey the uncertainty around an estimate.
A common misconception treats statistical significance as proof that a hypothesis is true, or that an effect matters in practice. It doesn't. A medicine can produce a statistically significant improvement that has no practical significance for patients.
The p-value causes particular trouble. Many readers treat it as the probability that the tested hypothesis is true or false. That conclusion cannot be drawn from a p-value, said Ronnie Pingel, senior lecturer at the Department of Statistics, Uppsala University.
"Such misinterpretations can lead us to attribute a greater degree of certainty to research findings than is warranted," Pingel said.
The takeaway: a small p-value signals something, but not what most people think it signals.
Why do these errors persist?
The review's authors point to several reasons. Statistical concepts are genuinely difficult and often clash with how people intuitively reason about probability and uncertainty. On top of that, statistical methods rest on different theoretical foundations and are sometimes used in combination, making it harder to see which conclusions are actually justified.
The problem is not carelessness. It's a mismatch between intuitive thinking and what the mathematics actually supports.
Why does this matter for science and teaching?
By compiling studies spanning more than 60 years, the review gives a broad picture of how widespread and persistent these misconceptions are — even among people with extensive statistical training.
The researchers hope the work will raise awareness of statistical misinterpretation, improve statistics teaching, and encourage more critical scrutiny of scientific findings.
Lytsky put the stakes plainly: "Being able to use statistical methods is not the same as understanding what the results mean. Our review shows that misunderstandings are common even among experienced researchers. This risks leading to incorrect conclusions and, ultimately, poorer decisions."
That distinction — between running the methods and understanding the output — is the heart of the problem. Statistical software will compute a p-value for anyone. Interpreting it correctly is a separate skill, and one that even professionals haven't reliably mastered.
For readers of science, the message is caution. A result labeled "statistically significant" is not a verdict. It's a piece of evidence, filtered through concepts that even experts frequently get wrong.
The study is a review of existing research, not new experiments, so it describes the pattern rather than testing fixes. But its scope — 55 studies, 15,000-plus participants, six decades — makes the pattern hard to dismiss.
via phys.org/rss-feed/ (Source)
More from Elena Vasquez
Show full bio
Correspondent covering money, funding and market shifts around general science news at SciBeat.
384 articles
Nearby plates
- 42 studies, one verdict: no single factor gets a Ph.D. finished
- Scientists Report Continuing Harms to U.S. Science in Second-Year Survey
- AI Reviewer ASP1 Checks Science for Causation, Not Truth
- FDA Urges Drug Makers to Release Data From Thousands of Unpublished Trials
- Nearly 30% of Patients Lie About Symptoms to Get Antibiotics