Plate Nº 93 · recorded November 9, 2026
Health & Medicine ResearchReported finding
AI Reviewer ASP1 Checks Science for Causation, Not Truth
ASP1, an AI paper-review system, checks whether studies support causal claims and drafts audience-specific briefs — its first test reinforced cautious human interpretation.
By Marcus Bennett3 min read637 words
In brief
- ASP1 generates evidence bundles and audience-specific briefs for scientists, lawyers, policymakers, and reporters
- A hazard ratio of 2 indicates twice the hazard in one group compared with another, but cannot prove causation
- Thousands of observational health-policy studies are published each year
- The demonstration study found lower mortality among overweight dementia patients, but no causal claim was established
- A proof-of-concept version accepts PDF uploads and generates evidence bundles on demand
A newly developed AI-assisted system called the Automated Scientific Paper Reviewer (ASP1) can review scientific papers transparently and auditably — and its first real-world test shows it reinforcing, rather than overriding, careful human judgment about a counterintuitive dementia study.
ASP1, described by its developer Dr. Cox, generates structured reports that apply predefined methodological standards to published research, focusing on one deceptively simple question: does the evidence actually support the causal conclusions a paper draws?
Why does correlation keep fooling people?
In health policy, thousands of studies appear each year testing whether an external factor links to a health outcome. The principle "correlation is not causation" sits at the center of the challenge facing regulators and policymakers, yet the public often misses it.
Take one of nutrition's most contested questions: is ultra-processed food associated with diseases like cardiovascular disease, cancer, or dementia — and if so, does it cause them?
Most health policy studies are observational: researchers examine data on people's exposures and outcomes rather than randomly assigning exposures. These studies report measures of association such as hazard ratios. A hazard ratio (HR) of 1 means equal risk between groups; 0.5 means half the hazard; 2 means twice the hazard.
But an HR alone cannot establish causation. Even a large association may reflect confounding, selection bias, reverse causation, or other explanations. A confounder — a factor that influences both the exposure and the outcome — can distort an apparent relationship. Scientists counter this with randomized assignment when feasible, careful study design, and statistical adjustment, and the credibility of any causal claim depends on how well those methods rule out alternatives.
Policy raises a further question: if ultra-processed food causes harm, which intervention would actually change diets, and toward what substitutions? Good policy needs evidence about the consequences of alternative actions, not just associations.
What does ASP1 actually do?
Unlike an unstructured request to a general-purpose chatbot, ASP1 uses specialized modules organized around explicit methodological questions. It first extracts and classifies a paper's key causal claims, then evaluates reporting quality, study design, statistical analysis, potential biases, and how well the paper justifies its causal interpretations. Human reviewers can inspect, question, and check every output.
The system does not determine truth. It evaluates the support a document presents, including its omissions and ambiguities. Its output — an "evidence bundle" consisting of a summary, module reports, and an overall synthesis — serves as an input for expert reviewers, not an authoritative verdict. It also produces audience-specific evidence briefs for scientists, lawyers, policymakers, and reporters. A proof-of-concept version is available online: users upload a PDF and click a button to generate a bundle.
How did it handle a tricky dementia study?
The demonstration used a recent study, discussed in an ACSH article, reporting a surprising association: people who were overweight or obese when diagnosed with dementia had lower subsequent mortality than those of normal weight.
The study's authors were clear the association might not be causal — possibly reflecting body reserves mitigating malnutrition and eating difficulties, though that mechanism was not established. ASP1's summary agreed, stating:
"The evidence is associative, not causal. The document explicitly warns against using these findings to support weight gain or maintenance as a survival intervention in dementia."
The reporter brief spelled out headline guidance: say "Study reports correlation between higher body weight and lower mortality after dementia diagnosis," and avoid headlines like "Obesity protects against death in dementia." The lawyer brief recommended describing the results as "statistical associations" and explicitly stating that causality is not established.
What changes for practitioners?
ASP1 addresses two chronic problems in modern science: evaluating vast research output without confusing correlation with causation, and communicating complex findings to different audiences. It cannot replace scientific judgment — but it can make that judgment more systematic, clear, and accessible.
via acsh.org (Source)
More from Marcus Bennett
Show full bio
News editor covering regulation, policy and their impact on general science news at SciBeat.
370 articles
Nearby plates
- Alzheimer's Drug Developers Accuse Trial Sites of Faking Data
- Mediterranean-Style Diets Show Brain Benefits—But Only in Snapshot
- Proposed Rule Would Politicize Medical Research, Scientists Warn
- FDA Urges Drug Makers to Release Data From Thousands of Unpublished Trials
- Even Trained Researchers Misread Basic Statistics, Review Finds