Plate Nº 71 · recorded November 17, 2026
Science Policy & FundingReported finding
AI Checked Ethics Applications in Minutes — But Only One Model Got It Right
Claude 3.5 Sonnet got 87.9% of ethics checks fully correct in seven minutes per application, a Bristol study with the UK Health Research Authority found. But the regulator isn't adopting AI yet.
By James Calloway4 min read712 words
In brief
- Claude 3.5 Sonnet achieved 87.9% fully correct checks, vs 66.5% for ChatGPT and 62.8% for Copilot.
- Claude averaged just over seven minutes per application; Copilot took 23.
- Claude spotted an average of 8 errors per application, against 2.3 for ChatGPT and 1.3 for Copilot.
- The study, published Nov. 16 in Scientific Reports, was a collaboration with the UK Health Research Authority.
- The HRA says further exploration is needed before recommending AI adoption in approvals.

One AI model got 87.9% of research ethics checks fully correct in an average of seven minutes per application — a task that currently eats substantial chunks of human reviewer time. That's the headline finding of a University of Bristol study published in Scientific Reports on Nov. 16, and it comes with a caveat the researchers repeat more than once: don't trust the machines without verification.
The study, run with the UK's Health Research Authority (HRA), pitted three widely used large language models — Claude 3.5 Sonnet, ChatGPT-4o and Microsoft Copilot — against a routine but demanding job. Before any research study can begin, its application must be checked against the HRA's published quality standards. That can mean reviewing hundreds of pages and cross-referencing information across multiple documents. Essential work. Slow work. A significant slice of the whole approval timeline.
Which model won, and by how much?
The performance gap between the three tools was wide enough to matter:
- Accuracy: Claude answered 87.9% of checks fully correctly. ChatGPT managed 66.5%. Copilot trailed at 62.8%.
- Speed: Claude averaged just over seven minutes per application. ChatGPT took 13 minutes. Copilot needed 23.
- Error-spotting: Claude identified an average of eight errors per application needing correction. ChatGPT found 2.3. Copilot found 1.3.
- Reliability gaps: Copilot failed to answer data protection questions in most applications.
Claude also answered every question posed to it and produced fewer hallucinations than its rivals — hallucinations being cases where an AI generates information that is incorrect or unsupported by anything in the source material.
The most common cause of wrong answers, across all three models, wasn't invention. It was failure of logical reasoning. That's worth pausing on. The tools didn't just make things up; they reasoned badly, which is arguably harder to catch.
What does this change for researchers?
In the near term, very little — and the HRA is explicit about that. The findings point to AI as an assistant, not a replacement. Lead author Dr. Panos Maghsoudlou, NIHR academic clinical lecturer in ophthalmology at Bristol's School of Biochemistry and Biomedical Sciences, said: "These results highlight both the potential of AI-assisted checking and the risks of relying on its outputs without verification."
His framing is careful. Large language models could handle the early stages of approval, he suggested, "identifying missing information and potential departures from published standards before applications reach an ethics committee." Automating routine checks could free approvals staff and committee members to concentrate on "the substantive ethical issues at the heart of research."
But Maghsoudlou draws a line the industry would do well to memorize: checking whether an application meets documented requirements is not the same as deciding whether a study is ethically acceptable. "The latter requires human judgment and cannot be reduced to a checklist," he said.
The regulator's view
Dr. Jonathan Fennelly-Barnwell, director of approvals at the HRA and co-author of the paper, said the collaboration was built to develop an evidence base on how AI might streamline research approvals. In his words, AI used this way "could release staff capacity to focus on more complex regulatory issues, making our processes more effective and potentially also helping reduce the timeline for regulatory review."
Note the hedging. "Potentially." He followed with the cold shower: "While the results of the study are promising, further exploration will be required before we can make recommendations for how we might be able to adopt AI to improve our processes." The HRA says it is continuing to test the safe and responsible use of AI as part of broader work on the technology's benefits and challenges.
So where does that leave things? A proof of concept, not a policy. One model performed well on one defined task under test conditions. Real approval workflows, with messier documents and higher stakes, remain untested territory. The study doesn't say how the models would fare on novel or ambiguous applications, or what verification by humans would cost in practice — and those unanswered questions are exactly where adoption will be decided.
The paper: Panagiotis Maghsoudlou et al., "Evaluating large language models for quality control in research ethics review," Scientific Reports (2026). DOI: 10.1038/s41598-026-62027-y.
via phys.org/rss-feed/ (Source)
More from James Calloway
Show full bio
Staff writer covering regulation, policy and their impact on general science news at SciBeat.
418 articles