This is not a rare glitch but a design gap, the researchers contend. AI companies, they write, lack any system that makes the health advice their chatbots generate visible to others, and this built-in lack of visibility "actively prevents oversight."

Can you be harmed without even asking?

Yes, the paper suggests. Chatbots can be wrong in several ways: they confidently generate erroneous answers, oversimplify medical information, or draw on outdated material and false claims. But the risk is no longer limited to people who actively seek advice.

AI-generated summaries now appear in search engines, and AI is integrated into social media platforms such as X and Meta. That means incorrect health information can reach users who never typed a medical question.

"Sometimes I'm not even looking for health information, but just by browsing my social media feeds, it's there. It just shows up, and we believe that could have undesirable outcomes, especially if there's medical misinformation or state actors trying to manipulate the online discussion," Yang said.

Whether the exposure is deliberate or incidental, the authors argue, the resulting harm rarely leaves traces that doctors, regulators or researchers can verify.

What fixes do the authors propose?

The paper lays out several recommendations aimed at different players.

• AI companies should give users access to their own health conversations, so patients can share them with clinicians and doctors can trace the pathway that led to a harmful event. They should also build a disclosure system allowing health guidance to be flagged, reported and investigated.
• Social media companies should improve the clarity of labels on AI-generated health content and withhold that content until medical governing bodies vet it.
• Search engines should use only vetted information in their AI summaries.
• Policymakers should extend physician malpractice liability to AI chatbot companies.

Will companies fix this themselves?

Yang is skeptical. "We do not think this is something we should rely on the companies to do, because their incentive is always to make more money," he said. "Building such a system goes against that incentive. So we have to have some kind of third-party monitoring system, an independent evaluation."

The paper is a perspective piece — an argument synthesizing existing evidence rather than a new experimental study — so its recommendations are proposals, not established policy. Still, the underlying case it cites, and the growing reach of AI summaries across search and social platforms, give the warning concrete weight. As chatbots field more health questions around the clock, the authors argue, the absence of any record may matter as much as the advice itself.

Publication details: DeVerna, M.R., et al., The Invisible Risks of AI-Generated Health Information, Nature Health (2026). DOI: 10.1038/s44360-026-00206-x