What's happened
A recent study has found that AI chatbots frequently provide problematic answers to medical questions, especially about cancer treatments and alternative therapies. The research highlights risks of misinformation, with some models showing up to 58% problematic responses, raising concerns about their use in healthcare guidance.
What's behind the headline?
The study confirms that AI chatbots are currently unreliable for medical advice, especially in misinformation-prone fields like cancer and nutrition. These models are generating responses based on training data patterns, not reasoning or evidence weighing, which leads to inaccuracies. The tendency of chatbots to produce 'hallucinations' and false balance risks misleading users into pursuing unproven or harmful treatments. This will likely increase the spread of health misinformation unless there is strict oversight and public education. The worst-performing model, Grok, demonstrates that deployment without safeguards will amplify misinformation, potentially causing real harm. The findings underscore the urgent need for regulatory frameworks and professional oversight to prevent AI from eroding public health efforts.
What the papers say
The Independent reports that chatbots like Grok, ChatGPT, and Meta AI have been found to deliver up to 58% problematic responses in medical questions, with concerns raised about their tendency to hallucinate and provide false balance. The Scotsman highlights that these models often generate incorrect or misleading answers due to biased training data, warning that models fine-tuned on human feedback exhibit sycophancy. Both sources emphasize that AI tools are not licensed to dispense medical advice and lack access to real-time data, which will continue to pose risks unless properly regulated. The Independent also notes that previous studies have shown that only 32% of citations from AI models are accurate, with many responses fabricated or incomplete, reinforcing the need for caution in their use in healthcare.
How we got here
Researchers have tested popular AI chatbots by asking them medical questions to evaluate their accuracy. The study was prompted by increasing public reliance on AI tools for health advice, despite their lack of licensing and access to current medical data. Previous research has shown that many AI-generated citations are fabricated or incomplete, fueling concerns about misinformation.
Go deeper
More on these topics