ChatGPT’s Health Advice Sends 60-Year-Old Man to the Hospital, Raises Questions on Its Reliability

ChatGPT's health advice was the reason behind a man's trip to the hospital, as per a new case study. The study highlights that a 60-year-old person was suffering from rare metal poisoning, which resulted in a range of symptoms, including psychosis. The study also mentions that the poisoning, identified as being caused by long-term sodium bromide consumption, occurred because the patient took advice from ChatGPT about dietary changes. Interestingly, with GPT-5, OpenAI is now focusing on health-related responses from the artificial intelligence (AI) chatbot, promoting it as a key feature.

ChatGPT Said to Have Asked a Man to Replace Table Salt With Sodium Bromide

According to an Annals of Internal Medicine Clinical Cases report titled “A Case of Bromism Influenced by Use of Artificial Intelligence,” a person developed bromism after consulting the AI chatbot ChatGPT for health information.

The patient, a 60-year-old man with no past psychiatric or medical history, was admitted to the emergency room, concerned that he was being poisoned by his neighbour, the case study stated. He suffered from paranoia, hallucinations and suspicion of water despite being thirsty, insomnia, fatigue, issues with muscle coordination (ataxia), and skin changes, including acne and cherry angiomas.

After immediate sedation and running a series of tests, including consultation with the Poison Control Department, the medical professionals were able to diagnose the condition as bromism. This syndrome occurs after long-term consumption of sodium bromide (or any bromide salt).

According to the case study, the patient reported consulting ChatGPT to replace sodium chloride in his diet, and after receiving sodium bromide as an alternative, he began consuming it regularly for three months.

The study claims, based on the undisclosed timeline of the case, that either GPT-3.5 or GPT-4 was used to receive the consultation. However, the researchers note that they did not have access to the conversation log, so it is not possible to assess the prompt and response from the AI. It is likely that the man took ChatGPT's answer out of context.

“However, when we asked ChatGPT 3.5 what chloride can be replaced with, we also produced a response that included bromide. Though the reply stated that context matters, it did not provide a specific health warning, nor did it inquire about why we wanted to know, as we presume a medical professional would do,” the study added.

Live Science reached out to OpenAI for a comment. A company spokesperson reported directed the publication was directed to the company's terms of use, which state that one should not rely on output from ChatGPT as a “sole source of truth or factual information, or as a substitute for professional advice.

After prompt action and a treatment that lasted three weeks, the study claimed that the person began displaying improvements. “It is important to consider that ChatGPT and other AI systems can generate scientific inaccuracies, lack the ability to critically discuss results, and ultimately fuel the spread of misinformation,” the researchers said.