The Problem of AI 'Hallucinations' in Medicine
In 2026, the issue of AI 'hallucinations' remains relevant as Google's AI issues medical recommendations that contradict conventional medicine, endangering patient lives. In particular, AI advised pancreatic cancer patients to avoid fatty foods, which is not in line with oncologists' recommendations. This has raised concerns among specialists, as following such advice could lead to critical weight loss, as noted by Anna Jewell, a representative of Pancreatic Cancer UK.
Research and Reaction to the Problem
Research from the University of Pennsylvania conducted in April 2025 found that nearly 80% of adults search for symptoms online. Moreover, two-thirds of them consider AI responses reliable. This phenomenon is driven by experiments from the Massachusetts Institute of Technology, which confirmed that users perceive AI advice as 'comprehensive and trustworthy.' However, medical associations worldwide indicate that such advice is 'prone to hallucinations' and outdated.
In response to these issues, the Chinese Cyber Administration published a draft of new rules aimed at mitigating the harm from chatbots. Given the risks associated with medical advice provided by artificial intelligence, this initiative underscores the need for greater caution in using such technologies in healthcare.
This situation highlights the importance of regulating the use of artificial intelligence in the medical field, as the growing reliance on technology can lead to serious health consequences for patients. With more people turning to online resources for medical advice, it is essential to ensure that these sources are reliable and safe. The new rules proposed by China could be a crucial step in preventing potential risks associated with the use of AI in medicine.