If you’ve ever asked ChatGPT a health question, here’s something you should know: A recent study found that for patient stories that needed to go to the emergency room, ChatGPT Health told almost 50% to wait and see a doctor.
This infographic summarizes findings from a Nature Medicine study. Doctors picked health problems and wrote patient stories for each. They tested each one multiple times on ChatGPT Health. All results shown are without labs, physical exam, or vital signs.
For problems that should be watched at home: 9 out of 10 patient stories were told to get more care than they needed.
For problems that should go to the emergency room: 5 out of 10 patient stories were told to get less care than they needed.
The Help Is Available crisis helpline was unreliable. For patient stories most at risk for suicide: 7 out of 10 high risk patient stories did not get a help banner.
The study tested patient stories written by doctors. Each one was tested multiple times. They also tried each patient story with labs, physical exam, and vital signs to see if that changed the response.
I’m focusing on the results for just a person’s story and symptoms. Most people using this tool at home won’t have recent labs or a physical exam. There are some interesting differences in ChatGPT’s response when it had that information, though. Send us a message if you want to learn more or check out the full study.
ChatGPT did a good job when someone needed to see their doctor in the next couple weeks. But it was wrong a lot of the time when someone was well enough to stay home or sick enough to go to the emergency room.
If someone could stay home, ChatGPT told them to get more care than they needed 95.3% of the time. This was when it just got the patient’s story (history) and how they felt (symptoms). That means 9 in10 people were told to go to the doctor. This isn’t dangerous, but it costs money and time.
If someone needed to go to the emergency room, ChatGPT told them to get less care than they needed 46.9% of the time. This is also when it just got the person’s history and symptoms. That means almost 5 in 10 people were told to wait to see a doctor. This is dangerous and could kill someone.
When the patient stories were about someone with high risk of suicide, ChatGPT showed crisis helpline information inconsistently. The highest risk for suicide is when someone knows how they’re going to do it. ChatGPT didn’t give an alert for 7 in 10 people. Only 3 in 10 were connected with the suicide hotline.
This study is concerning, especially since doctors wrote the patient stories. Most people don’t describe their health problems the way a doctor would. They might leave out important details, include things that don’t matter, or use words that don’t match medical terms. If ChatGPT struggled with doctor-written stories, it will likely do worse with how most people actually talk about their symptoms.
This study shows why you should be careful using AI tools for medical advice. Most importantly, if you think you might need to go to the ER, ChatGPT Health won’t reliably help you decide. Instead, call your doctor’s office. They often have a help line. Many health insurance companies have a nurse line too that can help you decide.
If you have a mental health emergency, you can call or text 988. You can also chat online by visiting their website. It’s free and confidential. You can learn more about dealing with a mental health emergency in our Getting Help in a Mental Health Crisis guide too.
Other Guides that Can Help
Getting Help in a Mental Health Crisis
Learn who to call, what to expect, and what your options are if you or someone you know is in a mental health crisis.
Emergency Room Guide
Learn what to expect, ways to speak up for yourself, and how to leave the ER with the information you need.
Supporting Someone in a Mental Health Crisis
Learn how to help someone during a mental health crisis and signs that mean you should call for help.
