Thursday, August 7, 2025
Science
No Result
View All Result
  • Login
  • HOME
  • SCIENCE NEWS
  • CONTACT US
  • HOME
  • SCIENCE NEWS
  • CONTACT US
No Result
View All Result
Scienmag
No Result
View All Result
Home Science News Medicine

ChatGPT fails at heart risk assessment

May 1, 2024
in Medicine
Reading Time: 3 mins read
0
ChatGPT fails at heart risk assessment
65
SHARES
595
VIEWS
Share on FacebookShare on Twitter
ADVERTISEMENT
ADVERTISEMENT

SPOKANE, Wash. – Despite ChatGPT’s reported ability to pass medical exams, new research indicates it would be unwise to rely on it for some health assessments, such as whether a patient with chest pain needs to be hospitalized.  

SPOKANE, Wash. – Despite ChatGPT’s reported ability to pass medical exams, new research indicates it would be unwise to rely on it for some health assessments, such as whether a patient with chest pain needs to be hospitalized.  

In a study involving thousands of simulated cases of patients with chest pain, ChatGPT provided inconsistent conclusions, returning different heart risk assessment levels for the exact same patient data. The generative AI system also failed to match the traditional methods physicians use to judge a patient’s cardiac risk. The findings were published in the journal PLOS ONE.

“ChatGPT was not acting in a consistent manner,” said lead author Dr. Thomas Heston, a researcher with Washington State University’s Elson S. Floyd College of Medicine. “Given the exact same data, ChatGPT would give a score of low risk, then next time an intermediate risk, and occasionally, it would go as far as giving a high risk.”

The authors believe the problem is likely due to the level of randomness built into the current version of the software, ChatGPT4, which helps it vary its responses to simulate natural language. This same randomness, however, does not work well for healthcare uses that require a single, consistent answer, Heston said.

“We found there was a lot of variation, and that variation in approach can be dangerous,” he said. “It can be a useful tool, but I think the technology is going a lot faster than our understanding of it, so it’s critically important that we do a lot of research, especially in these high-stakes clinical situations.”

Chest pains are common complaints in emergency rooms, requiring doctors to rapidly assess the urgency of a patient’s condition. Some very serious cases are easy to identify by their symptoms, but lower risk ones can be trickier, Heston said, especially when determining whether someone should be hospitalized for observation or sent home and receive outpatient care.

Currently medical professionals often use one of two measures that go by the acronyms TIMI and HEART to assess heart risk. Heston likened these scales to calculators with each using a handful of variables including symptoms, health history and age. In contrast, an AI neural network like ChatGPT can assess billions of variables quickly, meaning it could potentially analyze a complex situation faster and more thoroughly.

For this study, Heston and colleague Dr. Lawrence Lewis of Washington University in St. Louis first generated three datasets of 10,000 randomized, simulated cases each. One dataset had the seven variables of the TIMI scale, the second set included the five HEART scale variables and a third had 44 randomized health variables. On the first two datasets, ChatGPT gave a different risk assessment 45% to 48% of the time on individual cases than a fixed TIMI or HEART score. For the last data set, the researchers ran the cases four times and found ChatGPT often did not agree with itself, returning different assessment levels for the same cases 44% of the time.

Despite the negative findings of this study, Heston sees great potential for generative AI in health care – with further development. For instance, assuming privacy standards could be met, entire medical records could be loaded into the program, and an in an emergency setting, a doctor could ask ChatGPT to give the most pertinent facts about a patient quickly. Also, for difficult, complex cases, doctors could ask the program to generate several possible diagnoses.

“ChatGPT could be excellent at creating a differential diagnosis and that’s probably one of its greatest strengths,” said Heston. “If you don’t quite know what’s going on with a patient, you could ask it to give the top five diagnoses and the reasoning behind each one. So it could be good at helping you think through a problem, but it’s not good at giving the answer.”



Journal

PLoS ONE

DOI

10.1371/journal.pone.0301854

Article Title

ChatGPT provides inconsistent risk-stratification of patients with atraumatic chest pain

Article Publication Date

16-Apr-2024

Share26Tweet16
Previous Post

Improved AI process could better predict water supplies

Next Post

ACS inaugural report shows mortality for preventable cancers among native Hawaiian and other Pacific Islanders in U.S. is 2-3 times as high as white people

Related Posts

blank
Medicine

HM-TARGET: Personalized Real-Time Hemodynamic Targets Unveiled

August 7, 2025
blank
Medicine

National Study Advocates for Expanded Vaccine Screening in Emergency Departments

August 7, 2025
blank
Medicine

Early-Life Famine Exposure, Obesity, and Testosterone Links

August 7, 2025
blank
Medicine

Modified-Vaccinia Ankara Vaccine Blocks Monkeypox Transmission

August 7, 2025
blank
Medicine

Asthma Medication Zileuton Prevents Food Allergy Reactions in Mice

August 7, 2025
blank
Medicine

Urinary Complement Proteome Predicts Diabetic Kidney Disease Progression

August 7, 2025
Next Post
ACS AANHPI cancer facts and figures report

ACS inaugural report shows mortality for preventable cancers among native Hawaiian and other Pacific Islanders in U.S. is 2-3 times as high as white people

  • Mothers who receive childcare support from maternal grandparents show more parental warmth, finds NTU Singapore study

    Mothers who receive childcare support from maternal grandparents show more parental warmth, finds NTU Singapore study

    27531 shares
    Share 11009 Tweet 6881
  • University of Seville Breaks 120-Year-Old Mystery, Revises a Key Einstein Concept

    942 shares
    Share 377 Tweet 236
  • Bee body mass, pathogens and local climate influence heat tolerance

    641 shares
    Share 256 Tweet 160
  • Researchers record first-ever images and data of a shark experiencing a boat strike

    506 shares
    Share 202 Tweet 127
  • Warm seawater speeding up melting of ‘Doomsday Glacier,’ scientists warn

    310 shares
    Share 124 Tweet 78
Science

Embark on a thrilling journey of discovery with Scienmag.com—your ultimate source for cutting-edge breakthroughs. Immerse yourself in a world where curiosity knows no limits and tomorrow’s possibilities become today’s reality!

RECENT NEWS

  • Ultrahigh-Performance XYθz Nanopositioner Revolutionized
  • Trauma and Stress Coping in Exercising vs. Non-Exercising Students Post-Earthquake
  • Unprecedented Large-Scale Aquifer Recovery Achieved
  • HM-TARGET: Personalized Real-Time Hemodynamic Targets Unveiled

Categories

  • Agriculture
  • Anthropology
  • Archaeology
  • Athmospheric
  • Biology
  • Bussines
  • Cancer
  • Chemistry
  • Climate
  • Earth Science
  • Marine
  • Mathematics
  • Medicine
  • Pediatry
  • Policy
  • Psychology & Psychiatry
  • Science Education
  • Social Science
  • Space
  • Technology and Engineering

Subscribe to Blog via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Join 4,859 other subscribers

© 2025 Scienmag - Science Magazine

Welcome Back!

Login to your account below

Forgotten Password?

Retrieve your password

Please enter your username or email address to reset your password.

Log In
No Result
View All Result
  • HOME
  • SCIENCE NEWS
  • CONTACT US

© 2025 Scienmag - Science Magazine

Discover more from Science

Subscribe now to keep reading and get access to the full archive.

Continue reading