A startling new analysis has revealed that more than one-third (37%) of answers generated by ChatGPT are considered untrustworthy by verified experts in fields such as medicine, law, and veterinary science. The findings highlight significant risks associated with relying on AI-generated information, including instances of misleading and potentially hazardous advice.
Verified Experts Sound the Alarm
The analysis, conducted by the human experts behind JustAnswer’s newly launched Pearl.com AI Answer Review tool, underscores the pressing need for caution when using AI for critical decisions. These experts, who include doctors, lawyers, and veterinarians, evaluated ChatGPT’s responses and assigned them a “Trust Score” from 1 (least trustworthy) to 5 (most accurate and trustworthy). Alarmingly, 37% of the reviewed answers received a score of 3 or lower.
“These findings reflect the troubling inaccuracy rates observed in other studies of current AI models. With more people relying on AI for critical life and health issues, it is terrifying how often these AI tools deliver misinformation that can lead to devastating consequences, including health issues, financial ruin, or even loss of life,” said Andy Kurtzig, CEO of JustAnswer.
Concerning Statistics and Real-World Examples
In addition to the trust scores, the experts’ survey revealed even more concerning statistics about the reliability of AI-generated responses:
- 42% of ChatGPT answers were “missing crucial details”
- 14% were “misleading”
- 4% provided “dangerous information”
Several real-world examples illustrate the potentially life-threatening consequences of these inaccuracies:
- A veterinarian flagged a ChatGPT response about treating a pet’s wound as dangerous, noting it could lead to serious internal organ damage.
- A lawyer pointed out that incorrect information given about inheritance laws failed to account for jurisdiction-specific regulations, which could have severe legal repercussions.
- A doctor reviewed a ChatGPT response regarding a possible subarachnoid hemorrhage and found the medical advice to be misleading and potentially life-threatening.
Pearl AI Answer Review tool invites human experts to verify AI-generated responses for accuracy, generating a trust score of 1 to 5.
The Need for Human Oversight
These findings raise significant concerns about the reliability of AI-generated information, particularly in areas where accuracy is critical. “While AI technologies hold immense potential, human oversight and expertise are crucial to harnessing their benefits and mitigating their risks,” Kurtzig emphasized.
The Pearl AI Answer Review tool, which invites human experts to verify AI-generated responses for accuracy, aims to address these issues by providing a trust score and ensuring that users receive reliable information.
JustAnswer has been a trusted platform since 2003, connecting consumers with verified and vetted professionals online. The site boasts about 12,000 experts across more than 150 categories, including health, legal, finance, veterinary, tech, and home improvement. With millions of users seeking reliable answers to their questions, JustAnswer plays a vital role in providing expert advice in a wide range of fields.
For more information about JustAnswer and the Pearl AI Answer Review tool, visit JustAnswer.