Artificial intelligence tools show promise as a mental health resource but may also pose major safety concerns, according to research from the University of Southern California’s Viterbi School of Engineering released Tuesday.
The new USC study suggests that all the large language models (LLMs) they tested showed “strong performance in general communication, receiving high ratings for overall quality, empathy and specificity from the mental health professionals who evaluated the responses.” However, the LLMs tested by the USC researchers were flagged for “providing varying levels of unauthorized medical advice,” which raises “significant” safety concerns.
LLMs are AI systems that can understand and produce human language by processing huge swaths of text data, accordingto IBM. Some commonly reported issues among all of the LLMs include overgeneralization, making unsubstantiated assumptions and providing users unnecessary feedback, the study suggests.
Google’s Gemini 1.5 Pro was most frequently flagged for “lacking empathy or emotional attunement,” according to the study. Meanwhile, Meta’s Llama 3.3 was most frequently flagged for providing users “unauthorized” health advice, per the report.
OpenAI‘s GPT-4 was commonly flagged for offering users “unconstructive feedback and showing little personalization or relevance to the patient’s specific situation,” the study found.
“Other patterns observed to be unintentionally judgmental, like calling a behavior ‘not normal,’ or giving apathetic responses, when models were stress-tested,” according to the report. “The team also found that when asked to grade their own performance, AI judges were unreliable, as they consistently overestimated their own performance and missed safety risks that human experts easily identified.”
The study’s release comes as many Americans have recently been increasingly turning to AI tools such as chatbots when seeking mental health advice and information. A November 2025 study in JAMA Network Open found that 13.1% of U.S. adolescents and young adults aged 12 to 21 years said they use generative AI for mental health advice.
Some experts previously told the Daily Caller News Foundation that while utilizing AI programs as a mental health resource is affordable and can help patients access information efficiently, it may also result in major issues, such as providing users with biased data.
