Artificial intelligence models may recommend different treatments for the same medical condition based solely on a patient’s socioeconomic and demographic characteristics, researchers warn.

The researchers invented nearly three dozen different patients and asked nine healthcare large language AI models how each one should be managed, in a thousand different emergency room situations.

Despite identical clinical details, the AI models occasionally altered decisions based on patients’ personal characteristics, affecting priority for care, diagnostic testing, treatment approach, and mental health evaluation, the researchers reported in Nature Medicine.

For example, advanced diagnostic tests such as CT scans or MRI were more often recommended for high-income patients, while low-income patients were more frequently advised to undergo no further testing, somewhat mimicking real-world healthcare inequities.

The problems were seen in both proprietary and open-source AI models, the researchers found.

“AI has the power to revolutionize healthcare, but only if it’s developed and used responsibly,” study co-leader Dr. Girish Nadkarni of the Icahn School of Medicine at Mount Sinai in New York said in a statement.

“By identifying where these models may introduce bias, we can work to refine their design, strengthen oversight, and build systems that ensure patients remain at the heart of safe, effective care,” added coauthor Dr. Eyal Klang, also of the Icahn School.