• Posted April 8, 2025

AI Outperforms Urgent Care Docs, Study Says

Cough? Sore throat? UTI? Eye infection?

Artificial intelligence (AI) might one day be seeing you for these sorts of conditions, a new study says.

AI programs appear to outperform human doctors when it comes to urgent care, researchers reported in the Annals of Internal Medicine.

A medical AI program agreed with the clinical decisions of human doctors in about two-thirds of cases, according to a head-to-head test.

In the remaining third, AI’s recommendations were rated superior to those of human doctors twice as often, researchers found.

“The AI diagnosis and management recommendations were more likely to be rated as optimal compared with physicians and less likely to be rated as potentially harmful,” wrote the research team led by Dan Zeltzer, an associate professor of economics at Tel Aviv University in Israel.

If implemented responsibly under the guidance of human doctors, AI could improve patient care by providing an instant second opinion, researchers concluded.

“This study suggests that AI can enhance clinical decision making for common acute symptoms in a virtual urgent care setting,” researchers wrote.

For the study, they asked an AI program and human doctors to review 461 real-life primary care visits involving urinary, vaginal, respiratory, eye and dental complaints.

The AI was trained on a massive amount of high-quality real-world data, far beyond what any physician would see in their lifetimes, researchers said.

Four expert physicians specializing in family, internal and emergency medicine, each with at least a decade of experience, then compared the AI evaluations to those of the doctors, to see which provided the better clinical recommendations.

Most of the time, the AI’s clinical judgments jibed with those of human doctors, researchers found.

But in the third of cases where AI and human docs didn’t agree, the evaluators felt that AI did a better job twice as often.

AI also received half as many potentially harmful ratings, compared to human doctors.

This might be because AI is designed to follow medical guidelines much more strictly, and can pick up on subtle details or medical record notes that doctors might miss, researchers said.

Human doctors do have an advantage in that they can ask follow-up questions to clarify a patient’s symptoms during a visit, researchers noted.

However, AI shares human doctors’ level of caution, in that if it isn’t confident in its recommendation it will not offer one, researchers said.

In about 20% of cases, AI didn’t offer a recommendation due to caution, researchers said.

Doctors who plan to use an AI program “should know the history of any program’s use, its error rate, and its possible commercial bias,” Dr. Jerome Kassirer, a professor at Tufts University School of Medicine in Boston, wrote in an accompanying editorial.

“I believe that clinicians should first use AI programs only to confirm their own diagnoses and therapeutic recommendations,” Kassirer wrote. “If the machine output does not agree with their own judgment, they should seek to determine why the discrepancy exists. Exploration of these discrepancies can be the seeds for improving the AI tool.”

More information

The Cleveland Clinic has more on AI in health care.

SOURCES: American College of Physicians, news release, April 4, 2025; Annals of Internal Medicine, April 4, 2025