Frontier Models Outperform Specialized Clinical AI in Head-to-Head Study
A Nature Medicine study pitted specialized clinical AI tools against general frontier models—GPT-5.2, Claude Opus 4.6, and Gemini 3.1 Pro—and found the specialists lost across every test. Twelve US physicians conducted a blinded, randomized review of 1,800 annotations across 500 MedQA questions, 500 HealthBench tasks, and 100 real clinical queries, giving clinical AI tools a 49–87% lower odds of receiving a higher rating than Gemini. The researchers also found that RAG retrieval systems in medical products can degrade answers by pulling in poorly integrated or irrelevant material, concluding that a strong general model with solid reasoning often outperforms purpose-built clinical alternatives.