AI Panel Outperforms Doctors in Complex Medical Simulations

The gap between medical supply and demand is widening, and Microsoft argues its latest AI system is ready to close that chasm. In a study published Thursday in *Science*, the company revealed that its diagnostic AI, MAI-DxO, correctly identified complex medical conditions more than twice as often as generalist physicians. This result marks a pivotal moment in the race among major tech firms to prove that artificial intelligence can not only match but surpass human expertise in high-stakes clinical environments.

The system operates like a virtual medical board. Rather than relying on a single algorithm, MAI-DxO coordinates a panel of AI agents that simulate a multidisciplinary team. These agents interact with patient data, order virtual tests, and debate potential diagnoses before reaching a consensus. The study utilized complex cases from the *New England Journal of Medicine* to test this orchestration. In this simulation, the AI panel achieved an accuracy rate of 83%. By comparison, primary care doctors and general internists working individually with standard reference materials reached a 36% accuracy rate. It is important to note that these specific cases are typically handled by specialists or teams in practice, who were not included in the baseline comparison.

Microsoft has joined Google and OpenAI in claiming that AI can outperform human clinicians, but the data suggests the underlying models are doing most of the heavy lifting. The study indicated that much of the performance improvement stems from the power of the underlying AI models, with Microsoft’s orchestration layer adding incremental gains. While the company tested the system with models from OpenAI, Google, Anthropic, Meta, and DeepSeek, the highest results were achieved using OpenAI’s GPT-5 and GPT-5.5.

From a financial perspective, the trade-off between cost and accuracy is nuanced. When running GPT-5 alone with a basic prompt, the system scored 78.9% on complex cases at an estimated cost of $8,400 per case. Adding the MAI-DxO orchestration reduced that cost by approximately $3,000 but only improved accuracy by about four points. For emergency department cases, the improvement in both accuracy and cost was minimal. However, with the newer GPT-5.5 model, released in April 2026, the system showed a more significant leap for complex cases, raising accuracy from 72% to nearly 87% at a slightly lower cost. The gains remained negligible for emergency cases, where both configurations scored around 85%.

The orchestration system proves most valuable when paired with less capable models. For instance, Google’s Gemini 2.5 Flash scored 46.4% independently but rose to nearly 69% with the assistance of MAI-DxO. Despite this boost, the gap between weaker and stronger models remained larger than the improvement provided by the orchestration. Microsoft researchers noted that accuracy gains naturally diminish as models approach the theoretical maximum. A spokesperson stated that the system helps optimize both accuracy and cost across the broader challenge of clinical decision-making.

Mustafa Suleyman, CEO of Microsoft AI, views these results as a foundational step toward "medical superintelligence," defined as the capacity to surpass the strongest contemporary human-led health systems in medical reasoning. Suleyman, who previously co-founded DeepMind and led Hark before joining Microsoft in 2024, emphasized that the company is working with the Mayo Clinic to train a dedicated healthcare foundation model, with a release planned for next year. He believes this model will enable Microsoft to make "really enormous leaps" in the field.

Suleyman predicts that by 2027, the industry will see a "coding style breakthrough" for medical records, allowing for highly accurate predictions of how a patient's condition might unfold. He described this foresight as "gold dust" for clinicians. The executive expects healthcare models to become an order of magnitude better in capability within the next 24 to 36 months, citing the rapid progress seen in coding and chat applications as a precedent for the "flywheel" effect of real-world iteration.

To validate these capabilities outside of simulation, Microsoft has partnered with Beth Israel Deaconess Medical Center in Boston. This collaboration will evaluate the clinical reasoning of MAI-DxO across various hospital settings, including emergency departments and urgent care. Patients will receive standard treatment while consenting patients will also interact with the AI. Microsoft and its partners will then compare the AI’s recommendations against those of human doctors. Dominic King, vice president of Health at Microsoft AI, called this the "biggest study of its kind," noting that it will provide compelling evidence for the system's ability to be deployed at greater scale.

This initiative coincides with a broader manifesto from Microsoft outlining the path to medical superintelligence. The company aims to move beyond benchmarking individual tasks, such as answering exam questions, toward integrating with clinicians and healthcare systems for improved patient outcomes. Microsoft already sells dictation and radiology tools to clinicians and sees MAI-DxO as a natural expansion of its portfolio. Suleyman stated that the company is committed to having the "absolute number one state-of-the-art medical clinical decision support products" in the coming years, positioning this focus as a key differentiator against other frontier AI labs.

The study underscores a shifting landscape in healthcare economics, where the cost of expertise is a primary driver for innovation. While the current results show a clear advantage in complex, simulated scenarios, the real-world application remains the next critical test. As Microsoft prepares to roll out its healthcare-specific foundation model next year, the industry will be watching to see if the promise of "medical superintelligence" translates into tangible improvements in patient care and operational efficiency.