OpenAI's o1 Model Outperforms ER Doctors in Diagnostic Accuracy, Harvard Study Finds

A new study led by Harvard Medical School researchers reveals that OpenAI's o1 reasoning model surpassed emergency department physicians in diagnostic accuracy during medical triage. In a trial at a Boston hospital, the AI correctly identified diagnoses in 67% of cases using limited electronic health records, while human doctors achieved 50% to 55% success. The AI excelled in high-pressure situations requiring quick decisions with minimal information and scored 89% on long-term treatment plans, such as antibiotic regimens, versus 34% for doctors using conventional tools. However, researchers caution the AI was tested only on text-based data, missing visual cues or patient distress levels. In one case, the AI identified a lupus-related diagnosis overlooked by clinicians. The authors emphasize AI is meant to support, not replace, medical professionals, though challenges remain around legal liability and accountability frameworks. They envision a future triadic care model involving doctors, patients, and AI.