According to a diagnostic study reported in JAMA Dermatology by Anriot et al, a modern artificial intelligence (AI) foundation model outperformed less experienced clinicians but did not match the diagnostic performance of expert dermatologists when tested under realistic clinical conditions. The study examined whether the strong performance of AI systems reported in controlled settings translates to the complexity of routine clinical practice.
Study Details
The study included 652 physicians who completed 1,092 test iterations from March 2023 to August 2025 using 1,117 standardized clinical cases from the Test of Dermoscopy for International Validation (TODIV) platform. Each case included patient demographics, lesion history, at least one macroscopic photograph and one dermoscopic image, and associated metadata. Unlike many previous AI studies, the dataset intentionally included uncommon and diagnostically challenging lesions while preserving the variable image quality encountered in routine practice.
The researchers evaluated a first-generation convolutional neural network and two versions of the PanDerm foundation model: a unimodal model using dermoscopic images alone, and a multimodal model incorporating clinical photographs and metadata. Human participants ranged from physicians with less than 1 year of dermoscopy experience to dermatologists with more than 10 years of experience. The primary endpoint was multiclass diagnostic accuracy across nine diagnostic categories, with secondary analyses assessing sensitivity, specificity, and balanced accuracy.
Key Findings
Overall, physicians achieved a mean diagnostic accuracy of 65.9%—significantly outperforming the first-generation convolutional neural network (56.7%). Diagnostic performance increased with experience, from 59.1% among clinicians with less than 1 year of dermoscopy experience to 74.2% among experts with more than 10 years of experience. The PanDerm unimodal model achieved 72.2% accuracy, outperforming clinicians with fewer than 3 years of experience but remaining significantly less accurate than the most experienced dermatologists. The multimodal model reached 66.3% accuracy and showed no overall advantage over physicians.
For the task of distinguishing benign from malignant lesions, the unimodal foundation model achieved the highest balanced accuracy (0.82 vs 0.65 for human readers) and an area under the curve of 0.91 (95% confidence interval [CI] = 0.89–0.93, P < .001), compared with 0.78 (95% CI = 0.76–0.81, P < .001) for the human consensus assessment. It also demonstrated markedly higher specificity than physicians (94% vs 65%), although expert dermatologists maintained the highest sensitivity for detecting malignancies.
The investigators noted that strong performance in identifying malignant lesions did not translate into superior accuracy for assigning the correct diagnosis across multiple disease categories.
The study authors concluded: “In this diagnostic study, a modern foundation model surpassed readers with less than 3 years of experience on accuracy of skin lesion diagnosis and matched those with 3 to 10 years of experience but remained inferior to experts with more than 10 years of experience, highlighting both the promise and current limitations of AI in dermatologic diagnosis.”
Julien Anriot, MD, of Claude Bernard University Lyon 1, France, is the corresponding author for the JAMA Dermatology article.
DISCLOSURE: For full disclosures of the study authors, visit jamanetwork.com.
