TL;DR
Get oils, diffusers and self-care delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
A dermatologist’s report compares two frontier language models with a fine-tuned Inception v3 CNN on 200 biopsy-confirmed skin lesions. The CNN named the correct diagnosis in 125 cases (62%), compared with 84 (42%) for GPT-6 Astra and 85 (42%) for Claude Fable 5.1; the small, single-dataset test does not establish how the models would perform in clinical practice.
A dermatologist’s test of 200 biopsy-confirmed skin lesions found that a fine-tuned, decade-old Inception v3 convolutional neural network (CNN) named the correct diagnosis more often than two frontier language models. The CNN was right on 125 lesions (62%), while GPT-6 Astra and Claude Fable 5.1 were each right on about 42%, according to the report published by Lynch Lab.
The dermatologist gave all three systems clinical and dermoscopic images of the same lesions and asked them to choose among six diagnoses, state their confidence, name a second choice and advise whether the lesion could be discharged, followed up or biopsied. GPT-6 Astra classified 84 of 200 correctly and Claude Fable 5.1 classified 85 of 200. The CNN, based on Google’s Inception v3 architecture first published in 2015, classified 125 correctly.
The sample included 100 lesions the author categorized as not safe to discharge: 30 melanomas, 25 basal cell carcinomas, 25 squamous cell carcinomas and 20 actinic keratoses. The other 100 were 50 naevi, or moles, and 50 seborrhoeic keratoses. Every lesion had been biopsied and its diagnosis confirmed by histopathology, including the benign cases.
Performance differed by diagnosis. For melanoma, the CNN correctly identified 13 of 30 cases (43%); Claude Fable 5.1 identified 11 (37%) and GPT-6 Astra eight (27%). The figures show performance on this particular sample, not the systems’ general accuracy across patients or settings. The report says both language models called an invasive melanoma on the ear a benign seborrhoeic keratosis and said the patient could be discharged; the CNN identified the melanoma using the dermoscopic image.
Why Image-Specific Models Scored Higher
The result highlights that a model built specifically to analyze medical images can perform differently from a general-purpose language model, even when the latter is described as highly capable. In this test, the CNN processed paired clinical and dermoscopic views and relied most heavily on the dermoscopic image. That task and input may better match its image-classification design than the systems’ general conversational capabilities.
The gap matters because a mistaken reassurance about a potentially dangerous lesion could affect whether someone seeks further assessment. But the results do not show that the CNN is ready to guide patients or clinicians: it missed more than half of the melanomas in the sample, and its overall score combines diagnoses with different risks. A correct six-way label is also not the same as a safe clinical decision.
The author says the CNN can run in a phone browser without uploading images. That technical feature does not establish that it is a validated diagnostic product. The report’s test is best read as a comparison that raises questions for further evaluation, not as evidence that any of the three tools should replace examination, biopsy or professional judgment.
How the 200-Lesion Test Was Built
The lesions came from MILK10k, a public dataset released through the International Skin Imaging Collaboration in 2025. It contains two types of image for each lesion: a clinical close-up and a dermoscopic image taken with an illuminated magnifying lens. The report says the dermatologist drew 200 lesions at random from the dataset.
For the CNN, the author fine-tuned all layers of Inception v3 on 68,011 clinical and dermoscopic images from the ISIC Archive, using a single H100 GPU. The report says the test lesions, along with other images of the same lesion or patient, were excluded from the training and validation data and were not used to select the model or set its threshold. The CNN scored both images, gave most weight to the dermoscopic view and selected from the same six diagnoses as the language models.
The sample’s composition limits how directly its results can be transferred to everyday care. Since every lesion had been biopsied, the benign lesions were not a cross-section of ordinary moles or growths; they had already prompted a clinician to remove them. The author notes this may make them harder to classify than benign lesions encountered in routine screening.
“Both LLMs called it a seborrhoeic keratosis and said it could be discharged.”
— The dermatologist author, describing the melanoma case
Limits of This Single Test
The report describes a single evaluation of 200 lesions by one dermatologist. The source material does not provide results from independent replication, testing at other hospitals, or a prospective study involving patients as they seek care. It also does not establish whether performance would hold across different cameras, image quality, skin tones, age groups or clinical settings.
Overall diagnosis accuracy does not by itself measure whether a tool makes safe decisions. The report provides category-level accuracy figures, but the supplied material does not fully establish how often each model would recommend discharge for a cancer or whether its confidence estimates were calibrated. The sample included only 30 melanomas, and the CNN correctly named 13 of them.
The findings also compare a fine-tuned image classifier with language models prompted to interpret two images. They do not show how the LLMs would perform with additional clinical information, nor do they compare the systems against dermatologists or usual clinical care. No evidence in the report establishes regulatory clearance or suitability for patient use.
Independent Clinical Testing Needed
The next meaningful step would be independent testing on larger and more varied datasets, with patient populations and image sources kept separate from model training. Such studies would need to report results by diagnosis and relevant patient groups, as well as whether the systems’ recommendations could lead to missed cancers or unnecessary follow-up.
The report does not announce a clinical trial, regulatory review or deployment plan. Until further evidence is available, the reported scores should be treated as a finding from a limited image test, not as a basis for diagnosing a skin lesion or deciding to skip medical assessment.
Key Questions
Which model had the highest accuracy in the reported test?
The fine-tuned Inception v3 CNN named the correct diagnosis for 125 of 200 lesions (62%). GPT-6 Astra was right on 84 (42%) and Claude Fable 5.1 on 85 (42%).
Did the CNN identify most of the melanomas?
No. It correctly classified 13 of 30 melanomas (43%) in this sample. Claude Fable 5.1 identified 11 and GPT-6 Astra identified eight, according to the report.
Does this show the CNN is safe to use for diagnosis?
No. The report describes a limited test of images, not a clinical trial or validation for patient care. Its results do not show that the CNN can replace a clinician, biopsy or other medical assessment.
What images and cases were used?
The dermatologist tested 200 lesions from the MILK10k public dataset, using both clinical close-ups and dermoscopic images. All cases had diagnoses confirmed by histopathology, including the benign lesions.
Source: hn
Fall Picks
fall essentials
As an affiliate, we earn on qualifying purchases.
