AI Applications on AIdataNewsGlobal: How a Fine-Tuned Inception v3 Compared With GPT-6 Astra and Claude Fable 5.1 on 200 Skin Lesions

When an AI application compares image outputs with language-model replies, the evaluation design can matter as much as the model name. In the October 2026 skin-lesion trial reviewed here for AI Applications on AIdataNewsGlobal, two general frontier LLMs and one specialized convolutional network were asked to classify biopsy-proven cases. The ten-year-old Inception v3 network, fine-tuned for about twenty minutes, produced 125 correct first answers (62%) against 84 and 85 (42%) for GPT-6 Astra and Claude Fable 5.1. The application lesson is straightforward: for a narrow visual task, a small specialized CNN can beat larger chat models if the inputs, labels, thresholds, and caveats are inspected carefully.

Image: In the first case pair, the ear lesion was an invasive melanoma under 0.8 mm thick. The two LLMs answered seborrhoeic keratosis and said the patient could be discharged; the CNN answered melanoma. Left: clinical photograph; right: dermoscopic image. Source images: MILK10k, MILK study team, CC BY-NC.

The images came from MILK10k, a public dataset released through the International Skin Imaging Collaboration in 2025. In the dataset, each lesion is photographed twice: a clinical close-up and a dermoscopic image taken through the illuminated magnifying lens dermatologists use. From that pool, 200 lesions were randomly selected, and all were confirmed by histopathology.

• 100 lesions that should not be discharged: 30 melanomas, 25 basal cell carcinomas, 25 squamous cell carcinomas (including Bowen’s disease) and 20 actinic keratoses.
• 100 lesions that are safe to discharge: 50 benign naevi (moles) and 50 seborrhoeic keratoses.
• Each model answered from the 6 diagnosis labels, or answered unsure when it did not choose one label.

Every lesion was biopsied, including the benign ones, which makes the sample less forgiving: a mole that somebody decided to cut out is one that worried a clinician.

A basal cell carcinoma on the trunk of a 75-year-old man; all three models answered correctly.

An invasive melanoma under 0.8 mm thick, with very little pigment. GPT-6 Astra called it a basal cell carcinoma with 95% confidence, while Claude Fable 5.1 and the CNN called it a melanoma.

A seborrhoeic keratosis on the leg. Both LLMs called it an actinic keratosis, and the CNN called it a squamous cell carcinoma; all three were wrong, though all three would have kept it rather than discharge it.

Figure: Three additional cases from the 200 test lesions. The left panel is the clinical photograph, and the right panel is the dermoscopic image. Images: MILK10k, MILK study team, CC BY-NC.

Prompt shape and model setup

The LLMs received both images and nothing else. They were asked, as a clinician might use them, to give the most likely of the six diagnoses (or “unsure”), a confidence, a second choice, and whether the lesion could be safely discharged with no treatment, biopsy or follow-up. Both models received the same prompt at default settings, and neither refused a case.

The CNN is not state of the art by design. The original test used Inception v3, a network first published by Google in 2015 and pretrained on everyday photographs, then fine-tuned all layers on 68,011 dermoscopic and clinical images from the ISIC Archive on a single H100 GPU. Before training, these 200 test lesions and every other photograph of the same lesion or patient were removed from its data, including the validation set, so it had never seen them; they were not used to choose the model or set its threshold. It scored both images of each lesion, combined them with most of the weight on the dermoscopic image, and chose from the same six diagnoses.

The CNN is available for readers to try. It is small enough to run entirely in a phone browser using WebGPU. Nothing is uploaded, and its random sample button draws from these same 200 lesions.

Where the scoring gaps appeared

Overall and on most diagnoses, the CNN was correct more often. If a second choice is allowed, it reached 173 of 200, versus 134 and 138. It named 13 of the 30 melanomas, compared with 8 and 11. The largest difference came on harmless lesions: it recognised 37 of 50 moles and 26 of 50 seborrhoeic keratoses, while GPT-6 Astra recognised 16 and 13, and Claude Fable 5.1 recognised 33 and 6. On basal cell carcinoma, it was one short of GPT-6 Astra, and on actinic keratosis it trailed both LLMs.

The models also failed in different ways. Each LLM had a favourite answer. GPT-6 Astra called 75 lesions basal cell carcinoma when 25 were, including 14 of the 30 melanomas. Claude Fable 5.1 called 53 lesions actinic keratosis when 20 were. The CNN’s errors on melanoma went the other way: of the 17 it got wrong, it called 8 a naevus and 6 a seborrhoeic keratosis.

Score readout in a table

	GPT-6 Astra	Claude Fable 5.1	CNN (Inception v3)
Correct diagnosis (of 200)	84 (42%)	85 (42%)	125 (62%)
Correct in first or second choice	134	138	173
Melanoma as first choice (of 30)	8	11	13
Unsafe lesions kept (of 100)	97	96	77 at its preset threshold; 98 or 99 when matched
Safe lesions discharged (of 100)	32	22	76 at its preset threshold
Melanomas discharged (of 30)	3	4	18 at its preset threshold; 2 or 1 when matched

Limitations that matter for reuse

• Small numbers. Two hundred lesions can show the pattern, but they cannot rank close results; 30 melanomas leave wide uncertainty on the figure that matters most. Each LLM case was run once, and answers vary between runs.
• Hard cases. Every lesion was biopsied. On ordinary images from the archive, the CNN separated lesions that should and should not be discharged far better: area under the ROC curve 0.96, against 0.86 here.
• Contamination. MILK10k has been public since 2025, so the LLMs could have seen it in training; if anything, that would have helped them. The CNN certainly had not.
• Not state of the art. Inception v3 is a baseline. State of the art skin cancer diagnosis AI uses ensembles of image transformers trained on more data, and the regulated products built on them perform considerably better.

When the question becomes patient-style

When asked as a patient instead, using only the clinical photograph and the question "do I need to see a doctor?", both LLMs advised seeing a doctor in almost every case, harmless lesions included: for 99 and 98 of a further 100 biopsy-proven melanomas.

Application checklist before copying this comparison

• Compare the task wording with production use: the same prompt, the same six diagnoses, and the same "unsure" option.
• Track whether the input includes both clinical and dermoscopic images or only one.
• Record confidence, second choice, and discharge or escalation separately.
• Look for favorite-answer bias: one model may overcall a common diagnosis.
• Check whether the test set has been public and could already be inside training data.
• Remember that a small, old, specialised network with an outdated architecture can beat two frontier LLMs in a narrow recognition task.

A practical evaluation path

1. Define the decision to be supported: diagnosis, discharge, biopsy, or follow-up.
2. Prepare an evaluation set with confirmed outcomes, and remove duplicate lesion or patient photographs.
3. Give the model the exact inputs and prompt that the production task would use.
4. Score first choice, second choice, confidence, and keep or discharge decisions separately.
5. Check thresholds, failures, and favorite-answer patterns before sharing results.

Common mistakes

• Treating a public test set as unseen without checking contamination.
• Using only the first answer and ignoring whether a second answer is available.
• Assuming a chat model has been exposed to the same dermoscopic images.
• Ignoring image type; dermoscopy carries more weight in this test.
• Reading a 62% versus 42% result as deployable clinical performance rather than a limited sample.
• Leaving a preset threshold unchanged when the workflow requires a different keep or discharge rate.

Action recommendations for teams applying vision AI

• For narrow image tasks, include a small specialised CNN as a control or candidate.
• For language-model workflows, test patient-style and clinician-style prompts because they can behave differently.
• For user-facing tools, WebGPU may help when nothing should be uploaded.
• For regulated uses, compare with state of the art products rather than relying on a baseline Inception v3.
• For reproducible reviews, archive the cases, prompts, replies, scoring code, and method notes in one download.
• For transparent reporting, credit the data source: MILK10k, MILK study team, CC BY-NC.

What this comparison suggests for application teams

The cases, prompts, every LLM reply and the scoring code are available in one download, and the CNN is here. The original test author designed the test and made the clinical rulings. Claude Code, Anthropic's coding tool, which runs on a Claude model, produced the analysis and the first drafts; one of the two LLMs tested is also Anthropic's. Scoring was done by script against the histology.

For AI & Data News readers, the final takeaway is not that Inception v3 should replace every frontier model. It is that application teams should evaluate visual AI against the exact workflow they intend to run: the image type, the prompt, the label set, the threshold, and the contamination risks.

This AIdataNewsGlobal variant is credited to David Nakamura for AI & Data News. Contact: editorial@aidatanews.com.