Master's thesis · Monash University · 2025

From prompts
to probes.

Four foundation models, three public retinal benchmarks and two transfer regimes. The question was practical: which model works when specialist labels are scarce?

Explore the interactive benchmark Inspect the source

Models: 4
Benchmarks: 3
Best zero-shot AUROC: 0.921
Validation: 5-fold

Research question

How much supervision does a retinal model really need?

Specialist labels are expensive and agreement can vary even between experts. Foundation models promise useful transfer with little or no task specific annotation, but the best pretraining strategy changes with the disease and the label budget.

I compared zero-shot prompting with frozen encoder linear probing at 5%, 10% and 20% label budgets, using the same stratified folds for every model.

Model set

Three pretraining philosophies.

Domain contrastive, broad contrastive and self-supervised visual learning were evaluated under the same protocol.

FLAIR

Retinal vision-language specialist

ResNet-50 · BioClinicalBERT

About 288,000 fundus image and clinical text pairs

OpenCLIP

Web-scale visual generalist

ViT-H/14 · LAION-2B

About 2 billion web image and text pairs

BiomedCLIP

Biomedical visual generalist

ViT-B/16 · PubMedBERT

15 million biomedical figure and caption pairs

RETFound

Self-supervised retinal model

ViT-Large · masked autoencoder

1.6 million retinal images

Result 01

Zero-shot performance.

Mean across three clinically structured prompt templates. Higher is better.

Model	MESSIDOR Macro-F1	REFUGE AUROC	ODIR-200×3 Macro-F1
FLAIR	0.735 ± 0.049	0.921 ± 0.049	0.366 ± 0.016
BiomedCLIP	0.471 ± 0.042	0.649 ± 0.037	0.709 ± 0.032
OpenCLIP	0.353 ± 0.001	0.530 ± 0.046	0.399 ± 0.131

FindingDomain alignment dominated zero-shot transfer. FLAIR led on diabetic retinopathy and glaucoma, while broader biomedical pretraining helped BiomedCLIP on the mixed disease task.

Result 02

Limited labels reshuffled the ranking.

Representative 20% label results from frozen encoder linear probes.

Model	MESSIDOR Macro-F1	REFUGE AUROC	ODIR-200×3 Macro-F1
FLAIR	0.700	0.870	0.900
OpenCLIP	0.648	0.891	0.878
BiomedCLIP	0.613	0.874	0.870
RETFound	0.619	0.836	0.820

FindingEncoder richness mattered once a small classifier could learn from labels. OpenCLIP moved from weak zero-shot glaucoma alignment to the strongest REFUGE probe.

Interpretation

The best model depends on the constraint.

A single leaderboard winner would hide the useful engineering decision.

No labels

Choose a model whose vision and language pretraining already matches the target domain.

A few labels

Probe several frozen encoders. Generic features can become highly competitive once a classifier is allowed to learn.

Clinical use

AUROC is not enough. Select a threshold, report sensitivity at fixed specificity and calibrate probabilities.

Known limit

These are public benchmark results, not a deployed diagnostic system. Local population validation and clinical governance remain essential.

Method

Paired, reproducible comparison.

01DatasetsMESSIDOR for diabetic retinopathy, REFUGE for glaucoma and ODIR-200×3 for mixed retinal disease.

02ValidationStratified 5-fold cross-validation with identical fold assignments across models and budgets.

03StatisticsPaired tests, bootstrap confidence intervals, calibration analysis and sensitivity at 90% specificity.

04ComputeAll experiments ran with PyTorch and Apple MPS acceleration on a single Apple silicon laptop.