Accuracy is the wrong first question for clinical AI

Suppose 1 in 100 scans contains a particular finding. A model that labels every scan as normal is 99% accurate, and it never finds anything. Accuracy tells you very little when the condition you care about is rare.

Measure the errors separately

Sensitivity asks how many real findings the model catches. Specificity asks how many normal cases it correctly leaves alone. They trade off against each other, so the right balance depends on which mistake costs more in that workflow.

Prevalence changes the result too. Take a model with 90% sensitivity and 90% specificity at that same 1 in 100 rate. Out of 1,000 scans, it flags 9 of the 10 real findings and 99 of the 990 normal scans. Only about 1 in 12 alerts is correct, so a reviewer needs to expect that workload.

Test on data that resembles the site

• Keep a reference set labelled by clinicians, separate from any data used in training.
• Report results by subgroup, such as age, sex, scanner or site. A strong average can hide a weak group.
• Test on data from the place the model will run. Equipment, protocols and populations all differ.

Evaluate the workflow, not only the model

A model can score well and still make care worse if its output arrives late, sits outside the clinician's normal tool or cannot be overridden. Decide who reviews each output, how a disagreement is recorded and what happens when the model is unavailable.

Keep measuring after release

Performance can drift when equipment, coding practice or the patient mix changes. Sample outputs for review on a schedule, track overrides and agree in advance what triggers a pause. Depending on the intended use and the market, the software may also be regulated as a medical device, so check this early.