How to appraise a diagnostic test study
Read diagnostic accuracy studies with a two-by-two table, a careful look at recruitment and reference standards, and a check of real-world usefulness.
When reading a diagnostic accuracy study, find out how well the test identifies the condition compared with a reference standard. The headline statistics do not give you enough to judge whether the results are useful on their own. You also need to know about the population tested, how the condition was established and what happened to people whose results were unclear.
Begin by naming the question: in which population, for which condition, and at what stage of the testing process? A test designed to guide decisions about further investigation has a different role from one intended to replace an existing investigation.
Look at the people before the percentages
The way participants were recruited can inflate the apparent performance of a test. Comparing healthy volunteers with people who have advanced disease is easier than distinguishing between patients with similar symptoms and several possible explanations.
Look for a description of where and when participants were recruited, and the inclusion and exclusion criteria. Were participants selected consecutively or because suitable test results were available? Determine whether they resemble the population in which you wish to use the test. Factors such as age, disease severity, prior investigations and co-existing medical problems can affect how well a test performs.
Examine the reference standard
Reference standards are the methods used to establish whether the target condition is present. These may involve laboratory procedures, imaging techniques, clinical assessments or follow-up. Being labelled a reference standard does not mean that a method is error-free.
Was the same appropriate reference standard applied to all participants? If only people with positive test results underwent confirmation, some missed cases may never be detected. If positive and negative results led to confirmation through different methods, the groups may not have been assessed comparably.
Interpretation should also be sufficiently independent. Knowledge of one result can affect the reading of another, especially when judgement is involved. Studies should explain what information each assessor could see, the time between tests, and whether treatment or disease progression could have changed the underlying condition.
Rebuild the two-by-two table
In a hypothetical study of 1,000 people, the reference standard identifies the condition in 100. The new test produces positive results in 90 of those 100 and in 180 of the remaining 900 people without the condition.
There are four groups:
- True positives: 90 people.
- False positives: 180 people.
- False negatives: 10 people.
- True negatives: 720 people.
Sensitivity is the proportion of people with the condition who test positive: 90/100, or 90%. Specificity is the proportion of people without the condition who test negative: 720/900, or 80%. Both calculations start with groups whose condition status is known.
A different denominator answers a common question after testing. Of 270 positive results, 90 are true positives, so the positive predictive value is 33.3%. Of 730 negative results, 720 are true negatives, so the negative predictive value is approximately 98.6%. A test can therefore have fairly high sensitivity while most positive results in a particular population are false positives.
These numbers demonstrate how the calculations work. They do not identify a specific test nor suggest a testing strategy.
Check thresholds, uncertainty and missing results
For a numerical test, the cut-off separating positive and negative results changes both sensitivity and specificity. Was the threshold specified before the results were reviewed? A threshold chosen solely because it performed best in the same small dataset needs evaluation in new data.
Read confidence intervals alongside the point estimates. Find out how the investigators handled indeterminate tests, missing reference results and exclusions. Removing difficult cases can produce a cleaner table while making the estimate less realistic.
Ask what the result would change
Accuracy and utility relate to one another but address different questions. Consider the consequences of false positives and false negatives, and the burden of further investigations following test results. Could using the results improve an outcome that matters?
Conclude with a few sentences in ordinary language. Report the population, the main accuracy estimates and their uncertainty, the most important possible bias, and the settings to which the findings might reasonably apply. These details convey more than simply declaring a test accurate or inaccurate.
Methodology references
- STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies
- Cochrane Handbook for Diagnostic Test Accuracy, Chapter 8: Assessing risk of bias and applicability
- Interpreting diagnostic accuracy studies for patient care