How to judge evidence about screening
Separate earlier detection from improved outcomes, understand screening biases, and compare absolute benefits with false positives and overdiagnosis.
A screening test might identify more diseases and diagnose them sooner — both could seem like a strong argument for a programme's effectiveness, but each requires a much closer look before we consider it as proof that the programme will help.
Screening identifies conditions or risk markers in people without the symptoms that would normally lead to investigation. Screening programmes include a series of steps: invitation to take the screening test, multiple screenings, confirmation testing, treatment for those identified with the targeted condition, and follow-up care. All of these components contribute to the final outcome for participants in the screening programme; for example, individuals who receive a false positive screening result may undergo additional investigations.
Identify the Programme Being Evaluated
First, determine the specific programme being evaluated. Specifically, start with the details of the actual offer made to eligible participants: Who were invited? What type of test was administered? At what age range were participants screened? How often were participants rescreened? What actions followed a positive screening result?
Findings from one schedule should not be assumed to apply to a different scheduling model or to a different population.
Accuracy vs. Consequences
While accuracy measures how well a screening test can identify a particular condition, programme evaluations seek to measure the effects associated with conducting screening tests. For example, while screening tests may provide earlier identification of illnesses, early detection may offer limited benefits if earlier treatment does not lead to improved outcomes. Additionally, follow-up investigations and confirmatory diagnostic procedures may also have negative impacts upon patients. Therefore, having a good screening test is just one piece of the rationale for establishing a screening service.
Misleading Survival Data
Imagine a theoretical disease in which symptoms appear in year five and death occurs in year eight. Without screening, survival after diagnosis is three years. However, suppose this disease is detected through screening in year two, yet no alteration in the timing of death occurs. Because screening occurred three years prior to symptom-onset, the documented length-of-survival post-diagnosis now increases to six years.
No aspect of mortality has been altered due to screening. The increase in length-of-survival post-diagnosis is entirely attributed to an earlier calendar date on which diagnosis occurred. This phenomenon is called lead-time bias. Moreover, because lead-time bias cannot be resolved solely through documentation of survival after diagnosis, the use of length-of-survival post-diagnosis data as sole means of determining whether a screening programme has resulted in increased longevity is inappropriate.
An additional problem arises from length-bias. Disease entities that continue to be detectable for longer periods are more likely to be detected during repeated periodic screening. Thus, slower-progressing disease cases can comprise a greater proportion of all disease cases detected via screening. Consequently, this group can appear to perform better even if the intervention provides no health benefit.
Therefore, the comparative metric must be based on health outcomes in comparable populations offered screening versus those following alternative approaches. Documentation of survival only among diagnosed persons cannot address these problems. Similarly, comparing screen-positives with screen-negatives cannot demonstrate the effectiveness of a screening programme.
Separating False Positives and Overdiagnosis
False positives occur when a screening result indicates presence of a condition although the participant does not actually have that condition as confirmed by an appropriate verification procedure. The potential consequences of resolving false positive results (e.g., anxiety, additional doctor visits or invasive diagnostic testing) deserve consideration regardless of how quick or easy the original screening test was.
However, overdiagnosis refers to situations where a true positive screening result indicates that an individual has a condition that would never have produced clinical manifestations or death during his/her lifetime. In such instances, treatment may offer no corresponding health benefit while still causing burdens or adverse effects. The discovery may expose the person to unnecessary treatment and its risks or side effects.
Sensitivity-based improvements in detection capability can shift the proportions of false positives and overdiagnoses. Documenting only initial screens overlooks everything else that happens afterward. Both initial screens and subsequent investigations and treatment need to be documented.
Fair Comparison
When evaluating a screening programme for effectiveness, there are many ways that attendees and non-attendees may differ systematically in terms of behaviours related to their health status. Therefore, simply comparing the two groups may not accurately portray the degree of benefit attributable to participation in screening. Randomisation can reduce systematic differences between people invited for screening and those not invited. Analysis by invitation preserves that original comparison between invitation groups more reliably than analysing only those who attended.
Potential limitations remain (participation rates, screening outside the assigned programme, incomplete follow-up). A short follow-up period may reveal extra diagnoses quickly while missing later effects on illness or death. Potential limitations also arise from endpoints. Different types of endpoints, such as deaths from the target condition, deaths overall, serious illness and quality of life, provide differing answers to questions about the impact of a screening programme. Claims regarding one endpoint should not necessarily translate silently into claims regarding another endpoint.
Express Effects Using Comparable Numbers
Assume a fictional ten-year study reported forty deaths per ten thousand people from the target condition without screening and thirty deaths per ten thousand people offered screening. The difference in deaths is ten per ten thousand or 0.1 percentage points. The relative reduction is twenty-five percent.
If this difference reflects a causal relationship, approximately one thousand people would need to be offered the programme over ten years for one death from the target condition to be prevented on average. Read this estimate with recognition of uncertainty; it does not guarantee a benefit to any single individual.
When possible place harms side-by-side with benefits for equivalent numbers of people over the same period. Where applicable report false positives, additional testing and overdiagnosis together with estimates of benefits and uncertainties. Explain how well the evidence supports your proposal for implementation and/or extension of the programme.
Methodology references
- USPSTF Procedure Manual, Section 6: Methods for arriving at a recommendation
- Cancer Screening Overview, National Cancer Institute
- National Lung Screening Trial: Questions and Answers, National Cancer Institute