Reading a research paper beyond its headline
When reviewing an article begin with the question asked, then review the design used to obtain the data upon which the findings are based. Followed by the point estimate and measures of precision prior to determining if you agree or disagree with the author's conclusion.
Find the real question
Who did you study? What was the exposure or intervention being measured? What was the comparison group? What was the outcome being measured and how long did participants have to develop it? Measuring a lab marker at two-week post-exposure does not equate to measuring a meaningful long-term health-related outcome.
Don't accept conclusions without reading through the methodology. Is there a pre-registered protocol associated with this study? Was the primary outcome and statistical analysis specified prior to data collection? Abstracts often do not include information about key decisions made during the study design phase which could impact our interpretation.
Connect the design to the claim
Even though randomization can help minimize confounding variables, other aspects of trial conduct also matter. Think about issues such as allocation concealment, adherence to the treatment assignments, amount of missing data for outcomes, quality of outcome measurement and degree of selective reporting. When making comparisons using observational studies we must consider potential sources of bias due to confounders as well as reverse causality.
Repeated measurements from one person are not equivalent to independent measurements from different people. Check the unit of analysis and use methods that account for pairing, clustering or repeated observations where needed.
Extract the estimate, then its limits
Express the main result in ordinary units. For events, report both groups’ absolute risks alongside the relative comparison, keeping denominators and timeframes attached. Read the confidence interval for effect sizes that could change a practical decision.
A statistically non-significant result can remain compatible with meaningful benefit or harm; a significant result may still be modest. Check harms and applicability. A systematic review can put one study in context, but its reliability depends on its methods and underlying evidence. Evaluate the question, estimate, uncertainty and limitations before drawing a conclusion.
Worked example: translating a headline
The fictional headlines read: "Programme cuts events by 50%". This paper reports an incidence of 8 events out of 400 control participants and an incidence of 4 events out of 400 program participants during a six-month period.
Absolute risks = 2% and 1%; absolute reduction = 1 percentage point; relative reduction = 50%.
Next inspect uncertainty, randomisation, missing outcomes and harms. The headline provides only one part of the finding.
A reasonable interpretation of this study will include some combination of the following elements: Research Question/Study Design/Effect Size/Uncertainty/Applicability.