How to appraise a treatment trial
Read a treatment trial by checking its comparison, allocation, follow-up, effect size and relevance.
A treatment trial can report a small p-value and still leave the main question unanswered. Perhaps the comparison was unrealistic, too many participants disappeared from follow-up, or the measured improvement mattered little to the people taking part. Appraisal means working out which conclusions the trial can reasonably support.
Start by writing the question in one sentence. Identify the participants, the intervention, the comparison, the outcome and the follow-up period. Keep that sentence beside the results. It helps you notice when a confident conclusion reaches beyond the population or outcome actually studied.
Check what was compared
Read the methods before concentrating on the headline result. What did each group receive? A new service compared with no service answers a different question from the same service compared with good existing care. Other treatment, staff contact and follow-up arrangements can also differ between groups.
Check whether the main outcome was defined before the results were known. A laboratory measurement may be useful, but it does not automatically establish an improvement in symptoms, function or survival. Where several outcomes are reported, distinguish the planned primary outcome from additional analyses.
Look at allocation and measurement
Random allocation aims to create comparable groups without allowing treatment choice to depend on prognosis. The random sequence and the concealment of the next assignment are separate issues. A random list offers little protection if the person enrolling participants can anticipate the next group.
Blinding addresses another problem. Knowing the assigned intervention may affect behaviour, the care offered or the assessment of an outcome. Its importance depends partly on what is measured. A self-reported pain score presents different opportunities for influence from a clearly defined laboratory measurement assessed without knowledge of allocation.
A table of baseline characteristics is useful context. Small differences do not prove that randomisation failed, and apparently balanced groups do not prove that it worked properly. The description of the allocation process matters.
Follow participants through the study
Look for the numbers assigned, treated, followed up and analysed. Ask why those numbers differ. Missing outcomes are particularly concerning when the reasons may depend on how well participants were doing.
An intention-to-treat analysis estimates the effect of assignment by retaining participants in their allocated groups. Check what the authors actually did rather than relying on that label. Excluding people with missing results can still introduce bias. An analysis restricted to people who completed treatment also needs careful interpretation because completion may relate to prognosis.
Work through an absolute result
Consider a hypothetical trial with 1,000 participants in each group. During one year, the outcome occurs in 120 people receiving usual care and 80 receiving the new intervention.
The observed risks are 12% and 8%. The risk ratio is 8 divided by 12, approximately 0.67. The absolute risk reduction is 4 percentage points. Its inverse is 25, giving an estimated number needed to treat of 25 over one year to prevent one additional event under these trial conditions.
Those figures describe the same comparison from different angles. They do not show how precise the estimate is. Read the confidence interval, check for important harms and retain the follow-up period whenever you repeat the result. The example is arithmetic practice, not evidence about a real treatment.
Separate clear reporting from trustworthy methods
CONSORT is a reporting guideline. It helps authors explain what happened, including allocation, participant flow and outcomes. A complete report makes appraisal possible, but completing a checklist does not make the trial unbiased.
Risk-of-bias assessment instead considers how design and conduct could distort a particular result. A trial can be more convincing for one outcome than another. Avoid reducing every concern to a single reassuring total score.
Finally, compare the trial setting with the setting you care about. Consider eligibility restrictions, baseline risk, available support and the practical burden of the intervention. A useful appraisal ends with a qualified statement about the size, uncertainty and relevance of the effect, rather than a verdict based only on statistical significance.
Methodology references
- Cochrane Handbook, Chapter 8: Assessing risk of bias in a randomized trial
- CONSORT 2025 statement: updated guideline for reporting randomised trials
- Oxford CEBM: Critical appraisal tools