Evidence and decisions

Decision analysis: comparing options under uncertainty

Learn how decision models combine options, probabilities, outcomes and uncertainty without hiding assumptions.

Decision analysis provides a structured way to compare options when their consequences are uncertain. It makes the alternatives, possible events and values assigned to outcomes explicit. Its usefulness comes from showing how the conclusion follows from those assumptions, including where the conclusion could change.

A model is a simplified representation of a decision. It cannot manufacture evidence that does not exist. An impressive diagram or a detailed spreadsheet should therefore prompt questions about its structure and inputs, rather than replace those questions.

Define the decision before choosing the model

State who is making the decision, which population is relevant and what options are available. Include a realistic account of existing practice. Then decide which outcomes matter and over what period they should be considered.

This first description guides the choice of model. A short sequence of events may fit a decision tree. A process involving repeated events and movement between health states over time may need another approach. The simplest adequate structure is often easier to inspect than a complicated model containing unnecessary detail.

Ask whether the model leaves out an important consequence. A model that counts a desirable outcome but omits the burden or cost of obtaining it can produce an incomplete comparison, even when every formula is correct.

Distinguish choices from chance events

A decision tree separates options that can be chosen from events that may follow. At a chance point, the probabilities of the mutually exclusive branches should add to one. Each complete path must have a clearly described outcome.

Avoid confusing a conditional probability with an overall probability. The chance of a later event may depend on the route already taken through the tree. Where events are related, multiplying unrelated headline percentages can give the wrong answer.

Use a consistent outcome scale across alternatives. Costs, life years and QALYs describe different things. They cannot simply be added together without an explicit framework explaining what the resulting measure means.

Work through an expected outcome

Consider an entirely hypothetical one-year model. Under option A, a favourable path has probability 0.70 and produces 1.00 QALY; the other path has probability 0.30 and produces 0.60 QALYs. The expected outcome is (0.70 × 1.00) + (0.30 × 0.60), or 0.88 QALYs.

Under option B, the corresponding probabilities are 0.85 and 0.15, with outcomes of 0.95 and 0.50 QALYs. Its expected outcome is (0.85 × 0.95) + (0.15 × 0.50), or 0.8825 QALYs.

The expected difference is only 0.0025 QALYs in this invented model. The result does not mean that a particular person will experience either expected value. It is an average across the specified possible outcomes. It also does not settle a real decision: costs, omitted consequences and uncertainty in every input still need consideration.

Test which assumptions matter

A deterministic sensitivity analysis changes selected inputs to see how the conclusion responds. For example, varying the probability of the favourable path can reveal whether a small, plausible change reverses the ranking of the options.

A probabilistic sensitivity analysis represents uncertainty in inputs using probability distributions and propagates that uncertainty through the model. Its results depend on the chosen distributions and on how relationships between inputs are represented. Repeating a calculation many times cannot repair inappropriate assumptions.

Structural uncertainty concerns the model itself: which events are represented, which paths are possible and how long effects last. This often needs alternative scenarios or model structures rather than a small adjustment to one number.

Check the model against evidence and common sense

Ask where each input came from and whether it fits the population and decision. Examine calculations, extreme cases and the agreement between model predictions and suitable external observations. Validation is an ongoing set of checks, not a claim justified solely because similar models have been used before.

Keep a record of influential assumptions and explain the consequences of choosing differently. Where uncertainty could change an important decision, the model may help identify what additional information would be most useful.

A strong decision analysis leaves the reader able to trace an answer from the decision question through the evidence and assumptions to the result. It makes judgement easier to examine. It does not remove the need for judgement or turn a hypothetical comparison into advice for an individual.

Methodology references

  • ISPOR–SMDM: Conceptualizing a model, Good Research Practices Report 2
  • ISPOR–SMDM: Modeling good research practices, overview Report 1
  • ISPOR–SMDM: Model parameter estimation and uncertainty analysis, Report 6