Inference

Confidence intervals: putting uncertainty around an estimate

Know how to read an interval correctly and know why sample size, variability and confidence level change width of interval.

Put uncertainty beside the estimate

Sample mean estimates population mean. Random sample usually gives different answer than first one. Confidence intervals use specified method to represent sampling uncertainty.

Average recovery time of 12 days with interval from 11 to 13 has greater precision than same estimate with interval from 6 to 18. Estimate and interval belong together.

Read 95% confidence correctly

A frequentist 95 percent procedure is designed so that about 95 percent of intervals from repeated samples contain the fixed population parameter when its assumptions hold. The interval already calculated either includes that parameter or does not.

Not range containing 95 percent of individual observations nor usual frequentist statement that there is a 95 percent chance of fixed parameter being included. Prediction interval for future observation answers different question.

Use an appropriate method

For an independent sample mean with unknown population SD, a common interval is the mean plus or minus a t critical value times s/√n. It is exact for independent normal observations and approximate in other settings.

Proportions need suitable method (Wilson intervals). Larger samples generally narrower sampling intervals. Greater variability, higher confidence levels widen sampling intervals. None of those choices automatically removes selection bias, measurement problems or confounders.

Worked example: a mean

You have an independent sample of 25 data points. Their average is 18, their Standard Deviation (SD) is 5. The assumption of the t interval applies here.

SE = 5 / √25 = 1. A 95% t critical value with 24 df is about 2.064. Interval = 18 ± 2.064 = [15.936, 20.064].

A confidence interval for the population mean, not a range that covers 95% of individual measurements.

Use this information in conjunction with the "study design" and "assumptions" above to understand how best to interpret the effect size.

Sources for further reading