Inference

One-sample, paired and Welch t-tests

Match the t-test to your comparison. Then see how the estimated mean or difference is divided by its standard error to produce the test statistic.

A difference relative to uncertainty

A t statistic expresses an estimated difference in units of estimated standard error. Its signed value, the relevant t distribution and the specified alternative determine the p-value.

This addresses a comparison of population means, not every feature of two distributions. Equal means can occur alongside different variances, shapes or frequencies of extreme observations.

One sample or paired observations

A one-sample t-test uses (sample mean − reference mean)/(s/√n), with n minus 1 degrees of freedom. It compares a population mean with a specified value.

A paired test first calculates matched differences and applies the one-sample procedure to them, usually against zero. For the classical small-sample normality assumption, the differences need an appropriate normal model; neither original column must separately be normal.

Two independent groups

Welch’s t-test does not assume equal population variances. Its standard error is √(s₁²/n₁ + s₂²/n₂), with Welch–Satterthwaite degrees of freedom. A pooled t-test is a different procedure requiring an appropriate equal-variance assumption.

Check independence of observations, extreme values and distributional assumptions. Thirty observations are not a universal guarantee. Choose one-sided or two-sided hypotheses before testing, then report the difference, confidence interval, t statistic, degrees of freedom and p-value. Statistical significance alone does not establish practical importance.

Worked example: a one-sample test

There are twenty five independent observations (of which) all of the same thing, whose average was 52 and whose sample standard deviation is 5. The question here is to compare that population mean to a reference value of fifty using the t-model assumption.

SE = 5/√25 = 1. t = (52 − 50)/1 = 2, with 24 df. The two-sided p-value is approximately 0.057.

This result at a pre-specified level of .05 does not allow us to reject the null hypothesis but neither does it give us enough information to say they are equal.

Select t-test from the Design. Then look up and explain the estimated difference in terms of how much and in terms of its confidence interval or uncertainty.

Sources for further reading