“It’s easy to lie with statistics. It’s hard to tell the truth without statistics.” – Andrejs Dunkels
In the previous chapter we focused on categorical outcomes. Here our response variable is quantitative, meaning it is measured on a numeric scale such as weight, blood pressure, cholesterol level, enzyme activity, revenue or time. To decide on an appropriate analysis we must consider how many explanatory variables we have and whether the data are paired or independent. The table below summarizes the situations covered in this chapter:
| Scenario | Response variable | Explanatory variable | Appropriate method |
|---|---|---|---|
| No explanatory variable | Quantitative | None (one sample) | One‑sample t‑test for the mean and chi‑square test for the variance |
| Two paired measurements | Quantitative | One categorical variable with two matched levels (e.g., before/after on the same subject) | Paired t‑test |
| Two independent groups | Quantitative | One categorical variable with two independent levels (e.g., treatment vs. control) | Two‑sample t‑test for means (pooled or Welch) and F‑test for equality of variances |
When there is no explanatory variable, we have a single quantitative sample. Two natural questions arise:
We address these with the one‑sample t-test and the chi‑square test for the variance.
Suppose we have a random sample \(x_1, x_2,\dots,x_n\) from a population with unknown mean \(\mu\) and standard deviation \(\sigma\). We want to test
\[ \begin{align*} &H_0: \mu = \mu_0 \\ &H_a: \mu \neq \mu_0\quad (\text{or } \mu > \mu_0,\ \mu < \mu_0) \end{align*} \]
Like all inference methods, the t‑test rests on assumptions. In practice we require:
Under \(H_0\), the test statistic
\[ t = \frac{\bar{x} - \mu_0}{s/\sqrt{n}} \]
follows a Student’s \(t\) distribution with \(n-1\) degrees of freedom. We obtain the p‑value by comparing the observed \(|t|\) to the \(t\) distribution. For a two‑sided alternative the p‑value is \(2P(T_{n-1} > |t|)\); for a one‑sided alternative we compute \(P(T_{n-1} > t)\) or \(P(T_{n-1} < t)\) depending on the direction.
When the test is significant (or even when it is not), a \((1-\alpha)\times100\%\) confidence interval for \(\mu\) gives a range of plausible values. The interval is
\[ \bar{x} \pm t_{\alpha/2}\,\frac{s}{\sqrt{n}}, \]
where \(t_{\alpha/2}\) is the critical value from the \(t\) distribution.
A financial analyst reviews 15 months of returns for an actively managed mutual fund. The sample mean return is 3.4% with a standard deviation of 0.6%. The benchmark index averages 3.0% per month. Test whether the fund’s mean monthly return differs from the benchmark at the 5% significance level.
Solution. The hypotheses are \(H_0: \mu = 3.0\) vs. \(H_a: \mu \neq 3.0\). The test statistic is
\[ t = \frac{3.4 - 3.0}{0.6/\sqrt{15}} \approx 2.58. \]
With \(n-1 = 14\) degrees of freedom the two‑sided p‑value is \(2P(T_{14} > 2.58) \approx 0.021\). Since 0.021 < 0.05, we reject \(H_0\) and conclude that the fund’s mean monthly return differs from the 3.0% benchmark. A 95% confidence interval is \(3.4 \pm t_{0.025,14}\times0.6/\sqrt{15} = (3.05, 3.75)\).
We can reproduce this in R:
set.seed(10040000)
#simulate the data since all we have are summary stats
returns <- rnorm(15, mean = 3.4, sd = 0.6)
t.test(returns, mu = 3.0, alternative = "two.sided")
One Sample t-test
data: returns
t = 2.5764, df = 14, p-value = 0.02196
alternative hypothesis: true mean is not equal to 3
95 percent confidence interval:
3.071913 3.786693
sample estimates:
mean of x
3.429303 Use t.test() with a numeric vector and the hypothesized mean:
# Example: test whether mean monthly return differs from 3.0%
# x = your data vector
# mu = hypothesized population mean
# alternative = "two.sided", "greater", or "less"
t.test(returns, mu = 3.0, alternative = "two.sided")
One Sample t-test
data: returns
t = 2.5764, df = 14, p-value = 0.02196
alternative hypothesis: true mean is not equal to 3
95 percent confidence interval:
3.071913 3.786693
sample estimates:
mean of x
3.429303 The output reports the t statistic, degrees of freedom, p-value, sample mean, and a confidence interval for \(\mu\). Change alternative to "greater" or "less" for a one-sided test.
Sometimes we are interested in assessing whether the population variance equals a specified value \(\sigma_0^2\). For example, an engineer may want to know whether the variability of machine parts exceeds a tolerance level. Let \(s^2\) be the sample variance computed from a random sample of size \(n\). To test
\[ \begin{align*} &H_0: \sigma^2 = \sigma_0^2\\ &H_a: \sigma^2 \neq \sigma_0^2 \quad (\text{or } >, <) \end{align*} \]
we use the test statistic
\[ \chi^2 = \frac{(n-1)s^2}{\sigma_0^2}, \]
which has a chi‑square distribution with \(n-1\) degrees of freedom under \(H_0\). Large values of \(\chi^2\) indicate that \(s^2\) is bigger than expected under \(H_0\); small values indicate it is too small. For a two‑sided alternative the p‑value is \(P(\chi^2_{n-1} > \chi^2_\text{obs})\) for one tail plus \(P(\chi^2_{n-1} < \chi^2_\text{obs})\) for the other.
A \((1-\alpha)\times100\%\) confidence interval for the population variance is
\[ \left(\frac{(n-1)s^2}{\chi^2_{1-\alpha/2}},\ \frac{(n-1)s^2}{\chi^2_{\alpha/2}}\right), \]
where \(\chi^2_{\alpha/2, n-1}\) and \(\chi^2_{1-\alpha/2, n-1}\) are critical values from the chi‑square distribution. Taking square roots yields a confidence interval for the standard deviation.
A company manufactures springs with a target variance in spring length of \(\sigma_0^2 = 0.04\ \mathrm{cm}^2\). A quality‑control engineer measures 25 springs and computes a sample variance of \(s^2 = 0.065\ \mathrm{cm}^2\). Test whether the variance differs from the target at the 5% level.
Solution. With \(n-1 = 24\) degrees of freedom, the test statistic is \[
\begin{align*}
\chi^2 &= \frac{(24)(0.065)}{0.04}\\
&= 39.0
\end{align*}
\]
For a two‑sided test, the p‑value is about \(2P(\chi^2>39.0)=0.055\). We fail to reject \(H_0\). A 95% confidence interval for \(\sigma^2\) is
\[ \left(\frac{24\times0.065}{39.4},\ \frac{24\times0.065}{12.4}\right) = (0.0396, 0.1256), \]
so the variance may range from 0.04 to 0.13 cm².
Base R has no single function for this test, but the calculation is straightforward:
# Chi-square test for variance
# Example: 25 springs, s^2 = 0.065, H0: sigma^2 = 0.04
n <- 25
s2 <- 0.065
sigma0 <- 0.04
chi2_stat <- (n - 1) * s2 / sigma0
df <- n - 1
# Two-sided p-value
p_value <- 2 * min(pchisq(chi2_stat, df),
pchisq(chi2_stat, df, lower.tail = FALSE))
# 95% confidence interval for sigma^2
ci_lower <- (df * s2) / qchisq(0.975, df)
ci_upper <- (df * s2) / qchisq(0.025, df)
cat("chi-sq =", round(chi2_stat, 3),
" p-value =", round(p_value, 4), "\n")chi-sq = 39 p-value = 0.0546 cat("95% CI for sigma^2: (", round(ci_lower, 4),
",", round(ci_upper, 4), ")\n")95% CI for sigma^2: ( 0.0396 , 0.1258 )| Keyword/Concept | Definition/Formula |
|---|---|
| One‑sample t‑test | Tests \(H_0: \mu=\mu_0\) using \(t = (\bar{x}-\mu_0)/(s/\sqrt{n})\) with \(n-1\) df. |
| Confidence interval for mean | \(\bar{x} \pm t_{\alpha/2,n-1} \times s/\sqrt{n}\). |
| Chi‑square test for variance | Tests \(H_0: \sigma^2 = \sigma_0^2\) using \(\chi^2 = (n-1)s^2/\sigma_0^2\) with \(n-1\) df. |
| Confidence interval for variance | \(\left(\frac{(n-1)s^2}{\chi^2_{1-\alpha/2,n-1}},\ \frac{(n-1)s^2}{\chi^2_{\alpha/2,n-1}}\right)\). |
“The twin assumptions of normality of distribution and homogeneity of variance are not ever exactly fulfilled in practice, and often they do not even hold to a good approximation.” -John Tukey
When comparing means between two independent groups, the appropriate procedure depends on whether the population variances are equal. We consider two methods: the pooled (Student’s) t‑test and Welch’s t‑test.
Let \(x_{1i}\) and \(x_{2j}\) be independent random samples from populations with means \(\mu_1\) and \(\mu_2\) and common variance \(\sigma^2\). To test \[ \begin{align*} &H_0: \mu_1 = \mu_2\\ &H_a: \mu_1 \neq \mu_2 \qquad\text{(or one‑sided)} \end{align*} \]
we compute
\[ s_p^2 = \frac{(n_1-1)s_1^2 + (n_2-1)s_2^2}{n_1 + n_2 - 2} \]
\[ t = \frac{\bar{x}_1 - \bar{x}_2}{s_p\sqrt{1/n_1 + 1/n_2}} \]
The statistic follows a \(t\) distribution with \(n_1 + n_2 - 2\) degrees of freedom. Assumptions include:
A \((1-\alpha)\times100\%\) confidence interval for \(\mu_1 - \mu_2\) is
\[ (\bar{x}_1 - \bar{x}_2) \pm t_{\alpha/2,}\,s_p \sqrt{1/n_1 + 1/n_2}. \]
When the assumption of equal variances is violated, we use Welch’s t‑test (also called the nonpooled t-test). This test modifies the standard error and the degrees of freedom. The test statistic is
\[ t = \frac{\bar{x}_1 - \bar{x}_2}{\sqrt{s_1^2/n_1 + s_2^2/n_2}}, \]
and the degrees of freedom are approximated by
\[ \nu = \frac{\left(\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2}\right)^2}{\frac{(s_1^2/n_1)^2}{n_1 - 1} + \frac{(s_2^2/n_2)^2}{n_2 - 1}}. \]
This formula yields a non‑integer value; software uses it to compute the p‑value. The confidence interval for \(\mu_1 - \mu_2\) uses the same standard error and the \(t_{\nu}\) distribution.
The assumptions for two‑sample t‑tests: the groups must be independent, each sample should come from a normal distribution, and the variances should be equal for the pooled test. If variances are not equal, the Welch test is preferred. In practice, Welch’s t-test should be used by default unless it is known the variances are equal.
A marketing analyst compares sales increases after two different advertising campaigns. Sales increases for 20 regions under campaign A have mean 1.8% and SD = 1.0%; for 18 regions under campaign B the mean is 1.2% with SD = 0.6%. We do not know if the population variances are equal so we use Welch’s t-test:
\[ t = \frac{1.8 - 1.2}{\sqrt{1.0^2/20 + 0.6^2/18}} = 2.17. \]
The Welch degrees of freedom are
\[ \nu = \frac{\left(\frac{1.0^2}{20} + \frac{0.6^2}{18}\right)^2}{\frac{(1.0^2/20)^2}{19} + \frac{(0.6^2/18)^2}{17}} \approx 31. \]
With \(\nu\approx31\) df, the two‑sided p‑value is 0.036. We reject \(H_0\); campaign A yields a greater increase in sales. A 95% CI for the difference is \((0.037, 1.163)\) percentage points.
We can replicate this in R:
set.seed(1004)
campaign_A <- rnorm(20, mean = 1.8, sd = 1.0)
campaign_B <- rnorm(18, mean = 1.2, sd = 0.6)
# Welch's t-test (default; does not assume equal variances)
t.test(campaign_A, campaign_B,
alternative = "two.sided", var.equal = FALSE)
Welch Two Sample t-test
data: campaign_A and campaign_B
t = 2.6333, df = 30.429, p-value = 0.01317
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
0.1688933 1.3331008
sample estimates:
mean of x mean of y
1.927982 1.176985 # Pooled t-test (use only if equal variances can be assumed)
t.test(campaign_A, campaign_B,
alternative = "two.sided", var.equal = TRUE)
Two Sample t-test
data: campaign_A and campaign_B
t = 2.56, df = 36, p-value = 0.01481
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
0.1560317 1.3459624
sample estimates:
mean of x mean of y
1.927982 1.176985 Pass two numeric vectors to t.test(). The key argument is var.equal:
# var.equal = FALSE → Welch's t-test (recommended default)
# var.equal = TRUE → Pooled (Student's) t-test
t.test(campaign_A, campaign_B,
alternative = "two.sided",
var.equal = FALSE) # Welch — safe default
Welch Two Sample t-test
data: campaign_A and campaign_B
t = 2.6333, df = 30.429, p-value = 0.01317
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
0.1688933 1.3331008
sample estimates:
mean of x mean of y
1.927982 1.176985 var.equal = FALSE) is the default and is recommended unless you have strong evidence that the population variances are equal.var.equal = TRUE) assumes \(\sigma_1^2 = \sigma_2^2\); use only after confirming this with the F-test in the next section.| Keyword/Concept | Definition/Formula |
|---|---|
| Pooled t‑test | Tests \(H_0: \mu_1=\mu_2\) assuming equal variances; statistic \(t=(\bar{x}_1-\bar{x}_2)/(s_p\sqrt{1/n_1+1/n_2})\) with df \(n_1+n_2-2\). |
| Welch’s (nonpooled) t‑test | Tests \(H_0: \mu_1=\mu_2\) without assuming equal variances; statistic \((\bar{x}_1-\bar{x}_2)/\sqrt{s_1^2/n_1+s_2^2/n_2}\) with approximate df given by Welch’s formula. |
“To make the preliminary test on variances is rather like putting to sea in a rowing boat to find out whether conditions are sufficiently calm for an ocean liner to leave port.” -George Box
The F‑test compares two sample variances to evaluate
\[ H_0: \sigma_1^2 = \sigma_2^2 \quad\text{vs.}\quad H_a: \sigma_1^2 \neq \sigma_2^2. \]
Let \(s_1^2\) and \(s_2^2\) be the sample variances from independent random samples of sizes \(n_1\) and \(n_2\) drawn from normally distributed populations. Without loss of generality assume \(s_1^2 \ge s_2^2\). The test statistic
\[ F = \frac{s_1^2}{s_2^2} \]
has an F distribution with \((n_1-1, n_2-1)\) degrees of freedom under \(H_0\). Large values of \(F\) indicate that \(s_1^2\) is much larger than \(s_2^2\); very small values (which correspond to \(s_2^2\) being larger) are captured by the symmetry of the F distribution.
Assumptions include:
We can estimate the ratio of population variances \(\sigma_1^2/\sigma_2^2\) using the sample ratio \(F\) and obtain a \((1-\alpha)\times100\%\) confidence interval:
\[ \left(\frac{s_1^2}{s_2^2}\times\frac{1}{F_{\alpha/2,\,n_1-1,\,n_2-1}},\ \frac{s_1^2}{s_2^2}\times\frac{1}{F_{1-\alpha/2,\,n_1-1,\,n_2-1}}\right). \]
A sports scientist compares the consistency of 40-yard dash times produced by two training programs. Program A (\(n_1=10\) athletes) yields a sample variance of \(s_1^2=0.08\) seconds²; Program B (\(n_2=12\) athletes) yields \(s_2^2=0.03\) seconds². A more consistent program produces less variability. With \(s_1^2 > s_2^2\), the test statistic is \(F=0.08/0.03=2.67\) with degrees of freedom \((9,11)\). The two‑sided p‑value is \(2P(F_{9,11}>2.67)\approx0.16\). We fail to reject \(H_0\); the sprint-time variances of the two programs are not significantly different.
In R we use var.test():
set.seed(21)
program_A <- rnorm(10, mean = 4.5, sd = sqrt(0.08))
program_B <- rnorm(12, mean = 4.5, sd = sqrt(0.03))
var.test(program_A, program_B, alternative = "two.sided")
F test to compare two variances
data: program_A and program_B
F = 3.932, num df = 9, denom df = 11, p-value = 0.03638
alternative hypothesis: true ratio of variances is not equal to 1
95 percent confidence interval:
1.095907 15.382295
sample estimates:
ratio of variances
3.932005 Use var.test() with two numeric vectors:
# var.test(x, y) tests H0: sigma_x^2 / sigma_y^2 = 1
var.test(program_A, program_B, alternative = "two.sided")
F test to compare two variances
data: program_A and program_B
F = 3.932, num df = 9, denom df = 11, p-value = 0.03638
alternative hypothesis: true ratio of variances is not equal to 1
95 percent confidence interval:
1.095907 15.382295
sample estimates:
ratio of variances
3.932005 The output reports the F statistic, degrees of freedom, p-value, and a confidence interval for the ratio \(\sigma_1^2/\sigma_2^2\). If the normality assumption is doubtful, use leveneTest() from the car package as a robust alternative:
# Levene’s test (robust to non-normality)
# car::leveneTest(response ~ group, data = your_data)| Keyword/Concept | Definition/Formula |
|---|---|
| F‑test | Tests \(H_0: \sigma_1^2=\sigma_2^2\) with statistic \(F=s_1^2/s_2^2\). |
| Confidence interval for variance ratio | \(\big(\frac{s_1^2}{s_2^2}/F_{\alpha/2},\ \frac{s_1^2}{s_2^2}/F_{1-\alpha/2}\big)\) for df \((n_1-1,n_2-1)\). |
“Statistics is the art of stating in precise terms that which one does not know.” - William Kruskal
Suppose we want to compare two methods of teaching reading to students with below-average reading scores. Eight pairs of students with similar baseline reading ability are identified, and one member of each pair is randomly assigned to the standard teaching method while the other is assigned to the new method.
The reading test scores are recorded below.
| Pair | New Method | Standard Method |
|---|---|---|
| 1 | 77 | 72 |
| 2 | 74 | 68 |
| 3 | 82 | 76 |
| 4 | 73 | 68 |
| 5 | 87 | 84 |
| 6 | 69 | 68 |
| 7 | 66 | 61 |
| 8 | 80 | 76 |
Do the data support the hypothesis that the new method produces higher mean scores?
Using what we know so far, the hypotheses are \[ \begin{align*} H_0&:\mu_\text{new} = \mu_\text{std}\\ H_a&:\mu_\text{new} > \mu_\text{std} \end{align*} \] Suppose we mistakenly apply the two-sample Welch t-test, treating the samples as independent:
new_method <- c(77, 74, 82, 73, 87, 69, 66, 80)
std_method <- c(72, 68, 76, 68, 84, 68, 61, 76)
# Incorrect: treats matched pairs as independent samples
t.test(new_method, std_method,
alternative = "greater", var.equal = FALSE)
Welch Two Sample t-test
data: new_method and std_method
t = 1.2556, df = 13.998, p-value = 0.1149
alternative hypothesis: true difference in means is greater than 0
95 percent confidence interval:
-1.762061 Inf
sample estimates:
mean of x mean of y
76.000 71.625 The p-value is approximately 0.11. At \(\alpha=0.10\), we fail to reject \(H_0\) and conclude there is insufficient evidence that the new method improves scores.
Look at the data again. The new method score exceeds the standard method score for every one of the eight pairs. This strongly suggests the new method is better — so why did the test fail?
The independent samples t-test is not valid here. We matched students in pairs based on reading ability, so the two samples are not independent. The between-pair variability (some students are simply stronger readers than others) inflates the error term and masks the within-pair difference we care about.
We now consider the correct approach. We compute the difference within each pair:
| Pair | New Method | Standard Method | Difference (new \(-\) std) |
|---|---|---|---|
| 1 | 77 | 72 | 5 |
| 2 | 74 | 68 | 6 |
| 3 | 82 | 76 | 6 |
| 4 | 73 | 68 | 5 |
| 5 | 87 | 84 | 3 |
| 6 | 69 | 68 | 1 |
| 7 | 66 | 61 | 5 |
| 8 | 80 | 76 | 4 |
When each subject or experimental unit provides two related measurements, such as before and after treatment or left‑ and right‑handed performance, we cannot treat the two samples as independent. Instead, we analyze the differences within pairs. The matched‑pairs t‑test focuses on the mean difference.
Suppose we observe pairs \((x_{1i}, x_{2i})\) for \(i=1,\dots,n\). Let \(d_i = x_{1i} - x_{2i}\) denote the difference for each pair and let \(\bar{x}_x\) and \(s_d\) be the mean and standard deviation of the differences. To test whether the average difference \(\mu_d\) equals zero, we formulate
\[ \begin{align*} &H_0: \mu_d = 0\\ &H_a: \mu_d \neq 0\quad (\text{or } > 0,<0) \end{align*} \]
The test statistic is
\[ t = \frac{\bar{x}_d}{s_d/\sqrt{n}} \]
which follows a \(t\) distribution with \(n-1\) degrees of freedom under \(H_0\). We compute the p‑value as before. A \((1-\alpha)\times100\%\) confidence interval for \(\mu_d\) is \(\bar{d} \pm t_{\alpha/2,n-1}\, s_d/\sqrt{n}\).
For our example, we run the paired t-test in R using paired = TRUE:
new_method <- c(77, 74, 82, 73, 87, 69, 66, 80)
std_method <- c(72, 68, 76, 68, 84, 68, 61, 76)
t.test(new_method, std_method,
paired = TRUE, alternative = "greater")
Paired t-test
data: new_method and std_method
t = 7.3438, df = 7, p-value = 7.838e-05
alternative hypothesis: true mean difference is greater than 0
95 percent confidence interval:
3.246316 Inf
sample estimates:
mean difference
4.375 The p-value is well below 0.0001. Contrast this with the independent-samples test earlier (p ≈ 0.11): pairing eliminates between-student variability and reveals the consistent improvement that the unpaired test missed.
Set paired = TRUE in t.test(). The two vectors must be the same length and in matching order (row \(i\) of the first vector is paired with row \(i\) of the second):
# paired = TRUE performs the matched-pairs t-test
# alternative = "greater": H_a: mu_new > mu_std
t.test(new_method, std_method,
paired = TRUE, alternative = "greater")
Paired t-test
data: new_method and std_method
t = 7.3438, df = 7, p-value = 7.838e-05
alternative hypothesis: true mean difference is greater than 0
95 percent confidence interval:
3.246316 Inf
sample estimates:
mean difference
4.375 Equivalently, you can compute the differences yourself and run a one-sample t-test:
diff_scores <- new_method - std_method
t.test(diff_scores, mu = 0, alternative = "greater")
One Sample t-test
data: diff_scores
t = 7.3438, df = 7, p-value = 7.838e-05
alternative hypothesis: true mean is greater than 0
95 percent confidence interval:
3.246316 Inf
sample estimates:
mean of x
4.375 Both approaches give identical results. The output reports the mean difference, the t statistic, degrees of freedom (\(n-1\)), p-value, and a confidence interval for \(\mu_d\).
| Keyword/Concept | Definition/Formula |
|---|---|
| Paired t‑test | Tests \(H_0: \mu_d=0\) using \(t=\bar{d}/(s_d/\sqrt{n})\) with \(n-1\) df. |