Linear Regression III
Grayson White
Stat 243
Week 3 | Fall 2026
The Sampling Distribution of \(\hat{\beta}_1\)
Assume the following true model: \[ f(x) = 12 + 0.7 x; \quad \epsilon \sim N(0, 4) \]
set.seed(145)
n <- 60
beta_0 <- 12
beta_1 <- .7
sigma <- 2
x <- rnorm(n, mean = 20, sd = 3)
f_mean <- beta_0 + beta_1*x
y <- f_mean + rnorm(n, mean = 0, sd = sigma)
my_data <- data.frame(x,y)
ggplot(my_data,
aes(x = x, y = y)) +
geom_abline(slope = beta_1,
intercept = 12) +
geom_point(color = "steelblue") +
geom_abline(slope = beta_1,
intercept = 12,
color = "goldenrod",
size = 1) +
theme_bw() +
annotate("text", x = 26, y = 29, label = "f(x)", color = "goldenrod",parse = T) +
labs(title = "Simulated Data from true model")
The Sampling Distribution of \(\hat{\beta}_1\)
Assume the following true model: \[ f(x) = 12 + 0.7 x; \quad \epsilon \sim N(0, 4) \]
ggplot(my_data,
aes(x = x, y = y)) +
geom_abline(slope = beta_1,
intercept = 12,
color = "goldenrod",
size = 1) +
geom_point(color = "steelblue")+
geom_smooth(method = "lm",
se = F,
color = "black") +
theme_bw() +
annotate("text", x = 26, y = 29, label = "f(x)", color = "goldenrod", parse = T) +
annotate("text", x = 26, y = 32.5, label = "hat(f)(x)", color = "black", parse = T) +
labs(title = "Estimate for f based on 1 simulation")
The Sampling Distribution of \(\hat{\beta}_1\)
Assume the following true model: \[ f(x) = 12 + 0.7 x; \quad \epsilon \sim N(0, 4) \]
set.seed(1020)
it <- 1000
slope <- rep(NA, times = it)
intercept <- rep(NA, times = it)
for(i in 1:it) {
x <- rnorm(n, mean = 20, sd = 3)
f_mean <- beta_0 + beta_1 * x
y <- f_mean + rnorm(n, mean = 0, sd = sigma)
slope[i] <- lm(y ~ x)$coef[[2]]
intercept[i] <- lm(y ~ x)$coef[[1]]
}
many_mods <- data.frame(slope, intercept)
ggplot(my_data,
aes(x = x, y = y)) +
geom_point(color = "steelblue") +
geom_abline(data = many_mods,
slope = slope,
intercept = intercept,
alpha = .02) +
geom_smooth(method = "lm",
se = F,
color = "black") +
geom_abline(slope = beta_1,
intercept = 12,
color = "goldenrod",
size = 1) +
theme_bw() +
labs(title = "Estimates for f based on 1000 simulations")
The Sampling Distribution of \(\hat{\beta}_1\)
The Sampling Distribution of \(\hat{\beta}_1\)
Exploratory Data Analysis
ggplot(candy, aes(x = factor(chocolate),
y = winpercent,
fill = factor(chocolate))) +
geom_boxplot() +
stat_summary(fun = mean,
geom = "point",
color = "yellow",
size = 4) +
guides(fill = "none") +
scale_fill_manual(values =
c("0" = "deeppink",
"1" = "chocolate4")) +
scale_x_discrete(labels = c("No", "Yes"),
name =
"Does the candy contain chocolate?")
New example: Palmer Penguins
Exploratory data analysis
How do we handle more than 2 groups???
Before:
We predicted a penguin’s bill length based on their species.
- Pink dots represent the mean value in each group.
- For the single categorical variable model, those pink dots are the predicted values for each group.
Now:
We’ll incorporate bill depth for prediction!
- A moderate negative relationship between bill length and bill depth!
- Does this make sense?
What if we include both explanatory variables?
ggplot(penguins, aes(x = bill_depth_mm, y = bill_length_mm, color = species)) +
geom_point(size = 2) +
geom_smooth(inherit.aes = FALSE,
mapping = aes(x = bill_depth_mm,
y = bill_length_mm),
method = "lm",
se = FALSE,
color = "red") +
geom_parallel_slopes(se = FALSE) +
scale_color_manual(values = c("steelblue",
"goldenrod",
"plum3")) +
theme_bw() +
theme(legend.position = "bottom")
- Negative relationships between bill depth and bill length overall.
- Positive relationships between bill depth and bill length when accounting for species!
- What is going on here??
- This is a case of Simpson’s Paradox.
Returning to our MLR model
Are equal slopes a reasonable assumption here?
Different Slopes Model
Equal slopes models force the relationship between quantitative predictors and the response variable to be the same for each group in the model.
In contrast, different slopes models allow for different relationships between quantitative predictors and the response variable for each group in the model.
How can we allow our model to have different slopes?