Dr.Tremendous
Multicollinearity
ARE 455 · When OLS can't tell your variables apart
Multicollinearity means the model can't tell which variable deserves credit. What's going on? ›
The estimates bounce around even when overall model fit still looks strong. OLS asks what each X contributes holding the others fixed. There are two bins because there are two variables, but the signal separating them is weak. When X2 and X3 move together, you almost never observe X2 changing while X3 stays fixed. The model is trying to answer a question the data give very little information about. Standard errors explode and t-stats collapse, but overall model fit often stays strong.
Try: drag ① r toward 1 → watch the machine ② struggle and SE grow in the table ③ → draw samples ④ and watch β̂ swing while the intervals ⑤ jump
1Correlation between X2 and X3
r (X2, X3) r = 0.00
No correlationNear-perfect
demonstrates overlapping information
r = 0.00 — no correlation. Each variable's effect can be estimated with little interference from the others. Coefficients are precise with respect to multicollinearity.
3Regression output
Stylized: β̂ are held fixed to isolate the SE and t-stat effect. Use the resample panel to see the coefficients themselves move.
Variable
β̂
SE
t-stat
p (approx.)
X1
1.50
0.18
8.33
<0.01
X2
0.70
0.18
3.89
<0.01
X3
0.70
0.18
3.89
<0.01
4Draw a new sample
0 draws
Same true model, fresh noise each time. How far the dots spread is what a standard error describes — wide spread means a large SE.
β̂₂ = —
β̂₃ = —
R² = —
Each dot is one sample's (β̂₂, β̂₃). The true value is the cross at (0.70, 0.70).
—
Model fit often stays strong. Multicollinearity usually doesn't hurt in-sample fit, and the model can still predict well for similar data. What it destroys is attribution: the model knows the combined effect but can't separate it. (Prediction can still suffer for unusual X2–X3 combinations not seen in the data.)
VIF 1.00
SE inflation ×1.00
SE inflates by √VIF. Common rule of thumb: VIF > 10 = serious problem.
2OLS sorting machine
The bins are competing to explain the same signal. When neither can win cleanly, both estimates suffer.
Fine print (for the rigorous) ›
* X2 and X3 have the same expected SE here because the setup is symmetric: both have true coefficient 0.70 and only their pairwise correlation drives the inflation.
* The stat table uses an illustrative baseline SE of 0.18. The resample panel runs a real regression (n = 60, noise SD = 1.6), so its scatter spread at r = 0 is about 0.21. The two layers tell the same story; their baseline numbers differ slightly by design.
* The ball-sorting is a metaphor. OLS doesn't assign variance piece-by-piece. Correlated regressors make both estimates simultaneously sensitive to noise. The "Draw a new sample" panel shows the true mechanism.
* In multiple regression, multicollinearity depends on how each variable relates to all others, not just one pair. This widget shows pairwise correlation for illustration.
595% confidence intervals. The bold bar is the expected 95% range for estimates, centered on the true β. The thin grey bar above it is your latest sample's interval, centered on its β̂. Draw repeatedly and watch the grey bar jump while the bold one only widens.
Why are there two bins if there's only one signal? ›
Because the theory says there are two different effects. OLS builds a bin for each. But when X2 and X3 almost always move together, the data give very little guidance on how much belongs in each bin, so both estimates become unreliable.

Example: X2 = study hours, X3 = attendance. If students who study more almost always attend more, did grades improve because of studying, or because of attending? You need two bins conceptually. But the data give you mostly one combined signal.

When r is high: many different splits between the two bins explain the data almost equally well. That flat ridge of near-ties is the consequence of multicollinearity, and it's why the estimates swing so much from sample to sample.

"The ball didn't go to the wrong bin. The data just can't pin down which bin it belongs in."
What causes multicollinearity?
↑ Variables that are strongly correlated in the real world
Income and wealth. Years of experience and age. Square footage and number of rooms. They aren't the same thing, but they move together in most data, so OLS has very little independent variation to work with when separating their effects.
↑ Including a variable and its transformation
X and X², or X and log(X). When X is all positive and its range is narrow, these are often highly correlated in your sample, so their individual coefficients get noisy. Two standard fixes: center X before squaring, which usually cuts the correlation sharply, and test the variables jointly (F-test) rather than one t-stat at a time.
↑ Small sample with many predictors
Two problems compound here. Sample correlations between even unrelated variables bounce around more when n is small, so chance collinearity is more likely. And as the number of predictors approaches the number of observations, the data spreads thinner over more dimensions and every estimate gets noisier, collinear or not. More data helps both problems.
Guided questions 6 ›
1
At r = 0, all three variables are significant. Drag to r = 0.90. What happens to X2 and X3? What happens to X1?
Watch the SE and t-stat columns separately for each variable.
2
In the stylized table, the coefficients on X2 and X3 stay at 0.70 while their SEs grow. Why does that make them insignificant? (Then check the resample panel: across real samples the estimates also move.)
Think about what SE does to the t-statistic. In the table the estimate is held still to isolate that effect; in real data both the estimate and its SE move.
3
Use the resample panel: draw samples at r = 0, then at r = 0.95. The SEs explode, but what happens to R²? What does this tell you about multicollinearity and model fit?
A model can fit the data well and still have uninterpretable coefficients.
4
Look at the confidence intervals for X2 and X3 as r increases. What happens to them, and what does that mean for your ability to make decisions?
A wide CI that crosses zero means you can't distinguish the effect from zero.
5
You have two highly correlated variables: household income and household wealth. Both are insignificant (p > 0.05). Should you conclude neither affects the outcome?
Insignificant may mean indistinguishable, not unimportant. Consider: drop one, combine them, collect more data, or keep both if your goal is prediction rather than interpretation. Multicollinearity usually does not hurt in-sample fit much, but it mainly damages individual coefficient attribution.
6
Set r = 0.90. Note the SE and t-stat for X2. Now click "Drop X3" below the slider. What happens to X2's SE and t-stat? What does this tell you about where the problem came from?
The problem wasn't X2. It was the competition with X3.