Heteroskedasticity-consistent standard errors

Heteroskedasticity-consistent standard errors (HC standard errors) are standard-error estimators used in regression analysis to provide inference that is robust to heteroskedasticity of the regression errors.

In an ordinary least squares (OLS) regression:

$𝐲=𝐗𝜷+𝝐$

the conventional OLS covariance matrix assumes that the conditional variance of the error term is constant:

$\mathrm{Var}(𝝐\mid 𝐗)={\sigma }^2𝐈$

Under heteroskedasticity, this assumption may not hold:

$\mathrm{Var}(𝝐\mid 𝐗)=𝜴\ne {\sigma }^2𝐈$

HC estimators replace the conventional homoskedastic covariance estimate with an estimator that allows the error variance to differ across observations.

Basic formulation

The general heteroskedasticity-robust covariance estimator has the sandwich form:

$\widehat{\mathrm{Var}}(\widehat{𝜷})=({𝐗}^{𝖳}𝐗{)}^{-1}{𝐗}^{𝖳}\widehat{𝜴}𝐗({𝐗}^{𝖳}𝐗{)}^{-1}$

where:

• $\widehat{𝜷}$ = OLS coefficient estimates;
• $𝐗$ = design matrix;
• $\widehat{𝜴}$ = estimated covariance matrix of the regression errors;
• $\widehat{\mathrm{Var}}(\widehat{𝜷})$ = estimated covariance matrix of the OLS coefficients.

The resulting standard error for coefficient $j$ is:

${\mathrm{SE}}_{HC}({\hat{\beta }}_j)=\sqrt{{[\widehat{\mathrm{Var}}(\widehat{𝜷})]}_{jj}}$

HC0 estimator

The original heteroskedasticity-consistent estimator, commonly called HC0, uses the squared OLS residuals to estimate observation-specific error variances.

The covariance estimator is:

${\widehat{\mathrm{Var}}}_{HC0}(\widehat{𝜷})=({𝐗}^{𝖳}𝐗{)}^{-1}\left({𝐗}^{𝖳}\widehat{𝐃}𝐗\right)({𝐗}^{𝖳}𝐗{)}^{-1}$

where:

$\widehat{𝐃}=\mathrm{diag}(e_1^2,e_2^2,\dots ,e_n^2)$

and $e_i$ is the OLS residual for observation $i$.

Equivalently:

${\widehat{\mathrm{Var}}}_{HC0}(\widehat{𝜷})=({𝐗}^{𝖳}𝐗{)}^{-1}\left(\sum _{i=1}^n{e}_i^2{𝐱}_i{𝐱}_i^{𝖳}\right)({𝐗}^{𝖳}𝐗{)}^{-1}$

where ${𝐱}_i$ is the vector of explanatory variables for observation $i$.

HC1 estimator

HC1 applies a degrees-of-freedom adjustment to HC0:

${\widehat{\mathrm{Var}}}_{HC1}=\frac{n}{n-k}{\widehat{\mathrm{Var}}}_{HC0}$

where:

• $n$ = number of observations;
• $k$ = number of estimated regression coefficients, including the intercept where applicable.

HC1 is closely related to the conventional finite-sample degrees-of-freedom adjustment used in OLS variance estimation.

HC2 and HC3 estimators

HC2 and HC3 adjust the residual contribution using the leverage of each observation.

The leverage of observation $i$ is:

$h_{ii}={𝐱}_i^{𝖳}({𝐗}^{𝖳}𝐗{)}^{-1}{𝐱}_i$

The HC2 residual adjustment is:

$\frac{e_i^2}{1-h_{ii}}$

The corresponding estimator can be written as:

${\widehat{\mathrm{Var}}}_{HC2}=({𝐗}^{𝖳}𝐗{)}^{-1}\left(\sum _{i=1}^n\frac{e_i^2}{1-h_{ii}}{𝐱}_i{𝐱}_i^{𝖳}\right)({𝐗}^{𝖳}𝐗{)}^{-1}$

HC3 applies a stronger leverage adjustment:

$\frac{e_i^2}{(1-h_{ii}{)}^2}$

giving:

${\widehat{\mathrm{Var}}}_{HC3}=({𝐗}^{𝖳}𝐗{)}^{-1}\left(\sum _{i=1}^n\frac{e_i^2}{(1-h_{ii}{)}^2}{𝐱}_i{𝐱}_i^{𝖳}\right)({𝐗}^{𝖳}𝐗{)}^{-1}$

HC3 is often used when observations may have substantial leverage or when the sample is relatively small.

HC4 and later estimators

Additional estimators, including HC4, HC4m, and HC5, apply alternative adjustments for observations with high leverage.

These estimators are designed to reduce the influence of observations whose leverage may cause the HC covariance estimator to underestimate sampling variability.

The exact adjustment differs among the estimators.

Purpose

HC standard errors are designed primarily to improve statistical inference when the OLS error variance is not constant.

They can be used to calculate:

• robust standard errors;
• confidence intervals;
• t-statistics;
• hypothesis tests.

The OLS coefficient estimates themselves are generally unchanged when conventional standard errors are replaced with HC standard errors.

Thus:

$\text{OLS coefficients}\to \text{unchanged}$

while:

$\text{Estimated standard errors}\to \text{adjusted}$

and consequently:

$\text{t-statistics and confidence intervals}\to \text{may change}$

Example

Suppose an OLS regression estimates:

$Y_i={\beta }_0+{\beta }_1{X}_i+{ϵ}_i$

and the estimated coefficient is:

${\hat{\beta }}_1=0.80$

Suppose the conventional standard error is:

$\mathrm{SE}({\hat{\beta }}_1)=0.20$

The conventional t-statistic for testing:

$H_0:{\beta }_1=0$

is:

$t=\frac{0.80}{0.20}=4.00$

If heteroskedasticity-consistent estimation produces:

${\mathrm{SE}}_{HC3}({\hat{\beta }}_1)=0.30$

then the coefficient remains:

${\hat{\beta }}_1=0.80$

but the robust t-statistic becomes:

$t_{HC3}=\frac{0.80}{0.30}\approx 2.67$

Thus, robust standard errors can materially affect statistical inference without changing the estimated regression coefficient.

Limitations

Heteroskedasticity-consistent standard errors do not solve every regression problem.

They do not, by themselves, correct for:

• Omitted-variable bias;
• endogeneity;
• measurement error;
• incorrect functional form;
• Multicollinearity;
• Autocorrelation or other serial dependence unless an appropriate covariance estimator is used;
• non-stationarity;
• structural breaks.

HC standard errors are therefore a correction to the estimated covariance matrix rather than a replacement for appropriate model specification.

Heteroskedasticity versus autocorrelation

HC standard errors are primarily designed for heteroskedasticity where observations are otherwise appropriately treated as independent.

When regression errors are both heteroskedastic and serially correlated, a covariance estimator designed to account for both features may be required, such as a suitable Newey–West estimator in applicable time-series settings.

The distinction can be summarized as:

$\text{HC standard errors}\to \text{Heteroskedasticity}$

$\text{HAC standard errors}\to \text{Heteroskedasticity + Autocorrelation}$

See also

• Heteroskedasticity
• Ordinary least squares
• Regression analysis
• Multiple linear regression
• Homoskedasticity
• Newey–West estimator
• Autocorrelation
• Multicollinearity
• Robust regression
• Econometrics
• Statistical inference