Coefficient of determination

Coefficient of determination is a statistical measure that indicates the proportion of the variation in a dependent variable that is explained by a statistical model. It is commonly denoted by $R^2$ and is widely used in linear regression.

For a regression model with an intercept, the coefficient of determination can be expressed as:

$R^2=1-\frac{{\mathrm{S}\mathrm{S}}_{\mathrm{r}\mathrm{e}\mathrm{s}}}{{\mathrm{S}\mathrm{S}}_{\mathrm{t}\mathrm{o}\mathrm{t}}}$

where:

• $R^2$ = coefficient of determination;
• ${\mathrm{S}\mathrm{S}}_{\mathrm{r}\mathrm{e}\mathrm{s}}$ = residual sum of squares;
• ${\mathrm{S}\mathrm{S}}_{\mathrm{t}\mathrm{o}\mathrm{t}}$ = total sum of squares.

Sum of squares

The total sum of squares measures the total variation of the observed dependent variable around its mean:

${\mathrm{S}\mathrm{S}}_{\mathrm{t}\mathrm{o}\mathrm{t}}=\sum _{i=1}^n(Y_i-\overline{Y}{)}^2$

The residual sum of squares measures the variation that remains unexplained by the fitted model:

${\mathrm{S}\mathrm{S}}_{\mathrm{r}\mathrm{e}\mathrm{s}}=\sum _{i=1}^n(Y_i-{\hat{Y}}_i{)}^2$

where:

• $Y_i$ = observed value;
• ${\hat{Y}}_i$ = fitted value;
• $\overline{Y}$ = mean of the observed values;
• $n$ = number of observations.

The explained sum of squares is:

${\mathrm{S}\mathrm{S}}_{\mathrm{e}\mathrm{x}\mathrm{p}\mathrm{l}\mathrm{a}\mathrm{i}\mathrm{n}\mathrm{e}\mathrm{d}}=\sum _{i=1}^n({\hat{Y}}_i-\overline{Y}{)}^2$

For ordinary least-squares regression with an intercept:

${\mathrm{S}\mathrm{S}}_{\mathrm{t}\mathrm{o}\mathrm{t}}={\mathrm{S}\mathrm{S}}_{\mathrm{e}\mathrm{x}\mathrm{p}\mathrm{l}\mathrm{a}\mathrm{i}\mathrm{n}\mathrm{e}\mathrm{d}}+{\mathrm{S}\mathrm{S}}_{\mathrm{r}\mathrm{e}\mathrm{s}}$

Therefore:

$R^2=\frac{{\mathrm{S}\mathrm{S}}_{\mathrm{e}\mathrm{x}\mathrm{p}\mathrm{l}\mathrm{a}\mathrm{i}\mathrm{n}\mathrm{e}\mathrm{d}}}{{\mathrm{S}\mathrm{S}}_{\mathrm{t}\mathrm{o}\mathrm{t}}}$

Interpretation

An $R^2$ of 0 indicates that the model does not explain any of the variation in the dependent variable relative to a model containing only an intercept.

An $R^2$ of 1 indicates that the fitted model explains all of the observed variation in the dependent variable.

For example:

$R^2=0.70$

can be interpreted, in the corresponding regression setting, as the model accounting for 70% of the sample variation in the dependent variable around its mean.

The remaining 30% is represented by the residual component:

$1-R^2=0.30$

The interpretation should be made in the context of the model and data; $R^2$ does not establish causation.

Coefficient of determination in simple linear regression

In a simple linear regression with an intercept, the coefficient of determination is equal to the square of the Pearson correlation coefficient:

$R^2=r_{XY}^2$

where $r_{XY}$ is the Pearson correlation coefficient between the dependent variable and explanatory variable.

Because the correlation coefficient ranges from −1 to +1:

$0\le R^2\le 1$

for this standard simple-regression setting.

The squaring operation removes the sign of the relationship. Therefore, $R^2$ alone does not indicate whether the relationship is positive or negative.

Example

Suppose a regression model produces:

${\mathrm{S}\mathrm{S}}_{\mathrm{t}\mathrm{o}\mathrm{t}}=1000$

and:

${\mathrm{S}\mathrm{S}}_{\mathrm{r}\mathrm{e}\mathrm{s}}=250$

The coefficient of determination is:

$R^2=1-\frac{250}{1000}=0.75$

Thus, the model explains 75% of the sample variation in the dependent variable relative to the mean-only benchmark.

The unexplained proportion is:

$1-R^2=0.25$

or 25%.

Adjusted coefficient of determination

The adjusted coefficient of determination adjusts $R^2$ for the number of explanatory variables included in a regression model.

A commonly used formula is:

${\overline{R}}^2=1-(1-R^2)\frac{n-1}{n-k-1}$

where:

• ${\overline{R}}^2$ = adjusted coefficient of determination;
• $R^2$ = coefficient of determination;
• $n$ = number of observations;
• $k$ = number of explanatory variables.

Unlike ordinary $R^2$, adjusted $R^2$ can decrease when additional explanatory variables are added if they do not provide sufficient explanatory power relative to the additional model complexity.

R-squared in financial models

In finance, $R^2$ is commonly used when evaluating regression models of asset returns.

For example, a market model can be written as:

$R_i={\alpha }_i+{\beta }_i{R}_m+{ϵ}_i$

where:

• $R_i$ = asset return;
• ${\alpha }_i$ = regression intercept;
• ${\beta }_i$ = market sensitivity;
• $R_m$ = market return;
• ${ϵ}_i$ = residual return.

The resulting $R^2$ indicates the proportion of the variation in the asset's returns explained by the fitted market model over the specified sample period.

A higher $R^2$ indicates that the model explains a larger proportion of the observed variation, while a lower $R^2$ indicates that a larger proportion remains in the residual component.

R-squared and beta

$R^2$ and beta measure different aspects of a regression relationship.

Beta measures the estimated sensitivity of the dependent variable to changes in an explanatory variable:

${\beta }_i=\frac{\mathrm{Cov}(R_i,R_m)}{\mathrm{Var}(R_m)}$

By contrast, $R^2$ measures the proportion of variation in the dependent variable explained by the regression model.

Therefore:

$\text{Beta}\to \text{Sensitivity}$

$R^2\to \text{Explained Variation}$

Limitations

The coefficient of determination has several limitations:

• a high $R^2$ does not establish causation;
• a low $R^2$ does not necessarily mean that a model is useless;
• $R^2$ can increase when additional variables are added to a regression;
• $R^2$ does not by itself indicate whether estimated coefficients are statistically significant;
• $R^2$ does not establish that a model is correctly specified;
• a high $R^2$ can result from spurious relationships in some settings;
• the value of $R^2$ is dependent on the data and model specification.

Consequently, $R^2$ is generally considered alongside other measures such as coefficient estimates, standard errors, statistical tests, residual diagnostics, and economic or substantive reasoning.

See also

• Linear regression
• Ordinary least squares
• Adjusted R-squared
• Pearson correlation coefficient
• Correlation
• Covariance
• Coefficient of correlation
• Beta (finance)
• Alpha (finance)
• Regression analysis
• Factor model
• Statistical significance