The Best Subsets Regression command involves examining all the models for all possible combinations of predictor variables and determines the best set of predictors for each subset size. It can be used as an alternative to the stepwise regression procedures. The best subsets regression is also known as all possible subsets regression.

How To

Run: Statistics→Regression → Best Subsets Regression...

Select Dependent (Response) variable and Independent variables (Predictors).

Select the Show correlations option to display the partial correlation matrix at each step.

Select the Show descriptive statistics option to show the descriptive statistics (mean, variance and standard deviation) for each term.

Casewise deletion method is used for missing values removal.

Results

Best subset regression command selects the subset of predictors at each step that fits best, based on the criterion of having the largest R².  The report includes a set of best fitted models with standardized regression statistics and ANOVA summary for each subset size (from 1 to the number of predictors). Correlation coefficients matrix and descriptive statistics for predictors are displayed if the corresponding options are selected.

 

R² Increment – is the increment of R² in comparison with the previous subset.

R² (Coefficient of determination, R-squared) - is the square of the sample correlation coefficient between the Predictors  (independent variables) and Response (dependent variable). In general, R² is a percentage of response variable variation that is explained by its relationship with one or more predictor variables. The definition of the R² is

Adjusted R² (Adjusted R-squared) - is a modification of R² that adjusts for the number of explanatory terms in a model. Adjusted R² is computed using the formula
 where k is the number of predictors.

S – the estimated standard deviation of the error in the model. Identifying the model with the smallest mean square error (MSE) is equivalent to finding the model with the smallest S.

MS (Mean Square) - an estimate of the variation accounted for by this term.

F - the F-test value.

p-value – p-value for a F-test. A value less than  level (0.05) shows that the model estimated by the regression procedure is significant.

References

[NWK] Neter, J., Wasserman, W. and Kutner, M. H. (1996). Applied Linear Statistical Models, Irwin, Chicago.

[NRM] Nargundkar R. (2008) Marketing Research: Text and Cases. Third edition. Tata McGraw-Hill Publishing Company Ltd.

[HRA] Hocking, R. R. (1976) "The Analysis and Selection of Variables in Linear Regression," Biometrics, 32

[OMK] Olejnik, S. Mills, R. and Keselman, H. “Using Wherry’s Adjusted R2 and Mallows' Cp for Model Selection from All Possible Regressions”, The Journal of Experimental Education, 2000, 68(4), 365-380.