The Generalized Ridge Trace Plot: Visualizing Bias and Precision

• Michael Friendly.
• The Generalized Ridge Trace Plot: Visualizing Bias and Precision.
• Journal of Computational and Graphical Statistics, vol. 22, no. 1, pp. 50–68, 2013.

Download: from web | supplementary materials

In ridge regression and related shrinkage methods, the ridge trace plot, a plot of estimated coefficients against a shrinkage parameter, is a common graphical adjunct to help determine a favorable tradeoff of bias against precision (inverse variance) of the estimates. However, standard unidimensional versions of this plot are ill-suited for this purpose because they show only bias directly and ignore the multi-dimensional nature of the problem. We introduce a generalized version of the ridge trace plot, showing covariance ellipsoids in parameter space, whose centers show bias and whose size and shape show variance and covariance in relation to the criteria for which these methods were developed. These provide a direct visualization of both bias and precision. Even 2D bivariate versions of this plot show interesting features not revealed in the standard univariate version. Low-rank versions of this plot, based on an orthogonal transformation of predictor space extend these ideas to larger numbers of predictor variables, by focusing on the dimensions in the space of predictors that are likely to be most informative about the nature of bias and precision. Two well-known data sets are used to illustrate these graphical methods. The genridge package for R implements computation and display.

Keywords: biplot; model selection; multivariate bootstrap; regression shrinkage; ridge regression; ridge trace plot; singular value decomposition; variance-shrinkage tradeoff

@Article{Friendlygenridge2013,
  author = {Michael Friendly},
  title = {The Generalized Ridge Trace Plot: Visualizing Bias and Precision},
  journal = {Journal of Computational and Graphical Statistics},
  year = {2013},
  volume = {22},
  number = {1},
  pages = {50–68},
  url = {http://datavis.ca/papers/genridge-jcgs.pdf},
  supp = {http://datavis.ca/papers/genridge-supp.zip},
}