TY - JOUR
T1 - AIC for the lasso in generalized linear models
AU - Ninomiya, Yoshiyuki
AU - Kawano, Shuichi
N1 - Publisher Copyright:
© 2016 The Institute of Mathematical Statistics and the Bernoulli Society.
PY - 2016
Y1 - 2016
N2 - The Lasso is a popular regularization method that can simultaneously do estimation and model selection. It contains a regularization parameter, and several information criteria have been proposed for selecting its proper value.While any of them would assure consistency in model selection, we have no appropriate rule to choose between the criteria. Meanwhile, a finite correction to the AIC has been provided in a Gaussian regression setting. The finite correction is theoretically assured from the viewpoint not of the consistency but of minimizing the prediction error and does not have the above-mentioned difficulty. Our aim is to derive such a criterion for the Lasso in generalized linear models. Towards this aim, we derive a criterion from the original definition of the AIC, that is, an asymptotically unbiased estimator of the Kullback-Leibler divergence. This becomes the finite correction in the Gaussian regression setting, and so our criterion can be regarded as its generalization. Our criterion can be easily obtained and requires fewer computational tasks than does cross-validation, but simulation studies and real data analyses indicate that its performance is almost the same as or superior to that of cross-validation. Moreover, our criterion is extended for a class of other regularization methods.
AB - The Lasso is a popular regularization method that can simultaneously do estimation and model selection. It contains a regularization parameter, and several information criteria have been proposed for selecting its proper value.While any of them would assure consistency in model selection, we have no appropriate rule to choose between the criteria. Meanwhile, a finite correction to the AIC has been provided in a Gaussian regression setting. The finite correction is theoretically assured from the viewpoint not of the consistency but of minimizing the prediction error and does not have the above-mentioned difficulty. Our aim is to derive such a criterion for the Lasso in generalized linear models. Towards this aim, we derive a criterion from the original definition of the AIC, that is, an asymptotically unbiased estimator of the Kullback-Leibler divergence. This becomes the finite correction in the Gaussian regression setting, and so our criterion can be regarded as its generalization. Our criterion can be easily obtained and requires fewer computational tasks than does cross-validation, but simulation studies and real data analyses indicate that its performance is almost the same as or superior to that of cross-validation. Moreover, our criterion is extended for a class of other regularization methods.
KW - Convexity lemma
KW - Information criterion
KW - Kullback- Leibler divergence
KW - Statistical asymptotic theory
KW - Tuning parameter
KW - Variable selection
UR - http://www.scopus.com/inward/record.url?scp=84988708751&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84988708751&partnerID=8YFLogxK
U2 - 10.1214/16-EJS1179
DO - 10.1214/16-EJS1179
M3 - Article
AN - SCOPUS:84988708751
SN - 1935-7524
VL - 10
SP - 2537
EP - 2560
JO - Electronic Journal of Statistics
JF - Electronic Journal of Statistics
IS - 2
ER -