A New Regularization Path for Logistic Regression via Linearized Bregman

Simple item page
dc.contributor.authorShi, J.V.
dc.contributor.authorYin, W.
dc.contributor.authorOsher, S.J.
dc.date.accessioned2018-06-19T17:48:02Z
dc.date.available2018-06-19T17:48:02Z
dc.date.issued2012-10
dc.date.noteOctober 2012
dc.description.abstractSparse logistic regression is an important linear classifier in statistical learning, providing an attractive route for feature selection. A popular approach is based on minimizing an l1-regularization term with a regularization parameter lambda that affects the solution sparsity. To determine an appropriate value for the regularization parameter, one can apply the grid search method or the Bayesian approach. The grid search method requires constructing a regularization path, by solving a sequence of minimization problems with varying values of the regularization parameter, which is typically time consuming. In this paper, we introduce a fast procedure that generates a new regularization path without tuning the regularization parameter. We first derive the direct Bregman method by replacing the l1-norm by its Bregman divergence, and contrast it with the grid search method. For faster path computation, we further derive the linearized Bregman algorithm, which is algebraically simple and computationally efficient. Finally we demonstrate some empirical results for the linearized Bregman algorithm on benchmark data and study feature selection as an inverse problem. Compared with the grid search method, the linearized Bregman algorithm generates a different regularization path with comparable classification performance, in a much more computationally efficient manner.
dc.format.extent27 pp
dc.identifier.citationShi, J.V., Yin, W. and Osher, S.J.. "A New Regularization Path for Logistic Regression via Linearized Bregman." (2012) <a href="https://hdl.handle.net/1911/102213">https://hdl.handle.net/1911/102213</a>.
dc.identifier.digitalTR12-24
dc.identifier.urihttps://hdl.handle.net/1911/102213
dc.language.isoeng
dc.titleA New Regularization Path for Logistic Regression via Linearized Bregman
dc.typeTechnical report
dc.type.dcmiText
Files
Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
TR12-24.PDF
Size:
1.26 MB
Format:
Adobe Portable Document Format
Download
Collections
CAAM Technical Reports