Topics on LASSO and Approximate Message Passing

Date
2014-04-25
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract

This thesis studies the performance of the LASSO (also known as basis pursuit denoising) for recovering sparse signals from undersampled, randomized, noisy measurements. We consider the recovery of the signal $$x_o \in \mathbb{R}^N$$ from $$n$$ random and noisy linear observations $$y= Ax_o + w$$, where $$A$$ is the measurement matrix and $$w$$ is the noise. The LASSO estimate is given by the solution to the optimization problem $$x_o$$ with $$\hat{x}_{\lambda} = \arg \min_x \frac{1}{2} |y-Ax|_2^2 + \lambda |x|1$$. Despite major progress in the theoretical analysis of the LASSO solution, little is known about its behavior as a function of the regularization parameter $$\lambda$$. In this thesis we study two questions in the asymptotic setting (i.e., where $$N \rightarrow \infty$$, $$n \rightarrow \infty$$ while the ratio $$n/N$$ converges to a fixed number in $$(0,1)$$): (i) How does the size of the active set $$|\hat{x}\lambda|0/N$$ behave as a function of $$\lambda$$, and (ii) How does the mean square error $$|\hat{x}{\lambda} - x_o|_2^2/N$$ behave as a function of $$\lambda$$? We then employ these results in a new, reliable algorithm for solving LASSO based on approximate message passing (AMP). Furthermore, we propose a parameter-free approximate message passing (AMP) algorithm that sets the threshold parameter at each iteration in a fully automatic way without either having an information about the signal to be reconstructed or needing any tuning from the user. We show that the proposed method attains the minimum reconstruction error in the least number of iterations . Our method is based on applying the Stein unbiased risk estimate (SURE) along with a modified gradient descent to find the optimal threshold in each iteration. Motivated by the connections between AMP and LASSO, it could be employed to find the solution of the LASSO for the optimal regularization parameter. To the best of our knowledge, this is the first work concerning parameter tuning that obtains the smallest MSE in the least number of iterations with theoretical guarantees.

Description
Degree
Master of Science
Type
Thesis
Keywords
LASSO, Sparsity, Message passing
Citation

Mousavi, Ali. "Topics on LASSO and Approximate Message Passing." (2014) Master’s Thesis, Rice University. https://hdl.handle.net/1911/77216.

Has part(s)
Forms part of
Published Version
Rights
Copyright is held by the author, unless otherwise indicated. Permission to reuse, publish, or reproduce the work beyond the bounds of fair use or other exemptions to copyright law must be obtained from the copyright holder.
Link to license
Citable link to this page