Browse
Recent Submissions
Now showing 1 - 20 of 508
Item A Theoretical Analysis of Joint Manifolds(2009-01) Davenport, Mark A.; Hegde, Chinmay; Duarte, Marco; Baraniuk, Richard G.The emergence of low-cost sensor architectures for diverse modalities has made it possible to deploy sensor arrays that capture a single event from a large number of vantage points and using multiple modalities. In many scenarios, these sensors acquire very high-dimensional data such as audio signals, images, and video. To cope with such high-dimensional data, we typically rely on low-dimensional models. Manifold models provide a particularly powerful model that captures the structure of high-dimensional data when it is governed by a low-dimensional set of parameters. However, these models do not typically take into account dependencies among multiple sensors. We thus propose a new joint manifold framework for data ensembles that exploits such dependencies. We show that simple algorithms can exploit the joint manifold structure to improve their performance on standard signal processing applications. Additionally, recent results concerning dimensionality reduction for manifolds enable us to formulate a network-scalable data compression scheme that uses random projections of the sensed data. This scheme efficiently fuses the data from all sensors through the addition of such projections, regardless of the data modalities and dimensions.Item Sparse Coding with Population Sketches(BMC Neuroscience, 2009-07-13) Dyer, Eva L.; Baraniuk, Richard G.; Johnson, Don H.Item Fast, Exact Synthesis of Gaussian and nonGaussian Long-Range-Dependent Processes(2009-04-15) Baraniuk, Richard; Crouse, Matthew1/f noise and statistically self-similar random processes such as fractional Brownian motion (fBm) and fractional Gaussian noise (fGn) are fundamental models for a host of real-world phenomena, from network traffic to DNA to the stock market. Synthesis algorithms play a key role by providing the feedstock of data necessary for running complex simulations and accurately evaluating analysis techniques. Unfortunately, current algorithms to correctly synthesize these long-range dependent (LRD) processes are either abstruse or prohibitively costly, which has spurred the wide use of inexact approximations. To fill the gap, we develop a simple, fast (O(N logN) operations for a length-N signal) framework for exactly synthesizing a range of Gaussian and nonGaussian LRD processes. As a bonus, we introduce and study a new bi-scaling fBm process featuring a "kinked" correlation function that exhibits distinct scaling laws at coarse and fine scales.Item Tuning support vector machines for minimax and Neyman-Pearson classification(2008-08-19) Scott, Clayton D.; Baraniuk, Richard G.; Davenport, Mark A.This paper studies the training of support vector machine (SVM) classifiers with respect to the minimax and Neyman-Pearson criteria. In principle, these criteria can be optimized in a straightforward way using a cost-sensitive SVM. In practice, however, because these criteria require especially accurate error estimation, standard techniques for tuning SVM parameters, such as crossvalidation, can lead to poor classifier performance. To address this issue, we first prove that the usual cost-sensitive SVM, here called the 2C-SVM, is equivalent to another formulation called the 2nu-SVM. We then exploit a characterization of the 2nu-SVM parameter space to develop a simple yet powerful approach to error estimation based on smoothing. In an extensive experimental study we demonstrate that smoothing significantly improves the accuracy of cross-validation error estimates, leading to dramatic performance gains. Furthermore, we propose coordinate descent strategies that offer significant gains in computational efficiency, with little to no loss in performance.Item A simple proof of the restricted isometry property for random matrices(2007-01-18) Baraniuk, Richard G.; Davenport, Mark A.; DeVore, Ronald A.; Wakin, Michael B.We give a simple technique for verifying the Restricted Isometry Property (as introduced by Candès and Tao) for random matrices that underlies Compressed Sensing. Our approach has two main ingredients: (i) concentration inequalities for random inner products that have recently provided algorithmically simple proofs of the Johnson–Lindenstrauss lemma; and (ii) covering numbers for finite-dimensional balls in Euclidean space. This leads to an elementary proof of the Restricted Isometry Property and brings out connections between Compressed Sensing and the Johnson–Lindenstrauss lemma. As a result, we obtain simple and direct proofs of Kashin’s theorems on widths of finite balls in Euclidean space (and their improvements due to Gluskin) and proofs of the existence of optimal Compressed Sensing measurement matrices. In the process, we also prove that these measurements have a certain universality with respect to the sparsity-inducing basis.Item Single-pixel imaging via compressive sampling(2008-03-01) Duarte, Marco F.; Davenport, Mark A.; Takhar, Dharmpal; Laska, Jason N.; Sun, Ting; Kelly, Kevin F.; Baraniuk, Richard G.Item Multiscale random projections for compressive classification(2007-09-01) Duarte, Marco F.; Davenport, Mark A.; Wakin, Michael B.; Laska, Jason N.; Takhar, Dharmpal; Kelly, Kevin F.; Baraniuk, Richard G.We propose a framework for exploiting dimension-reducing random projections in detection and classification problems. Our approach is based on the generalized likelihood ratio test; in the case of image classification, it exploits the fact that a set of images of a fixed scene under varying articulation parameters forms a low-dimensional, nonlinear manifold. Exploiting recent results showing that random projections stably embed a smooth manifold in a lower-dimensional space, we develop the multiscale smashed filter as a compressive analog of the familiar matched filter classifier. In a practical target classification problem using a single-pixel camera that directly acquires compressive image projections, we achieve high classification rates using many fewer measurements than the dimensionality of the images.Item Minimax support vector machines(2007-08-01) Davenport, Mark A.; Baraniuk, Richard G.; Scott, Clayton D.We study the problem of designing support vector machine (SVM) classifiers that minimize the maximum of the false alarm and miss rates. This is a natural classification setting in the absence of prior information regarding the relative costs of the two types of errors or true frequency of the two classes in nature. Examining two approaches – one based on shifting the offset of a conventionally trained SVM, the other based on the introduction of class-specific weights – we find that when proper care is taken in selecting the weights, the latter approach significantly outperforms the strategy of shifting the offset. We also find that the magnitude of this improvement depends chiefly on the accuracy of the error estimation step of the training procedure. Furthermore, comparison with the minimax probability machine (MPM) illustrates that our SVM approach can outperform the MPM even when the MPM parameters are set by an oracle.Item The smashed filter for compressive classification and target recognition(2007-01-01) Davenport, Mark A.; Duarte, Marco F.; Wakin, Michael B.; Laska, Jason N.; Takhar, Dharmpal; Kelly, Kevin F.; Baraniuk, Richard G.The theory of compressive sensing (CS) enables the reconstruction of a sparse or compressible image or signal from a small set of linear, non-adaptive (even random) projections. However, in many applications, including object and target recognition, we are ultimately interested in making a decision about an image rather than computing a reconstruction. We propose here a framework for compressive classification that operates directly on the compressive measurements without first reconstructing the image. We dub the resulting dimensionally reduced matched filter the smashed filter. The first part of the theory maps traditional maximum likelihood hypothesis testing into the compressive domain; we find that the number of measurements required for a given classification performance level does not depend on the sparsity or compressibility of the images but only on the noise level. The second part of the theory applies the generalized maximum likelihood method to deal with unknown transformations such as the translation, scale, or viewing angle of a target object. We exploit the fact the set of transformed images forms a low-dimensional, nonlinear manifold in the high-dimensional image space. We find that the number of measurements required for a given classification performance level grows linearly in the dimensionality of the manifold but only logarithmically in the number of pixels/samples and image classes. Using both simulations and measurements from a new single-pixel compressive camera, we demonstrate the effectiveness of the smashed filter for target classification using very few measurements.Item Regression level set estimation via cost-sensitive classification(2007-06-01) Scott, Clayton D.; Davenport, Mark A.Regression level set estimation is an important yet understudied learning task. It lies somewhere between regression function estimation and traditional binary classification, and in many cases is a more appropriate setting for questions posed in these more common frameworks. This note explains how estimating the level set of a regression function from training examples can be reduced to cost-sensitive classification. We discuss the theoretical and algorithmic benefits of this learning reduction, demonstrate several desirable properties of the associated risk, and report experimental results for histograms, support vector machines, and nearest neighbor rules on synthetic and real data.Item Detection and estimation with compressive measurements(2006-11-01) Baraniuk, Richard G.; Davenport, Mark A.; Wakin, Michael B.The recently introduced theory of compressed sensing enables the reconstruction of sparse or compressible signals from a small set of nonadaptive, linear measurements. If properly chosen, the number of measurements can be much smaller than the number of Nyquist rate samples. Interestingly, it has been shown that random projections are a satisfactory measurement scheme. This has inspired the design of physical systems that directly implement similar measurement schemes. However, despite the intense focus on the reconstruction of signals, many (if not most) signal processing problems do not require a full reconstruction of the signal { we are often interested only in solving some sort of detection problem or in the estimation of some function of the data. In this report, we show that the compressed sensing framework is useful for a wide range of statistical inference tasks. In particular, we demonstrate how to solve a variety of signal detection and estimation problems given the measurements without ever reconstructing the signals themselves. We provide theoretical bounds along with experimental results.Item Small-time scaling behaviors of Internet backbone traffic: An empirical study(2003-04-20) Zhang, Zhi-Li; Ribeiro, Vinay Joseph; Moon, Sue; Diot, Christophe; Center for Multimedia Communications (http://cmc.rice.edu/); Digital Signal Processing (http://dsp.rice.edu/)We study the small-time (sub-seconds) scaling behaviors of Internet backbone traffic, based on traces collected from OC3/12/48 links in a tier-1 ISP. We observe that for a majority of these traces, the (second-order) scaling exponents at small time scales (1ms - 100ms) are fairly close to 0.5, indicating that traffic fluctuations at these time scales are (nearly) uncorrelated. In addition, the traces manifest mostly monofractal behaviors at small time scales. The objective of the paper is to understand the potential causes or factors that influence the small-time scalings of Internet backbone traffic via empirical data analysis. We analyze the traffic composition of the traces along two dimensions â flow size and flow density. Our study uncovers dense flows (i.e., flows with bursts of densely clustered packets) as the correlation-causing factor in small time scales, and reveals that the traffic composition in terms of proportions of dense vs. sparse flows plays a major role in influecing the small-time scalings of aggregate traffic.Item Multiscale Density Estimation(2003-08-20) Willett, Rebecca; Nowak, Robert David; Digital Signal Processing (http://dsp.rice.edu/)The nonparametric density estimation method proposed in this paper is computationally fast, capable of detecting density discontinuities and singularities at a very high resolution, spatially adaptive, and offers near minimax convergence rates for broad classes of densities including Besov spaces. At the heart of this new method lie multiscale signal decompositions based on piecewise-polynomial functions and penalized likelihood estimation. Upper bounds on the estimation error are derived using an information-theoretic risk bound based on squared Hellinger loss. The method and theory share many of the desirable features associated with wavelet-based density estimators, but also offers several advantages including guaranteed non-negativity, bounds on the L1 error, small-sample quantification of the estimation errors, and additional flexibility and adaptability. In particular, the method proposed here can adapt the degrees as well as the locations of the polynomial pieces. For a certain class of densities, the error of the variable degree estimator converges at nearly the parametric rate. Experimental results demonstrate the advantages of the new approach compared to traditional density estimators and wavelet-based estimators.Item Multiscale Likelihood Analysis and Image Reconstruction(2003-08-20) Willett, Rebecca; Nowak, Robert David; Digital Signal Processing (http://dsp.rice.edu/)The nonparametric multiscale polynomial and platelet methods presented here are powerful new tools for signal and image denoising and reconstruction. Unlike traditional wavelet-based multiscale methods, these methods are both well suited to processing Poisson or multinomial data and capable of preserving image edges. At the heart of these new methods lie multiscale signal decompositions based on polynomials in one dimension and multiscale image decompositions based on what the authors call platelets in two dimensions. Platelets are localized functions at various positions, scales and orientations that can produce highly accurate, piecewise linear approximations to images consisting of smooth regions separated by smooth boundaries. Polynomial and platelet-based maximum penalized likelihood methods for signal and image analysis are both tractable and computationally efficient. Polynomial methods offer near minimax convergence rates for broad classes of functions including Besov spaces. Upper bounds on the estimation error are derived using an information-theoretic risk bound based on squared Hellinger loss. Simulations establish the practical effectiveness of these methods in applications such as density estimation, medical imaging, and astronomy.Item Platelets for Multiscale Analysis in Photon-Limited Imaging(2002-09-20) Willett, Rebecca; Nowak, Robert David; Digital Signal Processing (http://dsp.rice.edu/)This paper proposes a new multiscale image decomposition based on platelets. Platelets are localized functions at various scales, locations, and orientations that produce piecewise linear image approximations. For smoothness measured in certain H¨older classes, the error of m-term platelet approximations can decay significantly faster than that of m-term approximations in terms of sinusoids, wavelets, or wedgelets. Platelet representations are especially well suited for the analysis of Poisson data, unlike most other multiscale image representations, and they can be rapidly computed. We propose a platelet-based maximum penalized likelihood criterion that encompasses denoising, deblurring, and tomographic reconstruction.Item Platelets for Multiscale Analysis in Medical Imaging(2002-04-20) Willett, Rebecca; Nowak, Robert David; Digital Signal Processing (http://dsp.rice.edu/)This paper describes the development and use of multiscale, platelet-based image reconstruction algorithms in medical imaging. Such algorithms are effective because platelets approximate images in certain (piecewise) smoothness classes significantly more efficiently than sinusoids, wavelets, or wedgelets. Platelet representations are especially well-suited to the analysis of Poisson data, unlike most other multiscale image representations, and they can be rapidly computed. We present a fast, platelet-based maximum penalized likelihood algorithm that encompasses denoising, deblurring, and tomographic reconstruction and its applications to photon-limited imaging.Item Multiscale Analysis for Intensity and Density Estimation(2002-04-20) Willett, Rebecca; Digital Signal Processing (http://dsp.rice.edu/)The nonparametric multiscale polynomial and platelet algorithms presented in this thesis are powerful new tools for signal and image denoising and reconstruction. Unlike traditional wavelet-based multiscale methods, these algorithms are both well suited to processing Poisson and multinomial data and capable of preserving image edges. At the heart of these new algorithms lie multiscale signal decompositions based on polynomials in one dimension and multiscale image decompositions based on platelets in two dimensions. This thesis introduces platelets, localized atoms at various locations, scales and orientations that can produce highly accurate, piecewise linear approximations to images consisting of smooth regions separated by smooth boundaries. Polynomial- and platelet-based maximum penalized likelihood methods for signal and image analysis are both tractable and computationally efficient. Simulations establish the practical effectiveness of these algorithms in applications such as medical and astronomical, density estimation, and networking; statistical risk bounds establish the theoretical near-optimality of these algorithms.Item Platelets: A Multiscale Approach for Recovering Edges and Surfaces in Photon-Limited Medical Imaging(2003) Willett, Rebecca; Nowak, Robert David; Digital Signal Processing (http://dsp.rice.edu/)This paper proposes a new multiscale image decomposition based on platelets. Platelets are localized functions at various scales, locations, and orientations that produce piecewise linear image approximations. Platelets are well suited for approximating images consisting of smooth regions separated by smooth boundaries. For smoothness measured in certain Holder classes, it is shown that the error of m-term platelet approximations can decay significantly faster than that of m-term approximations in terms of sinusoids, wavelets, or wedgelets. This suggests that platelets may outperform existing techniques for image denoising and reconstruction. Moreover, the platelet decomposition is based on a recursive image partitioning scheme which, unlike conventional wavelet decompositions, is very well suited to photon-limited medical imaging applications involving Poisson distributed data. Fast, platelet-based, maximum penalized likelihood methods for photon-limited image denoising, deblurring and tomographic reconstruction problems are developed. Because platelet decompositions of Poisson distributed images are tractable and computationally efficient, existing image reconstruction methods based on expectation-maximization type algorithms can be easily enhanced with platelet techniques. Experimental results demonstrate that platelet-based methods can outperform standard reconstruction methods currently in use in confocal microscopy, image restoration and emission tomography.Item Multiresolution Nonparametric Intensity and Density Estimation(2002-05-20) Willett, Rebecca; Nowak, Robert David; Digital Signal Processing (http://dsp.rice.edu/)This paper introduces a new multiscale method for nonparametric piecewise polynomial intensity and density estimation of point processes. Fast, piecewise polynomial, maximum penalized likelihood methods for intensity and density estimation are developed. The recursive partitioning scheme underlying these methods is based on multiscale likelihood factorizations which, unlike conventional wavelet decompositions, are very well suited to applications with point process data. Experimental results demonstrate that multiscale methods can outperform wavelet and kernel based density estimation methods.Item Multiresolution Intensity Estimation of Piecewise Linear Poisson Processes(2001-04-20) Willett, Rebecca; Digital Signal Processing (http://dsp.rice.edu/)Given observations of a one-dimensional piecewise linear, length-M Poisson intensity function, our goal is to estimate both the partition points and the parameters of each segment. In order to determine where the breaks lie, we develop a maximum penalized likelihood estimator based on information-theoretic complexity penalization. We construct a probabilistic model of the observations within a multiscale framework, and use this framework to devise a computationally efficient optimization algorithm, based on a tree-pruning approach, to compute the MPLE.