R-3 Repository :: Browsing by Author "Baraniuk, Richard G."

Browsing by Author "Baraniuk, Richard G."

Now showing 1 - 20 of 354

A Compressive Phase-Locked Loop
(2011) Schnelle, Stephen; Baraniuk, Richard G.
We develop a new method for tracking narrowband signals acquired through compressive sensing, called the compressive sensing phase-locked loop (CS-PLL). The CS-PLL enables one to track oscillating signals in very large bandwidths using a small number of measurements. Not only does the CS-PLL potentially operate below the Nyquist rate, it can extract phase and frequency information without the computational complexity normally associated with compressive sensing signal re-construction. The CS-PLL has a wide variety of applications, including but not limited to communications, phase tracking, robust control, sensing, and FM demodulation. In particular we emphasize the advantages of using this system in wideband surveillence systems. Our design modifies classical PLL designs to operate with CS-based sampling systems. Performance results are shown for PLLs operating on both real and complex data. In addition to explaining general performance tradeoffs, implementations using several different CS sampling systems are explored.
A New Approach to the High-Resolution Linear Radon Transform based on Compressive Sensing Theory: Application on Teleseismic Wavefields
(2013-04-19) Aharchaou, Mehdi; Levander, Alan R.; Niu, Fenglin; Baraniuk, Richard G.
The development of new tools for high-resolution seismic imaging has been for many years one of the key challenges faced by earthquake and exploration seismologists. In order to make data amenable to imaging analysis, pre-processing steps are of great importance. This thesis proposes a new method for pre-processing teleseismic data based on the linear radon transform implemented according to compressive sensing theory – a novel theory about acquiring and recovering the sparsest signals (with minimum significant coefficients) in the most efficient way possible with the help of incoherent measurements. The LRT works by mapping data into a sparsity-promoting domain (called the radon domain) where the desired signals can be easily isolated, classiﬁed, ﬁltered and enhanced; and where noise can be attenuated or completely removed. The performance of the LRT is enhanced in terms of both high-resolution and computational cost by formulating the problem as an inverse problem in the frequency domain. This work shows that, unlike the common wisdom, irregularity in spatial sampling of teleseismic wavefields can be beneficial because it provides the incoherency needed to solve the compressive sensing problem and therefore recover the sparsest solutions in the radon domain. The inverse problem formulation yields the added advantage of automatic spatial interpolation and phase isolation after data reconstruction, and enables to regularize the problem by imposing sparsity constraint (instead of smoothness, which is the constraint usually adopted). We discuss and investigate the resolving power and applicability of convex and non-convex types of regularizers inspired from compressive sensing theory, and we establish a lower bound on the number of measurements needed to resolve certain time dips related to signals of interest within the data. We finish by applying the method to synthetic and recorded datasets and show how we do signal extraction, noise removal and spatial interpolation on teleseismic wavefields.
A Probabilistic Framework for Deep Learning
(Neural Information Processing Systems Foundation, Inc., 2016) Patel, Ankit B.; Nguyen, Tan; Baraniuk, Richard G.
We develop a probabilistic framework for deep learning based on the Deep Rendering Mixture Model (DRMM), a new generative probabilistic model that explicitly capture variations in data due to latent task nuisance variables. We demonstrate that max-sum inference in the DRMM yields an algorithm that exactly reproduces the operations in deep convolutional neural networks (DCNs), providing a first principles derivation. Our framework provides new insights into the successes and shortcomings of DCNs as well as a principled route to their improvement. DRMM training via the Expectation-Maximization (EM) algorithm is a powerful alternative to DCN back-propagation, and initial training results are promising. Classification based on the DRMM and other variants outperforms DCNs in supervised digit classification, training 2-3x faster while achieving similar accuracy. Moreover, the DRMM is applicable to semi-supervised and unsupervised learning tasks, achieving results that are state-of-the-art in several categories on the MNIST benchmark and comparable to state of the art on the CIFAR10 benchmark.
A universal hidden Markov tree image model
(1999) Romberg, Justin K.; Baraniuk, Richard G.
Wavelet-domain hidden Markov models have proven to be useful tools for statistical signal and image processing. The hidden Markov tree (HMT) model captures the key features of the joint density of the wavelet coefficients of real-world data. One potential drawback to the HMT framework is the need for computationally expensive iterative training. We propose two reduced-parameter HMT models that capture the general structure of a broad class of real-world images. In the image HMT model, we use the fact that for real-world images the structure of the HMT is self-similar across scale, allowing us to reduce the complexity of the model to just nine parameters. In the universal HMT we fix these nine parameters, eliminating training while retaining nearly all of the key structure modeled by the full HMT. Finally, we propose a fast shift-invariant HMT estimation algorithm that outperforms all other wavelet-based estimators in the current literature.
Active learning and adaptive sampling for non-parametric inference
(2008) Castro, Rui M.; Baraniuk, Richard G.
This thesis presents a general discussion of active learning and adaptive sampling. In many practical scenarios it is possible to use information gleaned from previous observations to focus the sampling process, in the spirit of the "twenty-questions" game. As more samples are collected one can learn how to improve the sampling process by deciding where to sample next, for example. These sampling feedback techniques are generically known as active learning or adaptive sampling. Although appealing, analysis of such methodologies is difficult, since there are strong dependencies between the observed data. This is especially important in the presence of measurement uncertainty or noise. The main thrust of this thesis is to characterize the potential and fundamental limitations of active learning, particularly in non-parametric settings. First, we consider the probabilistic classification setting. Using minimax analysis techniques we investigate the achievable rates of classification error convergence for broad classes of distributions characterized by decision boundary regularity and noise conditions (which describe the observation noise near the decision boundary). The results clearly indicate the conditions under which one can expect significant gains through active learning. Furthermore we show that the learning rates derived are tight for "boundary fragment" classes in d-dimensional feature spaces when the feature marginal density is bounded from above and below. Second we study the problem of estimating an unknown function from noisy point-wise samples, where the sample locations are adaptively chosen based on previous samples and observations, as described above. We present results characterizing the potential and fundamental limits of active learning for certain classes of nonparametric regression problems, and also present practical algorithms capable of exploiting the sampling adaptivity and provably improving upon non-adaptive techniques. Our active sampling procedure is based on a novel coarse-to-fine strategy, based on and motivated by the success of spatially-adaptive methods such as wavelet analysis in nonparametric function estimation. Using the ideas developed when solving the function regression problem we present a greedy algorithm for estimating piecewise constant functions with smooth boundaries that is near minimax optimal but is computationally much more efficient than the best dictionary based method (in this case wedgelet approximations). Finally we compare adaptive sampling (where feedback guiding the sampling process is present) with non-adaptive compressive sampling (where non-traditional projection samples are used). It is shown that under mild noise compressive sampling can be competitive with adaptive sampling, but adaptive sampling significantly outperforms compressive sampling in lower signal-to-noise conditions. Furthermore this work also helps the understanding of the different behavior of compressive sampling under noisy and noiseless settings.
An Adaptive Optimal-Kernel Time-Frequency Representation
(1995-10-01) Jones, Douglas L.; Baraniuk, Richard G.; Digital Signal Processing (http://dsp.rice.edu/)
Time-frequency representations with fixed windows or kernels figure prominently in many applications, but perform well only for limited classes of signals. Representations with signal- dependent kernels can overcome this limitation. However, while they often perform well, most existing schemes are block-oriented techniques unsuitable for on-line implementation or for tracking signal components with characteristics that change with time. The time-frequency representation developed here, based on a signal-dependent radially Gaussian kernel that adapts over time, overcomes these limitations. The method employs a short-time ambiguity function both for kernel optimization and as an intermediate step in computing constant-time slices of the representation. Careful algorithm design provides reasonably efficient computation and allows on-line implementation. Certain enhancements, such as cone-kernel constraints and approximate retention of marginals, are easily incorporated with little additional computation. While somewhat more expensive than fixed-kernel representations, this new technique often provides much better performance. Several examples illustrate its behavior on synthetic and real-world signals.
Adaptive Wavelet Transforms for Image Coding
(1997-11-01) Claypoole, Roger L.; Davis, Geoffrey; Sweldens, Wim; Baraniuk, Richard G.; Digital Signal Processing (http://dsp.rice.edu/)
We introduce a new adaptive transform for wavelet-based image coding. The lifting framework for wavelet construction motivates our analysis and provides new insight into the problem. Since the adaptive transform is non-linear, we examine the central issues of invertibility, stability, and artifacts in its construction. We describe a new type of non-linearity: a set of linear predictors are chosen adaptively using a non-linear selection function. We also describe how earlier families of non-linear filter banks can be extended through the use of prediction functions operating on a causal neighborhood. We present preliminary results for a synthetic test image.
Adaptive Wavelet Transforms for Image Coding
(1997-11-01) Claypoole, Roger L.; Davis, Geoffrey; Sweldens, Wim; Baraniuk, Richard G.; Digital Signal Processing (http://dsp.rice.edu/)
We introduce a new adaptive transform for wavelet-based image coding. The lifting framework for wavelet construction motivates our analysis and provides new insight into the problem. Since the adaptive transform is non-linear, we examine the central issues of invertibility, stability, and artifacts in its construction. We describe a new type of non-linearity: a set of linear predictors are chosen adaptively using a non-linear selection function. We also describe how earlier families of non-linear filter banks can be extended through the use of prediction functions operating on a causal neighborhood. We present preliminary results for a synthetic test image.
Adaptive Wavelet Transforms for Image Coding using Lifting
(1998-03-01) Claypoole, Roger L.; Davis, Geoffrey; Sweldens, Wim; Baraniuk, Richard G.; Digital Signal Processing (http://dsp.rice.edu/)
Summary form only given. Image compression relies on efficient representations of images, and within smooth image regions, the wavelet transform provides such a representation. However, near edges, wavelet coefficients decay slowly and are expensive to code. We focus on improving the transform by incorporating adaptivity. Construction of nonlinear filter banks has been discussed, but the question of how to utilize the nonlinearities remained. We answer this question by describing our transform via lifting. Lifting provides a spatial domain framework for the wavelet transform. In the lifting formalism, wavelet coefficients are seen as prediction residuals from a linear prediction operation. Wavelet coefficients are large near edges because the linear predictors are built to interpolate low order polynomials. Our goal is to avoid this problem by adapting the predictor based on local image properties. In smooth regions of the image, we use high order polynomial predictors. We adaptively reduce the prediction order to avoid attempting to predict values across discontinuities.
Adaptive Wavelet Transforms via Lifting
(1998) Claypoole, Roger L.; Baraniuk, Richard G.; Nowak, Robert David; Digital Signal Processing (http://dsp.rice.edu/)
This paper develops new algorithms for adapted multiscale analysis and signal adaptive wavelet transforms. We construct our adaptive transforms with the lifting scheme, which decomposes the wavelet transform into prediction and update stages. We adapt the prediction stage to the signal structure and design the update stage to preserve the desirable properties of the wavelet transform. We incorporate this adaptivity into the redundant and non-redundant transforms; the resulting transforms are scale and spatially adaptive. We study applications to signal estimation; our new transforms show improved denoising performance over existing (non-adaptive) orthogonal transforms.
Adaptive Wavelet Transforms via Lifting
(1999-01-15) Claypoole, Roger L.; Baraniuk, Richard G.; Nowak, Robert David; Digital Signal Processing (http://dsp.rice.edu/)
This paper develops new algorithms for adapted multiscale analysis and signal adaptive wavelet transforms. We construct our adaptive transforms with the lifting scheme, which decomposes the wavelet transform into prediction and update stages. We adapt the prediction stage to the signal structure and design the update stage to preserve the desirable properties of the wavelet transform. We incorporate this adaptivity into the redundant and non-redundant transforms; the resulting transforms are scale and spatially adaptive. We study applications to signal estimation; our new transforms show improved denoising performance over existing (non-adaptive) orthogonal transforms.
Adaptive Wavelet Transforms via Lifting
(1998-05-01) Claypoole, Roger L.; Baraniuk, Richard G.; Nowak, Robert David; Digital Signal Processing (http://dsp.rice.edu/)
This paper develops two new adaptive wavelet transforms based on the lifting scheme. The lifting construction exploits a spatial-domain, prediction-error interpretation of the wavelet transform and provides a powerful framework for designing customized transforms. We use the lifting construction to adaptively tune a wavelet transform to a desired signal by optimizing data-based prediction error criteria. The performances of the new transforms are compared to existing wavelet transforms, and applications to signal denoising are investigated.
Adaptive wavelet transforms via lifting
(2000) Claypoole, Roger L., Jr; Baraniuk, Richard G.
Wavelet transforms have proven very useful for a variety of signal and image processing tasks. The wedgelet transform also shows promise for certain edge-dominated images. However, in many applications we desire to introduce adaptivity and non-linearities into the transforms. These are powerful extensions, but difficult to control within the wavelet framework. The lifting scheme provides a new, spatial intuition into the wavelet transform that simplifies the introduction of adaptivity. In this thesis, we develop several new adaptive wavelet transforms and adaptive multiresolution wedgelet transforms. The lifting construction permits control over the multiresolution properties of these transforms despite the adaptivity. We demonstrate the power of our new adaptive lifted transforms with successful applications to signal denoising and compression problems.
Adaptive Weighted Highpass Filters Using Multiscale Analysis
(1998-07-01) Nowak, Robert David; Baraniuk, Richard G.; Digital Signal Processing (http://dsp.rice.edu/)
In this paper, we propose a general framework for studying a class of weighted highpass filters. Our framework, based on a multiscale signal decomposition, allows us to study a wide class of filters and to assess the merits of each. We derive an automatic procedure to tune a filter to the local structure of the image under consideration. The entire algorithm is fully automatic and requires no parameter specification from the user. Several simulations demonstrate the efficacy of the method.
Additive and Multiplicative Mixture Trees for Network Traffic Modeling
(2002-05-01) Sarvotham, Shriram; Wang, Xuguang; Riedi, Rudolf H.; Baraniuk, Richard G.; Digital Signal Processing (http://dsp.rice.edu/)
Network traffic exhibits drastically different statistics, ranging from nearly Gaussian marginals and long range dependence at very large time scales to highly non-Gaussian marginals and multifractal scaling on small scales. This behavior can be explained by forming two components of the traffic according to the speed of connections, one component absorbing most traffic and being mostly Gaussian, the other constituting virtually all the small scale bursts. Towards a better understanding of this phenomenon, we propose a novel tree-based model which is flexible enough to accommodate Gaussian as well as bursty behavior on different scales in a parsimonious way.
An asymptotic minimax analysis of nonlocal means on edges
(2012) Narayan, Manjari; Baraniuk, Richard G.
This thesis analyzes the non-local means denoising algorithm using the criterion of minimax optimality from statistical decision theory. We show that nonlocal means is minimax suboptimal on images with smooth discontinuities [1] with a rate of convergence of [Special characters omitted.] ( n -1 ) comparable to that of wavelet thresholding. The suboptimality is a consequence of the isotropic nature of the algorithm, and its inability to adapt to the smoothness of the discontinuity. However, all is not lost for nonlocal methods. We also propose an anisotropic nonlocal means algorithm [2] that can attain the optimal rate of [Special characters omitted.] ( n -4/3 ) as well as deliver superior denoising performance using image gradients on synthetic and empirical images, respectively. Nonlocal means is an instance of exemplar based image processing methods. This result broadly implies that exemplar methods that respect anisotropy can yield superior performance in estimating edges in both theory and practice.
Analog system for computing sparse codes
(2010-08-24) Rozell, Christopher John; Johnson, Don H.; Baraniuk, Richard G.; Olshausen, Bruno A.; Ortman, Robert Lowell; Rice University; United States Patent and Trademark Office
A parallel dynamical system for computing sparse representations of data, i.e., where the data can be fully represented in terms of a small number of non-zero code elements, and for reconstructing compressively sensed images. The system is based on the principles of thresholding and local competition that solves a family of sparse approximation problems corresponding to various sparsity metrics. The system utilizes Locally Competitive Algorithms (LCAs), nodes in a population continually compete with neighboring units using (usually one-way) lateral inhibition to calculate coefficients representing an input in an over complete dictionary.
Analysis and modeling of bursty long-range-dependent network traffic
(2001) Sarvotham, Shriram; Baraniuk, Richard G.
In this thesis, we study the cause and impact of burstiness in computer network traffic. A connection-level analysis of traffic at coarse time scales (time scales greater than a round-trip-time) reveals that a single connection dominates during the period of the burst. The number of dominating connections that cause bursts is found to be a small fraction of the total number of connections. Removing the burst causing connections from the traffic yields a trace whose marginal is close to a Gaussian. This observation motivates a network traffic model comprised of two components, namely the Gaussian part and the bursty part. The Gaussian part of the traffic models the aggregate of majority of the connections, whereas the bursty part models the behavior of few dominant connections that transmit data at unusually high rates. The Gaussian component imparts long-range-dependence (LRD) to the traffic, whereas the bursty component gives rise to spikiness. We argue that heterogeneity in bottleneck link speeds gives rise to burstiness, and heavy tailed connection durations results in LRD. We perform simulations in ns to validate the proposed model and synthesize realistic traffic that is both non-Gaussian and LRD. We demonstrate the impact of the bursty component in queueing behavior. Although the bursty component constitutes a small fraction of the total traffic, it significantly affects the queueing behavior, in particular at large queue sizes.
Analysis of Multiscale Texture Segmentation using Wavelet-Domain Hidden Markov Trees
(1999-10-01) Choi, Hyeokho; Hendricks, Brent; Baraniuk, Richard G.; Digital Signal Processing (http://dsp.rice.edu/)
This paper describes a technique for estimating the Kullback-Leibler (KL) distance between two Hidden Markov Models (HMMs), and for measuring the quality of the estimator. It also provides some results based on applying the technique to wavelet domain Hidden Markov Tree (HMT) models used in image segmentation. The technique is easily applied, because in most situations the necessary tools (data generation and likelihood calculation) are already in place.
Analysis of the DCS one-stage Greedy Algorothm for Common Sparse Supports
(2005-11-01) Baron, Dror; Duarte, Marco F.; Wakin, Michael; Sarvotham, Shriram; Baraniuk, Richard G.; Digital Signal Processing (http://dsp.rice.edu/)
Analysis of the DCS one-stage Greedy Algorothm for Common Sparse Supports

Browsing by Author "Baraniuk, Richard G."

Results Per Page

Sort Options