Models in signal processing often deal with some notion of structure or conciseness suggesting that a signal really has "few degrees of freedom" relative to its actual size. Examples include: bandlimited signals, images containing low-dimensional geometric features, or collections of signals observed from multiple viewpoints in a camera or sensor network. In many cases, such signals can be expressed as sparse linear combinations of elements from some dictionary---the sparsity of the representation directly reflects the conciseness of the model and permits efficient algorithms for signal processing. Sparsity also forms the core of the emerging theory of Compressed Sensing (CS), which states that a sparse signal can be recovered from a small number of random linear measurements. In other cases, however, sparse representations may not suffice to truly capture the underlying structure of a signal. Instead, the conciseness of the signal model may in fact dictate that the signal class forms a low-dimensional manifold as a subset of the high-dimensional ambient signal space. To date, the importance and utility of manifolds for signal processing has been acknowledged largely through a research effort into "learning" manifold structure from a collection of data points. While these methods have proved effective for certain tasks (such as classification and recognition), they also tend to be quite generic and fail to consider the geometric nuances of specific signal classes. The purpose of this thesis is to develop new methods and understanding for signal processing based on low-dimensional signal models, with a particular focus on the role of geometry. Our key contributions include (i) new models for low-dimensional signal structure, including local parametric models for piecewise smooth signals and joint sparsity models for signal collections; (ii) multiscale representations for piecewise smooth signals designed to accommodate efficient processing; (iii) insight and analysis into the geometry of low-dimensional signal models, including the non-differentiability of certain articulated image manifolds and the behavior of signal manifolds under random low-dimensional projections, and (iv) dimensionality reduction algorithms for image approximation and compression, distributed (multi-signal) CS, parameter estimation, manifold learning, and manifold-based CS.