R-3 Repository :: Browsing by Author "Tapia, Richard"

Browsing by Author "Tapia, Richard"

Now showing 1 - 15 of 15

A Fast Newton's Algorithm for Entropy Maximization in Phase Determination
(1999-05) Wu, Zhijun; Phillips, George; Tapia, Richard; Zhang, Yin
A long-standing problem in X-ray crystallography, known as the phase problem, is to determine the phases for a large set of complex variables, called the structure factors of the crystal, given their magnitudes obtained from X-ray diffraction experiments. We introduce a statistical phase estimation approach to the problem. This approach requires solving a special class of entropy maximization problems repeatedly to obtain the joint probability distribution of the structure factors. The entropy maximization problem is a semi-infinite convex program, which can be solved in a finite dual space by using a standard Newton's method. The Newton's method converges quadratically, but is costly in general, requiring O(n log n) floating point operations in every iteration, where n is the number of variables. We present a fast Newton's algorithm for solving the entropy maximization problem. The algorithm requires only O(n log n) floating point operations for each of its iterates, yet has the same convergence rate as the standard Newton. We describe the algorithm and discuss related computational issues. Numerical results on simple test cases will also be presented to demonstrate the behavior of the algorithm.
A Spectrum-based Regularization Approach to Linear Inverse Problems: Models, Learned Parameters and Algorithms
(2015-04-20) Castanon, Jorge Castanon Alberto; Zhang, Yin; Tapia, Richard; Hand, Paul; Kelly, Kevin
In this thesis, we study the problem of recovering signals, in particular images, that approximately satisfy severely ill-conditioned or underdetermined linear systems. For example, such a linear system may represent a set of under-sampled and noisy linear measurements. It is well-known that the quality of the recovery critically depends on the choice of an appropriate regularization model that incorporates prior information about the target solution. Two of the most successful regularization models are the Tikhonov and Total Variation (TV) models, each of which is used in a wide range of applications. We design and investigate a class of spectrum-based models that generalize and improve upon both the Tikhonov and the TV methods, as well as their combinations or so-called hybrids. The proposed models contain "spectrum parameters" that are learned from training data sets through solving optimization problems. This parameter-learning feature gives these models the flexibility to adapt to desired target solutions. We devise efficient algorithms for all the proposed models and conduct comprehensive numerical experiments to evaluate their performance as compared to established models. Numerical results show a generally superior quality in recovered images by our approach from under-sampled linear measurements. Using the proposed algorithms, one can often obtain much improved quality at a moderate increase in computational time.
A Superlinearly Convergent Polynomial Primal-Dual Interior-Point Algorithm for Linear Programming
(1991-02) Zhang, Yin; Tapia, Richard
The choice of the centering (or barrier) parameter and the step length parameter are the fundamental issues in primal-dual interior-point algorithms for linear programming. Various choices for these two parameters have been proposed that lead to polynomial algorithms. Recently, Zhang, Tapia and Dennis gave conditions that these choices must satisfy in order to achieve quadratic or superlinear convergence. However, it has not been shown that these conditions for fast convergence are compatible with the choices that lead to polynomiality. It is worth noting that none of the existing polynomial algorithms satisfies these fast convergence requirements. This paper gives an affirmative answer to the question: can an algorithm be both polynomial and superlinearly convergent? We construct and analyze a "large step" algorithm that possesses both polynomiality and Q-superlinear convergence. For nondegenerate problems, the convergence rate is actually Q-quadratic.
An Interior-Point Krylov-Orthogonal Projection Method for Nonlinear Programming
(1997-06) Argáez, Miguel; Klíe, Héctor; Ramé, Marcelo; Tapia, Richard
In this work we consider an inexact Newton's method implementation of the primal-dual interior-point algorithm for solving general nonlinear programming problems recently introduced by Argáez and Tapia. This inexact method is designed to solve large scale problems. The iterative solution technique uses an orthogonal projection - Krylov subspace scheme to solve the highly indefinite and nonsymmetric linear systems associated with nonlinear programming. Our iterative method is a projection method that maintains linearized feasibility with respect to both the equality constraints and the complementarity conditions. This guarantees that in each iteration the linear solver generates a descent direction, so that the iterative solver is not required to find a Newton step but rather cheaply provides a way to march toward an optimal solution of the problem. This makes the use of a preconditioner inconsequential except near the solution of the problem, where the Newton step is effective. Moreover, we limit the problem to finding a good preconditioner only for the Hessian of the Lagrangian function associated with the problem plus a positive diagonal matrix. We report numerical experimentation for several large scale problems to illustrate the viability of the method.
An Interior-Point Method with Polynomial Complexity and Superlinear Convergence for Linear Complementarity Problems
(1991-07) Ji, Jun; Potra, Florian; Tapia, Richard; Zhang, Yin
For linear programming, a primal-dual interior-point algorithm was recently constructed by Zhang and Tapia that achieves both polynomial complexity and Q-superlinear convergence (Q-quadratic in the nondegenerate case). In this paper, we extend their results to quadratic programming and linear complementarity problems.
Block Coordinate Update Method in Tensor Optimization
(2014-08-19) Xu, Yangyang; Yin, Wotao; Zhang, Yin; Allen, Genevera; Tapia, Richard
Block alternating minimization (BAM) has been popularly used since the 50's of last century. It partitions the variables into disjoint blocks and cyclically updates the blocks by minimizing the objective with respect to each block of variables, one at a time with all others fixed. A special case is the alternating projection method to find a common point of two convex sets. The BAM method is often easy yet efficient particularly if each block subproblem is simple to solve. However, for certain problems such as the nonnegative tensor decomposition, the block subproblems can be difficult to solve, or even if they are solved exactly or to high accuracies, BAM can perform badly on solving the original problem, in particular on non-convex problems. On the other hand, in the literature, the BAM method is mainly analyzed for convex problems. Although it has been shown numerically to work well on many non-convex problems, theoretical results of BAM for non-convex optimization are still lacked. For these reasons, I propose different block update schemes and generalize the BAM method for non-smooth non-convex optimization problems. Which scheme is the most efficient depends on specific applications. In addition, I analyze convergence of the generalized method, dubbed as block coordinate update method (BCU), with different block update schemes for non-smooth optimization problems, in both convex and non-convex cases. BCU has found many applications, and the work in this dissertation is mainly motivated by tensor optimization problems, for which the BCU method is often the best choice due to their block convexity. I make contributions in modeling, algorithm design, and also theoretical analysis. The first part is about the low-rank tensor completion, for which I make a novel model based on parallel low-rank matrix factorization. The new model is non-convex, and it is difficult to guarantee global optimal solutions. However, the BAM method performs very well on solving this model. Global convergence in terms of KKT conditions is established, and numerical experiments demonstrate the superiority of the proposed model over several state-of-the-art ones. The second part is towards the solution of the nonnegative tensor decomposition. For this problem, each block subproblem is a nonnegative least squares problem and not simple to solve. Hence, the BAM method may be inefficient. I propose a block proximal gradient (BPG) method. In contrast to BAM that solves each block subproblem exactly, BPG solves relaxed block subproblems, which are often much simpler than the original ones and can thus make BPG converge faster. Through the Kurdyka-Lojasiewicz property, I establish its global convergence with rate estimate in terms of iterate sequence. Numerical experiments on sparse nonnegative Tucker decomposition demonstrates its superiority over the BAM method. The last part is motivated by tensor regression problems, whose block partial gradient is expensive to evaluate. For such problems, BPG becomes inefficient, and I propose to use inexact partial gradient and generalize BPG to a block stochastic gradient method. Convergence results in expectation are established for general non-convex case in terms of first-order optimality conditions, and for convex case, a sublinear convergence rate result is shown. Numerical tests on tensor regression problems show that the block stochastic gradient method significantly outperforms its deterministic counterpart.
Generalizations of the Alternating Direction Method of Multipliers for Large-Scale and Distributed Optimization
(2014-11-19) Deng, Wei; Zhang, Yin; Yin, Wotao; Jermaine, Christopher; Tapia, Richard
Due to the dramatically increasing demand for dealing with "Big Data", efficient and scalable computational methods are highly desirable to cope with the size of the data. The alternating direction method of multipliers (ADMM), as a versatile algorithmic tool, has proven to be very effective at solving many large-scale and structured optimization problems, particularly arising from the areas of compressive sensing, signal and image processing, machine learning and applied statistics. Moreover, the algorithm can be implemented in a fully parallel and distributed manner to process huge datasets. These benefits have mainly contributed to the recent renaissance of ADMM for modern applications. This thesis makes important generalizations to ADMM to improve its flexibility and efficiency, as well as extending its convergence theory. Firstly, we allow more options of solving the subproblems either exactly or approximately, such as linearizing the subproblems, taking one gradient descent step, and approximating the Hessian. Often, when subproblems are expensive to solve exactly, it is much cheaper to compute approximate solutions to the subproblems which are still good enough to guarantee convergence. Although it may take more iterations to converge due to less accurate subproblems, the entire algorithm runs faster since each iteration takes much less time. Secondly, we establish the global convergence of these generalizations of ADMM. We further show the linear convergence rate under a variety of scenarios, which cover a wide range of applications in practice. Among these scenarios, we require that at least one of the two objective functions is strictly convex and has Lipschitz continuous gradient, along with certain full rank conditions on the constraint coefficient matrices. The derived rate of convergence also provides some theoretical guidance for optimizing the parameters of the algorithm. In addition, we introduce a simple technique to improve an existing convergence rate from O(1/k) to o(1/k). Thirdly, we introduce a parallel and multi-block extension to ADMM for solving convex separable problems with N blocks of variables. The algorithm decomposes the original problem into N smaller subproblems and solves them in parallel at each iteration. It is well suited to distributed computing and is particularly attractive for solving certain large-scale problems. We show that extending ADMM straightforwardly from the classic Gauss-Seidel setting to the Jacobi setting, from 2 blocks to N blocks, will preserve convergence if the constraint coefficient matrices are mutually near-orthogonal and have full column-rank. For general cases, we propose to add proximal terms of different kinds to the N subproblems so that they can be solved in flexible and efficient ways and the algorithm converges globally at a rate of o(1/k). We introduce a strategy for dynamically tuning the parameters of the algorithm, often leading to substantial acceleration of the convergence in practice. Numerical results are presented to demonstrate the efficiency of the proposed algorithm in comparison with several existing parallel algorithms. We also implemented our algorithm on Amazon EC2, an on-demand public computing cloud, and report its performance on very large-scale basis pursuit problems with distributed data.
On Convergence of Minimization Methods: Attraction, Repulsion and Selection
(1999-03) Zhang, Yin; Tapia, Richard; Velazquez, Leticia
In this paper, we introduce a rather straightforward but fundamental observation concerning the convergence of the general iteration process. x^(k+1) = x^k - alpha(x^k) [B(x^k)]^(-1) gradf(x^k) for minimizing a function f(x). We give necessary and sufficient conditions for a stationary point of f(x) to be a point of strong attraction of the iteration process. We will discuss various ramifications of this fundamental result, particularly for nonlinear least squares problems.
On Secant Updates for Use in General Constrained Optimization
(1984-09) Tapia, Richard
In this paper we present two new classes of SQP secant methods for the equality constrained optimization problem. One class of methods uses the SQP augmented Lagrangian formulation, while the other class uses the SQP Lagrangian formulation. We demonstrate, under the standard assumptions, that in both cases the BFGS and DFP versions of the algorithm are locally q-superlinearly convergent. To our knowledge this is the first time that either local or q-superlinear convergence has been established for an SQP Lagrangian secant method which uses either the BFGS or DFP updating philosophy and assumes no more than the standard assumptions. Since the standard assumptions do not require positive definiteness of the Hessian of the Lagrangian at the solution, it is no surprise that our BFGS and DFP updates possess the hereditary positive definiteness property only on a proper subspace.
Radial MILO: A 4D Image Registration Algorithm Based on Filtering Block Match Data via l1-minimization
(2015-04-21) Vargas, Arturo; Zhang, Yin; Castillo, Edward; Tapia, Richard; Warburton, Tim
Minimal l1 Perturbation to Block Match Data (MILO) is a spatially accurate image registration algorithm developed for thoracic CT inhale/exhale images. The MILO algorithm consists of three components: (1) creating an initial estimate for voxel displacement via a Mutual Minimizing Block Matching Algorithm (MMBM), (2) a filtering step based on l1 minimization and a uniform B-spline parameterization, and (3) recovering a full displacement field based on the filtered estimates. This thesis presents a variation of MILO for 4DCT images. In practice, the use of uniform B-splines has led to rank deficient linear systems due to the spline's inability to conform to non-structured MMBM estimates. In order to adaptively conform to the data an octree is paired with radial functions. The l1 minimization problem had previously been addressed by employing QR factorization, which required substantial storage. As an alternative a block coordinate descent algorithm is employed, relieving the need for QR factorization. Furthermore, by modeling voxel trajectories as quadratic functions in time, the proposed method is able to register multiple images.
Selective Search for Global Optimization of Zero or Small Residual Least-Squares Problems: A Numerical Study
(1999-09) Velazquez, Leticia; Phillips, George; Tapia, Richard; Zhang,Yin
In this paper, we consider searching for global minima of zero or small residual, nonlinear least-squares problems. We propose a selective search approach based on the concept of selective minimization recently introduced in Zhang et al[14]. To test the viability of the proposed approach, we construct a simple implementation using a Levenberg-Marquardt type method combined with a multi-start scheme, and compare it with several existing global optimization techniques. Numerical experiments were performed on zero residual nonlinear least-squares problem chosen from structural biology applications as well as from the literature. On the problems of larger sizes, the performance of the new approach compared favorably with the other tested methods, indicating that the new approach is promising for the intended class of problems.
Some numerical experiments with the diagonalized Newton multiplier method on geometric programming problems with equality constraints
(1977) Lucas, Kristin Rose; Tapia, Richard
The diagonalized Newton multiplier method is applied to the problem of minimizing a posynomial objective function subject to posynomial equality constraints. Two transformations of the original problem are made and their relative merits examined. Two example problems are solved numerically, in their original formulation and also in their transformed versions. Our numerical experiments with the diagonalized Newton multiplier method seem to suggest that, for this class of problems, solving a transformed version yields a larger region of convergence.
The Bayesian Statistical Approach to the Phase Problem in Protein X-ray Crystallography
(1999-04) Wu, Zhijun; Phillips, George; Tapia, Richard; Zhang, Yin
We review a Bayesian statistical approach to the phase problem in protein X-ray crystallography. We discuss the mathematical foundations and the computational issues. The introduction to the theory and the algorithms does not require strong background in X-ray crystallography and related physical disciplines.
The Behavior of Newton-Type Methods on Two Equivalent Systems from Linear Programming
(1998-02) Villalobos, Cristina; Tapia, Richard; Zhang, Yin
Newton-type methods are fundamental techniques for solving optimization problems. However, it is often not fully appreciated that these methods can produce significantly different behavior when applied to two equivalent systems. In this paper, we investigate differences in local and global behavior of Newton-type methods when applied to the first-order optimality conditions for the logarithmic barrier formulation of the linear programming problem, and when applied to the perturbed first-order optimality conditions for the linear programming problem. Through theoretical analysis and numerical results, we show that Newton-type methods perform more effectively on the latter system than on the former system.
The Sphere of Convergence of Newton's Method on Two Equivalent Systems from Nonlinear Programming
(1999-04) Villalobos, Cristina; Tapia, Richard; Zhang, Yin
We study a local feature of a Newton logarithmic barrier function method and a Newton primal-dual interior-point method. In particular, we study the radius of the sphere of convergence of Newton's method on two equivalent systems associated with the two aforementioned interior-point methods for nondegenerate problems in inequality contrained optimization problems. Our theoretical and numerical results are clearly in favor of using Newton primal-dual methods for solving the optimization problem. This work is an extension of the authors' earlier work [10] on linear programming problems.

Browsing by Author "Tapia, Richard"

Results Per Page

Sort Options