Gradient Pursuits
Top Cited Papers
- 16 May 2008
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Signal Processing
- Vol. 56 (6), 2370-2382
- https://doi.org/10.1109/tsp.2007.916124
Abstract
Sparse signal approximations have become a fundamental tool in signal processing with wide-ranging applications from source separation to signal acquisition. The ever-growing number of possible applications and, in particular, the ever-increasing problem sizes now addressed lead to new challenges in terms of computational strategies and the development of fast and efficient algorithms has become paramount. Recently, very fast algorithms have been developed to solve convex optimization problems that are often used to approximate the sparse approximation problem; however, it has also been shown, that in certain circumstances, greedy strategies, such as orthogonal matching pursuit, can have better performance than the convex methods. In this paper, improvements to greedy strategies are proposed and algorithms are developed that approximate orthogonal matching pursuit with computational requirements more akin to matching pursuit. Three different directional optimization schemes based on the gradient, the conjugate gradient, and an approximation to the conjugate gradient are discussed, respectively. It is shown that the conjugate gradient update leads to a novel implementation of orthogonal matching pursuit, while the gradient-based approach as well as the approximate conjugate gradient methods both lead to fast approximations to orthogonal matching pursuit, with the approximate conjugate gradient method being superior to the gradient method.Keywords
This publication has 21 references indexed in Scilit:
- Signal Recovery From Random Measurements Via Orthogonal Matching PursuitIEEE Transactions on Information Theory, 2007
- Gradient Projection for Sparse Reconstruction: Application to Compressed Sensing and Other Inverse ProblemsIEEE Journal of Selected Topics in Signal Processing, 2007
- Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency informationIEEE Transactions on Information Theory, 2006
- On the exponential convergence of matching pursuits in quasi-incoherent dictionariesIEEE Transactions on Information Theory, 2005
- Sparse Bayesian Learning for Basis SelectionIEEE Transactions on Signal Processing, 2004
- Simple mixture model for sparse overcomplete ICAIEE Proceedings - Vision, Image, and Signal Processing, 2004
- An improved FOCUSS-based learning algorithm for solving sparse linear inverse problemsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2001
- 10.1162/15324430152748236Applied Physics Letters, 2000
- Sparse Approximate Solutions to Linear SystemsSIAM Journal on Computing, 1995
- Matching pursuits with time-frequency dictionariesIEEE Transactions on Signal Processing, 1993