An improved FOCUSS-based learning algorithm for solving sparse linear inverse problems
- 1 January 2001
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- Vol. 1 (10586393), 347-351 vol.1
- https://doi.org/10.1109/acssc.2001.986949
Abstract
We develop an improved algorithm for solving blind sparse linear inverse problems where both the dictionary (possibly overcomplete) and the sources are unknown. The algorithm is derived in the Bayesian framework by the maximum a posteriori method, with the choice of prior distribution restricted to the class of concave/Schur-concave functions, which has been shown previously to be a sufficient condition for sparse solutions. This formulation leads to a constrained and regularized minimization problem which can be solved in part using the FOCUSS (focal underdetermined system solver) algorithm for vector selection. We introduce three key improvements in the algorithm: an efficient way of adjusting the regularization parameter; column normalization that restricts the learned dictionary; reinitialization to escape from local optima. Experiments were performed using synthetic data with matrix sizes up to 64/spl times/128; the algorithm solves the blind identification problem, recovering both the dictionary and the sparse sources. The improved algorithm is much more accurate than the original FOCUSS-dictionary learning algorithm when using large matrices. We also test our algorithm on natural images, and show that a learned overcomplete representation can encode the data more efficiently than a complete basis at the same level of accuracy.Keywords
This publication has 10 references indexed in Scilit:
- A fast algorithm for estimating overcomplete ICA bases for image windowsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2003
- Basis selection in the presence of noisePublished by Institute of Electrical and Electronics Engineers (IEEE) ,2002
- Resolution, separation of retinal ganglion cells, and cortical magnification in humansVision Research, 2001
- FOCUSS-based dictionary learning algorithmsPublished by SPIE-Intl Soc Optical Eng ,2000
- Independence: a new criterion for the analysis of the electromagnetic fields in the global brain?Neural Networks, 2000
- Probabilistic framework for the adaptation and comparison of image codesJournal of the Optical Society of America A, 1999
- Independent Factor AnalysisNeural Computation, 1999
- An affine scaling methodology for best basis selectionIEEE Transactions on Signal Processing, 1999
- Feature Selection via Mathematical ProgrammingINFORMS Journal on Computing, 1998
- Neuromagnetic source imaging with FOCUSS: a recursive weighted minimum norm algorithmElectroencephalography and Clinical Neurophysiology, 1995