Recursive Least Squares Dictionary Learning Algorithm
Top Cited Papers
- 15 January 2010
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Signal Processing
- Vol. 58 (4), 2121-2130
- https://doi.org/10.1109/tsp.2010.2040671
Abstract
We present the recursive least squares dictionary learning algorithm, RLS-DLA, which can be used for learning overcomplete dictionaries for sparse signal representation. Most DLAs presented earlier, for example ILS-DLA and K-SVD, update the dictionary after a batch of training vectors has been processed, usually using the whole set of training vectors as one batch. The training set is used iteratively to gradually improve the dictionary. The approach in RLS-DLA is a continuous update of the dictionary as each training vector is being processed. The core of the algorithm is compact and can be effectively implemented. The algorithm is derived very much along the same path as the recursive least squares (RLS) algorithm for adaptive filtering. Thus, as in RLS, a forgetting factor ¿ can be introduced and easily implemented in the algorithm. Adjusting ¿ in an appropriate way makes the algorithm less dependent on the initial dictionary and it improves both convergence properties of RLS-DLA as well as the representation ability of the resulting dictionary. Two sets of experiments are done to test different methods for learning dictionaries. The goal of the first set is to explore some basic properties of the algorithm in a simple setup, and for the second set it is the reconstruction of a true underlying dictionary. The first experiment confirms the conjectural properties from the derivation part, while the second demonstrates excellent performance.Keywords
This publication has 22 references indexed in Scilit:
- $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse RepresentationIEEE Transactions on Signal Processing, 2006
- On the uniqueness of overcomplete dictionaries, and a practical way to retrieve themLinear Algebra and its Applications, 2006
- General design algorithm for sparse frame expansionsSignal Processing, 2006
- A simple design of sparse signal representations using overlapping framesPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2002
- Method of optimal directions for frame designPublished by Institute of Electrical and Electronics Engineers (IEEE) ,1999
- Ten Lectures on WaveletsPublished by Society for Industrial & Applied Mathematics (SIAM) ,1992
- Least squares quantization in PCMIEEE Transactions on Information Theory, 1982
- An Algorithm for Vector Quantizer DesignIEEE Transactions on Communications, 1980
- Note on GroupingJournal of the American Statistical Association, 1957
- Algorithms for the Assignment and Transportation ProblemsJournal of the Society for Industrial and Applied Mathematics, 1957