Sparse models are particularly useful in scientific applications, such as biomarker discovery in genetic or neuroimaging data, where the interpretability of a predictive model is essential. Sparsity can also dramatically improve the cost efficiency of signal processing.
Sparse Modeling: Theory, Algorithms, and Applications provides an introduction to the growing field of sparse modeling, including application examples, problem formulations that yield sparse solutions, algorithms for finding such solutions, and recent theoretical results on sparse recovery. The book gets you up to speed on the latest sparsity-related developments and will motivate you to continue learning about the field.
The authors first present motivating examples and a high-level survey of key recent developments in sparse modeling. The book then describes optimization problems involving commonly used sparsity-enforcing tools, presents essential theoretical results, and discusses several state-of-the-art algorithms for finding sparse solutions.
The authors go on to address a variety of sparse recovery problems that extend the basic formulation to more sophisticated forms of structured sparsity and to different loss functions. They also examine a particular class of sparse graphical models and cover dictionary learning and sparse matrix factorizations.
Table of Contents
Sparse Recovery in a Nutshell
Statistical Learning versus Compressed Sensing
Sparse Recovery: Problem Formulations
Noiseless Sparse Recovery
Convexity: Brief Review
Relaxations of (P0) Problem
The Effect of lq-Regularizer on Solution Sparsity
l1-norm Minimization as Linear Programming
Noisy Sparse Recovery
A Statistical View of Sparse Recovery
Beyond LASSO: Other Loss Functions and Regularizers
Theoretical Results (Deterministic Part)
The Sampling Theorem
Surprising Empirical Results
Signal Recovery from Incomplete Frequency Information
Spark and Uniqueness of (P0) Solution
Null Space Property and Uniqueness of (P1) Solution
Restricted Isometry Property (RIP)
Square Root Bottleneck for the Worst-Case Exact Recovery
Exact Recovery Based on RIP
Theoretical Results (Probabilistic Part)
When Does RIP Hold?
Johnson-Lindenstrauss Lemma and RIP for Subgaussian Random Matrices
Random Matrices Satisfying RIP
RIP for Matrices with Independent Bounded Rows and Matrices with Random Rows of Fourier Transform
Algorithms for Sparse Recovery Problems
Univariate Thresholding is Optimal for Orthogonal Designs
Algorithms for l0-norm Minimization
Algorithms for l1-norm Minimization (LASSO)
Beyond LASSO: Structured Sparsity
The Elastic Net
Group LASSO: l1/l2 Penalty
Simultaneous LASSO: l1/l∞ Penalty
Beyond LASSO: Other Loss Functions
Sparse Recovery from Noisy Observations
Exponential Family, GLMs, and Bregman Divergences
Sparse Recovery with GLM Regression
Sparse Graphical Models
Learning and Inference in Markov Networks
Learning Sparse Gaussian MRFs
Sparse Matrix Factorization: Dictionary Learning and Beyond
Sparse NMF for Blind Source Separation
Appendix: Mathematical Background
A Summary and Bibliographical Notes appear at the end of each chapter.
Irina Rish, Genady Grabarnik
"… an excellent introductory book for branching off into aspects of sparse modeling; it is also good for advanced students since it is contains an appendix with some of the mathematical background needed to learn from this book, including topics such as eigentheory, discrete Fourier transform, and subgaussian random variables. I very much recommend this book for researchers and students alike."
—Computing Reviews, May 2015
"A comprehensive, clear, and well-articulated book on sparse modeling. This book will stand as a prime reference to the research community for many years to come."
—Ricardo Vilalta, Department of Computer Science, University of Houston
"This book provides a modern introduction to sparse methods for machine learning and signal processing, with a comprehensive treatment of both theory and algorithms. Sparse Modeling is an ideal book for a first-year graduate course."
—Francis Bach, INRIA - École Normale Supérieure, Paris