Sparse Modeling : Theory, Algorithms, and Applications book cover
1st Edition

Sparse Modeling
Theory, Algorithms, and Applications

ISBN 9781439828694
Published December 1, 2014 by CRC Press
236 Pages 7 Color & 52 B/W Illustrations

SAVE ~ $18.19
was $90.95
USD $72.76

Prices & shipping based on shipping country


Book Description

Sparse models are particularly useful in scientific applications, such as biomarker discovery in genetic or neuroimaging data, where the interpretability of a predictive model is essential. Sparsity can also dramatically improve the cost efficiency of signal processing.

Sparse Modeling: Theory, Algorithms, and Applications provides an introduction to the growing field of sparse modeling, including application examples, problem formulations that yield sparse solutions, algorithms for finding such solutions, and recent theoretical results on sparse recovery. The book gets you up to speed on the latest sparsity-related developments and will motivate you to continue learning about the field.

The authors first present motivating examples and a high-level survey of key recent developments in sparse modeling. The book then describes optimization problems involving commonly used sparsity-enforcing tools, presents essential theoretical results, and discusses several state-of-the-art algorithms for finding sparse solutions.

The authors go on to address a variety of sparse recovery problems that extend the basic formulation to more sophisticated forms of structured sparsity and to different loss functions. They also examine a particular class of sparse graphical models and cover dictionary learning and sparse matrix factorizations.

Table of Contents

Motivating Examples
Sparse Recovery in a Nutshell
Statistical Learning versus Compressed Sensing

Sparse Recovery: Problem Formulations
Noiseless Sparse Recovery
Convexity: Brief Review
Relaxations of (P0) Problem
The Effect of lq-Regularizer on Solution Sparsity
l1-norm Minimization as Linear Programming
Noisy Sparse Recovery
A Statistical View of Sparse Recovery
Beyond LASSO: Other Loss Functions and Regularizers

Theoretical Results (Deterministic Part)
The Sampling Theorem
Surprising Empirical Results
Signal Recovery from Incomplete Frequency Information
Mutual Coherence
Spark and Uniqueness of (P0) Solution
Null Space Property and Uniqueness of (P1) Solution
Restricted Isometry Property (RIP)
Square Root Bottleneck for the Worst-Case Exact Recovery
Exact Recovery Based on RIP

Theoretical Results (Probabilistic Part)
When Does RIP Hold?
Johnson-Lindenstrauss Lemma and RIP for Subgaussian Random Matrices
Random Matrices Satisfying RIP
RIP for Matrices with Independent Bounded Rows and Matrices with Random Rows of Fourier Transform

Algorithms for Sparse Recovery Problems
Univariate Thresholding is Optimal for Orthogonal Designs
Algorithms for l0-norm Minimization
Algorithms for l1-norm Minimization (LASSO)

Beyond LASSO: Structured Sparsity
The Elastic Net
Group LASSO: l1/l2 Penalty
Simultaneous LASSO: l1/l∞ Penalty

Beyond LASSO: Other Loss Functions
Sparse Recovery from Noisy Observations
Exponential Family, GLMs, and Bregman Divergences
Sparse Recovery with GLM Regression

Sparse Graphical Models
Markov Networks
Learning and Inference in Markov Networks
Learning Sparse Gaussian MRFs

Sparse Matrix Factorization: Dictionary Learning and Beyond
Dictionary Learning
Sparse PCA
Sparse NMF for Blind Source Separation


Appendix: Mathematical Background



A Summary and Bibliographical Notes appear at the end of each chapter.

View More



Irina Rish, Genady Grabarnik


"… an excellent introductory book for branching off into aspects of sparse modeling; it is also good for advanced students since it is contains an appendix with some of the mathematical background needed to learn from this book, including topics such as eigentheory, discrete Fourier transform, and subgaussian random variables. I very much recommend this book for researchers and students alike."
Computing Reviews, May 2015

"A comprehensive, clear, and well-articulated book on sparse modeling. This book will stand as a prime reference to the research community for many years to come."
—Ricardo Vilalta, Department of Computer Science, University of Houston

"This book provides a modern introduction to sparse methods for machine learning and signal processing, with a comprehensive treatment of both theory and algorithms. Sparse Modeling is an ideal book for a first-year graduate course."
—Francis Bach, INRIA - École Normale Supérieure, Paris