Sparse Modeling: Theory, Algorithms, and Applications, 1st Edition (Hardback) book cover

Sparse Modeling

Theory, Algorithms, and Applications, 1st Edition

By Irina Rish, Genady Grabarnik

CRC Press

236 pages | 7 Color Illus. | 52 B/W Illus.

Purchasing Options:$ = USD
Hardback: 9781439828694
pub: 2014-12-01
SAVE ~$17.39
eBook (VitalSource) : 9780429151996
pub: 2014-12-01
from $43.48

FREE Standard Shipping!


Sparse models are particularly useful in scientific applications, such as biomarker discovery in genetic or neuroimaging data, where the interpretability of a predictive model is essential. Sparsity can also dramatically improve the cost efficiency of signal processing.

Sparse Modeling: Theory, Algorithms, and Applications provides an introduction to the growing field of sparse modeling, including application examples, problem formulations that yield sparse solutions, algorithms for finding such solutions, and recent theoretical results on sparse recovery. The book gets you up to speed on the latest sparsity-related developments and will motivate you to continue learning about the field.

The authors first present motivating examples and a high-level survey of key recent developments in sparse modeling. The book then describes optimization problems involving commonly used sparsity-enforcing tools, presents essential theoretical results, and discusses several state-of-the-art algorithms for finding sparse solutions.

The authors go on to address a variety of sparse recovery problems that extend the basic formulation to more sophisticated forms of structured sparsity and to different loss functions. They also examine a particular class of sparse graphical models and cover dictionary learning and sparse matrix factorizations.


"… an excellent introductory book for branching off into aspects of sparse modeling; it is also good for advanced students since it is contains an appendix with some of the mathematical background needed to learn from this book, including topics such as eigentheory, discrete Fourier transform, and subgaussian random variables. I very much recommend this book for researchers and students alike."

Computing Reviews, May 2015

"A comprehensive, clear, and well-articulated book on sparse modeling. This book will stand as a prime reference to the research community for many years to come."

—Ricardo Vilalta, Department of Computer Science, University of Houston

"This book provides a modern introduction to sparse methods for machine learning and signal processing, with a comprehensive treatment of both theory and algorithms. Sparse Modeling is an ideal book for a first-year graduate course."

—Francis Bach, INRIA - École Normale Supérieure, Paris

Table of Contents


Motivating Examples

Sparse Recovery in a Nutshell

Statistical Learning versus Compressed Sensing

Sparse Recovery: Problem Formulations

Noiseless Sparse Recovery


Convexity: Brief Review

Relaxations of (P0) Problem

The Effect of lq-Regularizer on Solution Sparsity

l1-norm Minimization as Linear Programming

Noisy Sparse Recovery

A Statistical View of Sparse Recovery

Beyond LASSO: Other Loss Functions and Regularizers

Theoretical Results (Deterministic Part)

The Sampling Theorem

Surprising Empirical Results

Signal Recovery from Incomplete Frequency Information

Mutual Coherence

Spark and Uniqueness of (P0) Solution

Null Space Property and Uniqueness of (P1) Solution

Restricted Isometry Property (RIP)

Square Root Bottleneck for the Worst-Case Exact Recovery

Exact Recovery Based on RIP

Theoretical Results (Probabilistic Part)

When Does RIP Hold?

Johnson-Lindenstrauss Lemma and RIP for Subgaussian Random Matrices

Random Matrices Satisfying RIP

RIP for Matrices with Independent Bounded Rows and Matrices with Random Rows of Fourier Transform

Algorithms for Sparse Recovery Problems

Univariate Thresholding is Optimal for Orthogonal Designs

Algorithms for l0-norm Minimization

Algorithms for l1-norm Minimization (LASSO)

Beyond LASSO: Structured Sparsity

The Elastic Net


Group LASSO: l1/l2 Penalty

Simultaneous LASSO: l1/l∞ Penalty



Beyond LASSO: Other Loss Functions

Sparse Recovery from Noisy Observations

Exponential Family, GLMs, and Bregman Divergences

Sparse Recovery with GLM Regression

Sparse Graphical Models


Markov Networks

Learning and Inference in Markov Networks

Learning Sparse Gaussian MRFs

Sparse Matrix Factorization: Dictionary Learning and Beyond

Dictionary Learning

Sparse PCA

Sparse NMF for Blind Source Separation


Appendix: Mathematical Background



A Summary and Bibliographical Notes appear at the end of each chapter.

About the Series

Chapman & Hall/CRC Machine Learning & Pattern Recognition

Learn more…

Subject Categories

BISAC Subject Codes/Headings:
COMPUTERS / Machine Theory