# Large Covariance and Autocovariance Matrices

## Preview

## Book Description

**Large Covariance and Autocovariance Matrices** brings together a collection of recent results on sample covariance and autocovariance matrices in high-dimensional models and novel ideas on how to use them for statistical inference in one or more high-dimensional time series models. The prerequisites include knowledge of elementary multivariate analysis, basic time series analysis and basic results in stochastic convergence.

Part I is on different methods of estimation of large covariance matrices and auto-covariance matrices and properties of these estimators. Part II covers the relevant material on random matrix theory and non-commutative probability. Part III provides results on limit spectra and asymptotic normality of traces of symmetric matrix polynomial functions of sample auto-covariance matrices in high-dimensional linear time series models. These are used to develop graphical and significance tests for different hypotheses involving one or more independent high-dimensional linear time series.

The book should be of interest to people in econometrics and statistics (large covariance matrices and high-dimensional time series), mathematics (random matrices and free probability) and computer science (wireless communication). Parts of it can be used in post-graduate courses on high-dimensional statistical inference, high-dimensional random matrices and high-dimensional time series models. It should be particularly attractive to researchers developing statistical methods in high-dimensional time series models.

**Arup Bose** is a professor at the Indian Statistical Institute, Kolkata, India. He is a distinguished researcher in mathematical statistics and has been working in high-dimensional random matrices for the last fifteen years. He has been editor of *Sankhyā* for several years and has been on the editorial board of several other journals. He is a Fellow of the Institute of Mathematical Statistics, USA and all three national science academies of India, as well as the recipient of the S.S. Bhatnagar Award and the C.R. Rao Award. His first book *Patterned Random Matrices* was also published by Chapman & Hall. He has a forthcoming graduate text *U-statistics, M-estimates and Resampling* (with Snigdhansu Chatterjee) to be published by Hindustan Book Agency.

**Monika Bhattacharjee** is a post-doctoral fellow at the Informatics Institute, University of Florida. After graduating from St. Xavier's College, Kolkata, she obtained her master’s in 2012 and PhD in 2016 from the Indian Statistical Institute. Her thesis in high-dimensional covariance and auto-covariance matrices, written under the supervision of Dr. Bose, has received high acclaim.

## Table of Contents

**LARGE COVARIANCE MATRIX** I

Consistency

Covariance classes and regularization

Covariance classes

Covariance regularization

Bandable Σ*p *

Parameter space

Estimation in U

Minimaxity

Toeplitz Σ*p*

Parameter space

Estimation in G*β* (*M*) or F*β* (*M*0, *M*)

Minimaxity

Sparse Σ*p*

Parameter space

Estimation in Uτ (*q*, C0(*p*), *M* ) or G*q *(C*n,p*)

Minimaxity

LARGE COVARIANCE MATRIX II

Bandable Σ*p *

Models and examples

Weak dependence

Estimation

Sparse Σ*p *

LARGE AUTOCOVARIANCE MATRIX III

Models and examples

Estimation of Γ0,*p*

Estimation of Γu,*p *

Parameter spaces

Estimation

Estimation in MA(*r*)

Estimation in IVAR(*r*)

Gaussian assumption

Simulations

SPECTRAL DISTRIBUTION

LSD

Moment method

Method of Stieltjes transform

Wigner matrix: semi-circle law

Independent matrix: Marčenko -Pastur law

Results on Z: *p/n* → y > 0

Results on Z: *p/n* → 0

NON-COMMUTATIVE PROBABILITY

NCP and its convergence

Essentials of partition theory

Möbius function

Partition and non-crossing partition

Kreweras complement

Free cumulant; free independence

Moments of free variables

Joint convergence of random matrices

Compound free Poisson

GENERALIZED COVARIANCE MATRIX I

Preliminaries

Assumptions

Embedding

NCP convergence

Main idea

Main convergence

LSD of symmetric polynomials

Stieltjes transform

Corollaries

GENERALIZED COVARIANCE MATRIX II

Preliminaries

Assumptions

Centering and Scaling

Main idea

NCP convergence

LSD of symmetric polynomials

Stieltjes transform

Corollaries

**SPECTRA OF AUTOCOVARIANCE MATRIX** I

Assumptions

LSD when *p/n* → y ∈ (0, ∞)

MA*(q), q* < ∞

MA(∞)

Application to specific cases

LSD when *p/n* → 0

Application to specific cases

Non-symmetric polynomials

SPECTRA OF AUTOCOVARIANCE MATRIX II

Assumptions

LSD when *p/n* → y ∈ (0, ∞)

MA(*q*), *q *< ∞

MA(∞)

LSD when *p/n* → 0

MA*(q), q* < ∞

MA(∞)

GRAPHICAL INFERENCE

MA order determination

AR order determination

Graphical tests for parameter matrices

TESTING WITH TRACE

One sample trace

Two sample trace

Testing

SUPPLEMENTARY PROOFS

Proof of Lemma

Proof of Theorem (a)

Proof of Theorem

Proof of Lemma

Proof of Corollary (c)

Proof of Corollary (c)

Proof of Corollary (c)

Proof of Lemma

Proof of Lemma

Lemmas for Theorem

## Author(s)

### Biography

**Arup Bose** is a professor at the Indian Statistical Institute, Kolkata, India. He is a distinguished researcher in mathematical statistics and has been working in high-dimensional random matrices for the last fifteen years. He has been editor of *Sankhyā* for several years and has been on the editorial board of several other journals. He is a Fellow of the Institute of Mathematical Statistics, USA and all three national science academies of India, as well as the recipient of the S.S. Bhatnagar Award and the C.R. Rao Award. His first book *Patterned Random Matrices* was also published by Chapman & Hall. He has a forthcoming graduate text *U-statistics, M-estimates and Resampling* (with Snigdhansu Chatterjee) to be published by Hindustan Book Agency.

**Monika Bhattacharjee** is a post-doctoral fellow at the Informatics Institute, University of Florida. After graduating from St. Xavier's College, Kolkata, she obtained her master’s in 2012 and PhD in 2016 from the Indian Statistical Institute. Her thesis in high-dimensional covariance and auto-covariance matrices, written under the supervision of Dr. Bose, has received high acclaim.

## Reviews

" . . . the authors should be congratulated for producing two highly relevant and well-written books. Statisticians would probably gravitate to LCAM in the first instance and those working in linear algebra would probably gravitate to PRM."

~Jonathan Gillard, Cardiff University"The book represents a monograph of the authors’ recent results about the theory of large covariance and autocovariance matrices and contains other important results from other research papers and books in this topic. It is very useful for all researchers who use large covariance and autocovariance matrices in their researches. Especially, it is very useful for post-graduate and PhD students in mathematics, statistics, econometrics and computer science. It is a well-written and organized book with a large number of solved examples and many exercises left to readers for homework. I would like to recommend the book to PhD students and researchers who want to learn or use large covariance and autocovariance matrices in their researches."

~ Miroslav M. Ristić (Niš), zbMath"This book brings together a collection of recent results on estimation of multidimensional time series covariance matrices. In the case where the time series consists of a sequence of independent (Chapter 1) or weakly dependent (Chapter 2) random vectors, the authors call it covariance estimation, whereas in the general case where the time series is only stationary, they call it autocovariance estimation. The framework of the results presented here is the one where the dimension of the observations (as well as the observation window size, otherwise nothing can be said) is high. The prerequisites include knowledge of elementary multivariate analysis, basic time series analysis, and basic results in stochastic convergence.

In Chapter 1, the authors consider the case where we have at our disposal a large time series of iid high-dimensional observations with common covariance matrix C and want to estimate C. They provide some results on how to regularize the empirical sample covariance matrix in order to accurately estimate C in the case where C is either quickly decaying away from its diagonal (or \bendable"), Toeplitz or sparse. The regularization techniques involved are the \banding" (zero-out of all entries above a certain distance from the diagonal), the tapering (instead of turning to zero, multiply by a factor which gets small as the distance to the diagonal gets large), and the thresholding (zero-out all entries smaller, in absolute value, than a certain well-chosen threshold).

In Chapter 2, the same questions and techniques are discussed in the case where the independence between the observations is replaced by weak dependence.

In Chapter 3, the authors suppress completely the hypothesis of independence of the observations, replace it by a stationarity hypothesis, and show how the techniques presented earlier allow one to still get estimations of the (auto)covariance matrix in the case of MA(r) and IVAR(r) models.

Chapters 4 and 5 collect the basic concepts and results from respectively random matrix theory (RMT), about the empirical spectral distribution of various random matrix models, and Voiculescu's free probability theory that are needed in Chapters 6 to 10.

Chapters 6 to 9, among other analogous questions, revisit the covariance matrix estimation results from Chapters 1 to 3 from the point of view of empirical spectral distribution (thanks to the framework defined in Chapters 4 and 5). In Chapter 10, it is demonstrated how the limiting spectral distribution (LSD) results obtained in Chapters 8 and 9 can be used in statistical graphical inference of high-dimensional time series. This includes estimation of unknown order of high-dimensional MA and AR processes.

In Chapter 11, central limit theorems (CLTs) for linear spectral statistics of random matrices are used in signi cance tests for di erent hypotheses on coe□cient matrices.

~Florent Benaych-Georges - Mathematical Reviews Clippings February 2019"Most of the materials covered in the book are at an advanced level. Fortunately, their exposition is clear, rigorous and highly self-contained. The book assumes a working knowledge in multivariate analysis, multivariate time series analysis and in stochastic convergence, that should be possessed by graduate students in econometrics, statistics or probability theory. A significant number of exercises are included in each chapter to help the reader master the introduced concepts, methods and results. The book is also an important reference for experienced researchers in the area of high-dimensional multivariate and time series analyses. One particular strength of the book is a thorough presentation of the most relevant concepts of non-commutative probability theory. In recent years and using this theory, the authors have developed several important results on the limiting proprieties of large sample covariance and autocovariance matrices. These results are now very accessible in this book."

~Journal of Time Series Analysis