2nd Edition

Introduction to Time Series Modeling with Applications in R

By Genshiro Kitagawa Copyright 2020
    340 Pages
    by Chapman & Hall

    340 Pages
    by Chapman & Hall

    Praise for the first edition:

    [This book] reflects the extensive experience and significant contributions of the author to non-linear and non-Gaussian modeling. … [It] is a valuable book, especially with its broad and accessible introduction of models in the state-space framework.

    Statistics in Medicine

    What distinguishes this book from comparable introductory texts is the use of state-space modeling. Along with this come a number of valuable tools for recursive filtering and smoothing, including the Kalman filter, as well as non-Gaussian and sequential Monte Carlo filters.

    MAA Reviews

    Introduction to Time Series Modeling with Applications in R, Second Edition covers numerous stationary and nonstationary time series models and tools for estimating and utilizing them. The goal of this book is to enable readers to build their own models to understand, predict and master time series. The second edition makes it possible for readers to reproduce examples in this book by using the freely available R package TSSS to perform computations for their own real-world time series problems.

    This book employs the state-space model as a generic tool for time series modeling and presents the Kalman filter, the non-Gaussian filter and the particle filter as convenient tools for recursive estimation for state-space models. Further, it also takes a unified approach based on the entropy maximization principle and employs various methods of parameter estimation and model selection, including the least squares method, the maximum likelihood method, recursive estimation for state-space models and model selection by AIC.

    Along with the standard stationary time series models, such as the AR and ARMA models, the book also introduces nonstationary time series models such as the locally stationary AR model, the trend model, the seasonal adjustment model, the time-varying coefficient AR model and nonlinear non-Gaussian state-space models.

    About the Author:

    Genshiro Kitagawa is a project professor at the University of Tokyo, the former Director-General of the Institute of Statistical Mathematics, and the former President of the Research Organization of Information and Systems.

    1   Introduction and Preparatory Analysis 


    1.1    Time Series Data                                                                    

    1.2    Classification of Time Series                                                

    1.3    Objectives of Time Series Analysis                                     

    1.4    Pre-processing of Time Series                                              

    1.4.1    Transformation of variables                                  

    1.4.2    Differencing                                                             

    1.4.3    Month-to-month basis and year-over-year          

    1.4.4    Moving average                                                      

    1.5    Organization of This Book 


    2   The Covariance Function     

    2.1   The Distribution of Time Series and Stationarity             

    2.2   The Autocovariance Function of Stationary Time Series      

    2.3   Estimation of the Autocovariance Function                     

    2.4   Multivariate Time Series and Scatterplots                        

    2.5   Cross-covariance Function and Cross-correlation Function   


    3  The Power Spectrum and the Periodogram     

    3.1   The Power Spectrum                                                           

    3.2   The Periodogram                                                                 

    3.3   Averaging and Smoothing of the Periodogram               

    3.4   Computational Method of Periodogram                         

    3.5   Computation of the Periodogram by Fast Fourier Transform     


    4     Statistical Modeling     

    4.1    Probability Distributions and Statistical Models            

    4.2    K-L Information and Entropy Maximization Principle    

    4.3    Estimation of the K-L Information and the Log-likelihood                                                     

    4.4    Estimation of Parameters by the Maximum Likelihood Method                                                                                  

    4.5    AIC (Akaike Information Criterion)                                 

    4.5.1    Evaluation of C1

    4.5.2    Evaluation of C3

    4.5.3    Evaluation of C2

    4.5.4    Evaluation of C and AIC

    4.6    Transformation of Data 

    5 The Least Squares Method

    5.1    Regression Models and the Least Squares Method

    5.2    Householder Transformation Method

    5.3    Selection of Order by AIC

    5.4    Addition of Data and Successive Householder Reduction

    5.5    Variable Selection by AIC

    Analysis of Time Series Using ARMA Models

    6.1    ARMA Model

    6.2    The Impulse Response Function

    6.3    The Autocovariance Function

    6.4    The Relation Between AR Coefficients and PARCOR 98

    6.5    The Power Spectrum of the ARMA Process 98

    6.6    The Characteristic Equation 102

    6.7    The Multivariate AR Model 106


    7 Estimation of an AR Model


    7.1    Fitting an AR Mode

    7.2    Yule-Walker Method and Levinson’s Algorithm

    7.3    Estimation of an AR Model by the Least Squares Method

    7.4    Estimation of an AR Model by the PARCOR Method

    7.5    Large Sample Distribution of the Estimates

    7.6    Estimation of Multivariate AR Model by Yule-Walker Method

    7.7    Estimation of Multivariate AR Model by Least Squares Method


    8 The Locally Stationary AR Model   


    8.1     Locally Stationary AR Model                                              

    8.2     Automatic Partitioning of the Time Interval                        

    8.3     Precise Estimation of the Change Point                               

    8.4     Posterior Probability of the Change Point     


    9       Analysis of Time Series with a State-Space Model 


    9.1    The State-Space Model                                                     

    9.2     State Estimation via the Kalman Filter                                 

    9.3     Smoothing Algorithms                                                         

    9.4     Long-term Prediction of the State                                        

    9.5     Prediction of Time Series                                                   

    9.6     Likelihood Computation and Parameter Estimation for Time Series Models                                                            

    9.7     Interpolation of Missing Observations 


    10    Estimation of the ARMA Model     


    10.1    State-Space Representation of the ARMA Model                

    10.2    Initial State Distribution for an AR Model                           

    10.3    Initial State Distribution of an ARMA Model                      

    10.4    Maximum Likelihood Estimates of an ARMA Model         

    10.5    Initial Estimates of Parameters   


    11    Estimation of Trends     


    11.1    The Polynomial Trend Model                                              

    11.2    Trend Component Model – Model for Gradual Changes    

    11.3    Trend Model   


    12    The Seasonal Adjustment Model   


    12.1    Seasonal Component Model                                                

    12.2    Standard Seasonal Adjustment Model                                 

    12.3    Decomposition Including an AR Component                      

    12.4    Decomposition Including a Trading-day Effect 


    13    Time-Varying Coefficient AR Model   


    13.1    Time-varying Variance Model                                             

    13.2    Time-varying Coefficient AR Model                                   

    13.3    Estimation of the Time-varying Spectrum                           

    13.4    The Assumption on System Noise for the Time-varying Coefficient AR Model                                                                                  

    13.5    Abrupt Changes of Coefficients 


    14    Non-Gaussian State-Space Model     

    14.1     Necessity of Non-Gaussian Models                                     

    14.2     Non-Gaussian State-Space Models and State Estimation    

    14.3     Numerical Computation of the State Estimation Formula    

    14.4     Non-Gaussian Trend Model                                                

    14.5     A Time-varying Variance Model                                     

    14.6     Further Applications of Non-Gaussian State-Space Model                                                                                

    14.6.1     Processing of the outliers by a mixture of Gaussian distributions                                             

    14.6.2     A nonstationary discrete process                            

    14.6.3     A direct method of estimating the time-varying variance                                                                  

    14.6.4     Nonlinear state-spece models 


    15    Particle Filter 

    15.1    The Nonlinear Non-Gaussian State-Space Model and Approximations of Distributions                                                                        

    15.2    Particle Filter                                                                      

    15.2.1    One-step-ahead prediction                                      

    15.2.2    Filtering                                                                 

    15.2.3    Algorithm for the particle filter                               

    15.2.4    Likelihood of a model                                           

    15.2.5    On the re-sampling method                                    

    15.2.6    Numerical examples                                               

    15.3    Particle Smoothing Method                                                 

    15.4    Nonlinear Smoothing 


    16    Simulation   

    16.1    Generation of Uniform Random Numbers                          

    16.2    Generation of White Noise                                                  

    16.2.1     χ2 distribution                                                       

    16.2.2     Cauchy distribution                                                 

    16.2.3     Arbitrary distribution                                              

    16.3    Simulation of ARMA models                                              

    16.4    Simulation Using a State-Space Model                                

    16.5    Simulation with Non-Gaussian Model                                

    A      Algorithms forNonlinearOptimization                                   
    B      Derivation ofLevinson’sAlgorithm                                          
    C      Derivation of the Kalman Filter and Smoother Algorithms    
    C.1      Kalman Filter                                                                     
    C.2      Smoothing                                                                          
    D      Algorithm for the Particle Filter                                                
    D.1      One-step-ahead Prediction                                              
    D.2      Filter                                                                                    
    D.3      Smoothing                                                                          



    Genshiro Kitagawa is a project professor at the University of Tokyo, the former Director-General of the Institute of Statistical Mathematics, and the former President of the Research Organization of Information and Systems.