#
Adaptive Filtering

Fundamentals of Least Mean Squares with MATLAB®

## Preview

## Book Description

Adaptive filters are used in many diverse applications, appearing in everything from military instruments to cellphones and home appliances. **Adaptive Filtering: Fundamentals of Least Mean Squares with MATLAB® **covers the core concepts of this important field, focusing on a vital part of the statistical signal processing area—the least mean square (LMS) adaptive filter.

This largely self-contained text:

- Discusses random variables, stochastic processes, vectors, matrices, determinants, discrete random signals, and probability distributions
- Explains how to find the eigenvalues and eigenvectors of a matrix and the properties of the error surfaces
- Explores the Wiener filter and its practical uses, details the steepest descent method, and develops the Newton’s algorithm
- Addresses the basics of the LMS adaptive filter algorithm
**,**considers LMS adaptive filter variants, and provides numerous examples - Delivers a concise introduction to MATLAB®, supplying problems, computer experiments, and more than 110 functions and script files

Featuring robust appendices complete with mathematical tables and formulas,** Adaptive Filtering: Fundamentals of Least Mean Squares with MATLAB® **clearly describes the key principles of adaptive filtering and effectively demonstrates how to apply them to solve real-world problems.

## Table of Contents

Preface

Author

Abbreviations

MATLAB® Functions

Vectors

Introduction

Multiplication by a Constant and Addition and Subtraction

Unit Coordinate Vectors

Inner Product

Distance between Two Vectors

Mean Value of a Vector

Direction Cosines

The Projection of a Vector

Linear Transformations

Linear Independence, Vector Spaces, and Basis Vectors

Orthogonal Basis Vectors

Problems

Hints–Suggestions–Solutions

Matrices

Introduction

General Types of Matrices

Diagonal, Identity, and Scalar Matrices

Upper and Lower Triangular Matrices

Symmetric and Exchange Matrices

Toeplitz Matrix

Hankel and Hermitian

Matrix Operations

Determinant of a Matrix

Definition and Expansion of a Matrix

Trace of a Matrix

Inverse of a Matrix

Linear Equations

Square Matrices (n × n)

Rectangular Matrices (n < m)

Rectangular Matrices (m < n)

Quadratic and Hermitian Forms

Eigenvalues and Eigenvectors

Eigenvectors

Properties of Eigenvalues and Eigenvectors

Problems

Hints–Suggestions–Solutions

Processing of Discrete Deterministic Signals: Discrete Systems

Discrete-Time Signals

Time-Domain Representation of Basic Continuous and Discrete Signals

Transform-Domain Representation of Discrete Signals

Discrete-Time Fourier Transform

The Discrete FT

Properties of DFT

The z-Transform

Discrete-Time Systems

Linearity and Shift Invariant

Causality

Stability

Transform-Domain Representation

Problems

Hints–Suggestions–Solutions

Discrete-Time Random Processes

Discrete Random Signals, Probability Distributions, and Averages of Random Variables

Stationary and Ergodic Processes

Averages of RV

Stationary Processes

Autocorrelation Matrix

Purely Random Process (White Noise)

Random Walk

Special Random Signals and pdf’s

White Noise

Gaussian Distribution (Normal Distribution)

Exponential Distribution

Lognormal Distribution

Chi-Square Distribution

Wiener–Khinchin Relations

Filtering Random Processes

Special Types of Random Processes

Autoregressive Process

Nonparametric Spectra Estimation

Periodogram

Correlogram

Computation of Periodogram and Correlogram Using FFT

General Remarks on the Periodogram

Proposed Book Modified Method for Better Frequency Resolution

Bartlett Periodogram

The Welch Method

Proposed Modified Welch Methods

Problems

Hints–Solutions–Suggestions

The Wiener Filter

Introduction

The LS Technique

Linear LS

LS Formulation

Statistical Properties of LSEs

The LS Approach

Orthogonality Principle

Corollary

Projection Operator

LS Finite Impulse Response Filter

The Mean-Square Error

The FIR Wiener Filter

The Wiener Solution

Orthogonality Condition

Normalized Performance Equation

Canonical Form of the Error-Performance Surface

Wiener Filtering Examples

Minimum MSE

Optimum Filter (wo)

Linear Prediction

Problems

Additional Problems

Hints–Solutions–Suggestions

Additional Problems

Eigenvalues of Rx: Properties of the Error Surface

The Eigenvalues of the Correlation Matrix

Karhunen–Loeve Transformation

Geometrical Properties of the Error Surface

Problems

Hints–Solutions–Suggestions

Newton’s and Steepest Descent Methods

One-Dimensional Gradient Search Method

Gradient Search Algorithm

Newton’s Method in Gradient Search

Steepest Descent Algorithm

Steepest Descent Algorithm Applied to Wiener Filter

Stability (Convergence) of the Algorithm

Transient Behavior of MSE

Learning Curve

Newton’s Method

Solution of the Vector Difference Equation

Problems

Edition Problems

Hints–Solutions–Suggestions

Additional Problems

The Least Mean-Square Algorithm

Introduction

The LMS Algorithm

Examples Using the LMS Algorithm

Performance Analysis of the LMS Algorithm

Learning Curve

The Coefficient-Error or Weighted-Error Correlation Matrix

Excess MSE and Misadjustment

Stability

The LMS and Steepest Descent Methods

Complex Representation of the LMS Algorithm

Problems

Hints–Solutions–Suggestions

Variants of Least Mean-Square Algorithm

The Normalized Least Mean-Square Algorithm

Power NLMS

Self-Correcting LMS Filter

The Sign-Error LMS Algorithm

The NLMS Sign-Error Algorithm

The Sign-Regressor LMS Algorithm

Self-Correcting Sign-Regressor LMS Algorithm

The Normalized Sign-Regressor LMS Algorithm

The Sign–Sign LMS Algorithm

The Normalized Sign–Sign LMS Algorithm

Variable Step-Size LMS

The Leaky LMS Algorithm

The Linearly Constrained LMS Algorithm

The Least Mean Fourth Algorithm

The Least Mean Mixed Norm LMS Algorithm

Short-Length Signal of the LMS Algorithm

The Transform Domain LMS Algorithm

Convergence

The Error Normalized Step-Size LMS Algorithm

The Robust Variable Step-Size LMS Algorithm

The Modified LMS Algorithm

Momentum LMS

The Block LMS Algorithm

The Complex LMS Algorithm

The Affine LMS Algorithm

The Complex Affine LMS Algorithm

Problems

Hints–Solutions–Suggestions

Appendix 1: Suggestions and Explanations for MATLAB Use

Suggestions and Explanations for MATLAB Use

Creating a Directory

Help

Save and Load

MATLAB as Calculator

Variable Names

Complex Numbers

Array Indexing

Extracting and Inserting Numbers in Arrays

Vectorization

Windowing

Matrices

Producing a Periodic Function

Script Files

Functions

Complex Expressions

Axes

2D Graphics

3D Plots

General Purpose Commands

Managing Commands and Functions

Managing Variables and Workplace

Operators and Special Characters

Control Flow

Elementary Matrices and Matrix Manipulation

Elementary Matrices and Arrays

Matrix Manipulation

Elementary Mathematical Functions

Elementary Functions

Numerical Linear Algebra

Matrix Analysis

Data Analysis

Basic Operations

Filtering and Convolution

Fourier Transforms

2D Plotting

2D Plots

Appendix 2: Matrix Analysis

Definitions

Special Matrices

Matrix Operation and Formulas

Eigendecomposition of Matrices

Matrix Expectations

Differentiation of a Scalar Function with respect to a Vector

Appendix 3: Mathematical Formulas

Trigonometric Identities

Orthogonality

Summation of Trigonometric Forms

Summation Formulas

Finite Summation Formulas

Infinite Summation Formulas

Series Expansions

Logarithms

Some Definite Integrals

Appendix 4: Lagrange Multiplier Method

Bibliography

Index

## Author(s)

### Biography

**Alexander D. Poularikas** is chairman of the electrical and computer engineering department at the University of Alabama in Huntsville, USA. He previously held positions at University of Rhode Island, Kingston, USA and the University of Denver, Colorado, USA. He has published, coauthored, and edited 14 books and served as an editor-in-chief of numerous book series. A Fulbright scholar, lifelong senior member of the IEEE, and member of Tau Beta Pi, Sigma Nu, and Sigma Pi, he received the IEEE Outstanding Educators Award, Huntsville Section in 1990 and 1996. Dr. Poularikas holds a Ph.D from the University of Arkansas, Fayetteville, USA.