#
Signal Processing

A Mathematical Approach, Second Edition

- Open access content is available for this title.

## Preview

## Book Description

**Signal Processing: A Mathematical Approach** is designed to show how many of the mathematical tools the reader knows can be used to understand and employ signal processing techniques in an applied environment. Assuming an advanced undergraduate- or graduate-level understanding of mathematics—including familiarity with Fourier series, matrices, probability, and statistics—this **Second Edition**:

- Contains new chapters on convolution and the vector DFT, plane-wave propagation, and the BLUE and Kalman filters
- Expands the material on Fourier analysis to three new chapters to provide additional background information
- Presents real-world examples of applications that demonstrate how mathematics is used in remote sensing

Featuring problems for use in the classroom or practice, **Signal Processing: A Mathematical Approach, Second Edition **covers topics such as Fourier series and transforms in one and several variables; applications to acoustic and electro-magnetic propagation models, transmission and emission tomography, and image reconstruction; sampling and the limited data problem; matrix methods, singular value decomposition, and data compression; optimization techniques in signal and image reconstruction from projections; autocorrelations and power spectra; high-resolution methods; detection and optimal filtering; and eigenvector-based methods for array processing and statistical filtering, time-frequency analysis, and wavelets.

## Table of Contents

Preface

Introduction

Chapter Summary

Aims and Topics

Some Examples of Remote Sensing

A Role for Mathematics

Limited Data

The Emphasis in this Book

Topics Covered

Applications of Interest

Sensing Modalities

Active and Passive Sensing

A Variety of Modalities

Using Prior Knowledge

An Urn Model of Remote Sensing

An Urn Model

Some Mathematical Notation

An Application to SPECT Imaging

Hidden Markov Models

Fourier Series and Fourier Transforms

Chapter Summary

Fourier Series

Complex Exponential Functions

Fourier Transforms

Basic Properties of the Fourier Transform

Some Fourier-Transform Pairs

Dirac Deltas

Convolution Filters

A Discontinuous Function

Shannon's Sampling Theorem

What Shannon Does Not Say

Inverse Problems

Two-Dimensional Fourier Transforms

The Basic Formulas

Radial Functions

An Example

The Uncertainty Principle

Best Approximation

The Orthogonality Principle

An Example

The DFT as Best Approximation

The Modified DFT (MDFT)

The PDFT

Analysis of the MDFT

Eigenvector Analysis of the MDFT

The Eigenfunctions of S_{r}

Remote Sensing

Chapter Summary

Fourier Series and Fourier Coefficients

The Unknown Strength Problem

Measurement in the Far Field

Limited Data

Can We Get More Data?

Measuring the Fourier Transform

Over-Sampling

The Modified DFT

Other Forms of Prior Knowledge

One-Dimensional Arrays

Measuring Fourier Coefficients

Over-Sampling

Under-Sampling

Using Matched Filtering

A Single Source

Multiple Sources

An Example: The Solar-Emission Problem

Estimating the Size of Distant Objects

The Transmission Problem

Directionality

The Case of Uniform Strength

The Laplace Transform and the Ozone Layer

The Laplace Transform

Scattering of Ultraviolet Radiation

Measuring the Scattered Intensity

The Laplace Transform Data

The Laplace Transform and Energy Spectral Estimation

The Attenuation Coefficient Function

The Absorption Function as a Laplace Transform

Finite-Parameter Models

Chapter Summary

Finite Fourier Series

The DFT and the Finite Fourier Series

The Vector DFT

The Vector DFT in Two Dimensions

The Issue of Units

Approximation, Models, or Truth?

Modeling the Data

Extrapolation

Filtering the Data

More on Coherent Summation

Uses in Quantum Electrodynamics

Using Coherence and Incoherence

The Discrete Fourier Transform

Complications

Multiple Signal Components

Resolution

Unequal Amplitudes and Complex Amplitudes

Phase Errors

Undetermined Exponential Models

Prony's Problem

Prony's Method

Transmission and Remote Sensing

Chapter Summary

Directional Transmission

Multiple-Antenna Arrays

The Array of Equi-Spaced Antennas

The Far-Field Strength Pattern

Can the Strength Be Zero?

Diffraction Gratings

Phase and Amplitude Modulation

Steering the Array

Maximal Concentration in a Sector

Scattering in Crystallography

The Fourier Transform and Convolution Filtering

Chapter Summary

Linear Filters

Shift-Invariant Filters

Some Properties of a SILO

The Dirac Delta

The Impulse Response Function

Using the Impulse-Response Function

The Filter Transfer Function

The Multiplication Theorem for Convolution

Summing Up

A Question

Band-Limiting

Infinite Sequences and Discrete Filters

Chapter Summary

Shifting

Shift-Invariant Discrete Linear Systems

The Delta Sequence

The Discrete Impulse Response

The Discrete Transfer Function

Using Fourier Series

The Multiplication Theorem for Convolution

The Three-Point Moving Average

Autocorrelation

Stable Systems

Causal Filters

Convolution and the Vector DFT

Chapter Summary

Nonperiodic Convolution

The DFT as a Polynomial

The Vector DFT and Periodic Convolution

The Vector DFT

Periodic Convolution

The vDFT of Sampled Data

Superposition of Sinusoids

Rescaling

The Aliasing Problem

The Discrete Fourier Transform

Calculating Values of the DFT

Zero-Padding

What the vDFT Achieves

Terminology

Understanding the Vector DFT

The Fast Fourier Transform (FFT)

Evaluating a Polynomial

The DFT and Vector DFT

Exploiting Redundancy

The Two-Dimensional Case

Plane-Wave Propagation

Chapter Summary

The Bobbing Boats

Transmission and Remote Sensing

The Transmission Problem

Reciprocity

Remote Sensing

The Wave Equation

Plane-wave Solutions

Superposition and the Fourier Transform

The Spherical Model

Sensor Arrays

The Two-Dimensional Array

The One-Dimensional Array

Limited Aperture

Sampling

The Limited-Aperture Problem

Resolution

The Solar-Emission Problem Revisited

Other Limitations on Resolution

Discrete Data

Reconstruction from Samples

The Finite-Data Problem

Functions of Several Variables

A Two-Dimensional Far-Field Object

Limited Apertures in Two Dimensions

Broadband Signals

The Phase Problem

Chapter Summary

Reconstructing from Over-Sampled Complex FT Data

The Phase Problem

A Phase-Retrieval Algorithm

Fienup's Method

Does the Iteration Converge?

Transmission Tomography

Chapter Summary

X-Ray Transmission Tomography

The Exponential-Decay Model

Difficulties to be Overcome

Reconstruction from Line Integrals

The Radon Transform

The Central Slice Theorem

Inverting the Fourier Transform

Back Projection

Ramp Filter, then Back Project

Back Project, then Ramp Filter

Radon's Inversion Formula

From Theory to Practice

The Practical Problems

A Practical Solution: Filtered Back Projection

Some Practical Concerns

Summary

Random Sequences

Chapter Summary

What is a Random Variable?

The Coin-Flip Random Sequence

Correlation

Filtering Random Sequences

An Example

Correlation Functions and Power Spectra

The Dirac Delta in Frequency Space

Random Sinusoidal Sequences

Random Noise Sequences

Increasing the SNR

Colored Noise

Spread-Spectrum Communication

Stochastic Difference Equations

Random Vectors and Correlation Matrices

The Prediction Problem

Prediction Through Interpolation

Divided Differences

Linear Predictive Coding

Discrete Random Processes

Wide-Sense Stationary Processes

Autoregressive Processes

Linear Systems with Random Input

Stochastic Prediction

Prediction for an Autoregressive Process

Nonlinear Methods

Chapter Summary

The Classical Methods

Modern Signal Processing and Entropy

Related Methods

Entropy Maximization

Estimating Nonnegative Functions

Philosophical Issues

The Autocorrelation Sequence fr(n)g

Minimum-Phase Vectors

Burg's MEM

The Minimum-Phase Property

Solving *Ra = δ* Using Levinson's Algorithm

A Sufficient Condition for Positive-Definiteness

The IPDFT

The Need for Prior Information in Nonlinear Estimation

What Wiener Filtering Suggests

Using a Prior Estimate

Properties of the IPDFT

Illustrations

Fourier Series and Analytic Functions

An Example

Hyperfunctions

Fejér-Riesz Factorization

Burg Entropy

Some Eigenvector Methods

The Sinusoids-in-Noise Model

Autocorrelation

Determining the Frequencies

The Case of Non-White Noise

Discrete Entropy Maximization

Chapter Summary

The Algebraic Reconstruction Technique

The Multiplicative Algebraic Reconstruction Technique

The Kullback-Leibler Distance

The EMART

Simultaneous Versions

The Landweber Algorithm

The SMART

The EMML Algorithm

Block-Iterative Versions

Convergence of the SMART

Analysis and Synthesis

Chapter Summary

The Basic Idea

Polynomial Approximation

Signal Analysis

Practical Considerations in Signal Analysis

The Finite-Data Problem

Frames

Bases, Riesz Bases, and Orthonormal Bases

Radar Problems

The Wideband Cross-Ambiguity Function

The Narrowband Cross-Ambiguity Function

Range Estimation

Time-Frequency Analysis

The Short-Time Fourier Transform

The Wigner-Ville Distribution

Wavelets

Chapter Summary

Background

A Simple Example

The Integral Wavelet Transform

Wavelet Series Expansions

Multiresolution Analysis

The Shannon Multiresolution Analysis

The Haar Multiresolution Analysis

Wavelets and Multiresolution Analysis

Signal Processing Using Wavelets

Decomposition and Reconstruction

Generating the Scaling Function

Generating the Two-Scale Sequence

Wavelets and Filter Banks

Using Wavelets

The BLUE and the Kalman Filter

Chapter Summary

The Simplest Case

A More General Case

Some Useful Matrix Identities

The BLUE with a Prior Estimate

Adaptive BLUE

The Kalman Filter

Kalman Filtering and the BLUE

Adaptive Kalman Filtering

Difficulties with the BLUE

Preliminaries from Linear Algebra

When are the BLUE and the LS Estimator the Same?

A Recursive Approach

Signal Detection and Estimation

Chapter Summary

The Model of Signal in Additive Noise

Optimal Linear Filtering for Detection

The Case of White Noise

Constant Signal

Sinusoidal Signal, Frequency Known

Sinusoidal Signal, Frequency Unknown

The Case of Correlated Noise

Constant Signal with Unequal-Variance Uncorrelated Noise

Sinusoidal signal, Frequency Known, in Correlated Noise

Sinusoidal Signal, Frequency Unknown, in Correlated Noise

Capon's Data-Adaptive Method

Appendix: Inner Products

Chapter Summary

Cauchy's Inequality

The Complex Vector Dot Product

Orthogonality

Generalizing the Dot Product: Inner Products

Another View of Orthogonality

Examples of Inner Products

An Inner Product for Infinite Sequences

An Inner Product for Functions

An Inner Product for Random Variables

An Inner Product for Complex Matrices

A Weighted Inner Product for Complex Vectors

A Weighted Inner Product for Functions

The Orthogonality Principle

Appendix: Wiener Filtering

Chapter Summary

The Vector Wiener Filter in Estimation

The Simplest Case

A More General Case

The Stochastic Case

The VWF and the BLUE

Wiener Filtering of Functions

Wiener Filter Approximation: The Discrete Stationary Case

Approximating the Wiener Filter

Adaptive Wiener Filters

An Adaptive Least-Mean-Square Approach

Adaptive Interference Cancellation (AIC)

Recursive Least Squares (RLS)

Appendix: Matrix Theory

Chapter Summary

Matrix Inverses

Basic Linear Algebra

Bases and Dimension

Systems of Linear Equations

Real and Complex Systems of Linear Equations

Solutions of Under-determined Systems of Linear Equations

Eigenvalues and Eigenvectors

Vectorization of a Matrix

The Singular Value Decomposition (SVD)

The SVD

An Application in Space Exploration

Pseudo-Inversion

Singular Values of Sparse Matrices

Matrix and Vector Differentiation

Differentiation with Respect to a Vector

Differentiation with Respect to a Matrix

Eigenvectors and Optimization

Appendix: Compressed Sensing

Chapter Summary

An Overview

Compressed Sensing

Sparse Solutions

Maximally Sparse Solutions

Minimum One-Norm Solutions

Minimum One-Norm as an LP Problem

Why the One-Norm?

Comparison with the PDFT

Iterative Reweighting

Why Sparseness?

Signal Analysis

Locally Constant Signals

Tomographic Imaging

Compressed Sampling

Appendix: Probability

Chapter Summary

Independent Random Variables

Maximum Likelihood Parameter Estimation

An Example: The Bias of a Coin

Estimating a Poisson Mean

Independent Poisson Random Variables

The Multinomial Distribution

Characteristic Functions

Gaussian Random Variables

Gaussian Random Vectors

Complex Gaussian Random Variables

Using A Priori Information

Conditional Probabilities and Bayes' Rule

An Example of Bayes' Rule

Using Prior Probabilities

Maximum A Posteriori Estimation

MAP Reconstruction of Images

Penalty-Function Methods

Basic Notions

Generating Correlated Noise Vectors

Covariance Matrices

Principal Component Analysis

Appendix: Using the Wave Equation

Chapter Summary

The Wave Equation

The Shallow-Water Case

The Homogeneous-Layer Model

The Pekeris Waveguide

The General Normal-Mode Model

Matched-Field Processing

Appendix: Reconstruction in Hilbert Space

Chapter Summary

The Basic Problem

Fourier-Transform Data

The General Case

Some Examples

Choosing the Inner Product

Choosing the Hilbert Space

Summary

Appendix: Some Theory of Fourier Analysis

Chapter Summary

Fourier Series

Fourier Transforms

Functions in the Schwartz Class

Generalized Fourier Series

Wiener Theory

Appendix: Reverberation and Echo Cancellation

Chapter Summary

The Echo Model

Finding the Inverse Filter

Using the Fourier Transform

The Teleconferencing Problem

Bibliography

Index

### Featured Author Profiles

## Reviews

"The second edition of this book further confirms the skill of its author to combine mathematical rigor with intuitive understanding. ... All chapters are carefully structured such that the reader can make easily an image on the main ideas of each topic, not only from the mathematical point of view but also regarding intuitive interpretations and applications. In fact, the book can be a very good course for advanced undergraduates who will find in it a kind of handbook of connections between mathematical concepts and applications in signal processing. ... The book is worth reading not only by students but also by people wishing to have a broad image on signal processing in its intrinsic relationship with mathematics."

—Liviu Goraş (Iaşi), fromZentralblatt MATH 1321 – 1

## Support Material

### Open Access Content

Open Access content has been made available under a Creative Commons Attribution-Non Commercial-No Derivatives (CCBY-NC-ND) license

Read Full Book - Open Access