1st Edition

# Computational Mathematics An introduction to Numerical Analysis and Scientific Computing with Python

528 Pages 92 B/W Illustrations
by Chapman & Hall

528 Pages 92 B/W Illustrations
by Chapman & Hall

Also available as eBook on:

This textbook is a comprehensive introduction to computational mathematics and scientific computing suitable for undergraduate and postgraduate courses. It presents both practical and theoretical aspects of the subject, as well as advantages and pitfalls of classical numerical methods alongside with computer code and experiments in Python. Each chapter closes with modern applications in physics, engineering, and computer science.

Features:

• No previous experience in Python is required.
• Includes simplified computer code for fast-paced learning and transferable skills development.
• Includes practical problems ideal for project assignments and distance learning.
• Presents both intuitive and rigorous faces of modern scientific computing.
• Provides an introduction to neural networks and machine learning.

Preface

1. Introduction to scientific computing with Python
2. 1 Introduction to Python

1.1 Basic Python

1.1.1 Python, numbers and variables

1.1.2 Basic mathematical operations

1.1.3 Mathematical functions and constants

1.1.4 Powers of 10 and scientific notation

1.2 Assignment operators

1.3 Strings, tuples, dictionaries and lists

1.3.1 Strings

1.3.2 Tuples

1.3.3 Dictionaries

1.3.4 Lists

1.3.5 Lists of lists

1.4 Flow control

1.4.1 for loops

1.4.2 Useful applications

1.4.3 Boolean type of variables

1.4.4 while loops

1.4.5 Decisions and the if statement

1.4.6 Error control and exceptions

1.4.7 The command break

1.5 Functions

1.6 Classes

1.7 Accessing data files

Chapter highlights

Exercises

2 Matrices and Python

2.1 Review of matrices

2.1.1 Matrix properties

2.1.2 Special matrices

2.2 The NumPy module

2.2.1 Matrices and arrays

2.2.2 Slicing

2.2.3 Special matrices, methods and properties

2.2.4 Matrix assignment

2.2.5 Matrix addition and scalar multiplication

2.2.6 Vectorized algorithms

2.2.7 Performance of algorithms

2.2.8 Matrix multiplication

2.2.9 Inverse of a matrix

2.2.10 Determinant of a matrix

2.2.11 Linear systems

2.2.12 Eigenvalues and eigenvectors

2.3 The SciPy module

2.3.1 Systems with banded matrices

2.3.2 Systems with positive definite, banded matrices

2.3.3 Systems with other special matrices

2.4 Drawing graphs with MatPlotLib

2.4.1 Plotting one-dimensional arrays

Chapter highlights

Exercises

3 Scientific computing

3.1 Computer arithmetic and sources of error

3.1.1 Catastrophic cancelation

3.1.2 The machine precision

3.2 Normalized scientific notation

3.2.1 Floats in Python

3.3 Rounding and chopping

3.3.1 Absolute and relative errors

3.3.2 Loss of significance (again)

3.3.3 Algorithms and stability

3.3.4 Approximations of π

Chapter highlights

Exercises

4 Calculus facts

4.1 Sequences and convergence

4.2 Continuous functions

4.2.1 Bolzano’s intermediate value theorem

4.3 Differentiation

4.3.1 Important theorems from differential calculus

4.3.2 Taylor polynomials

4.3.3 The big-O notation

4.4 Integration

4.4.1 Approximating integrals

4.4.2 Important theorems of integral calculus

4.4.3 The remainder in Taylor polynomials

Chapter highlights

Exercises

1. Introduction to computational mathematics

5 Roots of equations

5.1 Bisection method

5.1.1 Derivation and implementation

5.1.2 Proof of convergence

5.1.3 Rate of convergence

5.2 Fixed-point methods

5.2.1 Derivation and implementation

5.2.2 Theoretical considerations

5.2.3 Convergence of fixed-point methods

5.3 Newton’s method

5.3.1 Derivation and implementation

5.3.2 Rate of convergence

5.4 Secant method

5.4.1 Derivation and implementation

5.4.2 Rate of convergence

5.5 Other methods and generalizations

5.5.1 Stopping criteria

5.5.2 Steffensen’s method

5.5.3 Aitken’s delta-squared process

5.5.4 Multiple roots

5.5.5 High-order methods

5.5.6 Systems of equations

5.6 The module scipy.optimize

5.6.1 Bisection method

5.6.2 Fixed-point methods

5.6.3 Newton and secant methods

5.6.4 A hybrid method

5.6.5 Application in astrophysics

Chapter highlights

Exercises

6 Interpolation and approximation

6.1 Introduction

6.2 Lagrange interpolation

6.2.1 Naive construction of the interpolant

6.2.2 Lagrange polynomials

6.2.3 Failure of Lagrange interpolation

6.2.4 Error analysis

6.2.5 Newton’s representation with divided differences

6.2.6 More on divided differences

6.3 Hermite interpolation

6.3.1 Computation of the Hermite interpolant

6.3.2 Error analysis

6.3.3 Implementation details

6.4 Spline interpolation

6.4.1 Continuous piecewise linear interpolation

6.4.2 Some theoretical considerations

6.4.3 Error analysis of piecewise linear interpolation

6.4.4 Cubic spline interpolation

6.4.5 An algorithm for cubic splines

6.4.6 Theoretical considerations and error analysis

6.4.7 B-splines

6.5 Method of least squares

6.5.1 Linear least squares

6.5.2 Polynomial fit

6.5.3 Non-polynomial fit

6.6 The module scipy.interpolate

6.6.1 Lagrange interpolation

6.6.2 Cubic spline interpolation

6.6.3 Computations with B-splines

6.6.4 General purpose interpolation and the function interp1d

6.6.5 Interpolation in two dimensions

6.6.6 Application in image processing

Chapter highlights

Exercises

7 Numerical integration

7.1 Introduction to numerical quadrature and midpoint rule

7.2.1 Trapezoidal rule

7.2.2 Error estimate of the trapezoidal rule

7.2.3 Composite trapezoidal rule

7.2.4 Error estimate of the composite trapezoidal rule

7.2.5 Simpson’s rule

7.2.6 Composite Simpson’s rule

7.2.7 Error of Simpson’s rule

7.2.8 Degree of exactness

7.3.1 Choice of nodes and weights

7.3.3 Gaussian quadrature on general intervals

7.3.4 Error analysis of the Gaussian quadrature

7.4 The module scipy.integrate

7.4.1 Trapezoidal rule

7.4.2 Simpson’s rule

7.4.4 Application in classical mechanics

Chapter highlights

Exercises

8 Numerical differentiation and applications to differential equations

8.1 Numerical differentiation

8.1.1 First-order accurate finite difference approximations

8.1.2 Second-order accurate finite difference approximations

8.1.3 Error estimation

8.1.4 Approximation of second-order derivatives

8.1.5 Richardson’s extrapolation

8.2 Applications to ordinary differential equations

8.2.1 First-order ordinary differential equations

8.2.2 Euler method

8.2.3 Alternative derivation and error estimates

8.2.4 Implicit variants of Euler’s method

8.2.5 Improved Euler method

8.2.6 The notion of stability

8.3 Runge-Kutta methods

8.3.1 Explicit Runge-Kutta methods

8.3.2 Implicit and diagonally implicit Runge-Kutta methods

8.4 The module scipy.integrate again

8.4.1 Generic integration of initial value problems

8.4.2 Systems of ordinary differential equations

8.4.3 Application in epidemiology

Chapter highlights

Exercises

9 Numerical linear algebra

9.1 Numerical solution of linear systems

9.1.1 Algorithm for the naive Gaussian elimination

9.1.2 LU factorization

9.1.3 Implementation of LU factorization

9.2 Pivoting strategies

9.2.1 Gaussian elimination with partial pivoting

9.2.2 LU factorization with pivoting

9.3 Condition number of a matrix

9.3.1 Vector and matrix norms

9.3.2 Theoretical properties of matrix norms

9.3.3 Condition number

9.4 Other matrix computations

9.4.1 Computation of inverse matrix

9.4.2 Computation of determinants

9.5 Symmetric matrices

9.5.1 LDLT factorization

9.5.2 Cholesky factorization

9.6 The module scipy.linalg again

9.6.1 LU factorization

9.6.2 Cholesky factorization

9.7 Iterative methods

9.7.1 Derivation of iterative methods

9.7.2 Classical iterative methods – Jacobi and Gauss-Seidel methods

9.7.3 Convergence of iterative methods

9.7.4 Sparse matrices in Python and the module scipy.sparse

9.7.5 Sparse Gauss-Seidel implementation

9.7.6 Sparse linear systems and the module scipy.sparce

9.7.7 Application in electric circuits

Chapter highlights

Exercises

10 Best approximations

10.1 Vector spaces, norms, and approximations

10.1.1 Vector spaces

10.1.2 Inner products and norms

10.1.3 Best approximations

10.2 Linear least squares

10.2.1 Theoretical properties of linear least squares

10.2.2 Solution of linear least squares problem

10.2.3 Linear least squares problem in Python

10.3 Gram-Schmidt orthonormalization

10.3.1 Numerical implementation

10.4 QR factorization

10.4.1 Linear least squares using QR factorization

10.4.2 Computation of range

10.4.3 Householder matrices and QR factorization

10.4.4 Python implementation and orthonormalization

10.5 Singular value decomposition

10.5.1 Derivation of SVD

10.5.2 Theoretical considerations

10.5.3 Pseudoinverse and SVD

10.5.4 Linear least squares problem and SVD

10.5.5 Singular value decomposition in Python

10.5.6 Application in image compression

Chapter highlights

Exercises

11 Unconstrained optimization and neural networks

11.1 Gradient methods for unconstrained optimization

11.1.1 Estimation of convergence

11.1.2 Method of steepest descent

11.1.3 Solving linear systems using optimization

11.1.4 Theoretical considerations

11.2.1 Conjugate gradient method and linear systems

11.2.2 Convergence and preconditioning

11.2.3 Extensions to nonlinear systems

11.3 Newton’s method in optimization

11.3.1 Newton’s iteration

11.3.2 Quasi-Newton methods

11.3.3 Nonlinear optimization and the module scipy.optimize

11.3.4 The Gauss-Newton approximation

11.3.5 The Levenberg-Marquardt modification

11.4 Introduction to neural networks

11.4.1 A simple neural network

11.4.2 Activation functions

11.4.3 General feedforward neural networks

11.4.4 Machine learning and the module sklearn

11.4.5 Application in image recognition

Chapter highlights

Exercises

12 Eigenvalue problems

12.1 Eigenvalues and eigenvectors

12.1.1 Basic properties

12.1.2 Diagonalization of a matrix and the Jordan normal form

12.1.3 Other properties

12.1.4 First estimation of eigenvalues using Gerschgorin disks

12.2 Numerical approximation of eigenvalues and eigenvectors

12.2.1 Power method

12.2.2 Inverse power method

12.2.3 Shifted inverse power method

12.2.4 Basic QR iteration

12.2.5 Application in computer networks

Chapter highlights

Exercises

Appendix A Computing Jordan normal forms

Bibliography

Index

### Biography

Dimitrios Mitsotakis received a PhD in Mathematics in 2007 from the University of Athens. His experience with high-performance computing started while at the Edinburgh Parallel Computing Center at the University of Edinburgh. Dimitrios worked at the University Paris-Sud as a Marie Curie fellow, at the University of Minnesota as an associate postdoc and at the University of California, Merced as a Visiting Assistant Professor. Dimitrios is currently an associate professor/reader at the School of Mathematics and Statistics of Victoria University of Wellington. He has published his work in journals of numerical analysis and in more general audience journals in physics, coastal engineering, waves sciences, and in scientific computing. He develops numerical methods for the solution of equations for water waves, and he studies real-world applications such as the generation of tsunamis. Some of his main contributions are in the theory and numerical analysis of Boussinesq systems for nonlinear and dispersive water waves.