1st Edition
Computational Mathematics An introduction to Numerical Analysis and Scientific Computing with Python
This textbook is a comprehensive introduction to computational mathematics and scientific computing suitable for undergraduate and postgraduate courses. It presents both practical and theoretical aspects of the subject, as well as advantages and pitfalls of classical numerical methods alongside with computer code and experiments in Python. Each chapter closes with modern applications in physics, engineering, and computer science.
Features:
- No previous experience in Python is required.
- Includes simplified computer code for fast-paced learning and transferable skills development.
- Includes practical problems ideal for project assignments and distance learning.
- Presents both intuitive and rigorous faces of modern scientific computing.
- Provides an introduction to neural networks and machine learning.
Preface
- Introduction to scientific computing with Python
- Introduction to computational mathematics
1 Introduction to Python
1.1 Basic Python
1.1.1 Python, numbers and variables
1.1.2 Basic mathematical operations
1.1.3 Mathematical functions and constants
1.1.4 Powers of 10 and scientific notation
1.2 Assignment operators
1.3 Strings, tuples, dictionaries and lists
1.3.1 Strings
1.3.2 Tuples
1.3.3 Dictionaries
1.3.4 Lists
1.3.5 Lists of lists
1.4 Flow control
1.4.1 for loops
1.4.2 Useful applications
1.4.3 Boolean type of variables
1.4.4 while loops
1.4.5 Decisions and the if statement
1.4.6 Error control and exceptions
1.4.7 The command break
1.5 Functions
1.6 Classes
1.7 Accessing data files
1.8 Further reading
Chapter highlights
Exercises
2 Matrices and Python
2.1 Review of matrices
2.1.1 Matrix properties
2.1.2 Special matrices
2.2 The NumPy module
2.2.1 Matrices and arrays
2.2.2 Slicing
2.2.3 Special matrices, methods and properties
2.2.4 Matrix assignment
2.2.5 Matrix addition and scalar multiplication
2.2.6 Vectorized algorithms
2.2.7 Performance of algorithms
2.2.8 Matrix multiplication
2.2.9 Inverse of a matrix
2.2.10 Determinant of a matrix
2.2.11 Linear systems
2.2.12 Eigenvalues and eigenvectors
2.3 The SciPy module
2.3.1 Systems with banded matrices
2.3.2 Systems with positive definite, banded matrices
2.3.3 Systems with other special matrices
2.4 Drawing graphs with MatPlotLib
2.4.1 Plotting one-dimensional arrays
2.5 Further reading
Chapter highlights
Exercises
3 Scientific computing
3.1 Computer arithmetic and sources of error
3.1.1 Catastrophic cancelation
3.1.2 The machine precision
3.2 Normalized scientific notation
3.2.1 Floats in Python
3.3 Rounding and chopping
3.3.1 Absolute and relative errors
3.3.2 Loss of significance (again)
3.3.3 Algorithms and stability
3.3.4 Approximations of π
3.4 Further reading
Chapter highlights
Exercises
4 Calculus facts
4.1 Sequences and convergence
4.2 Continuous functions
4.2.1 Bolzano’s intermediate value theorem
4.3 Differentiation
4.3.1 Important theorems from differential calculus
4.3.2 Taylor polynomials
4.3.3 The big-O notation
4.4 Integration
4.4.1 Approximating integrals
4.4.2 Important theorems of integral calculus
4.4.3 The remainder in Taylor polynomials
4.5 Further reading
Chapter highlights
Exercises
5 Roots of equations
5.1 Bisection method
5.1.1 Derivation and implementation
5.1.2 Proof of convergence
5.1.3 Rate of convergence
5.2 Fixed-point methods
5.2.1 Derivation and implementation
5.2.2 Theoretical considerations
5.2.3 Convergence of fixed-point methods
5.3 Newton’s method
5.3.1 Derivation and implementation
5.3.2 Rate of convergence
5.4 Secant method
5.4.1 Derivation and implementation
5.4.2 Rate of convergence
5.5 Other methods and generalizations
5.5.1 Stopping criteria
5.5.2 Steffensen’s method
5.5.3 Aitken’s delta-squared process
5.5.4 Multiple roots
5.5.5 High-order methods
5.5.6 Systems of equations
5.6 The module scipy.optimize
5.6.1 Bisection method
5.6.2 Fixed-point methods
5.6.3 Newton and secant methods
5.6.4 A hybrid method
5.6.5 Application in astrophysics
5.7 Further reading
Chapter highlights
Exercises
6 Interpolation and approximation
6.1 Introduction
6.2 Lagrange interpolation
6.2.1 Naive construction of the interpolant
6.2.2 Lagrange polynomials
6.2.3 Failure of Lagrange interpolation
6.2.4 Error analysis
6.2.5 Newton’s representation with divided differences
6.2.6 More on divided differences
6.3 Hermite interpolation
6.3.1 Computation of the Hermite interpolant
6.3.2 Error analysis
6.3.3 Implementation details
6.4 Spline interpolation
6.4.1 Continuous piecewise linear interpolation
6.4.2 Some theoretical considerations
6.4.3 Error analysis of piecewise linear interpolation
6.4.4 Cubic spline interpolation
6.4.5 An algorithm for cubic splines
6.4.6 Theoretical considerations and error analysis
6.4.7 B-splines
6.5 Method of least squares
6.5.1 Linear least squares
6.5.2 Polynomial fit
6.5.3 Non-polynomial fit
6.6 The module scipy.interpolate
6.6.1 Lagrange interpolation
6.6.2 Cubic spline interpolation
6.6.3 Computations with B-splines
6.6.4 General purpose interpolation and the function interp1d
6.6.5 Interpolation in two dimensions
6.6.6 Application in image processing
6.7 Further reading
Chapter highlights
Exercises
7 Numerical integration
7.1 Introduction to numerical quadrature and midpoint rule
7.2 Newton-Cotes quadrature rules
7.2.1 Trapezoidal rule
7.2.2 Error estimate of the trapezoidal rule
7.2.3 Composite trapezoidal rule
7.2.4 Error estimate of the composite trapezoidal rule
7.2.5 Simpson’s rule
7.2.6 Composite Simpson’s rule
7.2.7 Error of Simpson’s rule
7.2.8 Degree of exactness
7.3 Gaussian quadrature rules
7.3.1 Choice of nodes and weights
7.3.2 Gauss-Legendre quadrature rules
7.3.3 Gaussian quadrature on general intervals
7.3.4 Error analysis of the Gaussian quadrature
7.4 The module scipy.integrate
7.4.1 Trapezoidal rule
7.4.2 Simpson’s rule
7.4.3 Gaussian quadrature
7.4.4 Application in classical mechanics
7.5 Further reading
Chapter highlights
Exercises
8 Numerical differentiation and applications to differential equations
8.1 Numerical differentiation
8.1.1 First-order accurate finite difference approximations
8.1.2 Second-order accurate finite difference approximations
8.1.3 Error estimation
8.1.4 Approximation of second-order derivatives
8.1.5 Richardson’s extrapolation
8.2 Applications to ordinary differential equations
8.2.1 First-order ordinary differential equations
8.2.2 Euler method
8.2.3 Alternative derivation and error estimates
8.2.4 Implicit variants of Euler’s method
8.2.5 Improved Euler method
8.2.6 The notion of stability
8.3 Runge-Kutta methods
8.3.1 Explicit Runge-Kutta methods
8.3.2 Implicit and diagonally implicit Runge-Kutta methods
8.3.3 Adaptive Runge-Kutta methods
8.4 The module scipy.integrate again
8.4.1 Generic integration of initial value problems
8.4.2 Systems of ordinary differential equations
8.4.3 Application in epidemiology
8.5 Further reading
Chapter highlights
Exercises
9 Numerical linear algebra
9.1 Numerical solution of linear systems
9.1.1 Algorithm for the naive Gaussian elimination
9.1.2 LU factorization
9.1.3 Implementation of LU factorization
9.2 Pivoting strategies
9.2.1 Gaussian elimination with partial pivoting
9.2.2 LU factorization with pivoting
9.3 Condition number of a matrix
9.3.1 Vector and matrix norms
9.3.2 Theoretical properties of matrix norms
9.3.3 Condition number
9.4 Other matrix computations
9.4.1 Computation of inverse matrix
9.4.2 Computation of determinants
9.5 Symmetric matrices
9.5.1 LDLT factorization
9.5.2 Cholesky factorization
9.6 The module scipy.linalg again
9.6.1 LU factorization
9.6.2 Cholesky factorization
9.7 Iterative methods
9.7.1 Derivation of iterative methods
9.7.2 Classical iterative methods – Jacobi and Gauss-Seidel methods
9.7.3 Convergence of iterative methods
9.7.4 Sparse matrices in Python and the module scipy.sparse
9.7.5 Sparse Gauss-Seidel implementation
9.7.6 Sparse linear systems and the module scipy.sparce
9.7.7 Application in electric circuits
9.8 Further reading
Chapter highlights
Exercises
- Advanced topics
10 Best approximations
10.1 Vector spaces, norms, and approximations
10.1.1 Vector spaces
10.1.2 Inner products and norms
10.1.3 Best approximations
10.2 Linear least squares
10.2.1 Theoretical properties of linear least squares
10.2.2 Solution of linear least squares problem
10.2.3 Linear least squares problem in Python
10.3 Gram-Schmidt orthonormalization
10.3.1 Numerical implementation
10.4 QR factorization
10.4.1 Linear least squares using QR factorization
10.4.2 Computation of range
10.4.3 Householder matrices and QR factorization
10.4.4 Python implementation and orthonormalization
10.5 Singular value decomposition
10.5.1 Derivation of SVD
10.5.2 Theoretical considerations
10.5.3 Pseudoinverse and SVD
10.5.4 Linear least squares problem and SVD
10.5.5 Singular value decomposition in Python
10.5.6 Application in image compression
10.6 Further reading
Chapter highlights
Exercises
11 Unconstrained optimization and neural networks
11.1 Gradient methods for unconstrained optimization
11.1.1 Estimation of convergence
11.1.2 Method of steepest descent
11.1.3 Solving linear systems using optimization
11.1.4 Theoretical considerations
11.2 Conjugate gradient method
11.2.1 Conjugate gradient method and linear systems
11.2.2 Convergence and preconditioning
11.2.3 Extensions to nonlinear systems
11.3 Newton’s method in optimization
11.3.1 Newton’s iteration
11.3.2 Quasi-Newton methods
11.3.3 Nonlinear optimization and the module scipy.optimize
11.3.4 The Gauss-Newton approximation
11.3.5 The Levenberg-Marquardt modification
11.4 Introduction to neural networks
11.4.1 A simple neural network
11.4.2 Activation functions
11.4.3 General feedforward neural networks
11.4.4 Machine learning and the module sklearn
11.4.5 Application in image recognition
11.5 Further reading
Chapter highlights
Exercises
12 Eigenvalue problems
12.1 Eigenvalues and eigenvectors
12.1.1 Basic properties
12.1.2 Diagonalization of a matrix and the Jordan normal form
12.1.3 Other properties
12.1.4 First estimation of eigenvalues using Gerschgorin disks
12.2 Numerical approximation of eigenvalues and eigenvectors
12.2.1 Power method
12.2.2 Inverse power method
12.2.3 Shifted inverse power method
12.2.4 Basic QR iteration
12.2.5 Application in computer networks
12.3 Further reading
Chapter highlights
Exercises
Appendix A Computing Jordan normal forms
Bibliography
Index
Biography
Dimitrios Mitsotakis received a PhD in Mathematics in 2007 from the University of Athens. His experience with high-performance computing started while at the Edinburgh Parallel Computing Center at the University of Edinburgh. Dimitrios worked at the University Paris-Sud as a Marie Curie fellow, at the University of Minnesota as an associate postdoc and at the University of California, Merced as a Visiting Assistant Professor. Dimitrios is currently an associate professor/reader at the School of Mathematics and Statistics of Victoria University of Wellington. He has published his work in journals of numerical analysis and in more general audience journals in physics, coastal engineering, waves sciences, and in scientific computing. He develops numerical methods for the solution of equations for water waves, and he studies real-world applications such as the generation of tsunamis. Some of his main contributions are in the theory and numerical analysis of Boussinesq systems for nonlinear and dispersive water waves.
