1st Edition

Computational Mathematics An introduction to Numerical Analysis and Scientific Computing with Python

By Dimitrios Mitsotakis Copyright 2023
    528 Pages 92 B/W Illustrations
    by Chapman & Hall

    528 Pages 92 B/W Illustrations
    by Chapman & Hall

    This textbook is a comprehensive introduction to computational mathematics and scientific computing suitable for undergraduate and postgraduate courses. It presents both practical and theoretical aspects of the subject, as well as advantages and pitfalls of classical numerical methods alongside with computer code and experiments in Python. Each chapter closes with modern applications in physics, engineering, and computer science.


    • No previous experience in Python is required.
    • Includes simplified computer code for fast-paced learning and transferable skills development.
    • Includes practical problems ideal for project assignments and distance learning.
    • Presents both intuitive and rigorous faces of modern scientific computing.
    • Provides an introduction to neural networks and machine learning.


      1. Introduction to scientific computing with Python
      2. 1 Introduction to Python

        1.1 Basic Python

        1.1.1 Python, numbers and variables

        1.1.2 Basic mathematical operations

        1.1.3 Mathematical functions and constants

        1.1.4 Powers of 10 and scientific notation

        1.2 Assignment operators

        1.3 Strings, tuples, dictionaries and lists

        1.3.1 Strings

        1.3.2 Tuples

        1.3.3 Dictionaries

        1.3.4 Lists

        1.3.5 Lists of lists

        1.4 Flow control

        1.4.1 for loops

        1.4.2 Useful applications

        1.4.3 Boolean type of variables

        1.4.4 while loops

        1.4.5 Decisions and the if statement

        1.4.6 Error control and exceptions

        1.4.7 The command break

        1.5 Functions

        1.6 Classes

        1.7 Accessing data files

        1.8 Further reading

        Chapter highlights


        2 Matrices and Python

        2.1 Review of matrices

        2.1.1 Matrix properties

        2.1.2 Special matrices

        2.2 The NumPy module

        2.2.1 Matrices and arrays

        2.2.2 Slicing

        2.2.3 Special matrices, methods and properties

        2.2.4 Matrix assignment

        2.2.5 Matrix addition and scalar multiplication

      2.2.6 Vectorized algorithms

      2.2.7 Performance of algorithms

      2.2.8 Matrix multiplication

      2.2.9 Inverse of a matrix

      2.2.10 Determinant of a matrix

      2.2.11 Linear systems

      2.2.12 Eigenvalues and eigenvectors

      2.3 The SciPy module

      2.3.1 Systems with banded matrices

      2.3.2 Systems with positive definite, banded matrices

      2.3.3 Systems with other special matrices

      2.4 Drawing graphs with MatPlotLib

      2.4.1 Plotting one-dimensional arrays

      2.5 Further reading

      Chapter highlights


      3 Scientific computing

      3.1 Computer arithmetic and sources of error

      3.1.1 Catastrophic cancelation

      3.1.2 The machine precision

      3.2 Normalized scientific notation

      3.2.1 Floats in Python

      3.3 Rounding and chopping

      3.3.1 Absolute and relative errors

      3.3.2 Loss of significance (again)

      3.3.3 Algorithms and stability

      3.3.4 Approximations of π

      3.4 Further reading

      Chapter highlights


      4 Calculus facts

      4.1 Sequences and convergence

      4.2 Continuous functions

      4.2.1 Bolzano’s intermediate value theorem

      4.3 Differentiation

      4.3.1 Important theorems from differential calculus

      4.3.2 Taylor polynomials

      4.3.3 The big-O notation

      4.4 Integration

      4.4.1 Approximating integrals

      4.4.2 Important theorems of integral calculus

      4.4.3 The remainder in Taylor polynomials

      4.5 Further reading

      Chapter highlights


    1. Introduction to computational mathematics

    5 Roots of equations

    5.1 Bisection method

    5.1.1 Derivation and implementation

    5.1.2 Proof of convergence

    5.1.3 Rate of convergence

    5.2 Fixed-point methods

    5.2.1 Derivation and implementation

    5.2.2 Theoretical considerations

    5.2.3 Convergence of fixed-point methods

    5.3 Newton’s method

    5.3.1 Derivation and implementation

    5.3.2 Rate of convergence

    5.4 Secant method

    5.4.1 Derivation and implementation

    5.4.2 Rate of convergence

    5.5 Other methods and generalizations

    5.5.1 Stopping criteria

    5.5.2 Steffensen’s method

    5.5.3 Aitken’s delta-squared process

    5.5.4 Multiple roots

    5.5.5 High-order methods

    5.5.6 Systems of equations

    5.6 The module scipy.optimize

    5.6.1 Bisection method

    5.6.2 Fixed-point methods

    5.6.3 Newton and secant methods

    5.6.4 A hybrid method

    5.6.5 Application in astrophysics

    5.7 Further reading

    Chapter highlights


    6 Interpolation and approximation

    6.1 Introduction

    6.2 Lagrange interpolation

    6.2.1 Naive construction of the interpolant

    6.2.2 Lagrange polynomials

    6.2.3 Failure of Lagrange interpolation

    6.2.4 Error analysis

    6.2.5 Newton’s representation with divided differences

    6.2.6 More on divided differences

    6.3 Hermite interpolation

    6.3.1 Computation of the Hermite interpolant

    6.3.2 Error analysis

    6.3.3 Implementation details

    6.4 Spline interpolation

    6.4.1 Continuous piecewise linear interpolation

    6.4.2 Some theoretical considerations

    6.4.3 Error analysis of piecewise linear interpolation

    6.4.4 Cubic spline interpolation

    6.4.5 An algorithm for cubic splines

    6.4.6 Theoretical considerations and error analysis

    6.4.7 B-splines

    6.5 Method of least squares

    6.5.1 Linear least squares

    6.5.2 Polynomial fit

    6.5.3 Non-polynomial fit

    6.6 The module scipy.interpolate

    6.6.1 Lagrange interpolation

    6.6.2 Cubic spline interpolation

    6.6.3 Computations with B-splines

    6.6.4 General purpose interpolation and the function interp1d

    6.6.5 Interpolation in two dimensions

    6.6.6 Application in image processing

    6.7 Further reading

    Chapter highlights


    7 Numerical integration

    7.1 Introduction to numerical quadrature and midpoint rule

    7.2 Newton-Cotes quadrature rules

    7.2.1 Trapezoidal rule

    7.2.2 Error estimate of the trapezoidal rule

    7.2.3 Composite trapezoidal rule

    7.2.4 Error estimate of the composite trapezoidal rule

    7.2.5 Simpson’s rule

    7.2.6 Composite Simpson’s rule

    7.2.7 Error of Simpson’s rule

    7.2.8 Degree of exactness

    7.3 Gaussian quadrature rules

    7.3.1 Choice of nodes and weights

    7.3.2 Gauss-Legendre quadrature rules

    7.3.3 Gaussian quadrature on general intervals

    7.3.4 Error analysis of the Gaussian quadrature

    7.4 The module scipy.integrate

    7.4.1 Trapezoidal rule

    7.4.2 Simpson’s rule

    7.4.3 Gaussian quadrature

    7.4.4 Application in classical mechanics

    7.5 Further reading

    Chapter highlights


    8 Numerical differentiation and applications to differential equations

    8.1 Numerical differentiation

    8.1.1 First-order accurate finite difference approximations

    8.1.2 Second-order accurate finite difference approximations

    8.1.3 Error estimation

    8.1.4 Approximation of second-order derivatives

    8.1.5 Richardson’s extrapolation

    8.2 Applications to ordinary differential equations

    8.2.1 First-order ordinary differential equations

    8.2.2 Euler method

    8.2.3 Alternative derivation and error estimates

    8.2.4 Implicit variants of Euler’s method

    8.2.5 Improved Euler method

    8.2.6 The notion of stability

    8.3 Runge-Kutta methods

    8.3.1 Explicit Runge-Kutta methods

    8.3.2 Implicit and diagonally implicit Runge-Kutta methods

    8.3.3 Adaptive Runge-Kutta methods

    8.4 The module scipy.integrate again

    8.4.1 Generic integration of initial value problems

    8.4.2 Systems of ordinary differential equations

    8.4.3 Application in epidemiology

    8.5 Further reading

    Chapter highlights


    9 Numerical linear algebra

    9.1 Numerical solution of linear systems

    9.1.1 Algorithm for the naive Gaussian elimination

    9.1.2 LU factorization

    9.1.3 Implementation of LU factorization

    9.2 Pivoting strategies

    9.2.1 Gaussian elimination with partial pivoting

    9.2.2 LU factorization with pivoting

    9.3 Condition number of a matrix

    9.3.1 Vector and matrix norms

    9.3.2 Theoretical properties of matrix norms

    9.3.3 Condition number

    9.4 Other matrix computations

    9.4.1 Computation of inverse matrix

    9.4.2 Computation of determinants

    9.5 Symmetric matrices

    9.5.1 LDLT factorization

    9.5.2 Cholesky factorization

    9.6 The module scipy.linalg again

    9.6.1 LU factorization

    9.6.2 Cholesky factorization

    9.7 Iterative methods

    9.7.1 Derivation of iterative methods

    9.7.2 Classical iterative methods – Jacobi and Gauss-Seidel methods

    9.7.3 Convergence of iterative methods

    9.7.4 Sparse matrices in Python and the module scipy.sparse

    9.7.5 Sparse Gauss-Seidel implementation

    9.7.6 Sparse linear systems and the module scipy.sparce

    9.7.7 Application in electric circuits

    9.8 Further reading

    Chapter highlights


      1. Advanced topics

    10 Best approximations

    10.1 Vector spaces, norms, and approximations

    10.1.1 Vector spaces

    10.1.2 Inner products and norms

    10.1.3 Best approximations

    10.2 Linear least squares

    10.2.1 Theoretical properties of linear least squares

    10.2.2 Solution of linear least squares problem

    10.2.3 Linear least squares problem in Python

    10.3 Gram-Schmidt orthonormalization

    10.3.1 Numerical implementation

    10.4 QR factorization

    10.4.1 Linear least squares using QR factorization

    10.4.2 Computation of range

    10.4.3 Householder matrices and QR factorization

    10.4.4 Python implementation and orthonormalization

    10.5 Singular value decomposition

    10.5.1 Derivation of SVD

    10.5.2 Theoretical considerations

    10.5.3 Pseudoinverse and SVD

    10.5.4 Linear least squares problem and SVD

    10.5.5 Singular value decomposition in Python

    10.5.6 Application in image compression

    10.6 Further reading

    Chapter highlights


    11 Unconstrained optimization and neural networks

    11.1 Gradient methods for unconstrained optimization

    11.1.1 Estimation of convergence

    11.1.2 Method of steepest descent

    11.1.3 Solving linear systems using optimization

    11.1.4 Theoretical considerations

    11.2 Conjugate gradient method

    11.2.1 Conjugate gradient method and linear systems

    11.2.2 Convergence and preconditioning

    11.2.3 Extensions to nonlinear systems

    11.3 Newton’s method in optimization

    11.3.1 Newton’s iteration

    11.3.2 Quasi-Newton methods

    11.3.3 Nonlinear optimization and the module scipy.optimize

    11.3.4 The Gauss-Newton approximation

    11.3.5 The Levenberg-Marquardt modification

    11.4 Introduction to neural networks

    11.4.1 A simple neural network

    11.4.2 Activation functions

    11.4.3 General feedforward neural networks

    11.4.4 Machine learning and the module sklearn

    11.4.5 Application in image recognition

    11.5 Further reading

    Chapter highlights


    12 Eigenvalue problems

    12.1 Eigenvalues and eigenvectors

    12.1.1 Basic properties

    12.1.2 Diagonalization of a matrix and the Jordan normal form

    12.1.3 Other properties

    12.1.4 First estimation of eigenvalues using Gerschgorin disks

    12.2 Numerical approximation of eigenvalues and eigenvectors

    12.2.1 Power method

    12.2.2 Inverse power method

    12.2.3 Shifted inverse power method

    12.2.4 Basic QR iteration

    12.2.5 Application in computer networks

    12.3 Further reading

    Chapter highlights


    Appendix A Computing Jordan normal forms




    Dimitrios Mitsotakis received a PhD in Mathematics in 2007 from the University of Athens. His experience with high-performance computing started while at the Edinburgh Parallel Computing Center at the University of Edinburgh. Dimitrios worked at the University Paris-Sud as a Marie Curie fellow, at the University of Minnesota as an associate postdoc and at the University of California, Merced as a Visiting Assistant Professor. Dimitrios is currently an associate professor/reader at the School of Mathematics and Statistics of Victoria University of Wellington. He has published his work in journals of numerical analysis and in more general audience journals in physics, coastal engineering, waves sciences, and in scientific computing. He develops numerical methods for the solution of equations for water waves, and he studies real-world applications such as the generation of tsunamis. Some of his main contributions are in the theory and numerical analysis of Boussinesq systems for nonlinear and dispersive water waves.