8 - Numerical Methods with SciPy and Curve Fitting — Topics & Learning Outcomes

Module Topics

Introduction to SciPy for Numerical Computing

• What is SciPy? — SciPy is an open-source Python library built on top of NumPy that provides a wide collection of algorithms for scientific and technical computing.
• SciPy vs NumPy: Complementary Roles — While NumPy handles array creation and basic mathematical operations, SciPy provides higher-level algorithms such as optimization, integration, and signal processing.
• Installing and Importing SciPy Submodules — SciPy submodules must be imported explicitly; importing the top-level scipy package does not automatically load all submodules.
• Overview of Numerical Problem Categories — This module focuses on three core numerical problem types: root finding, numerical integration (including ODEs), and curve fitting.

Root Finding with scipy.optimize

• Concept of Root Finding — Root finding is the process of locating values of x such that f(x) = 0, a fundamental task in engineering analysis such as finding equilibrium points or solving implicit equations.
• Bracket-Based Methods: brentq and bisect — Bracket-based methods require an interval [a, b] where the function changes sign, guaranteeing convergence to a root within that interval.
• Derivative-Based Methods: fsolve and newton — Derivative-based methods such as Newton-Raphson use the function's slope to converge rapidly to a root but may fail if started far from the solution.
• General Scalar Minimization and root_scalar — scipy.optimize.root_scalar provides a unified interface for choosing among multiple root-finding algorithms using a common function signature.
• Practical Considerations for Root Finding — Choosing an appropriate root-finding method depends on function continuity, availability of derivatives, and whether a bracketing interval can be identified.

Numerical Integration with scipy.integrate

• Numerical Quadrature: quad and dblquad — scipy.integrate.quad computes definite integrals of scalar functions over a finite or infinite interval using adaptive Gaussian quadrature.
• Integrating Tabular Data: simpson and trapezoid — When data is given as discrete arrays rather than a function, methods such as simpson and trapezoid approximate the integral from the sample points.
• Introduction to Ordinary Differential Equations (ODEs) — An ODE describes how a quantity changes with respect to an independent variable, and solve_ivp integrates such equations forward from a set of initial conditions.
• Solving ODEs with solve_ivp — scipy.integrate.solve_ivp is the modern interface for integrating systems of ODEs, supporting multiple integration methods and dense output.
• Defining the Derivative Function for solve_ivp — The user-defined function passed to solve_ivp must accept time t and state vector y and return the vector of derivatives dy/dt.
• Interpreting and Visualizing ODE Solutions — Once solve_ivp returns a result, the solution arrays can be plotted with Matplotlib to visualize the system's transient and steady-state behavior.

Linear Curve Fitting with numpy.polyfit

• What is Curve Fitting? — Curve fitting finds the parameters of a mathematical model that minimize the difference between the model's predictions and observed data points.
• Using numpy.polyfit for Polynomial Regression — numpy.polyfit fits a polynomial of specified degree to x-y data using the least-squares method and returns the polynomial coefficients.
• Selecting Polynomial Degree — Choosing the right polynomial degree requires balancing fit accuracy with model simplicity, guided by domain knowledge and residual analysis.
• Linearization of Nonlinear Data — Some nonlinear relationships such as exponential or power-law models can be transformed into linear form, allowing numpy.polyfit to be applied after a variable substitution.
• Generating and Plotting Fitted Curves — After obtaining polynomial coefficients, a smooth fitted curve is generated by evaluating the polynomial on a dense array of x values and overlaying it on the scatter plot of the data.

Nonlinear Curve Fitting with scipy.optimize.curve_fit

• When to Use Nonlinear Fitting — Nonlinear curve fitting is required when the relationship between variables cannot be described by a polynomial or linearized form, and the model has parameters that appear nonlinearly.
• Defining a Model Function for curve_fit — The user supplies a Python function that takes the independent variable as its first argument followed by the model parameters, returning the predicted dependent variable values.
• Calling curve_fit and Interpreting Output — scipy.optimize.curve_fit returns the optimal parameter array popt and the covariance matrix pcov, which together describe the best-fit model and parameter uncertainty.
• Setting Initial Guesses and Bounds — Providing good initial parameter estimates via the p0 argument and restricting parameter ranges with bounds improves convergence and prevents physically unreasonable solutions.
• Handling Convergence Failures — If curve_fit cannot converge, it raises a RuntimeError; diagnosing and resolving convergence failures is an important practical skill.

Residuals Analysis

• What are Residuals? — Residuals are the differences between observed data values and the corresponding values predicted by the fitted model, quantifying the pointwise error of the fit.
• Computing Residuals in Python — Residuals are computed by evaluating the fitted model at the original x data points and subtracting the predicted values from the observed y values.
• Residual Plots and What They Reveal — Plotting residuals against the independent variable (or fitted values) is the primary diagnostic tool for evaluating whether the chosen model is appropriate.
• Quantitative Fit Metrics — Alongside visual residual plots, quantitative metrics such as the coefficient of determination R² and root mean square error (RMSE) provide summary measures of fit quality.
• Applying Residuals Analysis to Engineering Datasets — On real engineering datasets, residuals analysis guides iterative model improvement by revealing systematic errors tied to physical phenomena not yet captured by the model.
• Overfitting and Model Complexity — Increasing model complexity will always reduce residuals on training data, but may degrade predictive performance on new data—a phenomenon known as overfitting.

Applying Numerical Methods to Engineering Datasets

• Structuring an End-to-End Numerical Workflow — A complete numerical workflow involves loading and inspecting data, selecting and applying an appropriate method, validating results, and communicating outputs clearly.
• Loading and Preprocessing Engineering Data — Real engineering datasets typically require preprocessing steps such as unit conversion, removal of missing values, and normalization before applying numerical methods.
• Combining Root Finding and Integration — Some engineering problems require both root finding and integration, for example locating where a cumulative quantity reaches a threshold by combining quad with a root solver.
• Fitting ODE Model Parameters to Experimental Data — A powerful workflow combines solve_ivp with curve_fit to estimate parameters of a dynamic model from time-series measurements.
• Communicating Numerical Results in Engineering Reports — Effective communication of numerical results requires presenting fitted parameters with uncertainties, fit quality metrics, and clearly labeled plots that support engineering decisions.

Student Learning Outcomes

By the end of this module, students will be able to:

Course Outcomes (reference)