Mathematical Methods is the practical side of mathematics: when an exact answer does not exist or is too costly, how do you get a good numerical one? That is what a computer does all day. Linear algebra sits under machine learning and graphics, and numerical methods sit under every simulation, solver and optimiser. This article revisits the five units of the JNTUK course in short form.
Unit 1: Solution of linear systems
A system of linear equations is written Ax = b, with A the coefficient matrix, x the unknowns and b the constants.
Matrix basics
- Rank of a matrix: the number of linearly independent rows (found by reducing to echelon form with row operations).
- Consistency: Ax = b has a solution only if rank(A) = rank([A | b]).
| Condition | Result |
|---|---|
| rank(A) = rank([A | b]) = number of unknowns | Unique solution |
| rank(A) = rank([A | b]) < number of unknowns | Infinitely many solutions |
| rank(A) < rank([A | b]) | No solution (inconsistent) |
A homogeneous system (Ax = 0) always has the trivial solution x = 0, and has non-trivial solutions when rank(A) is less than the number of unknowns, that is when det(A) = 0 for a square matrix.
Direct and iterative methods
| Method | Idea | Note |
|---|---|---|
| Gauss elimination | Reduce to upper triangular form, then back-substitute | Cost grows as n³ |
| Gauss-Jordan | Reduce all the way to the identity; also finds the inverse | More work than elimination |
| LU decomposition | Factor A = LU; solve two triangular systems | Reuses the factorisation for many b |
| Jacobi iteration | Update every unknown from the previous values of the others | Needs diagonal dominance to converge |
| Gauss-Seidel | Same, but uses newly computed values immediately | Usually converges faster than Jacobi |
Pivoting (swapping rows to put the largest element on the diagonal) controls rounding error.
Today: a large share of scientific computing is solving Ax = b: circuit simulation, structural analysis, weather models and the normal equations of linear regression. Libraries such as LAPACK and NumPy do this with optimised versions of LU.
Unit 2: Eigenvalues, eigenvectors and quadratic forms
For a square matrix A, a non-zero vector x is an eigenvector with eigenvalue λ if
A x = λ x
The matrix only stretches that vector, without changing its direction. Eigenvalues come from the characteristic equation det(A − λI) = 0.
Key properties
| Property | Statement |
|---|---|
| Sum of eigenvalues | Equals the trace of A (sum of the diagonal) |
| Product of eigenvalues | Equals det(A) |
| Triangular matrix | Eigenvalues are the diagonal entries |
| Powers | Ak has eigenvalues λk |
| Symmetric matrix | Real eigenvalues, orthogonal eigenvectors |
The Cayley-Hamilton theorem says every matrix satisfies its own characteristic equation, which gives a quick way to find A⁻¹ and powers of A. Diagonalisation writes A = PDP⁻¹, where D holds the eigenvalues and P the eigenvectors; it makes Ak easy to compute.
Quadratic forms
A quadratic form Q = xᵀAx (for example 2x² + 3xy + y²) can be reduced to a sum of squares by an orthogonal transformation. Its nature is read from the eigenvalues of A:
| Eigenvalues | Nature |
|---|---|
| All positive | Positive definite |
| All negative | Negative definite |
| Mixed signs | Indefinite |
| Some zero | Semi-definite |
Today: eigenvectors are behind Google’s original PageRank, principal component analysis (PCA) for reducing data, face recognition and vibration analysis. A covariance matrix is positive semi-definite, and a positive-definite Hessian means you are at a true minimum when training a model.
Unit 3: Solution of nonlinear equations
To solve f(x) = 0 when there is no formula, start from an interval or guess and improve it step by step. A root is first bracketed by finding a and b with f(a) f(b) < 0.
| Method | Update rule | Convergence |
|---|---|---|
| Bisection | Halve the interval, keep the side where f changes sign | Slow but always works (linear) |
| Regula falsi | Replace the interval by where the straight line through the ends crosses zero | Usually faster than bisection |
| Secant | xn+1 = xn − f(xn) (xn − xn−1) / (f(xn) − f(xn−1)) | Fast (order about 1.62), no derivative needed |
| Newton-Raphson | xn+1 = xn − f(xn) / f′(xn) | Very fast (quadratic) near the root, needs f′ |
| Fixed-point iteration | Rewrite as x = g(x) and iterate xn+1 = g(xn) | Converges if |
A small example: to find √2 by Newton’s method, solve x² − 2 = 0, giving xn+1 = (xn + 2/xn) / 2. Starting at 1, you get 1.5, 1.4167, 1.41422, correct to five places in three steps.
The order of convergence describes how fast the error shrinks: with quadratic convergence the number of correct digits roughly doubles each step. Newton’s method can fail if f′ is near zero or the guess is poor.
Today: Newton’s method is how calculators compute square roots, how solvers find the operating point of a circuit and how optimisers train logistic regression. Bisection is the same idea as
git bisect, which halves a commit range to find where a bug appeared.
Unit 4: Interpolation, curve fitting and numerical integration
Interpolation
Interpolation estimates a value between known data points by fitting a polynomial that passes through them.
| Method | Use |
|---|---|
| Newton forward and backward difference | Equally spaced points; near the start or end of the table |
| Lagrange | Unequally spaced points |
| Newton divided difference | Unequally spaced points, easy to add new points |
A polynomial through n + 1 points has degree at most n, and is unique. Interpolation error grows if the polynomial degree is high or you extrapolate beyond the data.
Curve fitting
When data is noisy, passing exactly through every point is wrong. Least squares chooses the curve minimising the sum of squared errors. For a straight line y = a + bx the normal equations are:
Σy = na + bΣx and Σxy = aΣx + bΣx²
The same idea fits parabolas, exponentials (after taking logs) and power laws.
Numerical differentiation and integration
Differences approximate derivatives; areas approximate integrals.
| Rule | Idea | Needs |
|---|---|---|
| Trapezoidal | Join points with straight lines | Any number of intervals |
| Simpson’s 1/3 | Fit parabolas over pairs of intervals | An even number of intervals |
| Simpson’s 3/8 | Fit cubics over triples | A multiple of 3 intervals |
Simpson’s rule is much more accurate than the trapezoidal rule for the same number of points, because it fits curves instead of lines.
Today: line of best fit is linear regression. Interpolation is how images are resized and how a game fills the frames between two animation key points. Numerical integration is how simulators and financial models compute quantities that have no formula.
Unit 5: Numerical solution of ordinary differential equations
For an initial value problem dy/dx = f(x, y), y(x₀) = y₀, step forward from the known point in small steps of size h.
| Method | Idea | Accuracy |
|---|---|---|
| Taylor series | Expand y around x₀ using derivatives | Depends on the terms kept |
| Euler | yn+1 = yn + h f(xn, yn) | First order: simple but error grows |
| Modified Euler (Heun) | Average the slope at the start and the predicted end | Second order |
| Runge-Kutta, 2nd order | Uses two slope estimates per step | Second order |
| Runge-Kutta, 4th order | Weighted average of four slopes per step | Fourth order, the common default |
| Predictor-corrector (Milne, Adams-Bashforth-Moulton) | Predict with one formula, correct with another, reusing past points | Efficient for smooth problems |
The classical fourth-order Runge-Kutta step is:
yn+1 = yn + (k₁ + 2k₂ + 2k₃ + k₄) / 6
where k₁ = h f(xn, yn), and k₂, k₃, k₄ are slopes at the midpoint and the end. Halving the step size h cuts the error of Euler by about 2 and of Runge-Kutta 4 by about 16.
Stability matters: for some (“stiff”) problems a large step makes the numerical solution blow up even though the true one is calm, so specialised solvers are used.
Today: SciPy’s
solve_ivp, MATLAB’sode45and physics engines in games use Runge-Kutta methods. Climate, epidemic and chemical reaction models, and the neural ODEs of machine learning, are all differential equations solved step by step.
One-minute recap
| Unit | Key idea | You meet it in |
|---|---|---|
| 1. Linear systems | Rank decides solutions; Gauss, LU, Jacobi, Gauss-Seidel | Regression, simulations, solvers |
| 2. Eigenvalues | A x = λ x; diagonalisation; definiteness | PCA, PageRank, optimisation |
| 3. Nonlinear equations | Bisection, secant, Newton-Raphson | Square roots, solvers, git bisect |
| 4. Interpolation and integration | Newton and Lagrange; least squares; trapezoidal and Simpson | Image resizing, curve fitting, simulation |
| 5. ODEs | Euler, Runge-Kutta, predictor-corrector | Physics engines, climate and epidemic models |
Next in Engineering Revisited: Advanced Data Structures & Algorithms.
Written by Claude, Anthropic’s AI assistant, as part of Engineering Revisited, a quick refresher of the B.Tech subjects one at a time.