Cheatsheet: Higher-order derivatives
The idea
Section titled “The idea”A derivative is a function, so differentiate again:
f -> f' -> f'' -> f''' -> f^(4) -> ...Leibniz: d²f/dx², d³f/dx³, ...Physics interpretation
Section titled “Physics interpretation”position s(t) -> velocity s'(t) -> acceleration s''(t) -> jerk s'''(t)F = ma: force = mass times the second derivative of position.
Concavity interpretation
Section titled “Concavity interpretation”f' is the slope; f'' is how the slope changes (curvature):
f'' | Shape |
|---|---|
> 0 | Cups upward (smiling); slope increasing |
< 0 | Cups downward (frowning); slope decreasing |
= 0 with sign change | Inflection point (curvature flips) |
The second-derivative test
Section titled “The second-derivative test”At a critical point (f'(x) = 0):
f''(x) > 0-> local minimum (cup holds water)f''(x) < 0-> local maximumf''(x) = 0-> inconclusive; look closer
Worked examples
Section titled “Worked examples”f | f' | f'' | Reading |
|---|---|---|---|
-16t^2+64t+32 | -32t+64 | -32 | acceleration = gravity (constant) |
x^3 - 3x | 3x^2-3 | 6x | min at x=1 (-2), max at x=-1 (2), inflection at 0 |
sin x | cos x | -sin x | f'' = -f (oscillation equation) |
e^x | e^x | e^x | every derivative is e^x |
Free-fall at t=2: velocity 0 (peak), height 96 ft, acceleration -32.
Why it matters for AI
Section titled “Why it matters for AI”Second-order (curvature) information drives optimization:
- Newton’s method uses the Hessian (matrix of second derivatives) to converge faster than gradient descent on well-behaved problems.
- Adam keeps a running estimate of how the gradient changes (an informal curvature signal); K-FAC approximates the Hessian.
- Loss-landscape analysis (saddle points, flat basins, sharp ridges) is second-derivative geometry.
Pitfalls to dodge
Section titled “Pitfalls to dodge”- f” is the slope. No,
f'is the slope;f''is how the slope changes. - Flipping the second-derivative test.
f'' > 0= minimum (cup holds water),f'' < 0= maximum. f'' = 0always means inflection. Only if the curvature actually changes sign.- Stopping at velocity. Second derivative of position is acceleration, not velocity.
The one-line version
Section titled “The one-line version”The second derivative is the rate the slope changes: acceleration in time, curvature on a graph, and the test that tells a minimum (f''>0) from a maximum (f''<0).