Chapter 4: The Gradient and Optimization

The last chapter ended with two numbers at each point of a surface — the slice slopes $f_x$ and $f_y$. This chapter packs them into a single vector, $\nabla f = (f_x, f_y)$, and discovers that the packaging is the point. The gradient is not a bookkeeping device: it is a compass. At every point of the map it is perpendicular to the level curve through that point, it aims in the direction of steepest climb, and its magnitude is the climb rate. We verify all three claims the honest way — on a contour map carpeted with gradient arrows, where a theorem about perpendicularity becomes something you can check arrow by arrow.

With the compass in hand, we go hunting for the flat spots. Where $\nabla f = \mathbf{0}$ the terrain has a critical point, and a contour map betrays its character before any algebra: nested ovals close around a peak or a pit, while the level curve through a saddle crosses itself in an X. Reading those signatures — and then confirming each read with the second derivative test — turns optimization from formula-hunting into map-reading. One thing the map alone cannot do is tell a peak from a pit, and that ambiguity is honest mathematics with an honest resolution.

Finally we constrain the hunt. Most real optimization happens on a leash — maximize this, subject to that — and the leash changes the geometry: the best point on a constraint curve is almost never a critical point of $f$ itself. It is the point where the constraint curve kisses a level curve of $f$ tangentially, which is the same as saying the two gradients line up: $\nabla f = \lambda \nabla g$. That one equation, Lagrange's, closes the chapter — derived not from algebraic manipulation but from watching where a ribbon of function values, ridden along the constraint, stalls.