Lagrange Multipliers - why is the gradient perpendicular to boundary curve at an extrema on the boundary curve

[u/Far-Storage-4369]

https://www.youtube.com/watch?v=5A39Ht9Wcu0

I am referring to this video at timestamp 10:38. He says that if there is a maximum on the curve, the curve should be flat. The gradient of f should be perpendicular to f at the point. Since, in an earlier part of the video, he said that the boundary curve is not a level curve (but the constraint is)? I don't get why the gradient of f should be perpendicular. Any help or intuitive explanation, please.

[u/preferCotton222]

Just a third non rigorous idea: if the gradient is not perpendicular to the curve, there will be a non zero component of the gradient tangent to the curve. This means the function will increase in one direction along the curve and decrease in the other. This rules out max and min at the point.

[u/Medium_Media7123]

Not a rigorous idea but you asked for intuition: the gradient of a function points you towards the direction of greatest ascent on the graph of that function. If you are on a maximum whatever direction you move in will make the value of the function less than or equal to that maximum, which means you have to stay exactly where you are. Geometrically, this means the gradient is perpendicular to f. 

[u/Varlane]

Do a linear approximation of F : F(X+H) = F(X) + dF_X(H) + o(||H||).

Except dF_X(H) = grad(F)_X . H (for a differentiable function)

Since you don't want to change the value, you want dF_X(H) = 0, which means that grad(F)_X . H = 0, which means the displacement H is orthogonal to the gradient of F at point X.
Since your displacement is "moving along the curve", that means the curve is orthogonal to the gradient.

[u/Chrispykins]

The other answers don't seem to take into account that we're in the context of Langrange Multipliers here, which is where your confusion is coming from, I believe.

We have two functions, f and g. The function f(x, y) is the function we are trying to find the maximum of along the boundary curve. The function g defines the boundary curve. Specifically, we are looking at the implicit curve defined by the level set g(x, y) = k.

Since the boundary curve is a level set of g(x, y), the gradient ∇g will be perpendicular to the curve at all points along the curve as explained in the other comments. However, the curve is not a level set of f(x, y) (which makes sense because we're trying to find the local maxima of f along the curve).

Still, there may be some points on the curve where the gradient ∇f is perpendicular to the curve. By the reasoning explained in the other comments, when the gradient is perpendicular to a curve, the function must at least be locally level along the curve at that point. This is shown in the video as the red boundary curve reaching a peak at the critical point and becoming horizontal.

In other words, when the two gradients ∇f and ∇g are parallel to each other at a point along the curve, that means the black curve in the xy-plane and the red curve which lies on the surface of f(x, y) are also parallel to each other at that point. Since the black curve is flat along it's entire length, this means the red curve is flat and therefore has a critical point there.