For a convex quadratic function (like the MSE loss in linear regression), the Lipschitz constant L of the gradient is equal to the largest eigenvalue of the Hessian.
Proof: Let’s define a general convex quadratic function: where , is a symmetric positive semi-definite matrix (to ensure convexity), , and . The gradient of this function is: Lipschitz Continuity A function is Lipschitz continuous…


