Skip to content

For a convex quadratic function (like the MSE loss in linear regression), the Lipschitz constant L of the gradient is equal to the largest eigenvalue of the Hessian.

Proof: Let’s define a general convex quadratic function: where , is a symmetric positive semi-definite matrix (to ensure convexity), , and . The gradient of this function is: Lipschitz Continuity A function is Lipschitz continuous… For a convex quadratic function (like the MSE loss in linear regression), the Lipschitz constant L of the gradient is equal to the largest eigenvalue of the Hessian.

error: Content is protected !!