On the Nesterov's acceleration: A NAIM perspective

摘要

We present a unifying Nearly Asymptotically Invariant Manifold (NAIM) framework for understanding Nesterovs Accelerated Gradient (NAG) method. By lifting the first-order gradient flow into a second-order phase space we construct a NAIM a slow, attracting graph and show that acceleration emerges from a curvature aware perturbation of this graph. The evolving slope of the perturbed manifold is governed by a Differential Riccati Equation (DRE), which enforces strict tangency of the vector field to the manifold surface. In the quadratic case the DRE reduces to an Algebraic Riccati Equation (ARE), and the requirement of spectral resonance equal contraction rates across all curvature modes uniquely determines the damping coefficient, directly yielding the continuous time Nesterov ODE. Fenichels theorem then extends this picture rigorously to general smooth, strongly convex landscapes: normal hyperbolicity guarantees persistence of the accelerated manifold despite varying Hessian curvature. The method is further extended to unified geometric derivation of NAG methods for smooth convex and strongly convex optimization in the discrete case. We exploit the underlying geometric structure and derive both cases from the same principle of preserving the projective structure under discretization process. A Lie Trotter splitting separates the linear dissipative dynamics from the nonlinear gradient flow. The dissipative subsystem is integrated by the Cayley (bilinear) transform, which preserves the underlying projective (Mobius) structure unconditionally and produces the classical Nesterov momentum coefficient as the unique Pade multiplier. For the convex case, projective flatness (vanishing Schwarzian derivative) uniquely selects the time-varying damping recovering the canonical Nesterov ODE for convex functions.