Analysis of gradient descent algorithms: Discrete to continuous domains and circuit equivalents

Abstract

In recent years, there have been several advances in iterative optimization algorithms seen as closed-loop control systems in discrete-time that solve unconstrained optimization problems. In this paper, we extend these advances to the continuous setting and leverage circuit equivalence to present a possible implementation of such controllers. Next, we address constrained Quadratic Programming (QP) challenges within a primal–dual framework that appears in many controller definitions. By drawing parallels between second-order ODEs and circuit dynamics, our study bridges theoretical optimization with practical electrical analogues that can be designed and implemented for problems in systems engineering, robotics, and autonomous vehicles that could benefit from the low power and latency of these optimization-based controllers.