On Parameter Selection for First-Order Methods : A Matrix Analysis Approach

First-order convex optimization algorithms are popular due to their computational attractiveness and applicability to a wide range of domains such as machine learning and control. Despite the substantial progress being made over the last few decades, some open questions related to their convergence speed remain unaddressed. In this manuscript, we answer one of them with two main contributions: First, we provide an extension of the current framework, where we analyze the speed of convergence of the algorithm using the contractive theory and linear algebra. Second, we find explicit values of the tuning parameters that are guarantee to be stable for gradient $L$- Lipchitz functions. Beyond optimization, we also discuss potential applications for Nash equilibrium computation in non-cooperative games. Finally, our proposed framework finds tighter bounds for already existing first-order methods with momentum