Provable Accelerated Convergence of Nesterov’s Momentum for Deep ReLU Neural Networks

In this talk, I will discuss the result in my paper “Provable Accelerated Convergence of Nesterov’s Momentum for Deep ReLU Neural Networks”, which introduces the notion of “partially strong convexity” that is tailored to the analysis of over-parameterized neural networks. I will also discuss the future directions of extending the framework to the so-called “feature learning” regime.

—

ICCOPT 2025