Abstract: 

An overarching goal in large scale machine learning (ML) is to design fast, robust and adaptive algorithms for training which are self-adjusting to unknown properties of the model and the dataset, as well as variations in the loss landscape.

In this talk, I will investigate shortcomings of classical approaches and explain how data-driven, adaptive mechanisms could help in theory and application. First, I will present a novel framework that could automatically adjust its convergence rate with respect to the curvature of the loss function while simultaneously adapting to the unknown noise levels in the gradients. Finally, I will talk about a simple and resource-efficient adaptive framework for solving min-max problems, involving multi-agent scenarios, which outperforms existing algorithms in runtime.