Thematic Einstein Semester on Mathematical Optimization for Machine Learning

Mathematical optimization often focuses on accuracy, computational efficiency, and robustness while machine learning (ML) aims at achieving effective results on real datasets, in particular aiming for generalization, robustness, and resilience (to e.g., perturbations of the inputs). Up to now, both research areas are only loosely intertwined with optimization being the ‘tool’ to execute the learning task from the ML perspective, and ML being just ‘yet another application’ from the optimization perspective. This is illustrated by the fact that certain variants of stochastic gradient descent (e.g., out-of-the-box Adam) are still the method of choice in machine learning despite the fact that the application of this method is not justified by theory for a large number of applications from machine learning. On the other hand, mathematical optimization often studies approximation properties for machine learning tasks but does so far not develop optimization approaches targeting the needs of machine learning (e.g., aiming for generalization or sparsity). Hence, there remains a dearth of work to advance optimization techniques to such an extent that machine learning problems can be handled with the required efficiency. Connecting mathematical optimization perspectives with machine learning approaches will help to generate ideas for new approaches or improvements in existing approaches. For instance, the problem of predicting cluster membership in unsupervised learning may be greatly aided by an optimization perspective; after all, as with many other data science approaches, the problem can be viewed as maximizing a (complicated) function subject to some constraints. Complementing this, machine learning heavily influences downstream decision making and more generally decision support systems in many real-world applications across industries. One prominent set of examples of such approaches that made headway is motivated by tasks originating from the transition to renewable energy sources including energy generation, storage, transmission and delivery, and trading fitting perfectly to the MATH+ slogan “Transforming the World through Mathematics”, AA3 “Networks” as well as AA4 “Energy and Markets”. As such the proposed TES naturally connects to MATH+ and sets out to explore new perspectives that might lead to new developments that will likely also impact Math+ research beyond the TES. Moreover, the TES will also help to maintain the current momentum in mathematical optimization and machine learning for cross-disciplinary research between the two fields