The DFG excellence cluster MATH+ in Berlin organizes a Thematic Einstein Semester on Optimization and Machine Learning during the summer term 2023. As a part of the TES, a focused workshop titled “Quintessential Learning Components: Diverse Domains of Optimization” will take place. This workshop, scheduled to take place from June 14 – 16, 2023 at the Humboldt-Universität zu Berlin, would be a small in-person meeting. The expectation is to host nearly 50 participants and the presentations will be divided into three sessions.
Scope of the Semester
Our workshop will roughly constitute sixteen talks. This includes both regular (with 25-30 minutes for the presentation and 5-10 minutes for questions) and keynote presentations (lasting 45 minutes with an additional 15 minutes for questions). The talks will focus on current research at the interstices of diverse aspects of Optimization, which are closely tied to data-science. The workshop will have three sessions (one for each day). Below is the detailed schedule.
We consider optimal control problems, where the state equation is not posed on a vector space, but on a manifold. We explain the mathematical framework for this class of problems, describe an algorithm for its solution and present a numerical example.
Classical methods for parameter identification problems usually employ some variation of gradient-based optimization where the mismatch between measurements and a simulation is minimized as part of a regularized objective function. This has the great benefit of typically fast convergence. On the downside, however, this also means that typically only locally optimal parameters can be found instead of the globally optimal solution. This puts a special emphasis on obtaining a very good initial guess for the correct parameter values.
Recent advances in machine learning have demonstrated that neural nets can be a powerful tool for a huge variety of problems including parameter identification problems. At its core the biggest obstacle is obtaining sufficient datapoints to train neural nets which are reliable enough to reconstruct the sought-after parameter values. Sufficient data points can be obtained locally for a given model by repeated simulation. However, for very nonlinear model behavior and many parameters that are to be reconstructed it seems unsurmountable to train a globally correct neural net due to the curse of dimensionality.
In this talk ideas and techniques are presented that combine classical methods with recent advances in machine learning. These ideas are then applied to the task of identifying material parameters for piezoelectric ceramics.
While many approaches were developed for obtaining worst-case complexity bounds for first-order optimization methods in the last years, there remain theoretical gaps in cases where no such bound can be found. In such cases, it is often unclear whether no such bound exists (e.g., because the algorithm might fail to systematically converge) or simply if the current techniques do not allow finding them. In this work, we propose an approach to automate the search for cyclic trajectories generated by first-order methods. This provides a constructive approach to show that no appropriate complexity bound exists, thereby complementing the approaches providing sufficient conditions for convergence. Using this tool, we provide ranges of parameters for which some of the famous heavy-ball, Nesterov accelerated gradient, inexact gradient descent, and three-operator splitting algorithms fail to systematically converge, and show that it nicely complements existing tools searching for Lyapunov functions.
This is a joint work with Baptiste Goujaud and Adrien Taylor.
Solving large scale optimization problems is a challenging task and exploiting their structure can alleviate its computational cost. This idea is at the core of multilevel optimization methods. They leverage the definition of coarse approximations of the objective function to minimize it. In this talk, we present a multilevel proximal algorithm IML FISTA that draws ideas from the multilevel optimization setting for smooth optimization to tackle non-smooth optimization. In the proposed method we combine the classical accelerations techniques of inertial algorithm such as FISTA with the multilevel acceleration. IML FISTA is able to handle state-of-the-art regularization techniques such as total variation and non-local total-variation, while providing a relatively simple construction of coarse approximations. The convergence guarantees of this approach are equivalent to those of FISTA. Finally we demonstrate the effectiveness of the approach on color images reconstruction problems and on hyperspectral images reconstruction problems.
Scientific machine learning (SciML) is a rapidly evolving field of research that combines techniques from scientific computing and machine learning. A major branch of SciML is the approximation of the solutions of partial differential equations (PDEs) using neural networks. In classical physics-informed
neural networks (PINNs) , simple feed-forward neural networks are employed to discretize a PDE. The loss function may include a combination of data (e.g., initial, boundary, and/or measurement data) and the residual of the PDE. Challenging applications, such as multiscale problems, require
neural networks with high capacity, and the training is often not robust and may take large numbers of iterations. In this talk, domain decomposition-based network architectures for PINNs using the finite basis physics-informed neural network (FBPINN) approach [3, 2] will be discussed. In particular, the global network function is constructed as a combination of local network functions defined on an overlapping domain decomposition. Similar to classical domain decomposition methods, the one-level method generally lacks scalability, but scalability can be achieved by introducing a multi-level hierarchy of overlapping domain decompositions. The performance of the multi-level FBPINN  method will be investigated based on numerical results for several model problems, showing robust convergence for up to 64 subdomains on the finest level and challenging multi-frequency problems. This talk is based on joint work with Victorita Dolean (University of Strathclyde, Côte d’Azur University), Siddhartha Mishra, and Ben Moseley (ETH Zürich).
 V. Dolean, A. Heinlein, S. Mishra, and B. Moseley. Multilevel domain decomposition-based architectures for physics-informed neural networks. In preparation.
 V. Dolean, A. Heinlein, S. Mishra, and B. Moseley. Finite basis physics-informed neural networks as a Schwarz domain decomposition method, November 2022. arXiv:2211.05560.
 B. Moseley, A. Markham, and T. Nissen-Meyer. Finite Basis Physics-Informed Neural Networks (FBPINNs): a scalable domain decomposition approach for solving differential equations, July 2021. arXiv:2107.07871.
 M. Raissi, P. Perdikaris, and G. E. Karniadakis. Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378:686–707, 2019.
Coffee to go
Looking forward to welcoming you all soon in Berlin!
Opening Day (June 14 at Humboldt Universität zu Berlin)
The workshop will take place at the Humboldt Kabinett (1st floor, House 3, Room 116) located at the Institut für Mathematik, Humboldt-Universität zu Berlin, Rudower Chausee 25, 12489 Berlin, DE.
Here you can find info on how to get to the institute and navigate the campus.
Workshop on Optimization (June 14-16 at Humboldt-Universität zu Berlin)
Local organizers: Daniel Walter and Aswin Kannan.
External organizers: Omri Weinstein, Claudia Totzeck, and Alena Kopanicakova.
UPDATE: registration closed on June 5th, 2023.
If you have any further inquiries, please contact us at firstname.lastname@example.org.