Project Heads
Tobias Breiten, Carsten Hartmann
Project Members
Siragan Gailus, Omar Kebiri
Project Duration
01.01.2021 − 31.12.2022
Located at
TU Berlin
Using sampling methods to analyse and improve the parametrisation of neural networks is a relatively new idea. This project is devoted to the systematic development of stochastic differential equation (SDE) approximations for momentum enriched stochastic gradient schemes for deep neural networks and corresponding numerical algorithms. Specifically, we want to study the underdamped Langevin model that can be understood as the SDE counterpart of momentum enriched optimisation schemes. The underdamped Langevin model has a long tradition in statistical mechanics, and, from a control perspective, it offers more flexibility in altering the dynamics while preserving the invariant measure than the overdamped Langevin model that has been a standard tool in computational statistics or statistical learning for more than 20 years. The starting point for our analysis will be a controlled underdamped Langevin equation where the control is adapted to the filtration generated by the Brownian motion. Here the role of the control is twofold: it should accelerate the convergence to equilibrium at low temperature, while preserving the stationary distribution of the dynamics.
Stochastic modified equations with momentum
External Website
Related Publications
[1] T. Breiten, C. Hartmann, L. Neureither and U. Sharma. Stochastic gradient descent and fast relaxation to thermodynamic equilibrium: a stochastic control approach. Journal of Mathematical Physics 62, 123302, 2021.
[2] R.D. Skeel, C. Hartmann. Choice of Damping Coefficient in Langevin Dynamics. EPJ B Topical Issue “Recent progress and emerging trends in Molecular Dynamics” 94, 178, 2021.
[3] C. Hartmann, L. Neureither, M. Strehlau. Reachability Analysis of Randomly Perturbed Hamiltonian Systems. 7th IFAC Workshop on Lagrangian and Hamiltonian Methods for Nonlinear Control (LHMNC21), IFAC-PapersOnLine, 54(19), 307-314, 2021.
[1] P. Benner, T. Breiten, C. Hartmann and B. Schmidt. Model reduction of controlled Fokker-Planck and Liouville-von Neumann equations. Journal of Computational Dynamics, 7(1): 1-33, 2020.
[3] T. Breiten, K. Kunisch and L. Pfeiffer. Control Strategies for the Fokker-Planck Equation. ESAIM: Control, Optimisation and Calculus of Variations 24(2) (2018), 741-763.
[4] C. Hartmann, L. Neureither, and U. Sharma. Coarse graining of nonreversible stochastic differential equations: Quantitative results and connections to averaging. SIAM Journal on Mathematical Analysis 52 (2020), 2689-2733.
Related Pictures