Method Development Unit

We investigate the training of deep neural networks, in particular of the Transformer architecture, using rough analysis. We reveal continuous-time scaling regimes and stability properties essential for understanding convergence, capturing the limiting behavior both at initialization and during training.