Neural networks are machine learning models for constructing universal function approximation algorithms.

The learning of these networks requires to iteratively solve large nonlinear optimization problems, and gradient methods

play a central role in this framework, which motivates the investigation of techniques for accelerating and robustifying these methods.
On the other hand, for some neural network architectures, the learning process can be interpreted as an optimal control problem
and, also in this case, efficient and robust solution techniques are required.