This website requires JavaScript.

# Controlled Descent Training

Mar 2023

In this work, a novel and model-based artificial neural network (ANN)training method is developed supported by optimal control theory. The methodaugments training labels in order to robustly guarantee training lossconvergence and improve training convergence rate. Dynamic label augmentationis proposed within the framework of gradient descent training where theconvergence of training loss is controlled. First, we capture the trainingbehavior with the help of empirical Neural Tangent Kernels (NTK) and borrowtools from systems and control theory to analyze both the local and globaltraining dynamics (e.g. stability, reachability). Second, we propose todynamically alter the gradient descent training mechanism via fictitious labelsas control inputs and an optimal state feedback policy. In this way, we enforcelocally $\mathcal{H}_2$ optimal and convergent training behavior. The novelalgorithm, \textit{Controlled Descent Training} (CDT), guarantees localconvergence. CDT unleashes new potentials in the analysis, interpretation, anddesign of ANN architectures. The applicability of the method is demonstrated onstandard regression and classification problems.

Q1论文试图解决什么问题？
Q2这是否是一个新的问题？
Q3这篇文章要验证一个什么科学假设？
0