This website requires JavaScript.

Exploring the Optimized Value of Each Hyperparameter in Various Gradient Descent Algorithms

Abel C. H. Chen
Dec 2022
摘要
In the recent years, various gradient descent algorithms including themethods of gradient descent, gradient descent with momentum, adaptive gradient(AdaGrad), root-mean-square propagation (RMSProp) and adaptive momentestimation (Adam) have been applied to the parameter optimization of severaldeep learning models with higher accuracies or lower errors. These optimizationalgorithms may need to set the values of several hyperparameters which includea learning rate, momentum coefficients, etc. Furthermore, the convergence speedand solution accuracy may be influenced by the values of hyperparameters.Therefore, this study proposes an analytical framework to use mathematicalmodels for analyzing the mean error of each objective function based on variousgradient descent algorithms. Moreover, the suitable value of eachhyperparameter could be determined by minimizing the mean error. The principlesof hyperparameter value setting have been generalized based on analysis resultsfor model optimization. The experimental results show that higher efficiencyconvergences and lower errors can be obtained by the proposed method.
展开全部
图表提取

暂无人提供速读十问回答

论文十问由沈向洋博士提出,鼓励大家带着这十个问题去阅读论文,用有用的信息构建认知模型。写出自己的十问回答,还有机会在当前页面展示哦。

Q1论文试图解决什么问题?
Q2这是否是一个新的问题?
Q3这篇文章要验证一个什么科学假设?
0
被引用
笔记
问答