This website requires JavaScript.

QuickNets: Saving Training and Preventing Overconfidence in Early-Exit Neural Architectures

Devdhar PatelHava Siegelmann
Dec 2022
摘要
Deep neural networks have long training and processing times. Early exitsadded to neural networks allow the network to make early predictions usingintermediate activations in the network in time-sensitive applications.However, early exits increase the training time of the neural networks. Weintroduce QuickNets: a novel cascaded training algorithm for faster training ofneural networks. QuickNets are trained in a layer-wise manner such that eachsuccessive layer is only trained on samples that could not be correctlyclassified by the previous layers. We demonstrate that QuickNets candynamically distribute learning and have a reduced training cost and inferencecost compared to standard Backpropagation. Additionally, we introducecommitment layers that significantly improve the early exits by identifying forover-confident predictions and demonstrate its success.
展开全部
图表提取

暂无人提供速读十问回答

论文十问由沈向洋博士提出,鼓励大家带着这十个问题去阅读论文,用有用的信息构建认知模型。写出自己的十问回答,还有机会在当前页面展示哦。

Q1论文试图解决什么问题?
Q2这是否是一个新的问题?
Q3这篇文章要验证一个什么科学假设?
0
被引用
笔记
问答