This website requires JavaScript.

Predictive Exit: Prediction of Fine-Grained Early Exits for Computation- and Energy-Efficient Inference

Xiangjie LiChenfei LouZhengping Zhu ...+3 An Zou
Jun 2022
摘要
By adding exiting layers to the deep learning networks, early exit canterminate the inference earlier with accurate results. The passivedecision-making of whether to exit or continue the next layer has to go throughevery pre-placed exiting layer until it exits. In addition, it is also hard toadjust the configurations of the computing platforms alongside the inferenceproceeds. By incorporating a low-cost prediction engine, we propose aPredictive Exit framework for computation- and energy-efficient deep learningapplications. Predictive Exit can forecast where the network will exit (i.e.,establish the number of remaining layers to finish the inference), whicheffectively reduces the network computation cost by exiting on time withoutrunning every pre-placed exiting layer. Moreover, according to the number ofremaining layers, proper computing configurations (i.e., frequency and voltage)are selected to execute the network to further save energy. Extensiveexperimental results demonstrate that Predictive Exit achieves up to 96.2%computation reduction and 72.9% energy-saving compared with classic deeplearning networks; and 12.8% computation reduction and 37.6% energy-savingcompared with the early exit under state-of-the-art exiting strategies, giventhe same inference accuracy and latency.
展开全部
图表提取

论文十问由沈向洋博士提出，鼓励大家带着这十个问题去阅读论文，用有用的信息构建认知模型。写出自己的十问回答，还有机会在当前页面展示哦。

  1. Q1
    论文试图解决什么问题？
    An Zou 2023/02/01

    降低神经网络推理过程中通用处理器所消耗的算力和功耗。

  2. Q2
    这是否是一个新的问题？
    An Zou 2023/02/01

    算是一个比较新的问题。在神经网络模型越来越大的今天，降低处理器所消耗的功耗至关重要。

  3. Q3
    这篇文章要验证一个什么科学假设？
    An Zou 2023/02/01

    提前预测神经网络在执行完一定的层数后可获得准确结果。并通过对处理器电压频率的调节，降低神经网络推理过程所消耗的功耗。

  4. Q4
    有哪些相关研究？如何归类？谁是这一课题在领域内值得关注的研究员？
  5. Q5
    论文中提到的解决方案之关键是什么？
  6. Q6
    论文中的实验是如何设计的？
  7. Q7
    用于定量评估的数据集是什么？代码有没有开源？
  8. Q8
    论文中的实验及结果有没有很好地支持需要验证的科学假设？
  9. Q9
    这篇论文到底有什么贡献？
  10. Q10
    下一步呢？有什么工作可以继续深入？
0
被引用
0
笔记
0
问答