This website requires JavaScript.

Hyperspherical Loss-Aware Ternary Quantization

Dan LiuXue Liu
Dec 2022
摘要
Most of the existing works use projection functions for ternary quantizationin discrete space. Scaling factors and thresholds are used in some cases toimprove the model accuracy. However, the gradients used for optimization areinaccurate and result in a notable accuracy gap between the full precision andternary models. To get more accurate gradients, some works gradually increasethe discrete portion of the full precision weights in the forward propagationpass, e.g., using temperature-based Sigmoid function. Instead of directlyperforming ternary quantization in discrete space, we push full precisionweights close to ternary ones through regularization term prior to ternaryquantization. In addition, inspired by the temperature-based method, weintroduce a re-scaling factor to obtain more accurate gradients by simulatingthe derivatives of Sigmoid function. The experimental results show that ourmethod can significantly improve the accuracy of ternary quantization in bothimage classification and object detection tasks.
展开全部
图表提取

暂无人提供速读十问回答

论文十问由沈向洋博士提出,鼓励大家带着这十个问题去阅读论文,用有用的信息构建认知模型。写出自己的十问回答,还有机会在当前页面展示哦。

Q1论文试图解决什么问题?
Q2这是否是一个新的问题?
Q3这篇文章要验证一个什么科学假设?
0
被引用
笔记
问答