Most of the existing works use projection functions for ternary quantizationin discrete space. Scaling factors and thresholds are used in some cases toimprove the model accuracy. However, the gradients used for optimization areinaccurate and result in a notable accuracy gap between the full precision andternary models. To get more accurate gradients, some works gradually increasethe discrete portion of the full precision weights in the forward propagationpass, e.g., using temperature-based Sigmoid function. Instead of directlyperforming ternary quantization in discrete space, we push full precisionweights close to ternary ones through regularization term prior to ternaryquantization. In addition, inspired by the temperature-based method, weintroduce a re-scaling factor to obtain more accurate gradients by simulatingthe derivatives of Sigmoid function. The experimental results show that ourmethod can significantly improve the accuracy of ternary quantization in bothimage classification and object detection tasks.
