This website requires JavaScript.

COLT: Cyclic Overlapping Lottery Tickets for Faster Pruning of Convolutional Neural Networks

Md. Ismail HossainMohammed RakibM. M. Lutfe ElahiNabeel MohammedShafin Rahman
Dec 2022
摘要
Pruning refers to the elimination of trivial weights from neural networks.The sub-networks within an overparameterized model produced after pruning areoften called Lottery tickets. This research aims to generate winning lotterytickets from a set of lottery tickets that can achieve similar accuracy to theoriginal unpruned network. We introduce a novel winning ticket called CyclicOverlapping Lottery Ticket (COLT) by data splitting and cyclic retraining ofthe pruned network from scratch. We apply a cyclic pruning algorithm that keepsonly the overlapping weights of different pruned models trained on differentdata segments. Our results demonstrate that COLT can achieve similar accuracies(obtained by the unpruned model) while maintaining high sparsities. We showthat the accuracy of COLT is on par with the winning tickets of Lottery TicketHypothesis (LTH) and, at times, is better. Moreover, COLTs can be generatedusing fewer iterations than tickets generated by the popular IterativeMagnitude Pruning (IMP) method. In addition, we also notice COLTs generated onlarge datasets can be transferred to small ones without compromisingperformance, demonstrating its generalizing capability. We conduct all ourexperiments on Cifar-10, Cifar-100 & TinyImageNet datasets and report superiorperformance than the state-of-the-art methods.
展开全部
图表提取

暂无人提供速读十问回答

论文十问由沈向洋博士提出,鼓励大家带着这十个问题去阅读论文,用有用的信息构建认知模型。写出自己的十问回答,还有机会在当前页面展示哦。

Q1论文试图解决什么问题?
Q2这是否是一个新的问题?
Q3这篇文章要验证一个什么科学假设?
0
被引用
笔记
问答