CSSL-MHTR: Continual Self-Supervised Learning for Scalable Multi-script Handwritten Text Recognition

Marwa DhiafMohamed Ali SouibguiKai WangYuyang LiuYousri KessentiniAlicia Forn\'esAhmed Cheikh Rouhou

Marwa DhiafMohamed Ali SouibguiKai Wang ...+3 Ahmed Cheikh Rouhou

Mar 2023

摘要

Self-supervised learning has recently emerged as a strong alternative indocument analysis. These approaches are now capable of learning high-qualityimage representations and overcoming the limitations of supervised methods,which require a large amount of labeled data. However, these methods are unableto capture new knowledge in an incremental fashion, where data is presented tothe model sequentially, which is closer to the realistic scenario. In thispaper, we explore the potential of continual self-supervised learning toalleviate the catastrophic forgetting problem in handwritten text recognition,as an example of sequence recognition. Our method consists in addingintermediate layers called adapters for each task, and efficiently distillingknowledge from the previous model while learning the current task. Our proposedframework is efficient in both computation and memory complexity. Todemonstrate its effectiveness, we evaluate our method by transferring thelearned model to diverse text recognition downstream tasks, including Latin andnon-Latin scripts. As far as we know, this is the first application ofcontinual self-supervised learning for handwritten text recognition. We attainstate-of-the-art performance on English, Italian and Russian scripts, whilstadding only a few parameters per task. The code and trained models will bepublicly available.

展开全部

图表提取