This website requires JavaScript.

Ask Question First for Enhancing Lifelong Language Learning

Han WangRuiliu FuXuejun ZhangJun ZhouQingwei Zhao
Aug 2022
摘要
Lifelong language learning aims to stream learning NLP tasks while retainingknowledge of previous tasks. Previous works based on the language model andfollowing data-free constraint approaches have explored formatting all data as"begin token (\textit{B}) + context (\textit{C}) + question (\textit{Q}) +answer (\textit{A})" for different tasks. However, they still suffer fromcatastrophic forgetting and are exacerbated when the previous task's pseudodata is insufficient for the following reasons: (1) The model has difficultygenerating task-corresponding pseudo data, and (2) \textit{A} is prone to errorwhen \textit{A} and \textit{C} are separated by \textit{Q} because theinformation of the \textit{C} is diminished before generating \textit{A}.Therefore, we propose the Ask Question First and Replay Question (AQF-RQ),including a novel data format "\textit{BQCA}" and a new training task to trainpseudo questions of previous tasks. Experimental results demonstrate thatAQF-RQ makes it easier for the model to generate more pseudo data that matchcorresponding tasks, and is more robust to both sufficient and insufficientpseudo-data when the task boundary is both clear and unclear. AQF-RQ canachieve only 0.36\% lower performance than multi-task learning.
展开全部
图表提取

暂无人提供速读十问回答

论文十问由沈向洋博士提出,鼓励大家带着这十个问题去阅读论文,用有用的信息构建认知模型。写出自己的十问回答,还有机会在当前页面展示哦。

Q1论文试图解决什么问题?
Q2这是否是一个新的问题?
Q3这篇文章要验证一个什么科学假设?
0
被引用
笔记
问答