This website requires JavaScript.

Generalizing Goal-Conditioned Reinforcement Learning with Variational Causal Reasoning

Wenhao DingHaohong LinBo LiDing Zhao
Jul 2022
摘要
As a pivotal component to attaining generalizable solutions in humanintelligence, reasoning provides great potential for reinforcement learning(RL) agents' generalization towards varied goals by summarizing part-to-wholearguments and discovering cause-and-effect relations. However, how to discoverand represent causalities remains a huge gap that hinders the development ofcausal RL. In this paper, we augment Goal-Conditioned RL (GCRL) with CausalGraph (CG), a structure built upon the relation between objects and events. Wenovelly formulate the GCRL problem into variational likelihood maximizationwith CG as latent variables. To optimize the derived objective, we propose aframework with theoretical performance guarantees that alternates between twosteps: using interventional data to estimate the posterior of CG; using CG tolearn generalizable models and interpretable policies. Due to the lack ofpublic benchmarks that verify generalization capability under reasoning, wedesign nine tasks and then empirically show the effectiveness of the proposedmethod against five baselines on these tasks. Further theoretical analysisshows that our performance improvement is attributed to the virtuous cycle ofcausal discovery, transition modeling, and policy training, which aligns withthe experimental evidence in extensive ablation studies.
展开全部
图表提取

暂无人提供速读十问回答

论文十问由沈向洋博士提出,鼓励大家带着这十个问题去阅读论文,用有用的信息构建认知模型。写出自己的十问回答,还有机会在当前页面展示哦。

Q1论文试图解决什么问题?
Q2这是否是一个新的问题?
Q3这篇文章要验证一个什么科学假设?
0
被引用
笔记
问答