Logical Implications for Visual Question Answering Consistency

Sergio Tascon-MoralesPablo M\'arquez-NeilaRaphael Sznitman

Mar 2023

摘要

Despite considerable recent progress in Visual Question Answering (VQA)models, inconsistent or contradictory answers continue to cast doubt on theirtrue reasoning capabilities. However, most proposed methods use indirectstrategies or strong assumptions on pairs of questions and answers to enforcemodel consistency. Instead, we propose a novel strategy intended to improvemodel performance by directly reducing logical inconsistencies. To do this, weintroduce a new consistency loss term that can be used by a wide range of theVQA models and which relies on knowing the logical relation between pairs ofquestions and answers. While such information is typically not available in VQAdatasets, we propose to infer these logical relations using a dedicatedlanguage model and use these in our proposed consistency loss function. Weconduct extensive experiments on the VQA Introspect and DME datasets and showthat our method brings improvements to state-of-the-art VQA models, while beingrobust across different architectures and settings.

展开全部

图表提取

论文十问

参考文献

被引用

提问与回答

暂无人提供速读十问回答

被引用

笔记

问答

领域