This website requires JavaScript.

A Generalized EigenGame with Extensions to Multiview Representation Learning

James ChapmanAna Lawry AguilaLennie Wells
Nov 2022
摘要
Generalized Eigenvalue Problems (GEPs) encompass a range of interestingdimensionality reduction methods. Development of efficient stochasticapproaches to these problems would allow them to scale to larger datasets.Canonical Correlation Analysis (CCA) is one example of a GEP for dimensionalityreduction which has found extensive use in problems with two or more views ofthe data. Deep learning extensions of CCA require large mini-batch sizes, andtherefore large memory consumption, in the stochastic setting to achieve goodperformance and this has limited its application in practice. Inspired by theGeneralized Hebbian Algorithm, we develop an approach to solving stochasticGEPs in which all constraints are softly enforced by Lagrange multipliers. Thenby considering the integral of this Lagrangian function, its pseudo-utility,and inspired by recent formulations of Principal Components Analysis and GEPsas games with differentiable utilities, we develop a game-theory inspiredapproach to solving GEPs. We show that our approaches share much of thetheoretical grounding of the previous Hebbian and game theoretic approaches forthe linear case but our method permits extension to general functionapproximators like neural networks for certain GEPs for dimensionalityreduction including CCA which means our method can be used for deep multiviewrepresentation learning. We demonstrate the effectiveness of our method forsolving GEPs in the stochastic setting using canonical multiview datasets anddemonstrate state-of-the-art performance for optimizing Deep CCA.
展开全部
图表提取

暂无人提供速读十问回答

论文十问由沈向洋博士提出,鼓励大家带着这十个问题去阅读论文,用有用的信息构建认知模型。写出自己的十问回答,还有机会在当前页面展示哦。

Q1论文试图解决什么问题?
Q2这是否是一个新的问题?
Q3这篇文章要验证一个什么科学假设?
0
被引用
笔记
问答