Orthogonal Directions Constrained Gradient Method: from non-linear equality constraints to Stiefel manifold

Sholom SchechtmanDaniil TiapkinMichael MuehlebachEric Moulines

Mar 2023

摘要

We consider the problem of minimizing a non-convex function over a smoothmanifold $\mathcal{M}$. We propose a novel algorithm, the Orthogonal DirectionsConstrained Gradient Method (ODCGM) which only requires computing a projectiononto a vector space. ODCGM is infeasible but the iterates are constantly pulledtowards the manifold, ensuring the convergence of ODCGM towards $\mathcal{M}$.ODCGM is much simpler to implement than the classical methods which require thecomputation of a retraction. Moreover, we show that ODCGM exhibits thenear-optimal oracle complexities $\mathcal{O}(1/\varepsilon^2)$ and$\mathcal{O}(1/\varepsilon^4)$ in the deterministic and stochastic cases,respectively. Furthermore, we establish that, under an appropriate choice ofthe projection metric, our method recovers the landing algorithm of Ablin andPeyr\'e (2022), a recently introduced algorithm for optimization over theStiefel manifold. As a result, we significantly extend the analysis of Ablinand Peyr\'e (2022), establishing near-optimal rates both in deterministic andstochastic frameworks. Finally, we perform numerical experiments which showsthe efficiency of ODCGM in a high-dimensional setting.

展开全部

图表提取

论文十问

参考文献

被引用

提问与回答

暂无人提供速读十问回答

被引用

笔记

问答

领域