This website requires JavaScript.

# Orthogonal Directions Constrained Gradient Method: from non-linear equality constraints to Stiefel manifold

Mar 2023

We consider the problem of minimizing a non-convex function over a smoothmanifold $\mathcal{M}$. We propose a novel algorithm, the Orthogonal DirectionsConstrained Gradient Method (ODCGM) which only requires computing a projectiononto a vector space. ODCGM is infeasible but the iterates are constantly pulledtowards the manifold, ensuring the convergence of ODCGM towards $\mathcal{M}$.ODCGM is much simpler to implement than the classical methods which require thecomputation of a retraction. Moreover, we show that ODCGM exhibits thenear-optimal oracle complexities $\mathcal{O}(1/\varepsilon^2)$ and$\mathcal{O}(1/\varepsilon^4)$ in the deterministic and stochastic cases,respectively. Furthermore, we establish that, under an appropriate choice ofthe projection metric, our method recovers the landing algorithm of Ablin andPeyr\'e (2022), a recently introduced algorithm for optimization over theStiefel manifold. As a result, we significantly extend the analysis of Ablinand Peyr\'e (2022), establishing near-optimal rates both in deterministic andstochastic frameworks. Finally, we perform numerical experiments which showsthe efficiency of ODCGM in a high-dimensional setting.

Q1论文试图解决什么问题？
Q2这是否是一个新的问题？
Q3这篇文章要验证一个什么科学假设？
0