This page provides a high level overview of the inner solvers supported by alpaqa, and how to use them.
Both PANOC [5] and PANTR solve problems of the form
\[ \begin{equation} \begin{aligned} \underset{x}{\text{minimize}}\; \psi(x) + h(x) \quad \quad \psi : \Rn \rightarrow \R, \; \; h : \Rn \rightarrow \overline{\R} \end{aligned} \label{eq:problem-orig} \tag{P} \end{equation} \]
where \(\psi\) is (locally) Lipschitz-smooth and \(h\) is proper, lowersemicontinuous and convex. Note that the (possibly nonsmooth) term \(h\) can be used to encode constraints or include a regularization term.
A well-known strategy for solving \(\eqref{eq:problem-orig}\) consists of iteratively applying the forward-backward operator
\[ \begin{equation} T_\gamma(x) = \prox_{\gamma h}(x - \gamma \nabla \psi(x)), \label{eq:fbs} \tag{FB} \end{equation} \]
where \(\prox_{\gamma h}(x) = \argmin_u \{ h(u) + \frac{1}{2\gamma} \Vert u - x \Vert^2 \}\) denotes the proximal operator of \(h\) with step size \(\gamma\). Remark that this scheme only requires evaluations of \(\nabla \psi\) and \(\prox_{\gamma h}\), which are assumed to be efficiently evaluated. Using the same oracle, both PANOC and PANTR aim to accelerate the standard forward-backward iterations \(\eqref{eq:fbs}\).
PANOC [5] combines forward-backward iterations \(\eqref{eq:fbs}\) with a quasi-Newton linesearch procedure to attain fast asymptotic convergence.
TO DO: API
PANTR is similar to PANOC, but replaces the quasi-Newton directions by solutions to trust-region subproblems, which can be seen as regularized Newton updates.
TO DO: API
1.9.8