Tips and tricks#

Problem formulation#

There are two main ways to formulate an optimization problem:

  1. Using symbolic CasADi expressions.

  2. By creating a class that implements the necessary methods for evaluating problem functions.

To decide which formulation is most suitable for your application, it helps to consider a few different aspects:

CasADi

Classes

Declarative

Imperative

Intuitive symbolic expressions

Any Python code (NumPy, JAX, TensorFlow, PyTorch …)

Automatic computation of derivatives

Requires explicit methods for evaluating derivatives

Pre-compilation and caching for excellent performance

Overhead because of calls to Python functions

Going beyond Python, there are other ways to formulate problems that are covered in the Doxygen documentation:

In the following sections, we’ll focus on the CasADi and classes-based problem formulations.

CasADi#

An example of using CasADi to build the problem generation is given on the Getting started page. It makes use of the High-level problem formulation:

# %% Generate and compile C code for the objective and constraints using alpaqa
from alpaqa import minimize

problem = (
    minimize(f, x)  #       Objective function f(x)
    .subject_to_box(C)  #   Box constraints x ∊ C
    .subject_to(g, D)  #    General ALM constraints g(x) ∊ D
    .with_param(p, [1])  #  Parameter with default value (can be changed later)
).compile()

Compilation#

The alpaqa.MinimizationProblemDescription.compile() method generates C code for the problem functions and their derivatives, and compiles them into an optimized binary. Since alpaqa solvers spend most of their time inside of problem function evaluations, this compilation can have a significant impact on solver performance.

By default, the CasADi SX class is used for code generation. This causes the subexpressions to be expanded, which is usually beneficial for performance. However, the resulting expression trees can grow quite massive, and compiling the generated C code can become very slow for large or complex problems. In such cases, you can use the MX class, by passing sym=casadi.MX.sym as an argument to the compile() function.

If you don’t want to compile the problem at all (e.g. because no C compiler is available, or because the resulting C files are too large), you can use the alpaqa.MinimizationProblemDescription.build() method instead of compile(). This will use CasADi’s VM to evaluate the expressions.

Classes#

The imperative class-based problem formulation allows users to select alternative frameworks to implement the problem functions, such as NumPy, JAX, TensorFlow, PyTorch, etc. However, this does mean that all required problem functions and functions to evaluate their derivatives have to be supplied by the user.

The following class can be used as a template:

class MyProblem:
    def __init__(self):
        self.n = 3 # Number of variables
        self.m = 2 # Number of constraints
    def eval_proj_diff_g(self, z: np.ndarray, e: np.ndarray) -> None: ...
    def eval_proj_multipliers(self, y: np.ndarray, M: float) -> None: ...
    def eval_prox_grad_step(self, γ: float, x: np.ndarray, grad_ψ: np.ndarray, x_hat: np.ndarray, p: np.ndarray) -> float: ...
    def eval_inactive_indices_res_lna(self, γ: float, x: np.ndarray, grad_ψ: np.ndarray, J: np.ndarray) -> int: ...
    def eval_f(self, x: np.ndarray) -> float: ...
    def eval_grad_f(self, x: np.ndarray, grad_fx: np.ndarray) -> None: ...
    def eval_g(self, x: np.ndarray, gx: np.ndarray) -> None: ...
    def eval_grad_g_prod(self, x: np.ndarray, y: np.ndarray, grad_gxy: np.ndarray) -> None: ...
    def eval_grad_gi(self, x: np.ndarray, i: int, grad_gi: np.ndarray) -> None: ...
    def eval_hess_L_prod(self, x: np.ndarray, y: np.ndarray, scale: float, v: np.ndarray, Hv: np.ndarray) -> None: ...
    def eval_hess_ψ_prod(self, x: np.ndarray, y: np.ndarray, Σ: np.ndarray, scale: float, v: np.ndarray, Hv: np.ndarray) -> None: ...
    def eval_f_grad_f(self, x: np.ndarray, grad_fx: np.ndarray) -> float: ...
    def eval_f_g(self, x: np.ndarray, g: np.ndarray) -> float: ...
    def eval_grad_f_grad_g_prod(self, x: np.ndarray, y: np.ndarray, grad_f: np.ndarray, grad_gxy: np.ndarray) -> None: ...
    def eval_grad_L(self, x: np.ndarray, y: np.ndarray, grad_L: np.ndarray, work_n: np.ndarray) -> None: ...
    def eval_ψ(self, x: np.ndarray, y: np.ndarray, Σ: np.ndarray, ŷ: np.ndarray) -> float: ...
    def eval_grad_ψ(self, x: np.ndarray, y: np.ndarray, Σ: np.ndarray, grad_ψ: np.ndarray, work_n: np.ndarray, work_m: np.ndarray) -> None: ...
    def eval_ψ_grad_ψ(self, x: np.ndarray, y: np.ndarray, Σ: np.ndarray, grad_ψ: np.ndarray, work_n: np.ndarray, work_m: np.ndarray) -> float: ...
    def get_box_C(self) -> alpaqa.Box: ...
    def get_box_D(self) -> alpaqa.Box: ...
    def check(self): ...

The meanings of different methods and their arguments are explained on the Problem formulations page. You can find a concrete example in Lasso JAX.

Note

To assign values to an output argument, you should use arg[:] = x, and not arg = x. For example:

def eval_grad_f(self, x: np.ndarray, grad_f: np.ndarray) -> None:
    grad_f[:] = A @ x - b

Compilation and caching#

Compiled CasADi problems are cached. To show the location of the cache, you can use the following command:

alpaqa cache path

If alpaqa is used inside of a virtual environment, it will create a cache directory in that environment. You can force the global cache to be used instead by setting the environment variable ALPAQA_GLOBAL_CACHE=1. To override the default cache directory, you can set the ALPAQA_CACHE_DIR environment variable.

To delete all cached problems, use:

alpaqa cache clean

For the compilation of C code generated by CasADi, alpaqa relies on CMake. Version 3.17 or later is required: by default, a recent version of CMake will be installed into your Python virtual environment when you install alpaqa. To use a different version of CMake, you can set the ALPAQA_CMAKE_PROGRAM environment variable.

If you want to change the compiler or the options used, you can clean alpaqa’s CMake build directory and then set the appropriate environment variables, for example:

alpaqa cache clean --cmake
export CC="/usr/bin/gcc"                   # C compiler to use
export CFLAGS="-march=native"              # Options to pass to the C compiler
python "/path/to/your/alpaqa/script.py"

Note

The CMake options set by these environment variables are cached: The values of the environment variables are picked up only during the very first compilation of a CasADi problem after clearing the cache. Later changes to the environment variables are ignored and do not affect the cached values.

To change the CMake build configuration, you can set the ALPAQA_BUILD_CONFIG environment variable. To change the number of parallel build jobs, you can set the ALPAQA_BUILD_PARALLEL environment variable. These two variables are not cached and take effect immediately.

Compiler installation#

See the resources below if you do not have a C compiler installed on your system:

Solver selection and parameter tuning#

You can find more information about the different solvers and their parameters in this presentation.