Riemannian black box function with JuMP/MOI

[blegat]

"""
    struct RiemannianFirstOrderFunction{F,G} <: MOI.AbstractScalarFunction
        value::F
        gradient!::G
    end

Function defined on a manifold. Given a point `x` of the manifold, `value(x)` provides the value at point `x`
and `gradient!(g, x)` set `g` to be the Riemannian gradient at `x`.

!!! note
    `x` is not in a vectorized format but in the format of points in the manifold.
"""
struct RiemannianFirstOrderFunction{F,G} <: MOI.AbstractScalarFunction
    value::F
    gradient::G
end

From JuMP, you can then do something like

@variable(model, X in manifold)
@objective(model, Min, RiemannianFirstOrderFunction(X -> ..., (g, X) -> ...))

If the user sets an objective without RiemannianFirstOrderFunction then it is transformed into these as in #264
The transformation from MOI's functions into RiemannianFirstOrderFunction's function may be doable transparently in a MOI.Bridges.Objective so that Manopt does not even have to care about anything other than RiemannianFirstOrderFunction an may assume the user always give that.

[kellertuer]

This transformation is indeed on my list for a while
#217
though basically the inverse, since Manopt always assumes the Riemannian case and I would like to offer the Euclidean as well, but I was a bit busy this semester with other duties.

Besides that, I like the notion of @variable(model, X in manifold) – it seems to be what I asked for with the domain discussion, just that you embed the domain as part of the variable – which for me is basically the same just a bit of a different viewpoint – and maybe even one that fits better into MOI.

Is this in deed an issue for Manopt or more one for MOI?

Also note that in Manopt, the signatures of cost/grad/hessian/prox/... have the manifold as their first variable, since there might also be different metrics “at work” and such.

[blegat]

A better one might indeed be

struct RiemannianFunction{MO<:AbstractManifoldObjective} <: MOI.AbstractScalarFunction
    func::MO
end

Is this in deed an issue for Manopt or more one for MOI?

We always start defining new sets and functions in the packages supporting them and them move to MOI if other solvers support it as well and they don't want more dependencies than MOI. I would just define it in Manopt for now.

[kellertuer]

Do you mean MO <: AbstractManifoldObjective like https://manoptjl.org/stable/plans/objective/#Manopt.AbstractManifoldObjective ? That would of course be a very clean very nice wrapper!

[blegat]

Yes corrected

[kellertuer]

Then I understand the approach – super neat!

[kellertuer]

HI @blegat,
what would be good next steps here to get this issue finished? Is it just something on the Manopt.jl side that is missing? I do not see links to issues over at JuMP, but I had in mind, there was also something missing on that side?

I would like to maybe tackle this one here next, to have even better support for JuMP/Manopt.jl together and think about / start a tutorial.

[blegat]

Yes, we should have everything we need at the JuMP/MOI side now, I should have some time to take a look soon