AbstractVariable interface; allow variables to carry constraints

[ericphanson]

This is a continuation of #313; I made a PR from the JuliaOpt/Convex.jl repo instead of my fork to get docs previews (available here: https://www.juliaopt.org/Convex.jl/previews/PR358/).

Remaining things to do:

• Wait for Switch to MathOptInterface #330 and then rebase
• Add examples and explanations to the documentation
• Add more tests (complex variable constructors, various numeric types, problems using the new variables, show methods)
• Add _value accessor discussed in AbstractVariable interface; allow variables to carry constraints #358 (comment)

This PR introduces an AbstractVariable type with a corresponding interface. User-defined types can implement the interface to have custom variable types which can be dispatched on in user-code. The interface consists of

• evaluate, set_value!: get or set the numeric value of the variable. value should return nothing when no numeric value is set. I decided to re-use evaluate instead of introducing e.g. value, since they would have the same meaning.
• vexity, vexity!: get or set the vexity of the variable. The vexity should be AffineVexity() unless the variable has been fix!'d, in which case it is ConstVexity().
• sign, vartype, and constraints: get the Sign, VarType (a new enum, for binary, integer, or continuous variables), numeric type, and a (possibly empty) vector of constraints which are to be applied to any problem in which the variable is used.
• constraints: return those constraints carried by the variable (e.g. that it is positive semi-definite)

Optionally, also users can also implement sign!, vartype!, and add_constraint! to allow users to modify those values or add a constraint to a list of constraints which are applied to any problem in which the variable is used.

Subtypes of AbstractVariable are expected to have fields (or getproperty overloads) for head, id_hash and size, as those are expected of all AbstractExpr's. Future work could be to clarify the AbstractExpr interface. Moreover, AbstractVariable's need to set their own hash as Variable's do. This sounds confusing but I added an example to the docs. Moreover, subtyping AbstractVariable is definitely not necessary for casual use (we got this far without it anyway!).

The venerable Variable is the first user of this interface. The new constraints and vartype methods lets us finish what we started in #299: getting rid of sets. The problem with sets is that it tried to serve several orthogonal purposes simultaneously while at the same time, it wasn't always very clear what you could put in there that would have an effect.

• sets was populated by :fixed to indicate a variable was constant; this was changed in Allow calling fix! twice in a row #299 by using the vexity to indicate constantness.
• sets was also used to add one particular constraint to a variable: semidefiniteness. This is replaced by the much more flexible constraints (and add_constraint!) method. Now, a Variable can carry around arbitrary constraints which automatically get applied to any problem the variable is in, just like :semidefinite in sets did.
• sets was finally used to indicate if a variable was binary, integer, or continuous. This was replaced by vartype which uses an enum. I prefer the enum because it makes clear what the possible choices are, whereas when entering symbols, it wasn't immediately obvious (e.g. :bin or :binary?)

What is the benefit? One nice thing you can do now is

function probabilityvector(d::Int)
    x = Variable(d, Positive())
    add_constraint!(x, sum(x) == 1)
    x
end

Then later, p = probabilityvector(3) can be used like any other variable, but the constraints of having non-negative entries which add to 1 will be automatically applied to any problem it is used in. This allows very simple implementations of DSLs.

The introduction of AbstractVariable allows more ambitious extensions as well. Since custom types can subtype AbstractVariable, variables can be used in dispatch, or even be used as callable types. Continuing with the probability example, let's say I often use my probability vectors for getting the expectation value of other variables, and I want to use function notation for this. I could define

using Convex
mutable struct ProbabilityVector <: Convex.AbstractVariable
    head::Symbol
    id_hash::UInt64
    size::Tuple{Int, Int}
    value::Convex.ValueOrNothing
    vexity::Vexity
    function ProbabilityVector(d)
        this = new(:ProbabilityVector, 0, (d,1), nothing, Convex.AffineVexity())
        this.id_hash = objectid(this)
        this
    end
end
Convex.constraints(p::ProbabilityVector) = [ sum(p) == 1 ]
Convex.sign(::ProbabilityVector) = Convex.Positive()
Convex.vartype(::ProbabilityVector) = Convex.ContVar

(p::ProbabilityVector)(x) = dot(p, x)

And then I can use the new type freely in Convex problems. E.g.

p = ProbabilityVector(3)
x = [1.0, 2.0, 3.0]
prob = minimize( p(x) )
solve!(prob, solver)
evaluate(p) # [1.0, 0.0, 0.0]

(Back in August, I was experimenting with a similar DSL for quantum information theory in https://github.com/ericphanson/QuantumSDPs.jl; I hope to pick that up at some point.)

I re-implemented the old constructors that accept symbols for set, and translate them to the new variable type, which could be deprecated now or in a later PR. Many of the internals changes are simply changing e.g. x.sign to sign(x), to use the interface instead.

[ericphanson]

One thing is that now evaluate(x::Variable) does not error even if x.value = nothing; it just returns nothing in that case. This used to give an informative error that x's value has not yet been set since the problem has not been solved. This was one benefit of separating x.value from evaluate(x), since internally we could just do field access. Maybe we should add an _value method to the AbstractVariable interface which never errors, for internal use. (I.e. other custom variables would need to define this method instead of evaluate, which would fall back to calling that method and erroring if it finds nothing).

[ericphanson]

Ah, looking at #341, I think there's another catch. We likely want evaluate(c::Constant) and evaluate(x::Variable) to e.g. yield scalars instead of (1,1)-matrices, but we may not want to do the conversion internally. I think it makes sense then to have a raw accessor like _value mentioned above.

[ericphanson]

I've rebased on top of #341 and add such an accessor. I think this just needs a few more tests and it's good to go.

[codecov]

Codecov Report

Merging #358 into master will increase coverage by 0.39%.
The diff coverage is 94.44%.

@@            Coverage Diff             @@
##           master     #358      +/-   ##
==========================================
+ Coverage   88.03%   88.42%   +0.39%     
==========================================
  Files          75       75              
  Lines        3460     3508      +48     
==========================================
+ Hits         3046     3102      +56     
+ Misses        414      406       -8     

Impacted Files	Coverage Δ
src/Convex.jl	100.00% <ø> (ø)
src/conic_form.jl	100.00% <ø> (ø)
src/utilities/broadcast.jl	50.00% <ø> (ø)
src/utilities/iteration.jl	100.00% <ø> (ø)
src/constraints/signs_and_sets.jl	50.00% <50.00%> (-33.34%)	⬇️
src/utilities/tree_interface.jl	31.57% <50.00%> (ø)
src/problems.jl	90.19% <66.66%> (+6.52%)	⬆️
src/utilities/show.jl	85.07% <66.66%> (ø)
src/variable.jl	96.73% <96.20%> (+10.37%)	⬆️
src/problem_depot/problems/constant.jl	100.00% <100.00%> (ø)
... and 8 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fc60bbe...26041a1. Read the comment docs.