add more tests, in particular for dual variables, cf. #290

[ranocha]

I've checked the dual variables either analytically or via cvx (and different solvers) in matlab.

[ericphanson]

These look great! I'll review them more carefully in the next couple days and then merge if it all seems ok. Thanks very much. (Same goes for the other PR).

One question: why the minimal norm tests? Just to test something more, or is it related to duality?

[ranocha]

I added the minimal norm tests because I was interested in some similar problems. Then I wanted to test additionally the dual variables for these problems but got different results for different solvers (SCS, ECOS). Strangely, cvx in matlab gave consistent results for different solvers (gurobi, mosek, sedumi and the other free one I don't remember now). Since I didn't have enough time to work out the dual variables by hand, I pushed them without tests of dual variables. But I calculated the optimal values and solutions by hand.
Are the dual variables unique?

[ericphanson]

Ok, interesting. My understanding is that in general they are not unique, but the optimal primal and dual variables should together satisfy the KTT conditions (see e.g. Section 5.5.3 of Boyd and Vandenberghe's book https://web.stanford.edu/~boyd/cvxbook/bv_cvxbook.pdf). The KTT conditions require a differentiable problem, but I think Convex is rewriting these norms into a formulation that is differentiable anyway.

So maybe for these problems the dual solutions are not unique? It looks like Slater's condition holds, since there is a feasible point, namely x = [1, 1], and there are no inequality constraints, so strong duality holds. So that means the dual problem and primal problem have the same optimal value. We could check that the dual values returned by Convex give that same value when we evaluate the dual objective function with them, and check that they satisfy the constraints of the dual program.

[ranocha]

How can I get the dual objective function in Convex.jl?

[ericphanson]

I was thinking we'd have to do so analytically; Convex itself does not provide such a method. However, there was a GsoC project this summer to do so at the level of MOI: https://github.com/JuliaOpt/Dualization.jl. So maybe with #330 this could be incorporated into Convex somehow.

[ericphanson]

(Adding dual tests to those minimal norm problems could be a separate PR though)

[ranocha]

I've added the tests of the dual objective function value. Indeed, the dual variable is not unique. In that case, it is a bit strange that all solvers of cvx in matlab I've tested return the same dual variable...

[ericphanson]

Looks great overall! Thanks very much for contributing these tests. I did not check all the dual values (though I checked a couple), so I'll trust you on those :).

I think this is good to merge once line 26 of test_lp.jl is fixed or removed.

[ericphanson]

This line doesn't seem to test anything; was it supposed to test something else?

[ranocha]

Good catch! I've fixed the typo.

[ericphanson]

Thanks very much!