Add complementarity constraints #913

odow · 2019-10-08T01:11:40Z

This PR adds the set Complements(dimension) which defines a mixed complementarity constraint.

The main targets are

KNITRO.jl, which will implement VectorOfVariables-in-Complements (cc @frapac)
PATH.jl, which will implement VectorQuadraticFunction-in-Complements and VectorAffineFunction-in-Complements, implementing both allows us to short-circuit the Jacobian evaluation the affine case. VectorOfVariables-in-Complements will be handled by bridges.

Closes #771

I've left more obscure complementarity constraints, like conic constrained x compared to lb <= x <= ub, un-mentioned. @frapac does KNITRO just look at the bounds? Or does it do something different?

cc @xhub

codecov-io · 2019-10-08T01:23:56Z

Codecov Report

Merging #913 into master will increase coverage by 0.07%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master     #913      +/-   ##
==========================================
+ Coverage    95.2%   95.27%   +0.07%     
==========================================
  Files          82       83       +1     
  Lines        8751     9059     +308     
==========================================
+ Hits         8331     8631     +300     
- Misses        420      428       +8

Impacted Files	Coverage Δ
src/Test/UnitTests/basic_constraint_tests.jl	`98% <ø> (ø)`	⬆️
src/Utilities/model.jl	`94.63% <ø> (ø)`	⬆️
src/Test/nlp.jl	`20.22% <100%> (+20.22%)`	⬆️
src/sets.jl	`95.06% <100%> (+0.06%)`	⬆️
src/attributes.jl	`86.32% <0%> (-0.84%)`	⬇️
src/Test/UnitTests/solve.jl	`100% <0%> (ø)`	⬆️
src/Bridges/Bridges.jl	`100% <0%> (ø)`	⬆️
src/Bridges/debug.jl	`100% <0%> (ø)`
src/Bridges/lazy_bridge_optimizer.jl	`93.85% <0%> (+0.6%)`	⬆️
src/Utilities/mockoptimizer.jl	`92.64% <0%> (+2.1%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e29f1ce...ad1d56b. Read the comment docs.

frapac

Thanks @odow for adding this in MOI!

src/Test/nlp.jl

src/sets.jl

blegat · 2019-10-08T13:09:34Z

If at some point we define VariationalInequality(C) that means $C^D\ni F(x) \perp x \in C$ as @xhub suggested then Complements(n) will simply be VariationalInequality(Nonnegatives(n)).
What should we do then ?

Define const Complements = VariationalInequality{Nonnegatives}
Define a bridge mapping VariationalInequality{Nonnegatives} to Complements and vice versa

In MOI, we don't currently have a set for which the type is a special case of the type of another one so I think 1. would make more sense.
If we go with 1., we should make sure we don't get any backward compatibility issue by transforming Complements from a struct to a const.

odow · 2019-10-08T13:28:11Z

Note that our definition of complements isn't VariationalInequality{Nonnegatives}, it's really VariationalInequality(Hypercube(lb, ub)) so I think a bridge is the way to go.

In MOI, we don't currently have a set for which the type is a special case of the type of another one

Sure we do: Integer and ZeroOne, GreaterThan and Interval.

odow · 2019-10-08T13:37:16Z

Really, there is nothing stopping us from interpreting the current Complements as VariationalInequality.

The current definition is just a relation between F(x) and x with the bounds added separately. So the cone C can just be inferred from the constraints on x. Maybe we should change the docstring to reflect this.

blegat · 2019-10-08T13:46:13Z

The current definition is just a relation between F(x) and x with the bounds added separately.

Bridging currently assume that we can treat each constraint independently so it will be easy to bridge VariationalInequality to Complements but not Complements to VariationalInequality as you would need a mechanism for the bridge to say to the bridge optimizer to notify him when the bounds on the variable changes (we would also need something similar for the QuadtoSOC bridge to support x^2 + y^2 <= t^2 as we need to be sure that there is a bound on t so maybe we could add that at some point.

I prefer VariationalInequality(Hyperrectangle(lb, ub)) over Complements as all the information is inside the constraint and it does not depend on other constraint.
However, if Complements is a close match to Knitro and PATH interface (EDIT I have checked your PATH interface, Complements is indeed an exact match) then it makes sense to have both. It's kind of similar to Quad vs SOC. SOC (resp. VI) are preferred because they are easy to bridge into Quad (resp. Complements) and the other side depends on variable bounds.

odow · 2019-10-08T13:54:19Z

Bridging currently assume that we can treat each constraint independently so it will be easy to bridge VariationalInequality to Complements but not Complements to VariationalInequality

Sure. I envisaged a one-way VariationalInequality to Complements bridge.

Since we have two solvers that currently support Complements as written, we could keep Complements for now. If, in the future, we have multiple solvers that support more general variational inequalities (e.g., PATH-VI), then we can revisit this discussion, probably after those solvers have experimented with the syntax as external sets.

blegat · 2019-10-08T15:27:12Z

We also need to add it to docs/src/apireference.md

odow · 2019-10-08T15:39:42Z

Before merging, we should wait for @xhub to sign-off.

frapac · 2019-10-09T08:33:45Z

@frapac does KNITRO just look at the bounds? Or does it do something different?

Once complementarity constraints defined inside Knitro, the solver forces the lower-bounds of the variables to be equal to 0.

blegat · 2019-10-09T10:11:14Z

Once complementarity constraints defined inside Knitro, the solver forces the lower-bounds of the variables to be equal to 0.

So Knitro.jl will throw an error of the variable lower are nonzero and if the upper bound is not +infinity ?

frapac · 2019-10-09T14:10:36Z

If the variable LB is lower than 0, no error is returned (Knitro should set automatically the lower bound equal to 0, but it appears that the control checks have been updated). If the variable LB is greater than 0, then Knitro returns:

ERROR: Infeasible problem specification.
       Variables x[2] and x[5] are complementary but
       constrained to be strictly positive.  At least one
       of these variables must be 0 at the solution.

* Test: add QP complementarity test * Add Mock test for QP complementarity test * Fix with comments in PR#918

src/Test/nlp.jl

xhub

Sorry for the late review. That looks very good. I just had a few suggestions about the wording in the doc.

src/Test/nlp.jl

src/sets.jl

xhub · 2019-10-14T13:31:37Z

I am not sure I followed the discussion of Complements vs VariationalInequality.
As @odow said, nothing prevents us from encoding a VI using Complements.
I understand that @blegat propose that we have a VariationalInequality set where the data for the set C is stored inside. That could be a great way to store the data.
It could be used whenever C can be described as Ax + b in K and K is a cone.

One just has to see how to educate the user about the need to specify the set C in this way and not via @constraints macros.

blegat · 2019-10-15T10:01:11Z

One just has to see how to educate the user about the need to specify the set C in this way and not via @constraints macros.

What do you mean ? The user can do @constraint(model, [A*x + b; x] in MOI.VariationalInequality(C)) to model $C^D\ni F(x) \perp x \in C$ where F(x) = Ax + b.

odow · 2019-10-17T02:08:20Z

As I've said before, I think we can leave the VariationalInequality set for a future solver. But I think @blegat's last suggestion is the way to go.

@xhub Are the latest changes okay?

xhub · 2019-10-21T13:48:05Z

@odow Thanks for making the changes. I am really happy with Complements as just some tagging information. The only part that I would maybe think about changing is the documentation for Complements. With the current implementation, and proper support from a solver, nothing prevents me from specifying a general polyhedral VI. Currently the doc mentions MCP, which is good. If we want to change the documentation, we could do that later.

@blegat I am not sure what V is in your example. Maybe it is best to move on on the Complements.
For VariationalInequalities I was thinking we could have the following

struct VariationalInequalities
   A::SparseMatrix
   b::AbstractVector
   K::Union{MOI.Zeros, MOI.Nonnegatives, MOI.Nonpositives}
end

I was thinking of a case where the user does the following in a variational inequality problem definition

@constraint(model, [A*x + b; x] in MOI.VariationalInequalities(H, k, MOI.Nonnegatives))
@constraint(model, x[2] >= 5)

Then, the full definition of C is the intersection of Hx <= k and x[2] >= 5. Then, the burden of
I must say that I may be too pessimistic about the user's knowledge. I may have in mind people who are not optimizers by training, rather use optimization algorithm as a tool.

blegat · 2019-10-21T13:54:10Z

I am not sure what V is in your example.

It should be C, I have corrected it. About the A and b field in VariationalInequality, I would prefer keeping VariationalInequality simple and have a new set AffineTransformation and so that the user does VariationalInequality(AffineTranformation(A, b, MOI.Nonnegatives(2))) but that's out the scope of this PR.

odow · 2019-10-21T15:00:34Z

If we want to change the documentation, we could do that later.

Yeah, it's easier to change to be more general later on.

odow added 2 commits October 7, 2019 20:01

Add Complements set

6adf621

Update docstring

097811e

Fix test/Test/Tests.jl

85c5a4f

frapac approved these changes Oct 8, 2019

View reviewed changes

src/Test/nlp.jl Outdated Show resolved Hide resolved

blegat reviewed Oct 8, 2019

View reviewed changes

src/Test/nlp.jl Outdated Show resolved Hide resolved

blegat reviewed Oct 8, 2019

View reviewed changes

src/sets.jl Show resolved Hide resolved

Update docstrings

730cc08

Fix optimal_status

b47d0eb

odow added 2 commits October 8, 2019 09:41

Update examples in docstring

0473168

More updates to docstring

a2a18f5

blegat approved these changes Oct 8, 2019

View reviewed changes

Add Complements to docs

9abb78e

Add AbstractVectorFunction to docs

d0b6f36

blegat added this to the v0.9.5 milestone Oct 8, 2019

blegat removed this from the v0.9.5 milestone Oct 9, 2019

frapac mentioned this pull request Oct 9, 2019

Wrap MOI.Complements to Knitro complementarity constraints jump-dev/KNITRO.jl#130

Merged

odow and others added 2 commits October 10, 2019 15:51

Tweak definition of complements

f812d34

Add QP complementarity test (#918)

2bc4f89

* Test: add QP complementarity test * Add Mock test for QP complementarity test * Fix with comments in PR#918

frapac reviewed Oct 11, 2019

View reviewed changes

src/Test/nlp.jl Outdated Show resolved Hide resolved

Update nlp.jl

e720cb5

xhub reviewed Oct 14, 2019

View reviewed changes

src/Test/nlp.jl Show resolved Hide resolved

src/sets.jl Show resolved Hide resolved

src/sets.jl Show resolved Hide resolved

Re-word definition of Complements and break tests into CP and MPCC

532cb68

Update example in Complements docstring

ad1d56b

blegat added this to the v0.9.6 milestone Oct 15, 2019

xhub approved these changes Oct 21, 2019

View reviewed changes

blegat approved these changes Oct 21, 2019

View reviewed changes

odow merged commit 97a6483 into master Oct 21, 2019

odow deleted the od/complements branch October 21, 2019 15:01

Add complementarity constraints #913

Add complementarity constraints #913

Uh oh!

Conversation

odow commented Oct 8, 2019

Uh oh!

codecov-io commented Oct 8, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

frapac left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

blegat commented Oct 8, 2019

Uh oh!

odow commented Oct 8, 2019

Uh oh!

odow commented Oct 8, 2019

Uh oh!

blegat commented Oct 8, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

odow commented Oct 8, 2019

Uh oh!

blegat commented Oct 8, 2019

Uh oh!

odow commented Oct 8, 2019

Uh oh!

frapac commented Oct 9, 2019

Uh oh!

blegat commented Oct 9, 2019

Uh oh!

frapac commented Oct 9, 2019

Uh oh!

Uh oh!

xhub left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

xhub commented Oct 14, 2019

Uh oh!

blegat commented Oct 15, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

odow commented Oct 17, 2019

Uh oh!

xhub commented Oct 21, 2019

Uh oh!

blegat commented Oct 21, 2019

Uh oh!

odow commented Oct 21, 2019

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

6 participants

codecov-io commented Oct 8, 2019 •

edited

Loading

blegat commented Oct 8, 2019 •

edited

Loading

blegat commented Oct 15, 2019 •

edited

Loading