How do I prevent one value from exceeding another value in a vector when minimizing it?

Question

When minimizing a function, I'm trying to prevent one value from exceeding another value in the used vector.

For example, this code optimizes init_x:

using Optim
f(x) = (1.0-x[1])^2+100.0*(-x[2]^2)^2

init_x = [0.0,0.0]
res = Optim.minimizer(Optim.optimize(f, init_x))

which returns:

2-element Vector{Float64}:
  0.9999860561511018
 -0.0006072923701139431

How would I prevent the first value of res (0.9999860561511018) from exceeding the second value of res (-0.0006072923701139431) during optimization? like using a constraint where res[1] < res[2]

Answer 1

I recommend using JuMP, but in case your model you can rewrite your model it by replacing x[2] with x[1]+x_[2] where x_[2] >=0 and using box constrained optimization eg.:

using Optim
f(x) = (1.0-x[1])^2+100.0*(-(x[1]+x[2])^2)^2

init_x = [0.01,0.01]

inner_optimizer = GradientDescent()
result = optimize(f, [-Inf, 0.0],[Inf,Inf], init_x, Fminbox(inner_optimizer))
res = Optim.minimizer(result)

This yields:

julia> final_res=[res[1], res[1]+res[2]]
2-element Vector{Float64}:
 0.16126202305154383
 0.16126202306334733

julia> f([res[1],res[2]])
0.7711096853639534

More general solution is to use JuMP

using JuMP, Ipopt
m = Model(optimizer_with_attributes(Ipopt.Optimizer, ("print_level"=>2))); 
@variable(m, x[1:2])

@constraint(m, x[1] <= x[2]  )

@NLobjective(m, Min, (1.0-x[1])^2+100.0*(-x[2]^2)^2)

optimize!(m)

To see the result:

julia> value.(x)
2-element Vector{Float64}:
 0.16126203113215182
 0.16126202262738015

julia> objective_value(m)
0.7711096710776033