Nonlinear Optimization of absolute value function using ipopt solver

Question

How to Solve Nonlinear Optimization Problem when Objective function is a Modulus (absolute) valued function using IPOPT Solver in python?

Hi! Your question is too broad. Which part are you struggling with? Did you try the python bindings https://github.com/mechmotum/cyipopt? — psarka, Oct 07 '21 at 10:07
Abs(x) is non-differentiable. Often we can reformulate things. — Erwin Kalvelagen, Oct 07 '21 at 10:42
Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. — Community, Oct 11 '21 at 11:28

score 0 · Answer 1 · answered Jul 04 '22 at 23:19

Absolute abs(x) has a point at x=0 that is not continuously differentiable. There are two popular ways to reformulate the abs() function for continuous derivatives that are discussed in the optimization course.

Approach 1: MPCC

Use Mathematical Program with Complementarity Constraints (MPCC) as a first approach. This is implemented in Python Gekko with the m.abs2() function.

from gekko import GEKKO
# define new GEKKO model
m = GEKKO()
# calculate x to minimize objective
x = m.Var(1)
# use abs2 to define y
y = m.abs2(x)
# define objective to minimize
m.Minimize((y+3)**2)
# solve
m.solve()
# print solution
print('x: ' + str(x.value))
print('y: ' + str(y.value))

This is solved with IPOPT:

Number of Iterations....: 8

                                   (scaled)                 (unscaled)
Objective...............:   9.0000075849548888e-01    9.0000075849548882e+00
Dual infeasibility......:   3.3306690738754696e-16    3.3306690738754696e-15
Constraint violation....:   2.8561682212339413e-21    2.8561682212339413e-21
Complementarity.........:   3.8526843291314332e-07    3.8526843291314330e-06
Overall NLP error.......:   3.8526843291314332e-07    3.8526843291314330e-06


Number of objective function evaluations             = 9
Number of objective gradient evaluations             = 9
Number of equality constraint evaluations            = 9
Number of inequality constraint evaluations          = 0
Number of equality constraint Jacobian evaluations   = 9
Number of inequality constraint Jacobian evaluations = 0
Number of Lagrangian Hessian evaluations             = 8
Total CPU secs in IPOPT (w/o function evaluations)   =      0.005
Total CPU secs in NLP function evaluations           =      0.000

EXIT: Optimal Solution Found.
 
 The solution was found.
 
 The final value of the objective function is    9.00000758495489     
 
 ---------------------------------------------------
 Solver         :  IPOPT (v3.12)
 Solution time  :   1.110000000335276E-002 sec
 Objective      :    9.00000758495489     
 Successful solution
 ---------------------------------------------------
 
x: [-2.0813825544e-17]
y: [1.264092301e-06]

The advantage of the MPCC form is that it avoids binary variables that can be challenging to solve with MILP or MINLP solvers. The disadvantage is that solutions at the switch point have poor numerical properties and can get stuck at the saddle point.

Approach 2: Mixed Integer Optimization

Binary Variables are alternative method to create continuous gradients. Here is a simple example in Python Gekko:

from gekko import GEKKO
# define new GEKKO model
m = GEKKO()
# calculate x to minimize objective
x = m.Var(1.0)
# define new binary variable
intb = m.Var(0,lb=0,ub=1,integer=True)
# define y
y = m.Var()
# define equations
m.Equation((1-intb)*x <= 0)
m.Equation(intb * (-x) <= 0)
# output
m.Equation(y==(1-intb)*(-x) + intb*x)
# solve with APOPT (MINLP solver)
m.options.SOLVER=1
m.solve()
# print solution
print('x: ' + str(x.value))
print('intb: ' + str(intb.value))
print('y: ' + str(y.value))

or simplify with the use of the gekko m.abs3() function to implement these equations.

# with abs3 object
from gekko import GEKKO
# define new GEKKO model
m = GEKKO()
# variable
x = m.Var(-0.5)
# calculate y=abs(x) with abs3
y = m.abs3(x)
# solve with APOPT (MINLP solver)
m.solve()
# print solution
print('x: ' + str(x.value))
print('y: ' + str(y.value))

The disadvantage of this approach is that standard NLP solvers such as IPOPT are not designed to handle binary variables. This approach uses the APOPT solver that is also in Gekko.

 APMonitor, Version 1.0.1
 APMonitor Optimization Suite
 ----------------------------------------------------------------
 
 
 --------- APM Model Size ------------
 Each time step contains
   Objects      :            0
   Constants    :            0
   Variables    :            5
   Intermediates:            0
   Connections  :            0
   Equations    :            3
   Residuals    :            3
 
 Number of state variables:              5
 Number of total equations: -            3
 Number of slack variables: -            2
 ---------------------------------------
 Degrees of freedom       :              0
 
 ----------------------------------------------
 Steady State Optimization with APOPT Solver
 ----------------------------------------------
Iter:     1 I:  0 Tm:      0.00 NLPi:    2 Dpth:    0 Lvs:    0 Obj:  0.00E+00 Gap:  0.00E+00
 Successful solution
 
 ---------------------------------------------------
 Solver         :  APOPT (v1.0)
 Solution time  :   5.720000001019798E-002 sec
 Objective      :   0.000000000000000E+000
 Successful solution
 ---------------------------------------------------
 
x: [-0.16666666667]
y: [0.16666666667]

Nonlinear Optimization of absolute value function using ipopt solver

1 Answers1