4

I have a function which accepts a list R. In this function, I have defined an optimization problem using "pulp", This is my function:

import pulp
from multiprocessing.dummy import Pool as ThreadPool

def optimize(R):

    variables = ["x1","x2","x3","x4"]

    costs = {"x1":R[0], "x2":R[1], "x3":R[2], "x4":R[3]}

    constraint = {"x1":5, "x2":7, "x3":4, "x4":3}

    prob_variables = pulp.LpVariable.dicts("Intg",variables,
                              lowBound=0,
                              upBound=1,
                              cat=pulp.LpInteger)


    prob = pulp.LpProblem("test1", pulp.LpMaximize)

    # defines the constraints
    prob += pulp.lpSum([constraint[i]*prob_variables[i] for i in variables]) <= 14

    # defines the objective function to maximize
    prob += pulp.lpSum([costs[i]*prob_variables[i] for i in variables])

    pulp.GLPK().solve(prob)

    # Solution
    return pulp.value(prob.objective)

To get the output, I used a list as my input and the output is correct:

my_input = [[1,2,3,4],[5,6,7,8],[9,10,11,12],[13,14,15,16]]

results =[]
for i in range(0,len(my_input)):
    results.append(optimize(my_input[i]))

    print("*"*20)
    print(results)

But, I want to use multi-threading instead of the for loop. So, I used:

my_input = [[1,2,3,4],[5,6,7,8],[9,10,11,12],[13,14,15,16]]
pool = ThreadPool(4)
results = pool.map(optimize, my_input)

But it gives me some errors:

Traceback (most recent call last):
  File "/Users/Mohammad/PycharmProjects/untitled10/multi_thread.py", line 35, in <module>
    results = pool.map(optimize, my_input)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/multiprocessing/pool.py", line 260, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/multiprocessing/pool.py", line 608, in get
    raise self._value
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/multiprocessing/pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/multiprocessing/pool.py", line 44, in mapstar
    return list(map(*args))
  File "/Users/Mohammad/PycharmProjects/untitled10/multi_thread.py", line 27, in optimize
    pulp.GLPK().solve(prob)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/PuLP-1.6.1-py3.5.egg/pulp/solvers.py", line 179, in solve
    return lp.solve(self)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/PuLP-1.6.1-py3.5.egg/pulp/pulp.py", line 1643, in solve
    status = solver.actualSolve(self, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/PuLP-1.6.1-py3.5.egg/pulp/solvers.py", line 377, in actualSolve
    raise PulpSolverError("PuLP: Error while executing "+self.path)
pulp.solvers.PulpSolverError: PuLP: Error while executing glpsol

Can anybody help me? In my actual code, my_input list has the length of 27 (instead of 4 in the above code) and for each one, in my function I have to perform 80k optimizations (instead of one in the above code). So, multi-threading is a big help for me.

m0_as
  • 451
  • 1
  • 5
  • 13
  • "But it gives me some errors." Please *add* the error message. Always help people help you. – MisterMiyagi Jul 06 '16 at 07:12
  • @MisterMiyagi I added the error. Thanks. – m0_as Jul 06 '16 at 07:13
  • Are you sure it is entirely related to threading? Check threads such as http://stackoverflow.com/questions/32688324/pulpsolvererror-pulp-error-while-trying-to-execute-glpsol-in-python-2-7 to try to debug the error message beforehand. – Tttt1228 Jul 06 '16 at 08:43
  • @Tttt1228 Yes, because as I said, if I use for loop, it works without any error. – m0_as Jul 06 '16 at 17:45
  • @Tttt1228 I think the problem is that the optimization problem for all threads has the same name "prob", and I think that's where that's where it gives errors. But I don't know how to change the name "prob" for different thread. – m0_as Jul 06 '16 at 17:52

1 Answers1

2

I have seen that class pulp.solvers.COIN_CMD has a threads argument, although the documentation is quite laconic. Taking a look at the code source, it seems to be indeed a way to provide threads to the solver.

If naming is indeed the issue, consider adding the desired name index for a given problem as an input argument to the function. Something like:

def optimize(tup): # here, tup contains (idx, R), so as to be callable using pool.map
    ...
    prob = pulp.LpProblem('test'+str(idx), pulp.LpMaximize)
    ...

and then something like:

my_input = [[1,2,3,4],[5,6,7,8],[9,10,11,12],[13,14,15,16]]
pool = ThreadPool(4)
results = pool.map(optimize, enumerate(my_input))
Tttt1228
  • 409
  • 5
  • 11