Looping through elements of dictionary key instead of enumerating all

Question

I am working with python dictionaries to define some variables and parameters for a mathematical mode.

The data frame looks like this:

    Service Bill Weight Zone    Resi    UPS FedEx   USPS    DHL
    1DEA           1       2    N      33.02    9999    9999    9999
    2DAM           2       2    N      33.02    9999    9999    9999

I have defined some input and variable from this as follow:

cost = {}
for carrier in carriers:
   for row in df.to_dict('records'):
            key = (row['Service'], row['Bill Weight'], 
            row['Zone'],row['Resi'], carrier)
            cost[key] = row[carrier]

services = df['Service'].unique().tolist()
weights = df['Bill Weight'].unique().tolist()
zones = df['Zone'].unique().tolist()
addresses = df['Resi'].unique().tolist()

My only valid combinations for assign and cost should be:

 ['1DEA',1,2,'N','UPS']
 ['1DEA',1,2,'N','FedEx']
 ['1DEA',1,2,'N','USPS']
 ['1DEA',1,2,'N','DHL']
 ['2DAM',2,2,'N','UPS']
 ['2DAM',2,2,'N','FedEx']
 ['2DAM',2,2,'N','USPS']
 ['2DAM',2,2,'N','DHL']

And the following with gurobi python but I am really only concerned about the construction of my loops through python than the gurobi syntax:

Approach A:

assign = {}
for carrier in carriers:
   for row in df.to_dict('records'):
            key = (row['Service'], row['Bill Weight'], 
            row['Zone'],row['Resi'], carrier)
            cost[key] = row[carrier]
 obj = quicksum(cost[key]*assign[key] \
            for key in assign)

Now this works fine for making sure the variables and parameters are generated only from dictionary keys and not all possible combinations of services, weights, zones, and addresses. But it won't work when I have specific constraints like below:

  m.addConstrs((assign['1DEA', w, z, r, 'UPS']+assign['1DEA', w, z, r, 'USPS']+assign['1DEA', w, z, r, 'USPS 1C']==1\
          for i in clients for s in services for w in weights for z in zones for r in addresses),"C02")

Approach 2:

  assign = m.addVars(services, weights, zones, addresses, carriers, name = "Assign", vtype=GRB.BINARY)
  obj = quicksum(cost[s, w, z, r, l]*assign[ s, w, z, r, l] \
           for s in services for w in weights for z in zones for r in addresses for l in carriers)

This way I can easily write all my constraints but it will create all combinations of services, weights, zones, addresses, carriers which makes my model wrong. for example ['2DAM',1,2,'N','UPS'] is not a valid combination.

Is there a way to limit this looping on services, weights, zones, addresses, carriers to only combinations that are defined in cost dictionary keys?

David Nehme · Answer 1 · 2018-10-08T23:14:17.487

Since you have your data in a pandas data frame already, you could use its features to create the variables and cosntraints. Create a column with the decision variable, then use 'groupby' and grb.quicksum to define your constraints.

First, more pythonic column names

df.columns = ['service', 'bill_weight', 'zone', 'resi', 'UPS', 'FedEx', 'USPS', 'DHL']

Then, reshape the dataframe into a convenient form.

df1 = (df.set_index(['service', 'bill_weight', 'zone', 'resi']).
          rename_axis('carrier', axis=1).stack().to_frame('cost'))

The new dataframe will have one row per variable.

                                          cost
service bill_weight zone resi carrier         
1DEA    1           2    N    UPS        33.02
                              FedEx    9999.00
                              USPS     9999.00
                              DHL      9999.00
2DAM    2           2    N    UPS        33.02
                              FedEx    9999.00
                              USPS     9999.00
                              DHL      9999.00

You can create the variable (and add them to the objective with.

df1['assign'] = [m.addVar(name=".".join(map(str, row.Index), 
                 obj=row.cost, vtype='B') 
                 for row in df1.itertuples()]
m.update()

Now the frame will include the decision variables.

                                          cost                         assign
service bill_weight zone resi carrier                                        
1DEA    1           2    N    UPS        33.02    <gurobi.Var 1DEA.1.2.N.UPS>
                              FedEx    9999.00  <gurobi.Var 1DEA.1.2.N.FedEx>
                              USPS     9999.00   <gurobi.Var 1DEA.1.2.N.USPS>
                              DHL      9999.00    <gurobi.Var 1DEA.1.2.N.DHL>
2DAM    2           2    N    UPS        33.02    <gurobi.Var 2DAM.2.2.N.UPS>
                              FedEx    9999.00  <gurobi.Var 2DAM.2.2.N.FedEx>
                              USPS     9999.00   <gurobi.Var 2DAM.2.2.N.USPS>
                              DHL      9999.00    <gurobi.Var 2DAM.2.2.N.DHL>

Finally, you can use pandas groupby to add a constraints like the one in your question

lhs = (df1.groupby(level=['service', 'bill_weight', 
                          'zone', 'resi']).assign.apply(grb.quicksum)
single_carrier_constrs = [m.addConstr(l == 1 for l in lhs]

`df1['assign'] = [m.addVar(name=".".join(map(str, row.Index), obj=row.cost, vtype='B')] m.update()` doesn't seem to be working for me. Do you mind sharing how you defined `row`? @David Nehme — Nazanin Zinouri, Sep 21 '18 at 19:01

Looping through elements of dictionary key instead of enumerating all

1 Answers1