CP-SAT is slow when it creates the goal function

Question

Could you explain to me what CP-SAT does when it created the goal function? With big models this action is very slow. the function is the following:

sum(sum(self.suggested_decisions[r][d]*(
    ((int(self.scenario.dataset['CAPITAL'][r])*self.scenario.dataset['BandA'][r]*self.scenario.dataset['SCORE_CLASS_1'][r]*int(self.scenario.constants['scaling_factor']*self.scenario.constants['recovery_rate_CA_S1'])*self.scenario.decisions[d].attributes['max_capital_recoverable']-int(self.scenario.constants['scaling_factor']*self.scenario.decisions[d].attributes['FA']*self.scenario.dataset['BandA'][r]*self.scenario.dataset['SCORE_CLASS_1'][r]))+
    (int(self.scenario.dataset['CAPITAL'][r])*self.scenario.dataset['BandB'][r]*self.scenario.dataset['SCORE_CLASS_1'][r]*int(self.scenario.constants['scaling_factor']*self.scenario.constants['recovery_rate_CB_S1'])*self.scenario.decisions[d].attributes['max_capital_recoverable']-int(self.scenario.constants['scaling_factor']*self.scenario.decisions[d].attributes['FB']*self.scenario.dataset['BandB'][r]*self.scenario.dataset['SCORE_CLASS_1'][r]))+
    (int(self.scenario.dataset['CAPITAL'][r])*self.scenario.dataset['BandC'][r]*self.scenario.dataset['SCORE_CLASS_1'][r]*int(self.scenario.constants['scaling_factor']*self.scenario.constants['recovery_rate_CC_S1'])*self.scenario.decisions[d].attributes['max_capital_recoverable']-int(self.scenario.constants['scaling_factor']*self.scenario.decisions[d].attributes['FC']*self.scenario.dataset['BandC'][r]*self.scenario.dataset['SCORE_CLASS_1'][r]))+
    (int(self.scenario.dataset['CAPITAL'][r])*self.scenario.dataset['BandD'][r]*self.scenario.dataset['SCORE_CLASS_1'][r]*int(self.scenario.constants['scaling_factor']*self.scenario.constants['recovery_rate_CD_S1'])*self.scenario.decisions[d].attributes['max_capital_recoverable']-int(self.scenario.constants['scaling_factor']*self.scenario.decisions[d].attributes['FD']*self.scenario.dataset['BandD'][r]*self.scenario.dataset['SCORE_CLASS_1'][r]))+
    (int(self.scenario.dataset['CAPITAL'][r])*self.scenario.dataset['BandE'][r]*self.scenario.dataset['SCORE_CLASS_1'][r]*int(self.scenario.constants['scaling_factor']*self.scenario.constants['recovery_rate_CE_S1'])*self.scenario.decisions[d].attributes['max_capital_recoverable']-int(self.scenario.constants['scaling_factor']*self.scenario.decisions[d].attributes['FE']*self.scenario.dataset['BandE'][r]*self.scenario.dataset['SCORE_CLASS_1'][r]))+
    (int(self.scenario.dataset['CAPITAL'][r])*self.scenario.dataset['BandA'][r]*self.scenario.dataset['SCORE_CLASS_2'][r]*int(self.scenario.constants['scaling_factor']*self.scenario.constants['recovery_rate_CA_S2'])*self.scenario.decisions[d].attributes['max_capital_recoverable']-int(self.scenario.constants['scaling_factor']*self.scenario.decisions[d].attributes['FA']*self.scenario.dataset['BandA'][r]*self.scenario.dataset['SCORE_CLASS_2'][r]))+
    (int(self.scenario.dataset['CAPITAL'][r])*self.scenario.dataset['BandB'][r]*self.scenario.dataset['SCORE_CLASS_2'][r]*int(self.scenario.constants['scaling_factor']*self.scenario.constants['recovery_rate_CB_S2'])*self.scenario.decisions[d].attributes['max_capital_recoverable']-int(self.scenario.constants['scaling_factor']*self.scenario.decisions[d].attributes['FB']*self.scenario.dataset['BandB'][r]*self.scenario.dataset['SCORE_CLASS_2'][r]))+
    (int(self.scenario.dataset['CAPITAL'][r])*self.scenario.dataset['BandC'][r]*self.scenario.dataset['SCORE_CLASS_2'][r]*int(self.scenario.constants['scaling_factor']*self.scenario.constants['recovery_rate_CC_S2'])*self.scenario.decisions[d].attributes['max_capital_recoverable']-int(self.scenario.constants['scaling_factor']*self.scenario.decisions[d].attributes['FC']*self.scenario.dataset['BandC'][r]*self.scenario.dataset['SCORE_CLASS_2'][r]))+
    (int(self.scenario.dataset['CAPITAL'][r])*self.scenario.dataset['BandD'][r]*self.scenario.dataset['SCORE_CLASS_2'][r]*int(self.scenario.constants['scaling_factor']*self.scenario.constants['recovery_rate_CD_S2'])*self.scenario.decisions[d].attributes['max_capital_recoverable']-int(self.scenario.constants['scaling_factor']*self.scenario.decisions[d].attributes['FD']*self.scenario.dataset['BandD'][r]*self.scenario.dataset['SCORE_CLASS_2'][r]))+
    (int(self.scenario.dataset['CAPITAL'][r])*self.scenario.dataset['BandE'][r]*self.scenario.dataset['SCORE_CLASS_2'][r]*int(self.scenario.constants['scaling_factor']*self.scenario.constants['recovery_rate_CE_S2'])*self.scenario.decisions[d].attributes['max_capital_recoverable']-int(self.scenario.constants['scaling_factor']*self.scenario.decisions[d].attributes['FE']*self.scenario.dataset['BandE'][r]*self.scenario.dataset['SCORE_CLASS_2'][r]))+
    (int(self.scenario.dataset['CAPITAL'][r])*self.scenario.dataset['BandA'][r]*self.scenario.dataset['SCORE_CLASS_3'][r]*int(self.scenario.constants['scaling_factor']*self.scenario.constants['recovery_rate_CA_S3'])*self.scenario.decisions[d].attributes['max_capital_recoverable']-int(self.scenario.constants['scaling_factor']*self.scenario.decisions[d].attributes['FA']*self.scenario.dataset['BandA'][r]*self.scenario.dataset['SCORE_CLASS_3'][r]))+
    (int(self.scenario.dataset['CAPITAL'][r])*self.scenario.dataset['BandB'][r]*self.scenario.dataset['SCORE_CLASS_3'][r]*int(self.scenario.constants['scaling_factor']*self.scenario.constants['recovery_rate_CB_S3'])*self.scenario.decisions[d].attributes['max_capital_recoverable']-int(self.scenario.constants['scaling_factor']*self.scenario.decisions[d].attributes['FB']*self.scenario.dataset['BandB'][r]*self.scenario.dataset['SCORE_CLASS_3'][r]))+
    (int(self.scenario.dataset['CAPITAL'][r])*self.scenario.dataset['BandC'][r]*self.scenario.dataset['SCORE_CLASS_3'][r]*int(self.scenario.constants['scaling_factor']*self.scenario.constants['recovery_rate_CC_S3'])*self.scenario.decisions[d].attributes['max_capital_recoverable']-int(self.scenario.constants['scaling_factor']*self.scenario.decisions[d].attributes['FC']*self.scenario.dataset['BandC'][r]*self.scenario.dataset['SCORE_CLASS_3'][r]))+
    (int(self.scenario.dataset['CAPITAL'][r])*self.scenario.dataset['BandD'][r]*self.scenario.dataset['SCORE_CLASS_3'][r]*int(self.scenario.constants['scaling_factor']*self.scenario.constants['recovery_rate_CD_S3'])*self.scenario.decisions[d].attributes['max_capital_recoverable']-int(self.scenario.constants['scaling_factor']*self.scenario.decisions[d].attributes['FD']*self.scenario.dataset['BandD'][r]*self.scenario.dataset['SCORE_CLASS_3'][r]))+
    (int(self.scenario.dataset['CAPITAL'][r])*self.scenario.dataset['BandE'][r]*self.scenario.dataset['SCORE_CLASS_3'][r]*int(self.scenario.constants['scaling_factor']*self.scenario.constants['recovery_rate_CE_S3'])*self.scenario.decisions[d].attributes['max_capital_recoverable']-int(self.scenario.constants['scaling_factor']*self.scenario.decisions[d].attributes['FE']*self.scenario.dataset['BandE'][r]*self.scenario.dataset['SCORE_CLASS_3'][r]))+
    (int(self.scenario.dataset['CAPITAL'][r])*self.scenario.dataset['BandA'][r]*self.scenario.dataset['SCORE_CLASS_4'][r]*int(self.scenario.constants['scaling_factor']*self.scenario.constants['recovery_rate_CA_S4'])*self.scenario.decisions[d].attributes['max_capital_recoverable']-int(self.scenario.constants['scaling_factor']*self.scenario.decisions[d].attributes['FA']*self.scenario.dataset['BandA'][r]*self.scenario.dataset['SCORE_CLASS_4'][r]))+
    (int(self.scenario.dataset['CAPITAL'][r])*self.scenario.dataset['BandB'][r]*self.scenario.dataset['SCORE_CLASS_4'][r]*int(self.scenario.constants['scaling_factor']*self.scenario.constants['recovery_rate_CB_S4'])*self.scenario.decisions[d].attributes['max_capital_recoverable']-int(self.scenario.constants['scaling_factor']*self.scenario.decisions[d].attributes['FB']*self.scenario.dataset['BandB'][r]*self.scenario.dataset['SCORE_CLASS_4'][r]))+
    (int(self.scenario.dataset['CAPITAL'][r])*self.scenario.dataset['BandC'][r]*self.scenario.dataset['SCORE_CLASS_4'][r]*int(self.scenario.constants['scaling_factor']*self.scenario.constants['recovery_rate_CC_S4'])*self.scenario.decisions[d].attributes['max_capital_recoverable']-int(self.scenario.constants['scaling_factor']*self.scenario.decisions[d].attributes['FC']*self.scenario.dataset['BandC'][r]*self.scenario.dataset['SCORE_CLASS_4'][r]))+
    (int(self.scenario.dataset['CAPITAL'][r])*self.scenario.dataset['BandD'][r]*self.scenario.dataset['SCORE_CLASS_4'][r]*int(self.scenario.constants['scaling_factor']*self.scenario.constants['recovery_rate_CD_S4'])*self.scenario.decisions[d].attributes['max_capital_recoverable']-int(self.scenario.constants['scaling_factor']*self.scenario.decisions[d].attributes['FD']*self.scenario.dataset['BandD'][r]*self.scenario.dataset['SCORE_CLASS_4'][r]))+
    (int(self.scenario.dataset['CAPITAL'][r])*self.scenario.dataset['BandE'][r]*self.scenario.dataset['SCORE_CLASS_4'][r]*int(self.scenario.constants['scaling_factor']*self.scenario.constants['recovery_rate_CE_S4'])*self.scenario.decisions[d].attributes['max_capital_recoverable']-int(self.scenario.constants['scaling_factor']*self.scenario.decisions[d].attributes['FE']*self.scenario.dataset['BandE'][r]*self.scenario.dataset['SCORE_CLASS_4'][r]))+
    (int(self.scenario.dataset['CAPITAL'][r])*self.scenario.dataset['BandA'][r]*self.scenario.dataset['SCORE_CLASS_5'][r]*int(self.scenario.constants['scaling_factor']*self.scenario.constants['recovery_rate_CA_S5'])*self.scenario.decisions[d].attributes['max_capital_recoverable']-int(self.scenario.constants['scaling_factor']*self.scenario.decisions[d].attributes['FA']*self.scenario.dataset['BandA'][r]*self.scenario.dataset['SCORE_CLASS_5'][r]))+
    (int(self.scenario.dataset['CAPITAL'][r])*self.scenario.dataset['BandB'][r]*self.scenario.dataset['SCORE_CLASS_5'][r]*int(self.scenario.constants['scaling_factor']*self.scenario.constants['recovery_rate_CB_S5'])*self.scenario.decisions[d].attributes['max_capital_recoverable']-int(self.scenario.constants['scaling_factor']*self.scenario.decisions[d].attributes['FB']*self.scenario.dataset['BandB'][r]*self.scenario.dataset['SCORE_CLASS_5'][r]))+
    (int(self.scenario.dataset['CAPITAL'][r])*self.scenario.dataset['BandC'][r]*self.scenario.dataset['SCORE_CLASS_5'][r]*int(self.scenario.constants['scaling_factor']*self.scenario.constants['recovery_rate_CC_S5'])*self.scenario.decisions[d].attributes['max_capital_recoverable']-int(self.scenario.constants['scaling_factor']*self.scenario.decisions[d].attributes['FC']*self.scenario.dataset['BandC'][r]*self.scenario.dataset['SCORE_CLASS_5'][r]))+
    (int(self.scenario.dataset['CAPITAL'][r])*self.scenario.dataset['BandD'][r]*self.scenario.dataset['SCORE_CLASS_5'][r]*int(self.scenario.constants['scaling_factor']*self.scenario.constants['recovery_rate_CD_S5'])*self.scenario.decisions[d].attributes['max_capital_recoverable']-int(self.scenario.constants['scaling_factor']*self.scenario.decisions[d].attributes['FD']*self.scenario.dataset['BandD'][r]*self.scenario.dataset['SCORE_CLASS_5'][r]))+
    (int(self.scenario.dataset['CAPITAL'][r])*self.scenario.dataset['BandE'][r]*self.scenario.dataset['SCORE_CLASS_5'][r]*int(self.scenario.constants['scaling_factor']*self.scenario.constants['recovery_rate_CE_S5'])*self.scenario.decisions[d].attributes['max_capital_recoverable']-int(self.scenario.constants['scaling_factor']*self.scenario.decisions[d].attributes['FE']*self.scenario.dataset['BandE'][r]*self.scenario.dataset['SCORE_CLASS_5'][r]))+
    (int(self.scenario.dataset['CAPITAL'][r])*self.scenario.dataset['BandA'][r]*self.scenario.dataset['SCORE_CLASS_6'][r]*int(self.scenario.constants['scaling_factor']*self.scenario.constants['recovery_rate_CA_S6'])*self.scenario.decisions[d].attributes['max_capital_recoverable']-int(self.scenario.constants['scaling_factor']*self.scenario.decisions[d].attributes['FA']*self.scenario.dataset['BandA'][r]*self.scenario.dataset['SCORE_CLASS_6'][r]))+
    (int(self.scenario.dataset['CAPITAL'][r])*self.scenario.dataset['BandB'][r]*self.scenario.dataset['SCORE_CLASS_6'][r]*int(self.scenario.constants['scaling_factor']*self.scenario.constants['recovery_rate_CB_S6'])*self.scenario.decisions[d].attributes['max_capital_recoverable']-int(self.scenario.constants['scaling_factor']*self.scenario.decisions[d].attributes['FB']*self.scenario.dataset['BandB'][r]*self.scenario.dataset['SCORE_CLASS_6'][r]))+
    (int(self.scenario.dataset['CAPITAL'][r])*self.scenario.dataset['BandC'][r]*self.scenario.dataset['SCORE_CLASS_6'][r]*int(self.scenario.constants['scaling_factor']*self.scenario.constants['recovery_rate_CC_S6'])*self.scenario.decisions[d].attributes['max_capital_recoverable']-int(self.scenario.constants['scaling_factor']*self.scenario.decisions[d].attributes['FC']*self.scenario.dataset['BandC'][r]*self.scenario.dataset['SCORE_CLASS_6'][r]))+
    (int(self.scenario.dataset['CAPITAL'][r])*self.scenario.dataset['BandD'][r]*self.scenario.dataset['SCORE_CLASS_6'][r]*int(self.scenario.constants['scaling_factor']*self.scenario.constants['recovery_rate_CD_S6'])*self.scenario.decisions[d].attributes['max_capital_recoverable']-int(self.scenario.constants['scaling_factor']*self.scenario.decisions[d].attributes['FD']*self.scenario.dataset['BandD'][r]*self.scenario.dataset['SCORE_CLASS_6'][r]))+
    (int(self.scenario.dataset['CAPITAL'][r])*self.scenario.dataset['BandE'][r]*self.scenario.dataset['SCORE_CLASS_6'][r]*int(self.scenario.constants['scaling_factor']*self.scenario.constants['recovery_rate_CE_S6'])*self.scenario.decisions[d].attributes['max_capital_recoverable']-int(self.scenario.constants['scaling_factor']*self.scenario.decisions[d].attributes['FE']*self.scenario.dataset['BandE'][r]*self.scenario.dataset['SCORE_CLASS_6'][r])))
    *(1*self.scenario.constants['scaling_factor']-int(self.scenario.constants['scaling_factor']*self.scenario.dataset['SCORE'][r]))
)for r in self.all_records) for d in self.all_decisions)

Where all self.all_records=60.000 and self.all_decisions=7

The language is python.

modified the question text with the info requested, The constraints with the same complexity takes few seconds to be created, while the goal function takes 20 minutes — stefano guerrieri, Apr 29 '20 at 09:16

score 1 · Answer 1 · answered Apr 29 '20 at 16:11

1

the sum() API creates an unbalanced syntax tree that is very expensive to parse for large objectives.

I suggest using cp_model.LinearExpr.ScalProd(variable_list, coefficient_list) (see doc entry)

answered Apr 29 '20 at 16:11

Laurent Perron

8,594
1
8
22

tried, but it's 10 seconds faster than previouse over about 20 minutes of total time needed to create the goal objective – stefano guerrieri Apr 30 '20 at 14:55
You are using only one suggest using cp_model.LinearExpr.ScalProd ? – Laurent Perron May 01 '20 at 13:31
Yes, I created a list containing the suggested_decisions and used it as variable_list, while the other part of my goal function is stored into the coefficient list. For clarification: the suggested_decisions are all boolean decision variables. – stefano guerrieri May 02 '20 at 16:50

CP-SAT is slow when it creates the goal function

1 Answers1