Optaplanner - Chained planning entity 'corruption' when using multiple planning variables

Question

I am new to Optaplanner, and I have been looking at extending the VRP into a different problem space. I'm using 6.1.0-Final. It is quite a difficult problem to aticulate, but here goes:

Planning Entity - Shipment (extends Standstill), anchored by Transporter (i.e. a cargo ship) Planning Variables - Terminal (a cargo destination), and of course implicitly Transporter (as a Shadow Variable like Vehicle in the VRP example)

In this routing problem, a shipment is transported directly to a terminal, then back to the Facility. i.e. a transporter serves one Terminal then returns to the Facility. There is a fleet of transporters with different capacities/speeds. The terminal has a Location, which defines the distance between it and the Facility. A transporter may server multiple shipments, but constrainted by the round trip journey time and 'scheduled departure window' of each shipment. Therefore, a chain is created with Transporter as the anchor, linked to all the Shipments it is serving.

I have pretty much replicated the VRP with time windows example taking into account this problem domain. There are shadow variables called previousStandstill and arrivalTime (this is the arrival at the Facility not the Terminal). I have modified the ArrivalTimeUpdatingVariableListener so that it calculates the round trip time from the departure of the last shipment back to the Facility. I have hard contraints to ensure that the Transporter associated with a Shipment is in fact available within the scheduled departure window, has enough capacity etc. I am also trying to maximise the profit based on a number of attributes of the shipment, transporter and terminal.

The problem I am having is that there seems to be a mismatch between the arrivalTime attribute against a Shipment and the calculated arrival time from the previous shipment (departure time + travel time). This only happens with some shipments, not all. Since there is a dependency on the terminal as well as the transporter in the arrival time calculation I also tried to add another shadow variable called 'previousTerminal', and used this to try and ensure that the previousTerminal against a Shipment is equal to the terminal against the previousStandstill (using a TerminalUpdatingVariableListener).

However, this does not seem to work either and I am seeing a mismatch between the 'previousTerminal' on the Shipment instance and the terminal of the previousStandstill (Shipment). Again, this only happens for some of the Shipments. It is as if the Terminal attribute against the previous shipment gets updated without triggering the afterVariableChanged event on the TerminalUpdatingVariableListener.

Here's an example output. Here you can see that the shipments with '***************Mismatching shipment transit times*******************' after them have a 'previous terminal' that does not match the terminal of the previous shipment in the chain.

Shipment 00001, amount 150000.0, terminal Terminal G, transporter North Sea LNG 001, previous shipment null, prev shp term Terminal J, lastShipmentTransitTime 0, thisShipmentTransitTime 5844  
Shipment 00002, amount 180000.0, terminal Terminal J, transporter North Sea LNG 004, previous shipment null, prev shp term Terminal J, lastShipmentTransitTime 0, thisShipmentTransitTime 9179  
Shipment 00003, amount 250000.0, terminal Terminal B, transporter North Sea LNG 002, previous shipment null, prev shp term Terminal J, lastShipmentTransitTime 0, thisShipmentTransitTime 6656   
Shipment 00004, amount 180000.0, terminal Terminal J, transporter North Sea LNG 003, previous shipment null, prev shp term Terminal G, lastShipmentTransitTime 0, thisShipmentTransitTime 10199  
Shipment 00005, amount 150000.0, terminal Terminal J, transporter North Sea LNG 006, previous shipment null, prev shp term Terminal J, lastShipmentTransitTime 0, thisShipmentTransitTime 11474  
Shipment 00006, amount 150000.0, terminal Terminal J, transporter North Sea LNG 004, previous shipment 00002, prev shp term Terminal J, lastShipmentTransitTime 9179, thisShipmentTransitTime 9179  
Shipment 00007, amount 100000.0, terminal Terminal J, transporter North Sea LNG 001, previous shipment 00001, prev shp term Terminal G, lastShipmentTransitTime 5844, thisShipmentTransitTime 12239  
Shipment 00008, amount 250000.0, terminal Terminal G, transporter North Sea LNG 002, previous shipment 00003, prev shp term Terminal B, lastShipmentTransitTime 6656, thisShipmentTransitTime 5157  
Shipment 00009, amount 200000.0, terminal Terminal C, transporter North Sea LNG 003, previous shipment 00004, prev shp term Terminal J, lastShipmentTransitTime 10199, thisShipmentTransitTime 11001  
Shipment 00010, amount 150000.0, terminal Terminal E, transporter North Sea LNG 001, previous shipment 00007, prev shp term Terminal G, lastShipmentTransitTime 5844, thisShipmentTransitTime 15085  
***************Mismatching shipment transit times*******************  
Shipment 00011, amount 200000.0, terminal Terminal J, transporter North Sea LNG 003, previous shipment 00009, prev shp term Terminal B, lastShipmentTransitTime 6286, thisShipmentTransitTime 10199  
***************Mismatching shipment transit times*******************  
Shipment 00012, amount 100000.0, terminal Terminal J, transporter North Sea LNG 006, previous shipment 00005, prev shp term Terminal J, lastShipmentTransitTime 11474, thisShipmentTransitTime 11474  
Shipment 00013, amount 250000.0, terminal Terminal G, transporter North Sea LNG 002, previous shipment 00008, prev shp term Terminal G, lastShipmentTransitTime 5157, thisShipmentTransitTime 5157  
Shipment 00014, amount 200000.0, terminal Terminal E, transporter North Sea LNG 003, previous shipment 00011, prev shp term Terminal J, lastShipmentTransitTime 10199, thisShipmentTransitTime 12571  
Shipment 00015, amount 150000.0, terminal Terminal E, transporter North Sea LNG 006, previous shipment 00012, prev shp term Terminal J, lastShipmentTransitTime 11474, thisShipmentTransitTime 14143  
Shipment 00016, amount 150000.0, terminal Terminal J, transporter North Sea LNG 001, previous shipment 00010, prev shp term Terminal E, lastShipmentTransitTime 15085, thisShipmentTransitTime 12239  
Shipment 00017, amount 200000.0, terminal Terminal J, transporter North Sea LNG 003, previous shipment 00014, prev shp term Terminal J, lastShipmentTransitTime 10199, thisShipmentTransitTime 10199  
***************Mismatching shipment transit times*******************

I have spent considerable time trying to understand what is going on here. I've tried changing the ordering in the ShadowVariable sources, and lots of other trial and error attempts, but nothing seems to resolve this chain'corruption'.

Perhaps it is my ignorance of the tool, but would appreciate some guidance/pointers as I have otherwise been very impressed so far with what I have seen.

Many thanks in advance for any support, and patience reading all this!

UPDATE - following 'FULL_ASSERT' mode. The following exception is now raised (didn't see this before)...not sure what it means though...

Exception in thread "AWT-EventQueue-0" java.lang.IllegalStateException: Solving failed. ... Caused by: " java.lang.IllegalStateException: Score corruption: the workingScore (-2957hard/57945soft) is not the uncorruptedScore (-2957hard/57964soft) after completedAction (Shipment 00002, amount 180000.0, terminal Terminal G, transporter North Sea LNG 002, previous shipment null, prev shp term Terminal B, lastShipmentTransitTime 0, thisShipmentTransitTime 5157 => [Terminal-8]): The corrupted scoreDirector has 5 ConstraintMatch(s) which are in excess (and should not be there): org.optaplanner.examples.shipscheduling.solver/arrivalAfterLoadingWindow/level0/[Shipment 00003, amount 250000.0, terminal Terminal B, transporter North Sea LNG 002, previous shipment 00002, prev shp term Terminal G, lastShipmentTransitTime 5157, thisShipmentTransitTime 6656]=-4456 org.optaplanner.examples.shipscheduling.solver/arrivalAfterLoadingWindow/level0/[Shipment 00008, amount 250000.0, terminal Terminal B, transporter North Sea LNG 002, previous shipment 00003, prev shp term Terminal B, lastShipmentTransitTime 6656, thisShipmentTransitTime 6656]=-112 org.optaplanner.examples.shipscheduling.solver/maximise profit/level1/[Shipment 00013, amount 250000.0, terminal Terminal B, transporter North Sea LNG 002, previous shipment 00008, prev shp term Terminal B, lastShipmentTransitTime 6656, thisShipmentTransitTime 6656]=4009 org.optaplanner.examples.shipscheduling.solver/maximise profit/level1/[Shipment 00008, amount 250000.0, terminal Terminal B, transporter North Sea LNG 002, previous shipment 00003, prev shp term Terminal B, lastShipmentTransitTime 6656, thisShipmentTransitTime 6656]=10410 org.optaplanner.examples.shipscheduling.solver/terminal not big enough for transporter/level0/[[Terminal-2], [Transporter-2], Shipment 00013, amount 250000.0, terminal Terminal B, transporter North Sea LNG 002, previous shipment 00008, prev shp term Terminal B, lastShipmentTransitTime 6656, thisShipmentTransitTime 6656]=-70000 The corrupted scoreDirector has 3 ConstraintMatch(s) which are missing: org.optaplanner.examples.shipscheduling.solver/arrivalAfterLoadingWindow/level0/[Shipment 00003, amount 250000.0, terminal Terminal B, transporter North Sea LNG 002, previous shipment 00002, prev shp term Terminal G, lastShipmentTransitTime 5157, thisShipmentTransitTime 6656]=-2957 org.optaplanner.examples.shipscheduling.solver/maximise profit/level1/[Shipment 00008, amount 250000.0, terminal Terminal B, transporter North Sea LNG 002, previous shipment 00003, prev shp term Terminal B, lastShipmentTransitTime 6656, thisShipmentTransitTime 6656]=10398 org.optaplanner.examples.shipscheduling.solver/maximise profit/level1/[Shipment 00013, amount 250000.0, terminal Terminal B, transporter North Sea LNG 002, previous shipment 00008, prev shp term Terminal B, lastShipmentTransitTime 6656, thisShipmentTransitTime 6656]=10316 Check your score constraints....

UPDATE - updated the Optaplanner core to 6.2.0.CR4 and there is a different exception with the 'FULL_ASSERT' enabled. If I disable the arrival time rule check, then I no longer see any exceptions...but I am struggling to see why this is. Is there something specific about the 'undo move' I need to know about?

Caused by: java.lang.IllegalStateException: The moveClass (class org.optaplanner.core.impl.heuristic.move.CompositeMove)'s move ([Shipment 00009, amount 200000.0, assigned terminal null, maxTerminalCapacity null, and transporter null, capacity null, profit 0, periodStart 18000, periodEnd 18200, arrival time18000, departure time = 18000, return time 18000, lastShipmentTransitTime 0, thisShipmentTransitTime 0, previous shipment null => [Transporter-2], Shipment 00009, amount 200000.0, assigned terminal null, maxTerminalCapacity null, and transporter null, capacity null, profit 0, periodStart 18000, periodEnd 18200, arrival time18000, departure time = 18000, return time 18000, lastShipmentTransitTime 0, thisShipmentTransitTime 0, previous shipment null => [Terminal-2]]) probably has a corrupted undoMove ([Shipment 00009, amount 200000.0, assigned terminal null, maxTerminalCapacity null, and transporter null, capacity null, profit 0, periodStart 18000, periodEnd 18200, arrival time18000, departure time = 18000, return time 18000, lastShipmentTransitTime 0, thisShipmentTransitTime 0, previous shipment null => null, Shipment 00009, amount 200000.0, assigned terminal null, maxTerminalCapacity null, and transporter null, capacity null, profit 0, periodStart 18000, periodEnd 18200, arrival time18000, departure time = 18000, return time 18000, lastShipmentTransitTime 0, thisShipmentTransitTime 0, previous shipment null => null]). Or maybe there are corrupted score rules. Check the Move.createUndoMove(...) method of that Move class and enable EnvironmentMode FULL_ASSERT to fail-faster on corrupted score rules. Score corruption: the lastCompletedStepScore (-6256hard/0soft) is not the undoScore (-19256hard/0soft).

UPDATE - FIX

Ok after a lot of digging around I have found that removal of:

&& ObjectUtils.notEqual(shadowShipment.getArrivalTime(), arrivalTime)

from

while (shadowShipment != null) && ObjectUtils.notEqual(shadowShipment.getArrivalTime(), arrivalTime)) {

within the ArrivalTimeUpdatingVariableListener seems to fix the problem, though I am not entirely sure why yet...I guess this will slow it down somewhat though.

Does turning on enviromnentMode FULL_ASSERT make it fail sooner? — Geoffrey De Smet, Mar 02 '15 at 10:54
Hi - I now see an exception raised, which wasn't there before...ScoreCorruption — Justin, Mar 02 '15 at 11:03
That means that it fails sooner, a lot nearer to when the corruption is first created. Turn on TRACE logging. Look at the move described in that score corruption exception. See if you can debug your variable listeners when that move happens to figure out what's going wrong. — Geoffrey De Smet, Mar 02 '15 at 12:46
Thanks Geoffrey - what is the best way to debug the variable listener when looking for a specific move? — Justin, Mar 03 '15 at 13:55

score 0 · Accepted Answer · edited Jan 10 '20 at 14:03

Thanks for the pointers Geoffrey.

The issue was related to the while loop breaking once it found a shipment where the arrival time had not changed. There must be certain situations where the arrival time was the same for the shadowShipment, but not for subsequent shipments in the chain.

I have updated the ArrivalTimeUpdatingListener as follows (and this works fine):

protected void updateArrivalTime(ScoreDirector scoreDirector, Shipment sourceShipment) {
    // First get the previous shipment and the departure time. 
    // If the the previousStandstill is a transporter then the departure time will be null.
    Standstill previousStandstill = sourceShipment.getPreviousStandstill();
    Integer departureTime = (previousStandstill instanceof Shipment)
            ? ((Shipment) previousStandstill).getDepartureTime() : null;
    Shipment shadowShipment = sourceShipment;
    // Calculate the arrival time of the source shipment based on the departure time of the last.
    Integer arrivalTime = calculateArrivalTime(shadowShipment, departureTime);
    // Loop through each shipment in the chain from the source shipment forwards.
    while (shadowShipment != null) {
        // Only update the arrival time if it has changed.
        if(ObjectUtils.notEqual(shadowShipment.getArrivalTime(), arrivalTime)) {
            scoreDirector.beforeVariableChanged(shadowShipment, "arrivalTime");
            shadowShipment.setArrivalTime(arrivalTime);
            scoreDirector.afterVariableChanged(shadowShipment, "arrivalTime");
        }
        // Set the variables for the next loop.
        departureTime = shadowShipment.getDepartureTime();
        shadowShipment = shadowShipment.getNextShipment();
        arrivalTime = calculateArrivalTime(shadowShipment, departureTime);
    }
}

Optaplanner - Chained planning entity 'corruption' when using multiple planning variables

1 Answers1