Change priority rule of a Queue block at runtime in Anylogic

Question

I am trying to implement reinforcement learning in Anylogic using pathmind library, the RL agent can take either of two actions which is changing the priority rule of a Queue block.

I have a Queue block where I'm using priority-based queueing. I have two priority rules: using agent's departure date & agent's wait time. I want to either of these rules during runtime using another function called doAction(action). A value 0 or 1 will be passed to this function. The function body would be like this:

doAction(action){

if(action==0){
//set departure_date as priority rule of Queueblock}

else{
//set wait_time as priority rule of Queueblock}

}

The expression of my queue block is given here. .

RL parameters are mentioned here.

What should be the code to set priority rule dynamically from the doAction(action) function?

Answer 1

To use priorities, you specify an expression to determine the priority of the agent in the Queue's "Agent priority" or "Agent 1 is preferred to agent 2" property (depending what priority scheme you're using).

So have that expression be calling a function (defined within the agent type in question) which returns either the departure date or wait-time alternative.

Also, you didn't say whether this is a global setting --- i.e., use either departure or wait-time-based priorities for the whole run --- or could change dynamically; if you want the latter, you potentially need to call the sortAgents function of the Queue block (which might be inside a Service or Seize block, depending what you're doing) at the appropriate times (i.e., when your prioritisation scheme changes) to re-calculate all the priorities for agents currently waiting in the queue.

EDIT: I see from your other comment that you're trying to use reinforcement learning, presumably learning how to make a decision on how to prioritise the agents. (You should put that in an edit to your question since it's pretty important and relevant!)

So if you view the queue as the 'learning agent', you need to separate the learning action (which will set up / decide which prioritisation scheme you're using) from then using that scheme in the prioritisation.

This depends on whether you're using a Queue on its own (with priority based or agent comparison queueing), or you're doing this within a Service or Seize block. It matters because the on-enter action of the latter runs before the priority calculation expression but, with a plain Queue, it runs after the priority calculation.

Case 1: Using Service or Seize block

Have the on-enter action be the RL action which would then, say, set some variable to say which prioritisation scheme it had chosen and then call sortAgents on its embedded queue (self.queue) to recalc all the priorities. Then have switches in the priority calculation expression as above to do the calculation for the incoming agent using the required scheme.

Case 2: Using a plain Queue block

As above, but do the prioritisation scheme decision in the on-at-exit actions of all immediately preceding blocks (i.e., so that this is run just before the agent arrives at the Queue block and has its prioritisation allocated).

Answer 2

I would suggest to rather make the priority rule dynamic inside the queue.

I assume you have some agent with a field for departureTime as well as for waitingTime.

Then you can do something like the following: You simply have a different priority level for each agent if the priority option changes. Here I am using boolean useDaprtureTime, but you can make it as complex as you need and even have a function in the "Agent Priority" field that returns the priority level.

Just remember that you need to call queue.sortAgents() if you change the rule since only the new agents that arrive are sorted, not the entire list of agents waiting in the queue since this will be too resource-intensive.

Answer 3

You can always use 2 queue blocks and send agents to only one using a SelectOutput block in front of them.

Each agent decides which queue to use based on your conditions.