0

I have built an orchestration with a loop to retreive paged data from REST web service. From page size and offset I am able to call the service for "next page" of data. Then I debatch it, map it to internal format and process it further. When one page is processed, I request next page from the REST web service.

As it turns out, the host running the orchestration and send ports causes the memory to constantly grow during processing of all the data, and eventually hit the throttling mode.

Why is memory not released when I am done with one page loop? Is it the "consumed" messages that are stored in the orchestration that builds up the memory? Is it possible to clear orchestration from these "consumed" messages, to release the memory used? (No message tracking active on the orchestration, or send ports.)

JERKER
  • 907
  • 8
  • 17

2 Answers2

1

Apparently, there is no way to prohibit BizTalk Orchestrations from building up a list of messages in Orchestration, including used/processed/consumed messages. Putting things in Scope does not prohibit this behaviour.

Hence, for long-running Orchestrations there can be a lot of messages building up. Especially for singleton Orchestrations, where the general solution proposed to deal with this problem is to make sure Orchestration shuts down once in a while (when idle, e.g.).

My solution was to split the Orchestration into two, and have the initial Orchestration start the second Orchestration with the Start Orchestration, which in turn calls the second Orchestration recursively, and so on, until last page is received and the last Orchestration ends.

JERKER
  • 907
  • 8
  • 17
0

Yes, what you need to do is to have scopes, and that the messages are initialised in the scope (Green highlight below) rather than an the top level (yellow), and that means they will also be disposed of at the end of the scope. Note: That means those message can't be used outside of the scope.

enter image description here

However if you are just re-using the same messages in the loop, then I wouldn't expect it to increase memory usage. So there is possibly something else going on. I suspect that you must be adding each page to a message, and that is what is growing

Dijkgraaf
  • 11,049
  • 17
  • 42
  • 54
  • Interesting input, thank you. I use scope partially, but apparently not for all the messages used in orchestration. I will try to incude all messages used in loop, and check the difference. Hasta la vista! – JERKER Sep 16 '21 at 12:25
  • No apparent difference to include more messages into the scope I already have. Orchestration keeps building up the memory as new pags/loops are traversed. I will try a bit further...any other input about transaction scope/types? – JERKER Sep 16 '21 at 14:47
  • @JERKER Are you adding each page traversed to a single message? – Dijkgraaf Sep 16 '21 at 20:49
  • I am calling Send-Receive port with the same message variable as assigned message. I have no logic to append to that message, but using standard orchestration shapes and editor. So, I assumed it would replace the variable reference, and garbage collect the memory...? – JERKER Sep 17 '21 at 07:08
  • Apparently, there is no way to prohibit the orchestration history / the "consumed" messages to be persisted during the lifetime of the orchestration. My solution was to split this orchestration into a recursive loop instead, using the Start Orchestration shape. Now orchestration(s) performs well. – JERKER Sep 21 '21 at 06:24