Cache results in a Mirth channel that performs batch processing

Question

I'm stuck with a problem in Mirth. I have a Mirth channel that will receive a .csv file I'm performing a batch process in that channel. So, channel will read each row in the .csv file. I'm communicating each row and transforming into SOAP to communicate to other end. This is working fine.

I wanted to consolidate all the rows from the .csv file again and create a new .csv file and send it as a attachment to the sender. I figured out a way to attach .csv file and send in mail. But I cannot consolidate the rows (cache the row results) in the same channel.

Please let me know how to do this?

score 1 · Accepted Answer · answered May 02 '17 at 15:08

Sure, this is technically possible within one channel, but you'd have to be willing to cache messages in memory so you can send it all consolidated at the end.

In your deploy script, initialize a map:

if (!globalChannelMap.containsKey('csvMap')) {
    $gc('csvMap', Maps.map());
}

In your preprocessor, add each message to a consolidated CSV (essentially un-batching your batch message):

var csvMap = $('csvMap');
var batchId = $('batchId');
if ($('batchSequenceId') == 1) {
    csvMap.put(batchId, new java.lang.String(message));
} else {
    csvMap.put(batchId, csvMap.get(batchId).concat(message));
}
return message;

Then on your SMTP Sender destination, add a filter rule so it only triggers when the last message in a batch is processing:

On the same destination, add a transformer step that pulls the consolidated CSV out of the global channel map and sets the transformed data to it:

msg = SerializerFactory.getSerializer('DELIMITED').toXML($('csvMap').remove($('batchId')));

And there you go. Your channel will continue to process one row at a time, but when it's processing the last row in a batch, it will also trigger this other destination. I'll attach the channel (exported from 3.5.0) for illustration: Example - Consolidate CSV.xml

Of course, another perhaps saner way to do this would be to split it up into two channels. The upstream channel does not do any batching, but sends to your downstream channel as well as sending that e-mail. The downstream channel would have batching enabled, and that's where you'd move your Web Service Sender.

I have a doubt : In the filter portion of SMTP sender destination what value does the $(batchComplete) will hold? how do we know it is processing the last message of the file? — Vibin Guevara, May 02 '17 at 15:52
Because that's how the batchComplete source map variable works when batch processing is enabled. — Nick Rupley, May 02 '17 at 15:53
I'm getting the error "Maps" is not defined at the first place on the deploy script. How should I overcome it? — Vibin Guevara, May 03 '17 at 14:49
As I said, that channel was built for version 3.5. Since you did not specify what version you're using, I assumed you're using the latest (3.5). In earlier versions you'd create a new Java HashMap directly instead of using Maps.map(). — Nick Rupley, May 03 '17 at 15:52

score 0 · Answer 2 · answered May 02 '17 at 14:48

0

I'm not sure if this is doable or not. Have you considered saving the row to a file (append each row), and having another channel as a file reader to pick up that file? Make sure the aging is set to say a minute so it's not grabbing the file while it's being written.

answered May 02 '17 at 14:48

Rob

2,363
7
36
54

I have certain limitation here : This solution is already a 2 channel interface, I dont want to include third channel and complicate further. Also I do not have a luxury of writing the message to file and reading. – Vibin Guevara May 02 '17 at 15:53

Cache results in a Mirth channel that performs batch processing

2 Answers2