0

In this question Mirth HTTP POST request with Parameters using Javascript I used a semblance of the first answer. Code seen below.

I'm running this code for a file that has nearly 46,000 rows. Which equates to about 46,000 requests hitting our external server. I'm noting that Mirth is making requests to our API endpoint about 1.6 times per second. This is unusually slow, and I would like some help to understand whether this is something related to Mirth, or related to the code above. Can repeated Imports in a for loop cause slow downs? Or is there a specific Mirth setting that limits the number of requests sent?

Version of Mirth is 3.12.0

Started the process at 2:27 PM and it's expected to be finished by almost 8:41 PM tonight, that's ridiculously slow.

//Skip the first header row
for (i = 1; i < msg['row'].length(); i++) {

    col1 = msg['row'][i]['column1'].toString();
    col2...
    ...

    //Insert into results if the file and sample aren't already present
    InsertIntoDatabase()
}

function InsertIntoDatabase() {
    with(JavaImporter(
        org.apache.commons.io.IOUtils,
        org.apache.http.client.methods.HttpPost,
        org.apache.http.client.entity.UrlEncodedFormEntity,
        org.apache.http.impl.client.HttpClients,
        org.apache.http.message.BasicNameValuePair,
        com.google.common.io.Closer)) {
        var closer = Closer.create();
        try {
            var httpclient = closer.register(HttpClients.createDefault());
            var httpPost = new HttpPost('http://<server_name>/InsertNewCorrection');

            var postParameters = [
                new BasicNameValuePair("col1", col1),
                new BasicNameValuePair(...
                ...
            ];

            httpPost.setEntity(new UrlEncodedFormEntity(postParameters, "UTF-8"));
            httpPost.setHeader('Content-Type', 'application/x-www-form-urlencoded')
            var response = closer.register(httpclient.execute(httpPost));
            var is = closer.register(response.entity.content);
            result = IOUtils.toString(is, 'UTF-8');
        } finally {
            closer.close();
        }
    }
    return result;
}
tisaconundrum
  • 2,156
  • 2
  • 22
  • 37
  • I was able to do some optimizations to get it to get up to 6 transactions per second. Still slow, but 3 times faster than before. This was done by removing the `Closer` package. I'm not sure what purpose that additional package does, but I can simply close the HTTPresponse like normal at the end without using Closer. Still hoping someone out there has some additional information on what Mirth might be doing to cause the slow down or if this is code related. – tisaconundrum Jul 22 '22 at 17:31
  • 1
    Try to make one call in Postman or curl and check how long it takes. Then you have your minimum time or in other words: Then you know to what extend you could theoretically optimize your performance. 6 transactions per seconds take like 166ms per transaction, which already sounds not too bad to me (depending on how your infrastructure looks like). If you really, really need to get faster, you could try working with java threads (like, slicing your data into 10 arrays and using 10 threads to work on each array or something like that), but then you're dealing with a whole different beast :) – Stefan Woehrer Jul 26 '22 at 13:32
  • 1
    The closer was closing the `response.entity.content` InputStream and the HttpClient itself. As @StefanWoehrer alluded to, wrapping this code in a loop is single threaded, and it will have to wait to process each response before sending the next request. – agermano Jul 27 '22 at 18:40

0 Answers0