Azure Function - port exhaustion in node app

Question

I'm trying to build an Azure Function that makes a large number of outgoing HTTP requests. Periodically it seems to hit a limit and all requests time out for a couple minutes. Then it starts working again and requests go through.

From searching around, it sounds like this might be due to port exhaustion on the machine running the function, but I don't understand how to debug this or work around it in a Node app using the request library. It also sounds like Node is supposed to pool connections to prevent this. I'm not at all sure port exhaustion is the problem, since I can't use netstat on the Function. And I never have an issue running on my laptop.

Here's a simple Azure Function that fires lots of requests, one at a time, and illustrates this issue:

const cp = require('child_process');
const request = require('request');

module.exports = function (context, myTimer) {
    context.log('Starting');

    function doCall(cb) {
        const url = 'https://jsonplaceholder.typicode.com/posts';
        request(url, (err) => {
            if (err) {
                context.log("error: " + err.toString());
            }

            cb();
        });
    }

    let i = 500;
    doCall(function iterate() {
        if (i-- > 0) {
            context.log('iterate ' + i);
            doCall(iterate);
        } else {
            context.log('done');
            context.done();
        }
    });
};

I see it run successfully, time out for a couple minutes, run successfully again...

Maybe reducing the pool side would help. Not super familiar with Node, but see is something on https://stackoverflow.com/questions/19043355/how-to-use-request-js-node-js-module-pools is helpful. — David Ebbo, Sep 26 '17 at 23:54

score 0 · Accepted Answer · edited Jan 17 '18 at 04:21

@david-ebbo, your suggestion was very helpful. From the link you posted, I can use an http Agent to limit the pool of sockets. It takes a while to run, but now no requests time out.

My example from above now looks like this:

const cp = require('child_process');
const request = require('request');
const http = require('http');

module.exports = function (context, myTimer) {
    context.log('Starting with agent');
    const pool = new http.Agent();
    pool.maxSockets = 4;

    function doCall(cb) {
        const url = 'https://jsonplaceholder.typicode.com/posts';
        request({ url, pool }, (err) => {
            if (err) {
                context.log("error: " + err.toString());
            }

            cb();
        });
    }

    let i = 500;
    doCall(function iterate() {
        if (i-- > 0) {
            context.log('iterate ' + i);
            doCall(iterate);
        } else {
            context.log('done');
            context.done();
        }
    });
};

Azure Function - port exhaustion in node app

1 Answers1