Reuse TCP connection with node-fetch in node.js

Question

I am using this function to call an external API

const fetch = require('node-fetch');

fetchdata= async function (result = {}) {
  var start_time = new Date().getTime();

    let response = await fetch('{API endpoint}', {
      method: 'post',
      body: JSON.stringify(result),
      headers: { 'Content-Type': 'application/json' },
      keepalive: true

    });

  console.log(response) 
  var time = { 'Respone time': + (new Date().getTime() - start_time) + 'ms' };
  console.log(time)
  return [response.json(), time];
  
}

The problem is that i am not sure that node.js is reusing the TCP connection to the API every time i use this function, eventhough i defined the keepalive property.

Reusing the TCP connection can significantly improve response time
Any suggestions will be welcomed.

score 20 · Accepted Answer · answered Jun 21 '20 at 14:53

As documented in https://github.com/node-fetch/node-fetch#custom-agent

const fetch = require('node-fetch');

const http = require('http');
const https = require('https');

const httpAgent = new http.Agent({ keepAlive: true });
const httpsAgent = new https.Agent({ keepAlive: true });
const agent = (_parsedURL) => _parsedURL.protocol == 'http:' ? httpAgent : httpsAgent;

const fetchdata = async function (result = {}) {
    var start_time = new Date().getTime();

    let response = await fetch('{API endpoint}', {
        method: 'post',
        body: JSON.stringify(result),
        headers: { 'Content-Type': 'application/json' },
        agent
    });

    console.log(response)
    var time = { 'Respone time': + (new Date().getTime() - start_time) + 'ms' };
    console.log(time)
    return [response.json(), time];

}

Thank you all! This was most helpful – Ofer B Jun 22 '20 at 09:17 — Ofer B, Jun 22 '20 at 09:17

score 2 · Answer 2 · edited Feb 21 '23 at 22:22

Here's a wrapper around node-fetch based on their documentation:

import nodeFetch, { RequestInfo, RequestInit, Response } from "node-fetch";
import http from "http";
import https from "https";

const httpAgent = new http.Agent({
  keepAlive: true
});

const httpsAgent = new https.Agent({
  keepAlive: true
});

export const fetch = (url: RequestInfo, options: RequestInit = {}): Promise<Response> => {
  return nodeFetch(url, {
    agent: (parsedURL) => {
      if (parsedURL.protocol === "http:") {
        return httpAgent;
      } else {
        return httpsAgent;
      }
    },
    ...options
  });
};

score 0 · Answer 3 · answered Jun 21 '20 at 14:54

Keep-alive is not enabled for the default used agent and is not currently implemented into node-fetch directly, but you can easily specify a custom-agent where you enable the keep-alive option:

const keepAliveAgent = new http.Agent({
    keepAlive: true
});

fetch('{API endpoint}', {
     ...
     agent: keepAliveAgent         
});

score 0 · Answer 4 · answered Apr 25 '21 at 10:36

here is a wrapper for node-fetch to add a keepAlive option, based on Ilan Frumer's answer

// fetch: add option keepAlive with default true
const fetch = (function getFetchWithKeepAlive() {
  const node_fetch = require('node-fetch');
  const http = require('http');
  const https = require('https');
  const httpAgent = new http.Agent({ keepAlive: true });
  const httpsAgent = new https.Agent({ keepAlive: true });
  return async function (url, userOptions) {
    const options = { keepAlive: true };
    Object.assign(options, userOptions);
    if (options.keepAlive == true)
      options.agent = (parsedUrl => parsedUrl.protocol == 'http:' ? httpAgent : httpsAgent);
    delete options.keepAlive;
    return await node_fetch(url, options);
  }
})();

const response = await fetch('https://github.com/');
const response = await fetch('https://github.com/', { keepAlive: false });

Reuse TCP connection with node-fetch in node.js

4 Answers4