JavaScript web server: Understanding blocking behavior during request handling

Question

I have a web server that needs to fetch and cache around 1000 items that come from a slow api.

I would like the first request and any that hit a ttl expiration to kick off this operation, but then coninue with the request. It has other values to return, and can serve stale or null for the soon-to-be refreshed data.

I've recreated a simplified example:

// ... within some handleRequest function
if (
  !cache.isFetching && 
  (!cache.isSet || cache.isStale)
) {
  // start fetch/cache, but don't block
  fetchAndCacheThatStuff();
}
// ... continue with request handling

I am trying to reason about how/when this will block when requests are coming in during cache refreshing:

async function fetchAndCacheThatStuff() {
  cache.isFetching = true;
  for (const twoHundoIds of chunk(oneThousandIds, 200) {
    const result = await fetchItems(twoHundoIds);
    await cache.mset(result.map(mapResultToCacheEntries));
  }
  cache.isSet = true;
  cache.isStale = false;
}

Is there a time when the server will be waiting for a response and unable to process another incoming request?

In a JavaScript web server, blocking behavior can occur during request handling when a certain operation takes a long time to complete. This can prevent the server from processing other requests until the blocking operation is finished, leading to slower response times and potential timeouts. For example, if a web server receives a request to read a large file, the server may need to wait for the entire file to be read before sending a response. — Yasir Mushtaq, Apr 04 '23 at 01:04

score 0 · Answer 1 · answered Apr 04 '23 at 02:31

The asynchronous function fetchAndCacheStuff will not block while waiting for a response.

From Node.js Design Patterns Chapter 5: Asynchronous Control Flow Patterns with Promises and Async/Await

At each await expression, the execution of the function is put on hold, its state saved, and the control returned to the event loop. Once the Promise that has been awaited resolves, the control is given back to the async function, returning the fulfillment value of the Promise.

JavaScript web server: Understanding blocking behavior during request handling

1 Answers1