Is there a way to slow down async-await fetch calls inside a for-loop/forEach in order to avoid per-second API quotas?

Question

I am working on an app that requires calls to the foursquare places api, which has a 2-calls-per-second quota. The app pulls a list of places, an then has to separately call the pictures for each place. I have attempted to do this within a forEach function, and within a For-In function. I have tried everything I can think of, and find research on, to make this work (from using setTimeout in various situations, to creating promises with timeouts included and incorporated tehm in many different ways), but I have been unable to find any solutions to assist in my particular async/await fetch situation.

To be clear - the application is operational, and my "else" statement is kicking in, but the else statement is kicking in because I am exceeding the per-second quota - so, the code is there, and working, I just want to be able to run the photos instead of the generic icons. I can get the photos to work if I wait long enough, as if the server forgets for a second. But my total daily quotas are well over anything I could ever reach in dev environment, so this has to be what is getting me in trouble!

If anyone can help, I would appreciate it greatly!

const renderVenues = (venues) => {
  for(let i=0; i < $venueDivs.length; i++){
    const $venue = $venueDivs[i];
    const venue = venues[i];
    let newUrl = `https://api.foursquare.com/v2/venues/${venue.id}/photos?client_id=${clientId}&client_secret=${clientSecret}&v=20190202`;
    const getPics = async () =>{
      try{
      const picResp = await fetch(newUrl);
      if(picResp.ok){
        const picJson = await picResp.json();
        const photo = picJson.response.photos.items[0];
        const venueImgSrc = `${photo.prefix}300x300${photo.suffix}`;
        let venueContent = `<h2>${venue.name}</h2><h4 style='padding-  top:15px'>${venue.categories[0].name}</h4>
        <img class="venueimage" src="${venueImgSrc}"/>
        <h3 style='padding-top:5px'>Address:</h3>
        <p>${venue.location.address}</p>
        <p>${venue.location.city}, ${venue.location.state}</p>
        <p>${venue.location.country}</p>`;
        $venue.append(venueContent);
      } else{
          const venueIcon = venue.categories[0].icon;
          const venueImgSrc = `${venueIcon.prefix}bg_64${venueIcon.suffix}`;
          let venueContent = `<h2>${venue.name}</h2><h4 style='padding-top:15px'>${venue.categories[0].name}</h4>
    <img class="venueimage" src="${venueImgSrc}"/>
    <h3 style='padding-top:5px'>Address:</h3>
    <p>${venue.location.address}</p>
    <p>${venue.location.city}, ${venue.location.state}</p>
    <p>${venue.location.country}</p>`;
      $venue.append(venueContent);
      }
    }
    catch(error){
        console.log(error)
        alert(error)
      }
    }
    getPics();
  }

  $destination.append(`<h2>${venues[0].location.city}, ${venues[0].location.state}</h2>`);
}

//and then below, I execute the promise(s) that this is included with.

getVenues().then(venues =>
    renderVenues(venues)
  )

CertainPerformance · Accepted Answer · 2019-02-06T04:27:10.363

7

On each iteration, you can await a Promise that resolves after 0.6 seconds:

const delay = ms => new Promise(resolve => setTimeout(resolve, ms));
const renderVenues = async (venues) => {
  for(let i=0; i < $venueDivs.length; i++){
    // ...
    await getPics();
    // no need for a trailing delay after all requests are complete:
    if (i !== $venueDivs.length - 1) {
      await delay(600);
    }
  }
  $destination.append(...)
};

edited Feb 06 '19 at 04:27

answered Feb 06 '19 at 04:19

CertainPerformance

356,069
52
309
320

This was perfect! Thank you! I had tried a very similar tactic on my own, but I see i did not `await` the `getPics()` call, and I did not async the `renderVenues(venues)` definition either. – E_C Feb 06 '19 at 22:11

score 2 · Answer 2 · answered Feb 06 '19 at 04:32

2

If you find yourself doing a bunch of throttling like this in your application, the module https://github.com/SGrondin/bottleneck provides a nice interface for expressing them.

answered Feb 06 '19 at 04:32

Elliot Nelson

11,371
3
30
44

this looks like it could be very useful! Thank you! – E_C Feb 06 '19 at 22:11

Is there a way to slow down async-await fetch calls inside a for-loop/forEach in order to avoid per-second API quotas?

2 Answers2

Linked