how to avoid (or handle) http 429 programmatically

Question

I have a page with a lot of <img> tags (I am assembling this page programmatically). Inevitably some of the requests for image encounter http 429. Thankfully, the remote server sends a bunch of helpful response headers as follows

HTTP/1.1 429 TOO MANY REQUESTS
X-RateLimit-Limit: 20
X-RateLimit-Remaining: 20
X-RateLimit-Reset: 1646307858.32
Retry-After: 1

I am wondering if I could somehow wrap my <img> tags in code that catches the http response.status and then uses the above headers (for example, the Retry-After header) to ask for the failed image again after a set delay.

Logically, it seems to me the above would work, but I can't figure out how to code this. Fwiw, this is a completely client-side web page.

Thanks in advance for any guidance.

You *might* be able to detect the type of failure if the images (a) come from the same origin or (b) the origin of the images also sends CORS headers. I'd start by implementing something like [this](https://developer.mozilla.org/en-US/docs/Web/API/GlobalEventHandlers/onerror#element.onerror) and inspecting the event that is sent to see if there is anything in it that can discern the kind of error that happened. — spender, Mar 03 '22 at 11:55
thanks, this looks promising but it didn't really work for me (using ``). My guess is that perhaps http 429 doesn't seem like an error. It is, after all, a valid response, just not an image — punkish, Mar 03 '22 at 12:06
I may be wrong… tested it again and it seems to work. I will test more and come back and report — punkish, Mar 03 '22 at 13:15
do you know a solution for python? My problem is:[1]: https://stackoverflow.com/questions/75110778/http-errors-on-chrome-while-instagram-scrapping?noredirect=1#comment132547047_75110778 — freeengineer, Jan 13 '23 at 15:09
do you know a solution for python my problem is: https://stackoverflow.com/questions/75110778/http-errors-on-chrome-while-instagram-scrapping?noredirect=1#comment132547047_75110778 — freeengineer, Jan 13 '23 at 15:10
@freeengineer sorry, I am not a Python programmer. But the logic is simple to implement. Query for the image, and if there is a 429 error, wait for a second or two (1000 or 2000 ms) and query again — punkish, Jan 14 '23 at 14:36

mplungjan · Answer 1 · 2022-03-03T12:55:50.420

try something like this

window.addEventListener("DOMContentLoaded", function() {
  const images = document.querySelectorAll("img[data-src]");
  let cnt = 0;

  const loadImage = src => {
    let img = new Image();
    let tgtImage = images[cnt];
    img.addEventListener("load", function() {
      tgtImage.src = this.src
    })
    img.addEventListener("error", function() {
      setTimeout(function() {
        loadImage(`${src}`)
      }, 1000)
    })
    img.src = src;
    cnt++
    if (cnt >= images.length) return;
    loadImage(images[cnt].dataset.src)
  }
  loadImage(images[cnt].dataset.src)
})

<img src="https://dummyimage.com/200x300&text=placeholder" data-src="https://dummyimage.com/200x300&text=image1" />
<img src="https://dummyimage.com/200x300&text=placeholder" data-src="https://dummyimage.com/200x300&text=image2" />
<img src="https://dummyimage.com/200x300&text=placeholder" data-src="https://ERROR.com/200x300&text=image3" />
<img src="https://dummyimage.com/200x300&text=placeholder" data-src="https://dummyimage.com/200x300&text=image4" />

thanks. I am not quite sure where to use your suggested workaround. As I mentioned, my page is really a bunch of image tags ``. As I noted in a comment above, I tried using `setTimeout()` in `onerror` but that didn't work. I am wondering if using a Promise and then catching the `response.status` is the solution — punkish, Mar 03 '22 at 12:09

score 0 · Answer 2 · answered Mar 03 '22 at 20:14

the following worked for me

<img 
   src="${uri}" 
   onerror="this.onerror=null; setTimeout(() => { this.src='${uri}' },1000);">

thanks @spender for the hint. Thanks also to mplungjan for an alternative solution that is useful to know even if it is not for me at this point.

how to avoid (or handle) http 429 programmatically

2 Answers2