0

I want to implement a node script which should download some images from remote sites, say once every hour, with a local cache, on filesystem.
I would like to know if it's possible to add something like an "if-modified-since" tag in the node fetch request headers, to let the remote server to return a 304 status, instead of a 200 with the full image body, if the image is not changed.

The code I use is something like:

const response = await fetch(imageUrl, {
  headers: {'If-Modified-Since': new Date().toUTCString()}
});

I.e.: I use the current date, but no 304 from the remote server...

UPDATE: I did finally realise my mistake: I was removing the double quotes from the etag... The starting and trailing quotes in etag must be included in If-None-Match value... With double quotes, server correctly returns 304 if image is not changed.

Thanks to @bergui for your help...

MarcoS
  • 17,323
  • 24
  • 96
  • 174
  • It's just a header. Sure you can send those. – Bergi Dec 28 '20 at 11:17
  • But I can't find any example on the web, so I'm dubious... – MarcoS Dec 28 '20 at 11:18
  • You mean for example [this](https://stackoverflow.com/q/64911396/1048572)? – Bergi Dec 28 '20 at 11:21
  • Exactly, yes, thanks. But, if you see the answer, it looks like impossible to do... :-( – MarcoS Dec 28 '20 at 13:01
  • The request is sent just fine, the answer complains about the response status 304 being mangled into 200. (It should be possible to detect that however if there is no body… I'm not sure how `node-fetch` deals with this, you should try it out or study the code) – Bergi Dec 28 '20 at 13:05
  • node-fetch always returns body and 200, even if image is not changed since a long time. So I suppose the answer to my question is: "No, it is not possible to let remote server return 403 on cached files"... – MarcoS Dec 28 '20 at 14:29
  • Are you talking about 403 or 304? And no, node-fetch does not handle caching at all (see their documentation or [this issue](https://github.com/node-fetch/node-fetch/issues/68)), so it does not change the status code either, you just [get status 304 and an empty body](https://github.com/node-fetch/node-fetch/blob/cbd4d895767e33491219e40013dd8daa5c7ac024/test/main.js#L643-L654). – Bergi Dec 28 '20 at 14:53
  • Sorry, I meant 304, not 403... :-) Getting status 304 and an empty body is exactly what I want, to handle a custom cache myself... But I can't obtain such a response from the remote server, not even using an "if-modified-since" tag set to current time, in the request headers... – MarcoS Dec 28 '20 at 15:14
  • Can you post the code you tried to send the headers? And are you certain that the remote server supports this header and will send 304 responses at all? – Bergi Dec 28 '20 at 15:17
  • I am not sure at all... I do not control remote server, and I need to fetch from many of them... I add the code I use in the question... – MarcoS Dec 28 '20 at 16:39
  • Then try to test with one where you have confirmed it supports the header and will sometimes send 304. – Bergi Dec 28 '20 at 17:01
  • I did try 4 more different servers. None of them never returns a 304 (Not Modified). I suspect it's non an issue with the remote servers, but something related to node-fetch architecture... – MarcoS Jan 02 '21 at 15:59
  • Did you try with some other tool to ensure that the server you test with supports the header and will send 304? Did you use a network sniffer or proxy to debug what requests were actually sent on the tcp layer? – Bergi Jan 02 '21 at 16:15
  • 2
    @Bergui: with postman I get 304! I did realize I did remove the double quotes from the etag... The starting and trailing quotes in etag must be included in `If-None-Match` value... That way with node.js too I get 304... Thanks for your help! – MarcoS Jan 02 '21 at 17:23

0 Answers0