if curl --head http://.... includes "301" or "404" then ... if curl --head http://.... has content-Length > 1000000 then ...
how to be put the least possible load on the destination server when doing many of such requests?
is it possible to only request the first line of the http answer or even only the 3 digits of the status code(/content length) but nothing else, to minimize traffic?
(actually, is there any alternative http field next to content-length indicating file size?)
thanks!