I am trying to stream data from a remote host on my server using python requests library but getting ChunkedEncodingError with ImcompleteRead in deficiency of requested bytes, I made a request with chunk of 4096bytes but server sent me partial at any step. I've go through the similar problems on stack overflow but all of them suggested to go for http1.0 instead 1.1, I also tried with urllib but no success so far.
I've done the code part under with context to make sure connection will close gracefully but all time I am getting exception at r = req.iter_content(chunk_size=4096) at any time. Had tried re-attempt in exception but exception was same.
What could be the resolution for the IncompleteRead Issue ?
UPDATE: Taking small chunk size (256 or 512) does not help much for large files or data , but small files are downloaded gracefully. Here is the snippet:
with requests.get("https://example.com/data", headers={ 'Content-Type': 'application/octet-stream'}, stream=True) as r:
with open("test10.dat", 'wb') as f:
for chunk in r.iter_content(chunk_size=256):
if chunk:
print("Chunk size to be write for another file", len(chunk))
f.write(chunk)
