2

I am using the MediaIoBaseDownload to implement download from GCS.

But I found that the download response is always about 5 seconds between each response.

if I download two files at the same time, the gap between each response will around 10 seconds.

upload speed is fine, it only occurred while downloading.

Is there any limitation about the download API cause I could not found the limitation.

after add some log information I found that the most time spent at the response.read() in httplib2

Could I think this is the limitation that GCS server holds or is there any setting of buckets(e.g. like DRA) that will affect the download speed?

I am using python of 2.7.8.

thanks!

Mike Chiu
  • 45
  • 7
  • 1
    Can you try running `gsutil perfdiag`? – jterrace Dec 24 '14 at 17:35
  • OK, I'll try gsutil for more detail. thanks! – Mike Chiu Dec 25 '14 at 01:44
  • I ran the gsutil and found that gsutil download use the storage_apitools wrapped in the gsutil package but not use the httplib2 to do the download. Also, the resumable download feature, gsutil handle it by itself. I think that the key point that gsutil downloads file faster than normal lib and apply resumable parameter. – Mike Chiu Dec 27 '14 at 03:01
  • 1
    finally I found why. Because I hook ssl_recv and do rate limit control, and I found that there are many responses return in a request with length smaller than 1k. I solve this by return the response directly if no rate limit control and I am curious is there any way to set the response buf_size of the google server ? – Mike Chiu Jan 20 '15 at 04:16

0 Answers0