Google Storage creating signed url for multiple objects at once

Question

I have been succesfully using signed urls for my image and video content in google storage. My Django Apis return 100 Google Storage objects and creating 100 signed urls takes literally a long time. Is there any other way to generate signed urls faster or multiple of them at once?

class FileUploadSignedURL(APIView):
@method_decorator(login_required(login_url='/login/'))
def post(self, request):
    filename = request.POST.get('filename')
    filetype = request.POST.get('type')
    filesize = request.POST.get('size', 0)

    uuid = get_random_string(11)
    path =  '{0}-{1}/{2}/'.format(request.user, request.user.id, uuid)
    logging.info(path);

    video = Video.objects.create(title=filename,
                                 uuid=uuid,
                                 path=path,
                                 user=request.user,
                                 type=filetype,
                                 size=filesize,
                                 status="signed")
    # create the blob - the blob has to be created in order to get a signed URL
    full_path = '{0}{1}'.format(video.path, video.name)
    blob = default_storage.open(full_path, 'wb')

    signed_url = blob.blob.generate_signed_url(expiration=default_storage.expiration, method='PUT', content_type=filetype)
    logging.debug(signed_url);

    logging.debug("FileUploadSignedURL(APIView) end")
    return Response({"uuid":video.uuid, "title": video.title, "signed_url":signed_url})

Could you please share your code? Keep in mind that it also depend on the throughput of the network. — Mario, Apr 10 '20 at 22:44
Hi @Mario, I have updated the question with the code. It still take a long time either the requests generated from development environment or from Google App Engine. What would be the strategy here ? Would "implementing a custom Signed Url generator" be a solution in this circumstance ? — london_utku, Apr 11 '20 at 21:20

score 1 · Accepted Answer · answered Apr 23 '20 at 15:36

1

My suggestion is to compare the performance with the base example:

https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/storage/cloud-client/storage_generate_signed_url_v4.py

Could you try to create a new service account for this purpose?

answered Apr 23 '20 at 15:36

Mario

406
2
6

I will try and comment here. – london_utku Apr 23 '20 at 17:06

Google Storage creating signed url for multiple objects at once

1 Answers1