PUT files to Google Cloud Storage (GCS) via Signed URLs

Question

I am trying to use "signed URLs" to access/upload files to Google Cloud Storage (GCS).

Followed the instructions at https://developers.google.com/storage/docs/accesscontrol#Signing-Strings

What I did:

Registered a "Web Application" at Google Cloud Console.
Generated new private key using the "Generate New Key" at the "Certificate" section of the registered web application screen.
Created my string to be signed and sign it using the sample code provided by "How to Sign" section of the page (just minor tweaks to hardcode values for the Java main program). Update: Created a gist for GcsSigner.java.
Ensure that the signed string is URL encoded.
HTTP headers specified: x-goog-api-version, x-goog-project-id, Content-Type.
URL Parameters specified: GoogleAccessId, Signature, Expires.
Use Postman (a Chrome extension) to simulate PUT request.

However, I am still getting this (Status: 403 Forbidden):

<?xml version='1.0' encoding='UTF-8'?>
<Error>
    <Code>SignatureDoesNotMatch</Code>
    <Message>The request signature we calculated does not match the signature you provided. Check your Google secret key and signing method.</Message>
    <StringToSign>PUT

text/plain
1384084959392
x-goog-api-version:2
x-goog-project-id:99999
/mybucket/myfile.txt</StringToSign>
</Error>

Even when I retry by signing the string based on values in "StringToSign", I still get the same error.

Read various related postings here but could not find any solution and most of it refers to x-goog-api-version:1 while i am using version 2.

What am i missing? Any help would be greatly appreciated.

Could you post the code you're using to perform the signature? — jterrace, Nov 10 '13 at 19:29
@jterrace I've created a gist for the modified GcsSigner.java - https://gist.github.com/kctang/2cfdf66c7f11391ae11b. — kctang, Nov 11 '13 at 00:09
@kctang can you please post an example complete signed URL that is constructed with given code? — Neelam Sharma, Nov 13 '13 at 05:58

score 10 · Accepted Answer · answered Nov 15 '13 at 01:58

10

Finally managed to PUT files to Google Cloud Storage using signed URLs. This was done by creating a simple Java program to simulate:

Server to sign & encode a string as signature.
Uploader as an unauthenticated user submitting the PUT request using only the signature provided by Server. Browser is simulated using Apache's HTTP Client library.

You can see the demo app here.

I do not really understand why it did not work when I submitted through Chrome's Postman extension.

answered Nov 15 '13 at 01:58

kctang

10,894
8
44
63

1

hi kctang. I just have seen your demo app. In that file i cant able to find out what is public static final String KEY_FILENAME = "my-privatekey.p12"; public static final String KEY_PASSWORD = "notasecret";. can u please tell me what are those fields in the file. Even i am also working on the google cloud storage to upload a file. – Santosh May 27 '14 at 10:01
1

@Santosh You can create these as [service account](https://developers.google.com/storage/docs/authentication#service_accounts) type of client ID from Google's Dev Console. – kctang May 27 '14 at 17:40
thats great. I am also searching for that solution. You saved my days. Can you please tell me how to upload the file using forms. Like providing all the parameters in the form. I want to upload a file from the form to the google cloud storage. I am unable to prepare the signature and policy. Can you please tell me the procedure. Waiting for your response. – Sreekanth May 28 '14 at 05:01
Expiration must be in seconds, not in milliseconds as in your program. – Andrei Volgin May 18 '16 at 18:24
@kctang, I think that did not work in Chrome's Postman because you maybe forgot to send the same Content Type in the HEADER. I use Postman, but need to send the same Content Type that used in Signed URL. – javaTry Nov 23 '17 at 01:13

score 5 · Answer 2 · answered Apr 29 '16 at 12:54

5

Though the doc says Content-Type is optional, it actually means you must set correct content-Type correspond to your http request header.

In this case, your must add content-type: text/plain in the signing string.

https://cloud.google.com/storage/docs/access-control/create-signed-urls-program

answered Apr 29 '16 at 12:54

gaga5lala

1,218
13
17

In my experience it works in most cases also without specifying it.... i.e. for simple text files at least I am sure - I am using it in production and it works; for more complex formats it probably isn't always able to infer it... so it remains a good practice to set it explicitly ;-) – Daniele Muscetta Mar 18 '21 at 08:14

score 2 · Answer 3 · edited Apr 02 '19 at 22:45

I was struggling with PUT and the signed URL (with GAE Cloud Endpoints) but here are the two things I needed to do:

Make sure you have the latest GAE gradle dependencies. The auto generated ones but android studio are not always the latest.
For PUT you need to have your string to sign look like this:

String url_signature = sign(verb + "\n" + contentMD5 + "\n" + contentType + "\n" + expiration + "\n" + "/" + BUCKET_NAME + "/" + objectName );

As explained here: https://cloud.google.com/storage/docs/access-control/signed-urls the format is:

StringToSign = HTTP_Verb + "\n" +
               Content_MD5 + "\n" +
               Content_Type + "\n" +
               Expiration + "\n" +
               Canonicalized_Extension_Headers +
               Canonicalized_Resource

Those return "\n" are important. If you have too many or not enough, you will get that error. If you are not passing a contentMD5, for example, just pass in an empty string so you get the right number of return "\n"

score 1 · Answer 4 · answered Nov 13 '13 at 06:20

On looking into your code, can you please try creating stringToSign by omitting following headers -

"x-goog-api-version:2\n" +
"x-goog-project-id:1234\n" +

As shown below -

 String stringToSign = "PUT\n" +
                "\n" +
                "text/plain\n" +
                "1384084959392\n" +
                "/test-bucket/bob.txt";

Now try generating Sign URL.

Thanks

score 1 · Answer 5 · edited Mar 18 '19 at 22:14

To run signed url from browser you have to set HTTP header . In https://developers.google.com/storage/docs/accesscontrol#Construct-the-String

Content_Type Optional. If you provide this value the client (browser) must provide this HTTP header set to the same value.There is a word must

So if you are providing Content_Type for sign string you must provide same Content_Type in browser http header.I was also running from the same problem when i set Content_Type in browser header i was able to download a document.

PUT files to Google Cloud Storage (GCS) via Signed URLs

5 Answers5

Linked