8

I'm uploading a dummy file to Google Cloud Storage, and then signing a read/write URL before passing it to another service hosted separately. Unfortunately, I'm receiving a 403 response, with the message:

The request signature we calculated does not match the signature you provided. Check your Google secret key and signing method.

The code I'm using to create the dummy object and sign the URL:

const string BASE64_JSON_CREDS = "UklQIFN0YW4gTGVl"; // credentials of service account with "Storage Admin" role (entire json file as received from Google's Console)
const string BUCKET = "testbucket";
const string FILENAME = "test.jpg";
byte[] imageBytes = File.ReadAllBytes(@"test.jpg");

GoogleCredential credentials = null;
using (var stream = new MemoryStream(Convert.FromBase64String(BASE64_JSON_CREDS)))
{
    credentials = GoogleCredential.FromStream(stream);
}

StorageClient storageClient = StorageClient.Create(credentials);
var bucket = await storageClient.GetBucketAsync(BUCKET);
await storageClient.UploadObjectAsync(bucket.Name, FILENAME, null, new MemoryStream());

var scopedCreds = credentials.CreateScoped("https://www.googleapis.com/auth/devstorage.read_write").UnderlyingCredential as ServiceAccountCredential;
var urlSigner = UrlSigner.FromServiceAccountCredential(scopedCreds);
var url = urlSigner.Sign(bucket.Name, FILENAME, TimeSpan.FromHours(100));

Some test code I've written for the sake of this question (I've also tried HttpWebRequest):

var handler = new HttpClientHandler()
{
//    Proxy = new WebProxy("localhost", 8888)
};
var client = new HttpClient(handler);
var content = new ByteArrayContent(imageBytes, 0, imageBytes.Length);
content.Headers.Remove("Content-Type");
var response = await client.PutAsync(url, content);
if (response.IsSuccessStatusCode)
{
    Console.WriteLine("yay");
}
else
{
    Console.WriteLine(await response.Content.ReadAsStringAsync());
}
Console.ReadKey();

When proxied through Fiddler, the request looks like this:

PUT https://storage.googleapis.com/testbucket/test.jpg?GoogleAccessId=testbucket@testproject.iam.gserviceaccount.com&Expires=1543209340&Signature=j1cagZ9MHZQAIeYrzbm95MWsIdFMvX1Em13il%2F2nEB1qx9xGB6%2BUzt6vo2OVuRp2TlW1G1TtyX32lxbH%2Fb51dr49eFBcSSm9H8rSXtuEXci02dY%2Fe%2FV0n4kpVwDjpiq4QVSMM%2BaCEdrUtPxT69BSoDuRqh6UHkeOL6VqLgcHGKQcXraZCrEaCXCJfNBwBlPcoXzOD708Nasl99ahxGwcPY6s1FXLCiAiP0VDJSRrPqbE8LHyRLLTgCk9r2H4pEW%2BpGpjEWj3DVpDC334%2BQQFttzDNuZQnUMtZi%2BGz5rqQbU5hBLgthb%2B13884uL4eUalnoSuRfR9JPKIJP7xk3%2FH4g%3D%3D HTTP/1.1
Content-Length: 21925
Host: storage.googleapis.com

{IMAGE_CONTENT}

And the response is:

HTTP/1.1 403 Forbidden
X-GUploader-UploadID: AEnB2UoklkZmIP8odWSx14Y0ZDgxjM8ZM94SCfNgAONG1giFTd9cncH8bAMK3s7I7v2DC1NwVirOrNbTjnBzdS2o1tOGX2pLBg
Content-Type: application/xml; charset=UTF-8
Content-Length: 314
Date: Thu, 22 Nov 2018 01:15:39 GMT
Server: UploadServer
Alt-Svc: quic=":443"; ma=2592000; v="44,43,39,35"

<?xml version='1.0' encoding='UTF-8'?><Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your Google secret key and signing method.</Message><StringToSign>PUT


1543209340
/testbucket/test.jpg</StringToSign></Error>

Looking at Google's UrlSigner code, the lines listed above are:

var signatureLines = new List<string>
{
    requestMethod.ToString(),
    contentMD5,
    contentType,
    expiryUnixSeconds
};

I found suggestion in this question that the Content-Type header does need to be set, so I made the following changes:

// New signing code
var headers = new Dictionary<string, IEnumerable<string>>() { { "Content-Type", new string[] { "image/jpeg" } } };
var url = urlSigner.Sign(bucket.Name, FILENAME, TimeSpan.FromHours(100), requestHeaders: headers);

// New put code (I removed the line removing Content-Type)
var content = new ByteArrayContent(imageBytes, 0, imageBytes.Length);
content.Headers.ContentType = new System.Net.Http.Headers.MediaTypeHeaderValue("image/jpeg");

but this hasn't fixed things. The new "StringToSign" value reflects the change:

PUT

image/jpeg
1543214247
/teams-storage-test-bucket/test.jpg

So the headers it should (to my mind) be checking for are correct vs what's being sent. The generated URL works for GET (I can download the empty file), but not for the PUT. Is there a solution?

ProgrammingLlama
  • 36,677
  • 7
  • 67
  • 86
  • For your line that is loading the credentials are you loading the entire json file base64 encoded or just part of it: `const string BASE64_JSON_CREDS=`. To create credentials correctly you need both the `client_email` and `private_key` which are included in the credentials json file. – John Hanley Nov 22 '18 at 04:01
  • @John The entire file as received from Google. The same credentials are allowing me to upload files, download files, and delete files via the `ServiceClient` and, also allowing me create resumable uploads (via REST API because I need an Angular web front-end to perform the upload). – ProgrammingLlama Nov 22 '18 at 04:08
  • You need to specify the HTTP method and headers in your calll to`urlSigner.Sign(bucket.Name, FILENAME, TimeSpan.FromHours(100), HttpMethod.Put, contentHeaders: new Dictionary> { { "Content-Type", new[] { "image/jpeg" } } });` – John Hanley Nov 22 '18 at 04:22
  • @John Can you add that as an answer so that I can accept my humiliation and give you the upvotes you deserve? :-) I naively assumed (not looking at the overloads carefully enough) that specifying the scope was the equivalent of specifying the verb for S3, etc. – ProgrammingLlama Nov 22 '18 at 04:25

2 Answers2

11

In your call to Sign() include the HTTP Method and Content-Type headers:

urlSigner.Sign(bucket.Name, FILENAME, TimeSpan.FromHours(100), HttpMethod.Put, contentHeaders: new Dictionary<string, IEnumerable<string>> { { "Content-Type", new[] { "image/jpeg" } } });
John Hanley
  • 74,467
  • 6
  • 95
  • 159
4

It worked for me by adding the same content type in both the signedURL and the PUT request during uploading

file.getSignedUrl({
    action:"write",
    expires:(Date.now() + expDuration),
    contentType: "text/csv",  <-- add your type 
});

and in the upload request, add the header

"Content-Type":"text/csv"

Abraham
  • 12,140
  • 4
  • 56
  • 92