1

I had been using Google Cloud CDN for caching content from a GCS bucket for months. Until yesterday (2019/09/19), I noticed that I cannot access an object with white spaces in its name. Usually, I apply encodeURIComponent to the object's name before signing the whole URL, which had been working fine until yesterday.

Here's what I've tried so far with gcloud utility:

  1. Sign the URL without URL-encoding the file's name:

    $ gcloud compute sign-url --key-name my-key --key-file my-key --expires-in 15m "https://cdn.example.com/file-with-white space.txt"

    I then accessed the URL with and without %20. The result is 403, shown in the picture.

  2. Sign the URL with the URL-encoded file's name (this is what I'd been doing for months, and it worked fine):

    $ gcloud compute sign-url --key-name my-key --key-file my-key --expires-in 15m "https://cdn.example.com/file-with-white%20space.txt"

    The result is also 403 but with different message:

Anonymous caller does not have storage.objects.get access to bucket/file name

I have also tried using the Go code from this link. The results are the same.

Please note that files without white spaces in its name can still be accessed successfully through the CDN.

Update

  1. To clarify, I think the CDN's behavior has changed.
  2. I have granted the CDN access to the GCS bucket. That's why the CDN worked without problems earlier. I actually have just run gsutil iam ch serviceAccount:service-PROJECT_NUM@cloud-cdn-fill.iam.gserviceaccount.com:objectViewer gs://[BUCKET] twice to ensure this.
  3. I have tried signing GCS URLs using gsutil directly without using the CDN, and the signed URL worked.

Update 2

I have tried out the --validate option. This is what I got:

$ gcloud compute sign-url --key-name cdn-signing-key \
  --key-file cdn-signing-key --expires-in 15m \
  --validate "https://cdn.domain.com/file%20with%20space"

signedUrl: https://cdn.domain.com/file%20with%20space?Expires=1569075302&KeyName=cdn-signing-key&Signature=e3SANudKHIT5txHWVlO1oijItXw=
validationResponseCode: 200

And yet, I still received 403 when accessing the "signedUrl" through a browser. The result is an XML page with <Code>AccessDenied</Code>.

Kohpai Bamboo
  • 11
  • 4
  • > which had been working fine until yesterday. To be clear, are you saying this behaviour changed? – elithrar Sep 20 '19 at 17:10
  • Further - has Cloud CDN been explicitly granted access to the backend bucket per https://cloud.google.com/cdn/docs/using-signed-urls#configuring_google_cloud_storage_permissions - ? – elithrar Sep 20 '19 at 17:11
  • @elithrar Yes, I think the CDN's behavior to interpret a URL has changed. And I have followed the instruction on the link already. That's why I was able to use the CDN earlier. I also updated the question so other people can understand the context clearer. – Kohpai Bamboo Sep 20 '19 at 18:07

2 Answers2

0

I'm not able to replicate this.

A (URL-encoded) filename with spaces in a GCS bucket can be signed and validated:

➜  gcloud compute sign-url --key-name "backend-key" \
  --key-file backend.key --expires-in 7d \
  --validate "https://cloud-cdn.questionable.services/file%20with%20spaces.txt"

signedUrl: https://cloud-cdn.questionable.services/file%20with%20spaces.txt?Expires=1569639576&KeyName=backend-key&Signature=pTsgDpBOpBcqHDNeTWFfFcTC2Ws=
validationResponseCode: 200

Without URL-encoding, the validation and signing fail (as expected), as gcloud doesn't automatically percent encode URLs:

➜  gcloud compute sign-url --key-name "backend-key" \
  --key-file backend.key --expires-in 7d \
  --validate "https://cloud-cdn.questionable.services/file with spaces.txt"

signedUrl: https://cloud-cdn.questionable.services/file with spaces.txt?Expires=1569639586&KeyName=backend-key&Signature=AiJEBO6sHGgJh8EshLAH2IXlxe0=
validationResponseCode: 400

There haven't been any changes to the signing algorithm on the Cloud CDN side, nor the sign-url command in the gcloud SDK. We haven't implicitly URL encoded the input URL in the past.

elithrar
  • 23,364
  • 10
  • 85
  • 104
  • I just tried out the `--validate` option. This is what I got back: `$ gcloud compute sign-url --key-name cdn-signing-key \ --key-file cdn-signing-key --expires-in 15m \ --validate "https://cdn.domain.com/file%20with%20space" signedUrl: https://cdn.domain.com/file%20with%20space?Expires=1569075302&KeyName=cdn-signing-key&Signature=e3SANudKHIT5txHWVlO1oijItXw= validationResponseCode: 200` But when accessing the signed URL through a browser, I still got the XML page with `AccessDenied`. Any idea? – Kohpai Bamboo Sep 21 '19 at 14:15
  • Without being able to test the URL, I can't validate your exact scenario. Take a look at the access logs in Stackdriver Logging and see what error is bubbled up that prevents access. I presume you don't have anything like VPC Service Controls or similar configured... – elithrar Sep 22 '19 at 17:29
  • I checked the logs and saw that when the CDN tries to get the file from the bucket, the bucket returns 301. Then CDN redirects me to the bucket URL, which in turn, returns 403. – Kohpai Bamboo Sep 23 '19 at 08:28
0

This morning I tried accessing a newly signed URL of a file with spaces again. I noticed something interesting.

  • Sometimes, the CDN works and returns 200 along with the content of the file.
  • Some other times, the CDN gets 301 as a response from the backend service (in this case, a GCS bucket), and then redirects me to the bucket URL which in turn returns 403 along with the XML page.
  • So I tried changing the permission of the bucket to Public. If the CDN works, I will receive 200 as before. If the CDN redirects me to the bucket, now I will receive 200 along with the file.

So I'm closing this question since I think it has nothing to do with the file name anymore. It seems like the CDN doesn't cache the content for me and tries to direct me the original file in the bucket instead. I'm not sure if this is a normal behavior of CDNs, but I'll do some more research on that.

Kohpai Bamboo
  • 11
  • 4