0

I have created a Transfer Job to import some of my website's static resources to Google storage. The job was supposed to import the data in a bucket named www.pretty-story.com.

It is importing from a tsv file located here.

For instance the first url is : https://www.pretty-story.com/wp-includes/js/jquery/jquery.min.js

so I would have expected the job to create the folder structure starting with wp-includes.

But instead the job created this folder structure www.pretty-story.com\wp-includes\js\jquery.

Therefore the complete path (including my bucket name) is : www.pretty-story.com\www.pretty-story.com\wp-includes\js\jquery.

How can I tell the data transfer job to use the bucket as first folder, instead of creating a subfolder with the same name ?

David
  • 9,288
  • 1
  • 20
  • 52
Sam
  • 13,934
  • 26
  • 108
  • 194
  • Have you checked this [document](https://cloud.google.com/storage-transfer/docs/create-transfers#create_a_transfer)? – Sathi Aiswarya Sep 19 '22 at 11:51
  • yes I have :) Anything in particular that I have missed ? I don't see documentation for solving my issue. – Sam Sep 19 '22 at 11:54
  • the link you given not working.are you trying this in console? if so please give screenshots and the steps or links you are following – Sathi Aiswarya Sep 19 '22 at 12:46
  • sorry, i have fixed the link. Again it is all working fine, except that the job is creating a subfolder `www.pretty-story.com` inside the bucket `www.pretty-story.com` . I would expect it would simply reuse the bucket and only create subfolders starting with `wp-includes`. I'm adding as screenshot in a minute. – Sam Sep 19 '22 at 12:54

1 Answers1

1

According to https://cloud.google.com/storage-transfer/docs/create-url-list:

When an object located at http(s)://[HOSTNAME]:[PORT]/[URL_PATH] is transferred to Cloud Storage, the name of the object in Cloud Storage is [HOSTNAME]/[URL_PATH].

You don't have an option to skip the [HOSTNAME]/ part of this, so what you are asking is not possible.

If the amount of data involved is reasonable, I recommend downloading it to a workstation and using gsutil to copy it into a bucket without the hostname prefix.

David
  • 9,288
  • 1
  • 20
  • 52
  • Thank you for the link. Indeed you are correct, I won't be able to achieve this. However since you seem to be a GC employee, may I ask if such an option could be added to the data transfer funtionnality ? Inded, Google Storage documentation mentions that the bucket name can be a domain name (which in this case must be a verified domain name). Therefore it would make sense that when importing files to Google Storage through a list of urls, the domain name would be considered as the bucket name, instead of creating a subfolder. – Sam Sep 20 '22 at 11:22
  • 1
    I have created a feature request to the transfer service team, but it's going to be fairly low priority. If you have an account manager, support contact or other business channel, let them know and they can help get it prioritized - reference issue 247805751. You can also create a public feature request at https://issuetracker.google.com/issues/new?component=815827&template=1395475. This can help explain the need, plus if other people "+1" the issue it can help get it prioritized. – David Sep 20 '22 at 17:03