How can I download the large file from google drive using rclone?

Question

Now I want to download the dataset on the website(http://pubchemqc.riken.jp/), it's in google drive and filesize is 2TB. the website recommend using rclone to download the file but not said how to use. As the picture shows

Answer 1

This was asked a while back already but I'd like to keep it recorded here as it seems hard to find. Unfortunately I don't know if there's a way to automate this process but it's still easy.

What you can do is create a new remote with the link's root_folder_id. The Root Folder ID is present in the url e.g: in the example link https://drive.google.com/drive/folders/a1b2c3deFgHi4JKlm56nOpqrStuv7w8xy9

the root_folder_id is the string after the slash /folders/ so in this case it would be "a1b2c3deFgHi4JKlm56nOpqrStuv7w8xy9". If the Google Drive shared link you got is a folder all you have to do is copy that id and use it in the rclone's new remote setup. Now if the shared link points directly to a file, like the one in the OP, there's no folder to download it from, so we gotta create our own! The whole process would look like:

• get the folder_id from Google Drive's link. If it is a link to a direct file then first we have to create a new folder anywhere inside our own Google Drive, it's name doesn't matter as we will point directly to it using the ID. After creating this new folder, open it and note the url, it should look something like https://drive.google.com/drive/u/1/folders/Fdrcv3nQvxQqXUGEEyvacwUxdYXpV33Ct

just copy everything after /folders/ and save it for later.

• now go back to the link of the direct file. Add a shortcut for that file to inside your recently created folder, so say the folder was named dl-with-rclone, click the "Add shortcut to Drive" icon and navigate to the "dl-with-rclone" folder to add the shortcut there.

heading to rclone, do:

• rclone config <to open settings>
• On the prompt e/n/d/r/c/s/q> hit n (the key relative to creating a new remote)
• On name> give it a name like sharedWithMe or anything else for personal reference later
• On the prompt Storage>, the possible remotes to connect to, write down drive or the number relative to it (currently 15)
• if you set up your own client_id and client_secret you put them next
• On the scope> prompt option 2 "Read-only" is enough

now this is the important one:

• on the root_folder_id> prompt input the ID of the folder of the shared link or the folder you created and pointed the file shortcut to, e.g. Fdrcv3nQvxQqXUGEEyvacwUxdYXpV33Ct

Now you can pretty much hit enter for everything, once you reach Use auto config? you will be redirected for a browser to login. Make sure the logged in account is the same that you created the folder for the shortcut. If the shared link is for a folder already your logged in account doesn't matter.

After finishing your remote setup you can exit rclone config

The command you would need would be something like rclone copy sharedWithMe: destination/folder

This being rclone you would surely be able to copy from one remote to another, that's up to the user.

In summary: I did test this method on your link, creating a folder in my Drive, pointing the file shortcut to it and using the root_folder_id to setup the rclone remote and it did begin to download:

rclone test download screenshot