1

A business partner wants to drop files for us via SFTP, and we want those files on a GCS bucket. As far as I can tell, GCS doesn't support SFTP access, but there's a pre-GA connector, described here.

If I understand the doc correctly, this connector enables reading initiated by us, but the partner wants to initiate the drop to us. Has anyone worked with this? Is this possible? (We are not in a position to ask the business partner to use any of the SFTP-GCS tools in GitHub).

Thank you

Yanay Lehavi
  • 166
  • 11
  • This is not a [programming question](https://stackoverflow.com/help/on-topic). – Martin Prikryl Apr 20 '23 at 06:15
  • Have a look at this [blog](https://thorntech.com/transfer-files-to-google-cloud-sftp-gateway/) SFTP Gateway to easily transfer files to Google Cloud Storage – Sathi Aiswarya Apr 20 '23 at 07:13
  • Your post is asking for recommendations but you do not include enough details to really get a solid recommendation. For example, what is your budget, how many users, transfer speed requirements, storage amount transferred per unit of time, fault tolerance, etc. Google does not have a native solution. For a low-cost solution, deploy a Ubuntu or Red Hat VM as a file server and sync/rsync uploaded files to Cloud Storage. – John Hanley Apr 20 '23 at 17:02
  • Thanks John. Trying to stay metal-free, so will try that pre GA connector cited above. – Yanay Lehavi Apr 25 '23 at 23:07
  • A connector does not provide SFTP services, it provides connections to an SFTP server. – John Hanley Apr 25 '23 at 23:32
  • For completion and closure, this connector is unidirectional (from GCS out) and I need support for incoming traffic. Dead end. Set myself up on S3, which the above mentioned partner does support, and looking forward to moving files from S3 to GCS. Fun fun! Thanks to all who chimed in and here's to better days! – Yanay Lehavi May 05 '23 at 23:05

3 Answers3

1

At the present moment, I am not aware about any GA GCP "native" services, which can be used directly from Google. From my point of view, there might be a few workarounds to consider.

1/ External third party service, which can be used as a storage, so you can fetch from it, or can be configured to forward to your GCP storage bucket.

2/ Market Place solutions. I think there quite a few of them there. WIth different functionality and cost.

3/ Deploy and maintain a proper COTS SFTP server (might have storage buckets as a backend).

4/ Convince your partner to use their own SFTP server, and fetch data files from it. In that case, there might be plenty of options how to do that - from self developed serverless solutions (my SO example is described here: serverless solution for external SFTP access), to GCP "native" solutions (you mentioned in your question), and to third party solutions.

5/ some variation/combination of the above.

I would guess that your decision might be based on the many non functional requirements, and heavily/ultimately influenced by direct and indirect costs (COTS software, GCP expenses, third party partners, people - might be the main driever, etc.).

al-dann
  • 2,545
  • 1
  • 12
  • 22
0

You can also try SFTPCloud. It's a managed solution that allows you to access Google Cloud Storage via SFTP. You can manage from a web dashboard the SFTP users, set bandwidth & storage quotas, key-based authentication, create multiple SFTP servers (each one can have a storage from a different provider), view the active connections in real-time, etc. Maybe it will help other users searching for a simple-to-use solution to access GCS via SFTP.

Bruno
  • 1
-1

One way to do it is to use a third party SFTP server to act as a bridge between your SFTP client and your GCP cloud storage buckets.

In the spirit of full disclosure, I'm one of the developers of the solution cited in the knowledge base article linked here above, but our SFTP server is not the only one that can do this. You should test several solutions and choose the one that best suits your needs.

FjodrSo
  • 301
  • 2
  • 8