How to read and modify a csv file on one bucket in cloud storage and save the results in another bucket using Cloud Functions

Question

I have CSV files coming to a folder on a Cloud Storage bucket and I want to create a Cloud Function that opens the CSV, adds a new column to it and then save the results to a new bucket as new.csv file.

Is there a way to do that using python Cloud Function???

Thanks in advance.

I would say yes probably: https://cloud.google.com/functions/docs/writing/background#functions-writing-background-hello-storage-python — alex, Sep 11 '19 at 15:10
@alex Thnak you for the tips but is it possible for you to direct me to an example where I can understand how does it work — Be2, Sep 12 '19 at 05:27

score 1 · Accepted Answer · answered Sep 12 '19 at 08:04

The idea that you’re trying to implement is totally possible and can be achieved using Google Cloud Functions.

For that, you would need to create a storage-triggered Cloud Function. More specifically, you can create your function in a way that it will respond to change notifications emerging from your Google Cloud Storage.

These notifications can be configured to respond to various events inside a bucket: object creation, deletion, archiving and metadata updates.

For the situation described, you will need to use the trigger google.storage.object.finalize. This event is sent when a new object is created in the bucket or an existing object is overwritten, and a new generation of that object is created.

Here you can find a sample code of a storage-triggered Cloud Function written in Python, while this tutorial will give you a more detailed overview of the usage of storage-triggered functions.

How to read and modify a csv file on one bucket in cloud storage and save the results in another bucket using Cloud Functions

1 Answers1

Linked