-1

I have a CSV file on Google Cloud Storage that I want to read data from without downloading it. Is this possible? I'm extensively checking the docs but I haven't found a direct way to do this. The docs say you have to download the file and then do the csv reading, but this is something not possible on my case because the app that will do this is hosted on Google App Engine and it won't allow you to do so.

This is my current function: 

def get_data_from_csv_file(filename):
    app_id = get_app_id()
    g_storage = storage.Client(project=app_id)
    bucket = g_storage.get_bucket('prod-data-migration-csv-exports')

After this though, I'm lost. Has anyone achieved this before? Is there a way to get the file and read it's content in a manner you can store all the .csv content in a variable?

Dan Cornilescu
  • 39,470
  • 12
  • 57
  • 97
CodeTrooper
  • 1,890
  • 6
  • 32
  • 54

1 Answers1

1

The general idea is to use the GCS client library's .open() function instead of the regular filesystem open().

Maybe you can expand (by using a csv.reader instead of the csv.writer) from Write a CSV to store in Google Cloud Storage.

Maybe something along these lines? (not tested):

gcs_file = gcs.open(os.path.join(bucket, filename), 'r')
csv_reader = csv.reader(gcs_file)  # add whatever options you need

# CSV processing 

gcs_file.close()
Dan Cornilescu
  • 39,470
  • 12
  • 57
  • 97