Python - generate csv file in memory and then encode its data into base64?

Question

I need to generate csv file like data in memory and then encode it to base64, so I could save it. So basically I don't want to create file in hard disk for that. Now I solve this by creating csv file then encoding its data, saving it and then just simply removing csv file (because it is no longer needed). But is there a way to skip file creation, but save data the same way? I mean that data would be used to open csv file again using base64.

import base64
import csv
from StringIO import StringIO
import os

def test_binary(self):
    mylist = [['a', 'b'], ['c', 'd']]
    with open("test.csv", "wb") as f:
        writer = csv.writer(f)
        writer.writerows(mylist)   

    myfile = open('test.csv', 'r')
    stream = StringIO(myfile.read())

    encoded = base64.b64encode(stream.getvalue())

    self.test = encoded
    myfile.close()
    os.remove('test.csv')

Drop `'test.csv'` and use `StringIO` – jfs Oct 16 '15 at 09:08 — jfs, Oct 16 '15 at 09:08

jfs · Answer 1 · 2021-02-08T15:33:39.557

55

You could pass StringIO() to the csv writer directly:

>>> import base64
>>> import csv
>>> from io import StringIO
>>> mylist = [['a', 'b'], ['c', 'd']]
>>> f = StringIO()
>>> csv.writer(f).writerows(mylist)
>>> base64.b64encode(f.getvalue().encode())
b'YSxiDQpjLGQNCg=='

Python 2

>>> import base64
>>> import csv
>>> from StringIO import StringIO
>>> mylist = [['a', 'b'], ['c', 'd']]
>>> f = StringIO()
>>> csv.writer(f).writerows(mylist)
>>> base64.b64encode(f.getvalue())
'YSxiDQpjLGQNCg=='

edited Feb 08 '21 at 15:33

answered Oct 20 '15 at 10:26

jfs

399,953
195
994
1,670

9

```from io import StringIO``` – priyankvex Jun 24 '17 at 20:06
`from StringIO import StringIO` works fine for python2, I do not know if it's deprecated and whatnot – Drachenfels Oct 30 '17 at 12:56
@priyankvex - this is new syntax for python3 – wittyurchin Mar 19 '19 at 11:27
2

A TypeError occurs when running the code using python3. To fix it, the final line should be `base64.b64encode(f.getvalue().encode('utf-8'))` – yvesva Oct 02 '19 at 05:52

score -4 · Accepted Answer · answered Oct 16 '15 at 09:35

If I understood correctly, your final goal is to have a list turned into a string in .csv format, and have it base64 encoded. If that is the case, you don't need to create the .csv file at all.

import base64

mylist = [['a', 'b'], ['c', 'd']]

# Creating the csv formatted string
csv_line_length = len(max(mylist,key=len))
csv_string = ''
for row in mylist:
    temp_row = ['"' + col + '"' for col in row]
    while len(temp_row) < csv_line_length:
      temp_row.append([])
    csv_string += ','.join(temp_row) + '\n'
# Encoding the string to base64
encoded = base64.b64encode(csv_string.encode('utf-8'))

don't reinvent csv, use `csv` module instead e.g., your code fails if `col` has a quote in it. — jfs, Oct 20 '15 at 10:27

Python - generate csv file in memory and then encode its data into base64?

2 Answers2

Python 2