How to use Python's CSV DictReader to parse a CSV output?

Question

I have integrated Salesforce's Bulk API for fetching records in my Python project. With 'Content-Type': 'text/csv; charset=UTF-8' header, it returns CSV Output.

"Id","Name","CreatedById","Salary","Base_Salary","Type","Pay_cycle","Description","Code"
"a0u90000003R4Y9AAK","Freelance Hadoop developer","005900000039GMdAAM","","","","","",""
"a0u90000003R5UPAA0","Senior Hadoop developer","005900000039GMdAAM","","","","","",""
"a0u90000003R5V3AAK","Freelance Webmaster","005900000039GMdAAM","","","","","",""
"a0u90000004ZMUwAAO","Full-Stack Web Developer - PHP and Python","005900000039GMdAAM","","","","","",""

Now, without storing this in a CSV file, I want to put these records this in my Db.

The DictReader is a really nice and straightforward solution for getting the field values, But I guess it only works with a (CSV) file.

import csv
with open('output.csv') as csvfile:
    reader = csv.DictReader(csvfile)
    for row in reader:
        print(row['Id'], row['Name'])

I am trying something but I don't like it. See

objects = []
for line in iter(csv_data.splitlines()):
    # Split by separator
    data = line.split(",")
    # Remove double quotes around the field value
    objects.append([i[1:-1] for i in data])

What will be the most elegant way of doing this?

hiro protagonist · Accepted Answer · 2015-11-03T11:45:50.453

5

you can wrap the csv string into a io.StringIO object. this will then work perfectly fine with the csv module:

data_str = '''"Id","Name","CreatedById","Salary","Base_Salary","Type","Pay_cycle","Description","Code"
"a0u90000003R4Y9AAK","Freelance Hadoop developer","005900000039GMdAAM","","","","","",""
"a0u90000003R5UPAA0","Senior Hadoop developer","005900000039GMdAAM","","","","","",""
"a0u90000003R5V3AAK","Freelance Webmaster","005900000039GMdAAM","","","","","",""
"a0u90000004ZMUwAAO","Full-Stack Web Developer - PHP and Python","005900000039GMdAAM","","","","","",""'''

# in python 2 you need to decode the string before passing it to StringIO
data_io = io.StringIO(data_str.decode('utf-8'))
## this is not needed in python 3
# data_io = io.StringIO(data_str)

reader = csv.DictReader(data_io)
for row in reader:
    print row['Id'], row['Name']

edited Nov 03 '15 at 11:45

answered Nov 03 '15 at 11:17

hiro protagonist

44,693
14
86
111

This is great. But I am getting an exception `TypeError: initial_value must be unicode or None, not str` in `data = io.StringIO(csv_output)` – Hussain Nov 03 '15 at 11:22
oh, you are on python 2.7, right? or what is you python version? – hiro protagonist Nov 03 '15 at 11:23
then you need to decode the string appending a `.decode('utf-8')` before you pass it to `io.StringIO`. – hiro protagonist Nov 03 '15 at 11:27
Yes. I see that here too http://stackoverflow.com/questions/22316333/how-can-i-resolve-typeerror-with-stringio-in-python-2-7 – Hussain Nov 03 '15 at 11:28
Now it works perfect! Thanks! Upvoted and accepted your answer. You should put `.decode('utf-8')` thing in your it. – Hussain Nov 03 '15 at 11:32
Recently I have started getting an error. `File "/usr/lib/python2.7/csv.py", line 104, in next row = self.reader.next() UnicodeEncodeError: 'ascii' codec can't encode character u'\xa0' in position 27: ordinal not in range(128) ` I guess this is happening because one of my column has HTML text value. – Hussain Nov 13 '15 at 07:22

How to use Python's CSV DictReader to parse a CSV output?

1 Answers1