how to upload and read csv file in django using csv.DictReader?

Question

I am reading csv file through upload and trying to store all values in a list

def upload(request):
    paramFile = request.FILES['file'].read()
    data = csv.DictReader(paramFile)
    list1 = []
    for row in data:
        list1.append(row)

    print list1

file.csv

12345,abcdef

output

[{'1': '', None: ['']}, {'1': '2'}]

I want to append all values in list1

I see nothing having to do with dictionaries, so why are you using `DictReader`? — Cory Kramer, Jul 11 '14 at 19:05
@shashisp are you sure you don't want a list of lists: `[['12345', 'abcdef'], ...]`? One for each line in the CSV? — Jamie Cockburn, Jul 11 '14 at 19:16

score 47 · Answer 1 · answered Sep 16 '17 at 08:18

This should work if you're using Python 3.

file = request.FILES['file'] 
decoded_file = file.read().decode('utf-8').splitlines()
reader = csv.DictReader(decoded_file)
for row in reader:
    # Get each cell value based on key-value pair. 
    # Key will always be what lies on the first row.

We can use the list that splitlines() creates. splitlines() is called because csv.DictReader expects "any object which supports the iterator protocol and returns a string each time its next() method is called — file objects and list objects are both suitable".

nice note on the decoded_file variable. This had me stumped all afternoon and that line helped, thanks! — ViaTech, Oct 19 '18 at 21:21

score 17 · Answer 2 · answered Jan 07 '18 at 16:14

17

in Python 3, to get the correct type (String not bytes) without reading the full file into memory you can use a generator to decode line by line:

def decode_utf8(input_iterator):
    for l in input_iterator:
        yield l.decode('utf-8')

def upload(request):
    reader = csv.DictReader(decode_utf8(request.FILES['file']))
    for row in reader:
        print(row)

answered Jan 07 '18 at 16:14

mattwarren

791
8
8

3

Or using generator expression: `reader = csv.DictReader(chunk.decode() for chunk in request.FILES['file'])` – piuspbd Apr 19 '19 at 09:05
What if the chunking algorithm splits apart a multi-byte UTF-8 character? This would cause a UnicodeDecodeError. – George Kettleborough Aug 07 '20 at 10:55

Jamie Cockburn · Accepted Answer · 2014-07-11T19:12:45.757

You have two problems:

You are passing a string to the constructor of DictReader. You must pass an iterable object that gives the individual lines in the input (a string is iterable, but will give each character one at a time). Luckily, an UploadedFile object (like those in the FILES dictionary) are already file-like objects that support iteration, so just do this:
```
data = csv.DictReader(request.FILES['file'])
```
Your input data only has one line. DictReader will use that line for the column "headers", which will become the key in the resulting dictionaries. You will then have no data left! It looks like you don't want a DictReader, just a regualar reader:
```
data = csv.reader(request.FILES['file'])
```

how to upload and read csv file in django using csv.DictReader?

3 Answers3

Linked

Related