django - how to parse an uploaded file and send the result to a different form

Question

I'm new to the django and in my first app,
I would like to upload a csv file, with unknown number of rows, parse it's content and display the parsed data, than manipulate this data and save it in a table.

What is the best approach for achieving this goal ?

Currently I have a view and a form for uploading the file,
my first attempt was to parse the uploaded file in that view and send the parsed data to a second view :

def UploadFileView(request):
    if request.method == 'POST':
        form = UploadFileForm(request.POST, request.FILES)
        if form.is_valid():
            upload_list = handle_uploaded_file(request.FILES['file'])
            return HttpResponseRedirect(reverse('preview_file', ?????))
    else:
        form = UploadFileForm()

    return render(request, 'upload_file.html', {'form': form,})

upload_list - a list of objects which describe a row in the file.
????? - what to put here in order to send the list to a second view ?

I understand that I need to serialize this list somehow in order to be able to send it to the second view, i.e preview_file.

1. How do I serialize it ? make it a JSON string ?
2. How do I send this serialized data to a second view ?

The csv file has unknown number of lines, it can be 3, 10 or 500.

While posting this question, I thought about another option and would like to know if it's possible and if it's a better option :
Not to parse the file in the first view, but to send it to the second view and call handle_uploaded_file in that view.

Answer 1

Save the list to the session:

import random
from django.shortcuts import redirect

key = 'preview%s' % random.randint(0, 999999)
request.session[key] = upload_list
return redirect('preview_file', key)

And the in the prefiew_file():

from django.http import Http404

def preview_file(request, key):
    upload_list = request.session.get(key)
    if not upload_list:
        raise Http404

UPDATE: Session size limitation depend on the used SESSION_ENGINE. The default django.contrib.sessions.backends.db engine stores session data in the database TEXT field. In case of Postgres/SQLite such fields can contain up to 1Gb of data. For MySQL the limit is 64Kb.

Answer 2

You could use Django formsets for this (https://docs.djangoproject.com/en/dev/topics/forms/formsets/). A formset is a collection of instances of the same form. You can define a form that displays the contents of one line of your CSV file and create a formset for it to be able to display the whole file.

In the examples below I use the following CSV file:

Name,Email
Bob,bob@example.com
Alice,alice@example.com

A form to represent this data could be the following (very simplified):

class PersonForm(forms.Form):
    name = forms.CharField()
    email = forms.EmailField()

To create a formset for this, do:

PersonFormSet = formset_factory(PersonForm)

To populate your formset, you should set initial data. You can do this when you instantiate your form in your view:

def some_view(request):
    # ....
    form = PersonFormSet(
               initial=[
                   {'name': "Bob", 'email': "bob@example.com"},
                   {'name': "Alice", 'email': "alice@example.com"},
               ]
           )

Or, using a class-based view:

class SomeView(FormView):
    form_class = PersonFormSet

    def get_initial(self):
        return [
                   {'name': "Bob", 'email': "bob@example.com"},
                   {'name': "Alice", 'email': "alice@example.com"},
               ]

As you can see, a list of dicts suffices, you need to do minimal manipulation on your CSV data.

The remaining challenge is getting the data at your form. There are several approaches here, but I recommend you handle everything in the same view or save the file and pass on a filename (although the latter can have security implications).