13

I've written a series of tests for my Django app, and would like to run them on a copy of my production database.

As far as I can tell, the best way to do this is using fixture loading like so:

  • Run manage.py dumpdata -o app.dump
  • Move the resulting app.dump file to a fixtures directory in the [app name] folder
  • Specify a 'fixtures' class attribute on my django.test.TestCase subclass

However, this approach is cumbersome. I have multiple apps, and running manage.py dumpdata for each of them and manually moving around fixtures files every time I want to test my app is a pain.

Is there an easier way to automatically generate a copy of my entire production database and test my Django apps against it?

Sam
  • 1,952
  • 1
  • 18
  • 31
  • 1
    This is only feasible for small websites. How long do you think it's going to take to just setup the test if there's several GB of data – e4c5 May 07 '17 at 13:26
  • I know! Fortunately, my website is quite small. – Sam May 08 '17 at 15:02

2 Answers2

5

Generally, testing against the live DB or a copy of the live DB is discouraged. Why? Because tests need to be predictable. When you make a copy of the live db, the input becomes unpredictable. The second problem is that you cannot obviously test on the live site, so you need to clone the data. That's slow for anything more than few MB in size.

Even if the DB is small, dumpdata followed by loaddata isn't the way. That's because dumpdata by default exports in a JSON format which has a large generating overhead, not to mention making the data file very bulky. Importing using loaddata is even slower.

The only realistic way to make a clone is using the database engines built in export/import mechanism. In the case of sqlite that's just copying the db file. For mysql it's SELECT INTO OUTFILE followed by LOAD DATA INFILE. And for postgresql it's COPY TO followed by COPY FROM and so on.

All of these export/import commands can be executed using the lowlevel connection object available in django and thus can be used to load fixtures.

e4c5
  • 52,766
  • 11
  • 101
  • 134
  • Thanks for the response, e4c5. Basically what I'm hearing is that I should find another way to do what I'm trying to do. I've posted the problem as a separate question here in case you have thoughts: http://stackoverflow.com/questions/43876310/how-should-i-test-a-database-driven-django-cms-for-404-errors – Sam May 09 '17 at 17:31
0

You didn't mention which version of Django you're using, but looking at the 1.11 documentation:

It's not clear from the 1.11 docs about fixture loading for tests whether they would also look in FIXTURE_DIRS, though. So this might not solve your problem entirely.

bouteillebleu
  • 2,456
  • 23
  • 32