5

How can I update an already populated Django database with records from a data fixture (or other serialized records)?

I know I can use Django data fixtures to provide initial data. Can I use the same already written features to update the database from data fixtures (or similar serialized data like a JSON document)?

The “insert or update from serialised data” operation should be idempotent:

  • If a record doesn't exist (by its key) in the database, it should be inserted.
  • If a record already exists (by its key) in the database, it should be updated to match the data fixture.
  • The end state should be that all data from the data fixture should be updated in the database, no matter if the records already existed.

Specifically, can I update existing rows by specifying pk=null and using natural keys?

How can I use the existing Django “load data” features (whether loaddata or something else similar in Django), to read serialised data, insert records if they don't exist and update them if they already exist?

bignose
  • 30,281
  • 14
  • 77
  • 110
eugene
  • 39,839
  • 68
  • 255
  • 489

1 Answers1

4

Yes, you can with loaddata:

python3 -m django loaddata path/to/fixture/some_new_data.json

BUT:

  • don't call your new data initial_data.json else it will be reloaded every time you syncdb or run a South migraton and will wipe out any changes made since the fixture was loaded. (As mentioned mid-way through this page)
  • related to the above, loading data via fixtures can be a brittle way to do things unless you're sure that you new data won't arrive while you're writing your fixture. If it does, you risk a clash and either a failure or 'incorrect' data in your DB.

Depending on the nature of your data, it may be better to use a data migration in South. A data migration might be good when you're setting just a couple of values for lots of records (eg, changing the default of a field, or filling in an empty field with data). A data migration also lets you apply some checks/logic to things, via Python, so that you can target which records you update, or update different records in different ways, rather than writing lots of fixtures. If, however, you want to load in a ton of new records, a fixture would make more sense.

bignose
  • 30,281
  • 14
  • 77
  • 110
Steve Jalim
  • 11,989
  • 1
  • 37
  • 54
  • I tried it, ah I should have been more clear. I'll update the question. question was how can I `update` existing rows? – eugene Jun 12 '13 at 10:36
  • 1
    Ah, that update has stuffed my original answer :o) Though a data migration may still be better. What kind of changes do you need to make? – Steve Jalim Jun 12 '13 at 10:41
  • Sorry for the vague question. :( I had model A and data_for_A.json. now I added a field to A and want to give initial value for the field. (each row has different initial values) – eugene Jun 12 '13 at 10:52
  • Sounds like you can either a) manually amend your fixture and reload it or b) write a data migration to set them intelligently. I'd go with b, but I haven't seen your data – Steve Jalim Jun 12 '13 at 11:17
  • I can't do the a) method. Except I don't know the pk for each rows. Is there a way to use natural key instead of pk? (I have seen examples using natural keys for foreign key only) – eugene Jun 13 '13 at 01:47
  • Yeah, I've only used it for foreign keys. However you may be able to do it with YAML rather than JSON, but I've not looked into this. – Steve Jalim Jun 14 '13 at 08:27