Django: "Unknown Column" when run makemigrations on an app after adding an Integer Field

Question

First of all, I have recently upgraded from Django 1.6 to 1.8, and never used South, so I'm new to migrations.

I just ran:

> python manage.py makemigrations myapp
> python manage.py migrate --fake initial

to create the initial migrations for my model with existing MySQL tables. All seemed good so far.

I then added an IntegerField to my model:

new_integer_field = models.IntegerField(default = 50) #can include blank=True, null=True; seems to make no difference

Now when I run:

>python manage.py makemigrations myapp

I get

django.db.utils.OperationalError: (1054, "Unknown column 'myapp_mymodel.new_integer_field' in 'field list'")

The traceback (starting where the problem occured) is:

Traceback (most recent call last):
    File "./manage.py", line 10, in <module>
        execute_from_command_line(sys.argv)
    File ".../django/core/management/__init__.py", line 338, in execute_from_command_line
        utility.execute()
    File ".../django/core/management/__init__.py", line 312, in execute
        django.setup()
    File ".../django/__init__.py", line 18, in setup
        apps.populate(settings.INSTALLED_APPS)
    File ".../django/apps/registry.py", line 115, in populate
        app_config.ready()
    File ".../autocomplete_light/apps.py", line 9, in ready
        autodiscover()
    File ".../autocomplete_light/registry.py", line 298, in autodiscover
        autodiscover_modules('autocomplete_light_registry')
    File ".../django/utils/module_loading.py", line 74, in autodiscover_modules
        import_module('%s.%s' % (app_config.name, module_to_search))
    File ".../python2.7/importlib/__init__.py", line 37, in import_module
        __import__(name)
    File ".../mymapp/autocomplete_light_registry.py", line 42, in <module>
        x = other_model.other_model_manager.manager_function(),
    File ".../myapp/models.py", line 759, in get_a_queryset
        stus = a_queryset
    File ".../myapp/models.py", line 92, in get_another_queryset
        if obj.model_function(prog):
    File ".../myapp/models.py", line 402, in model_function
        z = object.MyManagerObjects.manager_function(self)
    File ".../myapp/models.py", line 573, in get_type
        curstart = mymodel.MyManagerObjects.get().old_field
    File ".../python2.7/site-packages/django/db/models/manager.py", line 127, in manager_method
        return getattr(self.get_queryset(), name)(*args, **kwargs)
    File ".../python2.7/site-packages/django/db/models/query.py", line 328, in get
        num = len(clone)
    File ".../python2.7/site-packages/django/db/models/query.py", line 144, in __len__
        self._fetch_all()
    File ".../python2.7/site-packages/django/db/models/query.py", line 965, in _fetch_all
        self._result_cache = list(self.iterator())
    File ".../python2.7/site-packages/django/db/models/query.py", line 238, in iterator
        results = compiler.execute_sql()
    File ".../python2.7/site-packages/django/db/models/sql/compiler.py", line 840, in execute_sql
        cursor.execute(sql, params)
    File ".../python2.7/site-packages/django/db/backends/utils.py", line 79, in execute
        return super(CursorDebugWrapper, self).execute(sql, params)
    File ".../python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
        return self.cursor.execute(sql, params)
    File ".../python2.7/site-packages/django/db/utils.py", line 97, in __exit__
        six.reraise(dj_exc_type, dj_exc_value, traceback)
    File ".../python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
        return self.cursor.execute(sql, params)
    File ".../python2.7/site-packages/django/db/backends/mysql/base.py", line 124, in execute
        return self.cursor.execute(query, args)
    File ".../python2.7/site-packages/MySQLdb/cursors.py", line 205, in execute
        self.errorhandler(self, exc, value)
    File ".../python2.7/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
        raise errorclass, errorvalue
django.db.utils.OperationalError: (1054, "Unknown column 'myapp_mymodel.new_integer_field' in 'field list'")

MyManager is simple, as follows:

class MyManager(models.Manager):
    def get_queryset(self):
        return super(MyManager, self).get_queryset().filter(...some filters...) #this is guaranteed to return a queryset with one object in it.

And inside myapp/models.py:

MyManagerObjects = MyManager()

So, my question is, why can't I add a field to mymodel and run makemigrations? What is breaking in the manager? I'm starting the stacktrace there because I've tried a variety of different commenting out strategies that seem to point to this manager regardless of what the prior calls are.

I'm sure hoping someone can tell me it's something quick and simple that I'm just not seeing, but I'm not sure how that will be the case! Anyone else had this trouble or have any suggestions?

----****---- Update 1: the error doesn't just occur with makemigrations - since some commentors asked what happens when I run python manage.py migrate I thought I'd try it for things and giggles, even though there is nothing to migrate. I was expecting to be told there was nothing to migrate, but I got the same error. So it's not makemigrations code itself; it's something not able to function because it can't find the new field. Somehow it is looking at the model and finding a field that it doesn't recognize. But why?!

----****---- Update 2: I added the start of the traceback... I'm loathe to post any of the actual paths/model names, hence the edits. Hope that's ok. A note - it is not, as far as I can tell, related to autocomplete_light (unless it is!) because when I comment out the line x = other_model.other_model_manager.manager_function() (line 42 in autocomplete_light_registry.py) the error occurs without reference to autocomplete_light. I've added a second traceback here in case that helps too!

Traceback (most recent call last):
    File "./manage.py", line 10, in <module>
        execute_from_command_line(sys.argv)
    File ".../django/core/management/__init__.py", line 338, in execute_from_command_line
        utility.execute()
    File ".../django/core/management/__init__.py", line 312, in execute
        django.setup()
    File ".../django/__init__.py", line 18, in setup
        apps.populate(settings.INSTALLED_APPS)
    File ".../django/apps/registry.py", line 115, in populate
        app_config.ready()
    File ".../django/contrib/admin/apps.py", line 22, in ready
        self.module.autodiscover()
    File ".../django/contrib/admin/__init__.py", line 24, in autodiscover
        autodiscover_modules('admin', register_to=site)
    File ".../django/utils/module_loading.py", line 74, in autodiscover_modules
        import_module('%s.%s' % (app_config.name, module_to_search))
    File ".../python2.7/importlib/__init__.py", line 37, in import_module
        __import__(name)
    File ".../myapp/admin.py", line 151, in <module>
        class MyListFilter(SimpleListFilterWithDefault):
    File ".../myapp/admin.py", line 154, in MyListFilter
        x = mymodel.MyManagerObjects.get()
    File ".../django/db/models/manager.py", line 127, in manager_method
        return getattr(self.get_queryset(), name)(*args, **kwargs)
    File ".../django/db/models/query.py", line 328, in get
        num = len(clone)
    File ".../django/db/models/query.py", line 144, in __len__
        self._fetch_all()
    File ".../django/db/models/query.py", line 965, in _fetch_all
        self._result_cache = list(self.iterator())
    File ".../django/db/models/query.py", line 238, in iterator
        results = compiler.execute_sql()
    File ".../django/db/models/sql/compiler.py", line 840, in execute_sql
        cursor.execute(sql, params)
    File ".../django/db/backends/utils.py", line 79, in execute
        return super(CursorDebugWrapper, self).execute(sql, params)
    File ".../django/db/backends/utils.py", line 64, in execute
        return self.cursor.execute(sql, params)
    File ".../django/db/utils.py", line 97, in __exit__
        six.reraise(dj_exc_type, dj_exc_value, traceback)
    File ".../django/db/backends/utils.py", line 64, in execute
        return self.cursor.execute(sql, params)
    File ".../django/db/backends/mysql/base.py", line 124, in execute
        return self.cursor.execute(query, args)
    File ".../MySQLdb/cursors.py", line 205, in execute
        self.errorhandler(self, exc, value)
    File ".../MySQLdb/connections.py", line 36, in defaulterrorhandler
        raise errorclass, errorvalue
django.db.utils.OperationalError: (1054, "Unknown column 'myapp_mymodel.new_integer_field' in 'field list'")

did you run a migrate after you ran the makemigrations the second time ? — karthikr, Sep 03 '15 at 03:32
@karthikr I can't; the error happens when I run `python manage.py makemigrations myapp` so there's nothing to migrate. — jenniwren, Sep 04 '15 at 00:19
The whole traceback would also help to identify the stage of initialization the app is getting to when it fails. — Fush, Sep 04 '15 at 00:59
Obviously I could just add the field manually and move on. I'd rather figure out what's going on though :) — jenniwren, Sep 05 '15 at 05:01

Fush · Accepted Answer · 2017-12-18T08:18:16.293

13

It looks like you are making a db query either when your module is imported or when the app is being registered: curstart = mymodel.MyManagerObjects.get().old_field

This means when you run any management command like runserver, makemigrations or migrate, if you've changed your models then they will be out of sync with your database, and making the query will throw an exception before the command can do what it's supposed to (like make your migrations).

If possible, modify your code so MyManagerObjects.get() isn't called at load time, eg by using a lambda function or wrapping it in a method that you can call at run time.

If that's not possible, you'll need to comment out that section of the code every time you make or run migrations, but that's really messy.

Update: In the full traceback there's the line:

File ".../myapp/admin.py", line 154, in MyListFilter
    x = mymodel.MyManagerObjects.get()

which means the get is being run when the admin file is imported. I think you may need to put the call to mymodel.MyManagerObjects.get() in MyListFilter.queryset() instead of in the class declaration.

https://docs.djangoproject.com/en/1.8/ref/contrib/admin/#django.contrib.admin.ModelAdmin.list_filter

edited Dec 18 '17 at 08:18

answered Sep 04 '15 at 00:56

Fush

2,469
21
19

I tried a lambda function, which I've never used before so I want to explore the suggestions further. It's weird, though, isn't it, that the traceback doesn't point to the manager? Why does it ends at the query using the manager instead of inside the manager itself? – jenniwren Sep 04 '15 at 22:53
Ok, so now I've tried creating a lambda inside the manager (no change) and creating a lambda in the spot where the error occurs. The latter produces the error at the lambda function. Does the following make sense? `curstart = lambda: mymodel.MyManagerObjects.get().old_field` then `curstart = curstart()` If so, no success, the error still occurs at the lambda line. How does django's `objects` code get around this, I wonder? – jenniwren Sep 05 '15 at 02:57
The update was very helpful - it solved that version of the problem. The first traceback is still happening, and it's not clear how to fix it within autocomplete light. The problem lies in setting `choices` when registering an autocomplete. Using a lambda didn't help; it seems it is now perhaps a new, related question related to autocomplete light. At least I understand it a bit better now. Thanks! – jenniwren Sep 17 '15 at 21:27
Yes, the problem is it looks like `x = other_model.other_model_manager.manager_function()` is being called when `autocomplete_light_registry.py` is imported. – Fush Sep 18 '15 at 06:12
I can't really make good suggestions without seeing the code, but Django normally handles situations like this for querysets, choices, template names, etc by having an attribute like `choices` for when choices are static and a method like `get_choices()` for when choices are dynamic. Can you wrap the thing you're calculating in a `get_choices` method and call it later, eg in `__init__()`? – Fush Sep 18 '15 at 06:20

score 0 · Answer 2 · answered Sep 03 '15 at 03:55

0

It shouldn't be:

python manage.py makemigrations mymodel

it should be:

python manage.py makemigrations myapp

So comment out new_integer_field and do it again I think.

You also need to run migrate to add the new_integer_fieldcolumn in the database table.

python manage.py makemigrations myapp
./manage.py migrate --fake
# add new_integer_field
./manage.py makemigrations myapp
./manage.py migrate

answered Sep 03 '15 at 03:55

François Constant

5,531
1
33
39

Sorry, I was running `python manage.py makemigrations myapp` both times. Corrected above. – jenniwren Sep 04 '15 at 00:18
Yeah OK but did my second comment fixed your issue? – François Constant Sep 04 '15 at 00:19
Are you talking about where you say to run makemigrations, then migrate? I can't do either of those things. makemigrations doesn't work, so there is nothing to migrate. Further, migrate makes the same error. I get to step 3 in your "second comment" and get an error. If that's not what you mean then I need clarification. What you suggested is exactly what I was trying to do (except my understanding is that it's `--fake-initial`, not `--fake`. – jenniwren Sep 04 '15 at 00:27
That's really weird. What happens if you comment out `new_integer_field` and run `makemigrations myapp`? it should tell you there is no change but I'm curious. Just doing this will confirm if the --fake worked. – François Constant Sep 04 '15 at 00:48
Yes, it tells me there are no changes when I comment out the new field (i.e. put the model back to the way it was without changes). – jenniwren Sep 04 '15 at 22:35

Django: "Unknown Column" when run makemigrations on an app after adding an Integer Field

2 Answers2