7

I'm running Django 1.6.6 on production and have recently upgraded to 1.9.7 on staging (dev server). This update was performed on the server and I followed the steps outlined here Upgrading from South.

I noticed that the structure of the migration files have changed, and they no longer include a create statement. This causes issues because if I pull this new code from my GitHub repo and run python manage.py makemigrations or python manage.py migrate, it says:

django.db.utils.OperationalError: no such table: appname_modelname

The traceback points to my urls.py because I'm referencing the model in a queryset:

queryset=list(chain(models.modelname.objects.filter(booleanField=True).order_by(object), models.aDifferentModel.objects.all())),

Prior to the 1.9 upgrade, syncdb created the tables for me, but this isn't the case with migrate. I've also tried python manage.py migrate --run-syncdb but this gives the same error.

However, if I copy the SQLite database from my production or staging environments to my local machine and run the command, it works (because the table is already in the database).

Do I have to manually create these tables (though I assume not) or am I doing something wrong?

Edit: Added code snippets and tracebacks. Sorry for not doing this initially.

models.py

class HowToApply(models.Model):
    title = models.CharField(max_length=500, blank=True, null=True)
    notice = models.TextField(blank=True, null=True)
description = models.TextField(blank=True, null=True)
active = models.BooleanField(default=None)
image = models.FileField(upload_to='numeric/img/%Y', blank=True, null=True)
mobile_image = models.FileField(upload_to='mobile/img/%Y', blank=True, null=True)
sequence_number = models.IntegerField(unique=True)

...

urls.py

from django.conf.urls import patterns, include, url
from django.views.generic import RedirectView, TemplateView, ListView, CreateView
from numeric import models, forms, views
from honeypot.decorators import check_honeypot
from numeric.views import CheckDeviceView
from itertools import chain

urlpatterns = patterns('',
    url(r'^academy/howtoapply/$',
        ListView.as_view(
            queryset =  list(chain(models.HowToApply.objects.filter(active=True).order_by('sequence_number'), models.AcademyAdmin.objects.all())),
        template_name = 'numeric/apply.html'
    ),
    name='apply'),

...

traceback

Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/var/www/website_mig/venv/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 350, in execute_from_command_line
    utility.execute()
  File "/var/www/website_mig/venv/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 342, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/var/www/website_mig/venv/local/lib/python2.7/site-packages/django/core/management/base.py", line 348, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/var/www/website_mig/venv/local/lib/python2.7/site-packages/django/core/management/base.py", line 398, in execute
    self.check()
  File "/var/www/website_mig/venv/local/lib/python2.7/site-packages/django/core/management/base.py", line 426, in check
    include_deployment_checks=include_deployment_checks,
  File "/var/www/website_mig/venv/local/lib/python2.7/site-packages/django/core/checks/registry.py", line 75, in run_checks
    new_errors = check(app_configs=app_configs)
  File "/var/www/website_mig/venv/local/lib/python2.7/site-packages/django/core/checks/urls.py", line 10, in check_url_config
    return check_resolver(resolver)
  File "/var/www/website_mig/venv/local/lib/python2.7/site-packages/django/core/checks/urls.py", line 19, in check_resolver
    for pattern in resolver.url_patterns:
  File "/var/www/website_mig/venv/local/lib/python2.7/site-packages/django/utils/functional.py", line 33, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File "/var/www/website_mig/venv/local/lib/python2.7/site-packages/django/core/urlresolvers.py", line 417, in url_patterns
    patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
  File "/var/www/website_mig/venv/local/lib/python2.7/site-packages/django/utils/functional.py", line 33, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File "/var/www/website_mig/venv/local/lib/python2.7/site-packages/django/core/urlresolvers.py", line 410, in urlconf_module
    return import_module(self.urlconf_name)
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/var/www/website_mig/project/urls.py", line 14, in <module>
    (r'^', include('numeric.urls')),
  File "/var/www/website_mig/venv/local/lib/python2.7/site-packages/django/conf/urls/__init__.py", line 52, in include
    urlconf_module = import_module(urlconf_module)
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/var/www/website_mig/numeric/urls.py", line 144, in <module>
    queryset = list(chain(models.HowToApply.objects.filter(active=True).order_by('sequence_number'), models.AcademyAdmin.objects.all())),
  File "/var/www/website_mig/venv/local/lib/python2.7/site-packages/django/db/models/query.py", line 258, in __iter__
    self._fetch_all()
  File "/var/www/website_mig/venv/local/lib/python2.7/site-packages/django/db/models/query.py", line 1074, in _fetch_all
    self._result_cache = list(self.iterator())
  File "/var/www/website_mig/venv/local/lib/python2.7/site-packages/django/db/models/query.py", line 52, in __iter__
    results = compiler.execute_sql()
  File "/var/www/website_mig/venv/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 852, in execute_sql
    cursor.execute(sql, params)
  File "/var/www/website_mig/venv/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/var/www/website_mig/venv/local/lib/python2.7/site-packages/django/db/utils.py", line 95, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/var/www/website_mig/venv/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/var/www/website_mig/venv/local/lib/python2.7/site-packages/django/db/backends/sqlite3/base.py", line 323, in execute
    return Database.Cursor.execute(self, query, params)
django.db.utils.OperationalError: no such table: numeric_howtoapply`
naktinis
  • 3,957
  • 3
  • 36
  • 52
Keenan Lawrence
  • 1,446
  • 10
  • 16
  • The `migrate` management command is definitely supposed to create the tables for you. Why does `urls.py` even get imported when trying to run the migrations? – Sven Marnach Jun 09 '16 at 12:51
  • have you tried `python manage.py makemigrations ` ? – DeepSpace Jun 09 '16 at 12:52
  • You should very definitely not be making queries in your urls.py. – Daniel Roseman Jun 09 '16 at 12:55
  • 1
    Before upgrading to 1.9, you should upgrade incrementally (1.7 then 1.8 first). It will make it *much* easier – Sayse Jun 09 '16 at 13:19
  • Instead of fake error messages like "django.db.utils.OperationalError: no such table: appname_modelname" why don't you post the complete error trace? – e4c5 Jun 09 '16 at 14:39
  • @SvenMarnach I don't know. It doesn't happen when I use `syncdb` – Keenan Lawrence Jun 10 '16 at 08:04
  • @DeepSpace, yes I have. Same error. – Keenan Lawrence Jun 10 '16 at 08:05
  • @DanielRoseman I'm really new to Django (using it for a month) and I took this project over from someone and it had zero documentation. According to the [Django Docs](https://docs.djangoproject.com/en/1.9/intro/tutorial03/), the `queryset` should be executed in `views.py` and then returns a rendered response, which you then use in `urls.py` Would this solve the issue? – Keenan Lawrence Jun 10 '16 at 08:08
  • Kind of. The logic definitely belongs in the view; I'm confused about your description of returning the response to urls.py though. Perhaps more details of what your urls and views are doing would help. – Daniel Roseman Jun 10 '16 at 08:14
  • I take it that the query in `urls.py` is executed at import time (since usually all code in that file is executed at import time). If this is true, you can easily see why `urls.py` is imported by checking the traceback. More generally, I think we would need more information to be able to help you (relevant parts of your code as well as the full traceback of the error you get). – Sven Marnach Jun 10 '16 at 10:27
  • @Sayse Thank you for your suggestion. `python manage.py migrate` works perfectly well with Django 1.8. I'm studying the changelogs of Django 1.9 to see if I need to rewrite any of the code (P.S this project was started using Django 1.5) – Keenan Lawrence Jun 10 '16 at 10:57
  • @KeenanLawrence From the traceback, I can indeed see that Django is importing the url patterns as part of the checks. I didn't know it's doing that (and apparently it's only doing it since version 1.9). In any case, posting the actual code and the traceback helped getting an answer (which is almost always the case). – Sven Marnach Jun 10 '16 at 13:05
  • @SvenMarnach Yes, sorry for not posting the actual code and the traceback with the original post. Thank you for your help! – Keenan Lawrence Jun 10 '16 at 13:11

1 Answers1

8

The problem is that your queryset is being evaluated when the urls.py loads. When you run makemigrations for a new project, this causes the error because the table has not been created yet.

You can fix this by subclassing ListView and moving the queryset into get_queryset.

class MyListView(ListView):
    template_name = 'numeric/apply.html'

    def get_queryset(self):
        return list(chain(models.HowToApply.objects.filter(active=True).order_by('sequence_number'), models.AcademyAdmin.objects.all()))

Then change your url pattern to use your new view.

url(r'^academy/howtoapply/$',
    MyListView.as_view(),
    name='apply',
),

Django 1.9 runs some checks to validate your url patterns, which means that the url patterns are loaded before the makemigrations command runs. Django 1.8 does not have these checks, so you can get away with setting the queryset as you have done.

Alasdair
  • 298,606
  • 55
  • 578
  • 516
  • This solution is perfect! Thank you so much! – Keenan Lawrence Jun 10 '16 at 12:21
  • 2
    I having the same problem in a slightly different way. Check loads urls.py then that loads views.py, then that loads forms.py and then one of the forms has a ModelChoiceField and this causes an exception because of the queryset parameter. Is there anyway to get around this ? Are ModelChioceFields queryset supposed to be handled differently in Django 1.9 ? – thelinuxer Sep 08 '16 at 06:56
  • @thelinuxer as far as I know, ModelChoiceFields shouldn't cause a problem. I would ask a new question and include the code and traceback. – Alasdair Sep 08 '16 at 09:01
  • 1
    I found out my problem was related to initial attribute of the ModelChoiceField. Moved these to __init__ of the form and all working properly now. – thelinuxer Sep 25 '16 at 16:44
  • Happy I helped :) – thelinuxer Jan 15 '18 at 20:26