Django BigInteger auto-increment field as primary key?

Question

I'm currently building a project which involves a lot of collective intelligence. Every user visiting the web site gets created a unique profile and their data is later used to calculate best matches for themselves and other users.

By default, Django creates an INT(11) id field to handle models primary keys. I'm concerned with this being overflown very quickly (i.e. ~2.4b devices visiting the page without prior cookie set up). How can I change it to be represented as BIGINT in MySQL and long() inside Django itself?

I've found I could do the following (http://docs.djangoproject.com/en/dev/ref/models/fields/#bigintegerfield):

class MyProfile(models.Model):
    id = BigIntegerField(primary_key=True)

But is there a way to make it autoincrement, like usual id fields? Additionally, can I make it unsigned so that I get more space to fill in?

Thanks!

Answer 1

Django now has a BigAutoField built in if you are using Django 1.10:

https://docs.djangoproject.com/en/1.10/ref/models/fields/#bigautofield

Answer 2

Inspired by lfagundes but with a small but important correction:

class BigAutoField(fields.AutoField):
    def db_type(self, connection):  # pylint: disable=W0621
        if 'mysql' in connection.__class__.__module__:
            return 'bigint AUTO_INCREMENT'
        return super(BigAutoField, self).db_type(connection)

add_introspection_rules([], [r"^a\.b\.c\.BigAutoField"])

Notice instead of extending BigIntegerField, I am extending AutoField. This is an important distinction. With AutoField, Django will retrieve the AUTO INCREMENTed id from the database, whereas BigInteger will not.

One concern when changing from BigIntegerField to AutoField was the casting of the data to an int in AutoField.

Notice from Django's AutoField:

def to_python(self, value):
    if value is None:
        return value
    try:
        return int(value)
    except (TypeError, ValueError):
        msg = self.error_messages['invalid'] % str(value)
        raise exceptions.ValidationError(msg)

and

def get_prep_value(self, value):
    if value is None:
        return None
    return int(value)

It turns out this is OK, as verified in a python shell:

>>> l2 = 99999999999999999999999999999
>>> type(l2)
<type 'long'>
>>> int(l2)
99999999999999999999999999999L
>>> type(l2)
<type 'long'>
>>> type(int(l2))
<type 'long'>

In other words, casting to an int will not truncate the number, nor will it change the underlying type.

Answer 3

NOTE: This answer as modified, according to Larry's code. Previous solution extended fields.BigIntegerField, but better to extend fields.AutoField

I had the same problem and solved with following code:

from django.db.models import fields
from south.modelsinspector import add_introspection_rules

class BigAutoField(fields.AutoField):
    def db_type(self, connection):
        if 'mysql' in connection.__class__.__module__:
            return 'bigint AUTO_INCREMENT'
        return super(BigAutoField, self).db_type(connection)

add_introspection_rules([], ["^MYAPP\.fields\.BigAutoField"])

Apparently this is working fine with south migrations.

Answer 4

You could alter the table afterwards. That may be a better solution.

Answer 5

Since Django 3.2 the type of implicit primary key can be controlled with the DEFAULT_AUTO_FIELD setting (documentation). So, there is no need anymore to override primary keys in all your models.

#This setting will change all implicitly added primary keys to BigAutoField
DEFAULT_AUTO_FIELD = 'django.db.models.BigAutoField'

Note that starting with Django 3.2 new projects are generated with DEFAULT_AUTO_FIELD set to BigAutoField (release notes).

Answer 6

As stated before you could alter the table afterwards. That is a good solution.

To do that without forgetting, you can create a management module under your application package and use the post_syncdb signal.

https://docs.djangoproject.com/en/dev/ref/signals/#post-syncdb

This can cause django-admin.py flush to fail. But it is still the best alternative I know.

Answer 7

I also had the same problem. Looks like there is no support for BigInteger auto fields in django.

I've tried to create some custom field BigIntegerAutoField but I faced a problem with south migration system (south couldn't create sequence for my field).

After giving a try couple of different approaches I decided to follow Matthew's advice and do alter table (e.g. ALTER TABLE table_name ALTER COLUMN id TYPE bigint; in postgre)

Would be great to have solution supported by django (like built in BigIntegerAutoField) and south.