Django specify which database to use for module

Question

In my Django project I have a couple of applications, one of them is email_lists and this application does a lot of data handling reading data from the Model Customers. In my production environment I have two databases: default and read-replica. I would like all queries in a particular module to be made against the replica-set database.

I can do that if I explicitly tell the query to do so:

def get_customers(self):
    if settings.ENV == 'production':
        customers = Customer.objects.using('read-replica').filter()
    else:
        customers = Customer.objects.filter()

but this module has more than 100 queries to the Customer and other models. I also have queries to relations like:

def get_value(self, customer):
    target_sessions = customer.sessions.filter(status='open')
    carts = Cart.objects.filter(session__in=target_sessions)

the idea is that I want to avoid writing:

if settings.ENV == 'production':
    instance = Model.objects.using('read-replica').filter()
else:
    instance = Model.objects.filter()

for every query. There are other places in my project that do need to read from default database so it can't be a global setting. I just need this module or file to read using the replica.

Is this possible in Django, are there any shortcuts ?

Thanks

ibaguio · Accepted Answer · 2020-03-31T18:27:34.993

4

You can read on django database routers for this, some good examples can be found online as well and they should be straightforward.

--

Another solution would be to modify the Model manager.

from django.db import models


class ReplicaRoutingManager(models.Manager):
    def get_queryset(self):
        queryset = super().get_queryset(self)

        if settings.ENV == 'production':
            return queryset.using('read-replica')

        return queryset


class Customer(models.Model):

    ...
    objects = models.Manager()
    replica_objects = ReplicaRoutingManager()

with this, you can just use the normal Customer.objects.filter and the manager should do the routing.

I still suggest going with the database router solution, and creating a custom logic in the class. But if the manager works for you, its fine.

edited Mar 31 '20 at 18:27

answered Mar 31 '20 at 17:45

ibaguio

2,298
3
20
32

The database router receives a model, so I could route a model's queries to the replica, and the `hints` attribute only contains an instance. I can't wrap my head around how to make a module route to the replicate or all queries coming from an application to be routed to the replica. – PepperoniPizza Mar 31 '20 at 18:08
so you're saying you have a Model, and when a query is done by Module A, you need it to be queried to the default, and when a query is done by Module B, you need it routed to replica? – ibaguio Mar 31 '20 at 18:10
I have another idea using Model manager / queryset. give me a few minutes to whip up a pseudocode – ibaguio Mar 31 '20 at 18:14
Yes, something like all the queries from `email_lists.views.py` file should be made against the replica, all other queries go to default. – PepperoniPizza Mar 31 '20 at 18:21

score 1 · Answer 2 · answered Mar 31 '20 at 19:35

If you want All the queries in the email_lists app to query read-replica, then a router is the way to go. If you need to query different databases within the same app, then @ibaguio's solution is the way to go. Here's a basic router example similar to what I'm using:

project/database_routers.py

MAP = {'some_app': 'default',
       'some_other_app': 'default',
       'email_lists': 'read-replica',}

class DatabaseRouter:

    def db_for_read(self, model, **hints):
        return MAP.get(model._meta.app_label, None)

    def db_for_write(self, model, **hints):
        return MAP.get(model._meta.app_label, None)

    def allow_relation(self, object_1, object_2, **hints):
        database_object_1 = MAP.get(object_1._meta.app_label)
        database_object_2 = MAP.get(object_2._meta.app_label)
        return database_object_1 == database_object_2

    def allow_migrate(self, db, app_label, model=None, **hints):
        return MAP.get(app_label, None)

In settings.py:

DATABASE_ROUTERS = ['project.database_router.DatabaseRouter',]

It looks like you only want it in production, so I would think you could add it conditionally:

if ENV == 'production':
    DATABASE_ROUTERS = ['project.database_router.DatabaseRouter',]

Django specify which database to use for module

2 Answers2