7

I am using django restframework and want to handle multiple databases. I am using django function using(alias) and switch_db(alias) for manually switching between database whenever I want to Get, Post or update data.

I am facing problem while posting and updating data.i.e whenever serializer.is_valid() will be called.

serializer.is_valid() will go and first check for db_alias in model.py file. If I have not specified db_alias under meta it will select default database for validation. If I am specifying db_alias in model it will select that database for validation.

But I do not want to specify db_alias in model since my usecase is to store data on different database based on some logic in my view file. So dynamically I want to select database from view and want to store data in it.

Almost I have implemented but I am facing problem when my model is having Reference Field. In this case serializer.is_valid is going to default database for validating that reference field.

Required Details: I am using mongoengine(0.9.0), document, document serializer.

My files are as below:

model.py:

class ngroup(Document):

    groupname = StringField(max_length=100, required=True)
    description = StringField(max_length=100, required=False)
    parent = ReferenceField('ngroup',null=True)
    created_ts = DateTimeField(default=datetime.now())
    modified_ts = DateTimeField(default=datetime.now())
    is_deleted = BooleanField(default=False)

serializer.py:

from device_management.models import ngroup
from rest_framework_mongoengine.serializers import DocumentSerializer
from mongoengine import EmbeddedDocumentField, ReferenceField, StringField, ObjectIdField, IntField, BooleanField, FloatField, DateTimeField,ListField


class ngroupSerializer(DocumentSerializer):

    class Meta:
        model = ngroup

    def setOrgId(self, orgid):
        self.orgid = orgid

    def create(self, validated_data):
        ngroup_data = ngroup(**validated_data).switch_db(self.orgid)
        ngroup_data.save()
        return ngroup_data

    def update(self, instance, validated_data):
        ngroup_data = ngroup.objects.using(self.orgid).get(id = instance.id)
        ngroup_data = ngroup_data.switch_db(self.orgid)
        ngroup_data = ngroup_data.update(**validated_data)
        return validated_data

    def to_internal_value(self, data):
        print "data:" , data
        return super(DocumentSerializer, self).to_internal_value(data)  

view.py:

def create(self, request, format=None):
    orgid = str(request.user.orgid.id)
    data=request.data

    serializer = ngroupSerializer(data=data)
    if serializer.is_valid():
        try:
            serializer.save()
        except Exception as e:
            log.error("create" , extra={'extra':{'error': str(e),'message' :strings.DATA_VALIDATION_ERROR }})
            return response.errorResponse(message=strings.SERIALIZATION_ERROR_MSG,error=str(e),rstatus=status.HTTP_400_BAD_REQUEST)
        return response.successResponse(res_data=serializer.data, message=strings.POST_SUCCESS_MSG, rstatus=status.HTTP_201_CREATED)
    log.error("create" , extra={'extra':{'error': serializer.errors,'message' :strings.DATA_VALIDATION_ERROR }})
    return response.errorResponse(message=strings.DATA_VALIDATION_ERROR,error=serializer.errors,rstatus=status.HTTP_400_BAD_REQUEST)

settings.py:

DATABASES = {
     'default': {
        'ENGINE': 'django_mongodb_engine',
        'NAME': 'mydb',
        'USER': 'admin',
        'PASSWORD':'admin123',
        'HOST': '127.0.0.1',
        'PORT': 27017,
        'DBTYPE' : "mongo",
    },
    '586e47c784413825f2b5bc49': {
        'ENGINE': 'django_mongodb_engine',
        'NAME': 'mydb1',
        'USER': 'admin',
        'PASSWORD':'admin123',
        'HOST': '127.0.0.1',
        'PORT': 27017,
        'DBTYPE' : "mongo",
    },
    # Enter super_user organisation here. This DB will be same as default db only always
    '58996fb28441384430dc8ae6': {
        'ENGINE': 'django_mongodb_engine',
        'NAME': 'mydb',
        'USER': 'admin',
        'PASSWORD':'admin123',
        'HOST': '127.0.0.1',
        'PORT': 27017,
        'DBTYPE' : "mongo",
    },
}

pip freeze(Installation versions):

Django==1.5.11
django-browserid==2.0.2
django-classy-tags==0.8.0
django-missing==0.1.18
django-mongo-auth==0.1.3
django-mongodb-engine==0.6.0
django-mongoengine==0.2.1
django-redis-sessions==0.5.6
django-rest-framework-mongoengine==3.3.0
django-sekizai==0.10.0
django-websocket-redis==0.4.7
djangorestframework==3.1.2
djangorestframework-jwt==1.9.0
djangotoolbox==1.8.0
gevent==1.1.2
greenlet==0.4.10
httplib2==0.9.2
mongoengine==0.9.0
oauthlib==2.0.1
pika==0.10.0
Pygments==2.1.3
PyJWT==1.4.2
pymongo==2.8
python-dateutil==2.6.0
python-openid==2.2.5
pytz==2016.10
redis==2.10.5
requests==2.12.3
requests-oauthlib==0.7.0
rest-condition==1.0.3
six==1.10.0
tweepy==3.5.0
twilio==5.7.0

I have overide create in serializer to take care of database while calling serializer.save() but how to handle serializer.is_valid().

My project has been stuck at this point. Any help will be greatly appreciated...

2 Answers2

2

This is not the exact solution to above problem but we have 2 options.

1) Do not go for serializer.is_valid() or serializer.save(). Directly create ngroup:

def my_create(self, validated_data):
    gateway = Gateway(**validated_data).switch_db(self.orgid)
    gateway.save()
    return gateway

2) Another solution is to use django-mogodb-engine and django models and modelserializers instead of documents and documents serializers.

I have tried following this with Django-mongodb-engine and are working well:

-> JWT authentication
-> custom user
-> foreign key
-> embedded model
-> list of embedded model
-> dict field
-> **Routers for switching between databases.(Manual switching DB is not required)**

I can also use middleware_classes to specify runtime in each request which database to use. Reference Link: Django Authenticate Backend Multiple Databases

Community
  • 1
  • 1
0

Husain,

Unfortunately, you're mixing incompatible projects together. Mongoengine, django-mongoengine and Django-REST-Framework-Mongoengine projects are alternative to django-mongodb-engine, they are not meant to be used together.

As far as I know, django-mongodb-engine project's been dead for 2 years, or even longer, to be honest. At the same time, Mongoengine stack is working in production, though, development is not too active. I really want to create a proper django database backend out of Mongoengine to make it a first-class citizen in Django world, and it seems like Django guys are looking in that direction, too.

You might also want to look into this post.

This is my second attempt. I tried to switch the database connection in view create(). Didn't work for me:

settings.py

# We define 2 Mongo databases - default (project) and project2
MONGODB_DATABASES = {
    "project": {
        "name": "project",
        "host": "localhost",
        "port": 27017,
        "tz_aware": True,  # if you use timezones in django (USE_TZ = True)
    },

    "project2": {
        "name": "project2",
        "host": "localhost",
        "port": 27017,
        "tz_aware": True,  # if you use timezones in django (USE_TZ = True)
    }
}


mongoengine.register_connection(alias='default', name=MONGODB_DATABASES["project"]["name"], host="local")
mongoengine.register_connection(alias='project2', name=MONGODB_DATABASES["project2"]["name"], host="local")

connection = mongoengine.connect(db="project", alias='default')

views.py

class AuthorViewSet(MongoModelViewSet):
    lookup_field = 'id'
    serializer_class = AuthorSerializer

    def create(self, request, format=None):
        global Author
        mongoengine.connection.disconnect(alias='project')
        mongoengine.connect('project2', alias='project2')
        return super(AuthorViewSet, self).create(request, format)

    def get_queryset(self):
        return Author.objects.all()
Community
  • 1
  • 1
Boris Burkov
  • 13,420
  • 17
  • 74
  • 109
  • Hi Boris, Can I use django-mongodb-engine as database backend and model and modelserializer instead of document and document serializer. – Husain Pansoliwala Mar 02 '17 at 05:44
  • continue... i.e If I start using django-mongodb-engine instead of mongoengine and rest-framework-mongoengine, will that can have any problems. Because I have checked following things with django-mongodb-engine + Modeld.model + Modelserializer and Mongodb as database. 1) Database routers for database switching. 2) Embedded Model field 3) DictField and things are working really nice – Husain Pansoliwala Mar 02 '17 at 05:58
  • continue... and things are working really nice So if I migrate my project to models from documents, will that can stop me anywhere in future. Are there any limitations of django-mongodb-engine ??? (Since you mention that "django-mongodb-engine project's been dead for 2 years, or even longer,") – Husain Pansoliwala Mar 02 '17 at 05:59
  • @HusainPansoliwala That's actually a very interesting question - I really like this consistency with the rest of django in `django-mongodb-engine`. I've never really worked with `django-mongodb-engine` - I started using Mongo with Django in 2014 and it was already abandoned at that point. I'm not sure about how stable that implementation is and what's the level of support of Mongo-specific features in it, like EmbeddedDocuments and stuff. But if you managed to give it a try, can you share your experience with this approach? – Boris Burkov Mar 02 '17 at 08:40
  • I have replaced serializers like below: class ngroupserializer(DocumentSerializer): class Meta: def _db_switch(self): print "_db_switch......\n" return switch_db(ngroup, 'org') model = property(_db_switch) But it has no effect. It does not works. – Husain Pansoliwala Mar 05 '17 at 10:28
  • not able to understand your settings what you prefer. If I have multiple database than how can I connect to all databases. That setting will only connect to default one. And also where can I give command line argument to specify which database to use. – Husain Pansoliwala Mar 05 '17 at 10:35
  • @HusainPansoliwala You're right, I'm trying to fix the example now. – Boris Burkov Mar 05 '17 at 11:31
  • @HusainPansoliwala I tried to switch the database in viewset method.Currently it doesn't work, but may be you manage to tweak my code, using this approach? – Boris Burkov Mar 05 '17 at 15:52
  • I have also tried to use property in models as below: class ngroup(Document): title = StringField(max_length=30) des = StringField(max_length=30) def _db_switch(self): print "_db_switch......\n" meta = {'strict': False , 'db_alias' : 'org'} return meta meta = property(_db_switch) But this also does not works. – Husain Pansoliwala Mar 06 '17 at 05:43