$group aggregation with $sum returns nothing

Question

I have a query which runs as expected when it's directly executed in MongoDB, but I'm facing some troubles trying to make it work through MongoEngine. In fact, it returns nothing.

Query on MongoDB (which works correctly):

db.annual_account.aggregate([{ $group: { "_id":"$adress.city", "total"{$sum: 1} }} ])

Result (which is what I expect):

{ "_id" : "Genk", "total" : 1 }
{ "_id" : "Ottignies-Louvain-la-Neuve", "total" : 1 }
{ "_id" : "Ganshoren", "total" : 1 }
{ "_id" : "Mont-de-l'Enclus", "total" : 1 }
{ "_id" : "Charleroi", "total" : 1 }

And now, my query on MongoEngine in my views.py:

class StatisticsView(generics.ListAPIView):
    serializer_class = AnnualAccountSerializer

    def get_queryset(self):
        group = {"$group" : {"_id": "$adress.city", "total": {"$sum":1} } }
        response = AnnualAccount.objects.aggregate(group)
        return response

Results:

HTTP 200 OK
Allow: GET, HEAD, OPTIONS
Content-Type: application/json
Vary: Accept

[
    {},
    {},
    {},
    {},
    {},
]

Does someone has any idea of what is wrong, please? Why I don't have the same result between the shell Mongo and MongoEngine?

Thank you,

Simon

EDIT1

I have created a simple example for test but still have nothing in return:

Model.py:

class Adress(EmbeddedDocument):
    city = StringField()
class User(Document):
    name = StringField()
    adress = EmbeddedDocumentField(Adress)

Serializer.py:

class UserSerializer(serializers.DocumentSerializer):
    class Meta:
        model = User
        fields = '__all__'

Views.py:

class UserView(generics.ListAPIView):
    serializer_class = UserSerializer

    def get_queryset(self):

        group = {"$group": {"_id":"$adress.city", "total":{"$sum":1}}}
        response = User.objects.aggregate(group)

        return response

I triple checked that I have a collection named "user" and the fields corresponding. If I don't use the "$group" I have my data displayed. It seems that the problem comes from the "$group" operator but it's weird, I don't have any explications. Below, the versions that I use:

Django                            1.11.17
django-rest-framework-mongoengine 3.3.1
djangorestframework               3.7.7
mongoengine                       0.11.0
pymongo                           3.8.0

one is using `"_id":"$adress.city"`, the other `"_id": "$code"`, is that a typo or am I missing something? — bagerard, Jul 25 '19 at 17:42
Yes, it was the wrong that I copied, but the result is still the same — Simon23, Jul 25 '19 at 18:08

score 0 · Answer 1 · answered Jul 25 '19 at 18:32

0

Your code looks right, I'm able to get result with the following simple example

class Adress(EmbeddedDocument):
    city = StringField()

class AnnualAccount(Document):
    adress = EmbeddedDocumentField(Adress)

AnnualAccount(adress=Adress(city='A')).save()
AnnualAccount(adress=Adress(city='B')).save()

group = {"$group" : {"_id": "$adress.city", "total": {"$sum":1} } }
print(list(AnnualAccount.objects.aggregate(group)))
## [{u'_id': u'B', u'total': 1}, {u'_id': u'A', u'total': 1}]

answered Jul 25 '19 at 18:32

bagerard

5,681
3
24
48

Thanks for your help, I still have nothing in return. Check my edit 1, I created a simple example as yours but still nothing – Simon23 Jul 25 '19 at 20:41
Mongoengine 0.11 is quite old, try with a more recent version (I was using 0.18), also ensure pymongo gets updated, maybe it will help. What you may want to try is also to use pymongo directly to check if it’s a mongoengine problem or pymongo (pymongo is the driver underneath mongoengine). c=User._get_collection() then response = list(c.aggregate([group])) – bagerard Jul 25 '19 at 20:51
Hey Bagerard, I tried to use pymongo directly and it works correctly. However, I updated mongoengine to last version (0.18.2) and it still not works... nothing display. Could show me the package version you are using, please? – Simon23 Jul 27 '19 at 08:37
Have you tried to run my example from a python shell (or at least outside the view context)? In fact I’m suspecting that your problem comes from UserSerializer, I guess this one is meant to serialise User instance but the aggregation doesn’t return User instance – bagerard Jul 28 '19 at 08:15
Yes, I have tried your example in python shell, it works well, the data is displayed. I understand what you think about the serializer, it seems logical in fact... but the problem is I can't create a 2nd model and a 2nd serializer because these data doesn't exist in my database... Do you have an idea for that? – Simon23 Jul 28 '19 at 11:31
I m not familiar with that serializer framework but you can try to not specify any serializer and return list(response), since it is native python structure (dict, list, strings, et ) you shouldn’t need any serializers – bagerard Jul 28 '19 at 12:26

$group aggregation with $sum returns nothing

1 Answers1