4

Currently trying to grab an average of a sum of a group by, but when I try to execute the correct queryset it complains of incorrect syntax

objs = MyModel.objects.filter(...).values('user', 'value')
objs = objs.annotate(sum=Sum('value')) 
# i've also tried adding `.values('sum')` to the above
objs = objs.aggregate(Avg('sum')) # whines here

Is there any way to do this without dropping in SQL? The desired SQL is along the lines of

SELECT AVG(`sum`) as `avg` FROM ( 
    SELECT SUM(`appname_mymodel`.`value`) AS `sum`
    FROM `appname_mymodel` 
    GROUP BY 
        `appname_mymodel`.`user_id`, `appname_mymodel`.`value`
) as subquery ORDER BY NULL;

Not sure if django plays well with subqueries.

Here is the error message:

ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'FROM (SELECT SUM(`appname_mymodel`.`value`) AS `sum` FROM `appname_' at line 1")

Django version - 1.6.5

MySQL version - 5.5

Notes:

  • If I leave off the values (the group by), it works fine (i.e. objs.filter(...).annotate(...).aggregate(...)
  • The example below is sufficient as a fully reproducible example.

Model:

# Create your models here.

class UserProfile(models.Model):
    name = models.CharField(max_length=200)

class OtherFK(models.Model):
    name = models.CharField(max_length=200)

class MyModel(models.Model):
    # Happens if value is CharField or IntegerField
    value = models.CharField(max_length=200, blank=True, null=True)
    date = models.DateField()
    user = models.ForeignKey(UserProfile)
    other_fk = models.ForeignKey(OtherFK)

Example:

>>> from app_name.models import UserProfile, OtherFK, MyModel
>>> up = UserProfile.objects.create(name="test")
>>> fk = OtherFK.objects.create(name="test")
>>> v1 = MyModel.objects.create(user=up, other_fk=fk, value="12", date="2015-10-01")
>>> v2 = MyModel.objects.create(user=up, other_fk=fk, value="13", date="2015-10-02")
>>> from django.db.models import Sum,Avg
>>> MyModel.objects.all().values("user").distinct().annotate(sum=Sum("value")).aggregate(Avg('sum'))
ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'FROM (SELECT DISTINCT `app_name_mymodel`.`user_id` AS `user_id`, SUM(`app_name_m' at line 1")
>>> MyModel.objects.all().values("user").distinct().annotate(sum=Sum("value"))
[{'sum': 25.0, 'user': 1L}]
>>> print MyModel.objects.all().values("user").distinct().annotate(sum=Sum("value")).query
SELECT DISTINCT `app_name_mymodel`.`user_id`, SUM(`app_name_mymodel`.`value`) AS `sum` FROM `app_name_mymodel` GROUP BY `app_name_mymodel`.`user_id` ORDER BY NULL
C.B.
  • 8,096
  • 5
  • 20
  • 34
  • Why the `.values()`? I think that could be the problem, try removing it. -> `MyModel.objects.filter(...).annotate(sum_=Sum('value')).aggregate(Avg('sum_'))` – Gocht Oct 29 '15 at 21:46
  • @Gocht I need to have the average of the sums at the grouped level, i.e. the average of the sums per user. – C.B. Oct 29 '15 at 21:47
  • You need to use `.values()` to group the results right? That's the first one, why the second `.values()`? – Gocht Oct 29 '15 at 21:50
  • I should have mentioned it happens with or without the second `.values()` – C.B. Oct 29 '15 at 21:50
  • The sum per user is doing right? – Gocht Oct 29 '15 at 21:54
  • @Gocht yeah with or without the second `values()` – C.B. Oct 29 '15 at 21:55

2 Answers2

0

I think this is what you need, please give a feedback after test the line below:

MyModel.objects.filter(...).values('user__id').distinct().annotate(sum=Sum('value')).aggregate(Avg('sum'))
Gocht
  • 9,924
  • 3
  • 42
  • 81
  • Still getting a programming error. `ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'FROM (SELECT DISTINCT `appname_mymodel `.`user_id` AS `user_id`, SUM(`ap' at line 1")` – C.B. Oct 29 '15 at 22:50
  • @C.B. Show your model please. – Gocht Oct 29 '15 at 22:56
  • Added a watered down aliased version. – C.B. Oct 29 '15 at 22:59
  • @C.B. I edited the answer, note `user__id` instead of `user_id` and, why are you trying to `Sum` a `CharField` ? – Gocht Oct 29 '15 at 23:00
  • data is inherently an integer, just is stored as a char right now. `user` or `user__id` both fail with the same programming error. Again, the sum works fine. – C.B. Oct 29 '15 at 23:02
  • I understand you hide some part of your code, such as names, but I think something important is being missing there. – Gocht Oct 29 '15 at 23:03
  • Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/93738/discussion-between-gocht-and-c-b). – Gocht Oct 29 '15 at 23:06
0

Looks like this is a known bug in Django 1.6, and need to upgrade to 1.7+ to fix

For now, I will just do the math in python.

C.B.
  • 8,096
  • 5
  • 20
  • 34