return datetimes in the active timezone with a django query

Question

I am trying to retrieve the last n hour rows from a table and print their datetimes in a given timezone, the timezone to use when printing dates is given, I am trying to use activate to make django return the datetimes with the proper timezone but it returns dates as UTC.

here is my current code:

min_time = datetime.datetime.now(link.monitor.timezone) - datetime.timedelta(hours=period)

timezone.activate(link.monitor.timezone)
rows = TraceHttp.objects.values_list('time', 'elapsed').filter(time__gt=min_time,link_id=link_id,elapsed__gt=0)

array = []
for row in rows:
    array.append((row[0].astimezone(link.monitor.timezone),row[1]))

I want to avoid using the astimezone function and make Django do this for me, is there sometimes I'm missing about the activate function?

EDIT

Here are my models, as you can see the timezone to display is saved on the "monitor" model:

class Link(models.Model):
   ...
   monitor = models.ForeignKey(Monitor)
   ...

class Monitor(models.Model):
    ...
    timezone = TimeZoneField(default='Europe/London')

class TraceHttp(models.Model):
    link = models.ForeignKey(Link)
    time = models.DateTimeField()
    elapsed = models.FloatField()

Answer 1

After some research I noticed that Django allways returns datetimes as UTC and it's up to you to interpret them in the correct timezone either by using the datetime.astimezone(timezone) method or activating a certain timezone.

The django active function just changes the way that the datetime will be rendered on a template but doesn't actually localize a timezone.

Answer 2

If you ever find yourself doing timezone.localtime(dt_value) or dt_value.astimezone(tzifo) in a loop for a few million times to calculate what's the current date in your timezone, the likely best approach as of 1.10 <= django.VERSION <= 2.1 is to use django.db.models.functions.Trunc and related functions, i.e use a queryset like:

from django.db.models.functions import Trunc, TruncDate

qs = MyModel.objects.filter(...).values(
    'dtime',
    ...,
    dtime_at_my_tz=Trunc('dtime', 'second', tzinfo=yourtz),
    date_at_my_tz=TruncDate('dtime', tzinfo=yourtz),
    month=TruncDate(Trunc('dtime', 'month', tzinfo=yourtz)),
    quarter=TruncDate(Trunc('dtime', 'quarter', tzinfo=yourtz))
)

This will return datetimes or dates in the right timezone. You can use other Trunc* functions as shorthand. TruncDate is especially useful if all you need are datetime.dates

This will offload date calculations to the database, usually with a big reduction in code complexity and increased speed (in my case, over 6.5 million timezone.localtime(ts) were contributing 25% of total CPU time)

Note on TruncMonth and timezones

A while ago I found that I couldn't get 'proper' months out of TruncMonth or TruncQuarter: a January 1st would become a December 31st.

TruncMonth uses the currently active timezone, so (correctly) a datetime of 2019-01-01T00:00:00Z gets converted to the previous day for any timezone that has a positive offset from UTC (Western Europe and everywhere further East). If you're only interested in the 'pure month' of an event datetime (and you probably are if you're using TruncMonth) this isn't helpful, however if you timezone.activate(timezone.utc) before executing the query (that is, evaluating your QuerySet) you'll get the intended result. Keep in mind that events occurred from your midnight until UTC's midnight will fall under the previous month (and in the same way datetimes from your timezone's midnight to UTC's midnight will be converted to the 'wrong' month)

Answer 3

You can use now() function from django.utils, but you need to set two variables in settings: USE_TZ and TIME_ZONE, the first with True and the other with the default timezone that will be used to generate the datetime.

You can see more informations in django documentation here.