0

Timezones in Django...

I am not sure why this is so difficult, but I am stumped. I have a form that is overwriting the UTC dateTime in the database with the localtime of the user. I can't seem to figure out what is causing this.

my settings.py timezone settings look like:

LANGUAGE_CODE = 'en-us'
TIME_ZONE = 'America/Toronto'
USE_I18N = True
USE_L10N = False
USE_TZ = True

I am in Winnipeg, my server is hosted in Toronto. My users can be anywhere.

I have a modelfield for each user that is t_zone = models.CharField(max_length=50, default = "America/Winnipeg",) which users can change themselves.

with respect to this model:

class Build(models.Model):
    PSScustomer = models.ForeignKey(Customer, on_delete=models.CASCADE)
    buildStart = models.DateTimeField(null=True, blank=True)
    ...

I create a new entry in the DB using view logic like:

...
now = timezone.now()
newBuild = Build(author=machine,
                PSScustomer = userCustomer,
                buildStart = now,
                status = "building",
                addedBy = (request.user.first_name + ' ' +request.user.last_name),
                ...
                )
newBuild.save()

buildStart is saved to the database in UTC, and everything is working as expected. When I change a user's timezone in a view with timezone.activate(pytz.timezone(self.request.user.t_zone)) it will display the UTC time in their respective timezone.

All is good (I think) so far.

Here is where things go sideways: When I want a user to change buildStart in a form, I can't seem to get the form to save the date to the DB in UTC. It will save to the DB in whatever timezone the user has selected as their own.

Using this form:

class EditBuild_building(forms.ModelForm):
    buildStart = forms.DateTimeField(input_formats = ['%Y-%m-%dT%H:%M'],widget = forms.DateTimeInput(attrs={'type': 'datetime-local','class': 'form-control'},format='%Y-%m-%dT%H:%M'), label = "Build Start Time")
    def __init__(self, *args, **kwargs):# for ensuring fields are not left empty
        super(EditBuild_building, self).__init__(*args, **kwargs)
        self.fields['buildDescrip'].required = True

    class Meta:
        model = Build
        fields = ['buildDescrip', 'buildStart','buildLength'...]

        labels = {
            'buildDescrip': ('Build Description'),
            'buildStart': ('Build Start Time'),
            ...
        }

        widgets = {'buildDescrip': forms.TextInput(attrs={'class': 'required'}),

and this view:

class BuildUpdateView_Building(LoginRequiredMixin,UpdateView):
    model = Build
    form_class = EditBuild_building
    template_name = 'build_edit_building.html'
    login_url = 'login'

    def get(self, request, *args, **kwargs):
        proceed = True
        try:
            instance = Build.objects.get(id = (self.kwargs['pk']))
        except:
            return HttpResponse("<h2 style = 'margin:2em;'>This build is no longer available it has been deleted, please please return to dashboard</h2>")
        if instance.buildActive == False:
            proceed = False
        if instance.deleted == True:
            proceed = False
        #all appears to be well, process request
        if proceed == True:
            form = self.form_class(instance=instance)
            timezone.activate(pytz.timezone(self.request.user.t_zone))
            customer = self.request.user.PSScustomer
            choices = [(item.id, (str(item.first_name) + ' ' + str(item.last_name)))  for item in CustomUser.objects.filter(isDevice=False, PSScustomer = customer)]
            choices.insert(0, ('', 'Unconfirmed'))
            form.fields['buildStrategyBy'].choices = choices
            form.fields['buildProgrammedBy'].choices = choices
            form.fields['operator'].choices = choices
            form.fields['powder'].queryset = Powder.objects.filter(PSScustomer = customer)
            context = {}
            context['buildID'] = self.kwargs['pk']
            context['build'] = Build.objects.get(id = (self.kwargs['pk']))
            return render(request, self.template_name, {'form': form, 'context': context})
        else:
            return HttpResponse("<h2 style = 'margin:2em;'>This build is no longer editable here, or has been deleted, please return to dashboard</h2>")


    def form_valid(self, form):
        timezone.activate(pytz.timezone(self.request.user.t_zone))
        proceed = True
        try:
            instance = Build.objects.get(id = (self.kwargs['pk']))
        except:
            return HttpResponse("<h2 style = 'margin:2em;'>This build is no longer available it has been deleted, please please return to dashboard</h2>")
        if instance.buildActive == False:
            proceed = False
        if instance.deleted == True:
            proceed = False
        #all appears to be well, process request
        if proceed == True:
            form.instance.editedBy = (self.request.user.first_name)+ " " +(self.request.user.last_name)
            form.instance.editedDate = timezone.now()
            print('edited date ' + str(form.instance.editedDate))
            form.instance.reviewed = True
            next = self.request.POST['next'] #grabs prev url from form template
            form.save()
            build = Build.objects.get(id = self.kwargs['pk'])
            if build.buildLength >0:
                anticipated_end = build.buildStart + (timedelta(hours = float(build.buildLength)))
                print(anticipated_end)
            else:
                anticipated_end = None
            build.anticipatedEnd = anticipated_end
            build.save()
            build_thres_updater(self.kwargs['pk'])#this is function above, it updates threshold alarm counts on the build
            return HttpResponseRedirect(next) #returns to this page after valid form submission
        else:
            return HttpResponse("<h2 style = 'margin:2em;'>This build is no longer available it has been deleted, please please return to dashboard</h2>")

When I open this form, the date and time of buildStart are displayed in my Winnipeg timezone, so Django converted from UTC to my timezone, perfect, but when I submit this form, the date in the DB has been altered from UTC to Winnipeg Time. Why is this?

I have tried to convert the submitted time to UTC in the form_valid function, but this does not seem like the right approach. What am I missing here? I simply want to store all times as UTC, but display them in the user's timezone in forms/pages.

EDIT

When I remove timezone.activate(pytz.timezone(self.request.user.t_zone)) from both get and form_valid, UTC is preserved in the DB which is great. But the time displayed on the form is now in the default TIME_ZONE in settings.py. I just need this to be in the user's timezone....

EDIT 2

I also tried to add:

{% load tz %}

{% timezone "America/Winnipeg" %}
    {{form}}
{% endtimezone %}

Which displayed the time on the form correctly, but then when the form submits, it will again remove 1 hour from the UTC time in the DB.

If I change template to:

{% load tz %}

{% timezone "Europe/Paris" %}
    {{form}}
{% endtimezone %}

The time will be displayed in local Paris time. When I submit the form, it will write this Paris time to the DB in UTC+2. So, in summary:

  • Time record was created was 11:40 Winnipeg time, which writes 16:40 UTC to database, perfect
  • I access the form template, and time is displayed as local Paris time, 6:40pm, which is also what I would expect.
  • I submit form without changing any fields.
  • Record has been updated with the time as 22:40, which is UTC + 6 hours.

What is happening here!?

Lutz Prechelt
  • 36,608
  • 11
  • 63
  • 88
MattG
  • 1,682
  • 5
  • 25
  • 45
  • What database do you use? PostgreSQL? If so PostgreSQL supports timezones and the datetime is saved _with_ the timezone in the database. – Abdul Aziz Barkat May 19 '21 at 17:04
  • Currently just de-bugging this locally using SQLite. The production version uses PostgreSQL. I didn't think that would make a difference, but I'm not surprised this can get even more complicated... – MattG May 19 '21 at 17:21
  • You should not be having any problem if you are using SQLite, since it doesn't support timezones Django should be storing the time in UTC in the database. See this part in the database settings documentation: https://docs.djangoproject.com/en/3.2/ref/settings/#time-zone (Note this is different from the `TIME_ZONE` setting and is part of the _database settings_) By any chance do you set this for your database setting? – Abdul Aziz Barkat May 19 '21 at 17:43
  • No i have not changed any database settings from the default. I am wondering if it is something to do with the way I am rendering/saving the form – MattG May 19 '21 at 17:48
  • @KevinChristopherHenry, I added 'activate()` to my question, and I was hoping that was the problem, but the effect of adding `activate()` stops the form from saving localtime (which I suppose is a step in the right direction) but it still removes one hour from the UTC time in the database. So the time starts as the correct UTC time in the DB, it is displayed as local Winnipeg time when I view the form, but then when I hit save, all it does is overwrite the UTC time with UTC-1 hour. I'm stumped... – MattG May 20 '21 at 15:15
  • @KevinChristopherHenry thank you!! I'm depressed at how much time I spent on this, thank you for taking the time to comment and help me. Middleware worked as you suggested. Post as an answer and I will mark it as answered. I suppose you need to call `activate()` before it hits the view like you said, I never would have guessed that. – MattG May 20 '21 at 17:20
  • No problem, happy to help. And, yes, we've all been there, oscillating between euphoria at fixing a bug and despair at the time wasted. :-) – Kevin Christopher Henry May 20 '21 at 17:55

1 Answers1

3

Put simply: your activate() call in form_valid() comes too late to affect the form field, so the incoming datetime gets interpreted in the default timezone—which in your case is America/Toronto—before being converted to UTC and saved to the database. Hence the apparent time shift.

The documentation doesn't really specify when you need to call activate(). Presumably, though, it has to come before Django converts the string value in the request to the aware Python datetime in the form dictionary (or vice versa when sending a datetime). By the time form_valid() is called, the dictionary of field values is already populated with the Python datetime object.

The most common place to put activate() is in middleware (as in this example from the documentation), since that ensures that it comes before any view processing. Alternatively, if using generic class-based views like you are, you could put it in dispatch().

Kevin Christopher Henry
  • 46,175
  • 7
  • 116
  • 102