0

I have a pandas dataframe containing sensor data with timestamp columns

    id                      time                            temp        humi      
                             min                       max  mean  std   mean   std
0  1.0 2023-03-29 14:24:12-03:00 2023-03-29 14:24:25-03:00  19.9  0.0  84.73  0.06

Where the columns ('time', 'min') and ('time', 'max') are timezone-aware, e.g. 2023-03-29 14:24:12-03:00

When I load this dataframe to a PostgreSQL database using the to_sql method like this:

dataframe.to_sql(table, engine, if_exists='append', index=False)

I noticed that the timezone information is not being preserved in the database. For instance, the value 2023-03-29 14:24:12-03:00 is being stored as 2023-03-29 17:24:12+00

The table in the database is created with the following SQL:

CREATE TABLE IF NOT EXISTS public.sensors

(
    "('id', '')" smallint,
    "('time', 'min')" timestamp with time zone,
    "('time', 'max')" timestamp with time zone,
    "('temp', 'mean')" float4,
    "('temp', 'std')" float4,
    "('humi', 'mean')" float4,
    "('humi', 'std')" float4
)

I suspect that the problem is related to the use of to_sql method rather than Postgres or the dataframe, but I'm not sure how to fix it.

How can I ensure that the timezone information is properly stored in the database?

I try:

dataframe.to_sql(table, engine, if_exists='append', index=False)

But: I noticed that the timezone information is not being preserved in the database. For instance, the value 2023-03-29 14:24:12-03:00 is being stored as 2023-03-29 17:24:12+00

leadvic
  • 21
  • 4
  • Postgres does not store the timezone, it stores UTC and converts to/from local timezone, unless `at time zone ...` specified. Take a good look at the times you have indicated; you will see `2023-03-29 14:24:12-03:00` and `2023-03-29 17:24:12+00` are actually the same time, the first local time, the second UTC. – Belayer Mar 29 '23 at 18:16

1 Answers1

0

As Belayer commented on my question he/she is right!

Postgres is storing the time into UTC, and both hours I posted originaly are the same hour one is local and the other is UTC, so there is actualy no problem at all, I was the one that didn't know that postgres doesn't store the timezone and didn't realize that everything was perfectly fine.

Thank you Belayer!

leadvic
  • 21
  • 4