python3 fromtimestamp generates OSError

Question

We encountered an interesting mtime problem we encountered in the field. I'm planning to file a bug report, but wanted to get outside perspective.

A file stored on a windows server had the last-modified date '1/1/4501' (that's the year 4501). os.path.getmtime() returns a valid timestamp, but when we try to pass this back into datetime.datetime.fromtimestamp() we get an OSError.

There are several discussions on SO that talk around this issue, for example: Python fromtimestamp OSError. However, in most of the cases we could find, the OP were dealing with manufactured dates that clearly were outside the platform's supported epoch range. Generating an OSError when the date exceeds the epoch support on Windows is consistent with the python docs.

In our case, the date is clearly supported by Windows as evidenced by it's storage in the filesystem. Further, we can reproduce the situation using the cygwin touch utility.

>>> import os, datetime
>>> os.system('touch -d "4501-01-01" file.txt')
>>> t = os.path.getmtime('file.txt')
>>> datetime.datetime.fromtimestamp(t)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 22] Invalid argument

What's interesting is we can manually convert it with reference to the epoch

>>> datetime.datetime(1970, 1, 1) + datetime.timedelta(seconds=t)
datetime.datetime(4501, 1, 1, 5, 0)

We used Windows 10-Pro for our tests running python 3.8.1.

This worked for me on Ubuntu. `t` was `79870690800.0`. I assume you are using a 64 bit python? — tdelaney, May 25 '20 at 16:42

score 0 · Accepted Answer · answered May 27 '20 at 09:30

Ok, this is not a bug, it's a documented limitation of the platform's localtime()/gmtime(). While the datetime class itself can handle arbitrary dates, any references to timestamp requires a pass through the platform's localtime system. In my case it's Windows 10, 64bit.

According to docs: https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/localtime-s-localtime32-s-localtime64-s?view=vs-2019

_localtime64_s, which uses the __time64_t structure, allows dates to be expressed up through 23:59:59, January 18, 3001, coordinated universal time (UTC)

Passing in a value greater than the year 3001 causes localtime64_s to return EINVAL. According to python docs:

This may raise OverflowError, if the timestamp is out of the range of values supported by the platform C localtime() function, and OSError on localtime() failure.

The later is what we are seeing.

python3 fromtimestamp generates OSError

1 Answers1