Questions tagged [unix-timestamp]

The number of seconds between a particular date and the Unix Epoch on January 1st, 1970

POSIX definition

The POSIX.1 definition of Unix time is a number which is zero at the Unix epoch (1970-01-01T00:00:00Z), and increases by exactly 86 400 per day. Epoch and day ordinals are based on UTC.

The subtlety in this definition comes from the fact that days aren't exactly 86 400 seconds long. POSIX timestamps grow at 1Hz during the day, then end the day with small jumps to adjust for the duration of the UTC day.

For example, 2004-09-16T00:00:00Z, 12 677 days after the epoch, is represented by the Unix time number 12 677 × 86 400 = 1 095 292 800. The time interval between the epoch and 2004-09-16T00:00:00Z actually lasted 12 677 × 86 400 + 22 seconds.

This definition can be extended to represent instants before the epoch using negative numbers. 1957-10-04T00:00:00Z, 4 472 days before the epoch, is represented by the Unix time number -4 472 × 86 400 = -386 380 800. UTC is not defined for these instants, but universal time (any time standard that counts days from midnight at the reference meridian, such as the Julian Day) can be used, and the reduced accuracy is unlikely to matter.

POSIX provides for sub-second resolution with struct timespec, a fixed point format with a tv_nsec struct member for nanoseconds. This format is useful for system interfaces, but unsuitable for serialisation (naive range-checking could leave holes).

POSIX timestamps are ambiguous, discontinuous, and non-monotonic across leap seconds. When a leap second is inserted, a 1s range of Unix timestamps is repeated, first representing the leap second, then representing the first second of the next day (some implementations repeat the timestamp range immediately before the leap second instead). In the theoretical case of negative leap seconds, there would be 1s ranges of Unix time that do not represent any instant in time. The rest of the time, these Unix timestamps are continuous, unambiguous, and grow monotonically by 1s every second. The ambiguity isn't introduced by UTC, which measures time broken down in components and not as a single number.

System timestamps

On Unix systems, the CLOCK_REALTIME clock represents Unix time on a best-effort basis, based on hardware and network support. It may jump if the system clock is too far from reference time. Different clocks, representing different notions of system time, are exposed through clock_gettime. On Linux, CLOCK_MONOTONIC is monotonic and continuous (with no time elapsing when the system is suspended). It may speed up or slow down when adjtime is called, typically through NTP steering (clock slew). CLOCK_BOOTTIME is also monotonic, but will continue growing when the system is suspended. CLOCK_MONOTONIC_RAW is like CLOCK_MONOTONIC, but matches the speed of the hardware clock and ignores adjtime adjustments to clock speed. CLOCK_PROCESS_CPUTIME_ID and CLOCK_THREAD_CPUTIME_ID count CPU time consumed by the process and thread, respectively. Linux also provides coarse variants that may provide better performance.

Timestamps recorded by the kernel (for example, modification times on filesystem inodes) follow the CLOCK_REALTIME clock.

Assuming CLOCK_REALTIME follows POSIX time, getting unambiguous time (UTC or TAI) from the kernel is an unsolved problem; adjtimex might expose enough internal state but it is highly implementation dependent. Breaking from the standard brings its own tradeoffs.

Alternative timestamps

POSIX.1b-1993 switched the definition of Unix timestamps away from a simple second count from the epoch. This introduced a few drawbacks: timestamps do not represent instants unambiguously, and Unix time is discontinuous and jumps backwards. The jumps are rare, thus hard to test for. Bugs can be subtle and are most likely to be discovered in production, after developers have moved on.

TAI-10 (TAI minus ten seconds) hits midnight at the Unix epoch. TAI is an ideal timestamp format; it grows perfectly linearly at 1/s.

Redefining CLOCK_REALTIME to follow an alternative to POSIX time is doable, but not advisable unless you control the system entirely. Setting the clock to TAI-10, applications that use localtime will still work, with /etc/localtime pointing to the Olson "right" timezones, but many applications expect to compute UTC days from timestamp / 86_400. Redefining CLOCK_REALTIME indirectly, through a tweaked NTP server, is more feasible; many applications will survive slightly varying clock speeds. This is the leap smear technique, which silently replaces UTC with UTC-SLS (smoothed leap seconds).

Other proposals aim to extend the clock_gettime interface instead of replacing the default clock. One is CLOCK_UTC, which encodes the leap second by growing tv_nsec beyond the [0, NSEC_PER_SEC] range, removing the ambiguity of CLOCK_REALTIME. The other is CLOCK_TAI, which simply encodes TAI.

time_t binary representation

ABIs where time_t is 32 bits are unable to represent times beyond January 2038; their timestamps will jump into the early twentieth century instead. This will prove a problem for some embedded systems that are being deployed now. clock_gettime/timespec_get, 64 bit integers, or other fixed-point formats like TAI64 should be used instead.

Use in protocols and serialisation

Unix timestamps are sometimes persisted, for example through serialisation or archive formats. Most filesystems use them for inode metadata. Internet protocols and formats systematically prefer RFC 3339/ISO 8601 datetimes. The SQL timestamp type is a Unix timestamp; when (fixed-offset) timezones are used, naive datetimes are translated to UTC at the storage boundary. TAI64 has been proposed to address the interoperability shortcomings of POSIX timestamps (and of time_t). When the extra compactness of integers isn't required, RFC 3339 UTC datetimes are self-describing and provide better portability, readability and widespread support.

2326 questions
0
votes
0 answers

how do i get the query which executed more time from a log file

I need the unix command to fetch out the query that got executed more than 1 sec from a log file using sed/awk. $11 is time stamp. Pl suggest awk script to print all the query that executed more than 1 sec the log file pattern looks as below, "PUT…
0
votes
1 answer

Python: convert timestamp to datetime

I have the following dataframe: df=pd.DataFrame(index=[0]) df['timestamp'] = 1642104862000 df: timestamp 0 1642104862000 I am applying the following code: df['timestamp'] = pd.to_datetime(df['timestamp'], infer_datetime_format=True) But the…
MathMan 99
  • 665
  • 1
  • 7
  • 19
0
votes
1 answer

from_unixtime gave me an awkward value

Hi I used from_unixtime to convert this value 1632837232439 and I got 53712-07-21 01:53:59 is this right? I can't make sense of this, I used df = df.select(from_unixtime(df_sixty60['createdOn']).alias("date_key")) Thanks for you help even if you…
user3476582
  • 75
  • 1
  • 10
0
votes
1 answer

How can I get 7 days before last 7 days using UNIX_TIMESTAMP?

I use the code below to return results from a mySQL field called registered_at so I can get the users that registered on the site the last 7 days and it works fine: andWhere('registered_at > UNIX_TIMESTAMP(NOW() - INTERVAL 7 DAY)') My problem is…
Designer
  • 875
  • 7
  • 26
0
votes
1 answer

How can I display Unix day (not seconds) in date formatting?

The date is approaching (in fact, 2022-01-08), when Unix time turns 19000-days old. Currently, there is a way (namely, %s) to display Unix second in dates, but there seem to be no way to display Unix day in date formatting. How does one display Unix…
Mindey I.
  • 29
  • 1
  • 6
0
votes
1 answer

How to get yesterday's date in Epoch time? [python]

I can easily get the yesterday date like this: from datetime import datetime, timedelta yesterday = datetime.now() - timedelta(1) x= datetime.strftime(yesterday, '%Y-%m-%d') How can I do the same for Unix Epoch time? I tried this: >>>…
botafogo
  • 189
  • 7
0
votes
2 answers

Unix timestamp with only month and year

I have an array of different dates but they only include the month and the year. Is there a way to get the unix timestamp just from those two things? I have to iterate over the array to get all the timestamps. I have the solution but the solution is…
Sarah
  • 105
  • 1
  • 7
0
votes
1 answer

Convert a timestamp to a DateTime field?

I've got a DateTimeField(auto_now_add=True) in my model. However, I wish to update this field, and the format I'm receiving from my API is a UNIX timestamp. Can I somehow convert the format I receive from my API to the correct one? (eg 1640206232 to…
erikvm
  • 858
  • 10
  • 30
0
votes
1 answer

Compare client timezone to CET timezone

My website is using the timezone from my server, which is CET. So the unix-timestamp that is always parsed using new Date(timestamp * 1000) to the timezone of the server, and not the client. However, I know how to make the client view the time of…
0
votes
0 answers

Format and sort a complex Python list

I’m new to Python and have been struggling to find any resources on how to perform the below. I’m confident that my data set is a list as wrapping the variable which contains it with print(type(variable)) returns The list is formatted…
Usul
  • 36
  • 3
0
votes
1 answer

In react, I get too similar unix time stamp in

I have two Textfield which set the start date and due date. here is the code. const startDateChange = (e) => { setStartDate(Math.floor(new Date().getTime(e.target.value) / 1000)); console.log(startDate); }; const dueDateChange = (e) => { …
chiko
  • 11
  • 4
0
votes
1 answer

convert datetime to timestamp

First, what does this time format mean '2020-01-01T01:39:40.000000Z', my guess is that, it means timezone, I stand to be corrected. I want to convert it to Unix timestamp in python This works fine from datetime import…
se7en
  • 671
  • 4
  • 18
0
votes
1 answer

Apache Spark How to convert a datetime from Australia/Melbourne time to UTC?

How do I convert a datetime string like 21/10/2021 15:15:28 in Australia/Melbourne time to UTC in Apache Spark in Scala?
0
votes
4 answers

How do I process this format of timestamp?

How can I process an output like "TimeStamp: 1635251181000" into a readable format in Python. I tried datetime.fromtimestamp() but I got an error like OSError: [Errno 22] Invalid argument
zertalion
  • 3
  • 1
0
votes
1 answer

Convert String column which has AM/PM to Timestamp in Impala

I have a column which has values in string type like below: 31-Oct-2016 12:00 AM 31-May-2015 12:00 PM I want to convert the above column values to timestamp in IMPALA. Tried with cast, to_timestamp and other ways , but it is either showing syntax…
Shanti
  • 13
  • 4