Questions tagged [unix-timestamp]

The number of seconds between a particular date and the Unix Epoch on January 1st, 1970

POSIX definition

The POSIX.1 definition of Unix time is a number which is zero at the Unix epoch (1970-01-01T00:00:00Z), and increases by exactly 86 400 per day. Epoch and day ordinals are based on UTC.

The subtlety in this definition comes from the fact that days aren't exactly 86 400 seconds long. POSIX timestamps grow at 1Hz during the day, then end the day with small jumps to adjust for the duration of the UTC day.

For example, 2004-09-16T00:00:00Z, 12 677 days after the epoch, is represented by the Unix time number 12 677 × 86 400 = 1 095 292 800. The time interval between the epoch and 2004-09-16T00:00:00Z actually lasted 12 677 × 86 400 + 22 seconds.

This definition can be extended to represent instants before the epoch using negative numbers. 1957-10-04T00:00:00Z, 4 472 days before the epoch, is represented by the Unix time number -4 472 × 86 400 = -386 380 800. UTC is not defined for these instants, but universal time (any time standard that counts days from midnight at the reference meridian, such as the Julian Day) can be used, and the reduced accuracy is unlikely to matter.

POSIX provides for sub-second resolution with struct timespec, a fixed point format with a tv_nsec struct member for nanoseconds. This format is useful for system interfaces, but unsuitable for serialisation (naive range-checking could leave holes).

POSIX timestamps are ambiguous, discontinuous, and non-monotonic across leap seconds. When a leap second is inserted, a 1s range of Unix timestamps is repeated, first representing the leap second, then representing the first second of the next day (some implementations repeat the timestamp range immediately before the leap second instead). In the theoretical case of negative leap seconds, there would be 1s ranges of Unix time that do not represent any instant in time. The rest of the time, these Unix timestamps are continuous, unambiguous, and grow monotonically by 1s every second. The ambiguity isn't introduced by UTC, which measures time broken down in components and not as a single number.

System timestamps

On Unix systems, the CLOCK_REALTIME clock represents Unix time on a best-effort basis, based on hardware and network support. It may jump if the system clock is too far from reference time. Different clocks, representing different notions of system time, are exposed through clock_gettime. On Linux, CLOCK_MONOTONIC is monotonic and continuous (with no time elapsing when the system is suspended). It may speed up or slow down when adjtime is called, typically through NTP steering (clock slew). CLOCK_BOOTTIME is also monotonic, but will continue growing when the system is suspended. CLOCK_MONOTONIC_RAW is like CLOCK_MONOTONIC, but matches the speed of the hardware clock and ignores adjtime adjustments to clock speed. CLOCK_PROCESS_CPUTIME_ID and CLOCK_THREAD_CPUTIME_ID count CPU time consumed by the process and thread, respectively. Linux also provides coarse variants that may provide better performance.

Timestamps recorded by the kernel (for example, modification times on filesystem inodes) follow the CLOCK_REALTIME clock.

Assuming CLOCK_REALTIME follows POSIX time, getting unambiguous time (UTC or TAI) from the kernel is an unsolved problem; adjtimex might expose enough internal state but it is highly implementation dependent. Breaking from the standard brings its own tradeoffs.

Alternative timestamps

POSIX.1b-1993 switched the definition of Unix timestamps away from a simple second count from the epoch. This introduced a few drawbacks: timestamps do not represent instants unambiguously, and Unix time is discontinuous and jumps backwards. The jumps are rare, thus hard to test for. Bugs can be subtle and are most likely to be discovered in production, after developers have moved on.

TAI-10 (TAI minus ten seconds) hits midnight at the Unix epoch. TAI is an ideal timestamp format; it grows perfectly linearly at 1/s.

Redefining CLOCK_REALTIME to follow an alternative to POSIX time is doable, but not advisable unless you control the system entirely. Setting the clock to TAI-10, applications that use localtime will still work, with /etc/localtime pointing to the Olson "right" timezones, but many applications expect to compute UTC days from timestamp / 86_400. Redefining CLOCK_REALTIME indirectly, through a tweaked NTP server, is more feasible; many applications will survive slightly varying clock speeds. This is the leap smear technique, which silently replaces UTC with UTC-SLS (smoothed leap seconds).

Other proposals aim to extend the clock_gettime interface instead of replacing the default clock. One is CLOCK_UTC, which encodes the leap second by growing tv_nsec beyond the [0, NSEC_PER_SEC] range, removing the ambiguity of CLOCK_REALTIME. The other is CLOCK_TAI, which simply encodes TAI.

time_t binary representation

ABIs where time_t is 32 bits are unable to represent times beyond January 2038; their timestamps will jump into the early twentieth century instead. This will prove a problem for some embedded systems that are being deployed now. clock_gettime/timespec_get, 64 bit integers, or other fixed-point formats like TAI64 should be used instead.

Use in protocols and serialisation

Unix timestamps are sometimes persisted, for example through serialisation or archive formats. Most filesystems use them for inode metadata. Internet protocols and formats systematically prefer RFC 3339/ISO 8601 datetimes. The SQL timestamp type is a Unix timestamp; when (fixed-offset) timezones are used, naive datetimes are translated to UTC at the storage boundary. TAI64 has been proposed to address the interoperability shortcomings of POSIX timestamps (and of time_t). When the extra compactness of integers isn't required, RFC 3339 UTC datetimes are self-describing and provide better portability, readability and widespread support.

2326 questions
0
votes
1 answer

24h clock with from_unixtime

I need to transform a dataframe with a column of timestamps in Unixtime/LongType-Format to actual TimestampType. According to epochconverter.com: 1646732321 = 8. März 2022 10:38:41 GMT+1 1646768324 = 8. March 2022 20:38:44 GMT+1 However, when I…
Cribber
  • 2,513
  • 2
  • 21
  • 60
0
votes
0 answers

Bash script to rename files with current timestamp

master_sheet_2022_03_25_1141.csv master_sheet_2021_03_30_1034.csv master_sheet_2021_03_31_1857.csv master_sheet_2021_03_31_1930.csv master_sheet_2021_03_31_2037.csv master_sheet_2021_03_31_2109.csv For each file in above directory, rename the…
0
votes
1 answer

Unix epoch to date String – Slightly wrong month day

I am trying to format SystemTime as a String in Rust without using external crates, but I am facing a bizarre issue regarding the "day" field. Can anyone explain me why the line preceded by the "FIXME" comment yields slightly wrong month days as the…
user11874489
0
votes
0 answers

How can I convert from a date to unix in Safari? My current solution only works in chrome

The dateToUnix() function below is meant to return a unix timestamp given a date as the parameter (in the form of "YYYY.MM.DD"). function dateToUnix(date) { return Math.floor(new Date(date).getTime() /…
jgpixel
  • 103
  • 2
  • 6
0
votes
2 answers

Getting a date in properties file in string format

I want to get a date from a key dynamically which will be used by java program to perform some tasks. I have to get values from property file to java. cannot do vice versa So basically the value for this key, job.date=2022-03-23 i can get through…
Namy
  • 1
  • 1
0
votes
0 answers

Problems in UTC time (unix timestamp) convert to human readable format then merge the other data

I had used the API to get a data and then asign to "sol_tvl_df" class. from defillama import DefiLlama import requests import pandas import ccxt from math import sqrt import time import json from typing import List, Union import numpy _binance =…
Steven
  • 1
  • 1
0
votes
1 answer

Is it safe to use time.Now().Unix(), UnixNano(), UnixMilli() etc? What about integer overflow?

I read here https://pkg.go.dev/time#Unix that Unix() returns the local Time corresponding to the given Unix time, sec seconds and nsec nanoseconds since January 1, 1970 UTC And you can also get the milliseconds since, or nanoseconds since 1970 using…
Shisui
  • 1,051
  • 1
  • 8
  • 23
0
votes
1 answer

Firebase query data of last 24 hour return same result

let before1day = new Date().getTime() - (24 * 3600 * 1000 ); const result = await this.firestore.collection(collection).doc(id).collection(subcollection). orderBy('time').startAt(before1day).get(); const list = []; result.forEach(doc…
0
votes
1 answer

how to add hours in python? in the format below:

how to add hours in python? in the format below: hour1= '20:00' hour2= '25:00' sum= hour1 + hour2 so the correct print and formatted as I want it would be: result of the sum 45:00:00
0
votes
0 answers

Mysql WHERE ... ORDER BY taking too long in large database

I have a large database. There are 22 million lines for now, but there will be billions of lines in the future. I have attached example lines where timestamp is in milliseconds. What I want to do is get the last line every second. SELECT * FROM …
0
votes
1 answer

Unix time oddity, or something else?

I'm writing an app in C# that calls methods on a 3rd party web service (but this question is language independent). One of the methods I have to call sets a "start time." The method expects the date and time to be passed as a long, in Unix time…
Jimmy D
  • 5,282
  • 16
  • 54
  • 70
0
votes
1 answer

Convert a string into date in AIX unix

I need to extract the date part from a file in unix and add +1 day to the date and rename the file with the new date in it. for ex: sample file name: sample_file_name_01_31_2022_14_01_45_loadid.csv I tried to extract the timestamp using substr which…
nag
  • 75
  • 5
0
votes
0 answers

Separating List of unix times into separate nested list in C#

I have a list of unix times that looks like this: List unixTimes = new List() { 1635123603, // -> Monday, October 25, 2021 1:00:03 AM 1635123608, // -> Monday, October 25, 2021 1:00:08 AM …
pewocis495
  • 117
  • 1
  • 11
0
votes
1 answer

Why do I need to divide the timestamp by 1 billion?

I'm using this public Postgres DB of NEAR protocol: https://github.com/near/near-indexer-for-explorer#shared-public-access There is a field called included_in_block_timestamp whose "data type" = "numeric", and "length/precision" = 20. This code…
Ryan
  • 22,332
  • 31
  • 176
  • 357
0
votes
1 answer

How to get IST in Unix if my current timezone is UTC. Also I need to get the time format to be dd/mm/yyyy hr:min:sec?

I have just started learning about the date command in Unix. I have typed the below code to get the IST. But I don't know how to convert it to a specific format. TZ = IST-5:30 date This gives the result as Thu Feb 10:31:51 IST 2022. But I want the…