Questions tagged [unix-timestamp]

The number of seconds between a particular date and the Unix Epoch on January 1st, 1970

POSIX definition

The POSIX.1 definition of Unix time is a number which is zero at the Unix epoch (1970-01-01T00:00:00Z), and increases by exactly 86 400 per day. Epoch and day ordinals are based on UTC.

The subtlety in this definition comes from the fact that days aren't exactly 86 400 seconds long. POSIX timestamps grow at 1Hz during the day, then end the day with small jumps to adjust for the duration of the UTC day.

For example, 2004-09-16T00:00:00Z, 12 677 days after the epoch, is represented by the Unix time number 12 677 × 86 400 = 1 095 292 800. The time interval between the epoch and 2004-09-16T00:00:00Z actually lasted 12 677 × 86 400 + 22 seconds.

This definition can be extended to represent instants before the epoch using negative numbers. 1957-10-04T00:00:00Z, 4 472 days before the epoch, is represented by the Unix time number -4 472 × 86 400 = -386 380 800. UTC is not defined for these instants, but universal time (any time standard that counts days from midnight at the reference meridian, such as the Julian Day) can be used, and the reduced accuracy is unlikely to matter.

POSIX provides for sub-second resolution with struct timespec, a fixed point format with a tv_nsec struct member for nanoseconds. This format is useful for system interfaces, but unsuitable for serialisation (naive range-checking could leave holes).

POSIX timestamps are ambiguous, discontinuous, and non-monotonic across leap seconds. When a leap second is inserted, a 1s range of Unix timestamps is repeated, first representing the leap second, then representing the first second of the next day (some implementations repeat the timestamp range immediately before the leap second instead). In the theoretical case of negative leap seconds, there would be 1s ranges of Unix time that do not represent any instant in time. The rest of the time, these Unix timestamps are continuous, unambiguous, and grow monotonically by 1s every second. The ambiguity isn't introduced by UTC, which measures time broken down in components and not as a single number.

System timestamps

On Unix systems, the CLOCK_REALTIME clock represents Unix time on a best-effort basis, based on hardware and network support. It may jump if the system clock is too far from reference time. Different clocks, representing different notions of system time, are exposed through clock_gettime. On Linux, CLOCK_MONOTONIC is monotonic and continuous (with no time elapsing when the system is suspended). It may speed up or slow down when adjtime is called, typically through NTP steering (clock slew). CLOCK_BOOTTIME is also monotonic, but will continue growing when the system is suspended. CLOCK_MONOTONIC_RAW is like CLOCK_MONOTONIC, but matches the speed of the hardware clock and ignores adjtime adjustments to clock speed. CLOCK_PROCESS_CPUTIME_ID and CLOCK_THREAD_CPUTIME_ID count CPU time consumed by the process and thread, respectively. Linux also provides coarse variants that may provide better performance.

Timestamps recorded by the kernel (for example, modification times on filesystem inodes) follow the CLOCK_REALTIME clock.

Assuming CLOCK_REALTIME follows POSIX time, getting unambiguous time (UTC or TAI) from the kernel is an unsolved problem; adjtimex might expose enough internal state but it is highly implementation dependent. Breaking from the standard brings its own tradeoffs.

Alternative timestamps

POSIX.1b-1993 switched the definition of Unix timestamps away from a simple second count from the epoch. This introduced a few drawbacks: timestamps do not represent instants unambiguously, and Unix time is discontinuous and jumps backwards. The jumps are rare, thus hard to test for. Bugs can be subtle and are most likely to be discovered in production, after developers have moved on.

TAI-10 (TAI minus ten seconds) hits midnight at the Unix epoch. TAI is an ideal timestamp format; it grows perfectly linearly at 1/s.

Redefining CLOCK_REALTIME to follow an alternative to POSIX time is doable, but not advisable unless you control the system entirely. Setting the clock to TAI-10, applications that use localtime will still work, with /etc/localtime pointing to the Olson "right" timezones, but many applications expect to compute UTC days from timestamp / 86_400. Redefining CLOCK_REALTIME indirectly, through a tweaked NTP server, is more feasible; many applications will survive slightly varying clock speeds. This is the leap smear technique, which silently replaces UTC with UTC-SLS (smoothed leap seconds).

Other proposals aim to extend the clock_gettime interface instead of replacing the default clock. One is CLOCK_UTC, which encodes the leap second by growing tv_nsec beyond the [0, NSEC_PER_SEC] range, removing the ambiguity of CLOCK_REALTIME. The other is CLOCK_TAI, which simply encodes TAI.

time_t binary representation

ABIs where time_t is 32 bits are unable to represent times beyond January 2038; their timestamps will jump into the early twentieth century instead. This will prove a problem for some embedded systems that are being deployed now. clock_gettime/timespec_get, 64 bit integers, or other fixed-point formats like TAI64 should be used instead.

Use in protocols and serialisation

Unix timestamps are sometimes persisted, for example through serialisation or archive formats. Most filesystems use them for inode metadata. Internet protocols and formats systematically prefer RFC 3339/ISO 8601 datetimes. The SQL timestamp type is a Unix timestamp; when (fixed-offset) timezones are used, naive datetimes are translated to UTC at the storage boundary. TAI64 has been proposed to address the interoperability shortcomings of POSIX timestamps (and of time_t). When the extra compactness of integers isn't required, RFC 3339 UTC datetimes are self-describing and provide better portability, readability and widespread support.

2326 questions
0
votes
1 answer

Pyspark: to_date and unix_timestamp return null for some records for others valid values

The above command is giving null for few records. Can anyone please tell what might be the reason for this. I have tried using to_date and unix_timestamp function but both are giving the same result ( null for few…
Divyam
  • 31
  • 2
0
votes
2 answers

i want to print all the unique dates in each iteration from customer_data

The aim of below code is to find the number of hours a customer id spends online on a #particular date. I am unable to get the online hours for all of the #dates(i.e. 01/06/2017 to 21/06/2017). I have attached the images of o/p of current code, csv…
0
votes
0 answers

I am trying to get code to convert epoch time (Ex. 1621690499.4047496) to unix hex timestamp (Ex.60a90883XXXXXXXX) using python

I tried below line of code, to convert but I need complete conversion (8 bytes of hex value Ex. 60a79ed81c256a2b, which gives accurate time (Nano seconds)) print (hex(int(time.time()))) 0x60a90883 (4 bytes) print (time.time()) 1621690499.4047496
0
votes
1 answer

calculating date differences with datetime python

Example code: from datetime import datetime, date from dateutil.relativedelta import relativedelta import time timestamp = 1620013967000 cunix = int(time.time()) ounix = timestamp / 1000 outc_time = datetime.utcfromtimestamp(ounix) cutc_time =…
humid
  • 13
  • 5
0
votes
0 answers

Invalid Integer (UNIX Timestamp) Returned By Stripe

My app is sending a POST request to stripe's API for a usage update. The Body is as follows: quantity=2×tamp=1614916656 However, the issue is Stripe returns the following error: parameter_invalid_integer - timestamp I can't figure out why this…
Ben Smith
  • 43
  • 1
  • 5
0
votes
1 answer

Pyspark converting string to UTC timestamp [Getting null]

I am new to pyspark and Spark SQL. I have a dataframe with one column having date time values in string which I need to convert/cast to timestamp. Dataframe Format: +--------------------+------------------------------+ | value| …
phoenix007
  • 71
  • 1
  • 6
0
votes
0 answers

Convert unix date to utc

I have table with list of date, but date in unix format. I need to convert unix time to utc. The function FROM_UNIXTIME is not clear for me. Cause when i try to use it, i have error: "FROM_UNIXTIME" is not recognized built-in function name I create…
0
votes
1 answer

MySQL epoch time conversion with microseconds

I have time column stored with epoch timestamp values. I am trying to convert this in readable format as following: YYYYMMDD HH:MM:SS.uuuuuu Where last .uuuuuu are microseconds of time stamps. I can display YYYYMMDD HH:MM:SS with this: SELECT…
Mr. Wolf
  • 29
  • 4
0
votes
1 answer

Redis How to set "key name EXAT how_to_specify_format_unix_timestamp " in redis-cli

I like to know how to specify unix-timestamp in redis. for command like SET name xyz EXAT 1617875638 //is wrong format cli says On my terminal timestamp is something like this 1617875638. this is exception causing. wrong format.
user786
  • 3,902
  • 4
  • 40
  • 72
0
votes
1 answer

How to convert UNIX timestamp to date using chart.js?

I am using chart.js to display "price" data over the past 30 days. I am using a price API to do so. However the .time parameter returns the time as a UNIX timestamp. I would like the time to be displayed as a readable date. To get the time and…
seabass118
  • 21
  • 3
0
votes
1 answer

Batch renaming wave audio files with their time stamp info

I have approximately 60 audio files with a partial time stamp in their names. These names were created automatically by the recording system and are shown as below 201106_0099.wav 201106_0100.wav 201106_0101.wav . . . 201106_0163.wav The files are…
0
votes
4 answers

Change unix time to arrange the data in diurnal and hourly pattern

I have an input file with 9 columns where 6th and 7th columns are for start and end unix time. (The overall span of the timeline is 1290895200 to 1291154399, 3 days) I am looking for a perl script which can take in the unix time to specify the hour…
user744121
  • 467
  • 2
  • 7
  • 17
0
votes
2 answers

Unix TimeStamp value gets Corrupted

I'm trying to fetch Unix Timestamp value using below snippet but sometimes the value get corrupted like 11030 or 810 something like random number we don,t get actual date public static int getUnixTimeStamp() { Date date = new Date(); return…
0
votes
2 answers

Which sort by value to give in order to indicate LAST in a unix timestamps array?

I'm using lodash's sortBy to sort an array of objects based on their timestamps (Unix time). However, I want to set one of the objects to always be last (based on a certain condition). If I wanted it to be first, I would give it a value of 0. let…
amiregelz
  • 1,833
  • 7
  • 25
  • 46
0
votes
1 answer

How to convert timestamp to unix timestamp in Laravel?

Hello great people of SO I hope you all have a great day! I have an issue with my project timestamps I have 2 Models, News and Comments During working on my project, I tried to replicate 'real world data' So in my Seeder (NewsSeeder), use…
Gwein
  • 109
  • 3
  • 14