Generating monotonically increasing integers (max 64bit)

Question

As part of a new project we need a service which can generate monotonically increasing integers. Requirements for the service are:

1. service does not need to produce contiguous integer as long as it produces monotonically increasing integers it should be fine
2. It should produce 64 bit integer
3. the service should be highly available
4. services should be resilient to failure (or restarts)

I was planing to use redis (INCR) as a back-end store with replication enable but the issue is if the master redis fails then it may be possible that the service may loose some integers (think that before even the update flows down to slave, master crashes then the service may loose some integers i.e. service can generate a number which is less than already generated value).

Can someone help me in design such a system/service.

Answer 1

Your requirements 1 and 3 are somewhat contradictory. The requirement for monotonic increase means a single service has to supply the values, and that is not going to be highly available.

For instance, an Oracle database sequence will do the trick if you have a single database server. But once you go to a cluster of machines, batches of integers are provisioned into each cluster member and they are allocated sequentially as the member is hit. They are guaranteed to be unique, but not monotonically increasing.

Possibly you could build a system based on time (to nanosecond accuracy if possible), with logic to detect and adjust duplicates. This won't be highly available...

Personally I would revisit the business requirement that drives this need. Is it real? For instance, in the old days order numbers were contiguous if all sales people used the same order book. But once travelling sales people became the norm, this was difficult to achieve without having a single clerk that issued order numbers over the phone. High availability? Not really.

Answer 2

You are asking for a clock.

Relatively speaking it is impossible

See this wikipedia article for an excellent tutorial.

According to the special theory of relativity, it is impossible to say in an absolute sense that two distinct events occur at the same time if those events are separated in space. For example, a car crash in London and another in New York, which appear to happen at the same time to an observer on the earth, will appear to have occurred at slightly different times to an observer on an airplane flying between London and New York. The question of whether the events are simultaneous is relative: in the stationary earth reference frame the two accidents may happen at the same time but in other frames (in a different state of motion relative to the events) the crash in London may occur first, and in still other frames the New York crash may occur first. However, if the two events could be causally connected (i.e. the time between event A and event B is greater than the distance between them divided by the speed of light), the order is preserved (i.e., "event A precedes event B") in all frames of reference.

Practically speaking it is trivial

Most likely, an analysis of your business requirements would reveal that,

• All participants are earth bound (sharing the same frame of reference), Or
• The events of interest are causally connected.

in which case a simple NTP will suffice. The government already provides this service. http://tf.nist.gov/tf-cgi/servers.cgi