4

I am undertaking the algorithms course on Coursera, there is a section where the author mentions the following

the running time of weighted quick union with path compression is going be linear in the real world and actually could be improved to even a more interesting function called the Ackermann function, which is even more slowly growing than lg. And another point about this is it seems that this is so close to being linear that is time proportional to N instead of time proportional to N times the slowly growing function in N. Is there a simple algorithm that is linear? And people, looked for a long time for that, and actually it works out to be the case that we can prove that there is no such algorithm. (emphasis added)

(You can find the entire transcript here)

In all other sources including Wikipedia "linear" is used when time increases proportionally with the input size, and in weighted quick-union with path compression this is certainly not the case.

What exactly is meant by "linear in the real world" here?

templatetypedef
  • 362,284
  • 104
  • 897
  • 1,065
Sudarshan
  • 8,574
  • 11
  • 52
  • 74
  • "Linear in the real world" is an observation that the practical motivation for big-O notation has broken down here, not a formal statement about the complexity of union-find. – David Eisenstat Aug 08 '14 at 03:41

2 Answers2

3

The runtime of m operations on a union-find data structure with path compression and union-by-rank is O(mα(m)), where α(m) is the inverse Ackermann function. This function is so slowly-growing that you cannot express an input to it for which the output is 6 in scientific notation. In other words, for any possible value of m that fits into the universe (or even that has size around 2num atoms in the universe), we have that α(m) ≤ 5. Therefore, for any "reasonable" input the cost of m operations will be O(m · 6) = O(m), which is linear.

Of course, the runtime isn't linear because α(m) does indeed grow, just very, very slowly. However, it's usually fine to approximate the runtime as O(m) because there's no possible way you'd ever notice the runtime of the function deviating from a simple linear function.

Hope this helps!

templatetypedef
  • 362,284
  • 104
  • 897
  • 1,065
2

Here are some chunks from the transcript:

And what was proved by Hopcroft Ulman and Tarjan was that if you have N objects, any sequence of M union and find operations will touch the array at most a c (N + M lg star N) times. And now, lg N is kind of a funny function....

And another point about this is it< /i> seems that this is so close to being linear that is t ime proportional to N instead of time proportional to N times the slowly growing function in N.

(end quote)

You are pointing out that the cost of an individual operation grows very slowly with the number of objects, but they are looking at how the total cost of a number of operations grows with the number of objects involved so N times a per-operation cost that grows only very slowly with N is still just over linear in N.

mcdowella
  • 19,301
  • 2
  • 19
  • 25