2

There are cases that a brute force approach on a problem has a complexity that is not good enough performance-wise.

Let's take for example e.g. Theta(n^2).

Using a recursive approach it can be improved to Theta(nlogn).

Obviously, asymptotically one would opt to use the recursive approach since for larger and larger input N the order of growth is lower i.e. better performance.

My question is the following:

If asymptotically as N becomes larger and larger the recursive(i.e. divide and conquer approach) performs better, isn't it unrealistic to ignore that as we recurse on huge inputs of N we could eventually run out of stack?
As a result for huge input we actually never get a result.

So how can we be sure that we choose the best algorithm for a specific problem if we ignore these details?

Cratylus
  • 52,998
  • 69
  • 209
  • 339
  • related: [Worse is better. Is there an example?](http://stackoverflow.com/q/471544) – jfs Oct 28 '11 at 13:01

2 Answers2

4

It is possible to implement recursive algorithms without using the hardware stack. If the possibility of a stack overflow is a serious concern, you can implement your own stack or simply limit the depth of the recursion.

However, if the algorithm is dividing a set of data (of size n) recursively, then the number of recursions will be proportional to log n. This means that to exhaust a stack of size s you need data of size on the order of 2^s, which grows extremely quickly with s. People tend to not appreciate how quickly the exponential function is growing. My point is that with this type of algorithm (splitting a set of data in each recursive step) it is quite unlikely that the input data can be large enough to need so many recursions as to exhaust the stack.

EDIT: But then I'm not a professional programmer, so I'm lacking on the practical experience front.

Szabolcs
  • 24,728
  • 9
  • 85
  • 174
  • 1
    eg: on a 64 bit architecture the addressable space limits this to s=64, so unless you store serious amount of information on the stack for each step, it won't exhaust the stack. – Karoly Horvath Oct 28 '11 at 13:07
  • Not sure what you mean "without using hardware stack".Recursive functions, as the keep calling itself, use the stack to store the current function frame.@yi_H:Now you are taking into account the hardware.But RAM model ignores such things... – Cratylus Oct 28 '11 at 13:10
  • @user384706 I meant that you can implement (mathematically) *the same algorithm* without using a function that calls itself in some particular language. Re your comment to yi: in practice you simply can't have enough ram to exhaust a 64-bit address space. Apart from practical considerations, my point was that for any potential future architecture (suppose we'll have 1000 times the memory we have today), the stack space is likely to grow linearly with the usable memory while the *needed* stack space (for this algo) grown logarithmically (much slower). – Szabolcs Oct 28 '11 at 13:56
  • @user384706 For the *type of algorithm you described* you'll simply never run out of stack space in practice. For other types of recursive algorithms of course it is important to consider the possibility of a stack overflow ... – Szabolcs Oct 28 '11 at 13:57
  • @Szabolcs:From my side you have a +1.But on the other side you besides hardware there is also the language you use.If you use Java which has a shallow stack you will get a stackoverflow unless you start tampering with JVM parameters. – Cratylus Oct 28 '11 at 15:04
0

Here is a useful trick that I first saw done in quicksort:

f(x)
{
  for (;;)
  {
    if (x is trivial)
    {
      return;
    }
    x1 = one half of problem x
    x2 = other half of problem x
    if (x1 is the little half)
    {
      x = x2;
      f(x1);
    }
    else
    {
      x = x1;
      f(x2);
    }
  }
}

The point of this is that each recursive call is on the little half of the problem - typically less than half the size of the original problem - and the larger half of the problem is done via iteration. If the problem is of size 2^n, the stack need be of size only n, even if - because of very uneven splits - you end up taking time close to n^2 instead of n log n.

As far as the "real cost" goes of algorithms which use n log n time but loads of stack, note that an algorithm that runs in time n log n does not have the time to use more than n log n memory, so loads and loads of stack can't actually be huge amounts of memory, although it might be enough to make e.g. a Java interpreter crash out your program under the impression that you have runaway recursion.

mcdowella
  • 19,301
  • 2
  • 19
  • 25
  • mcdowella:I don't really get the pseudocode you posted.Where is the iteration for the larger part?You do recursion for both large and small parts.`an algorithm that runs in time n log n does not have the time to use more than n log n memory`.You are comparing time vs space.I don't see how this statement is valid.I mean if an algorithm takes 1 time step, how much memory does it use according to your analogy? – Cratylus Oct 28 '11 at 19:35
  • Sorry - I over-simplified the structure of the quick-sort like program. There is a more reasonable, if longer, explanation at http://www.petebecker.com/js/js200101.html - look where it says "In order to get rid of the problem here, too, we need to make the code that eliminates the tail recursion just a little smarter. If we call sort for the smaller of the two pieces, and loop on the larger, we’ll reduce stack usage significantly" – mcdowella Oct 29 '11 at 04:18
  • If you measure time in clock cycles - or better, memory bus cycles, then one unit of time can hold at most one memory write. More theoretically, if your program takes time n log n but uses space n^2, then it is writing n / log n memory units per unit of time, which does not makes sense as n gets larger and larger. – mcdowella Oct 29 '11 at 04:22