2

Suppose I have a recursive algorithm that splits input into 2 inputs of size n-1 and solves them recursively. It combines the results in a constant time say c.

So formulating an equation for this,

T(n) = 2T(n-1) + c

In order to find the complexity of this one, I used the recursive tree method. Since input is divided into 2 at each step, the number of nodes will get squared at each step while since only one element is getting eliminated, every level will lose only one element from the list.

So I think that the complexity of this problem should be Θ(n2)

Am I right in this thought process. If not, What am I doing wrong?

Thank you.

Greg Hewgill
  • 951,095
  • 183
  • 1,149
  • 1,285
mihsathe
  • 8,904
  • 12
  • 38
  • 54
  • 1
    `c + 2(c + 2(c + 2(c + 2(...))))`: if you "unloop" the whole thing, the most wrapped `c` gets multiplied by `2^(n-1)`. – akappa Sep 06 '12 at 19:11

3 Answers3

5

The number of nodes at each step doesn't get squared. Instead, it doubles at every level. For example, the number of nodes at

  • problem size n: 1
  • problem size n - 1: 2
  • problem size n - 2: 4
  • problem size n - 3: 8
  • problem size n - 4: 16
  • ...
  • problem size n - i: 2i

As a result, the total number of nodes in your recursion tree will be 1 + 2 + 4 + 8 + ... + 2n = 2n+1 - 1. Accordingly, the work done will be c2n+1 - c, which is Θ(2n). This is exponential time, rather than quadratic time.

Hope this helps!

templatetypedef
  • 362,284
  • 104
  • 897
  • 1,065
  • Yes. Great help indeed. I knew I was missing something but could not figure out what. Thank you so much sir. – mihsathe Sep 06 '12 at 19:13
  • @templatetypedef it is not 1+2+3+... It is 1+2+4+8 ...I think it was a typo from your end. But better to correct it . – Geek Sep 07 '12 at 09:23
2

The complexity is actually exponential: Omega(2^n).

Let's prove it using mathematical induction. For simplicity - assume c=1

Claim: T(n) >= 2^n
Base: T(0) = 1 (Let's take it as assumption)
Assume the claim is true for each k<n and using it let's prove:
T(n) = 2*T(n-1) + 1 >= 2*2^(n-1) + 1 = 2^n + 1 > 2^n

Now, to wrap it up let's show it is also O(2^n) and we will conclude this is Theta(2^n):

Claim: T(n) <= 2*2^n - 1
Base: T(0) = 1 = 2 -1 
Assume the claim is true for each k<n and using it let's prove:
T(n) = 2*T(n-1) + 1 <= 2 * (2*2^(n-1) -1 ) + 1 = 2*2^n -2 + 1 = 2*2^n -1

As you can see, we actually get exactly 2*2^n-1 (The <= can be easily replaced with = in the above proof), which fits what is shown by @templatetypedef

amit
  • 175,853
  • 27
  • 231
  • 333
  • What if I am not sure it is 2^n (which I was not in this case) – mihsathe Sep 06 '12 at 19:25
  • 1
    @mihsathe: (1) You can use other methods to "see" what it is, and use induction to proof it is the case. In this case, if you tried to prove by induction the `O(n^2)` claim, and you would have failed - and it would provide red flags (2) with experience - you get better clue. More experienced you are, you are more likely to understand what is missing in the inductive proof and it will lead you to the right answer. Again, with experience, if you tried to show O(n^2), the failing attempt will give you a hint what is missing and guide you to the solution. – amit Sep 06 '12 at 19:28
1

from T(n+1)=2T(n)+c we have :

T(n+1) + c = 2T(n) + 2c
T(n+1) + c = 2( T(n) + c )
T(n+1) + c = 2^n (T(0) + c)
T(n+1) = 2^n (T(0) + c) - c

which yields a Θ(2^n) complexity.

Kwariz
  • 1,306
  • 8
  • 11
  • I did not understand the third step. – mihsathe Sep 06 '12 at 19:16
  • It's a classical [sum of numbers in geometric progression](http://en.wikipedia.org/wiki/Geometric_progression#Geometric_series), very useful to keep in mind when you have to solve complexity exercices. If we let U(n+1)=T(n+1)+c, then U(n+1)=2 U(n) ... then apply the formula – Kwariz Sep 06 '12 at 19:19