3

Whenever I see a recursive solution, or I write recursive code for a problem, it is really difficult for me to figure out the time complexity, in most of the cases I just say its exponential? How is it exponential actually? How people say it is 2^n, when it is n!, when it is n^n or n^k.

I have some questions in mind -

  1. let say find all permutations of a string (O(n!))
  2. find all sequences which sum up to k in an array (exponential, how exactly do I calculate).
  3. Find all subsets of size k whose sum is 0 (will k come somewhere in complexity , it should come right?).

Can any1 help me how to calculate the exact complexity of such questions, I am able to wrote code for them, but its hard understanding the exact time complexity.

Srikar Appalaraju
  • 71,928
  • 54
  • 216
  • 264
Peter
  • 2,719
  • 4
  • 25
  • 55
  • 1
    It was a subject of one of my maths courses at university. I could point to web articles on the subject, e.g. http://en.wikipedia.org/wiki/Big_O_notation but I suspect that the general answer/explanation for your question (which I think is "how can I calculate time complexity for any arbitary algorithm?") might be too long/complicated to post as an answer on this forum. For this reason, I will vote to close this question. – ChrisW Dec 09 '12 at 21:37
  • @ChrisW can you just take the example of finding subset of size k whose sum is 0 and discuss complexity of that. It will help me a lot. Lets Discuss its code and TC , code is trivial but how do i calculate TC – Peter Dec 09 '12 at 21:40
  • Moreover These kind of questions have been discussed here earlier too but not for difficult recursive algorithms – Peter Dec 09 '12 at 21:43
  • 1
    Do you ever try to write a recurrence relation to describe to algorithm? Once you do that, then you can solve via guessing and checking via substituion, or masters method. Some are still difficult, but it covers many cases. – goat Dec 09 '12 at 21:45
  • @rambocoder I know the masters theorum , i can solve if given the recurrence.Writing recurrence for some problems is what i find difficult :(. can you take one of the problems mentioned and help me write the recurrence, when i write and calculate i find it n^n but people say it is 2^m , i m really confused on this – Peter Dec 09 '12 at 21:47
  • what is "it"? be more clear. – goat Dec 09 '12 at 21:53
  • @rambocoder lets say i am writing code for finding all subsets which sum upto say K. the recurrence which i calculate is T(n) = n*T(n-1)+C , which comes to T(n) = n(n-1)(n-2)(n-(n-1)).. which is n^n But people say the order is 2^n , i am confused on TC of such problems – Peter Dec 09 '12 at 21:57
  • 1
    @Peter For that one, a set of `n` elements has `2^n` subsets. For all those subsets compute the sum, and compare. Upper bound of `n*2^n` immediate. To get the `2^n` bound, a bit more careful analysis is needed. There are two kinds of subsets, those that include the last element, and those that don't. Find all subsets of the subset of the first `(n-1)` elements that sum to `K`, and those that sum to `K - S(n)`. Both subproblems take `T(n-1)` time, so `T(n) = 2*T(n-1)`. – Daniel Fischer Dec 09 '12 at 22:09
  • @DanielFischer thanks , but what if i have to find any k elements that sum to some C, for that again 2 subproblems , k elements in first (n-1) elements i.e T(n-1) and K-1 elements that sum to C-a[last] , now how to write this recurrence – Peter Dec 09 '12 at 22:23
  • @DanielFischer, Also can you explain me the first part with help of recursion tree and calculating sum, as at every index i am braking problem into n-1 subproblems , so here when i calculate using recursion tree it comes to n + n(n-1)+n^2(n-2)+..n^n.. here it is not coming as 2^n , can you please correct this approach of mine. I'll be very grateful – Peter Dec 09 '12 at 22:26

2 Answers2

3

find all sequences which sum up to k in an array (exponential, how exactly do I calculate).

Let F(a, n, k) be the number of all subsets of S ⊂ {0, 1, ..., n-1} so that

 ∑ a[i] == k
i∈S

Then we can compute F(array, length of array, k) recursively by splitting the problem in two subproblems (for n > 0).

The subsets of {0, 1, ..., n-1} can be partitioned into two classes, those that contain n-1 and those that don't.

We obtain the recursion

F(a,n,k) = F(a,n-1,k) + F(a,n-1, k-a[n-1])

Let T(n) be the time necessary to compute F(_,n,_) (the underscores indicating that T(n) does only depend on n, not on the array or on k [although for specific arrays and k faster algorithms are possible]. The recursion for F then immediately implies

T(n) = 2 * T(n-1)

for n > 0.

For n == 0, we can compute the solution in constant time,

F(a, 0, k) = k == 0 ? 1 : 0

so we have T(n) = 2^n * T(0) inductively.

If the subsets shall not only be counted, but output, the complexity becomes O(n * 2^n) and that bound is tight (for an array of all 0s, with k == 0, all subsets meet the condition, and printing them takes Θ(n * 2^n) time).

Find all subsets of size k whose sum is 0 (will k come somewhere in complexity , it should come right?).

Yes, the complexity of that problem depends on n and k.

Let F(a,n,k,s) be the number of subsets S ⊂ {0, 1, ..., n-1} of cardinality k such that

 ∑ a[i] == s
i∈S

For k == 0, we again have a constant time answer, there is one such subset (the empty set) if s == 0, and none otherwise. For k > n the set {0, 1, ..., n-1} has no subsets of cardinality k, so F(a,n,k,s) = 0 if k > n.

If 0 < k <= n, we can, like above, consider the subsets containing n-1 and those that don't separately, giving

F(a,n,k,s) = F(a,n-1,k,s) + F(a,n-1,k-1,s - a[n-1])

and for the time complexity we find

T(n,k) = T(n-1,k) + T(n-1,k-1)

That recursion is known from the binomial coefficients, and we have

T(n,k) = n `choose` k = n! / (k! * (n-k)!)

(with T(n,0) = 1).

Once again, if the sets shall not only be counted, but output, the time complexity increases, here all sets have cardinality k, so it becomes

k * n! / (k! * (n-k)!)
Daniel Fischer
  • 181,706
  • 17
  • 308
  • 431
0

Take a look at Master Theorem. It is perfectly explained in prof. Tim Roughgarden's Algorithms: Design and Analysis: Part I, from Stanford. This are online classes, which I recommend, but you can watch the videos without enrolling to the course. There is also part II of this course, if you are interested.

Adam Stelmaszczyk
  • 19,665
  • 4
  • 70
  • 110
  • 1
    I know the masters theorum , i can solve if given the recurrence.Writing recurrence for some problems is what i find difficult :(. can you take one of the problems mentioned and help me write the recurrence.When i write and calculate i find it n^n but people say it is 2^m , i m really confused on this – Peter Dec 09 '12 at 21:45
  • The lecture which you have given explains only those standard trivial algorithms like merge sort , binary search etc, i need help on these non trivial complex problems in which its difficult to judge complexity. I just dont want to say its exponential , i want to know what it is and how it came – Peter Dec 09 '12 at 21:54
  • The master theorem applies to a restricted set of recurrences, namely those occurring in traditional divide-and-conquer algorithms. It's not a magic bullet for deriving asymptotic complexities. – Fred Foo Dec 09 '12 at 22:08