4

I've seen a few interesting discussions recently debating whether or not a given ("hard") problem has at-best an 2^n or n! known solution.

My question is, aside from actually walking through the algorithm and seeing the growth rate, is there a heuristic for quickly spotting one versus the other? Ie. are there certain quickly-observable properties of an algorithm that make it obviously one or the other?

Related discussions:

Community
  • 1
  • 1
Tom Lianza
  • 4,012
  • 4
  • 41
  • 50
  • 1
    big-O notation says about the running time of an algorithm.. and in your first sentence you have mentioned about n^2 or n! known solution... Can you modify the question.. I find it a little confusing... – Ajai Feb 22 '12 at 21:04
  • By "known solution" I'm referring to the algorithm that solves the problem. Sorry... I think it's clear but I'm happy to reword it if you have a suggestion. – Tom Lianza Feb 22 '12 at 21:07
  • Hhhmm I get it now.. Thought it must have been running time... never mind. Amit has the answer.. – Ajai Feb 22 '12 at 21:17

2 Answers2

6

There is no algorithm that can determine a complexity of a program [at all]. It is a part of the Halting Problem - you cannot determine if a certain algorithm will stop or not. [You cannot estimate if it is Theta(infinity) or anything less then it]

As a rule of thumb - usually O(n!) algorithms are invoking recursive call in a loop with a decreasing range, while O(2^n) algorithms invoke a recursive call twice in each call.

Note: Not all algorithms that invokes a recursive call twice are O(2^n) - a quicksort is a good example for an O(nlogn) algorithm which also invokes a recursive call twice.

EDIT: For example:
SAT brute-force solution O(2^n):

SAT(formula,vars,i):
  if i == vars.length:
      return formula.isSatisfied(vars)
  vars[i] = true
  temp = SAT(formula,vars,i+1)  //first recursive call
  if (temp == true) return true
  vars[i] = false
  return SAT(formula,vars,i+1)  //second recursive call

Find all permutations: O(n!)

permutations(source,sol):
  if (source.length == 0): 
      print sol
      return
  for each e in source: 
      sol.append(e)
      source.remove(e)
      permutations(source,sol) //recursive call in a loop
      source.add(e)
      sol.removeLast()
amit
  • 175,853
  • 27
  • 231
  • 333
  • 2
    This is an incorrect interpretation of the Halting Problem. It is impossible to write a program that, when given _any_ other program and _any_ input to it, can determine whether the program will halt on that input, _and which will always work, whichever program+input you feed it with_. It is, however, fully possible to write a program that, for _many_ programs and inputs, will be able to determine whether it will halt, but will fail to determine this for other programs. – Aasmund Eldhuset Feb 24 '12 at 02:11
  • @AasmundEldhuset: actually, the problem is not even in [RE](http://en.wikipedia.org/wiki/RE_%28complexity%29). There is a reduction from the Not_Halting_Problem. Given `(M,x)`: create `M'` that on input `w` runs 2^|w| steps on M with input x. If M doesn't halt - M' stops. Else: It continue to run |w|! steps. It is easy to prove that (M,x) doesn't halt if and only if M' is `O(2^n)` – amit Feb 24 '12 at 08:36
  • My point was just that your formulation (as I read it) makes it sound like "given any program, it is impossible to programmatically determine the complexity of that particular program", rather than "it is impossible to write a program that can determine the complexity of any possible program". My apologies for the somewhat crass opening, by the way. – Aasmund Eldhuset Feb 24 '12 at 17:51
1

As amit mentioned it is theoretically not possible to check if an algorithm is O(2^n) or O(n!). However you can use the following heuristics:

  1. For different values of n calculate the number of steps, F(n), to solve
  2. Plot n vs log( F(n) )/n
  3. If it looks like a flat line (or levels off as a flat line) then it is O(2^n)
  4. If it looks like a strictly increasing function, then is is super exponential
  5. If it looks more line a x vs log(x) plot, then it is "probably" O(n!)
ElKamina
  • 7,747
  • 28
  • 43