53

I have a question in algorithm design about complexity. In this question a piece of code is given and I should calculate this code's complexity. The pseudo-code is:

for(i=1;i<=n;i++){
    j=i
    do{
        k=j;
        j = j / 2;
    }while(k is even);
}

I tried this algorithm for some numbers. and I have gotten different results. for example if n = 6 this algorithm output is like below

i = 1 -> executes 1 time
i = 2 -> executes 2 times
i = 3 -> executes 1 time
i = 4 -> executes 3 times
i = 5 -> executes 1 time
i = 6 -> executes 2 times

It doesn't have a regular theme, how should I calculate this?

John Kugelman
  • 349,597
  • 67
  • 533
  • 578
Behzad Hassani
  • 2,129
  • 4
  • 30
  • 51

5 Answers5

98

The upper bound given by the other answers is actually too high. This algorithm has a O(n) runtime, which is a tighter upper bound than O(n*logn).

Proof: Let's count how many total iterations the inner loop will perform.

The outer loop runs n times. The inner loop runs at least once for each of those.

  • For even i, the inner loop runs at least twice. This happens n/2 times.
  • For i divisible by 4, the inner loop runs at least three times. This happens n/4 times.
  • For i divisible by 8, the inner loop runs at least four times. This happens n/8 times.
  • ...

So the total amount of times the inner loop runs is:

n + n/2 + n/4 + n/8 + n/16 + ... <= 2n

The total amount of inner loop iterations is between n and 2n, i.e. it's Θ(n).

Jhon
  • 582
  • 7
  • 28
interjay
  • 107,303
  • 21
  • 270
  • 254
  • 15
    +1 Suprised to not see this as the accepted answer. While it's technically true that it is O(n log n), it's not really asymptotic... Also, here's a Θ if you want to edit it in – Sabre May 14 '15 at 13:33
  • 4
    "The upper bound given by the other answers is actually too high." is false. A better formulation is "... is actually higher than they need to be." – Taemyr May 15 '15 at 09:38
6

You always assume you get the worst scenario in each level.
now, you iterate over an array with N elements, so we start with O(N) already.
now let's say your i is always equals to X and X is always even (remember, worst case every time).
how many times you need to divide X by 2 to get 1 ? (which is the only condition for even numbers to stop the division, when they reach 1).
in other words, we need to solve the equation X/2^k = 1 which is X=2^k and k=log<2>(X) this makes our algorithm take O(n log<2>(X)) steps, which can easly be written as O(nlog(n))

David Haim
  • 25,446
  • 3
  • 44
  • 78
  • 5
    I originally did the same analysis, but I think it is incomplete. While the outer loop is executed `n` times and the inner loop has worst-case complexity of O(log(n)), it does not follow that the overall complexity is O(n log(n)); it depends on how often the worst-case complexity of the inner loop occurs as `i` varies from 1 to `n`. My guess is that this frequency is not O(n) and that the overall complexity is actually lower (perhaps O(log(n)^2)). – Ted Hopp May 14 '15 at 09:21
  • again, the worst case is a compound calculation of all the worst cases the problem poses. you are talking about the average complexity, where *usually* the input does not pose worst case scenario each time. – David Haim May 14 '15 at 09:23
  • 3
    The worst case here is O(n), as shown by my answer. (It's somewhat meaningless to talk about worst-case vs. average case though, because the algorithm doesn't have input other than `n` itself). – interjay May 14 '15 at 09:24
  • please demonstrate you answer on array with n elements, each with the value of 2^n. – David Haim May 14 '15 at 09:26
  • 4
    But that's an unfair request. The array is specifically [1, 2, ..., n]. I don't think O(n log(n)) necessarily follows. To take an extreme example, suppose the inner loop had worst-case behavior of O(log(i)) for exactly one value of `i` and O(1) for all other values. Then the fact that the outer loop executes `n` times would not result in an overall complexity of O(n log(n)); it would be O(n) + O(log(n)) = O(n). The actual inner loop doesn't have this extreme behavior, of course, but it's far from clear (to me) that the density of worst-case behavior warrants the conclusion of O(n log(n)). – Ted Hopp May 14 '15 at 09:28
  • but we're talking about big O notation, which is the upper bound, you are correct that tight bound is not Theta(nlog(n)) , but O(nlog n) bind the complexity on the top. – David Haim May 14 '15 at 09:33
  • 7
    By that logic, you could also say that the complexity is O(2^n) and be correct. I don't think that reasoning is in the spirit of the question. – Ted Hopp May 14 '15 at 09:35
  • 4
    @RobOcel: No, it's not a matter of worst vs. average. The worst-case is still O(n) *(which technically means it's also O(n log n))*. If n is odd it is still O(n) - you are enumerating all values from 1 to n, not just n itself. – BlueRaja - Danny Pflughoeft May 14 '15 at 14:59
  • 4
    If you want to lawyer, every algorithm is O(infinity) and Omega(1), but that doesn't actually mean anything. Theta bounds are more interesting in most cases. – Kevin May 14 '15 at 18:45
4

For such loop, we cannot separate count of inner loop and outer loop -> variables are tighted!

We thus have to count all steps.

In fact, for each iteration of outer loop (on i), we will have 

1 + v_2(i) steps

where v_2 is the 2-adic valuation (see for example : http://planetmath.org/padicvaluation) which corresponds to the power of 2 in the decomposition in prime factor of i.

So if we add steps for all i we get a total number of steps of :

n_steps = \sum_{i=1}^{n} (1 + v_2(i)) 
        = n + v_2(n!)    // since v_2(i) + v_2(j) = v_2(i*j)
        = 2n - s_2(n)    // from Legendre formula (see http://en.wikipedia.org/wiki/Legendre%27s_formula with `p = 2`)

We then see that the number of steps is exactly :

n_steps = 2n - s_2(n)

As s_2(n) is the sum of the digits of n in base 2, it is negligible (at most log_2(n) since digit in base 2 is 0 or 1 and as there is at most log_2(n) digits) compared to n.

So the complexity of your algorithm is equivalent to n:

n_steps = O(n)

which is not the O(nlog(n)) stated in many other solutions but a smaller quantity!

Samuel Caillerie
  • 8,259
  • 1
  • 27
  • 33
  • the example provided loops n times and then in it's worst case where i is even and is the result of 2^x 1 + log_2(i) iterations to become "odd" or more accurately `1 + Omega(n)` where `Omega(n)` returns the number of prime factors .e.g `6 = 2,3`, so it's not n +... but n logn. I think you missed the for loop on line 1? – Alexander Holman Jul 27 '18 at 13:45
  • 1
    @AlexanderHolman I disagree : as stated in this solution, the n_steps = 2*n - s_2(n) is an exact solution and a worst case for one iteration can be extended but this remains a worst estimation that an exact one... The sum for the loop over the i loop is indicated by the line : n_steps = \sum_{i=1}^n ... – Samuel Caillerie Aug 20 '18 at 13:59
  • 1
    Yep I was wrong! At this time of night I honestly can't work out if you are right... Your magic seems to work! +1 for you :) – Alexander Holman Aug 28 '18 at 22:59
0

lets start with worst case:

if you keep dividing with 2 (integral) you don't need to stop until you get to 1. basically making the number of steps dependent on bit-width, something you find out using two's logarithm. so the inner part is log n. the outer part is obviously n, so N log N total.

sp2danny
  • 7,488
  • 3
  • 31
  • 53
0

A do loop halves j until k becomes odd. k is initially a copy of j which is a copy of i, so do runs 1 + power of 2 which divides i:

  • i=1 is odd, so it makes 1 pass through do loop,
  • i=2 divides by 2 once, so 1+1,
  • i=4 divides twice by 2, so 1+2, etc.

That makes at most 1+log(i) do executions (logarithm with base 2).

The for loop iterates i from 1 through n, so the upper bound is n times (1+log n), which is O(n log n).

CiaPan
  • 9,381
  • 2
  • 21
  • 35