I need help understanding how to find the Big-Oh of a code segment

Question

My Computer Science II final is tomorrow, and I need some help understanding how to find the Big-Oh for segments of code. I've searched the internet and haven't been able to find any examples of how I need to understand it.

Here's a problem from our sample final:

for(int pass = 1; i <= n; pass++)
{
    for(int index = 0; index < n; index++) 
        for(int count = 1; count < n; count++) 
        {
            //O(1) things here.
        }
    }
}

We are supposed to find the order (Big-Oh) of the algorithm.

I think that it would be O(n^3), and here is how I came to that conclusion

for(int pass = 1; i <= n; pass++) // Evaluates n times
{
    for(int index = 0; index < n; index++) // Evaluates n * (n+1) times
        for(int count = 1; count < n; count++) // Evaluates n * n * (n) times
        {
            //O(1) things here. 
        }
    }
}
// T(n) = (n) + (n^2 + n) + n^3
// T(n) = n^3 + n^2 + 2n
// T(n) <= c*f(x)
// n^3 + n^2 + 2n <= c * (n^3)
// O(n) = n^3

I'm just not sure if I'm doing it correctly. Can someone explain how to evaluate code like this and/or confirm my answer?

Your answer is correct, but the number of iterations you count for each loop is not. The first and second both iterate `n` times, and the third iterates `n - 1` times. Obviously that doesn't affect the result though. — Joseph Mansfield, May 05 '13 at 22:29
Things are pretty bad if you must use an O(n^3) algorithm to solve a real world problem. — john, May 05 '13 at 22:32
@john: Also it depends on many situations and the amount of `n` :-) — masoud, May 05 '13 at 22:34
@MM. That's the problem, the boss is delighted with your O(n^3) code when it works with 5 widgets, then he asks you to run it on 10,000. — john, May 05 '13 at 22:36
Need to define `i` in `for(int pass = 1; i <= n; pass++)`. Also, `pass` isn't tested. The code is O(∞) if i <= n, else O(1). — James Waldby - jwpat7, May 05 '13 at 23:49
`for(int pass = 1; i <= n; pass++) // Evaluates n times` : `i` is not defined. Is it a typo (should be `pass`) or is some code missing (before to define `i` and in `//O(1) things here` maybe to change the value of `i`) ? — Tony Morris, May 06 '13 at 11:50

taocp · Answer 1 · 2013-05-07T10:42:27.633

2

Yes, it is O(n^3). However:

for(int pass = 1; pass <= n; pass++) // Evaluates n times
{                 //^^i should be pass

    for(int index = 0; index < n; index++) //Evaluates n times
       for(int count = 1; count < n; count++)  // Evaluates n-1 times
       {
            //O(1) things here. 
       }
    }
}

Since you have three layer of nested for loops, the nested loop will be evaluated n *n * (n-1) times, each operation inside the most inner for loop takes O(1) time, so in total you have n^3 - n^2 constant operations, which is O(n^3) in order of growth.

A good summary of how to measure order of growth in Big O notation can be found here:

Big O Notation MIT

Quoting part from the above file:

Nested loops
 for I in 1 .. N loop
    for J in 1 .. M loop
      sequence of statements
    end loop;
 end loop;
The outer loop executes N times. Every time the outer loop executes, the inner loop executes M times. As a result, the statements in the inner loop execute a total of N * M times. Thus, the complexity is O(N * M). In a common special case where the stopping condition of the inner loop is J <N instead of J <M (i.e., the inner loop also executes N times), the total complexity for the two loops is O(N^2).

Similar rationale can be applied in your case.

edited May 07 '13 at 10:42

answered May 05 '13 at 22:30

taocp

23,276
10
49
62

Okay, so why does the second for loop only evaluate the `index < n` condition n times? Shouldn't it be `n + 1` because it would have to evaluate the condition when `index = n`, and then stop? – chutch1122 May 05 '13 at 22:36
@chutch1122 what we are computing is the number of operations that inside the nested for loops gets executed, not the conditions of the for loop gets evaluated. Therefore, even what you said is correct, but the body of the for loop only gets executed n times. – taocp May 05 '13 at 22:40
@chutch1122 Its how many times you enter the body of the loop that counts, not how many times `index < n` is evaluated. – john May 05 '13 at 22:40
How can you say that `for(int pass = 1; i <= n; pass++)` evaluates `n` times ? We don't know nothing about `i`. (see `jwpat7` and my comments of the question. – Tony Morris May 07 '13 at 08:16
@TonyMorris that's great point, I thought it should be pass <=n, not i by OP – taocp May 07 '13 at 10:42

score 0 · Answer 2 · answered May 05 '13 at 22:40

You are absolutely correct. It is O(n^3) for your example.

To find the Big Oh running time of any segment of code, you should think about how many times the piece of code does O(1) things.

Let me simplify your example to give a better idea of this:

for(int index = 0; index < n; index++) // Evaluates n * (n+1) times
    for(int count = 1; count < n; count++) // Evaluates n * n * (n) times
    {
        //O(1) things here. 
    }
}

In the above case, the inner loop runs n times for each run of the outer loop. And your outer loop also runs n times. This means you're doing n things, n number of times. Making it O(n^2).

One other thing to take care of is that Big Oh is an upper bound limit. This means that you should always think about what's going to happen to the code when you have a large input (in your case, a large value of n. Another implication of this fact is that multiplying or adding by constants has no effect on the Big Oh bound. For example:

for(int index = 0; index < n; index++) // Evaluates n * (n+1) times
    for(int count = 1; count < 2*n; count++) // Runs 2*n times
    {
        //O(1) things here. 
    }
}

The Big Oh running time of this code is also O(n^2) since O(n*(2n)) = O(n^2).

Also check this out: http://ellard.org/dan/www/Q-97/HTML/root/node7.html

I need help understanding how to find the Big-Oh of a code segment

2 Answers2