Binary insertion sort and complexity

Question

I have a simple question about using binary search in the insertion sort algorithm. More precisely, at each step of the usual insertion sort, instead of linearly comparing the element with all the elements in the previous (sorted) subarray, we just use binary search in that sorted subarray to find the place where the element belongs.

I know that this reduced the number of comparisons that the algorithm makes (O(log n) instead of O(n)), but the number of swaps needed at each step still dominates and the complexity is still O(n^2).

I also know that complexity is not so easily related to running time. I have tried to compare the running time for both algorithms for "small" values of n (array size), up to about 500000. Binary insertion sort was always faster than usual insertion sort.

The fact that both are O(n^2) tells me that as n gets large enough, the running time should be similar, right? Any idea on what "large enough" would be in this situation to actually see similar running times?

Answer 1

The fact that both are O(n^2) tells me that as n gets large enough, the running time should be similar, right?

Careful - this isn't true. n^2 and 2n^2 will never get closer together as n gets bigger; they get farther apart. But both are O(n^2).

What does it mean, then, to say that both your algorithms are O(n^2)? Well, it means that eventually each one can be bounded above by some constant multiple of n^2. For your binary insertion sort, it might be 10n^2, whereas for your standard insertion sort it might be 1000n^2. Both are n^2 though the efficiency may differ by a factor of 100 (in this example).

Complexity tells you more about a particular function's behavior than it does about how that function stacks up against others. If you know your function is O(n^2), for instance, you know that for large values of n, f(n+1) will be grow by no more than some constant times n + 1 (why? because the derivative of n^2 is 2n, linear, which tells you that the difference between consecutive terms grows linearly).

Answer 2

This is my Binary Insertion Sort: Time complexity is (N^2)/4 + Nlog2(N/(2e)). It is True that if you perform swapping, it would be extremely costlier than array-rewriting. The time complexity in this case is NK/4 + Nlog2(N/(2*e)) which is between O(N^2) and O(NlogN)

def BinaryIndexing(arr: Union[Iterable, List, Tuple, np.ndarray], value, reverse: bool = False):
    if value < arr[0]:
        return 0
    elif value > arr[-1]:
        return len(arr)

    start, end = 0, len(arr)
    while True:
        if end - start <= 1:
            if reverse is False:
                if arr[start] > value:
                    return start
                else:
                    return start + 1
            else:
                if arr[start] < value:
                    return start
                else:
                    return start + 1

        mid = (start + end) // 2

        if reverse is False:
            if value < arr[mid]:
                end = mid
            else:
                start = mid
        else:
            if value > arr[mid]:
                end = mid
            else:
                start = mid


def BinaryInsertionSortOptimized(array: Union[Iterable, List, Tuple, np.ndarray], reverse: bool = False):
    if reverse is False:
        for index in range(1, len(array)):
            if array[index - 1] < array[index]:
                continue
            i = BinaryIndexing(arr=array[0:index], value=array[index], reverse=reverse)
            key_point = array[index]
            array[i + 1:index + 1] = array[i:index]
            array[i] = key_point

    else:
        for index in range(1, len(array)):
            if array[index - 1] > array[index]:
                continue
            i = BinaryIndexing(arr=array[0:index], value=array[index], reverse=reverse)
            key_point = array[index]
            array[i + 1:index + 1] = array[i:index]
            array[i] = key_point
    return array

Answer 3

In theory binary insertion sort complexity is O(log_2(n!)), Wolframalpha.

This is between O(n²) and O(n log(n)) ,more near from O(n log(n)) actually.