Binary Search using Geometric Means

Question

Background: When doing a binary search on an array, we initialise an upper h and lower l bound and test the mid-point (the arithmetic mean) half-way between them m := (h+l)/2. Then we either change the upper or lower bound and iterate to convergence.

Question: We can use a similar search strategy on the (unbounded) real numbers (well their floating point approximation). We could use a tolerance of convergence to terminate the search. If the search range is on the positive real numbers (0<l<h), we could instead take the geometric mean m :=(hl)^.5. My question is, when is the geometric mean faster (taking few iterations to converge)?

Motivation: This cropped up when I tried a binary search for a continuous variable where the initial bounds were very wide and it took many iterations to converge. I could use an exponential search before a binary search, but I got curious about this idea.

What I tried: I tried to get a feel for this by picking a million random (floating point) numbers between 2 and an initial h that I picked. I kept the initial l=1 fixed and the target had to be within a tolerance of 10^-8. I varied h between 10^1 and 10^50. The arithmetic mean had fewer iterations in about 60-70% of cases.

But the geometric mean is skewed (below the arithmetic mean). So when I restricted the targets to be less than the geometric mean of the initial bounds sqrt(lh) (still keeping l=1) the geometric mean was almost always faster (>99%) for large h>10^10. So it seems that the both h and the ratio of target / h could be involved in the number of iterations.

Code Example: Here's a simple Julia code to demonstrate:

function geometric_search(target, high, low)
    current = sqrt(high * low)
    converged = false
    iterations = 0
    eps = 1e-8

    while !converged
        if abs(current - target) < eps
            converged = true
        elseif current < target
            low = current
        elseif current > target
            high = current
        end

        current = sqrt(high * low)
        iterations += 1
    end
    return iterations
end

target = 3.0 
low = 1.0 
high = 1e10

println(geometric_search(target, high, low))

The Julia code isn't doing a binary search, because there is no array. All the Julia code does is measure how long it takes for the current index to converge to some target index. So obviously, geometric mean is faster than arithmetic mean when the target index is small. — user3386109, Dec 02 '22 at 07:26
To be clear, what the Julia code is doing is not representative of a binary search because in a binary search, `high`, `low`, and `current` are all integer values, and the search ends abruptly when `high` and `low` are the same. There is no gentle convergence between the `current` index and the target index, to within some small epsilon. The absolute difference between the `current` index and target index is 0, 1, or some other positive integer. It's never some small value between 0 and 1. — user3386109, Dec 02 '22 at 07:47
@mattapow could you phrase the bold text like a question, what kind of question is "When the geometric mean is faster (few iterations)?"? Perhaps [edit] the question and state the actual problem at the beginning in a clearer manner. — user692942, Dec 02 '22 at 09:11
@user692942 I've made the structure of the question clearer, which helps highlight the problem from where the text becomes "what I tried". — mattapow, Dec 11 '22 at 22:00

score 1 · Answer 1 · answered Dec 02 '22 at 13:36

The optimal worst-case time for a binary search is achieved when each decision divides the "possible answers" in half.

For searching in an array, it is therefore optimal in this respect to test the center of the possible range every time.

If you're not searching in an array, then it depends on your stopping criteria. In your case, you are searching for, for example, a square root among all possible floating-point values.

If your stopping criteria is an absolute difference, which is what you wrote, then choosing the arithmetic mean of the possible range is still best in the worst case.

If your stopping criteria is a percentage difference, however, then there are many more "small" answers than large ones (you know what I mean). This is using an absolute difference criteria while searching for the log of the answer. In this case, it is the geometric mean that is optimal in the worst case.

As mentioned above, you wrote an absolute difference stopping condition, but this is confused by the details of the floating point number representation. A floating point number is represented as 2^em, where e and m are integers of limited precision. There are, therefore, more small answers than large ones, even if you code an absolute difference stopping condition, so a geometric mean can again be better.

This explains why the geometric mean may be better for your specific implementation... but I'm afraid that your implementation will fail to converge for certain very large inputs.

score 1 · Accepted Answer · answered Dec 09 '22 at 05:38

The heart of this problem is "how fast is a geometric-mean binary search?"

In an ordinary (arithmetic mean) binary search among n discrete elements, the number of steps is s = log₂n (in the large n limit). In the continuous case, if we are searching for an interval of size d in a larger interval of size R, the number of steps is s = log₂(R/d).

Now consider the continuous case, but using the geometric mean. We are searching over a continuous variable x for an interval [k, k+d] in a larger interval [l, h], and our first test will be (lh)^1/2. Now we "take the log of the whole problem", which is to say we change to a new variable y which is defined as ln(x). So now we are searching for the interval [ln(k), ln(k+d)] in the larger interval [ln(l), ln(h)], and our first test will be y=(ln(l)+ln(h))/2. This is an arithmetic-mean binary search in y, but the size of the target varies according to its location. The number of steps will be

s = log₂((ln(h)-ln(l))/(ln(k+d)-ln(k)))
= log₂(ln(h/l)/ln((k+d)/k))

And notice that ln((k+d)/k) = ln(1 + d/k).

So the number of steps to find the target depends on k, the location of the target. Targets nearer to the low end of the range will be bigger, and will therefore be found more quickly than targets nearer the higher end. Near the low end, geometric-mean searches are faster than arithmetic-mean, and near the high end, arithmetic-mean are faster than geometric mean.

So where is the critical point? When will arithmetic-mean and geometric-mean searches be equally fast? We set the number of steps equal:

log₂(ln(h/l)/ln(1 + d/k)) = log₂((h-l)/d)
ln(h/l)/ln(1 + d/k) = (h-l)/d

And solve for k:

ln(1 + d/k) = d ln(h/l)/(h-l)
1 + d/k = (h/l)^d/(h-l)
d/k = (h/l)^d/(h-l) - 1
k = d/((h/l)^d/(h-l) - 1)

(This seems correct, but I have not yet written code to test it.)

Binary Search using Geometric Means

2 Answers2