We're looking for an algorithm to solve this problem in under O(N).
given two real numbers a and b (without loss of generality you can assume they are both between 0 and 1) Find an integer n between -N and N that minimizes the expression:
|a n - b - round(a n - b)|
We have thought that the Euclidean Algorithm might work well for this, but can't figure it out. It certainly looks like there should be much faster ways to do this than via an exhaustive search over integers n.
Note: in our situation a and b could be changing often, so fixing a and b for a lookup table is possible, it gets kind of ugly as N can vary as well. Haven't looked in detail into the lookup table yet, to see how small we can get it as a function of N.