'Less than' comparison with tolerance

Question

I am trying to write some code that computes whether a is less than b with n-bit tolerance, where a and b are double precision variables.

For example, 4.000000001 < 4.00000001 would be TRUE, but 4.0000000001 < 4.000000001 would be FALSE. 3.99999999 < 4.00000000 would also be FALSE. Also note that a and b could be negative.

Do you want to compare with a certain tolerance, or really the first n digits? 4.000000000<3.9999999999 will not be true, but is a very common derivation. — Daniel, Nov 10 '13 at 20:01
I'm sorry, i phrased it incorrectly. I meant to say with n-bit tolerance — Κωστας Ιωαννου, Nov 10 '13 at 20:11
You might have a look at [`eps`](http://www.mathworks.com/help/matlab/ref/eps.html). You can get a variety of tolerances appropriate for different data types (e.g. `eps('single')`) — chappjc, Nov 10 '13 at 21:05

score 3 · Answer 1 · answered Nov 10 '13 at 20:01

3

This should work:

ndigits = 7;
round(a*10^ndigits) < round(b*10^ndigits)

answered Nov 10 '13 at 20:01

Stewie Griffin

14,889
11
39
70

score 2 · Answer 2 · answered Nov 10 '13 at 20:06

2

If you really look for a < relationship, I would do

x < y + tolerance

where tolerance is a value which indicates which values above y shall count as below it.

answered Nov 10 '13 at 20:06

glglgl

89,107
13
149
217

'Less than' comparison with tolerance

2 Answers2