Floor and Ceil Are off by One
- October 16th, 2011
- By Rok Breulj
- Write comment
There are dozens of articles and posts about floating point values and their accuracy problems, but I hadn’t encountered this instance of it before.
Consider the following:
cout << (cost - research) / researchPerTurn << endl; cout << ceil((cost - research) / researchPerTurn) << endl;
with
cost = 30 research = 27 researchPerTurn = 3
The desired value from both lines is without a doubt a “1″. However, if the variables are defined as float, it really results in “1.0000000298023223876953125″ which is hidden by the fact that cout’s precision is set to 6 by default. This results in a “2″ being printed in the second case, and is really quite obvious, but only after the initial surprise of such a simple calculation producing an accuracy problem.
It’s not very awesome. Declaring them as double fixes the issue in this case, but since it’s still floating point it still feels like threading in risky territory.
In fact the almighty Wikipedia reveals:
In addition to loss of significance, inability to represent numbers such as π and 0.1 exactly, and other slight inaccuracies, the following phenomena may occur:
- Cancellation: subtraction of nearly equal operands may cause extreme loss of accuracy.[8][clarification needed] This is perhaps the most common and serious accuracy problem.
- Conversions to integer are not intuitive: converting (63.0/9.0) to integer yields 7, but converting (0.63/0.09) may yield 6. This is because conversions generally truncate rather than round. Floor and ceiling functions may produce answers which are off by one from the intuitively expected value.
- Limited exponent range: results might overflow yielding infinity, or underflow yielding a subnormal number or zero. In these cases precision will be lost.
- Testing for safe division is problematic: Checking that the divisor is not zero does not guarantee that a division will not overflow.
- Testing for equality is problematic. Two computational sequences that are mathematically equal may well produce different floating-point values. Programmers often perform comparisons within some tolerance (often a decimal constant, itself not accurately represented), but that does not necessarily make the problem go away.[citation needed]
This answers why we’ve had problems with our tickrate-capping code when it used floats. The floor function exhibited unexpected behavior on some machines. I’ll go on a hunch and assume the decreased chance of problems with doubles is also why the original C standard uses doubles exclusively and a “1.0″ is interpreted as a double, not a float.
My new rule is to use doubles by default and of course always prefer integers if fractions are not required.
Update: As suggested by xnor, in this particular case we could stick fully to integers and replace ceil with the following:
if ((cost - research) % researchPerTurn > 0) {
++result;
}
No comments yet.