niedziela, 20 listopada 2016

How not to compare floating point variables

During the introduction of floating point arithmetics my students often see a code like: Such an example is an occasion to explain the binary representation of floating point data and the problems with rounding errors.

Having in mind that floating point is not easy I am not eager to expose freshmen students to such problems specifically during tests and for a given problem:

Given the positive integer number x read from input, please print all numbers from [1..x] which are squares of natural numbers.

I have expected something like: But in one case I got something like (comment is mine): Beside efficiency issues, the above code is of course wrong. But, for some specific compiler, environment and small inputs it looked like working. To be honest, I was probably more surprised with this "correct" behavior than the student who was explained that the code is indeed wrong.

Comparing doubles by "==" operator is not a good idea but after several tests this example showed me that floating point arithmetic is less deterministic I had expected. For example it is not clear how much precision the internal calculations are evaluated at: double or maybe "register precision"? The answer is: "it depends", for more details I am redirecting you to Random ASCII, Floating-Point Determinism by Bruce Dawson and his earlier post: Intermediate Floating-Point Precision.