class Float

Practical examples and pitfalls for Float

Float examples: compare measurements and handle non-finite values

Practical notes by Ruby-Doc.org

Compare with a meaningful tolerance

A calculated value and an expected value can differ slightly in their binary floating-point representations. There is no way to store every decimal fraction exactly in a binary float. The first example illustrates that difference, then compares the values using a small absolute tolerance.

Example 1
actual = 0.1 + 0.2
p actual == 0.3
p (actual - 0.3).abs < 1e-12
Expected output
false
true

This tolerance choice isn't meant to be used across your application. The leeway allowed for a distance or temperature reading depends on the units, scale and significance of the measurement. Determine which tolerance rule best fits the context and size of your data; don't hide that determination by rounding both sides of a calculation to the appropriate number of decimal places and then comparing their string representations.

Decide how to handle non-finite values

An expression can also produce infinity or NaN (not a number). Use nan? to recognize NaN; equality with itself does not do that job. Those are floating-point values as well, so simply checking whether the object belongs to a certain class won't exclude those either. finite? clearly expresses your question if an output must be an ordinary finite measurement.

Example 2
values = [1.5, Float::INFINITY, Float::NAN]
p values.map(&:finite?)
p Float::NAN.nan?
p Float::NAN == Float::NAN
Expected output
[true, false, false]
true
false

If a calculated output produces infinity or NaN, do not silently replace that with zero in a report. Zero may actually be a valid measurement. Do not confuse a measured zero with missing input or an expression that couldn't possibly produce a finite result. The interface can then explain which case occurred, rather than presenting a fabricated measurement.

Don't put exact quantities into an approximate representation

Integers are the natural home for counts. For exact fractional arithmetic, consider rational numbers. For decimal arithmetic, consider a decimal library such as BigDecimal. Converting a decimal string to float prior to converting it to another type could have lost information before the second conversion even occurs.

When testing a new numerical helper, test zero, values near the comparison boundary and any non-finite quantities that callers might pass to it. Give these cases separate expected results. A properly formatted number does not guarantee that the original calculations were performed within some acceptable bounds of accuracy.

API reference: Float API reference

Related: Integer · Range

A Float object stores a real number using the native architecture’s double-precision floating-point representation.

Float Imprecisions

Some real numbers can be represented precisely as Float objects:

37.5    # => 37.5
98.75   # => 98.75
12.3125 # => 12.3125

Others cannot; among these are the transcendental numbers, including:

Some floating-point computations in Ruby give precise results:

1.0/2   # => 0.5
100.0/8 # => 12.5

Others do not:

See:

Note that precise storage and computation of rational numbers is possible using Rational objects.

Creating a Float

You can create a Float object explicitly with:

You can convert certain objects to Floats with:

What’s Here

First, what’s elsewhere. Class Float:

Here, class Float provides methods for:

Querying

Comparing

Converting