andishgar opened a new issue, #48123:
URL: https://github.com/apache/arrow/issues/48123
### Describe the enhancement requested
Currently, Arrow employs an **absolute tolerance** approach for approximate
floating-point comparisons. However, this method is suboptimal for `Float16`
values. The default absolute tolerance is too small relative to the
representational granularity of half-precision floats. Specifically, a
difference of one ULP after `1.0` corresponds to an increment of `1/1024`,
which is approximately **100 times larger** than Arrow’s default tolerance.
Consequently, the following assertion incorrectly passes, even though the two
values differ by exactly one ULP:
```c++
union Float16Union {
explicit Float16Union(Float16 value) : value(value) {}
uint16_t u_int_val;
Float16 value;
};
TEST(Compare, Float16ULPs) {
Float16 a = Float16::FromBits(0b0011110000000000);
ASSERT_EQ(a.ToFloat(), 1);
Float16Union a_union(a);
// Increment by one ULP
a_union.u_int_val += 1;
// One ULP after 1.0 in Float16 equals 1/1024
ASSERT_EQ(a_union.value.ToFloat(), (1.0f + 1.0f / 1024.0f));
auto b = a_union.value;
HalfFloatScalar a_scalar(a);
HalfFloatScalar b_scalar(b);
// This assertion passes because one ULP in [1, 2)
// is roughly 100× larger than Arrow’s default tolerance
ASSERT_FALSE(a_scalar.ApproxEquals(b_scalar));
}
```
One possible solution is to use a different absolute tolerance value for
`Float16`. However, this approach is not very practical, because the size of
one ULP doubles with each power of two. For example, after the number 1024, one
ULP equals 1.0.
There are other comparison methods—such as **relative difference** or
**ULP-based distance**—tend to work better for numbers larger than one.
### Component(s)
C++
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
