Hi, This is about an fp issue I stumbled upon while determining the order of magnitude of some fp number. It occured to me while determining the appropriate *prefix* of a unit, given some *numerical value* carrying the unit.
The prefix to determine (denoted by an underscore '_') can arise in the unit (based on e.g. 'm' for meter) in any arbitrary power. In the most simple case, as in '_m', the power of the prefix is 1, but e.g. in '1 / _m^2', the power is -2. I am explaining the problem here as it occurs with: - A prefix "u" (micro), carrying 1000 ** -2 as the coefficient; - A power of the prefix of -2; - A numerical value of 10 ** 12, for which the prefix shall be determined. I am comparing the numerical value with the prefix coefficient, raised to the prefix power. The power of the prefix coefficient is called the "prefix value". For *negative* prefix powers, the correct prefix is the *smallest* one, where it holds: numerical value >= prefix value Here, for a prefix power of -2, and the prefix 'u' (1000 ** -2 = 10 ** -6), mathematically the prefix value is (10 ** -6) ** -2 = 10 ** 12, which is just the numerical value, so the abovementioned comparison should be fulfilled as numerical value == prefix value. Technically I am obtaining: 1. The numerical value (10 ** 12) is 1000000000000. 2. The prefix value however, (10 ** -6) ** -2, is 10000000000.0001. Here, where the numerical value and the prefix value should coincide, the numerical value is smaller than the prefix value: numerical value - prefix value = -0.000122... so the 'u' prefix is mistakenly rejected, as "numerical value >= prefix value" is not fulfilled. I kind-of-fixed it utilising numpy.nextafter: stepnumerical = numericalvalue - numpy.nextafter(numericalvalue, 0) stepprefix = prefixvalue - numpy.nextafter(prefixvalue, 0) and replacing the condition "numerical value >= prefix value" by: numerical value - prefix value >= -(stepnumerical + stepprefix) In the singleton example discussed here, this remedies the flaw. It *seems*, that the error of the final figures "numerical value" and "prefix value" is not larger than the error resulting from their limited number of fp digits. However, I am unsure if this can be used *always* as the decisive threshold. I could speculate that the fp uncertainty of intermediate numbers might carry through, so that the overall error margin is even larger than the one implied by the limitedness of the *final* number of binary digits. This suspection applies most prominently to the "prefix value". One idea would be to use a multiple of the numpy.nextafter result. However, 2x (2 bits), 4x (3 bits), ...? I think that the approach using numpy.nextafter is valid, but my use is currently a little heuristic. Any idea, comment or pointer would be welcome very much. I'd like to sort this problem thoroughly. Best, Friedrich
_______________________________________________ NumPy-Discussion mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3//lists/numpy-discussion.python.org Member address: [email protected]
