[issue14381] Intern certain integral floats for memory savings and performance

Kristján Valur Jónsson added the comment: Dang. I yield! -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: h

[issue14381] Intern certain integral floats for memory savings and performance

Mark Dickinson added the comment: > I declare that this rule does not apply here ... Clearly the gcc developers disagree. :-) iwasawa:~ mdickinson$ cat test2.c int is_positive_zero(double f) { return *(long long*)&f == 0; } iwasawa:~ mdickinson$ gcc -fstrict-aliasing -O3 -Wall -Wextra -Wstr

[issue14381] Intern certain integral floats for memory savings and performance

Kristján Valur Jónsson added the comment: Interesting. I declare that this rule does not apply here since the code is a deliberate hack: We are pretending that a certain address points to integers and checking those integers. If you insist on following the standard, you could do double cm

[issue14381] Intern certain integral floats for memory savings and performance

Changes by Kristján Valur Jónsson : -- Removed message: http://bugs.python.org/msg158808 ___ Python tracker ___ ___ Python-bugs-list m

[issue14381] Intern certain integral floats for memory savings and performance

Kristján Valur Jónsson added the comment: Interesting. I declare that this rule does not apply here since the code is a deliberate hack: We are pretending that a certain address points to integers and checking those integers. If you insist on following the standard, you could do double cm

[issue14381] Intern certain integral floats for memory savings and performance

Mark Dickinson added the comment: > Because "pfval" in this example doesn't exist but is merely a temporary, > > there is no aliasing. Maybe aliasing wasn't the right word to use, then. The exact rule being violated is C99 6.5, para. 7 (or C11 6.5 para. 7 if you prefer): """ An object shall

[issue14381] Intern certain integral floats for memory savings and performance

Kristján Valur Jónsson added the comment: return *(PY_LONG_LONG*)&fval == 0; There is no aliasing, because there are no pointer variables in existence. If we did this: double *pfval = &fval; PY_LONG_LONG *pl = (PY_LONG_LONG*)pfval return *pfval == 0 Then we would have aliasing. Because "pfval

[issue14381] Intern certain integral floats for memory savings and performance

Mark Dickinson added the comment: > We only support IEEE platforms. I don't think that's true, BTW. -- ___ Python tracker ___ ___ Py

[issue14381] Intern certain integral floats for memory savings and performance

Mark Dickinson added the comment: > Why do you think it isn't safe, Antoine? It violates C's strict aliasing rules; Google for 'C strict aliasing' or 'C type punning' for more information. This isn't just a theoretical concern: gcc is known to make optimizations based on the assumption tha

[issue14381] Intern certain integral floats for memory savings and performance

Changes by Jesús Cea Avión : -- nosy: +jcea ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.

[issue14381] Intern certain integral floats for memory savings and performance

Changes by Eric Snow : -- nosy: +eric.snow ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.o

[issue14381] Intern certain integral floats for memory savings and performance

Kristján Valur Jónsson added the comment: Why do you think it isn't safe, Antoine? We only support IEEE platforms, and IEEE defines positive 0.0 to be all bits cleared. I personally don't think that even a small measurable performance degradation in creating new floats is problematic since th

[issue14381] Intern certain integral floats for memory savings and performance

Raymond Hettinger added the comment: FWIW, I'm -1 on making the change. Any memory savings would be microscopic for most Python programs. And anything that slows down float creation (even a teeny bit) is detrimental to all Python programs. We're better off not adding special case logic unl

[issue14381] Intern certain integral floats for memory savings and performance

Serhiy Storchaka added the comment: PyBench contains a lot of different tests and executes them not too long. Random fluctuations of a few percent are possible. We need a more stable realistic test working with floats. $ ./python -m timeit 'x,y,d=1,0,0.01 for i in range(628): x,y=x-d*y,y+d*

[issue14381] Intern certain integral floats for memory savings and performance

Antoine Pitrou added the comment: Ok. In the current state it doesn't look detrimental, although I'm not convinced it's that useful either. That said, the implementation needs a bit of work, I don't think it's ok for float_is_positive_zero() to use the Py_LONG_LONG casting thing. --

[issue14381] Intern certain integral floats for memory savings and performance

Kristján Valur Jónsson added the comment: PyFloat_FromString() is very restrictive. In our case, for example, the floats in question don't come from text files. I'm adding a new file with changes suggested by Antoine. only 0.0 and 1.0 are interned. It turns out that the "int" test is expen

[issue14381] Intern certain integral floats for memory savings and performance

Serhiy Storchaka added the comment: Try moving your changes from PyFloat_FromDouble to PyFloat_FromString. Look at memory and perfomance. -- ___ Python tracker ___

[issue14381] Intern certain integral floats for memory savings and performance

Antoine Pitrou added the comment: > Right, I'll do some test with the hardcoded values you mentioned. > Btw, I don't think -0.0 is worth it, that value is a typical > arithmetic result and not something you typically get from input data. I was suggesting -0.0 in the hope that it might make the

[issue14381] Intern certain integral floats for memory savings and performance

Kristján Valur Jónsson added the comment: The -10..11 range was determined empirically. As you see from the values, only -10 shows up as significant... In fact, just storing 0 to 5 would capture the bulk of the savings. Right, I'll do some test with the hardcoded values you mentioned. Btw,

[issue14381] Intern certain integral floats for memory savings and performance

Antoine Pitrou added the comment: > Integers don't fall out of arithmetic that often, true. But integral > floats are incredibly common in tabular data. In a game such as Eve > Online, configuration data contains a lot of 0.0, 1.0, -1.0 and so > on. This patch saved us many megabytes on the

[issue14381] Intern certain integral floats for memory savings and performance

Kristján Valur Jónsson added the comment: Yes, there is a measurable performance decrease in pybench arithmetic tests. Integers don't fall out of arithmetic that often, true. But integral floats are incredibly common in tabular data. In a game such as Eve Online, configuration data contain

[issue14381] Intern certain integral floats for memory savings and performance

Mark Dickinson added the comment: This looks like a duplicate of issue 4024. -- nosy: +mark.dickinson ___ Python tracker ___ ___ Pyth

[issue14381] Intern certain integral floats for memory savings and performance

Serhiy Storchaka added the comment: Does it not decrease the performance? Falling out integral floating point numbers in the mathematical floating point calculations seems unlikely. I suggest interning integral floats only in conversions str -> float and int -> float. Exception can be made to

[issue14381] Intern certain integral floats for memory savings and performance

New submission from Kristján Valur Jónsson : Michael Foord reminded me of this issue recently. It was discussed on pydev a few years back and met with limited enthusiasm. I speak from experience in live production with EVE that this simple change saved us a lot of memory. Integral floating