On Tue, Aug 12, 2014 at 11:21 PM, Stephen J. Turnbull
wrote:
> Redirecting to python-ideas, so trimming less than I might.
reasonable enough -- you are introducing some more significant ideas for
changes.
I've said all I have to say about this -- I don't seem to see anything
encouraging form c
On 12 Aug 2014, at 10:02, Armin Rigo wrote:
> Hi all,
>
> The core of the matter is that if we repeatedly __add__ strings from a
> long list, we get O(n**2) behavior. For one point of view, the
> reason is that the additions proceed in left-to-right order. Indeed,
> sum() could proceed in a m
Redirecting to python-ideas, so trimming less than I might.
Chris Barker writes:
> On Mon, Aug 11, 2014 at 11:07 PM, Stephen J. Turnbull
> wrote:
>
> > I'm referring to removing the unnecessary information that there's a
> > better way to do it, and simply raising an error (as in Python 3.
Chris Barker writes:
> What I fail to see is why it's better to raise an exception and point users
> to a better way, than to simply provide an optimization so that it's a mute
> issue.
>
> The only justification offered here is that will teach people that summing
> strings (and some other objects
August 2014 um 21:11 Uhr
Von: "Chris Barker"
An: Kein Empfänger
Cc: "Python Dev"
Betreff: Re: [Python-Dev] sum(...) limitation
On Mon, Aug 11, 2014 at 11:07 PM, Stephen J. Turnbull <step...@xemacs.org> wrote:
I'm referring to removing the unnecessary information that
On Mon, Aug 11, 2014 at 11:07 PM, Stephen J. Turnbull
wrote:
> I'm referring to removing the unnecessary information that there's a
> better way to do it, and simply raising an error (as in Python 3.2,
> say) which is all a RealProgrammer[tm] should ever need!
>
I can't imagine anyone is sugges
Hi all,
The core of the matter is that if we repeatedly __add__ strings from a
long list, we get O(n**2) behavior. For one point of view, the
reason is that the additions proceed in left-to-right order. Indeed,
sum() could proceed in a more balanced tree-like order: from [x0, x1,
x2, x3, ...], r
On 12 Aug 2014 11:21, "Chris Barker - NOAA Federal"
wrote:
>
> Sorry for the bike shedding here, but:
>
>> The quadratic behaviour of repeated str summation is a subtle, silent
error.
>
> OK, fair enough. I suppose it would be hard and ugly to catch those
instances and raise an exception pointing
Ethan Furman writes:
> On 08/11/2014 08:50 PM, Stephen J. Turnbull wrote:
> > Chris Barker - NOAA Federal writes:
> >
> >> It seems pretty pedantic to say: we could make this work well,
> >> but we'd rather chide you for not knowing the "proper" way to do
> >> it.
> >
> > Nobody disagrees.
On 08/11/2014 08:50 PM, Stephen J. Turnbull wrote:
Chris Barker - NOAA Federal writes:
It seems pretty pedantic to say: we could make this work well,
but we'd rather chide you for not knowing the "proper" way to do
it.
Nobody disagrees. But backward compatibility gets in the way.
Something
Chris Barker - NOAA Federal writes:
> Is there anything in the language spec that says string concatenation is
> O(n^2)? Or for that matter any of the performs characteristics of build in
> types? Those striker as implementation details that SHOULD be particular to
> the implementation.
Conta
Sorry for the bike shedding here, but:
The quadratic behaviour of repeated str summation is a subtle, silent error.
OK, fair enough. I suppose it would be hard and ugly to catch those
instances and raise an exception pointing users to "".join.
*is* controversial that CPython silently optimises s
On Mon, Aug 11, 2014 at 8:19 PM, Nick Coghlan wrote:
> Teaching users the difference between linear time operations and quadratic
> ones isn't about purity, it's about passing along a fundamental principle
> of algorithm scalability.
I would understand if this was done in reduce(operator.add, .
On 12 Aug 2014 03:03, "Chris Barker - NOAA Federal"
wrote:
>
> My confusion is still this:
>
> Repeated summation of strings has been optimized in cpython even
> though it's not the recommended way to solve that problem.
The quadratic behaviour of repeated str summation is a subtle, silent
error.
On 8/11/2014 8:26 AM, Ben Hoyt wrote:
It seems to me this is something of a pointless discussion -- I highly
doubt the current situation is going to change, and it works very well.
Even if not perfect, sum() is for numbers, sep.join() for strings.
However, I will add one comment:
I'm overall
On 04.08.2014 22:22, Jim J. Jewett wrote:
>
>
>
> Sat Aug 2 12:11:54 CEST 2014, Julian Taylor wrote (in
> https://mail.python.org/pipermail/python-dev/2014-August/135623.html ) wrote:
>
>
>> Andrea Griffini wrote:
>
>>>However sum([[1,2,3],[4],[],[5,6]], []) concatenates the lists.
>
> I'm very sympathetic to Steven's explanation that "we
> wouldn't be having this discussion if we used a different operator for
> string concatenation".
Sure -- but just imagine the conversations we could be having instead
: what does bit wise and of a string mean? A bytes object? I cod see
it as
It seems to me this is something of a pointless discussion -- I highly
doubt the current situation is going to change, and it works very well.
Even if not perfect, sum() is for numbers, sep.join() for strings. However,
I will add one comment:
I'm overall -1 on trying to change the current situatio
Glenn Linderman writes:
> On 8/10/2014 1:24 AM, Stephen J. Turnbull wrote:
> > Actually ... if I were a fan of the "".join() idiom, I'd seriously
> > propose 0.sum(numeric_iterable) as the RightThang{tm]. Then we could
> > deprecate "".join(string_iterable) in favor of "".sum(string_iterable)
On Sun, 10 Aug 2014 13:12:26 -0700, Glenn Linderman
wrote:
> On 8/10/2014 1:24 AM, Stephen J. Turnbull wrote:
> > Actually ... if I were a fan of the "".join() idiom, I'd seriously
> > propose 0.sum(numeric_iterable) as the RightThang{tm]. Then we could
> > deprecate "".join(string_iterable) in
On Sun, 10 Aug 2014 13:12:26 -0700, Glenn Linderman
wrote:
> On 8/10/2014 1:24 AM, Stephen J. Turnbull wrote:
> > Actually ... if I were a fan of the "".join() idiom, I'd seriously
> > propose 0.sum(numeric_iterable) as the RightThang{tm]. Then we could
> > deprecate "".join(string_iterable) in
On 8/10/2014 1:24 AM, Stephen J. Turnbull wrote:
Actually ... if I were a fan of the "".join() idiom, I'd seriously
propose 0.sum(numeric_iterable) as the RightThang{tm]. Then we could
deprecate "".join(string_iterable) in favor of "".sum(string_iterable)
(with the same efficient semantics).
Act
On Aug 10, 2014, at 05:24 PM, Stephen J. Turnbull wrote:
>Actually ... if I were a fan of the "".join() idiom, I'd seriously
>propose 0.sum(numeric_iterable) as the RightThang{tm]. Then we could
>deprecate "".join(string_iterable) in favor of "".sum(string_iterable)
>(with the same efficient sema
Alexander Belopolsky writes:
> On Sat, Aug 9, 2014 at 3:08 AM, Stephen J. Turnbull
> wrote:
>
> > All the suggestions
> > I've seen so far are (IMHO, YMMV) just as ugly as the present
> > situation.
> >
>
> What is ugly about allowing strings? CPython certainly has a way to to
> make
On Sat, Aug 9, 2014 at 12:20 PM, Alexander Belopolsky
wrote:
>
> On Sat, Aug 9, 2014 at 1:08 AM, Steven D'Aprano wrote:
>>
>> We wouldn't be having
>> these interminable arguments about using sum() to concatenate strings
>> (and lists, and tuples) if the & operator was used for concatenation and
On Sat, Aug 9, 2014 at 1:08 AM, Steven D'Aprano wrote:
> We wouldn't be having
> these interminable arguments about using sum() to concatenate strings
> (and lists, and tuples) if the & operator was used for concatenation and
> + was only used for numeric addition.
>
But we would probably have a
On Sat, Aug 9, 2014 at 2:02 PM, Alexander Belopolsky <
alexander.belopol...@gmail.com> wrote:
> y='';for in in x; y+= x
Should have been
y=''
for i in x; y += i
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo
On Sat, Aug 9, 2014 at 3:08 AM, Stephen J. Turnbull
wrote:
> All the suggestions
> I've seen so far are (IMHO, YMMV) just as ugly as the present
> situation.
>
What is ugly about allowing strings? CPython certainly has a way to to
make sum(x, '') at least as efficient as y='';for in in x; y+= x
On 9 August 2014 06:08, Steven D'Aprano wrote:
> py> with Stopwatch():
> ... sum(carray) # carray is a numpy array of 7500 floats.
> ...
> 11250.0
> time taken: 52.659770 seconds
> py> with Stopwatch():
> ... numpy.sum(carray)
> ...
> 11250.0
> time taken: 0.161263 seconds
>
>
Alexander Belopolsky writes:
> Why have builtin sum at all if its use comes with so many caveats?
Because we already have it. If the caveats had been known when it was
introduced, maybe it wouldn't have been. The question is whether you
can convince python-dev that it's worth changing the defi
Le 09/08/2014 01:08, Steven D'Aprano a écrit :
On Fri, Aug 08, 2014 at 10:20:37PM -0400, Alexander Belopolsky wrote:
On Fri, Aug 8, 2014 at 8:56 PM, Ethan Furman wrote:
I don't use sum at all, or at least very rarely, and it still irritates me.
You are not alone. When I see sum([a, b, c]),
Steven D'Aprano wrote:
I've long believed that + is the wrong operator for concatenating
strings, and that & makes a much better operator.
Do you have a reason for preferring '&' in particular, or
do you just want something different from '+'?
Personally I can't see why "bitwise and" on string
On Fri, Aug 08, 2014 at 10:20:37PM -0400, Alexander Belopolsky wrote:
> On Fri, Aug 8, 2014 at 8:56 PM, Ethan Furman wrote:
>
> > I don't use sum at all, or at least very rarely, and it still irritates me.
>
>
> You are not alone. When I see sum([a, b, c]), I think it is a + b + c, but
> in Py
On Fri, Aug 8, 2014 at 8:56 PM, Ethan Furman wrote:
> I don't use sum at all, or at least very rarely, and it still irritates me.
You are not alone. When I see sum([a, b, c]), I think it is a + b + c, but
in Python it is 0 + a + b + c. If we had a "join" operator for strings
that is different
On 08/08/2014 05:34 PM, Raymond Hettinger wrote:
On Aug 8, 2014, at 11:09 AM, Ethan Furman mailto:et...@stoneleaf.us>> wrote:
So why not apply a similar optimization to sum() for strings?
That I cannot answer -- I find the current situation with sum highly irritating.
It is only irritatin
On Aug 8, 2014, at 11:09 AM, Ethan Furman wrote:
>> So why not apply a similar optimization to sum() for strings?
>
> That I cannot answer -- I find the current situation with sum highly
> irritating.
>
It is only irritating if you are misusing sum().
The str.__add__ optimization was put in
On 08/08/2014 08:23 AM, Chris Barker wrote:
So my final question is this:
repeated string concatenation is not the "recommended" way to do this -- but
nevertheless, cPython has an optimization
that makes it fast and efficient, to the point that there is no practical
performance reason to pref
On Thu, Aug 7, 2014 at 4:01 PM, Ethan Furman wrote:
> I don't remember where, but I believe that cPython has an optimization
> built in for repeated string concatenation, which is probably why you
> aren't seeing big differences between the + and the sum().
>
Indeed -- clearly so.
A little test
On 08/07/2014 04:01 PM, Ethan Furman wrote:
On 08/07/2014 03:06 PM, Chris Barker wrote:
--> timeit.Timer("for string in ['booya'] * 10: blah = blah + string", "blah =
''").repeat(3, 1)
[0.021117210388183594, 0.013692855834960938, 0.00768280029296875]
--> timeit.Timer("for string in ['booya
On 08/07/2014 04:01 PM, Ethan Furman wrote:
On 08/07/2014 03:06 PM, Chris Barker wrote:
the + and the sum().
Yeah, that 'sum' should be 'join' :/
--
~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo
On 08/07/2014 03:06 PM, Chris Barker wrote:
[snip timings, etc.]
I don't remember where, but I believe that cPython has an optimization built in for repeated string concatenation, which
is probably why you aren't seeing big differences between the + and the sum().
A little testing shows how t
On Mon, Aug 4, 2014 at 11:10 AM, Steven D'Aprano
wrote:
> On Mon, Aug 04, 2014 at 09:25:12AM -0700, Chris Barker wrote:
>
> > Good point -- I was trying to make the point about .join() vs + for
> strings
> > in an intro python class last year, and made the mistake of having the
> > students test
Sat Aug 2 12:11:54 CEST 2014, Julian Taylor wrote (in
https://mail.python.org/pipermail/python-dev/2014-August/135623.html ) wrote:
> Andrea Griffini wrote:
>>However sum([[1,2,3],[4],[],[5,6]], []) concatenates the lists.
> hm could this be a pure python case that would profit from
Steven D'Aprano schrieb am 04.08.2014 um 20:10:
> On Mon, Aug 04, 2014 at 09:25:12AM -0700, Chris Barker wrote:
>
>> Good point -- I was trying to make the point about .join() vs + for strings
>> in an intro python class last year, and made the mistake of having the
>> students test the performanc
On Mon, Aug 04, 2014 at 09:25:12AM -0700, Chris Barker wrote:
> Good point -- I was trying to make the point about .join() vs + for strings
> in an intro python class last year, and made the mistake of having the
> students test the performance.
>
> You need to concatenate a LOT of strings to see
On Sat, Aug 2, 2014 at 1:35 PM, David Wilson wrote:
> > Repeated list and str concatenation both have quadratic O(N**2)
> > performance, but people frequently build up strings with +
>
> join() isn't preferable in cases where it damages readability while
> simultaneously providing zero or negat
On Sat, Aug 02, 2014 at 05:39:12PM +1000, Steven D'Aprano wrote:
> Repeated list and str concatenation both have quadratic O(N**2)
> performance, but people frequently build up strings with + and rarely
> do the same for lists. String concatenation with + is an attractive
> nuisance for many peopl
On Sat, Aug 2, 2014 at 11:06 AM, Stefan Behnel wrote:
> I don't think sum(strings) is beautiful enough
sum(strings) is more beautiful than ''.join(strings) in my view, but
unfortunately it does not work even for lists because the initial value
defaults to 0.
sum(strings, '') and ''.join(string
On 2014-08-02 16:27, Steven D'Aprano wrote:
On Sat, Aug 02, 2014 at 10:52:07AM -0400, Alexander Belopolsky wrote:
On Sat, Aug 2, 2014 at 3:39 AM, Steven D'Aprano wrote:
> String concatenation with + is an attractive
> nuisance for many people, including some who actually know better but
> neve
On Sat, Aug 02, 2014 at 10:52:07AM -0400, Alexander Belopolsky wrote:
> On Sat, Aug 2, 2014 at 3:39 AM, Steven D'Aprano wrote:
>
> > String concatenation with + is an attractive
> > nuisance for many people, including some who actually know better but
> > nevertheless do it. Also, for reasons I d
Alexander Belopolsky schrieb am 02.08.2014 um 16:52:
> On Sat, Aug 2, 2014 at 3:39 AM, Steven D'Aprano wrote:
>
>> String concatenation with + is an attractive
>> nuisance for many people, including some who actually know better but
>> nevertheless do it. Also, for reasons I don't understand, many
On Sat, Aug 2, 2014 at 3:39 AM, Steven D'Aprano wrote:
> String concatenation with + is an attractive
> nuisance for many people, including some who actually know better but
> nevertheless do it. Also, for reasons I don't understand, many people
> dislike or cannot remember to use ''.join.
>
Sin
Julian Taylor schrieb am 02.08.2014 um 12:11:
> On 02.08.2014 08:35, Terry Reedy wrote:
>> On 8/2/2014 1:57 AM, Allen Li wrote:
>>> On Fri, Aug 01, 2014 at 02:51:54PM -0700, Guido van Rossum wrote:
No. We just can't put all possible use cases in the docstring. :-)
On Fri, Aug 1,
On 02.08.2014 08:35, Terry Reedy wrote:
> On 8/2/2014 1:57 AM, Allen Li wrote:
>> On Fri, Aug 01, 2014 at 02:51:54PM -0700, Guido van Rossum wrote:
>>> No. We just can't put all possible use cases in the docstring. :-)
>>>
>>>
>>> On Fri, Aug 1, 2014 at 2:48 PM, Andrea Griffini wrote:
>>>
>>>
On Fri, Aug 01, 2014 at 10:57:38PM -0700, Allen Li wrote:
> On Fri, Aug 01, 2014 at 02:51:54PM -0700, Guido van Rossum wrote:
> > No. We just can't put all possible use cases in the docstring. :-)
> >
> >
> > On Fri, Aug 1, 2014 at 2:48 PM, Andrea Griffini wrote:
> >
> > help(sum) tells cle
On 8/2/2014 1:57 AM, Allen Li wrote:
On Fri, Aug 01, 2014 at 02:51:54PM -0700, Guido van Rossum wrote:
No. We just can't put all possible use cases in the docstring. :-)
On Fri, Aug 1, 2014 at 2:48 PM, Andrea Griffini wrote:
help(sum) tells clearly that it should be used to sum numbers
On Fri, Aug 01, 2014 at 02:51:54PM -0700, Guido van Rossum wrote:
> No. We just can't put all possible use cases in the docstring. :-)
>
>
> On Fri, Aug 1, 2014 at 2:48 PM, Andrea Griffini wrote:
>
> help(sum) tells clearly that it should be used to sum numbers and not
> strings, and wi
No. We just can't put all possible use cases in the docstring. :-)
On Fri, Aug 1, 2014 at 2:48 PM, Andrea Griffini wrote:
> help(sum) tells clearly that it should be used to sum numbers and not
> strings, and with strings actually fails.
>
> However sum([[1,2,3],[4],[],[5,6]], []) concatenates
help(sum) tells clearly that it should be used to sum numbers and not
strings, and with strings actually fails.
However sum([[1,2,3],[4],[],[5,6]], []) concatenates the lists.
Is this to be considered a bug?
Andrea
___
Python-Dev mailing list
Python-De
We're still working on the implementation details for math.sum().
When it's finished, the cmath equilvalent will be added.
Raymond
- Original Message -
From: "A.M. Kuchling" <[EMAIL PROTECTED]>
To:
Sent: Thursday, June 19, 2008 7:16 PM
Subject: [Python-Dev
In the comments before the implementation of sum()
in mathmodule.c, note 4 says:
Note 4: A similar implementation is in Modules/cmathmodule.c.
Be sure to update both when making changes.
I can't find a sum implementation or similar code in cmathmodule.c.
Did someone intend to port the sum f
[Tim]
> For contrast, here's one that doesn't use frexp(), and is probably
> faster because of that; internally, len(sums) probably won't exceed 5
> in real life (but could get as large as 2K for pathological inputs,
> spanning fp's full dynamic range):
>
> def summer4(iterable):
> sums = [0.0
[Raymond Hettinger]
> The approach I'm using relies on being able to exactly multiply the 0 or
> 1 bit error term mantissas by an integer (a count of how many times the
> term occurs). With a Python dictionary keeping the counts, many steps
> are saved and the tool becomes much more memory friendl
You guys have way too much time on your hands and neurons devoted to
this stuff. I'm glad that means that I can spend the afternoon playing
w/ my kids and searching python-dev when I need to add numbers =).
___
Python-Dev mailing list
Python-Dev@python.or
> the queue can be expected to grow to about half the length
> of the original iterable by the time the original iterable is
> exhausted.
>
> >x = z
> >mant, exp = frexp(x)
> >exp2sum[exp] = x
> >return sum(sorted(exp2sum.itervalues(), key=abs), 0.0)
> >
> > The
[Raymond Hettinger]
> Computing an error term can get the bit back and putting that term back
> in the input queue restores the overall sum.
Right!
> Once the inputs are exhausted, the components of exp2sum should be exact.
Ditto. I'll cover some subtleties below:
> from math import frexp
> f
> [Tim Peters]
> Summer.add() _can_ lose info -- it needs additional
> trickery to make it loss-free, and because x+y can lose (at most one
> bit) when x and y have the same binary exponent.
Computing an error term can get the bit back and putting that term back
in the input queue restores the ove
[Tim Peters]
...
> One clever way is to build on that whenever |x| and |y| are within a
> factor of 2 of each other, x+y is exact in 754 arithmetic.
Ack, I'm fried. Forget that, it's wrong. The correct statement is
that x-y is always exact whenever x and y are within a factor of two
of each othe
FYI, there are a lot of ways to do accurate fp summation, but in
general people worry about it too much (except for those who don't
worry about it at all -- they're _really_ in trouble <0.5 wink>).
One clever way is to build on that whenever |x| and |y| are within a
factor of 2 of each other, x+y
[Alex]
> > FWIW, when accuracy is an issue, I use:
> >
> >sum(sorted(data, key=abs))
>
> ...and you may still lose a LOT of accuracy that way:-(.
>
> Raymond, you technically reviewed the 2nd ed Cookbook -- don't you
> recall the sidebar about why partials are the RIGHT way to do
> summations
On Mar 11, 2005, at 19:39, Raymond Hettinger wrote:
[Alex]
If you're considering revisions to sum's design, my suggestion would
be
to find a way to let the user tell sum to use a more accurate approach
when summing floats.
FWIW, when accuracy is an issue, I use:
sum(sorted(data, key=abs))
...and
[Alex]
> If you're considering revisions to sum's design, my suggestion would
be
> to find a way to let the user tell sum to use a more accurate approach
> when summing floats.
FWIW, when accuracy is an issue, I use:
sum(sorted(data, key=abs))
Raymond
__
72 matches
Mail list logo
