Re: [Python-Dev] PATCH submitted: Speed up + for string concatenation, now as fast as "".join(x) idiom
> I've never liked the "".join([]) idiom for string concatenation; in my > opinion it violates the principles "Beautiful is better than ugly." and > "There should be one-- and preferably only one --obvious way to do it.". > (And perhaps several others.) To that end I've submitted patch #1569040 > to SourceForge: > > http://sourceforge.net/tracker/index.php?func=detail&aid=1569040&group_id=5470&atid=305470 > This patch speeds up using + for string concatenation. yay! i'm glad to see this. i hate the "".join syntax. i still write that as string.join() because thats at least readable). it also fixes the python idiom for fast string concatenation as intended; anyone whos ever written code that builds a large string value by pushing substrings into a list only to call join later should agree. mystr = "prefix" while bla: #... mystr += moredata is much nicer to read than mystr = "prefix" strParts = [mystr] while bla: #... strParts.append(moredata) mystr = "".join(strParts) have you run any generic benchmarks such as pystone to get a better idea of what the net effect on "typical" python code is? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PATCH submitted: Speed up + for string concatenation, now as fast as "".join(x) idiom
"Gregory P. Smith" <[EMAIL PROTECTED]> wrote: > > > I've never liked the "".join([]) idiom for string concatenation; in my > > opinion it violates the principles "Beautiful is better than ugly." and > > "There should be one-- and preferably only one --obvious way to do it.". > > (And perhaps several others.) To that end I've submitted patch #1569040 > > to SourceForge: > > > > http://sourceforge.net/tracker/index.php?func=detail&aid=1569040&group_id=5470&atid=305470 > > This patch speeds up using + for string concatenation. > > yay! i'm glad to see this. i hate the "".join syntax. i still write > that as string.join() because thats at least readable). it also fixes > the python idiom for fast string concatenation as intended; anyone > whos ever written code that builds a large string value by pushing > substrings into a list only to call join later should agree. > > mystr = "prefix" > while bla: > #... > mystr += moredata Regardless of "nicer to read", I would just point out that Guido has stated that Python will not have strings implemented as trees. Also, Python 3.x will have a data type called 'bytes', which will be the default return of file.read() (when files are opened as binary), which uses an over-allocation strategy like lists to get relatively fast concatenation (on the order of lst1 += lst2). - Josiah ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PATCH submitted: Speed up + for string concatenation, now as fast as "".join(x) idiom
On 5 Oct 2006, at 20:28, Gregory P. Smith wrote: >> I've never liked the "".join([]) idiom for string concatenation; >> in my >> opinion it violates the principles "Beautiful is better than >> ugly." and >> "There should be one-- and preferably only one --obvious way to do >> it.". >> (And perhaps several others.) To that end I've submitted patch >> #1569040 >> to SourceForge: >> >> http://sourceforge.net/tracker/index.php? >> func=detail&aid=1569040&group_id=5470&atid=305470 >> This patch speeds up using + for string concatenation. > > yay! i'm glad to see this. i hate the "".join syntax. Here here. Being able to write what you mean and have the language get decent performance none the less seems to me to be a "good thing". > have you run any generic benchmarks such as pystone to get a better > idea of what the net effect on "typical" python code is? Yeah, "real world" performance testing is always important with anything that uses lazy evaluation. If you get to control if and when the computation actually happens you have even more scope than usual for getting the benchmark answer you want to see! Cheers, Nicko ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PATCH submitted: Speed up + for string concatenation, now as fast as "".join(x) idiom
Gregory P. Smith wrote: > have you run any generic benchmarks such as pystone to get a better > idea of what the net effect on "typical" python code is? I hadn't, but I'm happy to. On my machine (a fire-breathing Athlon 64 x2 4400+), best of three runs: Python 2.5 release: Pystone(1.1) time for 5 passes = 1.01757 This machine benchmarks at 49136.8 pystones/second Python 2.5 concat: Pystone(1.1) time for 5 passes = 0.963191 This machine benchmarks at 51910.8 pystones/second I'm surprised by this; I had expected it to be slightly *slower*, not the other way 'round. I'm not sure why this is. A cursory glance at pystone.py doesn't reveal any string concatenation using +, so I doubt it's benefiting from my speedup. And I didn't change the optimization flags when I compiled Python, so that should be the same. Josiah Carlson wrote: > Regardless of "nicer to read", I would just point out that Guido has > stated that Python will not have strings implemented as trees. > I suspect it was more a caution that Python wouldn't *permanently* store strings as "ropes". In my patch, the rope only exists until someone asks for the string's value, at which point the tree is rendered and dereferenced. From that point on the object is exactly like a normal PyStringObject to the external viewer. But you and I are, as I believe the saying goes, "channeling Guido (badly)". Perhaps some adult supervision will intervene soon and make its opinions known. For what it's worth, I've realized two things I want to change about my patch: * I left in a couple of /* lch */ comments I used during development as markers to find my own code. Whoops; I'll strip those out. * I realized that, because of struct packing, all PyStringObjects are currently wasting an average of two bytes apiece. (As in, that's something Python 2.5 does, not something added by my code.) I'll change my patch so strings are allocated more precisely. If my string concatenation patch is declined, I'll be sure to submit this patch separately. I'll try to submit an updated patch today. Cheers, /larry/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PATCH submitted: Speed up + for string concatenation, now as fast as "".join(x) idiom
Gregory P. Smith wrote:
>>I've never liked the "".join([]) idiom for string concatenation; in my
>>opinion it violates the principles "Beautiful is better than ugly." and
>>"There should be one-- and preferably only one --obvious way to do it.".
>>(And perhaps several others.) To that end I've submitted patch #1569040
>>to SourceForge:
>>
>>http://sourceforge.net/tracker/index.php?func=detail&aid=1569040&group_id=5470&atid=305470
>>This patch speeds up using + for string concatenation.
>
>
> yay! i'm glad to see this. i hate the "".join syntax. i still write
> that as string.join() [...]
instance.method(*args) <==> type.method(instance, *args)
You can nowadays spell this as str.join("", lst) - no need to import a
whole module!
regards
Steve
--
Steve Holden +44 150 684 7255 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://holdenweb.blogspot.com
Recent Ramblings http://del.icio.us/steve.holden
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PATCH submitted: Speed up + for string concatenation, now as fast as "".join(x) idiom
Steve Holden wrote:
> instance.method(*args) <==> type.method(instance, *args)
>
> You can nowadays spell this as str.join("", lst) - no need to import a
> whole module!
except that str.join isn't polymorphic:
>>> str.join(u",", ["1", "2", "3"])
Traceback (most recent call last):
File "", line 1, in
TypeError: descriptor 'join' requires a 'str' object but received a
'unicode'
>>> string.join(["1", "2", "3"], u",")
u'1,2,3'
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
