[Python-Dev] [ANN] VPython 0.1
Hi, I implemented a variant of the CPython VM on top of Gforth's Vmgen; this made it fairly straightforward to add direct threaded code and superinstructions for the various permutations of LOAD_CONST, LOAD_FAST, and most of the two-argument VM instructions. Sources: http://svirfneblin.org/stuff/VPython-0.1.tar.gz Pybench output: Test minimum run-timeaverage run-time thisother diffthisother diff --- BuiltinFunctionCalls: 104ms 126ms -17.3% 110ms 128ms -14.0% BuiltinMethodLookup:98ms 136ms -27.9% 100ms 139ms -28.1% CompareFloats:59ms 121ms -51.1%60ms 123ms -50.7% CompareFloatsIntegers:67ms 121ms -45.0%73ms 136ms -46.7% CompareIntegers:57ms 157ms -63.7%58ms 161ms -64.1% CompareInternedStrings:55ms 143ms -61.6%57ms 161ms -64.5% CompareLongs:56ms 122ms -54.0%61ms 141ms -56.7% CompareStrings:71ms 126ms -43.6%72ms 131ms -44.7% CompareUnicode:82ms 123ms -33.5%83ms 131ms -36.7% ConcatStrings: 119ms 146ms -18.4% 129ms 158ms -18.4% ConcatUnicode:90ms 109ms -17.2%97ms 121ms -19.8% CreateInstances: 116ms 124ms -6.6% 118ms 127ms -7.0% CreateNewInstances: 109ms 119ms -7.9% 113ms 121ms -6.6% CreateStringsWithConcat:97ms 162ms -40.3%99ms 169ms -41.5% CreateUnicodeWithConcat:90ms 116ms -22.8%97ms 122ms -20.8% DictCreation:87ms 122ms -28.6%91ms 127ms -28.0% DictWithFloatKeys:98ms 139ms -29.5% 105ms 148ms -29.3% DictWithIntegerKeys:71ms 133ms -46.7%74ms 136ms -46.0% DictWithStringKeys:62ms 126ms -51.0%64ms 128ms -50.3% ForLoops:68ms 135ms -49.2%69ms 136ms -49.2% IfThenElse:63ms 130ms -51.6%64ms 134ms -51.9% ListSlicing: 122ms 123ms -0.9% 126ms 125ms +0.8% NestedForLoops:89ms 149ms -40.2%93ms 152ms -38.9% NormalClassAttribute:88ms 132ms -33.1%95ms 134ms -29.5% NormalInstanceAttribute:72ms 116ms -37.9%77ms 118ms -34.8% PythonFunctionCalls:90ms 122ms -26.1%94ms 125ms -24.7% PythonMethodCalls: 117ms 144ms -18.8% 121ms 147ms -17.8% Recursion: 121ms 180ms -32.6% 124ms 184ms -32.4% SecondImport: 144ms 139ms +3.5% 150ms 143ms +4.8% SecondPackageImport: 151ms 145ms +3.9% 156ms 149ms +4.3% SecondSubmoduleImport: 178ms 168ms +5.8% 186ms 176ms +5.4% SimpleComplexArithmetic:71ms 112ms -36.7%76ms 123ms -38.3% SimpleDictManipulation:77ms 139ms -44.3%78ms 140ms -44.3% SimpleFloatArithmetic:61ms 124ms -50.7%63ms 126ms -50.2% SimpleIntFloatArithmetic:61ms 121ms -49.4%62ms 123ms -49.5% SimpleIntegerArithmetic:61ms 121ms -49.5%62ms 123ms -49.8% SimpleListManipulation:58ms 116ms -50.0%58ms 117ms -50.2% SimpleLongArithmetic:89ms 121ms -26.3%91ms 124ms -27.0% SmallLists:79ms 116ms -31.8%82ms 122ms -32.6% SmallTuples:91ms 117ms -22.6%93ms 122ms -23.6% SpecialClassAttribute:84ms 132ms -36.4%93ms 134ms -30.4% SpecialInstanceAttribute: 111ms 153ms -27.6% 114ms 155ms -26.2% StringMappings: 102ms 115ms -11.1% 104ms 117ms -10.9% StringPredicates: 100ms 136ms -26.7% 101ms 137ms -26.1% StringSlicing:79ms 114ms -30.2%84ms 119ms -29.9% TryExcept:68ms 145ms -53.2%69ms 148ms -53.6% TryRaiseExcept: 106ms 104ms +2.7% 109ms 106ms +2.8% TupleSlicing: 108ms 126ms -14.4% 113ms 130ms -13.0% UnicodeMappings: 150ms 150ms -0.4% 152ms 154ms -1.7% UnicodePredicates: 106ms 130ms -18.3% 108ms 133ms -18.3% UnicodeProperties:94ms 111ms -15.2%97ms 115ms -15.5% UnicodeSlicing: 101ms 130ms -22.5% 105ms 136ms -22.6% --- Totals: 4750ms 6788ms -30.0% 4929ms 7035ms -29.9% ``other'' is vanilla Python 2.5.2. For details, see README, TODO, and PYBENCH which come with the sources.
Re: [Python-Dev] [ANN] VPython 0.1
2008/10/22 J. Sievers <[EMAIL PROTECTED]>: > I implemented a variant of the CPython VM on top of Gforth's Vmgen; this made > it fairly straightforward to add direct threaded code and superinstructions > for > the various permutations of LOAD_CONST, LOAD_FAST, and most of the > two-argument > VM instructions. [...] > Totals: 4750ms 6788ms -30.0% 4929ms 7035ms > -29.9% > > ``other'' is vanilla Python 2.5.2. > For details, see README, TODO, and PYBENCH which come with the sources. > Feedback is, of course, very welcome and it'd be great to have some pybench > results from different machines. Am I reading this right? You got a 30% speed improvement over CPython??? (I never read pybench results right, so if this is actually 30% slower than CPython, let me know and I'll shut up :-)) If so, what (if any) limitations are there to the implementation? Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [ANN] VPython 0.1
On 2008-10-22 14:16, J. Sievers wrote: > Hi, > > I implemented a variant of the CPython VM on top of Gforth's Vmgen; this made > it fairly straightforward to add direct threaded code and superinstructions > for > the various permutations of LOAD_CONST, LOAD_FAST, and most of the > two-argument > VM instructions. I suppose you get most of the speedup by using threaded code. Unfortunately, that is only supported by gcc. Do you get similar results for the switch based method that appears to be available in VMgen ? http://www.complang.tuwien.ac.at/anton/vmgen/html-docs/VM-engine.html Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Oct 22 2008) >>> Python/Zope Consulting and Support ...http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-3000] Backporting multiprocessing?
[EMAIL PROTECTED] wrote: I checked in the contents of my multiprocessing.tar file and opened issues #1 and #2. I added a setup.py, disabled recv_bytes_into for now and fixed lots of naming issues. The multiprocessing code is using the new names of the threading module (current_thread, is_alive etc.) but Python 2.5 just have the old names (currentThread, isAlive). $ python2.5 setup.py build_ext -i $ PYTHONPATH=Lib python2.5 Lib/test/test_multiprocessing.py == ERROR: test_connection (__main__.WithProcessesTestConnection) -- Traceback (most recent call last): File "Lib/test/test_multiprocessing.py", line 1220, in test_connection self.assertEqual(conn.recv_bytes_into(buffer), AttributeError: '_multiprocessing.Connection' object has no attribute 'recv_bytes_into' -- Ran 123 tests in 12.309s FAILED (errors=1) :) Christian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-3000] Backporting multiprocessing?
Maybe we should backport those handy pep8 threading names ... ... Ok maybe not. On Oct 22, 2008 9:02am, Christian Heimes <[EMAIL PROTECTED]> wrote: [EMAIL PROTECTED] wrote: I checked in the contents of my multiprocessing.tar file and opened issues #1 and #2. I added a setup.py, disabled recv_bytes_into for now and fixed lots of naming issues. The multiprocessing code is using the new names of the threading module (current_thread, is_alive etc.) but Python 2.5 just have the old names (currentThread, isAlive). $ python2.5 setup.py build_ext -i $ PYTHONPATH=Lib python2.5 Lib/test/test_multiprocessing.py == ERROR: test_connection (__main__.WithProcessesTestConnection) -- Traceback (most recent call last): File "Lib/test/test_multiprocessing.py", line 1220, in test_connection self.assertEqual(conn.recv_bytes_into(buffer), AttributeError: '_multiprocessing.Connection' object has no attribute 'recv_bytes_into' -- Ran 123 tests in 12.309s FAILED (errors=1) :) Christian ___ Python-3000 mailing list [EMAIL PROTECTED] http://mail.python.org/mailman/listinfo/python-3000 Unsubscribe: http://mail.python.org/mailman/options/python-3000/jnoller%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [ANN] VPython 0.1
On Oct 22, 2008, at 10:16 AM, J. Sievers wrote: Hi, I implemented a variant of the CPython VM on top of Gforth's Vmgen; this made it fairly straightforward to add direct threaded code and superinstructions for the various permutations of LOAD_CONST, LOAD_FAST, and most of the two-argument VM instructions. Is it complete? I don't want to be rude or anything, but the idea I seem repeated a lot is that it is really easy to do a very fast VM that supports 95% of the language. How much of the standard python tests your VM pass? What is missing, and why? But besides that, it is always very cool to see another vm project starting, Good Luck -- Leonardo Santagada santagada at gmail.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] heapq, min and max
Hello there.
I was surprised to find recently that the heapq module is still a pure python
implementation.
A few years ago we wrote our own in C for use in Eve-online, and we usually do a
import blue.heapq as heapq.
Having a python implementation of it almost completely negates any benefit of
using that in place of sort() unles the comparison is really expensive.
I'd be happy to donate an implementation if there is any interest.
I also recently came across the recommendation that one should use the "min"
and "max" builtins instead of using heapq or sort() when trying to find
a single smallest element.
Well, this is also not true, for simple comparisons, because currently "min"
and "max" use the iterator protocol, whereas sort() (and blue.heapq.heapify)
use direct list access, thus halving the number of python method calls
approximately.
s="""
import random
l = [random.random() for i in xrange(1)]
"""
timeit.Timer("(l.sort(), l[-1])", s).timeit(1000)
0.29406761513791935
timeit.Timer("max(l)", s).timeit(1000)
0.4760155766743992
>>>
Now, this is easy enough to rectify, by providing a list specialization for
min_max(). This would also make sure that "min" is truly the fastest
way to find the minimum member of a list.
Would there be interest in this? (I will be patching the CCP version of python
for it anyway).
We are using 2.5.3, but these changes should be directly applicable to 2.6 too.
K
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] heapq, min and max
Kristján Valur Jónsson ccpgames.com> writes:
> timeit.Timer("(l.sort(), l[-1])",
> s).timeit(1000)
>
> 0.29406761513791935
This is clearly wrong. l.sort() will sort the list in place when it is first
invoked, and therefore will be very fast in subsequent calls.
Compare with:
timeit.Timer("sorted(l)[-1]", s).timeit(1000)
which really does what you are looking for (sorting then taking the max *on
every iteration*), and is much slower.
As for heapq, its critical parts are already implemented in C as of Python
2.5.2:
>>> import heapq, _heapq
>>> _heapq
>>> heapq.heapify is _heapq.heapify
True
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] heapq, min and max
Ooops, you are right. Silly error on my part. Still, my comment about heapq still stands, and [hack, slash] 0.39713821814841893 (old) 0.35184029691278162 (hakced, for special list treatment) So, there is a 12% performance boost to be had by specializing for lists. How about it? K > -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf > Of Antoine Pitrou > Sent: Wednesday, October 22, 2008 14:06 > To: python-dev@python.org > Subject: Re: [Python-Dev] heapq, min and max > > This is clearly wrong. l.sort() will sort the list in place when it is > first > invoked, and therefore will be very fast in subsequent calls. > ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] heapq, min and max
Ok. And sorry, I missed your part about heapq now having a c implementation. This is indeed good, I was misled by the presence of heapq.py. However, our own heapify() implementation is some 10% faster on a 1 element list of floats than the _heapq.heapify() version. I'll investigate the difference. K > -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf > Of Antoine Pitrou > Sent: Wednesday, October 22, 2008 14:41 > To: python-dev@python.org > Subject: Re: [Python-Dev] heapq, min and max > > Kristján Valur Jónsson ccpgames.com> writes: > > > > 0.39713821814841893 (old) > > 0.35184029691278162 (hakced, for special list treatment) > > > > So, there is a 12% performance boost to be had by specializing for > lists. > > How about it? > > It depends on the added code complexity. In any case, you should open > an entry > on the tracker and post your patch there. > > Regards > > Antoine. > > > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python- > dev/kristjan%40ccpgames.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-3000] Backporting multiprocessing?
Christian> I just implemented the recv_bytes_into function with the old Christian> buffer protocol. All tests are passing on my Linux box Christian> (Ubuntu 8.04 with gcc 4.2, AMD64 processor). Using Python v < 2.6? So I don't need to horse around making test_multiprocessing.py API compatible with processing 0.52? Skip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] heapq, min and max
Kristján Valur Jónsson ccpgames.com> writes: > > 0.39713821814841893 (old) > 0.35184029691278162 (hakced, for special list treatment) > > So, there is a 12% performance boost to be had by specializing for lists. > How about it? It depends on the added code complexity. In any case, you should open an entry on the tracker and post your patch there. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Backporting multiprocessing?
[EMAIL PROTECTED] wrote: Using Python v < 2.6? So I don't need to horse around making test_multiprocessing.py API compatible with processing 0.52? With Python 2.5.2 and 2.6.0 all tests are passing with any error. With Python 2.4.5 seven tests are failing because 2.4 doesn't support mmap with a negative file number. File ".../python-multiprocessing/Lib/multiprocessing/heap.py", line 56, in __init__ self.buffer = mmap.mmap(-1, size) EnvironmentError: [Errno 9] Bad file descriptor Christian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] heapq, min and max
I have submitted a patch for min() and max(): http://bugs.python.org/issue4174 > -Original Message- > > > > It depends on the added code complexity. In any case, you should open > > an entry > > on the tracker and post your patch there. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [ANN] VPython 0.1
J> I implemented a variant of the CPython VM on top of Gforth's Vmgen; this made J> it fairly straightforward to add direct threaded code and superinstructions for J> the various permutations of LOAD_CONST, LOAD_FAST, and most of the two-argument J> VM instructions. J> Sources: J> http://svirfneblin.org/stuff/VPython-0.1.tar.gz Very interesting. Trying to build with your changes on my Mac (OS X 10.5.5) I get an error caused by a definition in code.h: ... gcc -c -fno-strict-aliasing -Wno-long-double -no-cpp-precomp -mno-fused-madd -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I. -IInclude -I./Include -I/opt/local/include -DPy_BUILD_CORE -o Python/mactoolboxglue.o Python/mactoolboxglue.c In file included from /System/Library/Frameworks/Carbon.framework/Frameworks/HIToolbox.framework/Headers/ControlDefinitions.h:36, from /System/Library/Frameworks/Carbon.framework/Frameworks/HIToolbox.framework/Headers/HIToolbox.h:201, from /System/Library/Frameworks/Carbon.framework/Headers/Carbon.h:29, from Include/pymactoolbox.h:10, from Python/mactoolboxglue.c:27: /System/Library/Frameworks/Carbon.framework/Frameworks/HIToolbox.framework/Headers/Lists.h:50: error: conflicting types for 'Cell' Include/code.h:15: error: previous declaration of 'Cell' was here make: *** [Python/mactoolboxglue.o] Error 1 Note that I'm not actually doing a Framework build. Any way you can easily rename your macros to avoid obvious name clashes like this? Thanks, -- Skip Montanaro - [EMAIL PROTECTED] - http://www.webfast.com/~skip/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] heapq, min and max
Kristján Valur Jónsson wrote: Ok. And sorry, I missed your part about heapq now having a c implementation. This is indeed good, I was misled by the presence of heapq.py. Duplicate Python/C code will probably become more common. Even if the Python is not used for prototyping (which I believe it was for heapq), it serves to document the intention of the C code and to be a ready to go version for non-C implementations. And it can be used as a basis for modification by Pythoneers who want slightly different behavior. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [ANN] VPython 0.1
J> I implemented a variant of the CPython VM on top of Gforth's Vmgen; J> this made it fairly straightforward to add direct threaded code and J> superinstructions for the various permutations of LOAD_CONST, J> LOAD_FAST, and most of the two-argument VM instructions. Skip> Trying to build with your changes on my Mac (OS X 10.5.5) I get an Skip> error caused by a definition in code.h: I renamed Cell to _PyV_Cell and Inst to _PyV_Inst and got it to build. I get this pybench output on my Mac: --- PYBENCH 2.0 --- * using Python 2.5.3a0 * disabled garbage collection * system check interval set to maximum: 2147483647 * using timer: time.time --- Benchmark: py25t.pybench --- Rounds: 10 Warp: 10 Timer: time.time Machine Details: Platform ID: Darwin-9.5.0-i386-32bit Processor:i386 Python: Executable: /Users/skip/src/python/release25-maint/python.exe Version: 2.5.3a0 Compiler: GCC 4.0.1 (Apple Inc. build 5465) Bits: 32bit Build:Oct 22 2008 13:12:03 (#release25-maint:66444M) Unicode: UCS2 --- Comparing with: py25.pybench --- Rounds: 10 Warp: 10 Timer: time.time Machine Details: Platform ID: Darwin-9.5.0-i386-32bit Processor:i386 Python: Executable: /Users/skip/local/bin/python2.5 Version: 2.5.3a0 Compiler: GCC 4.0.1 (Apple Inc. build 5465) Bits: 32bit Build:Sep 13 2008 09:17:06 (#release25-maint:66444) Unicode: UCS2 Test minimum run-timeaverage run-time thisother diffthisother diff --- BuiltinFunctionCalls: 119ms 133ms -10.6% 121ms 141ms -14.1% BuiltinMethodLookup: 107ms 113ms -5.2% 108ms 116ms -6.5% CompareFloats:67ms92ms -27.7%68ms95ms -28.4% CompareFloatsIntegers:75ms92ms -17.9%76ms94ms -19.1% CompareIntegers:53ms89ms -39.8%54ms91ms -40.6% CompareInternedStrings:78ms 102ms -24.3%78ms 109ms -28.3% CompareLongs:64ms85ms -24.2%65ms85ms -23.5% CompareStrings:60ms77ms -22.0%62ms80ms -22.5% CompareUnicode:91ms 106ms -13.8%93ms 111ms -16.0% ConcatStrings: 140ms 141ms -0.3% 144ms 144ms +0.0% ConcatUnicode:83ms82ms +2.4%91ms85ms +6.6% CreateInstances: 138ms 145ms -5.1% 140ms 148ms -5.2% CreateNewInstances: 123ms 128ms -3.7% 125ms 129ms -3.5% CreateStringsWithConcat: 104ms 111ms -7.0% 105ms 258ms -59.1% CreateUnicodeWithConcat:81ms93ms -13.2%83ms 158ms -47.7% DictCreation: 104ms96ms +9.1% 106ms 102ms +4.0% DictWithFloatKeys:88ms 104ms -15.9%92ms 110ms -16.4% DictWithIntegerKeys:79ms 111ms -29.2%81ms 113ms -28.4% DictWithStringKeys:77ms90ms -13.8%84ms93ms -9.8% ForLoops:66ms78ms -15.2%67ms79ms -14.8% IfThenElse:57ms91ms -37.4%57ms91ms -37.4% ListSlicing: 119ms 120ms -0.4% 121ms 121ms -0.3% NestedForLoops:86ms98ms -12.0%87ms99ms -12.5% NormalClassAttribute: 113ms 111ms +2.1% 118ms 113ms +4.0% NormalInstanceAttribute:89ms 102ms -12.5%91ms 104ms -12.6% PythonFunctionCalls:89ms 106ms -15.8%91ms 108ms -15.0% PythonMethodCalls: 141ms 158ms -11.0% 149ms 161ms -6.9% Recursion: 122ms 137ms -11.2% 123ms 140ms -11.7% SecondImport:90ms91ms -0.3%91ms91ms -0.3% SecondPackageImport:95ms95ms +0.9%97ms9
Re: [Python-Dev] Backporting multiprocessing?
[EMAIL PROTECTED] wrote: Using Python v < 2.6? So I don't need to horse around making test_multiprocessing.py API compatible with processing 0.52? I've backported the Python 2.5 svn version of mmap to 2.4 and added it as multiprocessing._mmap25. The port is just a proof of concept and most like contains issues with ssize_t -> long transitions. But it's working. With the latest svn checkout all tests are passing for 2.4.5, 2.5.2 and 2.6.0 on my 64bit Ubuntu box. Somebody needs to test it on Windows, 32bit Linux and BSD. Christian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] heapq, min and max
I
Kt just occurred to me:
Even though l.sort() is sorting a presorted array, it still must be doing
1-1 RichCompares minimum, just like max. So how do we explain the
large difference?
> -Original Message-
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED] On Behalf
> Of Antoine Pitrou
> Sent: Wednesday, October 22, 2008 14:06
> To: python-dev@python.org
> Subject: Re: [Python-Dev] heapq, min and max
>
> Kristján Valur Jónsson ccpgames.com> writes:
> > timeit.Timer("(l.sort(), l[-1])",
> > s).timeit(1000)
> >
> > 0.29406761513791935
>
> This is clearly wrong. l.sort() will sort the list in place when it is
> first
> invoked, and therefore will be very fast in subsequent calls.
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] heapq, min and max
I think this should be taken off of python-dev until
you have some quality measurements,
know what's going on, and have an actionable idea.
Aside from list specialization versus a general iterator
protocol, there is no fat in the min/max implementation.
It loops, it compares, it returns.
If we wanted to go the distance and type specialize,
it is possible to use the same ideas that used in
Py2.6's builtin sum() to speed-up compares for particular types.
Personally, I think that would be a waste and would rather
keep the code simple.
Also, a primary use case for min/max is with just two inputs.
We don't want to slow that down in order to provide negligible
improvements to min/max for long sequences.
Raymond
- Original Message -
From: "Kristján Valur Jónsson" <[EMAIL PROTECTED]>
To: "Antoine Pitrou" <[EMAIL PROTECTED]>;
Sent: Wednesday, October 22, 2008 1:37 PM
Subject: Re: [Python-Dev] heapq, min and max
I
Kt just occurred to me:
Even though l.sort() is sorting a presorted array, it still must be doing
1-1 RichCompares minimum, just like max. So how do we explain the
large difference?
-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf
Of Antoine Pitrou
Sent: Wednesday, October 22, 2008 14:06
To: python-dev@python.org
Subject: Re: [Python-Dev] heapq, min and max
Kristján Valur Jónsson ccpgames.com> writes:
> timeit.Timer("(l.sort(), l[-1])",
> s).timeit(1000)
>
> 0.29406761513791935
This is clearly wrong. l.sort() will sort the list in place when it is
first
invoked, and therefore will be very fast in subsequent calls.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/python%40rcn.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] heapq, min and max
Raymond Hettinger rcn.com> writes: > > If we wanted to go the distance and type specialize, > it is possible to use the same ideas that used in > Py2.6's builtin sum() to speed-up compares for particular types. Would it be useful to create a macro that packs some of the optimizations in http://bugs.python.org/issue3106 , so that interested code can get the optimizations for free simply by using the macro? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] heapq, min and max
On Wed, Oct 22, 2008 at 3:51 PM, Raymond Hettinger <[EMAIL PROTECTED]> wrote: > Aside from list specialization versus a general iterator > protocol, there is no fat in the min/max implementation. > It loops, it compares, it returns. > > Also, a primary use case for min/max is with just two inputs. > We don't want to slow that down in order to provide negligible > improvements to min/max for long sequences. > If min/max were specialized for tuples, it would speed up the common case where a handful of arguments are passed in. Since tuples are immutable, evil comparison functions can't permute the input, either. I threw together a quick modification with tuple specialization, here are the results: 2.6-pristine 2.6-patched 0.38usec 0.243usec -36% two integers 284usec 198usec -30%5000 integers 0.455usec 0.308usec -32% two datetime objects 600usec536usec -11%5000 datetime objects Not bad :-) Raw commands and results are below: Cashew:~/src-other/Python-2.6-pristine$ ./python.exe Lib/timeit.py 'max(2,8)' 100 loops, best of 3: 0.38 usec per loop Cashew:/tmp/Python-2.6$ ./python.exe Lib/timeit.py 'max(2,8)' 100 loops, best of 3: 0.243 usec per loop Cashew:~/src-other/Python-2.6-pristine$ ./python.exe Lib/timeit.py -s 'x=tuple(xrange(5000))' 'max(x)' 1000 loops, best of 3: 284 usec per loop Cashew:/tmp/Python-2.6$ ./python.exe Lib/timeit.py -s 'x=tuple(xrange(5000))' 'max(x)' 1 loops, best of 3: 198 usec per loop Cashew:~/src-other/Python-2.6-pristine$ ./python.exe Lib/timeit.py -s 'import datetime' -s 'x = datetime.datetime.now()' -s 'y = datetime.datetime.now()' 'max(x,y)' 100 loops, best of 3: 0.455 usec per loop Cashew:/tmp/Python-2.6$ ./python.exe Lib/timeit.py -s 'import datetime' -s 'x = datetime.datetime.now()' -s 'y = datetime.datetime.now()' 'max(x,y)' 100 loops, best of 3: 0.308 usec per loop Cashew:~/src-other/Python-2.6-pristine$ ./python.exe Lib/timeit.py -s 'import datetime' -s 'x = tuple(datetime.datetime.now() for x in range(5000))' 'max(x)' 1000 loops, best of 3: 600 usec per loop Cashew:/tmp/Python-2.6$ ./python.exe Lib/timeit.py -s 'import datetime' -s 'x = tuple(datetime.datetime.now() for x in range(5000))' 'max(x)' 1000 loops, best of 3: 536 usec per loop (the one in /tmp/ is patched) -- Daniel Stutzbach, Ph.D. http://stutzbachenterprises.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Backporting multiprocessing?
The latest svn version is now working with Python 2.4.4, Python 2.5.2 and Python 2.6.0 on Linux (Ubuntu AMD64, Debian i386) and Windows XP. On Windows the multiprocessing module requires ctypes and pywin32 under Python 2.4.4. Some of the examples aren't working correctly under 2.4 and 2.5. Jesse is looking into it. Christian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [ANN] VPython 0.1
> Feedback is, of course, very welcome and it'd be great to have some pybench > results from different machines. My results are very similar to Jakob's. Gentoo Linux, 32-bit x86, Athlon 6400+ underclocked to 3.0 GHz. make test: 282 tests OK. 5 tests failed: test_doctest test_hotshot test_inspect test_subprocess test_trace --- PYBENCH 2.0 --- * using Python 2.5.2 (r252:60911, Oct 22 2008, 13:47:58) [GCC 4.1.2 20070214 ( (gdc 0.24, using dmd 1.020)) (Gentoo 4.1.2 p1.0.2)] * disabled garbage collection * system check interval set to maximum: 2147483647 * using timer: time.time Calibrating tests. Please wait... done. Running 10 round(s) of the suite at warp factor 10: * Round 1 done in 8.474 seconds. * Round 2 done in 8.389 seconds. * Round 3 done in 8.438 seconds. * Round 4 done in 8.411 seconds. * Round 5 done in 8.484 seconds. * Round 6 done in 8.471 seconds. * Round 7 done in 8.492 seconds. * Round 8 done in 8.549 seconds. * Round 9 done in 8.429 seconds. * Round 10 done in 8.542 seconds. --- Benchmark: 2008-10-22 20:45:22 --- Rounds: 10 Warp: 10 Timer: time.time Machine Details: Platform ID: Linux-2.6.26-gentoo-r1-i686-AMD_Athlon-tm-_64_X2_Dual_Core_Processor_6400+-with-glibc2.3 Processor: AMD Athlon(tm) 64 X2 Dual Core Processor 6400+ Python: Implementation: n/a Executable: /var/tmp/VPython-2.5.2/python Version:2.5.2 Compiler: GCC 4.1.2 20070214 ( (gdc 0.24, using dmd 1.020)) (Gentoo 4.1.2 p1.0.2) Bits: 32bit Build: Oct 22 2008 13:47:58 (#r252:60911) Unicode:UCS2 --- Comparing with: /tmp/vanilla252.pybench --- Rounds: 10 Warp: 10 Timer: time.time Machine Details: Platform ID: Linux-2.6.26-gentoo-r1-i686-AMD_Athlon-tm-_64_X2_Dual_Core_Processor_6400+-with-glibc2.3 Processor: AMD Athlon(tm) 64 X2 Dual Core Processor 6400+ Python: Implementation: n/a Executable: /usr/local/bin/python2.5 Version:2.5.2 Compiler: GCC 4.1.2 20070214 ( (gdc 0.24, using dmd 1.020)) (Gentoo 4.1.2 p1.0.2) Bits: 32bit Build: Oct 22 2008 20:39:10 (#r252:60911) Unicode:UCS2 Test minimum run-timeaverage run-time thisother diffthisother diff --- BuiltinFunctionCalls: 144ms 205ms -30.1% 162ms 240ms -32.5% BuiltinMethodLookup: 164ms 222ms -26.2% 167ms 236ms -29.2% CompareFloats:90ms 211ms -57.5% 103ms 222ms -53.7% CompareFloatsIntegers:88ms 182ms -51.4% 107ms 200ms -46.6% CompareIntegers:63ms 258ms -75.5%84ms 272ms -69.1% CompareInternedStrings:93ms 252ms -63.0% 103ms 261ms -60.5% CompareLongs:65ms 180ms -63.9%87ms 203ms -57.1% CompareStrings: 113ms 211ms -46.5% 120ms 218ms -44.9% CompareUnicode: 187ms 273ms -31.7% 228ms 290ms -21.4% ComplexPythonFunctionCalls: 261ms 330ms -20.9% 277ms 336ms -17.5% ConcatStrings: 204ms 255ms -20.2% 209ms 297ms -29.7% ConcatUnicode: 143ms 118ms +20.3% 159ms 228ms -30.0% CreateInstances: 172ms 112ms +53.0% 187ms 211ms -11.5% CreateNewInstances: 165ms 100ms +65.0% 171ms 196ms -12.6% CreateStringsWithConcat: 141ms 133ms +5.8% 160ms 256ms -37.3% CreateUnicodeWithConcat: 145ms 126ms +14.8% 167ms 242ms -30.9% DictCreation: 129ms98ms +31.6% 131ms 184ms -28.8% DictWithFloatKeys: 185ms 143ms +29.6% 216ms 268ms -19.6% DictWithIntegerKeys: 122ms 115ms +6.0% 126ms 227ms -44.4% DictWithStringKeys:92ms 112ms -17.6% 104ms 216ms -51.8% ForLoops:98ms 224ms -56.2% 117ms 243ms -52.0% IfThenElse:89ms 221ms -59.9%97ms 237ms -59.1% ListSlicing: 123ms 111ms +10.8% 131ms 141ms -6.8% NestedForLoops: 138ms 234ms -41.1% 153ms 262ms -41.6% NormalClassAttribute: 131ms 225ms -41.5% 139ms 243ms -42.9% NormalInstanceA
[Python-Dev] Python2.5 _sre deepcopy regression?
In version of Python prior to 2.5, it would appear that deepcopying
compiled regular expressions worked by accident:
2.4:
>>> copy.deepcopy(re.compile(''))
<_sre.SRE_Pattern object at 0xb7d53ef0>
2.5:
>>> copy.deepcopy(re.compile(''))
Traceback (most recent call last):
File "", line 1, in
File "/usr/lib/python2.5/copy.py", line 173, in deepcopy
y = copier(memo)
TypeError: cannot deepcopy this pattern object
I say "by accident", since the SRE_Pattern object in 2.4 has
a __deepcopy__ method which raises the "cannot deepcopy this pattern
object" TypeError, however this method isn't found by copy.deepcopy()
in 2.4, and copy.deepcopy() falls back to using the pickle logic.
The _sre source has #ifdef-out support for __deepcopy__, issue 416670
has the gory details:
http://bugs.python.org/issue416670
Changeset 38430 on the release24-maint branch introduced the changes
that stopped SRE_Pattern.__deepcopy__ being found. r38430 was a patch
forward ported from 2.3, but never ported to the trunk (probably a good
thing, too).
Thoughts?
--
Andrew McNamara, Senior Developer, Object Craft
http://www.object-craft.com.au/
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [ANN] VPython 0.1
David Ripton wrote: Feedback is, of course, very welcome and it'd be great to have some pybench results from different machines. My results are very similar to Jakob's. From looking thru the vmgen manual, there are two things it is doing that CPython is not. 1. gcc-specific threaded code; claim doubles speed. 2. caching top of stack in a register; claim increases speed 0-40%, depending on system. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
