Re: [Python-Dev] "error: None" when building extensions under 2.6

2008-08-24 Thread Amaury Forgeot d'Arc
Hello,

2008/8/23 Fredrik Lundh <[EMAIL PROTECTED]>:
> when I'm trying to build extensions under Python 2.6 on Windows XP, the
> build process terminates with single line that says:
>
>error: None
>
> which is about as useless as an error message can be.  Googling for this
> brings up a few hits which all seem to involve setuptools (and none of the
> hits contain any answers), but I'm using straightforward distutils stuff
> that has worked without a glitch since 1.5.2.  Has anyone else seen this for
> 2.6?

I tried to rebuild some extensions, and it either succeeds,
or fails with a meaningful (for an expert) error message.

Which extensions did you try? is there some output before the error?

-- 
Amaury Forgeot d'Arc
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] "error: None" when building extensions under 2.6

2008-08-24 Thread Fredrik Lundh

Amaury Forgeot d'Arc wrote:


when I'm trying to build extensions under Python 2.6 on Windows XP, the
build process terminates with single line that says:

   error: None

which is about as useless as an error message can be.  Googling for this
brings up a few hits which all seem to involve setuptools (and none of the
hits contain any answers), but I'm using straightforward distutils stuff
that has worked without a glitch since 1.5.2.  Has anyone else seen this for
2.6?


I tried to rebuild some extensions, and it either succeeds,
or fails with a meaningful (for an expert) error message.

Which extensions did you try? is there some output before the error?


pick any:

http://effbot.org/downloads/

;-)

I'm beginning to suspect that I have a botched VS install on this 
machine, though.  I'll investigate.




___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] "error: None" when building extensions under 2.6

2008-08-24 Thread Thomas Heller
Fredrik Lundh schrieb:
> Amaury Forgeot d'Arc wrote:
> 
>>> when I'm trying to build extensions under Python 2.6 on Windows XP, the
>>> build process terminates with single line that says:
>>>
>>>error: None
>>>
>>> which is about as useless as an error message can be.  Googling for this
>>> brings up a few hits which all seem to involve setuptools (and none of the
>>> hits contain any answers), but I'm using straightforward distutils stuff
>>> that has worked without a glitch since 1.5.2.  Has anyone else seen this for
>>> 2.6?
>> 
>> I tried to rebuild some extensions, and it either succeeds,
>> or fails with a meaningful (for an expert) error message.
>> 
>> Which extensions did you try? is there some output before the error?
> 
> pick any:
> 
> http://effbot.org/downloads/
> 
> ;-)
> 
> I'm beginning to suspect that I have a botched VS install on this 
> machine, though.  I'll investigate.
> 

Do you get a traceback when you set the DISTUTILS_DEBUG environment variable?

Thomas

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] current problems that should be fixed before release (2.6)

2008-08-24 Thread skip
Neal>   Lib/gzip.py:114: Using property (filename) in classic class GzipFile
Neal> may not work

Seems shallow.  Just inherit from object.

Neal>   Lib/sched.py:122: Using property (queue) in classic class scheduler
Neal> may not work

Ditto.

Neal>   Lib/tempfile.py:535: Using property (closed) in classic class
Neal> SpooledTemporaryFile may not work
...

Ditto.

Neal>   Lib/multiprocessing/connection.py:378: No global 
(AuthenticationError) found

This is defined in the __init__ module of the package.  Is it accessible
from connection.py without further import?  (My guess would be no.)  A
slight code rearrangement or adding imports at the point of the raise
statements probably fixes that.

Neal> I haven't looked into the property complaint to see how bad it is.

I'm testing out the inherit-from-object fixes now.  Will report back later.

Skip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] performance

2008-08-24 Thread Antoine Pitrou
Neal Norwitz  gmail.com> writes:
> Can someone (else) compare performance of 2.5, 2.6, and 3.0?

Tests done on a 32-bit Linux installation on an Athlon 3600+ X2. Compiled with
gcc in UCS2 mode.

pystone
---

- 2.5: 43859.6 pystones/second
- 2.6: 42016.8 pystones/second 
- 3.0: 38759.7 pystones/second

richards.py
---

(roughly, richards is an object-oriented method-calling benchmark)

- 2.5: 770.54 ms per iteration
- 2.6: 572.84 ms per iteration
- 3.0: 566.69 ms per iteration

stringbench
---

- 2.5: unicode: 265.84 s / bytes: 180.50 s
- 2.6: unicode: 256.22 s / bytes: 184.45 s
- 3.0: unicode: 248.07 s / bytes: not tested

pybench: 2.6 vs. 2.5


"this" is 2.6, "other" is 2.5.

Test minimum run-timeaverage  run-time
 thisother   diffthisother   diff
---
  BuiltinFunctionCalls:   177ms   181ms   -2.2%   178ms   182ms   -2.1%
   BuiltinMethodLookup:   157ms   177ms  -11.1%   158ms   177ms  -11.0%
 CompareFloats:   171ms   171ms   -0.3%   171ms   171ms   -0.3%
 CompareFloatsIntegers:   158ms   162ms   -2.1%   160ms   167ms   -4.8%
   CompareIntegers:   208ms   208ms   -0.0%   209ms   209ms   -0.3%
CompareInternedStrings:   189ms   183ms   +3.4%   189ms   184ms   +2.7%
  CompareLongs:   154ms   154ms   -0.2%   154ms   154ms   -0.2%
CompareStrings:   162ms   159ms   +1.7%   163ms   159ms   +2.5%
CompareUnicode:   144ms   149ms   -3.6%   144ms   152ms   -5.0%
ComplexPythonFunctionCalls:   173ms   242ms  -28.5%   176ms   243ms  -27.6%
 ConcatStrings:   196ms   198ms   -0.9%   203ms   200ms   +1.4%
 ConcatUnicode:   153ms   151ms   +1.5%   155ms   157ms   -1.5%
   CreateInstances:   172ms   169ms   +1.5%   173ms   170ms   +1.4%
CreateNewInstances:   131ms   148ms  -11.6%   132ms   151ms  -12.2%
   CreateStringsWithConcat:   209ms   206ms   +1.4%   209ms   208ms   +0.8%
   CreateUnicodeWithConcat:   128ms   124ms   +3.7%   129ms   124ms   +3.4%
  DictCreation:   115ms   149ms  -22.8%   116ms   150ms  -22.8%
 DictWithFloatKeys:   208ms   207ms   +0.4%   208ms   208ms   +0.0%
   DictWithIntegerKeys:   173ms   173ms   -0.0%   174ms   173ms   +0.5%
DictWithStringKeys:   162ms   162ms   -0.1%   162ms   162ms   +0.1%
  ForLoops:   181ms   181ms   -0.2%   181ms   182ms   -0.2%
IfThenElse:   169ms   168ms   +0.2%   169ms   169ms   -0.2%
   ListSlicing:   109ms   108ms   +0.2%   109ms   109ms   +0.1%
NestedForLoops:   198ms   197ms   +0.2%   198ms   197ms   +0.1%
  NormalClassAttribute:   176ms   172ms   +2.0%   176ms   173ms   +1.7%
   NormalInstanceAttribute:   162ms   161ms   +1.0%   163ms   161ms   +1.1%
   PythonFunctionCalls:   161ms   151ms   +6.6%   162ms   153ms   +6.0%
 PythonMethodCalls:   188ms   189ms   -0.7%   189ms   193ms   -2.0%
 Recursion:   235ms   230ms   +2.2%   236ms   233ms   +1.2%
  SecondImport:   113ms   114ms   -1.0%   115ms   115ms   +0.0%
   SecondPackageImport:   120ms   116ms   +3.6%   120ms   117ms   +2.9%
 SecondSubmoduleImport:   154ms   146ms   +6.0%   156ms   148ms   +5.7%
   SimpleComplexArithmetic:   163ms   151ms   +8.3%   163ms   151ms   +8.6%
SimpleDictManipulation:   177ms   173ms   +2.6%   180ms   174ms   +3.2%
 SimpleFloatArithmetic:   169ms   164ms   +3.0%   169ms   166ms   +2.2%
  SimpleIntFloatArithmetic:   153ms   152ms   +1.2%   156ms   153ms   +1.8%
   SimpleIntegerArithmetic:   156ms   152ms   +2.5%   156ms   153ms   +2.0%
SimpleListManipulation:   156ms   158ms   -1.0%   157ms   158ms   -0.5%
  SimpleLongArithmetic:   159ms   150ms   +6.1%   161ms   151ms   +6.3%
SmallLists:   156ms   154ms   +1.1%   160ms   155ms   +3.0%
   SmallTuples:   156ms   155ms   +0.4%   157ms   156ms   +0.8%
 SpecialClassAttribute:   173ms   172ms   +0.6%   173ms   172ms   +0.7%
  SpecialInstanceAttribute:   202ms   198ms   +2.4%   203ms   199ms   +2.2%
StringMappings:   164ms   170ms   -3.2%   165ms   171ms   -3.9%
  StringPredicates:   160ms   185ms  -13.6%   160ms   186ms  -13.9%
 StringSlicing:   169ms   178ms   -5.3%   174ms   180ms   -3.4%
 TryExcept:   181ms   184ms   -1.5%   181ms   184ms   -1.5%
TryFinally:   157ms   158ms   -0.3%   159ms   161ms   -1.2%
TryRaiseExcept:   183ms   122ms  +49.6%   184ms   124ms  +48.2%
  TupleSlicing:   142ms   140ms   +1.5%   144ms   141ms   +2.0%
   UnicodeMappings:   198ms 

Re: [Python-Dev] current problems that should be fixed before release (2.6)

2008-08-24 Thread skip

me> Seems shallow.  Just inherit from object.
...
me> I'm testing out the inherit-from-object fixes now.

They all passed.  Here's the patch:

http://bugs.python.org/issue3658

In addition, I get these two test failures on trunk:

% ./python.exe ../Lib/test/regrtest.py test_distutils test_multiprocessing
test_distutils
test test_distutils failed -- Traceback (most recent call last):
  File 
"/Users/skip/src/python/trunk/Lib/distutils/tests/test_build_ext.py", line 23, 
in setUp
shutil.copy(xx_c, self.tmp_dir)
  File "/Users/skip/src/python/trunk/Lib/shutil.py", line 88, in copy
copyfile(src, dst)
  File "/Users/skip/src/python/trunk/Lib/shutil.py", line 52, in copyfile
fsrc = open(src, 'rb')
IOError: [Errno 2] No such file or directory: 
'/Users/skip/src/python/trunk/regular/Modules/xxmodule.c'

test_multiprocessing
test test_multiprocessing failed -- Traceback (most recent call last):
  File "/Users/skip/src/python/trunk/Lib/test/test_multiprocessing.py", 
line 1223, in test_connection
res = conn.recv_bytes_into(buffer)
TypeError: argument 1 must be pinned buffer, not bytearray

2 tests failed:
test_distutils test_multiprocessing

The former is almost certainly because I'm building in a subdirectory of my
trunk sandbox.  (Doesn't anybody else build that way these days?  I
routinely encounter problems with this setup.)  Looks like
distutils.sysconfig.project_base is set incorrectly:

% ./python.exe 
Python 2.6b3+ (trunk:66009M, Aug 24 2008, 07:17:12) 
[GCC 4.0.1 (Apple Inc. build 5465)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> import sys
>>> project_base = os.path.dirname(os.path.abspath(sys.executable))
>>> project_base
'/Users/skip/src/python/trunk/regular'

I think project_base should be /Users/skip/src/python/trunk, not
.../trunk/regular.

Skip

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] Things to Know About Super

2008-08-24 Thread [EMAIL PROTECTED]
Recently I have opened a blog on Artima and I am publishing a
few Python-related essays I had in store. In particular a trilogy
of papers about "super". From the foreword:

"""
In 2004 I decided to write a comprehensive paper documenting
``super`` pitfalls and traps, with the goal of publishing it on the
Python web site, just as I had published my essay on multiple
inheritance and the `Method Resolution Order`_. With time the paper
grew longer and longer but I never had the feeling that I had covered
everything I needed to say: moreover I have a full time job, so I
never had the time to fully revise the paper as a whole. As a
consequence, four years have passed and the paper is still in draft
status. This is a pity, since it documents issues that people
encounter and that regularly come out on the Python newsgroups and
forums.

Keeping the draft sitting on my hard disk is doing a disservice to the
community. Still, I lack to time to finish it properly. To come out
from the impasse, I decided to split the long paper in a series of
short blog posts, which I do have the time to review properly.
Moreover people are free to post comments and corrections in case I am
making
mistakes (speaking about ``super`` this is always possible). Once I
finish the series, I may integrate the corrections, put it together
again and possibly publish it as whole on the Python website.
In other words, in order to finish the task,
I am trying the strategies of *divide et conquer*
and *release early, release often*. We will see how it goes.
"""

It seems the strategy worked since I have finished
the paper, by splitting it in three post. Actually, it it not
really finished because it does not say anything about
the new super in Python 3.0, but that could do in a separate
essay. Here I am asking for feedback/corrections.
Moreover if people think it is worthy idea, I can re-assemble
the paper again, donate it to the PSF and publish it on the
Python website.

Michele Simionato
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] "error: None" when building extensions under 2.6

2008-08-24 Thread Fredrik Lundh

Thomas Heller wrote:

I'm beginning to suspect that I have a botched VS install on this 
machine, though.  I'll investigate.


Do you get a traceback when you set the DISTUTILS_DEBUG environment

> variable?

Indeed I do:

  ...
  File "c:\python26\lib\distutils\msvc9compiler.py", line 436, in compile
self.initialize()
  File "c:\python26\lib\distutils\msvc9compiler.py", line 347, in 
initialize

vc_env = query_vcvarsall(VERSION, plat_spec)
  File "c:\python26\lib\distutils\msvc9compiler.py", line 239, in 
query_vcvarsall

raise IOError("Unable to find vcvarsall.bat")

Oops.  Guess that's what you get for having too many Windows development 
boxes ;-)


(a nicer error message in non-debug mode would be a good thing, I think.)



PS. Can any resident Microsoft compiler expert perhaps summarize the 
differences between the Express Edition and the "real" editions wrt. 
generated code?  Are there any differences at all?


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] "error: None" when building extensions under 2.6

2008-08-24 Thread Christian Heimes

Fredrik Lundh wrote:
PS. Can any resident Microsoft compiler expert perhaps summarize the 
differences between the Express Edition and the "real" editions wrt. 
generated code?  Are there any differences at all?


The express edition does neither support PGO (profile guided 
optimization) nor cross compilation of 64bit binaries on 32bit 
platforms. The other missing features are mostly uninteresting from the 
PoV of a extension developer.


From wikipedia:
Visual C++ 2008 Express does not include OpenMP support, 64-bit 
compilers, or a resource editor. The higher-end commercial editions of 
Visual Studio, specifically the Professional and Team Suite editions, 
have these features.


From MS:
http://msdn.microsoft.com/en-us/library/hs24szh9.aspx

Christian

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Things to Know About Super

2008-08-24 Thread Matt Giuca
Hi Michele,

Do you have a URL for this blog?

Matt
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] Fwd: Things to Know About Super

2008-08-24 Thread Matt Giuca
Had a brief offline discussion with Michele - forwarding.

-- Forwarded message --
From: [EMAIL PROTECTED] <[EMAIL PROTECTED]>
Date: Mon, Aug 25, 2008 at 12:13 AM

On Aug 24, 3:43 pm, "Matt Giuca" <[EMAIL PROTECTED]> wrote:
> Hi Michele,
>
> Do you have a URL for this blog?

Sorry, here it is:
http://www.artima.com/weblogs/index.jsp?blogger=micheles

-- Forwarded message --
From: Matt Giuca <[EMAIL PROTECTED]>
Date: Mon, Aug 25, 2008 at 1:15 AM

I skimmed (will read in detail later). As an "intermediate" (I'll describe
myself as) Python developer, I tend not to use/understand super (I just call
baseclassname.methodname(self,...) directly, so I guess I'm the target
audience of this article. It's good - very informative and thorough.

It's a bit too informal, personal, and opinionative to be used as
"documentation" IMHO but it could certainly be cleaned up without being
rewritten.

Of interest though, is this:
"The first sentence is just plain wrong: super does not return the
superclass."

>From what I remember of using super, this statement is true, and the
documentation is wrong (or at least over-simplifies things).
http://docs.python.org/dev/library/functions.html#super
http://docs.python.org/dev/3.0/library/functions.html#super
Perhaps this should be amended? (A brief statement to the effect of super
creating a proxy object which can call the methods of any base class). It
actually mentions the "super object" later, even though it claims to be
returning the superclass.

Also Michele, looks as if super in Python 3 works about the same but has the
additional feature of supporting 0 arguments, in which case it defaults to
super(this_class, first_arg). (Does not create unbound super objects).

Matt
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] performance

2008-08-24 Thread Neal Norwitz
Thanks Antoine!

On Sun, Aug 24, 2008 at 5:58 AM, Antoine Pitrou <[EMAIL PROTECTED]> wrote:
> Neal Norwitz  gmail.com> writes:
>> Can someone (else) compare performance of 2.5, 2.6, and 3.0?
>
> Tests done on a 32-bit Linux installation on an Athlon 3600+ X2. Compiled with
> gcc in UCS2 mode.
>
> pystone
> ---
>
> - 2.5: 43859.6 pystones/second
> - 2.6: 42016.8 pystones/second
> - 3.0: 38759.7 pystones/second

So 3.0 is about 10% slower than 2.x.  Given all the changes, that
doesn't seem too bad.  Though, see below.  It looks like at least
class attribute lookup is much, much slower.  Can you investigate some
of these and file bugs as appropriate?

> richards.py
> ---
>
> (roughly, richards is an object-oriented method-calling benchmark)
>
> - 2.5: 770.54 ms per iteration
> - 2.6: 572.84 ms per iteration
> - 3.0: 566.69 ms per iteration

I'm a little concerned about why the big change here.  Though if I'm
reading this right it's a speed up...or am I just being optimistic?

The only things I remember that changed between 2.5 and 2.6 that might
affect this (without looking at any code) were:  1) Armin's method
caching, and 2) the slowdown to isinstance/issubclass which has a
release blocker:  http://bugs.python.org/issue2534

Can you dig into this and see what caused the slowdown?

>
> stringbench
> ---
>
> - 2.5: unicode: 265.84 s / bytes: 180.50 s
> - 2.6: unicode: 256.22 s / bytes: 184.45 s
> - 3.0: unicode: 248.07 s / bytes: not tested
>
> pybench: 2.6 vs. 2.5
> 
>
> "this" is 2.6, "other" is 2.5.

Hopefully anything >10% is a real change and not just noise.

> Test minimum run-timeaverage  run-time
> thisother   diffthisother   diff
> ---
>  BuiltinFunctionCalls:   177ms   181ms   -2.2%   178ms   182ms   -2.1%
>   BuiltinMethodLookup:   157ms   177ms  -11.1%   158ms   177ms  -11.0%

Maybe explained by Armin's patch.

> CompareFloats:   171ms   171ms   -0.3%   171ms   171ms   -0.3%
> CompareFloatsIntegers:   158ms   162ms   -2.1%   160ms   167ms   -4.8%
>   CompareIntegers:   208ms   208ms   -0.0%   209ms   209ms   -0.3%
>CompareInternedStrings:   189ms   183ms   +3.4%   189ms   184ms   +2.7%
>  CompareLongs:   154ms   154ms   -0.2%   154ms   154ms   -0.2%
>CompareStrings:   162ms   159ms   +1.7%   163ms   159ms   +2.5%
>CompareUnicode:   144ms   149ms   -3.6%   144ms   152ms   -5.0%
>ComplexPythonFunctionCalls:   173ms   242ms  -28.5%   176ms   243ms  -27.6%

Maybe explained by Armin's patch.

> ConcatStrings:   196ms   198ms   -0.9%   203ms   200ms   +1.4%
> ConcatUnicode:   153ms   151ms   +1.5%   155ms   157ms   -1.5%
>   CreateInstances:   172ms   169ms   +1.5%   173ms   170ms   +1.4%
>CreateNewInstances:   131ms   148ms  -11.6%   132ms   151ms  -12.2%

Maybe explained by Armin's patch.

>   CreateStringsWithConcat:   209ms   206ms   +1.4%   209ms   208ms   +0.8%
>   CreateUnicodeWithConcat:   128ms   124ms   +3.7%   129ms   124ms   +3.4%
>  DictCreation:   115ms   149ms  -22.8%   116ms   150ms  -22.8%

Why?  What changed?

> DictWithFloatKeys:   208ms   207ms   +0.4%   208ms   208ms   +0.0%
>   DictWithIntegerKeys:   173ms   173ms   -0.0%   174ms   173ms   +0.5%
>DictWithStringKeys:   162ms   162ms   -0.1%   162ms   162ms   +0.1%
>  ForLoops:   181ms   181ms   -0.2%   181ms   182ms   -0.2%
>IfThenElse:   169ms   168ms   +0.2%   169ms   169ms   -0.2%
>   ListSlicing:   109ms   108ms   +0.2%   109ms   109ms   +0.1%
>NestedForLoops:   198ms   197ms   +0.2%   198ms   197ms   +0.1%
>  NormalClassAttribute:   176ms   172ms   +2.0%   176ms   173ms   +1.7%
>   NormalInstanceAttribute:   162ms   161ms   +1.0%   163ms   161ms   +1.1%
>   PythonFunctionCalls:   161ms   151ms   +6.6%   162ms   153ms   +6.0%
> PythonMethodCalls:   188ms   189ms   -0.7%   189ms   193ms   -2.0%
> Recursion:   235ms   230ms   +2.2%   236ms   233ms   +1.2%
>  SecondImport:   113ms   114ms   -1.0%   115ms   115ms   +0.0%
>   SecondPackageImport:   120ms   116ms   +3.6%   120ms   117ms   +2.9%
> SecondSubmoduleImport:   154ms   146ms   +6.0%   156ms   148ms   +5.7%
>   SimpleComplexArithmetic:   163ms   151ms   +8.3%   163ms   151ms   +8.6%
>SimpleDictManipulation:   177ms   173ms   +2.6%   180ms   174ms   +3.2%
> SimpleFloatArithmetic:   169ms   164ms   +3.0%   169ms   166ms   +2.2%
>  SimpleIntFloatArithmetic:   153ms   152ms   +1.2%   156ms   153ms   +1.8%
>   SimpleIntegerArithmetic:   156ms   152ms   +2.5%   156ms   153ms   +2.0%
> 

Re: [Python-Dev] performance

2008-08-24 Thread Antoine Pitrou

Hi,

> So 3.0 is about 10% slower than 2.x.  Given all the changes, that
> doesn't seem too bad.

Yes, I think it's rather good.

> > - 2.5: 770.54 ms per iteration
> > - 2.6: 572.84 ms per iteration
> > - 3.0: 566.69 ms per iteration
> 
> I'm a little concerned about why the big change here.  Though if I'm
> reading this right it's a speed up...or am I just being optimistic?

Yes, it is a speed up. It is thanks to Armin's method cache.

> >ComplexPythonFunctionCalls:   173ms   242ms  -28.5%   176ms   243ms  
> > -27.6%
> 
> Maybe explained by Armin's patch.

Actually it is I and Raymond's patch speeding up function calls with named
parameters.

> >  DictCreation:   115ms   149ms  -22.8%   116ms   150ms  
> > -22.8%
> 
> Why?  What changed?

The opcode sequence for creation of dict literals has been optimized.

> >  StringPredicates:   160ms   185ms  -13.6%   160ms   186ms  
> > -13.9%
> 
> Maybe explained by Armin's patch.

I don't think so.

> >TryRaiseExcept:   183ms   122ms  +49.6%   184ms   124ms  
> > +48.2%
> 
> Whoa, that's a big slowdown.  I wonder if it's consistent?

Yes, I can definitely reproduce it.

> > UnicodePredicates:   157ms   175ms  -10.2%   157ms   176ms  
> > -10.6%
> 
> ?

Probably thanks to the speedup in whitespace detection.

> > CompareFloatsIntegers:274ms274ms0.30us0.630ms
> 
> Much slower, but probably due to switch from int -> long.  There could
> be potential for optimizing this case.

Well honestly you don't often compare different types. I think the most common
exception to this rule would be None vs. non-None.

> >   CompareIntegers:277ms277ms0.15us1.272ms
> 
> That's really slow!
> 
> >CompareInternedStrings:261ms261ms0.17us3.201ms
> 
> Much slower, but probably str -> unicode conversion.
> 
> >  CompareLongs:162ms162ms0.15us0.736ms
> 
> But this is comparable to 2.x.  I don't understand that?

I think CompareIntegers and CompareLongs aren't the same tests exactly, although
they test the same operations.

I've already proposed a patch to speedup comparisons in py3k:
http://bugs.python.org/issue3106

> > ConcatStrings:260ms273ms0.55us1.222ms
> 
> This is much slower in 3.0, even taking into account str -> unicode?

The number of rounds is not the same (6 for 2.6 with unicode, 10 for 3.0
with str). It should probably be fixed.

> >IfThenElse:212ms212ms0.16us1.597ms
> 
> Slower?

It is based on integer comparisons though.

> >  NormalClassAttribute:339ms340ms0.28us1.111ms
> 
> Over twice as slow?

Yes, should be investigated.

> >SimpleDictManipulation:280ms285ms0.24us1.059ms
> 
> Slower.  str -> unicode?

No, there are just integers.

> > SpecialClassAttribute:534ms535ms0.45us1.121ms
> 
> ~4x slower!

Should be investigated as well.

> >StringMappings:471ms471ms1.87us0.884ms
> 
> ~3X slower.

It's normal. This test measures functions like upper(), etc., which are much
expensive using the full unicode database than when doing a straight lookup
in a 256-entry table.

> >   WithFinally:185ms188ms1.18us0.881ms
> >   WithRaiseExcept:274ms275ms3.44us1.105ms
> 
> Slower.

Yes, the new exception semantics have a cost.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Unicode 5.1.0

2008-08-24 Thread Martin v. Löwis
> is the suggestion to *replace* the 4.1.0 database with a 5.1.0
> database, or to add yet another database in that module?

I would replace it.

> (how's the 3.2/4.1 dual support implemented?

The compiler needs data files for all supported versions, with
old_versions listing the, well, old versions. It then computes
deltas, expecting that they should mostly consist of new
assignments (i.e. characters unassigned in 3.2 might be assigned
in newer versions). It detects all differences, but might not be
able to represent all changes.

> do we have two distinct
> datasets, or are the differences encoded in some clever way?

The latter. It doesn't really need to be that clever: primarily
just a compressed list of "new" characters is needed, per version.

> would it
> make sense to split the unicodedata module into three separate
> modules, one for each major Unicode version?)

You couldn't use the space savings then, I suppose.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Unicode 5.1.0

2008-08-24 Thread Martin v. Löwis
> That's up to us. I don't know what the reason was for keeping the
> 3.2.0 database around -- does anyone here recall ever using it? For
> what?

It's needed for IDNA. The IDNA RFC requires that Unicode 3.2 is used
for performing IDNA (in particular, for determining what a valid domain
name is).

The IDNA people consider it security-relevant that it is
really the 3.2 database, and would probably consider it a serious
security bug if newer Python versions suddenly started to use newer
Unicode databases for IDNA.

At some point, IDNA might get updated to a newer version of the Unicode
spec; we can then drop 3.2 (and stick with whatever the RFC then
specifies).

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Unicode 5.1.0

2008-08-24 Thread Martin v. Löwis
> I can tinker a little with this over the weekend, unless Martin tells
> me not to ;-)

Go ahead; I can't work on this at the moment, anyway. I would also be
confident that a mere replacement of 4.1 with 5.1 should be easy, and
I see no reason to keep the 4.1 version.

Perhaps makeunicodedata should list *why* certain old versions remain
supported; for 3.2, the use case is IDNA.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] IDNA

2008-08-24 Thread Antoine Pitrou
Martin v. Löwis  v.loewis.de> writes:
> 
> It's needed for IDNA. The IDNA RFC requires that Unicode 3.2 is used
> for performing IDNA (in particular, for determining what a valid domain
> name is).

Speaking of which, Martin, did you take a look at 
http://bugs.python.org/issue3232 ?
I suppose the fix is trivial, but I don't know what it should be :-)

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] performance

2008-08-24 Thread Georg Brandl
Antoine Pitrou schrieb:
> Hi,

pystone
---

- 2.5: 43859.6 pystones/second
- 2.6: 42016.8 pystones/second
- 3.0: 38759.7 pystones/second


>> So 3.0 is about 10% slower than 2.x.  Given all the changes, that
>> doesn't seem too bad.
> 
> Yes, I think it's rather good.

Well, pystone really doesn't test much of what causes the largest
slowdowns in 3.0...

Georg


-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] performance

2008-08-24 Thread Steven D'Aprano
On Mon, 25 Aug 2008 05:04:16 am Antoine Pitrou wrote:
> > >         CompareFloatsIntegers:    274ms    274ms    0.30us  
> > >  0.630ms
> >
> > Much slower, but probably due to switch from int -> long.  There
> > could be potential for optimizing this case.
>
> Well honestly you don't often compare different types. I think the
> most common exception to this rule would be None vs. non-None.


Surely the most common None vs non-None comparison would be the "is" 
operator, which I hope remains fast.

I don't know about other people, but I often compare floats with ints. 
Here's a contrived example:

x = some_float()
if x == -1:
return -2
else:
return (x**2-1)/(x+1)


I suppose it's no hardship to start writing float literals instead of 
int literals, if needed.


-- 
Steven
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] what version of bsddb to use/accept for 2.6/3.0

2008-08-24 Thread Neal Norwitz
Many buildbots are running bsddb 4.7, particularly the debian/ubuntu
ones (4.7.25 which seems to be the latest).  Some of them are
crashing, others are not.  The max version we support in both 2.6 and
3.0 is 4.7.  Should we allow this version or should we use a lower
maximum that is more likely to work (ie, not crash)?

It looks like the WIndows buildbots use 4.4.20.  Unfortunately, the
Windows bots aren't in great shape either.

Additionally, there are reference leaks in both 2.6 and 3.0:
  test_bsddb3 leaked [80, 80] references, sum=160 (2.6)
  test_bsddb3 leaked [63, 63] references, sum=126 (3.0)

It would be nice to get as many of these things fixed up before
release.  Jesus, Greg, Trent, can you make some time over the next
week to stabilize bsddb support?

Thanks,
n
PS.  To change the max version of bsddb supported in Unix, modify
max_db_ver in setup.py.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] what version of bsddb to use/accept for 2.6/3.0

2008-08-24 Thread Matthias Klose
[CCing Clint]

4.7.25 generally looks fine, except for sparc (afaics); I asked the Debian bsddb
maintainer to have a look at the sparc problems. At least on sparc, there's no
difference between 4.6 and 4.7. I usually try to run the buildbots with the
recent software in the distribution, as this is what users will see with the
next release.

  Matthias

PS: currently travelling, won't be able to answer in time.

Neal Norwitz schrieb:
> Many buildbots are running bsddb 4.7, particularly the debian/ubuntu
> ones (4.7.25 which seems to be the latest).  Some of them are
> crashing, others are not.  The max version we support in both 2.6 and
> 3.0 is 4.7.  Should we allow this version or should we use a lower
> maximum that is more likely to work (ie, not crash)?
> 
> It looks like the WIndows buildbots use 4.4.20.  Unfortunately, the
> Windows bots aren't in great shape either.
> 
> Additionally, there are reference leaks in both 2.6 and 3.0:
>   test_bsddb3 leaked [80, 80] references, sum=160 (2.6)
>   test_bsddb3 leaked [63, 63] references, sum=126 (3.0)
> 
> It would be nice to get as many of these things fixed up before
> release.  Jesus, Greg, Trent, can you make some time over the next
> week to stabilize bsddb support?
> 
> Thanks,
> n
> PS.  To change the max version of bsddb supported in Unix, modify
> max_db_ver in setup.py.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Things to Know About Super

2008-08-24 Thread [EMAIL PROTECTED]


On Aug 24, 3:43 pm, "Matt Giuca" <[EMAIL PROTECTED]> wrote:
> Hi Michele,
>
> Do you have a URL for this blog?
>

Sorry, here it is:
http://www.artima.com/weblogs/index.jsp?blogger=micheles
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com