Re: [Python-Dev] __file__ is not always an absolute path
On 7 Feb 2010, at 05:27, exar...@twistedmatrix.com wrote: > Do you know of a case where it's actually slow? If not, how convincing > should this argument really be? Perhaps we can measure it on a few platforms > before passing judgement. On Mac OS X at least, system calls are notoriously slow. I think it has to do with Mach overhead, or something… $ arch -arch ppc /usr/bin/python2.6 -m timeit -s 'def f(): pass' 'f()' 100 loops, best of 3: 0.476 usec per loop $ arch -arch ppc /usr/bin/python2.6 -m timeit -s 'from os import getcwd' 'getcwd()' 1 loops, best of 3: 21.9 usec per loop $ arch -arch i386 /usr/bin/python2.6 -m timeit -s 'def f(): pass' 'f()' 100 loops, best of 3: 0.234 usec per loop $ arch -arch i386 /usr/bin/python2.6 -m timeit -s 'from os import getcwd' 'getcwd()' 10 loops, best of 3: 14.1 usec per loop $ arch -arch x86_64 /usr/bin/python2.6 -m timeit -s 'def f(): pass' 'f()' 1000 loops, best of 3: 0.182 usec per loop $ arch -arch x86_64 /usr/bin/python2.6 -m timeit -s 'from os import getcwd' 'getcwd()' 10 loops, best of 3: 11 usec per loop For maximum reproducibility, I used the stock Python 2.6.1 included in Mac OS X 10.6.2. In other words ‘os.getcwd()’ is more than fifty times as slow as a regular function call when using Mac OS X. -- Dan Villiom Podlaski Christiansen dan...@gmail.com smime.p7s Description: S/MIME cryptographic signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] embedding Python interpreter in non-console windows application
stephen, 17.02.2010 06:49: > THE PROBLEM: > I am having a problem that I have seen asked quite a bit on the web, with > little to no follow up. Note that this list is about developing the CPython core runtime, not about solving problems with Python code or Python usage. See the comp.lang.python newsgroup for that (or the corresponding mailing list mirror). Stefan ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add UTC to 2.7 (PyCon sprint idea)
-On [20100217 03:19], Stuart Bishop (stu...@stuartbishop.net) wrote: >The Debian, Ubuntu and I think Redhat packages all use the system >zoneinfo database - there are hooks in there to support package >maintainers that want to do this. This way the package can be included >in the supported release but still receive timezone information >updates via the OS (but no code updates, but these are rare and >usually irrelevant unless you where the person who filed the bug ;) ). This can also work for all the BSDs since they include the Olson zoneinfo data in the base system as well. And that will probably mean Mac OS X as well, if they stuck to what FreeBSD had in place for that. Can anyone verify that? -- Jeroen Ruigrok van der Werven / asmodai イェルーン ラウフロック ヴァン デル ウェルヴェン http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B Anything becomes possible, after you find the courage to admit that nothing is certain. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add UTC to 2.7 (PyCon sprint idea)
On Tue, Feb 16, 2010 at 13:42, Dirkjan Ochtman wrote: > On Tue, Feb 16, 2010 at 13:05, wrote: >> Maybe an alternate sprint idea would be to incorporate dateutil into the >> Python core: http://labix.org/python-dateutil >> >> Whoops... (just waking up - still need that first cup of coffee) >> >> While incorporating dateutil into the core would be nice (in my opinion at >> least), I was really thinking of pytz: http://pytz.sourceforge.net/ > > I think dateutil is fairly heavy for the stdlib, but I think pytz > would be a very good candidate for inclusion. Without it, the timezone > support in datetime is hardly usable. The timezone database is updated several times per year. You can *not* include it in the standard library. On Tue, Feb 16, 2010 at 16:43, Dirkjan Ochtman wrote: > By using what the OS provides. The OS often does not. > At least on Linux, the basic timezone > data is usually updated by other means (at least on the distro I'm > familiar with, it's updated quite often, too; through the package > manager). I'm assuming Windows and OS X would also be able to provide > something like this. The Windows timezone data sucks donkeyballs through a hose. Thus, if the timezone implementations from pytz was in the standard library, and the timezone data not, they would not be useable on Windows. So, no can do. Also, different Unices often have slightly different names and organisations of the Olsen database, which would create confusions and incompatibilities, so that's probably also not the best solution. == So, what to do? Use Pytz! == There is no need to stick Pytz in the standard library. It's available on PyPI, updated frequently, etc. What we can do is point to it from the documentation. But before that, it needs a fix. Pytz is great, but missing one thing: Wrappers for the current locale settings. This is necessary, because there is no way of realiably figuring out the current locale. See http://regebro.wordpress.com/2008/05/10/python-and-time-zones-part-2-the-beast-returns/ (and http://regebro.wordpress.com/2007/12/18/python-and-time-zones-fighting-the-beast/ for other timezone issues). These kinds of wrappers exist in dateutils.tz. It would be great if that type of functionality could get into Pytz as well. A sprint to do this and fix the issues in the tracker should solve the issues, I think. There is no need to move things into the core. An Pytz could use more maintainers, Stuart tends not to answer emails, I assume this is because he is overw -- Lennart Regebro: http://regebro.wordpress.com/ Python 3 Porting: http://python-incompatibility.googlecode.com/ +33 661 58 14 64 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add UTC to 2.7 (PyCon sprint idea)
On Wed, Feb 17, 2010 at 12:32, Lennart Regebro wrote: > These kinds of wrappers exist in dateutils.tz. It would be great if > that type of functionality could get into Pytz as well. A sprint to do > this and fix the issues in the tracker should solve the issues, I > think. There is no need to move things into the core. An Pytz could > use more maintainers, Stuart tends not to answer emails, I assume this > is because he is overw Bloody gmail! I did NOT press send. Glah. Stuart tends not to answer emails, I assume this is because he is overworked, so more eyes on Pytz is probably a good idea. He is welcome to correct me if this is not so. :) -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] math.hypot, complex.__abs__, and documentation
On Wed, 17 Feb 2010 10:06:01 am Mark Dickinson wrote: > On Tue, Feb 16, 2010 at 10:46 PM, Steven D'Aprano > > > > What's the justification for that convention? It seems wrong to me. > > It's difficult to do better than to point to Kahan's writings. See > > http://www.eecs.berkeley.edu/~wkahan/ieee754status/IEEE754.PDF Well, who am I to question Kahan? I guess if you interpret nan as "indeterminate", than hypot(inf, nan) should be inf; but if you interpret it as "not a number", then it should be nan. Since NANs can be both, I guess we're stuck with one or the other. So I'm satisfied that there's a good reason for the behaviour, even if I'm not 100% convinced it's the best reason. On a related note, why the distinction here? >>> inf*inf inf >>> inf**2 Traceback (most recent call last): File "", line 1, in OverflowError: (34, 'Numerical result out of range') -- Steven D'Aprano ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] math.hypot, complex.__abs__, and documentation
[With apologies for Steven for the duplicate email.]
On Wed, Feb 17, 2010 at 12:22 PM, Steven D'Aprano wrote:
> Well, who am I to question Kahan?
Yes, there I go with the argument from authority. But while we
shouldn't instantly accept Kahan's arguments just because he's Kahan,
it would be equally foolish for us mere mortals to ignore words from
one of the prime movers of the IEEE 754 standard. :-)
> I guess if you interpret nan as "indeterminate", than hypot(inf, nan)
> should be inf; but if you interpret it as "not a number", then it
> should be nan. Since NANs can be both, I guess we're stuck with one or
> the other.
Apart from the 'should be's, I think there's also a practical aspect
to consider: I'm guessing that part of the reason for this sort of
behaviour is that it make it more likely for numerical code to 'do the
right thing' without extra special-case handling, in much the same way
that infinities can appear and disappear during a numerical
calculation, leaving a valid finite result, without the user having
had to worry about inserting special cases to handle those infinities.
As an example of the latter behaviour, consider evaluating the
function f(x) = 1/(1+1/x) naively at x = 0; if this formula appears
in any real-world circumstances, the chances are that you want a
result of 0, and IEEE 754's non-stop mode gives it to you. (This
doesn't work in Python, of course, because it doesn't really have a
non-stop mode; more on this below.) Unfortunately, to back this
argument up properly I'd need lots of real-world examples, which I
don't have. :(
> So I'm satisfied that there's a good reason for the
> behaviour, even if I'm not 100% convinced it's the best reason.
>From Python's point of view, the real reason for implementing it this
way is that it follows current standards (C99 and IEEE 754; probably
also the Fortran standards too, but I haven't checked), so this
special case behaviour (a) likely matches expectations for numerical
users, and (b) has been thought about carefully by at least some
experts.
> On a related note, why the distinction here?
>
inf*inf
> inf
inf**2
> Traceback (most recent call last):
> File "", line 1, in
> OverflowError: (34, 'Numerical result out of range')
For that particular example, it's because you haven't upgraded to
Python 2.7 yet. :)
Python 2.7a3+ (trunk:78206M, Feb 17 2010, 10:19:00)
[GCC 4.2.1 (Apple Inc. build 5646) (dot 1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> float('inf') ** 2
inf
See http://bugs.python.org/issue7534. But there are similar problems
that aren't fixed, and can't reasonably be fixed without causing
upset:
>>> 1e300 ** 2
Traceback (most recent call last):
File "", line 1, in
OverflowError: (34, 'Result too large')
>>> 1e300 * 1e300
inf
Here I'd argue that the ideal Python behaviour would be to produce an
OverflowError in both cases; more generally, arithmetic with finite
numbers would never produce infinities or nans, but always raise
Python exceptions instead. But some users need or expect some kind of
'non-stop mode' for arithmetic, so changing this probably wouldn't go
down well.
Mark
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __file__ is not always an absolute path
I did some quick measures out of curiosity. Performances seems clearly filesystem and O.S. dependent (and are likely deployment/configuration dependent). I did each test 3 times to ensure measure where consistent. Tests were done with ActivePython 2.6.3.7. * AIX 5.3: python26 -m timeit -s 'def f(): pass' 'f()' 100 loops, best of 3: 0.336 usec per loop cwd is NFS mount: users/baplepil/sandbox> python26 -m timeit -s 'from os import getcwd' 'getcwd()' 1000 loops, best of 3: 1.09 msec per loop cwd is /tmp: /tmp> python26 -m timeit -s 'from os import getcwd' 'getcwd()' 1000 loops, best of 3: 323 usec per loop * Solaris 10 (Sparc): python26 -m timeit -s 'def f(): pass' 'f()' 100 loops, best of 3: 0.495 usec per loop cwd is NFS mount: users/baplepil/sandbox> python26 -m timeit -s 'from os import getcwd' 'getcwd()' 10 loops, best of 3: 12.1 usec per loop cwd is /tmp: /tmp> python26 -m timeit -s 'from os import getcwd' 'getcwd()' 10 loops, best of 3: 4.58 usec per loop * Windows XP SP2: python -m timeit -s "def f(): pass; f()" 1000 loops, best of 3: 0.0531 usec per loop cwd is network drive (same as previous NFS mount): R:\...\users\baplepil>python -m timeit -s "from os import getcwd" "getcwd()" 10 loops, best of 3: 5.14 usec per loop cwd is C:\temp>: C:\temp>python -m timeit -s "from os import getcwd" "getcwd()" 10 loops, best of 3: 4.27 usec per loop 2010/2/17 Dan Villiom Podlaski Christiansen > On 7 Feb 2010, at 05:27, exar...@twistedmatrix.com wrote: > > > Do you know of a case where it's actually slow? If not, how convincing > should this argument really be? Perhaps we can measure it on a few > platforms before passing judgement. > ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] embedding Python interpreter in non-console windows application
Hi,
2010/2/17 stephen
> Hello,
>
> THE PROBLEM:
> I am having a problem that I have seen asked quite a bit on the web, with
> little to no follow up.
> The problem is essentially this. When embedding (LoadLibraryA()) the python
> interpreter dll
> in a non-windows application the developer must first create a console for
> python to do output/input with.
> I properly initialize the CRT and AllocConsole() to do this. I then
> GetSTDHandle() for stdin and stdout accordingly
> and open those handles with the requisite flags "read" for STDIN and
> "write" for stdout. This all works great
> and is then verified and tested to work by printf() and fgets(). This issue
> however happens when attempting
> to PyRun_InteractiveLoop() and PyRun_SimpleString(). A
> PyRun_SimpleString("print 'test'") displays nothing in my
> freshly allocated console window. Similarly a PyRun_InteractiveLoop(stdin,
> NULL); yields nothing either even though
> the line printf("testing"); directly ahead of it works just fine. Does
> anyone have insight on how I can make this work
> with the freshly allocated console's stdin/stdout/stderr?
>
> SPECULATION:
> That is the question, so now on to the speculation. I suspect that
> something in the python runtime doesn't "get handles"
> correctly for STDIN and STDOUT upon initialization. I have perused the
> source code to find out exactly how this is done
> and I suspect that it starts in PyInitializeEx with calls to
> PySys_GetObject("stdin") and "stdout" accordingly. However I
> don't actually see where this translates into the Python runtime checking
> with the C-runtime for the "real" handles to STDIN and STDOUT. I dont ever
> see the Python runtime "ask the system" where his handles to STDIN and
> STDOUT are.
>
Are you using the same compiler as the one used to compile Python? It's
important that your program and python use the same C runtime library
(MSVCR90.dll for python 2.6), otherwise "stdout" refers to different things.
--
Amaury Forgeot d'Arc
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add UTC to 2.7 (PyCon sprint idea)
Defaulting to UTC is not a good idea, which is why relevant methods take an argument to specify whether to be UTC (exact details are in the patch; don't remember exact details). On Feb 16, 2010 4:07 PM, "Greg Ewing" wrote: Brett Cannon wrote: > Issue 5094 already has a patch that is nearly complete to provide a > default... Are you sure it's really a good idea to default to UTC? I thought it was considered a feature that datetime objects are naive unless you explicitly specify a timezone. -- Greg ___ Python-Dev mailing list Python-Dev@python.org http:... ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add UTC to 2.7 (PyCon sprint idea)
Lennart> The timezone database is updated several times per year. You Lennart> can *not* include it in the standard library. My guess is the data are updated several times per year, not the code. Can they not be separated? Skip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add UTC to 2.7 (PyCon sprint idea)
On Wed, Feb 17, 2010 at 23:46, wrote: > > Lennart> The timezone database is updated several times per year. You > Lennart> can *not* include it in the standard library. > > My guess is the data are updated several times per year, not the code. Can > they not be separated? Yes, but that would mean we have an implementation in stdlib that relies on a dataset which may not exist. That is just going to be confusing. Moving pytz into the stdlib doesn't solve anything, really. So why do it? It's not like pytz is hard to install. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add UTC to 2.7 (PyCon sprint idea)
s...@pobox.com writes: > My guess is the data are updated several times per year, not the code. > Can they not be separated? AIUI this discussion is about getting the ‘pytz’ library into the Python standard library. If the data is separate from the modules, the question then becomes how users on various platforms can update the data without installing a new version of the whole standard library. -- \ “The best ad-libs are rehearsed.” —Graham Kennedy | `\ | _o__) | Ben Finney ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [issue2636] Regexp 2.7 (modifications to current re 2.2.2)
Vlastimil Brom wrote:
Vlastimil Brom added the comment:
I just tested the fix for unicode tracebacks and found some possibly weird
results (not sure how/whether it should be fixed, as these inputs are indeed
rather artificial...).
(win XPp SP3 Czech, Python 2.6.4)
Using the cmd console, the output is fine (for the characters it can accept and
display)
regex.findall(ur"\p{InBasicLatinĚ}", u"aé")
Traceback (most recent call last):
...
File "C:\Python26\lib\regex.py", line 1244, in _parse_property
raise error("undefined property name '%s'" % name)
regex.error: undefined property name 'InBasicLatinĚ'
(same result for other distorted "proprety names" containing e.g.
ěščřžýáíéúůßäëiöüîô ...
However, in Idle the output differs depending on the characters present
regex.findall(ur"\p{InBasicLatinÉ}", u"ab c")
yields the expected
...
File "C:\Python26\lib\regex.py", line 1244, in _parse_property
raise error("undefined property name '%s'" % name)
error: undefined property name 'InBasicLatinÉ'
but
regex.findall(ur"\p{InBasicLatinĚ}", u"ab c")
Traceback (most recent call last):
...
File "C:\Python26\lib\regex.py", line 1244, in _parse_property
raise error("undefined property name '%s'" % name)
File "C:\Python26\lib\regex.py", line 167, in __init__
message = message.encode(sys.stdout.encoding)
File "C:\Python26\lib\encodings\cp1250.py", line 12, in encode
return codecs.charmap_encode(input,errors,encoding_table)
UnicodeEncodeError: 'charmap' codec can't encode character u'\xcc' in position 37:
character maps to
which might be surprising, as cp1250 should be able to encode "Ě", maybe there
is some intermediate ascii step?
using the wxpython pyShell I get its specific encoding error:
regex.findall(ur"\p{InBasicLatinÉ}", u"ab c")
Traceback (most recent call last):
...
File "C:\Python26\lib\regex.py", line 1102, in _parse_escape
return _parse_property(source, info, in_set, ch)
File "C:\Python26\lib\regex.py", line 1244, in _parse_property
raise error("undefined property name '%s'" % name)
File "C:\Python26\lib\regex.py", line 167, in __init__
message = message.encode(sys.stdout.encoding)
AttributeError: PseudoFileOut instance has no attribute 'encoding'
(the same for \p{InBasicLatinĚ} etc.)
Maybe it shouldn't show the property name at all. That would avoid the
problem.
In python 3.1 in Idle, all of these exceptions are displayed correctly, also in
other scripts or with special characters.
Maybe in python 2.x e.g. repr(...) of the unicode error messages could be used
in order to avoid these problems, but I don't know, what the conventions are in
these cases.
Another issue I found here (unrelated to tracebacks) are backslashes or
punctuation (except the handled -_) in the property names, which just lead to
failed mathces and no exceptions about unknown property names
regex.findall(u"\p{InBasic.Latin}", u"ab c")
[]
In the re module a malformed pattern is sometimes treated as a literal:
>>> re.match(r"a{1,2", r"a{1,2").group()
'a{1,2'
which is what I'm trying to replicate, as far as possible.
Which characters should it accept when parsing the property name, even
if it subsequently rejects the name? I don't want it to accept every
character until it sees the closing '}'. I currently include
alphanumeric, whitespace, '&', '_' and '-'. '.' might be a reasonable
addition.
I was also surprised by the added pos/endpos parameters, as I used flags as a
non-keyword third parameter for the re functions in my code (probably my fault
...)
re.findall(pattern, string, flags=0)
regex.findall(pattern, string, pos=None, endpos=None, flags=0, overlapped=False)
(is there a specific reason for this order, or could it be changed to maintain
compatibility with the current re module?)
Oops! I'll fix that.
I hope, at least some of these remarks make some sense;
thanks for the continued work on this module!
All constructive remarks are welcome! :-)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add UTC to 2.7 (PyCon sprint idea)
On 17 February 2010 23:01, Lennart Regebro wrote: > On Wed, Feb 17, 2010 at 23:46, wrote: > > > >Lennart> The timezone database is updated several times per year. You > >Lennart> can *not* include it in the standard library. > > > > My guess is the data are updated several times per year, not the code. > Can > > they not be separated? > > Yes, but that would mean we have an implementation in stdlib that > relies on a dataset which may not exist. That is just going to be > confusing. Moving pytz into the stdlib doesn't solve anything, really. > So why do it? It's not like pytz is hard to install. > > Some of the Linux distributions *already* patch pytz to use the system information, which they keep updated separately. That information is also available from the system on Mac OS and Windows. It would seem to be very useful to have a version of pytz that defaults to using the system information if available, has a mechanism for using separate data for systems that don't provide the information or raises an error when neither system information nor separate data is available. The data could then still be available and released regularly without being tied to the Python release schedule. That assumes that the author of pytz *wants* it to come into the standard library of course. Michael Foord > -- > Lennart Regebro: Python, Zope, Plone, Grok > http://regebro.wordpress.com/ > +33 661 58 14 64 > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
