Re: [Python-Dev] Musings on concurrency and scoping ("replacing" Javascript)

[Michael Hudson]

Ka-Ping Yee <[EMAIL PROTECTED]> writes:

> Client-side web scripting tends to have a callback/continuation-ish
> concurrency style because it has to deal with network transactions
> (which can stall for long periods of time) in a user interface that
> is expected to stay always responsive.  The Firefox API is full of
> listeners/observers, events, and continuation-like things.  So one
> thing to consider is that, when Python is used for these purposes,
> it may be written in a specialized style.

You could even call it a Twisted style :-)

Cheers,
mwh

-- 
  Considering that this thread is completely on-topic in the way only
  c.l.py threads can be, I think I can say that you should replace
  "Oblivion" with "Gravity", and increase your Radiohead quotient.
  -- Ben Wolfson, comp.lang.python
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] User's complaints

[Michael Hudson]

"A.M. Kuchling" <[EMAIL PROTECTED]> writes:

> On Mon, Jul 10, 2006 at 05:13:53PM +0200, Armin Rigo wrote:
>> didn't draw much applause.  It certainly gave me the impression that
>> many changes in Python are advocated and welcomed by only a small
>> fraction of users.
>
> The benefits of changes are usually clear, but I don't think the costs
> of changes are fully assessed.  python-dev considers the cost of
> changes to CPython's implementation, but I don't think the cost
> changes to Jython, IronPython, or PyPy are taken into account here.

I don't want to attempt to speak for Armin, but I don't think that was
quite the point he was trying to make.  The cost of implementation
isn't really that high -- most of the changes from 2.3 to 2.4 were
implemented for PyPy in about a week and we have implementations of
most of the 2.5 features already, and you can also consider the
decorators discussion where each of the proposed syntaxes had solid
tested implementations.

At least from my POV, the cost of bad design, whether through simple
lack of sufficient thought or aggregation of features into cruft, is
much much higher.

> PyPy probably has enough representatives here who would squawk if
> something was difficult, but I don't think Jython or IronPython do.

People implementing Python should accept the fact that Python is a
changing language, IMHO.

> I think if we assessed those costs fully, the bar for changes to the
> language would be a good deal higher.

I think if you assessed these other costs fully, the bar would be
higher still.

Cheers,
mwh

-- 
  This is the fixed point problem again; since all some implementors
  do is implement the compiler and libraries for compiler writing, the
  language becomes good at writing compilers and not much else!
 -- Brian Rogoff, comp.lang.functional
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Community buildbots

[Michael Hudson]

No real time to respond in detail here, but one small comment.

[EMAIL PROTECTED] writes:

> I see some responses to that post which indicate that the specific bug will be
> fixed, and that's good, but there is definitely a pattern he's talking about
> here, not just one issue.  I think there is a general pattern of small,
> difficult to detect breakages in Python.

Remember these words...

> For example, did anyone here know that the new-style exceptions stuff in 2.5
> caused hundreds of unit-test failures in Twisted?  I am glad the change was
> made, and one of our users did catch it, so the process isn't fatally broken,
> but it is still worrying.

When implementing this stuff, I could have merely made it possible for
exceptions to be new-style, and left the builtin exceptions as classic
classes.  This didn't seem to be an especially good idea, as it would
leave code working _most_ of the time, only to break horribly when
confronted with a rare new-style exception.  So I made the decision
(and I don't think I ever got around to explicitly discussing this
with anyone, nor if the people who actually updated my patch and
checked it in thought about it at all) to make the built in exceptions
new-style, precisely to make it a screamingly obvious change.  I
didn't _know_ when I was doing it that I'd break Twisted unit tests,
but I was hardly a surprise.

I think the idea of a buildbot running various projects tests suites
with svn HEAD is a great idea -- back when I was doing 2.2.1 I did run
a few third party test suites (zodb comes to mind) but as always with
these things, automation is a good idea.

Cheers,
mwh

-- 
  > Why are we talking about bricks and concrete in a lisp newsgroup?
  After long experiment it was found preferable to talking about why
  Lisp is slower than C++...
-- Duane Rettig & Tim Bradshaw, comp.lang.lisp
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Community buildbots

[Michael Hudson]

[EMAIL PROTECTED] writes:

> I just would have appreciated the opportunity to participate in the
> discussion before the betas were out and the featureset frozen.

I think something that has happened to some extent with this release
is that there was a long-ish period where stuff got discussed and
implemented (PEPs 343 and 344, new-style exceptions, the new compiler,
the ssize_t branch) and then suddenly we went "crap!  we have to
release this!" and then the whole alpha/beta/release part has happened
much more quickly.  Maybe we should have had a more explicit
discussion on what was going to be in 2.5 before beta1, but that kind
of thing is difficult to make happen (it's entirely possible that it
did happen and I just missed it by being snowed under, but I don't
think so).

Cheers,
mwh

-- 
 Have you considered downgrading your arrogance to a reasonable level?
-- Erik Naggum, comp.lang.lisp, to yet another C++-using troll
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Ireland PyPy sprint 21th-27th August 2006

[Michael Hudson]

The next PyPy sprint will happen in the nice city of 
Limerick in Ireland from 21st till 27th August.  
(Most people intend to arrive 20th August). 

The main focus of the sprint will be on JIT compiler works, 
various optimization works, porting extension modules, 
infrastructure works like a build tool for PyPy, or 
extended (distributed) testing. 

It's also open to new topics.  If you are a student
consider to participate in `Summer of PyPy`_ in order 
get funding for your travels and accomodation. 

The sprint is being hosted by University of Limerick
(http://www.ul.ie/) - and is arranged in co-operation
with people from our sister project Calibre (www.calibre.ie).
Our contact at the University is Pär Ågerfalk and Eoin
Oconchuir. 

.. _`Summer of PyPy`: http://codespeak.net/pypy/dist/pypy/doc/summer-of-pypy

First day: introduction and workshop (possible to attend only this day)


During the first day (21st of August) there will be talks on various subjects
related to PyPy:

* A tutorial and technical introduction to the PyPy codebase  
  (suited for people interested in getting an overview of PyPy´s architecture
  and/or contributing to PyPy)

* a workshop covering more in-depth technical aspects of PyPy and what PyPy
  can do for you. The workshop will also cover methodology, aiming at explaining
  the pros and cons of sprint-driven development.
  (suited for sprint attendants, students, staff and other 
  interested parties from/around the University and the local area)

The tutorial will be part of the sprint introduction - the workshop will take 
place if 
there is enough interest raised before the 21st of August from people planning 
to 
attend. You are of course welcome to attend just for this first day of the 
sprint.


If you want to come ...


If you'd like to come, please subscribe to the `pypy-sprint mailing list`_
and drop a note about your interests and post any questions.  More 
organisational information will be send to that list.  We'll keep
a list of `people`_ which we'll update (which you can do so yourself
if you have codespeak commit rights). 

.. _`Calibre`: http://www.calibre.ie 

A small disclaimer:
There might be people visiting the sprint in order to do research on 
how open source communities work, organize and communicate. This 
research might be done via filming, observing or interviewing. 
But of course you will be able to opt-out of being filmed at the sprint. 

Logistics 
--

NOTE: you need a UK style of power adapter (220V). 

The sprint will be held in the Computer Science Building, room CSG-025, 
University of Limerick (no 7 on http://www.ul.ie/main/places/campus.shtml). 

Bus 308 from Limerick city will take you to no 30 (approx.).
See http://www.ul.ie/main/places/travel.shtml for more on how to get to UL.

We will have access to the sprint facilities from 09:00-19:00 every day (it 
might
be even later than 19:00). Monday-Wednesday, Friday-Sunday are sprint days,
Thursday is likely a break day.

Food on campus varies in price and quality ;-) : from ca 4 EUR to 7-8 EUR for 
a lunch. There are of course a lot more food alternatives in down town Limerick.

Next Airports
--

Shannon Airport (SNN) is the nearest airport (Ryanair flies
there) - you may check out more information about flights
to/from the airport at

http://www.shannonairport.com/index.html 

There are busses from there to downtown Limerick, and busses
from Limerick to the UL campus. Taxis are about 35 EUR.

Accomodation
-

There is a website address for campus accomodation at 

http://www.ul.ie/conference/accommodation.htm.  

The rate should be 49 euro for Bed and Breakfast.  If you are interested 
in booking campus accommodation, please contact deborah.tudge at ul ie
and make reference to the PyPy workshop and sprint.  Please try 
to book as soon as possible. 

As an off-campus accommodation alternative you can also try:

Castletroy Lodge and Castletroy Inn (Bed and Breakfast)
Dublin Road (15 to 20 mins walk to UL)
Tel: +353 61 338385 / +353 61 331167

.. _`pypy-sprint mailing list`: 
http://codespeak.net/mailman/listinfo/pypy-sprint
.. _`people`: people.html 

-- 
  surely, somewhere, somehow, in the history of computing, at least
  one manual has been written that you could at least remotely
  attempt to consider possibly glancing at.  -- Adam Rixey
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] remaining issues from Klocwork static analysis

[Michael Hudson]

"Neal Norwitz" <[EMAIL PROTECTED]> writes:

> On 7/25/06, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote:
>> >
>> > Yes, I definitely think dropping the X would make the warning go away.
>> > Do we want to check for a NULL pointer and raise an exception?  The
>> > docs don't address the issue, so I think if we added a check, ie:  if
>> > (closure && PyTuple_Check(closure)) and got rid of the X that would be
>> > fine as well.
>>
>> The docs do address the issue:
>>
>> \var{closure} must be \var{Py_None} or a tuple of cell objects.
>>
>> It doesn't allow for NULL, and None indicates that the closure
>> should become NULL. The only caller of it in the core will never
>> pass NULL.
>>
>> If you want to check that this is not NULL on the grounds that
>> somebody may call it incorrectly, then you should also check that
>> op is not NULL, because somebody may call it incorrectly.
>
> We never really did address this issue did?  A while back we talked
> about whether to assert vs check and do PyErr_BadInternalCall().  I
> don't remember a clear resolution (though my memory).  I vaguely
> remember a preference towards asserting, but I don't know if that was
> in all cases or maybe it was just my preference. :-)
>
> I'm happy to assert here too.  But it's really a broader question.  I
> guess I'm even happy to just remove the X.  It would be nice to handle
> this consistently going forward.

I think I'm rather in favour of assert()ing this sort of thing.  If
you're programming in C, you can cause crashes any which way and
removing one doesn't seem worth making correct usage pay any kind of
(admittedly miniscule) performance penalty.  It would be nice if API
docs explicitly stated which pointer arguments could be NULL, and then
it would be a programming error to pass a NULL pointer argument in any
other place.  I have no idea how far away from this we are already :-)

Cheers,
mwh

-- 
  Gullible editorial staff continues to post links to any and all
  articles that vaguely criticize Linux in any way.
 -- Reason #4 for quitting slashdot today, from
http://www.cs.washington.edu/homes/klee/misc/slashdot.html
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Py2.5 release schedule

[Michael Hudson]

Anthony Baxter <[EMAIL PROTECTED]> writes:

> On Friday 28 July 2006 15:32, Raymond Hettinger wrote:
>> I suggest that there be a third beta release and that we then wait
>> just a bit before going final.
>>
>> The bugs that were found and fixed in the first two beta releases
>> suggest that Py2.5 is not yet as stable as we would like.  Over the
>> next few days, I'll try to run it on as much third-party code as
>> possible. That would have detected the recently surfaced grammar
>> error a little bit earlier (the one where "for x, in listOfTuples"
>> would not unpack).
>>
>> The release process itself is going well but I don't think the
>> pervasive AST changes have been fully shaken-out yet.
>
> I've been thinking the same thing, too. A quick chat to Neal says that 
> he also agrees.
>
> There's still a lot more bugs popping up than I'm really comfortable 
> with. I guess this is inevitable - there's a lot of new stuff in 2.5. 
>
> Does anyone disagree with making the next release beta3?

It seems like a good idea to me.  I guess this will mean the final
release will be pushed back a bit?

Cheers,
mwh

-- 
  Gevalia is undrinkable low-octane see-through only slightly
  roasted bilge water. Compared to .us coffee it is quite
  drinkable.  -- Måns Nilsson, asr
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Bad interaction of __index__ and sequence repeat

[Michael Hudson]

David Hopwood <[EMAIL PROTECTED]> writes:

> Armin Rigo wrote:
>> Hi,
>> 
>> There is an oversight in the design of __index__() that only just
>> surfaced :-(  It is responsible for the following behavior, on a 32-bit
>> machine with >= 2GB of RAM:
>> 
>> >>> s = 'x' * (2**100)   # works!
>> >>> len(s)
>> 2147483647
>> 
>> This is because PySequence_Repeat(v, w) works by applying w.__index__ in
>> order to call v->sq_repeat.  However, __index__ is defined to clip the
>> result to fit in a Py_ssize_t.
>
> Clipping the result sounds like it would *never* be a good idea. What was
> the rationale for that? It should throw an exception.

Why would you expect range(10)[:2**32-1] and range(10)[:2**32] to do
different things?

Cheers,
mwh

-- 
  This makes it possible to pass complex object hierarchies to
  a C coder who thinks computer science has made no worthwhile
  advancements since the invention of the pointer.
   -- Gordon McMillan, 30 Jul 1998
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] segmentation fault in Python 2.5b3 (trunk:51066)

[Michael Hudson]

Duncan Booth <[EMAIL PROTECTED]> writes:

> Does Coverity recognise objects on Python's internal pools as deallocated? 

Coverity doesn't work on that level; it analyzes source code, and
knows about Python's INCREFs and DECREFs.

> The moral is to regard the reference counting rules as law: no matter how 
> sure you are that you can cheat, don't or you'll regret it.

This is the truth.

Cheers,
mwh

-- 
 As it seems to me, in Perl you have to be an expert to correctly make
 a nested data structure like, say, a list of hashes of instances.  In
 Python, you have to be an idiot not  to be able to do it, because you
 just write it down. -- Peter Norvig, comp.lang.functional
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] unicode hell/mixing str and unicode as dictionary keys

[Michael Hudson]

"M.-A. Lemburg" <[EMAIL PROTECTED]> writes:

> The point here is that a typical user won't expect any comparisons
> to be made when dealing with dictionaries, simply because the fact
> that you do need to make comparisons is an implementation detail.

Of course looking things up in a dictionary involves comparisons!  How
could it not?

> So in this particular case silencing the exception might be the
> more user friendly way of dealing with the problem.

Please, no.

> That said, the problem still lingers in that dictionary, so it may
> bite you in some other context, e.g. when iterating over the list
> of keys.

For this reason, and others.

Cheers,
mwh

-- 
   web in my head get it out get it out
-- from Twisted.Quotes
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Dicts are broken Was: unicode hell/mixing str and unicode asdictionarykeys

[Michael Hudson]

Michael Chermside <[EMAIL PROTECTED]> writes:

> I'm changing the subject line because I want to convince everyone that
> the problem being discussed in the "unicode hell" thread has nothing
> to do with unicode and strings. It's all about dicts.

I'd say it's more to do with __eq__.  It's a strange __eq__ method
that raises an Exception, IMHO.

Please do realize that the motivation for this change was hours and
hours of tortous debugging caused by a buggy __eq__ method making keys
"impossibly" seem to not be in dictionaries.

> I have not observed real breakage in my own code, but I will present
> a sample of made-up-but-completely-reasonable code that works in
> 2.4, fails in 2.5, and arguably ought to work fine. I think we should
> restore the behavior of dicts that when they compare keys for
> equality they suppress exceptions (treating the objects as unequal),
> or at LEAST retain the behavior for one more release making it a
> warning this time.

Please no.  Here's just one piece of evidence that the 2.4 semantics
are pretty silly too:

>>> d = {u'\xe0':1, '\xe0\:2}
  File "", line 1
d = {u'\xe0':1, '\xe0\:2}
^
SyntaxError: EOL while scanning single-quoted string
>>> d = {u'\xe0':1, '\xe0':2}
>>> '\xe0' in d
True
>>> '\xe0' in d.keys()
Traceback (most recent call last):
  File "", line 1, in ?
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe0 in position 0: ordinal 
not in range(128)

Cheers,
mwh

-- 
  same software, different verbosity settings (this one goes to
  eleven) -- the effbot on the martellibot
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] SyntaxError: can't assign to function call

[Michael Hudson]

Neal Becker <[EMAIL PROTECTED]> writes:

> 1) Should assignment to a temporary object be allowed?

The question doesn't make sense: in Python, you assign to a name,
an attribute or a subscript, and that's it.

Cheers,
mwh

-- 
   I think there's a rather large difference between a stale
twinkie and a kernel swap daemon
-- from Twisted.Quotes
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Dict suppressing exceptions

[Michael Hudson]

"Jim Jewett" <[EMAIL PROTECTED]> writes:

>> It wasn't my idea to stop ignoring exceptions in dict lookups; I would
>> gladly have put this off until Py3k, where the main problem
>> (str-unicode __eq__ raising UnicodeError) will go away.
>
>> But since people are adamant that they want this in sooner,
>
> Is this true for dictionaries specifically?
> 
> Would there really be strong objections to continuing to swallow
> any Exception (not BaseException) raised by __eq__ ?

Armin's reason for changing dictionaries in this way was that enormous
debugging pain was caused by dicts swallowing exceptions raised by
__eq__ methods.  Having the __eq__ methods swallow the exceptions by
themselves would make the situation *worse*, not better.

Cheers,
mwh

-- 
  * vegai wears his reading bra.
   umm, I mean glasses   -- from Twisted.Quotes
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] What should the focus for 2.6 be?

[Michael Hudson]

"A.M. Kuchling" <[EMAIL PROTECTED]> writes:

> On Mon, Aug 21, 2006 at 12:24:54PM -0700, Brett Cannon wrote:
>> As for the docs, they just need a thorough updating.  
>
> Michael Hudson was working on a new guide to extending/embedding
> Python.  Incorporating that should be a goal for 2.6 (the document may
> still need to be finished -- I'm not sure what state it's in).

Oh my word, even I had nearly forgotten about that.

I think the latest version is here:

   http://starship.python.net/crew/mwh/toext/

and source is here:

   http://codespeak.net/svn/user/mwh/toext/

It's certainly not even half-finished, and I'm not sure how much time
or enthusiasm I'm likely to be able to find for it in the medium term.

Cheers,
mwh

-- 
  ZAPHOD:  OK, so ten out of ten for style, but minus several million
   for good thinking, eh?
-- The Hitch-Hikers Guide to the Galaxy, Episode 2
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Signals, threads, blocking C functions

[Michael Hudson]

"Gustavo Carneiro" <[EMAIL PROTECTED]> writes:

> According to [1], all python needs to do to avoid this problem is
> block all signals in all but the main thread;

Argh, no: then people who call system() from non-main threads end up
running subprocesses with all signals masked, which breaks other
things in very mysterious ways.  Been there...

No time to read the rest of the post, maybe in a few days...

Cheers,
mwh

-- 
  Agh, the braindamage!  It's not unlike the massively
  non-brilliant decision to use the period in abbreviations
  as well as a sentence terminator.  Had these people no
  imagination at _all_? -- Erik Naggum, comp.lang.lisp
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Signals, threads, blocking C functions

[Michael Hudson]

"Gustavo Carneiro" <[EMAIL PROTECTED]> writes:

> On 9/4/06, Nick Maclaren <[EMAIL PROTECTED]> wrote:
>> "Gustavo Carneiro" <[EMAIL PROTECTED]> wrote:
>> >   I am now thinking of something along these lines:
>> > typedef void (*PyPendingCallNotify)(void *user_data);
>> > PyAPI_FUNC(void) Py_AddPendingCallNotify(PyPendingCallNotify callback,
>> > void *user_data);
>> > PyAPI_FUNC(void) Py_RemovePendingCallNotify(PyPendingCallNotify
>> > callback, void *user_data);
>>
>> Why would that help?  The problems are semantic, not syntactic.
>>
>> Anthony Baxter isn't exaggerating the problem, despite what you may
>> think from his posting.
>
>   You guys are tough customers to please. 

Yes.

> I am just trying to solve a problem here, not create a new one; you
> have to believe me.

We believe you, but you are stirring the ashes of old problems.

>  1. In PyGTK we have a gobject.MainLoop.run() method, which blocks
> essentially forever in a poll() system call, and only wakes if/when it
> has to process timeout or IO event;
>  2. When we only have one thread, we can guarantee that e.g.
> SIGINT will always be caught by the thread running the
> g_main_loop_run(), so we know poll() will be interrupted and a EINTR
> will be generated, giving us control temporarily back to check for
> python signals;
>  3. When we have multiple thread, we cannot make this assumption,
> so instead we install a timeout to periodically check for signals.
>
>   We want to get rid of timeouts.  Now my idea: add a Python API to say:
>  "dear Python, please call me when you start having pending calls,
> even if from a signal handler context, ok?"

This seems a reasonable proposal.  But it's totally a Python 2.6
thing, so how about taking a deep breath, working on a patch and
submitting it when it's ready?

Having to wake a process up a few times a second is ugly and annoying,
sure, but it is not a release delaying problem.

Cheers,
mwh

-- 
  It is never worth a first class man's time to express a majority
  opinion.  By definition, there are plenty of others to do that.
-- G. H. Hardy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Change in file() behavior in 2.5

[Michael Hudson]

"Michael Urman" <[EMAIL PROTECTED]> writes:

> Hi folks,
>
> Between 2.4 and 2.5 the behavior of file or open with the mode 'wU'
> has changed. In 2.4 it silently works. in 2.5 it raises a ValueError.
> I can't find any more discussion on it in python-dev than tangential
> mentions in this thread:
>   http://mail.python.org/pipermail/python-dev/2006-June/065939.html
>
> It is (buried) in NEWS. First I found:
>   Bug #1462152: file() now checks more thoroughly for invalid mode
>   strings and removes a possible "U" before passing the mode to the
>   C library function.
> Which seems to imply different behavior than the actual entry:
>   bug #967182: disallow opening files with 'wU' or 'aU' as specified by PEP
>   278.
>
> I don't see anything in pep278 about a timeline, and wanted to make
> sure that transitioning directly from working to raising an error was
> a desired change. 

That it was silently ignored was never intentional; it was a bug and
it was fixed.  I don't think having a release with deprecation
warnings and so on is worth it.

> This actually caught a bug in an application I work with, which used
> an explicit 'wU', that will currently stop working when people
> upgrade Python but not our application.

I would hope they wouldn't do that without careful testing anyway.

Cheers,
mwh

-- 
  No.  In fact, my eyeballs fell out just from reading this question,
  so it's a good thing I can touch-type.
-- John Baez, sci.physics.research
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] _PyGILState_NoteThreadState should be static or not?

[Michael Hudson]

On 11 Sep 2006, at 09:34, Neal Norwitz wrote:

> Michael,
>
> In Python/pystate.c, you made this checkin:
>
> """
> r39044 | mwh | 2005-06-20 12:52:57 -0400 (Mon, 20 Jun 2005) | 8 lines
>
> Fix bug:  [ 1163563 ] Sub threads execute in restricted mode
> basically by fixing bug 1010677 in a non-broken way.
> """
>
> _PyGILState_NoteThreadState is declared as static on line 54, but the
> definition on line 508 is not static. (HP's cc is complaining.)  I
> don't see this referenced in any header file, it seems like this
> should be static?

It seems very likely, yes.  I think at one point (in my working copy)  
there was a call in Modules/threadmodule.c, which may partially  
account for my confusion.

Seems we have lots of HP users tracking SVN HEAD, then...

Cheers,
mwh

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Signals, threads, blocking C functions

[Michael Hudson]

"Martin v. Löwis" <[EMAIL PROTECTED]> writes:

> Michael Hudson schrieb:
>>> According to [1], all python needs to do to avoid this problem is
>>> block all signals in all but the main thread;
>> 
>> Argh, no: then people who call system() from non-main threads end up
>> running subprocesses with all signals masked, which breaks other
>> things in very mysterious ways.  Been there...
>
> Python should register a pthread_atfork handler then, which clears
> the signal mask. Would that not work?

Not for system() at least:

http://mail.python.org/pipermail/python-dev/2003-December/041303.html

Cheers,
mwh

-- 
  ROOSTA:  Ever since you arrived on this planet last night you've
   been going round telling people that you're Zaphod
   Beeblebrox, but that they're not to tell anyone else.
-- The Hitch-Hikers Guide to the Galaxy, Episode 7
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] AST structure and maintenance branches

[Michael Hudson]

Anthony Baxter <[EMAIL PROTECTED]> writes:

> I'd like to propose that the AST format returned by passing PyCF_ONLY_AST to 
> compile() get the same guarantee in maintenance branches as the bytecode 
> format - that is, unless it's absolutely necessary, we'll keep it the same. 
> Otherwise anyone trying to write tools to manipulate the AST is in for a 
> massive world of hurt.
>
> Anyone have any problems with this, or can it be added to PEP 6?

Sounds like a good idea.

Cheers,
mwh

-- 
  Reading Slashdot can [...] often be worse than useless, especially
  to young and budding programmers: it can give you exactly the wrong
  idea about the technical issues it raises.
 -- http://www.cs.washington.edu/homes/klee/misc/slashdot.html#reasons
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Minipython

[Michael Hudson]

Milan Krcmar <[EMAIL PROTECTED]> writes:

> I would like to run Python scripts on an embedded MIPS Linux platform
> having only 2 MiB of flash ROM and 16 MiB of RAM for everything.
>
> Current (2.5) stripped and gzipped (I am going to use a compressed
> filesystem) CPython binary, compiled with defaults on a i386/glibc
> Linux, results in 500 KiB of "flash". How to make the Python interpreter
> even smaller?
>
> - can I completely drop out lexical analysis of sourcecode and compilation
>   to bytecode? is it relevant enough to the size of interpreter?

I don't think there's an configure flag for this or anything, and it
might be a bit hairy to do it, but it's possible and it would probably
save a bit.

There is a configure option to remove unicode support.  It's not
terribly well supported and stops working every now and again, but
it's probably much easier to start with.

There was at one point and may still be an option to not include the
complex type.

> - should I drop "useless" compiled-in modules? (what I need is a
>   replacement for advanced bash scripting, being able to write more
>   complex scripts and avoid forking tens of processes for things like
>   searching filesystem, formating dates etc.)

Yes, definitely.

> I don't want to re-invent the wheel, but all my attempts at finding
> Python for embedded systems ended in instructions for embedding
> Python in another program :-)
>
> Can you give me any information to start with? I would prefer stripping
> current version of Python rather than returning to a years-old (but
> smaller) version and remembering what of the new syntax/functionality to
> avoid.

Well, I would start by looking at what is taking up the space...

Cheers,
mwh

-- 
  C++ is a siren song.  It *looks* like a HLL in which you ought to
  be able to write an application, but it really isn't.
   -- Alain Picard, comp.lang.lisp
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Typo.pl scan of Python 2.5 source code

[Michael Hudson]

"Neal Norwitz" <[EMAIL PROTECTED]> writes:

> I ignored these as I'm not certain all the platforms we run on accept
> free(NULL).

It's mandated by C99, and I don't *think* it changed from the previous
version (I only have a bootleg copy of C99 :).

Cheers,
mwh

-- 
  TRSDOS: Friendly old lizard. Or, at least, content to sit there
  eating flies.-- Jim's pedigree of operating systems, asr
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 355 status

[Michael Hudson]

[EMAIL PROTECTED] writes:

> I hope that eventually Python will include some form of OO
> filesystem access, but I am equally hopeful that the current PEP 355
> path.py is not it.

I think I agree with this too.  For another source of ideas there is
the 'py.path' bit of the py lib, which, um, doesn't seem to be
documented terribly well, but allows access to remote svn repositories
as well as local filesytems (at least).

Cheers,
mwh

-- 
3. Syntactic sugar causes cancer of the semicolon.
  -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Caching float(0.0)

[Michael Hudson]

"Martin v. Löwis" <[EMAIL PROTECTED]> writes:

> Kristján V. Jónsson schrieb:
>> I can't see how this situation is any different from the re-use of
>> low ints.  There is no fundamental law that says that ints below 100
>> are more common than other, yet experience shows that  this is so,
>> and so they are reused.
>
> There are two important differences:
> 1. it is possible to determine whether the value is "special" in
>constant time, and also fetch the singleton value in constant
>time for ints; the same isn't possible for floats.

I don't think you mean "constant time" here do you?  I think most of
the code posted so far has been constant time, at least in terms of
instruction count, though some might indeed be fairly slow on some
processors -- conversion from double to integer on the PowerPC
involves a trip off to memory for example.  Even so, everything should
be fairly efficient compared to allocation, even with PyMalloc.

> 2. it may be that there is a loss of precision in reusing an existing
>value (although I'm not certain that this could really happen).
>For example, could it be that two values compare successful in
>==, yet are different values? I know this can't happen for
>integers, so I feel much more comfortable with that cache.

I think the only case is that the two zeros compare equal, which is
unfortunate given that it's the most compelling value to cache...

I don't know a reliable and fast way to distinguish +0.0 and -0.0.

Cheers,
mwh

-- 
  The bottom tier is what a certain class of wanker would call
  "business objects" ...  -- Greg Ward, 9 Dec 1999
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Segfault in python 2.5

[Michael Hudson]

"Mike Klaas" <[EMAIL PROTECTED]> writes:

> [http://sourceforge.net/tracker/index.php?func=detail&aid=1579370&group_id=5470&atid=105470]
>
> Hello,
>
> I'm managed to provoke a segfault in python2.5 (occasionally it just a
> "invalid argument to internal function" error).  I've posted a
> traceback and a general idea of what the code consists of in the
> sourceforge entry. 

I've been reading the bug report with interest, but unless I can
reproduce it it's mighty hard for me to debug, as I'm sure you know.

> Unfortunately, I've been attempting for hours to
> reduce the problem to a completely self-contained script, but it is
> resisting my efforts due to timing problems.
>
> Should I continue in that vein, or is it more useful to provide more
> detailed results from gdb?

Well, I don't think that there's much point in posting masses of
details from gdb.  You might want to try trying to fix the bug
yourself I guess, trying to figure out where the bad pointers come
from, etc.

Are you absolutely sure that the fault does not lie with any extension
modules you may be using?  Memory scribbling bugs have been known to
cause arbitrarily confusing problems...

Cheers,
mwh

-- 
  I'm not sure that the ability to create routing diagrams 
  similar to pretzels with mad cow disease is actually a 
  marketable skill. -- Steve Levin
   -- http://home.xnet.com/~raven/Sysadmin/ASR.Quotes.html
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Last chance to join the Summer of PyPy!

[Michael Hudson]

Hopefully by now you have heard of the "Summer of PyPy", our program
for funding the expenses of attending a sprint for students.  If not,
you've just read the essence of the idea :-)

However, the PyPy EU funding period is drawing to an end and there is
now only one sprint left where we can sponsor the travel costs of
interested students within our program. This sprint will probably take
place in Leysin, Switzerland from 8th-14th of January 2007.

So, as explained in more detail at:

http://codespeak.net/pypy/dist/pypy/doc/summer-of-pypy.html

we would encourage any interested students to submit a proposal in the
next month or so.  If you're stuck for ideas, you can find some at

http://codespeak.net/pypy/dist/pypy/doc/project-ideas.html

but please do not feel limited in any way by this list!

Cheers,
mwh

... and the PyPy team

-- 
   the highest calling of technical book writers is to
  destroy the sun   -- from Twisted.Quotes
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] fpectl: does a better implementation make sense?

[Michael Hudson]

Giovanni Bajo <[EMAIL PROTECTED]> writes:

> Hello,
>
> I spent my last couple of hourse reading several past threads about fpectl. 
> If 
> I'm correct
>
> 1) fpectl is scheduled for deletion in 2.6.
> 2) The biggest problem is that the C standard says that it's undefined to 
> return from a SIGFPE handler.

Well, I'm not sure that's the "biggest" problem in any sense: C also
doesn't say anything about how to set things up so that 1.0/0.0 gets
you a SIGFPE.

> Thus, it's impossible to traps floating point 
> exceptions and convert them to Python exceptions in a way that really works.

"Impossible" is a strong word :-)  But yeah.

> 3) Moreover, the current implementation of PyFPE_* macros (turning on/off the 
> traps + setjmp) are pretty slow, so they're off by default.
>
> Now, I would like Python to rause exceptions (FloatingPointError) whenever a 
> Inf or NaN is computed or used in calculations (which to the best of my 
> little 
> understanding of 754 basically means that I want all FPU errors to be 
> detected and handled). 

I suggest starting from here and forgetting about fpectl.

> I am not arguing that this should be the default behaviour,

Good :-)

> I'm just saying that I would like if there was a way to enable this
> (even if only at Python compile time, in fact).

I think I would too.

> I read that Tim Peters suggested several times to rewrite fpectl so that it 
> does not use traps/signals at all, but just checks the FPU bits to see if 
> there was a FPU error. Basically, the PyFPE BEGIN macro would clear the FPU 
> bits, and the STOP macro would check for FPU errors and raise an appropriate 
> exception if needed.

This is part of the reason I want to rip out fpectl: I think a new set
of macros is called for, if only so they can have better names.  But
you'll end up fighting a series of aggravating battles with compiler
optimizations and platform support and so on.  Did you read this post:

http://mail.python.org/pipermail/python-list/2005-July/331509.html

and the thread it came from?

> Is this suggestion still valid or people changed their mind meanwhile?

I haven't changed my mind appreciably.

> Would such a rewrite of fpectl (or a new module with a different
> name) be accepted?

I would at least try to review it and press for its inclusion.

Cheers,
mwh

-- 
  The bottom tier is what a certain class of wanker would call
  "business objects" ...  -- Greg Ward, 9 Dec 1999
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-3000] Warning for 2.6 and greater

[Michael Hudson]

Georg Brandl <[EMAIL PROTECTED]> writes:

> Armin Rigo schrieb:
>> Hi Paul,
>> 
>> On Wed, Jan 10, 2007 at 11:10:10PM +, Paul Moore wrote:
>>> How many other projects/packages anticipate *not* migrating to Py3K, I 
>>> wonder?
>> 
>> FWIW: Psyco.
>
> What will PyPy do? It will certainly support compiling Py3k code at some 
> point,
> but will the codebase itself be converted?

For practical reasons (we have enough work to be getting on with) PyPy
is more-or-less ignoring Python 2.5 at the moment.  After funding and
so on, when there's less pressure, maybe it will seem worth it.  Not
soon though.

Cheers,
mwh

-- 
  I'm okay with intellegent buildings, I'm okay with non-sentient
  buildings. I have serious reservations about stupid buildings.
 -- Dan Sheppard, ucam.chat (from Owen Dunn's summary of the year)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-3000] Warning for 2.6 and greater

[Michael Hudson]

[EMAIL PROTECTED] writes:

> On 10:12 am, [EMAIL PROTECTED] wrote:
>
>>For practical reasons (we have enough work to be getting on with) PyPy
>>is more-or-less ignoring Python 2.5 at the moment.  After funding and
>>so on, when there's less pressure, maybe it will seem worth it.  Not
>>soon though.
>
> I think I know what you mean from previous conversations, but the
> context of the question makes the answer ambiguous.  Are you
> ignoring 2.5 in favor of 2.4, or 3.0?

In favour of 2.4 -- the one that exists now :) 

Cheers,
mwh

-- 
  > It might get my attention if you'd spin around in your chair,
  > spoke in tongues, and puked jets of green goblin goo.
  I can arrange for this.  ;-)-- Barry Warsaw & Fred Drake
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Deletion order when leaving a scope?

[Michael Hudson]

Nick Coghlan <[EMAIL PROTECTED]> writes:

> Neil Toronto wrote:
>> I imagine this would be important to someone expecting system resources 
>> to be cleaned up, closed, deallocated, or returned inside of __del__ 
>> methods. Someone coming from C++ might expect LIFO behavior because 
>> common idioms like RAII (Resource Allocation Is Instantiation) don't 
>> work otherwise. A Java programmer wouldn't care, being used to cleaning 
>> up resources manually with a try...catch...finally.
>> 
>> I'm just putting a possible motivation on the concern. It happens that 
>> the Pythonic Way is also the Java Way in this area: don't expect any 
>> specific deletion order (don't even expect a guaranteed call to 
>> __del__), and manually clean up after yourself. As a warning, this has 
>> been the subject of a great many flame wars between C++ and Java 
>> programmers...
>
> We know. Python 2.5 added a new statement (with) and a new standard 
> library module (contextlib) to allow resource deallocation to be dealt 
> with cleanly without requiring assumptions about the interpreter's 
> memory model.
>
> While RAII isn't mentioned explicitly in the corresponding PEP (PEP 
> 343), it was certainly a factor in the python-dev discussions.

It's mentioned in PEP 310, the predecessor to PEP 343.

Cheers,
mwh

-- 
 "I lacked the courage to investigate the weaknesses of the wicked,
  because I discovered they are the same as the weaknesses of the
  saintly."   -- The Name Of The Rose, Umberto Eco
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Eliminating f_tstate

[Michael Hudson]

"Martin v. Löwis" <[EMAIL PROTECTED]> writes:

> Bug #1579370 reports a crash when accessing the thread state of
> a terminated thread, when releasing a generator object.
>
> In analysing the problem, I found that f_tstate doesn't have much
> utility: it is used in very few places, and in these places, it
> might be as good or better to use the *current* thread state
> (rather than the frame's thread state); in some (most?) of these
> places, the current thread should be identical to the frame's
> thread, anyway.
>
> So I would like to propose that the f_tstate member is removed
> from the frame object.
>
> For Python 2.5, for compatibility, it probably has to remain
> where it is, and only PyTraceBack_Here should stop using it.
> As a consequence, a generator .send() makes exceptions
> occur in the current thread, not in the thread where the
> generator was created.
>
> What do you think?

Without having read the code in detail, I think what you say makes a
great deal of sense.

Cheers,
mwh

-- 
  (FREE|OPEN) BSD: Shire horse. Solid, reliable, only occasionally
  prone to crushing you against a wall and then only because
  you've told it to without knowing.
   -- Jim's pedigree of operating systems, asr
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] buildbot failure in amd64 gentoo 2.5

[Michael Hudson]

Giovanni Bajo <[EMAIL PROTECTED]> writes:

> On 23/01/2007 10.20, Brian Warner wrote:
>
 Do I miss something here, or is the buildbot hit by spammers now?
>>> It looks like it is. If that continues, we have to disable the web
>>> triggers.
>> 
>> Good grief. If anyone has any bright ideas about simple ways to change that
>> form to make it less vulnerable to the spambots, I'd be happy to incorporate
>> them into Buildbot.
>
> I'd throw a CAPTCHA in. There are even some written in Python.

I'd guess even the simplest thing would work:

"Type "Python" into this box: __"

Cheers,
mwh

-- 
  ZAPHOD:  Who are you?
  ROOSTA:  A friend.
  ZAPHOD:  Oh yeah? Anyone's friend in particular, or just generally 
   well-disposed to people?   -- HHGttG, Episode 7
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] head crashing (was: Fwd: [Python-c heckins] buildbot warnings in x86 mvlgcc trunk)

[Michael Hudson]

Neal Norwitz  gmail.com> writes:

> 
> Can you call PyMem_FREE() without the GIL held?  I couldn't find it
> documented either way.

Nope.  See comments at the top of Python/pystate.c.

Cheers,
mwh

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] test_shutils.py fails for 2.4 install

[Michael Hudson]

"Edward C. Jones" <[EMAIL PROTECTED]> writes:

> I have a PC with an AMD cpu and Mandrake 10.1. While installing Python
> 2.4 "make test" failed in "test_shutils.py". Here is the output of
> "./python ./Lib/test/test_shutil.py":

Are you running 'make test' as root?  Don't do that (although possibly
the test suite should guard against it).

Also, did you search the bug tracker?  Please do do that.

Cheers,
mwh

-- 
   CDATA is not an integration strategy.
-- from Twisted.Quotes
___
Python-Dev mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Any reason why CPPFLAGS not used in compiling?

[Michael Hudson]

"Brett C." <[EMAIL PROTECTED]> writes:

> I noticed that Makefile.pre.in uses the value from the environment
> variable LDFLAGS but not CPPFLAGS.  Any reason for this?
> ./configure -h`` lists both (and some other environment variables
> which are not really used either) so it seems a little misleading to
> have those listed but to not utilize them.

The whole story of how the compiling/linker flags get set up is a bit
of a mess, AFAIK.  I don't have the energy or the desire to try to
make what we want to happen precise (the hard bit) and then make that
happen (probably rather easier).

> This should allow for the removal of the direct support for Fink and
> DarwinPorts.

+lots.  I don't trust fink, but need it for latex...

Cheers,
mwh

-- 
  After a heavy night I travelled on, my face toward home - the comma
  being by no means guaranteed.   -- paraphrased from cam.misc
___
Python-Dev mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Deprecated xmllib module

[Michael Hudson]

"Martin v. Löwis" <[EMAIL PROTECTED]> writes:

> Anthony Baxter wrote:
>> The library docs for, e.g. xmllib already say deprecated at the top - maybe
>> it should be larger?
>
> I think it would be better to *remove* the xmllib documentation.
> Existing code which needs the module will continue to work even
> without documentation, and new code is unlikely to be written for
> a module that has no documentation, and where importing the module
> gives a DeprecationWarning.

+1, at least for *hiding* the documentation of xmllib + similar
modules.  I'm not sure wholesale removal is really a good idea.

It's like the new "Non-essential Built-in Functions" section of the
lib ref.

Cheers,
mwh

-- 
   jemfinch: What's to parse?  A numeric code, perhaps a
  chicken, and some arguments
-- from Twisted.Quotes
___
Python-Dev mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Re: [Python-checkins] python/dist/src/Lib/test test_shutil.py, 1.10, 1.11

[Michael Hudson]

[EMAIL PROTECTED] writes:

> Update of /cvsroot/python/python/dist/src/Lib/test
> In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv20531
>
> Modified Files:
>   test_shutil.py 
> Log Message:
> SF bug #1076467: don't run test_on_error as root, as the permission
> errors don't get provoked that way. Also add a bunch of cross-references
> to bugs.
>
>
> Index: test_shutil.py
> ===
> RCS file: /cvsroot/python/python/dist/src/Lib/test/test_shutil.py,v
> retrieving revision 1.10
> retrieving revision 1.11
> diff -u -d -r1.10 -r1.11
> --- test_shutil.py23 Nov 2004 09:27:27 -  1.10
> +++ test_shutil.py6 Dec 2004 20:50:15 -   1.11
> @@ -16,7 +16,10 @@
>  filename = tempfile.mktemp()
>  self.assertRaises(OSError, shutil.rmtree, filename)
>  
> -if hasattr(os, 'chmod') and sys.platform[:6] != 'cygwin':
> +# See bug #1071513 for why we don't run this on cygwin
> +# and bug #1076467 for why we don't run this as root.
> +if (hasattr(os, 'chmod') and sys.platform[:6] != 'cygwin'
> +and os.getenv('USER') != 'root'):

Is that really the best way to check for root?  It may be, I guess,
but I'd have expected os.geteuid() to be more reliable...

Or is it os.getuid()?  I always get confused by these functions...

Cheers,
mwh

-- 
   CDATA is not an integration strategy.
-- from Twisted.Quotes
___
Python-Dev mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python 2.4] PyInt_FromLong returning NULL

[Michael Hudson]

Andreas Jung <[EMAIL PROTECTED]> writes:

> While using Zope 2.7 with Python 2.4 we discovered some strange
> behaviour of the security machinery.
> I could track this down to some Zope code in cAccessControl.c where an
> Unauthorized exception is
> raised because of a call to PyInt_FromLong(1) which returns NULL. What
> could be the reason that
> such a "stupid" call return NULL in a reproducable way?

A memory scribble somewhere else?  Possibly a DECREF too many somewhere?

Is an exception set?  Have you tried a debug build?  Etc.

Cheers,
mwh

-- 
  All obscurity will buy you is time enough to contract venereal
  diseases.  -- Tim Peters, python-dev
___
Python-Dev mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Re: 2.4 news reaches interesting places

[Michael Hudson]

"Robert Brewer" <[EMAIL PROTECTED]> writes:

> Anthony Baxter wrote:
>> FWIW, I was planning on writing a tutorial (working title:
>> "Making Python Code Not Suck") for some conference
>> or another...
>
> Perhaps, given your high profile in the Python developer community, you
> might reconsider the title? Little details like that are what PR is made
> of. Imagine Bush's next Executive Order being titled "Making American
> [Business|Military|People] Not Suck"... ;)

Anthony's Australian, people expect this sort of thing from him :)

Cheers,
mwh

-- 
  Also, remember to put the galaxy back when you've finished, or an
  angry mob of astronomers will come round and kneecap you with a
  small telescope for littering. 
   -- Simon Tatham, ucam.chat, from Owen Dunn's review of the year
___
Python-Dev mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 2.4 news reaches interesting places

[Michael Hudson]

Jeremy Hylton <[EMAIL PROTECTED]> writes:

> I agree, although it's not clear to me how much faster it will be in
> the future.  Making a *fast* Python based on our own virtual execution
> environment (as opposed to piggybacking a JVM or CLR) is a big
> project.  It's not clear than anyone has enough resources to make
> credible effort

I think the EU does!

Cheers,
mwh

-- 
  NUTRIMAT:  That drink was individually tailored to meet your
 personal requirements for nutrition and pleasure.
ARTHUR:  Ah.  So I'm a masochist on a diet am I?
-- The Hitch-Hikers Guide to the Galaxy, Episode 9
___
Python-Dev mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PyOS_InputHook enhancement proposal

[Michael Hudson]

Keith Dart <[EMAIL PROTECTED]> writes:

> I did modify the readline module that hooks this and can call back
> to a Python function. There are also methods for installing and
> removing the Python function. I did this for a different reason. I
> need Python signal handlers to run, and they don't run when the
> execution path is currently in some C code (such as readline). The
> hook forces the interpreter to run, and check for signals as a side
> effect. Using this I can be sitting in raw_input(), or interactive
> mode, and signal handlers are still run.

Have you seen what happened to the signal handling code in readline in
Python 2.4?  I don't think your modifications will be needed anymore,
but you should check.

Cheers,
mwh

-- 
  I don't remember any dirty green trousers.
 -- Ian Jackson, ucam.chat
___
Python-Dev mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Re: Re: 2.4 news reaches interesting places

[Michael Hudson]

Keith Dart <[EMAIL PROTECTED]> writes:

> A.M. Kuchling wrote:
>> On Sun, Dec 12, 2004 at 03:32:03PM -0200, Carlos Ribeiro wrote:
>> 
>>>Of course, the point here is not Perl-bashing. The point here is that
>>>we should be able to "sell" Python better than we do now, even without
>>>the need to resort to such poor measures. I'm sure the Python
>>>community does have good & creative people that can write a good
>>>"selling" FAQ for Python, emphasizing the main points of the language.
>> No one disagrees that Python needs better marketing material.  At
>> the
>> last PyCon a group of people sat down in a pydotorg BoF and agreed
>> that yes, we do need a management-friendly marketing site, and that we
>> could put it on a separate hostname (something.python.org) so that the
>> current www.python.org wouldn't have to be changed.
>> However, no one has actually sat down and written such a site, or
>> even
>> outlined it.  Let me encourage you to go ahead and do that.  You could
>> draft the outline on a Wiki page, and then later figure out an
>> attractive design and organization for a new site.
>
>
> Whatever it looks like, it should probably run on Zope plus Plone. 8-) 
> You know... eat your own dog food. 8-)

This is SO not the problem that needs discussion...

> The kind folks over at Zettai! have provided some space for
> me. Perhaps they will be glad to host the main Python site, as well?

Neither is this!

Cheers,
mwh

-- 
  Just point your web browser at http://www.python.org/search/ and
  look for "program", "doesn't", "work", or "my". Whenever you find
  someone else whose program didn't work, don't do what they
  did. Repeat as needed.-- Tim Peters, on python-help, 16 Jun 1998
___
Python-Dev mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] mmap feature or bug?

[Michael Hudson]

Josiah Carlson <[EMAIL PROTECTED]> writes:

> Quick questions:
> Is mmap's inability to be subclassed a feature or bug?

Like Martin said, a missing feature.  I'd also quite like the mmap
module to have a C API, but as my use case is especially evil (

http://starship.python.net/crew/mwh/hacks/PPY.html

) I'm not going to push that hard :)

> Is one's inability to use a = mmapinstance.__setslice__;a(1,2,'a') (and
> others) a feature or bug?

I'm not entirely sure about this, though.  Does 

mmapinstance.__setitem__(slice(1,2), 'a')

work?  AIUI __setslice__ and __getslice__ are out of favour these days
(not sure though).

Any patch to fix one of these would probably also fix the other almost
as a sside-effect...

Cheers,
mwh

-- 
  ... with these conditions cam the realisation that ... nothing
  turned a perfectly normal healthy individual into a great political
  or military leader better than irreversible brain damage.
   -- The Hitch-Hikers Guide to the Galaxy, Episode 11
___
Python-Dev mailing list
[EMAIL PROTECTED]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Patches: 1 for the price of 10.

[Michael Hudson]

Titus Brown <[EMAIL PROTECTED]> writes:

> -> > 1067760 -- float-->long conversion on fileobj.seek calls, rather than
> -> >float-->int.  Permits larger floats (2.0**62) to match large
> -> >int (2**62) arguments.  rhettinger marked as "won't fix" in
> -> >the original bug report; this seems like a clean solution,
> -> >tho.  Recommend apply.
> -> 
> -> Wouldn't this cause subtle errors when the float -> long conversion is
> -> no longer precise? Or is this a non issue because it could only happen
> -> when seeking on impossibly large files?
>
> When would the float --> long conversion not be precise?

When the float is so large that it is the closest approximation to
more than one integer? (i.e. larger than 2**53 for 754 doubles).

Cheers,
mwh

-- 
  #ifndef P_tmpdir
  printf( "Go buy a better computer" );
  exit( ETHESKYISFALLINGANDIWANTMYMAMA );
 -- Dimitri Maziuk on writing secure code, asr
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Patches: 1 for the price of 10.

[Michael Hudson]

Titus Brown <[EMAIL PROTECTED]> writes:

> -> > Apparently file.seek doesn't have this DeprecationWarning though..
> -> > Strange, that.
> -> >  >>> f.seek(3.6)
> -> >  >>> f.tell()
> -> > 3L
> -> 
> -> That's a bug. Who'll fix it?
>
> Added:
>
> + if (PyFloat_Check(offobj))
> + PyErr_Warn(PyExc_DeprecationWarning,
> +"integer argument expected, got float" );
>
> see attached diff.  I also attached it to the patch at SourceForge,
> with a comment, just to keep the record straight.

You should check the return value of PyErr_Warn in case the user
passed -Werror on the command line.

> This seems like a reasonable resolution to patch #1067760, to me...

I guess I could look at that -- but when I'm not on dialup at my
parents' house...

Cheers,
mwh

-- 
  I love the way Microsoft follows standards.  In much the same
  manner that fish follow migrating caribou.   -- Paul Tomblin
   -- http://home.xnet.com/~raven/Sysadmin/ASR.Quotes.html
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Build extensions for windows python 2.4 what are the compiler rules?

[Michael Hudson]

Barry Scott <[EMAIL PROTECTED]> writes:

> I recursive grep'ed and missed this ref. However I did read this in
> README.TXT:

The top-level README file is hilariously out-of-date, in some ways.  I
meant to do something about this before 2.4 final, but didn't get
around to it...

Cheers,
mwh

-- 
   speak of the devil
   exarkun: froor
   not you -- from Twisted.Quotes
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Build extensions for windows python 2.4 what are the compiler rules?

[Michael Hudson]

Armin Rigo <[EMAIL PROTECTED]> writes:

> Hi,
>
> On Fri, Dec 24, 2004 at 12:17:49AM +, Barry Scott wrote:
>> I recursive grep'ed and missed this ref. However I did read this in 
>> README.TXT:
>
> The "extending and embedding" tutorial is similarily out-of-date.

Well, I've half re-written that (the entire thing, see doc-sig passim)
but this hasn't attracted a great deal of interest.  I've just
uploaded my most recent version to:

http://starship.python.net/crew/mwh/toext/

I haven't done any substantial work on it in a while, and probably
won't have time to do any for some time.  It would still be nice to
have it ready in the 2.5 timeframe.

Cheers,
mwh

-- 
  A difference which makes no difference is no difference at all.
-- William James (I think.  Reference anyone?)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 2.3.5 schedule, and something I'd like to get in

[Michael Hudson]

"Martin v. Löwis" <[EMAIL PROTECTED]> writes:

> Bob Ippolito wrote:
>> It doesn't for reasons I care not to explain in depth, again.
>> Search  the pythonmac-sig archives for longer explanations.  The
>> gist is that  you specifically do not want to link directly to the
>> framework at all  when building extensions.
>
> Because an Apple-built extension then may pick up a user-installed
> Python? Why can this problem not be solved by adding -F options,
> as Jack Jansen proposed?
>
>> This is not the wrong way to do it.
>
> I'm not convinced.

Martin, can you please believe that Jack, Bob, Ronald et al know what
they are talking about here?

Cheers,
mwh

-- 
  Q: Isn't it okay to just read Slashdot for the links?
  A: No. Reading Slashdot for the links is like having "just one hit"
 off the crack pipe.
 -- http://www.cs.washington.edu/homes/klee/misc/slashdot.html#faq
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] an idea for improving struct.unpack api

[Michael Hudson]

Ilya Sandler <[EMAIL PROTECTED]> writes:

> A problem:
>
> The current struct.unpack api works well for unpacking C-structures where
> everything is usually unpacked at once, but it
> becomes  inconvenient when unpacking binary files where things
> often have to be unpacked field by field. Then one has to keep track
> of offsets, slice the strings,call struct.calcsize(), etc...

IMO (and E), struct.unpack is the primitive atop which something more
sensible is built.  I've certainly tried to build that more sensible
thing at least once, but haven't ever got the point of believing what
I had would be applicable to the general case... maybe it's time to
write such a thing for the standard library.

Cheers,
mwh

-- 
  ARTHUR:  Ford, you're turning into a penguin, stop it.
-- The Hitch-Hikers Guide to the Galaxy, Episode 2
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] an idea for improving struct.unpack api

[Michael Hudson]

Bob Ippolito <[EMAIL PROTECTED]> writes:

> On Jan 6, 2005, at 8:17, Michael Hudson wrote:
>
>> Ilya Sandler <[EMAIL PROTECTED]> writes:
>>
>>> A problem:
>>>
>>> The current struct.unpack api works well for unpacking C-structures
>>> where
>>> everything is usually unpacked at once, but it
>>> becomes  inconvenient when unpacking binary files where things
>>> often have to be unpacked field by field. Then one has to keep track
>>> of offsets, slice the strings,call struct.calcsize(), etc...
>>
>> IMO (and E), struct.unpack is the primitive atop which something more
>> sensible is built.  I've certainly tried to build that more sensible
>> thing at least once, but haven't ever got the point of believing what
>> I had would be applicable to the general case... maybe it's time to
>> write such a thing for the standard library.
>
> This is my ctypes-like attempt at a high-level interface for struct.
> It works well for me in macholib:
> http://svn.red-bean.com/bob/py2app/trunk/src/macholib/ptypes.py

Unsurprisingly, that's fairly similar to mine :)

Cheers,
mwh

-- 
  If trees could scream, would we be so cavalier about cutting them
  down? We might, if they screamed all the time, for no good reason.
-- Jack Handey
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] an idea for improving struct.unpack api

[Michael Hudson]

Thomas Heller <[EMAIL PROTECTED]> writes:

>> Bob Ippolito <[EMAIL PROTECTED]> writes:
>
>>> This is my ctypes-like attempt at a high-level interface for struct.
>>> It works well for me in macholib:
>>> http://svn.red-bean.com/bob/py2app/trunk/src/macholib/ptypes.py
>
> Michael Hudson <[EMAIL PROTECTED]> writes:
>>
>> Unsurprisingly, that's fairly similar to mine :)
>
> So, why don't you both use the original?
> ctypes works on the mac, too ;-)

Well, I probably wrote mine before ctypes worked on the Mac... and
certainly when I was away from internet access.  I guess I should look
at ctypes' interface, at least...

Cheers,
mwh

-- 
  I located the link but haven't bothered to re-read the article,
  preferring to post nonsense to usenet before checking my facts.
  -- Ben Wolfson, comp.lang.python
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 246, redux

[Michael Hudson]

Alex Martelli <[EMAIL PROTECTED]> writes:

> I didn't know about the "let the object lie" quirk in isinstance.  If
> that quirk is indeed an intended design feature, rather than an
> implementation 'oops', it might perhaps be worth documenting it more
> clearly; I do not find that clearly spelled out in the place I'd
> expect it to be, namely
>  under 'isinstance'.

Were you not at the PyPy sprint where bugs in some __getattr__ method
caused infinite recursions on the isinstance's code attempting to
access __class__?  The isinstance code then silently eats the error,
so we had (a) a massive slowdown and (b) isinstance failing in an
"impossible" way.  A clue was that if you ran the code on OS X with
its silly default stack limits the code dumped core instead of going
slowly insane.

This is on quirk I'm not likely to forget in a hurry...

Cheers,
mwh

-- 
  If trees could scream, would we be so cavalier about cutting them
  down? We might, if they screamed all the time, for no good reason.
-- Jack Handey
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Exceptions *must*? be old-style classes?

[Michael Hudson]

Guido van Rossum <[EMAIL PROTECTED]> writes:

> To be honest, I don't recall the exact reasons why this wasn't fixed
> in 2.2; I believe it has something to do with the problem of
> distinguishing between string and class exception, and between the
> various forms of raise statements.

A few months back I hacked an attempt to make all exceptions
new-style.  It's not especially hard, but it's tedious.  There's lots
of code (more than I expected, anyway) to change and my attempt ended
up being pretty messy.  I suspect allowing both old- and new-style
classes would be no harder, but even more tedious and messy.

It would still be worth doing, IMHO.

Cheers,
mwh

-- 
  If you are anal, and you love to be right all the time, C++
   gives you a multitude of mostly untimportant details to fret about
   so you can feel good about yourself for getting them "right", 
   while missing the big picture entirely   -- from Twisted.Quotes
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Exceptions *must*? be old-style classes?

[Michael Hudson]

Guido van Rossum <[EMAIL PROTECTED]> writes:

> [Michael]
>> It would still be worth doing, IMHO.
>
> Then let's do it. Care to resurrect your patch? (And yes, classic
> classes should also be allowed for b/w compatibility.)

I found it and uploaded it here:

http://starship.python.net/crew/mwh/new-style-exception-hacking.diff

The change to type_str was the sort of unexpected change I was talking
about.

TBH, I'm not sure it's really worth working from my patch, a more
sensible course would be to just do the work again, but paying a bit
more attention to getting a maintainable result.

Questions:

a) Is Exception to be new-style?

b) Somewhat but not entirely independently, would demanding that all
   new-style exceptions inherit from Exception be reasonable?

Cheers,
mwh


-- 
  ZAPHOD:  You know what I'm thinking?
FORD:  No.
  ZAPHOD:  Neither do I.  Frightening isn't it?
   -- The Hitch-Hikers Guide to the Galaxy, Episode 11
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Exceptions *must*? be old-style classes?

[Michael Hudson]

"Martin v. Löwis" <[EMAIL PROTECTED]> writes:

> Guido van Rossum wrote:
a) Is Exception to be new-style?
>>>
>>>Probably not in 2.5; Martin and others have suggested that this could
>>>introduce instability for users' existing exception classes.
>> Really? I thought that was eventually decided to be a very small
>> amount of code.
>
> I still think that only an experiment could decide: somebody should
> come up with a patch that does that, and we will see what breaks.
>
> I still have the *feeling* that this has significant impact, but
> I could not pin-point this to any specific problem I anticipate.

Well, some code is certainly going to break such as this from
warnings.py:

assert isinstance(category, types.ClassType), "category must be a class"

or this from traceback.py:

if type(etype) == types.ClassType:
stype = etype.__name__
else:
stype = etype

I hope to have a new patch (which makes PyExc_Exception new-style, but
allows arbitrary old-style classes as exceptions) "soon".  It may even
pass bits of "make test" :)

Cheers,
mwh

-- 
  SPIDER:  'Scuse me. [scuttles off]
  ZAPHOD:  One huge spider.
FORD:  Polite though.
   -- The Hitch-Hikers Guide to the Galaxy, Episode 11
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Exceptions *must*? be old-style classes?

[Michael Hudson]

Michael Hudson <[EMAIL PROTECTED]> writes:

> I hope to have a new patch (which makes PyExc_Exception new-style, but
> allows arbitrary old-style classes as exceptions) "soon".  It may even
> pass bits of "make test" :)

Done: http://www.python.org/sf/1104669

It passed 'make test' apart from failures I really don't think are my
fault.  I'll run "regrtest -uall" overnight...

Cheers,
mwh

-- 
[1] If you're lost in the woods, just bury some fibre in the ground
carrying data. Fairly soon a JCB will be along to cut it for you
- follow the JCB back to civilsation/hitch a lift.
   -- Simon Burr, cam.misc
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Strange segfault in Python threads and linux kernel 2.6

[Michael Hudson]

Donovan Baarda <[EMAIL PROTECTED]> writes:

> G'day,
>
> I've Cc'ed this to zope-coders as it might affect other Zope developers
> and it had me stumped for ages. I couldn't find anything on it anywhere,
> so I figured it would be good to get something into google :-).
>
> We are developing a Zope2.7 application on Debian GNU/Linux that is
> using fop to generate pdf's from xml-fo data. fop is a java thing, and
> we are using popen2.Popen3(), non-blocking mode, and select loop to
> write/read stdin/stdout/stderr. This was all working fine.
>
> Then over the Christmas chaos, various things on my development system
> were apt-get updated, and I noticed that java/fop had started
> segfaulting. I tried running fop with the exact same input data from the
> command line; it worked. I wrote a python script that invoked fop in
> exactly the same way as we were invoking it inside zope; it worked. It
> only segfaulted when invoked inside Zope.
>
> I googled and tried everything... switched from j2re1.4 to kaffe, rolled
> back to a previous version of python, re-built Zope, upgraded Zope from
> 2.7.2 to 2.7.4, nothing helped. Then I went back from a linux 2.6.8
> kernel to a 2.4.27 kernel; it worked!
>
> After googling around, I found references to recent attempts to resolve
> some signal handling problems in Python threads. There was one post that
> mentioned subtle differences between how Linux 2.4 and Linux 2.6 did
> signals to threads.

You've left out a very important piece of information: which version
of Python you are using.  I'm guessing 2.3.4.  Can you try 2.4?

> So it seems this is a problem with Python threads and Linux kernel 2.6.
> The attached program demonstrates that it has nothing to do with Zope.
> Using it to run "fop-test /usr/bin/fop  fop installed will show the segfault. Running the same thing on a
> machine with 2.4 kernel will instead get the fop "usage" message. It is
> not a generic fop/java problem with 2.6 because the commented
> un-threaded line works fine. It doesn't seem to segfault for any
> command... "cat -" works OK, so it must be something about java
> contributing.
>
> After searching the Python bugs, the closest I could find was
> #971213
> .
>  Is
> this the same bug? Should I submit a new bug report? Is there any
> other way I can help resolve this?

I'd be astonished if this is the same bug.

The main oddness about python threads (before 2.3) is that they run
with all signals masked.  You could play with a C wrapper (call
setprocmask, then exec fop) to see if this is what is causing the
problem.  But please try 2.4.

> BTW, built in file objects really could use better non-blocking
> support... I've got a half-drafted PEP for it... anyone interested in
> it?

Err, this probably should be in a different mail :)

Cheers,
mwh

-- 
  If trees could scream, would we be so cavalier about cutting them
  down? We might, if they screamed all the time, for no good reason.
-- Jack Handey
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Strange segfault in Python threads and linux kernel 2.6

[Michael Hudson]

Donovan Baarda <[EMAIL PROTECTED]> writes:

> On Wed, 2005-01-19 at 13:37 +, Michael Hudson wrote:
>> Donovan Baarda <[EMAIL PROTECTED]> writes:
> [...]
>> You've left out a very important piece of information: which version
>> of Python you are using.  I'm guessing 2.3.4.  Can you try 2.4?
>
> Debian Python2.3 (2.3.4-18), Debian kernel-image-2.6.8-1-686 (2.6.8-10),
> and Debian kernel-image-2.4.27-1-686 (2.4.27-6)
>
>> I'd be astonished if this is the same bug.
>> 
>> The main oddness about python threads (before 2.3) is that they run
>> with all signals masked.  You could play with a C wrapper (call
>> setprocmask, then exec fop) to see if this is what is causing the
>> problem.  But please try 2.4.
>
> Python 2.4 does indeed fix the problem. 

That's good to hear.

> Unfortunately we are using Zope 2.7.4, and I'm a bit wary of
> attempting to migrate it all from 2.3 to 2.4. 

That's not so good to hear, albeit unsurprising.

> Is there any way this "Fix" can be back-ported to 2.3?

Probably not.  It was quite invasive and a bit scary.  OTOH, it hasn't
been the cause of any bug reports yet, so it can't be all bad.

> Note that this problem is being triggered when using 
> Popen3() in a thread. Popen3() simply uses os.fork() and os.execvp().
> The segfault is occurring in the excecvp'ed process. I'm sure there must
> be plenty of cases where this could happen. I think most people manage
> to avoid it because the processes they are popen'ing or exec'ing happen
> to not use signals.

Indeed.

> After testing a bit, it seems the fork() in Popen3 is not a contributing
> factor. The problem occurs whenever os.execvp() is executed in a thread.
> It looks like the exec'ed command inherits the masked signals from the
> thread.

Yeah.  I could have told you that, sorry :)

> I'm not sure what the correct behaviour should be. The fact that it
> works in python2.4 feels more like a byproduct of the thread mask change
> than correct behaviour. 

Well, getting rid of the thread mask changes was one of the goals of
the change.

> To me it seems like execvp() should be setting the signal mask back
> to defaults or at least the mask of the main process before doing
> the exec.

Possibly.  I think the 2.4 change -- not fiddling the process mask at
all -- is the Right Thing, but that doesn't help 2.3 users.  This has
all been discussed before at some length, on python-dev and in various
bug reports on SF.

In your situation, I think the simplest thing you can do is dig out an
old patch of mine that exposes sigprocmask + co to Python and either
make a custom Python incorporating the patch and use that, or put the
code from the patch into an extension module.  Then before execing
fop, use the new code to set the signal mask to something sane.  Not
pretty, particularly, but it should work.

>> > BTW, built in file objects really could use better non-blocking
>> > support... I've got a half-drafted PEP for it... anyone interested in
>> > it?
>> 
>> Err, this probably should be in a different mail :)
>
> The verboseness of the attached test code because of this issue prompted
> that comment... so vaguely related :-)

Oh right :) Didn't actually read the test code, not having fop to
hand...

Cheers,
mwh

-- 
  The ability to quote is a serviceable substitute for wit.
-- W. Somerset Maugham
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Strange segfault in Python threads and linux kernel 2.6

[Michael Hudson]

Donovan Baarda <[EMAIL PROTECTED]> writes:

> On Thu, 2005-01-20 at 14:12 +, Michael Hudson wrote:
>> Donovan Baarda <[EMAIL PROTECTED]> writes:
>> 
>> > On Wed, 2005-01-19 at 13:37 +, Michael Hudson wrote:
>> >> Donovan Baarda <[EMAIL PROTECTED]> writes:
> [...]
>> >> The main oddness about python threads (before 2.3) is that they run
>> >> with all signals masked.  You could play with a C wrapper (call
>> >> setprocmask, then exec fop) to see if this is what is causing the
>> >> problem.  But please try 2.4.
>> >
>> > Python 2.4 does indeed fix the problem. 
>> 
>> That's good to hear.
> [...]
>
> I still don't understand what Linux 2.4 vs Linux 2.6 had to do with
> it.

I have to admit to not being that surprised that behaviour appears
somewhat inexplicable.

As you probably know, linux 2.6 has a more-or-less entirely different
threads implementation (NPTL) than 2.4 (LinuxThreads) -- so changes in
behaviour aren't exactly surprising.  Whether they were intentional, a
good thing, etc, I have a careful lack of opinion :)

> Reading the man pages for execve(), pthread_sigmask() and sigprocmask(),
> I can see some ambiguities, but mostly only if you do things they warn
> against (ie, use sigprocmask() instead of pthread_sigmask() in a
> multi-threaded app).

Uh, I don't know how much I'd trust documentation in this situation.
Really.

Threads and signals are almost inherently incompatible, unfortunately.

> The man page for execve() says that the new process will inherit the
> "Process signal mask (see sigprocmask() )". This implies to me it will
> inherit the mask from the main process, not the thread's signal mask.

Um.  Maybe.  But this is the sort of thing I meant above -- if signals
are delivered to threads, not processes, what does the "Process signal
mask" mean?  The signal mask of the thread that executed main()?  I
guess you could argue that, but I don't know how much I'd bet on it.

> It looks like Linux 2.4 uses the signal mask of the main thread or
> process for the execve(), whereas Linux 2.6 uses the thread's signal
> mask.

I'm not sure that this is the case -- I'm reasonably sure I saw
problems caused by the signal masks before 2.6 was ever released.  But
I could be wrong.

> Given that execve() replaces the whole process, including all
> threads, I dunno if using the thread's mask is right. Could this be
> a Linux 2.6 kernel bug?

You could ask, certainly...

Although I've done a certain amount of battle with these problems, I
don't know what any published standards have to say about these things
which is the only real criteria by which it could be called "a bug".

>> > I'm not sure what the correct behaviour should be. The fact that it
>> > works in python2.4 feels more like a byproduct of the thread mask change
>> > than correct behaviour. 
>> 
>> Well, getting rid of the thread mask changes was one of the goals of
>> the change.
>
> I gathered that... which kinda means the fact that it fixed execvp in
> threads is a side effect...(though I also guess it fixed a lot of other
> things like this too).

Um.  I meant "getting rid of the thread mask" was one of the goals
*because* it would fix the problems with execve and system() and
friends.

>> > To me it seems like execvp() should be setting the signal mask back
>> > to defaults or at least the mask of the main process before doing
>> > the exec.
>> 
>> Possibly.  I think the 2.4 change -- not fiddling the process mask at
>> all -- is the Right Thing, but that doesn't help 2.3 users.  This has
>> all been discussed before at some length, on python-dev and in various
>> bug reports on SF.
>
> Would a simple bug-fix for 2.3 be to have os.execvp() set the mask to
> something sane before executing C execvp()?

Perhaps.  I'm not sure I want to go fiddling there.  Maybe someone
else does.  system(1) presents a problem too, though, which is harder
to worm around unless we want to implement it ourselves, in practice.

> Given that Python does not have any visibility of the procmask...
>
> This might be a good idea regardless as it will protect against this bug
> resurfacing in the future if someone decides fiddling with the mask for
> threads is a good idea again.

In the long run, everyone will use 2.4.  There are some other details
to the changes in 2.4 that have a slight chance of breaking programs
which is why I'm uneasy about putting them in 2.3.5 -- for a bug fix
release it's much much worse to break a program that was working than
to fail to fix one that wasn't.

>> I

Re: [Python-Dev] Re: PEP 309

[Michael Hudson]

Paul Moore <[EMAIL PROTECTED]> writes:

> Also, while looking at patches I noticed 1077106. It doesn't apply to
> me - I don't use Linux - but it looks like this may have simply been
> forgotten. The last comment is in December from from Michael Hudson,
> saying in effect "I'll commit this tomorrow". Michael?

Argh.  Committed.

Cheers,
mwh

-- 
  LINTILLA:  You could take some evening classes.
ARTHUR:  What, here?
  LINTILLA:  Yes, I've got a bottle of them.  Little pink ones.
   -- The Hitch-Hikers Guide to the Galaxy, Episode 12
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Re: Moving towards Python 3.0

[Michael Hudson]

Evan Jones <[EMAIL PROTECTED]> writes:

> On Jan 31, 2005, at 0:17, Guido van Rossum wrote:
>> The "just kidding" applies to the whole list, right? None of these
>> strike me as good ideas, except for improvements to function argument
>> passing.
>
> Really? You see no advantage to moving to garbage collection, nor
> allowing Python to leverage multiple processor environments? I'd be
> curious to hear your reasons why not.

Obviously, if one could wave a wand and make it so, we would.  The
argument about whether the cost (in backwards compatibility,
portability, uniprocessor performace, developer time, etc) outweighs
the benefit.

> My knowledge about garbage collection is weak, but I have read a
> little bit of Hans Boehm's work on garbage collection. For example,
> his "Memory Allocation Myths and Half Truths" presentation
> (http://www.hpl.hp.com/personal/Hans_Boehm/gc/myths.ps) is quite
> interesting. On page 25 he examines reference counting. The biggest
> disadvantage mentioned is that simple pointer assignments end up
> becoming "increment ref count" operations as well, which can "involve
> at least 4 potential memory references." The next page has a
> micro-benchmark that shows reference counting performing very
> poorly.

Given the current implementations *extreme* malloc-happyness I posit
that it would be more-or-less impossible to make any form of
non-copying garabage collector go faster for Python that refcounting.
I may be wrong, but I don't think so and I have actually thought about
this a little bit :)

The "non-copying" bit is important for backwards compatibility of C
extensions (unless there's something I don't know).

> Not to mention that Python has a garbage collector *anyway,* so
> wouldn't it make sense to get rid of the reference counting?

Here you're confused.  Python's cycle collector depends utterly on
reference counting.

(And what is it with this "let's ditch refcounting and use a garbage
collector" thing that people always wheel out?  Refcounting *is* a
form of garbage collection by most reasonable definitions, esp. when
you add Python's cycle collector).

> My only argument for making Python capable of leveraging multiple
> processor environments is that multithreading seems to be where the
> big performance increases will be in the next few years. I am
> currently using Python for some relatively large simulations, so
> performance is important to me.

I'm sure you're tired of hearing it, but I think processes are your
friend...

Cheers,
mwh

-- 
  It is time-consuming to produce high-quality software. However,
  that should not alone be a reason to give up the high standards
  of Python development.  -- Martin von Loewis, python-dev
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Re: [Python-checkins] python/dist/src/Python compile.c, 2.340, 2.341

[Michael Hudson]

[EMAIL PROTECTED] writes:

> Update of /cvsroot/python/python/dist/src/Python
> In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv26507/Python
>
> Modified Files:
>   compile.c 
> Log Message:
> Transform "x in (1,2,3)" to "x in frozenset([1,2,3])".
>
> Inspired by Skip's idea to recognize the throw-away nature of sequences
> in this context and to transform their type to one with better performance.

This breaks code:

>>> [] in (1,)
Traceback (most recent call last):
  File "", line 1, in ?
TypeError: list objects are unhashable

(and so breaks test_email -- is noone else running the test suite?).

It's a cute idea, but IMHO violates the principle of least surprise
too much.

Cheers,
mwh

-- 
  ZAPHOD:  Who are you?
  ROOSTA:  A friend.
  ZAPHOD:  Oh yeah? Anyone's friend in particular, or just generally 
   well-disposed to people?   -- HHGttG, Episode 7
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Patch review: [ 1098732 ] Enhance tracebacks and stack traces with vars

[Michael Hudson]

"Phillip J. Eby" <[EMAIL PROTECTED]> writes:

> At 08:20 PM 2/9/05 +0100, BJörn Lindqvist wrote:
>>Does Skip's idea have
>>any merit?
>
> Yes, but not as a default behavior.  Many people already consider the
> fact that tracebacks display file paths to be a potential security
> problem.  If anything, the default traceback display should have less
> information, not more.  (E.g., display module __name__ instead of the
> code's __file__).

Oh, come on.  Making tracebacks less useful to protect people who
accidentally spray them across the internet seems absurd.  Would you
like them not to show source, either?

Cheers,
mwh

-- 
  Many of the posts you see on Usenet are actually from moths.  You
  can tell which posters they are by their attraction to the flames.
  -- Internet Oracularity #1279-06
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Re: [Python-checkins] python/dist/src/Python compile.c, 2.343, 2.344

[Michael Hudson]

Jim Jewett <[EMAIL PROTECTED]> writes:

> On Wed, 09 Feb 2005 17:42:41 -0800, [EMAIL PROTECTED]
> <[EMAIL PROTECTED]> wrote:
>> Update of /cvsroot/python/python/dist/src/Python
>> In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv31172
>> 
>> Modified Files:
>> compile.c
>> Log Message:
>> Remove the set conversion which didn't work with:  [] in (0,)
>
> Why is this a problem?

It broke the test suite...

> If there were *any* unhashable objects in the container, then the
> compiler would have bailed on the initial set-conversion.

Also, the RHS wouldn't have been a tuple of constants, as far as the
compiler saw it.

> If there aren't any unhashable values, then the (unhashable) item
> being checked is not in the set. ==> Return False.

This would seem to require changing the frozenset implementation.  I
don't know if the option of unhashable implying returning false from
frozenset.__contains__() was considered at the time it was implemented
but it doesn't feel right to me.

> Are you worried about unhashable objects (as item) which 
> compare == to something that is hashable (in container)?
> Custom rich compares can already confuse the "in" tests.

This was a concern of mine, yes.  Although any custom object
(particularly an unhashable one!) that compares equal to something so
simple as an integer, string or tuple seems bad design, I'm not sure
that's the point.

> Or is the problem that guarding against/trapping this case is 
> somehow so expensive that it overrides the expected savings?

If you want to compile the expression

x in (1,2,3)

to contain the moral equivalent of a try:except: block, I'd question
your sanity.

Cheers,
mwh

-- 
  > It might get my attention if you'd spin around in your chair,
  > spoke in tongues, and puked jets of green goblin goo.
  I can arrange for this.  ;-)-- Barry Warsaw & Fred Drake
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] pymalloc on 2.1.3

[Michael Hudson]

"Fredrik Lundh" <[EMAIL PROTECTED]> writes:

> does anyone remember if there were any big changes in pymalloc between
> the 2.1 series (where it was introduced) and 2.3 (where it was enabled by
> default).

Yes.  (Was it really 2.1?  Time flies!)

> or in other words, is the 2.1.3 pymalloc stable enough for production use?

Well, Tim posted ways of making it crash, but I don't know how likely
they are to occur in non-malicious code.

"cvs log Objects/obmalloc.c" might enlighten, or at least give an idea
which months of the python-dev archive to search.

Cheers,
mwh

-- 
   this "I hate c++" is so old
   it's as old as C++, yes
-- from Twisted.Quotes
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Exceptions *must*? be old-style classes?

[Michael Hudson]

Michael Hudson <[EMAIL PROTECTED]> writes:

> Michael Hudson <[EMAIL PROTECTED]> writes:
>
>> I hope to have a new patch (which makes PyExc_Exception new-style, but
>> allows arbitrary old-style classes as exceptions) "soon".  It may even
>> pass bits of "make test" :)
>
> Done: http://www.python.org/sf/1104669

Now I think it's really done, apart from documentation.

My design decision was to make Exception new-style.  Things can be
raised if they are instances of old-style classes or instances of
Exception.  If this meets with general agreement, I'd like to check
the above patch in.  It will break some highly introspective code, so
it's IMO best to get it in early in the 2.5 cycle.

The other option is to keep Exception old-style but allow new-style
subclasses, but I think all this will do is break the above mentioned
introspective code in a quieter way...

The patch also updates the PendingDeprecationWarning on raising a
string exception to a full DeprecationWarning (something that should
be done anyway).

Cheers,
mwh

-- 
  python py.py ~/Source/python/dist/src/Lib/test/pystone.py
  Pystone(1.1) time for 5000 passes = 19129.1
  This machine benchmarks at 0.261381 pystones/second
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Exceptions *must*? be old-style classes?

[Michael Hudson]

Guido van Rossum <[EMAIL PROTECTED]> writes:

>> My design decision was to make Exception new-style.  Things can be
>> raised if they are instances of old-style classes or instances of
>> Exception.  If this meets with general agreement, I'd like to check
>> the above patch in.
>
> I like it, but didn't you forget to mention that strings can still be
> raised? I think we can't break that (but we can insert a deprecation
> warning for this in 2.5 so we can hopefully deprecate it in 2.6, or
> 2.7 at the latest).

I try to forget that as much as possible :)

>> The patch also updates the PendingDeprecationWarning on raising a
>> string exception to a full DeprecationWarning (something that should
>> be done anyway).
>
> What I said. :-)

:)

I'll try to bash the documentation into shape next.

Cheers,
mwh

-- 
  please realize that the Common  Lisp community is more than 40
  years old.  collectively, the community has already been where
  every clueless newbie  will be going for the next three years.
  so relax, please. -- Erik Naggum, comp.lang.lisp
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] subclassing PyCFunction_Type

[Michael Hudson]

Nick Rasmussen <[EMAIL PROTECTED]> writes:

[five days ago]

> Should boost::python functions be modified in some way to show
> up as builtin function types or is the right fix really to patch
> pydoc?

My heart leans towards the latter.

> Is PyCFunction_Type intended to be subclassable?

Doesn't look like it, does it? :) More seriosly, "no".

Cheers,
mwh

-- 
  ARTHUR:  Don't ask me how it works or I'll start to whimper.
   -- The Hitch-Hikers Guide to the Galaxy, Episode 11
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 2.4 func.__name__ breakage

[Michael Hudson]

Tim Peters <[EMAIL PROTECTED]> writes:

> Rev 2.66 of funcobject.c made func.__name__ writable for the first
> time.  That's great, but the patch also introduced what I'm pretty
> sure was an unintended incompatibility:  after 2.66, func.__name__ was
> no longer *readable* in restricted execution mode. 

Yeah, my bad.

> I can't think of a good reason to restrict reading func.__name__,
> and it looks like this part of the change was an accident.  So,
> unless someone objects soon, I intend to restore that func.__name__
> is readable regardless of execution mode (but will continue to be
> unwritable in restricted execution mode).
>
> Objections?

Well, I fixed it on reading the bug report and before getting to
python-dev mail :) Sorry if this duplicated your work, but hey, it was
only a two line change...

Cheers,
mwh

-- 
  The only problem with Microsoft is they just have no taste.
  -- Steve Jobs, (From _Triumph of the Nerds_ PBS special)
and quoted by Aahz on comp.lang.python
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Re: [ python-Bugs-1124637 ] test_subprocess is far tooslow (fwd)

[Michael Hudson]

"Fredrik Lundh" <[EMAIL PROTECTED]> writes:

> Nick Coghlan wrote:
>
>>> One thing that actually can motivate that test_subprocess takes 20% of the
>>> overall time is that this test is a good generic Python stress test - this
>>> test might catch some other startup race condition, for example.
>>
>> test_decimal has a short version which tests basic functionality and always 
>> runs, but 
>> enabling -udecimal also runs the specification tests (which take a fair bit 
>> longer).
>>
>> So keeping the basic subprocess tests unconditional, and running the long 
>> ones only if -uall 
>> or -usubprocess are given would seem reasonable.
>
> does anyone ever use the -u options when running tests?

Yes, occasionally.  Esp. with test_compiler a testall run is an
overnight job but I try to do it every now and again.

Cheers,
mwh

-- 
  If design space weren't so vast, and the good solutions so small a
  portion of it, programming would be a lot easier.
-- maney, comp.lang.python
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 2.4 func.__name__ breakage

[Michael Hudson]

Tim Peters <[EMAIL PROTECTED]> writes:

> [Michael Hudson]
>> ...
>> Well, I fixed it on reading the bug report and before getting to
>> python-dev mail :) Sorry if this duplicated your work, but hey, it was
>> only a two line change...
>
> Na, the real work was tracking it down in the bowels of Zope's C-coded
> security machinery -- we'll let you do that part next time .
>
> Did you add a test to ensure this remains fixed?

Yup.

> A NEWS blurb (at least for 2.4.1 -- the test failures under 2.4 are
> very visible in the Zope world, due to auto-generated test runner
> failure reports)?

No, I'll do that now.  I'm not very good at remembering NEWS blurbs...

Cheers,
mwh

-- 
6. The code definitely is not portable - it will produce incorrect 
   results if run from the surface of Mars.
   -- James Bonfield, http://www.ioccc.org/2000/rince.hint
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 2.4 func.__name__ breakage

[Michael Hudson]

Tim Peters <[EMAIL PROTECTED]> writes:

> [sorry for the near-duplicate msgs -- looks like gmail lied when it claimed 
> the
>  first msg was still in "draft" status]
>
>>> Did you add a test to ensure this remains fixed?
>
> [mwh]
>> Yup.
>
> Bless you.  Did you attach a contributor agreement and mark the test
> as being contributed under said contributor agreement, adjacent to
> your valid copyright notice ?

Fortunately 2 lines < 25 lines, so I think I'm safe on this one :)

Cheers,
mwh

-- 
   glyph: I don't know anything about reality.
-- from Twisted.Quotes
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [ python-Bugs-1124637 ] test_subprocess is far tooslow (fwd)

[Michael Hudson]

"Raymond Hettinger" <[EMAIL PROTECTED]> writes:

>> Let's keep the really long-running tests out
>> of the regular test suite.
>
> For test_subprocess, consider adopting the technique used by
> test_decimal.  When -u decimal is not specified, a small random
> selection of the resource intensive tests are run.  That way, all of the
> tests eventually get run even if no one is routinely using -u all.

I do like this strategy but I don't think it applies to this test --
it has to try to create more than 'ulimit -n' processes, if I
understand it correctly.  Which makes me think there might be other
ways to write the test if the resource module is available...

Cheers,
mwh

-- 
34. The string is a stark data structure and everywhere it is
passed there is much duplication of process.  It is a perfect
vehicle for hiding information.
  -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Requesting that a class be a new-style class

[Michael Hudson]

Nick Coghlan <[EMAIL PROTECTED]> writes:

> This is something I've typed way too many times:
>
> Py> class C():
>File "", line 1
>  class C():
>  ^
> SyntaxError: invalid syntax
>
> It's the asymmetry with functions that gets to me - defining a
> function with no arguments still requires parentheses in the
> definition statement, but defining a class with no bases requires the
> parentheses to be omitted.

Yeah, this has annoyed me for ages too.

However!  You obviously haven't read Misc/HISTORY recently enough :)

The surprising thing is that "class C():" used to work (in fact before
0.9.4 the parens mandatory).  It became a syntax error in 0.9.9,
seemingly because Guido was peeved that people hadn't updated all
their old code to the new syntax.  I wonder if he'd like to try that
trick again today :)

I'd still vote for it to be changed.

> Which leads in to the real question: Does this *really* need to be a
> syntax error? Or could it be used as an easier way to spell "class
> C(object):"?

-1.  Too magical, too opaque.

> Then, in Python 3K, simply drop support for omitting the parentheses
> from class definitions - require inheriting from ClassicClass
> instead.

HISTORY repeats itself...

Cheers,
mwh

-- 
  [Perl] combines all the worst aspects of C and Lisp: a billion
  different sublanguages in one monolithic executable.  It combines
  the power of C with the readability of PostScript. -- Jamie Zawinski
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Requesting that a class be a new-style class

[Michael Hudson]

Alex Martelli <[EMAIL PROTECTED]> writes:

> On 2005 Feb 20, at 04:35, Jack Diederich wrote:
>
>>   I didn't dig into the C but does having 'type'
>> as metaclass guarantee the same behavior as inheriting 'object' or
>> does object
>> provide something type doesn't?  *wince*
>
> I believe the former holds, since for example:

I was going to say that 'type(object) is type' is everything you need
to know, but you also need the bit of code in type_new that replaces
an empty bases tuple with (object,) -- but 

class C:
__metaclass__ = Type

and

class C(object):
pass

produce identical classes.

> This is because types.ClassType turns somersaults to enable this: in
> this latter construct, Python's mechanisms determine ClassType as the
> metaclass (it's the metaclass of the first base class), but then
> ClassType in turn sniffs around for another metaclass to delegate to,
> among the supplied bases, and having found one washes its hands of the
> whole business;-).

It's also notable that type_new does exactly the same thing!

Cheers,
mwh

-- 
   Jokes around here tend to get followed by implementations.
-- from Twisted.Quotes
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Store x Load x --> DupStore

[Michael Hudson]

Guido van Rossum <[EMAIL PROTECTED]> writes:

>> Any objections to new peephole transformation that merges a store/load
>> pair into a single step?
>> 
>> There is a tested patch at:  www.python.org/sf/1144842
>> 
>> It folds the two steps into a new opcode.  In the case of
>> store_name/load_name, it saves one three byte instruction, a trip around
>> the eval-loop, two stack mutations, a incref/decref pair, a dictionary
>> lookup, and an error check (for the lookup).  While it acts like a dup
>> followed by a store, it is implemented more simply as a store that
>> doesn't pop the stack.  The transformation is broadly applicable and
>> occurs thousands of times in the standard library and test suite.

I'm still a little curious as to what code creates such opcodes...

> What exactly are you trying to accomplish? Do you have examples of
> code that would be sped up measurably by this transformation? Does
> anybody care about those speedups even if they *are* measurable?
>
> I'm concerned that there's too much hacking of the VM going on with
> too little benefit. The VM used to be relatively simple code that many
> people could easily understand. The benefit of that was that new
> language features could be implemented relatively easily even by
> relatively inexperienced developers. All that seems to be lost, and I
> fear that the end result is going to be a calcified VM that's only 10%
> faster than the original, since we appear to have reached the land of
> diminishing returns here.

In the case of the bytecode optimizer, I'm not sure this is a fair
accusation.  Even if you don't understand it, you can ignore it and
not have your understanding of the rest of the VM affected (I'm not
sure that compile.c has ever been "easily understood" in any case :).

> I don't see any concentrated efforts trying to figure out where the
> biggest pain is and how to relieve it; rather, it looks as if the
> easiest targets are being approached. Now, if these were low-hanging
> fruit, I'd happily agree, but I'm not so sure that they are all that
> valuable.

I think some of the peepholer's work are pure wins -- x,y = y,x
unpacking and the creation of constant tuples certainly spring to
mind.

If Raymond wants to spend his time on this stuff, that's his choice.
I don't think the obfuscation cost is all that high.

> Where are the attempts to speed up function/method calls? That's an
> area where we could *really* use a breakthrough...

The problem is that it's hard!

> Eventually we'll need a radically different approach, maybe PyPy,
> maybe Starkiller.

Yup.

Cheers,
mwh

-- 
  Gevalia is undrinkable low-octane see-through only slightly
  roasted bilge water. Compared to .us coffee it is quite
  drinkable.  -- Måns Nilsson, asr
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Store x Load x --> DupStore

[Michael Hudson]

"Phillip J. Eby" <[EMAIL PROTECTED]> writes:

> At 08:17 AM 2/20/05 -0800, Guido van Rossum wrote:
>>Where are the attempts to speed up function/method calls? That's an
>>area where we could *really* use a breakthrough...
>
> Amen!
>
> So what happened to Armin's pre-allocated frame patch?  Did that get into 2.4?

No, because it slows down recursive function calls, or functions that
happen to be called at the same time in different threads.  Fixing
*that* would require things like code specific frame free-lists and
that's getting a bit convoluted and might waste quite a lot of memory.

Eliminating the blockstack would be nice (esp. if it's enough to get
frames small enough that they get allocated by PyMalloc) but this
seemed to be tricky too (or at least Armin, Samuele and I spent a
cuple of hours yakking about it on IRC and didn't come up with a clear
approach).  Dynamically allocating the blockstack would be simpler,
and might acheive a similar win.  (This is all from memory, I haven't
thought about specifics in a while).

> Also, does anybody know where all the time goes in a function call,
> anyway?

I did once...

> I assume that some of the pieces are:
>
> * tuple/dict allocation for arguments (but some of this is bypassed on
>   the fast branch for Python-to-Python calls, right?)

All of it, in easy cases.  ISTR that the fast path could be a little
wider -- it bails when the called function has default arguments, but
I think this case could be handled easily enough.

> * frame allocation and setup (but Armin's patch was supposed to
>   eliminate most of this whenever a function isn't being used
>   re-entrantly)

Ah, you remember the wart :) I think even with the patch, frame setup
is a significant amount of work.  Why are frames so big?

> * argument "parsing" (check number of args, map kwargs to their
>   positions, etc.; but isn't some of this already fast-pathed for
>   Python-to-Python calls?)

Yes.  With some effort you could probably avoid a copy (and incref) of
the arguments from the callers to the callees stack area.  BFD.

> I suppose the fast branch fixes don't help special methods like
> __getitem__ et al, since those don't go through the fast branch, but I
> don't think those are the majority of function calls.

Indeed.  I suspect this fails the effort/benefit test, but I could be
wrong.

> And whatever happened to CALL_METHOD?

It didn't work as an optimization, as far as I remember.  I think the
patch is on SF somewhere.  Or is a branch in CVS?  Oh, it's patch
#709744.

> Do we need a tp_callmethod that takes an argument array, length, and
> keywords, so that we can skip instancemethod allocation in the
> common case of calling a method directly?

Hmm, didn't think of that, and I don't think it's how the CALL_ATTR
attempt worked.  I presume it would need to take a method name too :)

I already have a patch that does this for regular function calls (it's
a rearrangement/refactoring not an optimization though).

Cheers,
mwh

-- 
  I think perhaps we should have electoral collages and construct
  our representatives entirely of little bits of cloth and papier 
  mache.  -- Owen Dunn, ucam.chat, from his review of the year
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Store x Load x --> DupStore

[Michael Hudson]

"Phillip J. Eby" <[EMAIL PROTECTED]> writes:

> At 07:00 PM 2/20/05 +0000, Michael Hudson wrote:
>>"Phillip J. Eby" <[EMAIL PROTECTED]> writes:
>>
>> > At 08:17 AM 2/20/05 -0800, Guido van Rossum wrote:
>> >>Where are the attempts to speed up function/method calls? That's an
>> >>area where we could *really* use a breakthrough...
>> >
>> > Amen!
>> >
>> > So what happened to Armin's pre-allocated frame patch?  Did that
>> get into 2.4?
>>
>>No, because it slows down recursive function calls, or functions that
>>happen to be called at the same time in different threads.  Fixing
>>*that* would require things like code specific frame free-lists and
>>that's getting a bit convoluted and might waste quite a lot of memory.
>
> Ah.  I thought it was just going to fall back to the normal case if
> the pre-allocated frame wasn't available (i.e., didn't have a refcount
> of 1).

Well, I don't think that's the test, but that might work.  Someone
should try it :) (I'm trying something else currently).

>>Eliminating the blockstack would be nice (esp. if it's enough to get
>>frames small enough that they get allocated by PyMalloc) but this
>>seemed to be tricky too (or at least Armin, Samuele and I spent a
>>cuple of hours yakking about it on IRC and didn't come up with a clear
>>approach).  Dynamically allocating the blockstack would be simpler,
>>and might acheive a similar win.  (This is all from memory, I haven't
>>thought about specifics in a while).
>
> I'm not very familiar with the operation of the block stack, but why
> does it need to be a stack?  

Finally blocks are the problem, I think.

> For exception handling purposes, wouldn't it suffice to know the
> offset of the current handler, and have an opcode to set the current
> handler location?  And for "for" loops, couldn't an anonymous local
> be used to hold the loop iterator instead of using a stack variable?
> Hm, actually I think I see the answer; in the case of module-level
> code there can be no "anonymous local variables" the way there can in
> functions.  Hmm.

I don't think this is the killer blow.  I can't remember the details
and it's too late to think about them, so I'm going to wait and see if
Samuele replies :)

>>All of it, in easy cases.  ISTR that the fast path could be a little
>>wider -- it bails when the called function has default arguments, but
>>I think this case could be handled easily enough.
>
> When it has *any* default arguments, or only when it doesn't have
> values to supply for them?

When it has *any*, I think.  I also think this is easy to change.

>>Why are frames so big?
>
> Because there are CO_MAXBLOCKS * 12 bytes in there for the block
> stack.  If there was no need for that, frames could perhaps be
> allocated via pymalloc.  They only have around 100 bytes or so in
> them, apart from the blockstack and locals/value stack.

What I'm trying is allocating the blockstack separately and see if two
pymallocs are cheaper than one malloc.

>> > Do we need a tp_callmethod that takes an argument array, length, and
>> > keywords, so that we can skip instancemethod allocation in the
>> > common case of calling a method directly?
>>
>>Hmm, didn't think of that, and I don't think it's how the CALL_ATTR
>>attempt worked.  I presume it would need to take a method name too :)
>
> Er, yeah, I thought that was obvious.  :)

Someone should try this too :)

Cheers,
mwh

-- 
  It is never worth a first class man's time to express a majority
  opinion.  By definition, there are plenty of others to do that.
-- G. H. Hardy
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Store x Load x --> DupStore

[Michael Hudson]

Michael Hudson <[EMAIL PROTECTED]> writes:

>> Because there are CO_MAXBLOCKS * 12 bytes in there for the block
>> stack.  If there was no need for that, frames could perhaps be
>> allocated via pymalloc.  They only have around 100 bytes or so in
>> them, apart from the blockstack and locals/value stack.
>
> What I'm trying is allocating the blockstack separately and see if two
> pymallocs are cheaper than one malloc.

This makes no difference at all, of course -- once timeit or pystone
gets going the code path that actually allocates a new frame as
opposed to popping one off the free list simply never gets executed.
Duh!

Cheers,
mwh
(and despite what the sigmonster implies, I wasn't drunk last night :)

-- 
  This is an off-the-top-of-the-head-and-not-quite-sober suggestion,
  so is probably technically laughable.  I'll see how embarassed I
  feel tomorrow morning.-- Patrick Gosling, ucam.comp.misc
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Decimal & returning NotImplemented (or not)

[Michael Hudson]

Nick Coghlan <[EMAIL PROTECTED]> writes:

> Neil Schemenauer wrote:
>>  IMO, Decimal should
>> be returning NotImplemented instead of raising TypeError.
>
> I agree, but I'm also interested in the fact that nobody commented on
> this before Python 2.4 went out. I read the reference page on numeric
> coercion more times than I care to count, and still didn't register
> that there might be an issue with raising the TypeError - and that's
> only me. Relative to people like Raymond and Facundo, I spent
> comparatively little time working on Decimal :)
>
> I think the problem arose because the natural thing to do in Python
> code is to push the type inspection for operations into a common
> method, and have that method raise TypeError if the other argument
> isn't acceptable.
>
> Trying to convert that exception to a 'NotImplemented' return value is
> a pain (it's less of a pain in C, since you're already checking for
> error returns instead of using exceptions). 

It seems to me that the most straightforward thing would be for
_convert_other to return NotImplemented, or to split it into two
functions, one that returns NotImplemented and one that raises and use
the former in __methods__.  This would mean a certain amount of

   other = _convert_other_ni(other)
   if other is NotImplemented: return other

but this seems mild in comparisons to the contortions the module
already performs in the name of correctness.

Cheers,
mwh

PS: this beahviour seems odd already:

>>> decimal.getcontext().abs(1)
Traceback (most recent call last):
  File "", line 1, in ?
  File "/Users/mwh/Source/python/dist/src/Lib/decimal.py", line 2321, in abs
return a.__abs__(context=self)
TypeError: wrapper __abs__ doesn't take keyword arguments

couldn't/shouldn't the context methods do a _convert_other (of the
erroring out kind) of their arguments?

-- 
  Gullible editorial staff continues to post links to any and all
  articles that vaguely criticize Linux in any way.
 -- Reason #4 for quitting slashdot today, from
http://www.cs.washington.edu/homes/klee/misc/slashdot.html
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Re: [Python-checkins] python/dist/src/Python import.c, 2.240, 2.241

[Michael Hudson]

[EMAIL PROTECTED] writes:

> Update of /cvsroot/python/python/dist/src/Python
> In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv22139/Python
>
> Modified Files:
>   import.c 
> Log Message:
> Patch #1043890: tarfile: add extractall() method.

Uh, did you mean to check import.c in here?

Cheers,
mwh

-- 
  > So what does "abc" / "ab" equal?
  cheese
 -- Steve Holden defends obscure semantics on comp.lang.python
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Outdating import.c 2.241

[Michael Hudson]

"Martin v. Löwis" <[EMAIL PROTECTED]> writes:

> I just mistakenly checked in import.c; I have "outdated"
> this checkin (cvs admin -o 2.241 import.c). If you happened
> to have checked-out import.c in-between, don't be surprised
> if the version numbers go backwards.

Oh!  Oops.  python-checkins sorts before python-dev...

Cheers,
mwh

-- 
  Remember - if all you have is an axe, every problem looks 
  like hours of fun.-- Frossie
   -- http://home.xnet.com/~raven/Sysadmin/ASR.Quotes.html
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Failing tests: marshal, warnings

[Michael Hudson]

Greg Ward <[EMAIL PROTECTED]> writes:

> On 06 March 2005, I said:
>> I'll check this in and merge to the trunk once I see all tests passing.
>
> Checked in on 2.4 branch.  Not merged to trunk since Raymond hasn't
> merged his stuff to the trunk yet.

I don't think that code is going onto HEAD.

Cheers,
mwh

-- 
  Our lecture theatre has just crashed. It will currently only
  silently display an unexplained line-drawing of a large dog
  accompanied by spookily flickering lights.
 -- Dan Sheppard, ucam.chat (from Owen Dunn's summary of the year)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Re: Useful thread project for 2.5?

[Michael Hudson]

"Phillip J. Eby" <[EMAIL PROTECTED]> writes:

> Which reminds me, btw, it would be nice while we're adding more
> execution control functions to have a way to get the current trace
> hook and profiling hook,

Well, there's the f_trace member of frame objects, but I honestly
can't remember what it's for...

> not to mention ways to set them on non-current threads.  You can do
> these things from C of course; I mean accessible as part of the
> Python API.

Again, given sys._current_frames(), this shouldn't be very hard.

Cheers,
mwh

-- 
  And then the character-only displays went away (leading to
  increasingly silly graphical effects and finally to ads on
  web pages).  -- John W. Baxter, comp.lang.python
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Re: @deprecated

[Michael Hudson]

Greg Ward <[EMAIL PROTECTED]> writes:

> On 08 March 2005, Michael Chermside said:
>> Greg writes:
>> > This is one of my top two Java features [1].
>>   ...
>> > [1] The other is compiler recognition of "@deprecated" in doc comments.
>> 
>> Hmm... what a good idea! Let's see... what exactly does "@deprecated" DO in
>> Java? Why it causes the compiler to emit a warning if you use the function in
>> question. Not a bad idea.
>
> Isn't it, though?
>
>> Amusingly enough, the syntax is even the same in
>> Python:
> [...code omitted...]
>
> Cl.  So simple, and yet so powerful.  One might even call it Pythonic!

One difference: I imagine (hope!) the java compiler would complain if
the deprecated function is referenced, whereas the python version only
complains if it is called...

Cheers,
mwh

-- 
  ROOSTA:  Ever since you arrived on this planet last night you've
   been going round telling people that you're Zaphod
   Beeblebrox, but that they're not to tell anyone else.
-- The Hitch-Hikers Guide to the Galaxy, Episode 7
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Re: No new features

[Michael Hudson]

"Donovan Baarda" <[EMAIL PROTECTED]> writes:

>
> Just my 2c;
>
> I don't mind new features in minor releases, provided they meet the
> following two criteria;
>
> 1) Don't break the old API! The new features must be pure extensions that in
> no way change the old API. Any existing code should be un-effected in any
> way by the change.
>
> 2) The new features must be clearly documented as "New in version X.Y.Z".
> This way people using these features will know the minium Python version
> required for their application.

No no no!  The point of what Anthony is saying, as I read it, is that
experience suggests it is exactly this sort of change that should be
avoided.  Consider the case of Mac OS X 10.2 which came with Python
2.2.0: this was pretty broken anyway because of some apple snafus but
it was made even more useless by the fact that people out in the wild
were writing code for 2.2.1 and using True/False/bool.  Going from
2.x.y to 2.x.y+1 shouldn't break anything, going from 2.x.y+1 to 2.x.y
shouldn't break anything that doesn't whack into a bug in 2.x.y -- and
"not having bool" isn't a bug in this sense.

My 2p.

Cheers,
mwh

-- 
  Well, you pretty much need Microsoft stuff to get misbehaviours
  bad enough to actually tear the time-space continuum.  Luckily 
  for you, MS Internet Explorer is available for Solaris.
  -- Calle Dybedahl, alt.sysadmin.recovery
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Re: @deprecated

[Michael Hudson]

"Irmen de Jong" <[EMAIL PROTECTED]> writes:

> mwh wrote:
>
>
>> One difference: I imagine (hope!) the java compiler would complain if
>> the deprecated function is referenced, whereas the python version only
>> complains if it is called...
>
> The java compiler actually already produces deprecation warnings
> during the *compilation phase* (when used with the -deprecation option).
> During runtime, no warnings are produced.

Yeah, that's what I meant to say, even if it wasn't what I said :)

Cheers,
mwh

-- 
  LINTILLA:  You could take some evening classes.
ARTHUR:  What, here?
  LINTILLA:  Yes, I've got a bottle of them.  Little pink ones.
   -- The Hitch-Hikers Guide to the Galaxy, Episode 12
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] rationale for the no-new-features approach

[Michael Hudson]

Anthony Baxter <[EMAIL PROTECTED]> writes:

> My google-fu returned, and I found the piece I was looking for earlier.
> This discusses (from an internal Sun perspective) some of the problems
> with Java. They make quite a big deal about the problem of changing
> code across minor releases. I recall (re)reading this at some point and it
> helped me clarify some ideas floating around in my head.
>
> http://www.internalmemos.com/memos/memodetails.php?memo_id=1321

Thanks for posting that, it was interesting.

Cheers,
mwh

-- 
  NUTRIMAT:  That drink was individually tailored to meet your
 personal requirements for nutrition and pleasure.
ARTHUR:  Ah.  So I'm a masochist on a diet am I?
-- The Hitch-Hikers Guide to the Galaxy, Episode 9
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Re: No new features

[Michael Hudson]

"Donovan Baarda" <[EMAIL PROTECTED]> writes:

> G'day again,

[...]

> You missed the "minor releases" bit in my post.
>
> major releases, ie 2.x -> 3.0, are for things that can break existing code.
> They change the API so that things that run on 2.x may not work with 3.x.
>
> minor releases, ie 2.2.x ->2.3.0, are for things that cannot break existing
> code. They can extend the API such that code for 2.3.x may not work on
> 2.2.x, but code that runs on 2.2.x must work on 2.3.x.
>
> micro releases, ie 2.2.1 ->2.2.2, are for bug fixes only. There can be no
> changes to the API, so that all code that runs on 2.2.2 should work with
> 2.2.1, barring the bugs fixed.
>
> The example you cited of adding bool was an extension to the API, and hence
> should have been a minor release, not a micro release.
>
> I just read the PEP-6, and it doesn't seem to use this terminology, or make
> this distinction... does something else do this anywhere? I thought this
> approach was common knowledge...

I see.  You were proposing a much larger change to the way Python
releases work than I (and perhaps you? :) realised.

Not breaking any code 2.x to 2.x+1 is a nice idea, but doesn't really
seem feasible in practice.

Cheers,
mwh

-- 
  nonono,  while we're making wild  conjectures about the behavior
  of completely irrelevant tasks, we must not also make serious
  mistakes, or the data might suddenly become statistically valid.
-- Erik Naggum, comp.lang.lisp
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] LinkedHashSet/LinkedHashMap equivalents

[Michael Hudson]

"Delaney, Timothy C (Timothy)" <[EMAIL PROTECTED]> writes:

> Set: Items are iterated over in the order that they are added. Adding an
> item that compares equal to one that is already in the set does not
> replace the item already in the set, and does not change the iteration
> order. Removing an item, then re-adding it moves the item to the end of
> the iteration order.

Well, this could be satisfied by an append_new operation on lists,
right (thinking of Common Lisps #'cl:pushnew)?  Complexity not that
great, of course, but I've written code like:

if a not in l:
l.append(a)

and not suffered that badly for it before now...

> Dict: Keys are iterated over in the order that they are added. Setting a
> value using a key that compares equal to one already in the dict
> replaces the value, but not the key, and does not change the iteration
> order. Removing a key (and value) then re-adding it moves the key to the
> end of the iteration order.

And these are what CL types call association lists, in effect.

Cheers,
mwh

-- 
  #ifndef P_tmpdir
  printf( "Go buy a better computer" );
  exit( ETHESKYISFALLINGANDIWANTMYMAMA );
 -- Dimitri Maziuk on writing secure code, asr
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

new-style sxceptions (was Re: [Python-Dev] Open issues for 2.4.1)

[Michael Hudson]

Robey Pointer <[EMAIL PROTECTED]> writes:

> (I've been lurking for a while but this is my first post.  I maintain
> paramiko and enjoy harrassing people on sourceforge to fix the
> new-style Exception bug that vexes me.  Nice to meet you all.) :)

Well, hey, you can review my patch if you like:

http://python.org/sf/1104669

Writing documentation would be even better :)

Cheers,
mwh

-- 
  Also, whenever a programmer thinks, "Hey, skins, what a cool idea",
  their computer's speakers should create some sort of cock-shaped
  soundwave and plunge it repeatedly through their skulls.
  -- http://www.livejournal.com/talkread.bml?journal=jwz&itemid=123070
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] can we stop pretending _PyTyple_Lookup is internal?

[Michael Hudson]

It's needed to implement things that behave like instance methods, for
instance, and I notice that at at least some point in the past Zope3
has used the function with a note attached saying "Guido says this is
OK".

Also, the context is that I want to submit a patch to PyObjC using the
function, and doing a little digging revealed that PyObjC already has
a copy & paste copy of _PyType_Lookup, presumably to avoid using an
internal API.  I don't think this does anyone any good.

Thoughts?

Cheers,
mwh

-- 
  I wouldn't trust the Anglo-Saxons for much anything else.  Given
  they way English is spelled, who could trust them on _anything_ that
  had to do with writing things down, anyway?
-- Erik Naggum, comp.lang.lisp
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] can we stop pretending _PyTyple_Lookup is internal?

[Michael Hudson]

Armin Rigo <[EMAIL PROTECTED]> writes:

> Hi Michael,
>
>> ... _PyType_Lookup ...
>
> There has been discussions about copy_reg.py and at least one other place in
> the standard library that needs this; it is an essential part of the
> descriptor model of new-style classes.  In my opinion it should be made part
> not only of the official C API but the Python one too, e.g. as a method of
> 'type' instances:  type(x).lookup('name')

Yes, this would be good too.  The patch should be trivial; I guess a
little work is required to ensure b/w compat, but not very much
really.

Cheers,
mwh

-- 
  how am I expected to quit smoking if I have to deal with NT
  every day-- Ben Raia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Exception needed: Not enough arguments to PyArg_ParseTuple

[Michael Hudson]

"Edward C. Jones" <[EMAIL PROTECTED]> writes:

> I had
>
> PyArg_ParseTuple(args, "s#s#i:compare", &p1, &bytes1, &p2, &bytes2)
>
> in a program. There are not enough arguments to PyArg_ParseTuple. Does
> PyArg_ParseTuple know how many arguments it is getting?

I don't think so.

> If so, I suggest that an exception should be raised here.

I think you'd need to do battle with ISO C first.

Cheers,
mwh

-- 
  Counting lines is probably a good idea if you want to print it out
  and are short on paper, but I fail to see the purpose otherwise.
-- Erik Naggum, comp.lang.lisp
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] using SCons to build Python

[Michael Hudson]

Adam MacBeth <[EMAIL PROTECTED]> writes:

> Has anyone ever considered using SCons to build Python?

Well, er, there's an obvious problem here somewhere...

> SCons is a great build tool written in Python that provides some
> Autoconf-like functionality (http://www.scons.org). It seems like
> this type of self-hosting would be a great testament to the power of
> Python as well as helping to reinforce the strength of SCons as a
> next generation build tool.

As an active Python developer, I'd like some less fluffy benefits than
that.

Cheers,
mwh

-- 
34. The string is a stark data structure and everywhere it is
passed there is much duplication of process.  It is a perfect
vehicle for hiding information.
  -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Re: [Python-checkins] python/dist/src/Modules readline.c, 2.82, 2.83

[Michael Hudson]

[EMAIL PROTECTED] writes:

> Update of /cvsroot/python/python/dist/src/Modules
> In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv10274
>
> Modified Files:
>   readline.c 
> Log Message:
> Fixes for
>
> [ 110 ] The readline module can cause python to segfault
>
> It seems to me that the code I'm rewriting here attempted to call any
> user-supplied hook functions using the thread state of the thread that
> called the hook-setting function, as opposed to that of the thread
> that is currently executing.  This doesn't work, in general.
>
> Fix this by using the PyGILState API (It wouldn't be that hard to
> define a dummy version of said API when #ifndef WITH_THREAD, would
> it?).
>
> Also, check the conversion to integer of the return value of a hook
> function for errors (this problem was mentioned in the ipython bug
> report linked to in the above bug).

Oh, and I forgot to say: this should go into realease24-maint when
that opens again.

Cheers,
mwh

-- 
  You're going to have to remember that I still think of Twisted as
  a big multiplayer game, and all this HTTP stuff is just kind of a
  grotty way to display room descriptions.  -- Glyph Lefkowitz
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] longobject.c & ob_size

[Michael Hudson]

Asking mostly for curiousity, how hard would it be to have longs store
their sign bit somewhere less aggravating?  It seems to me that the
top bit of ob_digit[0] is always 0, for example, and I'm sure this
would result no less convolution in longobject.c it'd be considerably
more localized convolution.

Cheers,
mwh

-- 
   CDATA is not an integration strategy.
-- from Twisted.Quotes
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] longobject.c & ob_size

[Michael Hudson]

Tim Peters <[EMAIL PROTECTED]> writes:

> [Michael Hudson]
>> Asking mostly for curiousity, how hard would it be to have longs store
>> their sign bit somewhere less aggravating?
>
> Depends on where that is.
>
>> It seems to me that the top bit of ob_digit[0] is always 0, for example,
>
> Yes, the top bit of ob_digit[i], for all relevant i, is 0 on all
> platforms now.
>
>> and I'm sure this would result no less convolution in longobject.c it'd be
>> considerably more localized convolution.
>
> I'd much rather give struct _longobject a distinct sign member (say, 0
> == zero, -1 = non-zero negative, 1 == non-zero positive). 

Well, that would indeed be simpler.

> That would simplify code.  It would cost no extra bytes for some
> longs, and 8 extra bytes for others (since obmalloc rounds up to a
> multiple of 8); I don't care about that (e.g., I never use millions
> of longs simultaneously, but often use a few dozen very big longs
> simultaneously; the memory difference is in the noise then).
>
> Note that longintrepr.h isn't included by Python.h.  Only longobject.h
> is, and longobject.h doesn't reveal the internal structure of longs. 
> IOW, changing the internal layout of longs shouldn't even hurt binary
> compatibility.

Bonus.

> The ob_size member of PyObject_VAR_HEAD would also be redeclared as
> size_t in an ideal world.

As nature intended.

I might do a patch, at some point...

Cheers,
mwh

-- 
  Indeed, when I design my killer language, the identifiers "foo" and
  "bar" will be reserved words, never used, and not even mentioned in
  the reference manual. Any program using one will simply dump core
  without comment. Multitudes will rejoice. -- Tim Peters, 29 Apr 1998
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] longobject.c & ob_size

[Michael Hudson]

Michael Hudson <[EMAIL PROTECTED]> writes:

> Tim Peters <[EMAIL PROTECTED]> writes:
>
>> [Michael Hudson]
>>> Asking mostly for curiousity, how hard would it be to have longs store
>>> their sign bit somewhere less aggravating?
>>
>> Depends on where that is.

[...]

>> I'd much rather give struct _longobject a distinct sign member (say, 0
>> == zero, -1 = non-zero negative, 1 == non-zero positive). 

I ended up doing -1 non-zero negative, 1 zero and positive, but I
don't know if this is really clearer than what you suggest overall.  I
suspect it's a wash.

[...]

> I might do a patch, at some point...

http://python.org/sf/119

Assigned to you, but unassign if you don't have time (testing the
patch is probably more worthwhile than reading it!).

Cheers,
mwh

-- 
  Linux: Horse. Like a wild horse, fun to ride. Also prone to
  throwing you and stamping you into the ground because it doesn't
  like your socks. -- Jim's pedigree of operating systems, asr
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] threading (GilState) question

[Michael Hudson]

I recently redid how the readline module handled threads around
callbacks into Python (the previous code was insane).

This resulted in the following bug report:

http://www.python.org/sf/1176893

Which is correctly assigned to me as it's clearly a result of my
recent checkin.  However, I think my code is correct and the fault
lies elsewhere.

Basically, if you call PyGilState_Release before PyEval_InitThreads
you crash, because PyEval_ReleaseThread gets called while
interpreter_lock is NULL.  This is very simple to make go away -- the
problem is that there are several ways!

Point the first is that I really think this is a bug in the GilState
APIs: the readline API isn't inherently multi-threaded and so it would
be insane to call PyEval_InitThreads() in initreadline, yet it has to
cope with being called in a multithreaded situation.  If you can't use
the GilState APIs in this situation, what are they for?

Option 1) Call PyEval_ThreadsInitialized() in PyGilState_Release().
Non-invasive, but bleh.

Option 2) Call PyEval_SaveThread() instead of
PyEval_ReleaseThread()[1] in PyGilState_Release().  This is my
favourite option (PyGilState_Ensure() calls PyEval_RestoreThread which
is PyEval_SaveThread()s "mate") and I guess you can distill this long
mail into the question "why doesn't PyGilState_Release do this
already?"

Option 3) Make PyEval_ReleaseThread() not crash when interpreter_lock
== NULL.  Easy, but it's actually documented that you can't do this.

Opinions?  Am I placing too much trust into PyGilState_Release()s
existing choice of function?

Cheers,
mwh

[1] The issue of having almost-but-not-quite identical variations of
API functions -- here

 PyEval_AcquireThread/PyEval_ReleaseThread
 vs. PyEval_RestoreThread/PyEval_SaveThread

-- is something I can rant about at length, if anyone is
interested :)

--
  I located the link but haven't bothered to re-read the article,
  preferring to post nonsense to usenet before checking my facts.
  -- Ben Wolfson, comp.lang.python
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com