Re: [Python-Dev] Deletion order when leaving a scope?

[Thomas Wouters]

On 1/18/07, Larry Hastings <[EMAIL PROTECTED]> wrote:




I just ran a quickie experiment and determined: when leaving a scope,
variables are deleted FIFO, aka in the same order they were created.  This
surprised me; I'd expected them to be deleted LIFO, aka last first.  Why is
it thus?  Is this behavior an important feature or an irrelevant
side-effect?



Please regard it as an irrelevant side-effect. If you want objects to be
cleaned up in a particular order, you should enforce it by having one of
them refer to the other. A great many details can affect the order in which
variables are cleaned up, and that only decreases refcounts of the actual
objects -- a great many other details can then affect the order in which any
objects left with a 0 refcount are actually cleaned up. Even not counting
the more complicated stuff like GC and funky __del__ methods, just having
'import *' or a bare 'exec' in your function can change the order of
DECREFs.

--
Thomas Wouters <[EMAIL PROTECTED]>

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-3000] dict.items as attributes [Was: The bytes type]

[Nick Coghlan]

Jim Jewett wrote:
> On 1/17/07, Thomas Wouters <[EMAIL PROTECTED]> wrote:
>> On 1/16/07, Jim Jewett <[EMAIL PROTECTED]> wrote:
>>> Other than dict.items (and .keys and .values) returning a non-list,
>>> are there any other cases where the Py3K idiom can't already be used
>>> in (or at least backported to) Py 2.x?
> 
>> The aim for 2.6 should be to have all the new features that 3.0 has,
>> as well as full backward compatibility ...
> 
> And I'm asking if there are any real barriers to this.  Some people
> have suggested that they'll have to maintain separate code bases.  So
> far, the closest I've seen[1] to something that can't use
> common-subset is wanting an iterator over a mapping.

To help with the transition code, could we have a 'py3migration' module 
that looked something like:

   if sys.version_info >= (3, 0, 0):
   from _py2compat import *
   else:
   from _py3compat import *

By providing functions like dkeys(), ditems(), dvalues() and maybe a few 
others in that module, this could allow code that needs to work in both 
environments to be written, without causing significant pain to pure 
Python 2 code or to pure Python 3 code.

OK, so it would mean there's still some backward compatibility cruft in 
Py3k, but one module seems a small price to pay for a single obvious way 
to handle semantic changes in the builtin APIs.

Regards,
Nick.

-- 
Nick Coghlan   |   [EMAIL PROTECTED]   |   Brisbane, Australia
---
 http://www.boredomandlaziness.org
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] buglet in long("123\0", 10)

[Nick Coghlan]

Calvin Spealman wrote:
> Added a check in test_long.LongTest.test_misc() that long("123\0", 10)
> fails properly and adapted the patch to int_new to long_new. I get
> this weird feeling that if its impossible for the function
> (PyLong_FromString) to know if its being given bad data, having know
> way to know if the string is supposed to continue past the zero-byte,
> then doesn't it make sense to say that the function by design is
> broken?

It makes sense to say the function is being misused in this case - it's 
designed to convert *C* strings to PyLong objects, so the assumption 
that there are no embedded NULs is a valid one. That said, I think a 
better patch for 2.6 would be to provide a separate PyLong_FromPyString 
function which did the embedded NULL check (and update abstract.c to use 
that instead of its own internal function).

Cheers,
Nick.

-- 
Nick Coghlan   |   [EMAIL PROTECTED]   |   Brisbane, Australia
---
 http://www.boredomandlaziness.org
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Testers wanted: mailbox.py bugfix

[A.M. Kuchling]

On Wed, Jan 17, 2007 at 03:36:58PM -0600, Charles Cazabon wrote:
> I can't speak for MMDF or Babyl, but it's a well-known requirement that all
> programs on a system must use the same type of locking when mbox files are
> used; the administrator picks his favourite (fcntl, dotlocking, whatever) and
> then has to ensure all mbox-aware programs are compiled to use that locking
> style.

That's not actually the bug,  The problem is that code that does:



... wait around a bit ...




can corrupt the mailbox if something else comes in and modifies the
mailbox during the "wait around a bit" phase.  This happens because an
table of contents is generated when the mailbox is read and then never
updated, missing changes made by other processes after the ToC is
read.

--amk
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Deletion order when leaving a scope?

[Neil Toronto]

Thomas Wouters wrote:
> On 1/18/07, *Larry Hastings* <[EMAIL PROTECTED] 
> > wrote:
>
>
>
> I just ran a quickie experiment and determined: when leaving a
> scope, variables are deleted FIFO, aka in the same order they were
> created.  This surprised me; I'd expected them to be deleted LIFO,
> aka last first.  Why is it thus?  Is this behavior an important
> feature or an irrelevant side-effect?
>
>
> Please regard it as an irrelevant side-effect. If you want objects to 
> be cleaned up in a particular order, you should enforce it by having 
> one of them refer to the other. A great many details can affect the 
> order in which variables are cleaned up, and that only decreases 
> refcounts of the actual objects -- a great many other details can then 
> affect the order in which any objects left with a 0 refcount are 
> actually cleaned up. Even not counting the more complicated stuff like 
> GC and funky __del__ methods, just having 'import *' or a bare 'exec' 
> in your function can change the order of DECREFs.

I imagine this would be important to someone expecting system resources 
to be cleaned up, closed, deallocated, or returned inside of __del__ 
methods. Someone coming from C++ might expect LIFO behavior because 
common idioms like RAII (Resource Allocation Is Instantiation) don't 
work otherwise. A Java programmer wouldn't care, being used to cleaning 
up resources manually with a try...catch...finally.

I'm just putting a possible motivation on the concern. It happens that 
the Pythonic Way is also the Java Way in this area: don't expect any 
specific deletion order (don't even expect a guaranteed call to 
__del__), and manually clean up after yourself. As a warning, this has 
been the subject of a great many flame wars between C++ and Java 
programmers...

Neil

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] buglet in long("123\0", 10)

[Calvin Spealman]

On 1/18/07, Nick Coghlan <[EMAIL PROTECTED]> wrote:
> Calvin Spealman wrote:
> > Added a check in test_long.LongTest.test_misc() that long("123\0", 10)
> > fails properly and adapted the patch to int_new to long_new. I get
> > this weird feeling that if its impossible for the function
> > (PyLong_FromString) to know if its being given bad data, having know
> > way to know if the string is supposed to continue past the zero-byte,
> > then doesn't it make sense to say that the function by design is
> > broken?
>
> It makes sense to say the function is being misused in this case - it's
> designed to convert *C* strings to PyLong objects, so the assumption
> that there are no embedded NULs is a valid one. That said, I think a
> better patch for 2.6 would be to provide a separate PyLong_FromPyString
> function which did the embedded NULL check (and update abstract.c to use
> that instead of its own internal function).

Thats more or less the route that would make sense to me! If they
would be accepted I would gladly provide patches for both int and long
types (and float? anything else?).

Is there often a usecase that anyone wants to use PyLong_FromString()
where the c-string is not the contents of PyString? I suppose when
used from extensions creating them.

> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   [EMAIL PROTECTED]   |   Brisbane, Australia
> ---
>  http://www.boredomandlaziness.org
>


-- 
Read my blog! I depend on your acceptance of my opinion! I am interesting!
http://ironfroggy-code.blogspot.com/
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Problem between deallocation of modules and func_globals

[Brett Cannon]

I have discovered an issue relating to func_globals for functions and
the deallocation of the module it is contained within.  Let's say you
store a reference to the function encodings.search_function from the
'encodings' module (this came up in C code, but I don't see why it
couldn't happen in Python code).  Then you delete the one reference to
the module that is stored in sys.modules, leading to its deallocation.
 That triggers the setting of None to every value in
encodings.__dict__.

Oops, now the global namespace for that module has everything valued
at None.  The dict doesn't get deallocated since a reference is held
by encodings.search_function.func_globals and there is still a
reference to that (technically held in the interpreter's
codec_search_path field).  So the function can still execute, but
throws exceptions like AttributeError because a module variable that
once held a dict now has None and thus doesn't have the 'get' method.

My question is whether this is at all worth trying to rectify.  Since
Google didn't turn anything up I am going to guess this is not exactly
a common thing.  =)  That would lead me to believe some (probably
most) of you will say, "just leave it alone and work around it".

The other option I can think of is to store a reference to the module
instead of just to its __dict__ in the function.  The problem with
that is we end up with a circular dependency of the functions in
modules having a reference to the module but then the module having a
reference to the functions.  I tried not having the values in the
module's __dict__ set to None if the reference count was above 1 and
that solved this issue, but that leads to dangling references on
anything in that dict that does not have a reference stored away
somewhere else like encodings.search_function.

Anybody have any ideas on how to deal with this short of rewriting
some codecs stuff so that they don't depend on global state in the
module or just telling me to just live with it?

-Brett
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Problem between deallocation of modules and func_globals

[Josiah Carlson]

"Brett Cannon" <[EMAIL PROTECTED]> wrote:
> I have discovered an issue relating to func_globals for functions and
> the deallocation of the module it is contained within.  Let's say you
> store a reference to the function encodings.search_function from the
> 'encodings' module (this came up in C code, but I don't see why it
> couldn't happen in Python code).  Then you delete the one reference to
> the module that is stored in sys.modules, leading to its deallocation.
>  That triggers the setting of None to every value in
> encodings.__dict__.
[snip]
> Anybody have any ideas on how to deal with this short of rewriting
> some codecs stuff so that they don't depend on global state in the
> module or just telling me to just live with it?

I would have presumed that keeping a reference to a function should have
kept the module "alive".  Why?  If a function keeps a reference to a
module's globals, then even if the module is deleted, the module's
dictionary should still persist, because there exists a reference to it,
through the reference to the function.

Seems to me like a bug, but the bug could be fixed if the module's
dictionary kept a (circular) reference to the module object.  Who else
has been waiting for a __module__ attribute?


 - Josiah

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] adding _Py prefix to names in 2.5.1?

[Martin v. Löwis]

Neal Norwitz schrieb:
> http://python.org/sf/1637022 points out a problem caused by the lack
> of a _Py prefix on Ellipsis.  Most (all?) of the new AST names are not
> prefixed.  These are all meant to be internal names.  Are there any
> issues with changing this?  If we do so, it means that any module
> built with 2.5 that is using these names will fail to work in 2.5.1.
> No code outside the core *should* be using these names.

I'll look into this. I will create macros in the header file for them,
so that existing source code will continue to compile.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Deletion order when leaving a scope?

[Martin v. Löwis]

Larry Hastings schrieb:
> I just ran a quickie experiment and determined: when leaving a scope,
> variables are deleted FIFO, aka in the same order they were created. 

Your experiment was apparently incomplete: Variables are *not* deleted
in the same order in which they are created:

py> class A:
...   def __init__(self, n):self.n = n
...   def __del__(self): print "Deleting", self.n
...
py> def f(x):
...   if x:
... a = A("a")
... b = A("b")
...   else:
... b = A("b")
... a = A("a")
...
py> f(0)
Deleting a
Deleting b

Here, it creates b first, then a (it's the else case), yet
deletes them in reverse order.

As others have pointed out, the deletion order is the one
indicated by the locals array:

py> f.func_code.co_varnames
('x', 'a', 'b')

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Problem between deallocation of modules and func_globals

[Martin v. Löwis]

Brett Cannon schrieb:
> Anybody have any ideas on how to deal with this short of rewriting
> some codecs stuff so that they don't depend on global state in the
> module or just telling me to just live with it?

There is an old patch by Armin Rigo ( python.org/sf/812369 ), which
attempts to implement shutdown based on gc, rather than the explicit
clearing of modules. It would be good if that could be put to work;
I don't know what undesirable side effects doing so would cause.

Short of that, I don't think Python needs to support explicit deletion
of the encodings module from sys.modules when somebody still has a
reference to the search function. Don't do that, then.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Problem between deallocation of modules and func_globals

[Martin v. Löwis]

Josiah Carlson schrieb:
> Seems to me like a bug, but the bug could be fixed if the module's
> dictionary kept a (circular) reference to the module object.  Who else
> has been waiting for a __module__ attribute?

This is the time machine at work:

py> import encodings
py> encodings.search_function.__module__
'encodings'

It's a string, rather than the module object, precisely to avoid cyclic
references.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] adding _Py prefix to names in 2.5.1?

[Raghuram Devarakonda]

On 1/18/07, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote:
> Neal Norwitz schrieb:
> > http://python.org/sf/1637022 points out a problem caused by the lack
> > of a _Py prefix on Ellipsis.  Most (all?) of the new AST names are not
> > prefixed.  These are all meant to be internal names.  Are there any
> > issues with changing this?  If we do so, it means that any module
> > built with 2.5 that is using these names will fail to work in 2.5.1.
> > No code outside the core *should* be using these names.
>
> I'll look into this. I will create macros in the header file for them,
> so that existing source code will continue to compile.
>
> Regards,
> Martin

Hi,

I started to look into this based on Neal's suggestions (in offline
conversation). Please let me know if you want me to continue.

Thanks,
Raghu.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com