[ python-Bugs-1338264 ] Memory keeping
Bugs item #1338264, was opened at 2005-10-26 15:37 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1338264&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Interpreter Core Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: sin (sin_avatar) Assigned to: Nobody/Anonymous (nobody) Summary: Memory keeping Initial Comment: I execute this code on python 2.4.2 (authentic copy from console): Python 2.4.2 (#1, Oct 26 2005, 14:45:33) [GCC 2.95.4 20020320 [FreeBSD]] on freebsd4 Type "help", "copyright", "credits" or "license" for more information. >>> a = range(1,1000) >>> del a before i type del - i run top and get (see console output below): 16300 sin 2 0 162M 161M poll 0:02 35.76% 9.28% python2.4 after del (console below): 16300 sin 2 0 162M 161M poll 0:03 7.18% 6.05% python2.4 I tried gc too ... but python didn't free memory. I checked this on windows - memory was freed, but interpreter with 0 defined variables "eat" about 75 Mb!. I think this is bug in interpereter core. some text from dmesg for you: Copyright (c) 1992-2003 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD 4.8-RELEASE #0: Thu Apr 3 10:53:38 GMT 2003 [EMAIL PROTECTED]:/usr/obj/usr/src/sys/ GENERIC Timecounter "i8254" frequency 1193182 Hz CPU: Pentium III/Pentium III Xeon/Celeron (499.15-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0x673 Stepping = 3 Features=0x387f9ff real memory = 268369920 (262080K bytes) avail memory = 255901696 (249904K bytes) -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1338264&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1336623 ] tarfile can't extract some tar archives..
Bugs item #1336623, was opened at 2005-10-24 19:47
Message generated for change (Comment added) made by gustaebel
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1336623&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: A. Murat EREN (meren)
Assigned to: Nobody/Anonymous (nobody)
Summary: tarfile can't extract some tar archives..
Initial Comment:
Here is a small demo to reproduce the same problem:
-8<-8<-8<-8<---
[EMAIL PROTECTED] /home/meren $ wget
ftp://ftp.sleepycat.com/releases/db.1.85.tar.gz
(...)
100%[>] 270,953 17.13K/sETA 00:00
20:21:09 (15.25 KB/s) - `db.1.85.tar.gz' saved [270,953]
[EMAIL PROTECTED] /home/meren $ file db.1.85.tar.gz
db.1.85.tar.gz: gzip compressed data, from Unix
[EMAIL PROTECTED] /home/meren $ python
>>> tar = tarfile.open("db.1.85.tar.gz", "r:gz")
>>> for tarinfo in tar:
... print tarinfo.name
...
db.1.85
db.1.85/btree
db.1.85/btree/Makefile.inc
db.1.85/btree/bt_close.c
db.1.85/btree/bt_conv.c
db.1.85/btree/bt_debug.c
db.1.85/btree/bt_delete.c
db.1.85/btree/btree.h
db.1.85/btree/bt_get.c
db.1.85/btree/bt_open.c
(...)
>>> for tarinfo in tar:
... tar.extract(tarinfo)
...
>>> Ctrl + D
[EMAIL PROTECTED] /home/meren $ ls db*
db.1.85
db.1.85.tar.gz
[EMAIL PROTECTED] /home/meren $ file db.1.85
db.1.85: empty
[EMAIL PROTECTED] /home/meren $ cat db.1.85
[EMAIL PROTECTED] /home/meren $
->8->8->8->8---
Also this file is extracting with the same result too:
ftp://ftp.linux.org.tr/pub/mirrors/gentoo/distfiles/ncompress-4.2.4.tar.gz
This thing is very rarely happening, but it is
happening. Also, I could extract these archives
properly via the native 'tar' binary..
Thanks in advance,
Ciao.
--
Comment By: Lars Gustäbel (gustaebel)
Date: 2005-10-26 12:31
Message:
Logged In: YES
user_id=642936
I submitted patch #1338314 which fixes this problem.
tarfile.py is aware of these "buggy" archives, but the
workaround did not work anymore.
Thank you for your report, especially for the vast number of
test archives.
--
Comment By: A. Murat EREN (meren)
Date: 2005-10-25 20:58
Message:
Logged In: YES
user_id=718263
no files attached.. sorry, here it is..
--
Comment By: A. Murat EREN (meren)
Date: 2005-10-25 20:36
Message:
Logged In: YES
user_id=718263
I figured out that this is a very pesky problem.
The problem is coming from the tar archives themselves.
Simply, the "tarinfo.isdir()" check in the library returns
false for the directories and they are extracting like a
regular file.. How did they create these fool archives I
don't know, and I couldn't reproduce similar buggy archives
by myself. It would be nice to know.
Because of the problem is not in the python library, it is
very difficult to implement an efficent workaround to
properly extract this kind of buggy tar archives. I'm going
to attach a dirty workaround to show the idea (is anyone
reading these reports?)..
Ciao..
--
Comment By: A. Murat EREN (meren)
Date: 2005-10-24 20:19
Message:
Logged In: YES
user_id=718263
more examples:
ftp://ftp.porcupine.org/pub/security/portmap_5beta.tar.gz
ftp://ftp.porcupine.org/pub/security/tcp_wrappers_7.6.tar.gz
additionally, the same problem appearing when trying to extract
these archives with the "ark" (yet another kde tool, just a simple
front-end for the tar command. interesting, isn't it).
Ciao.
--
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1336623&group_id=5470
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1336623 ] tarfile can't extract some tar archives..
Bugs item #1336623, was opened at 2005-10-24 17:47
Message generated for change (Comment added) made by meren
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1336623&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: A. Murat EREN (meren)
Assigned to: Nobody/Anonymous (nobody)
Summary: tarfile can't extract some tar archives..
Initial Comment:
Here is a small demo to reproduce the same problem:
-8<-8<-8<-8<---
[EMAIL PROTECTED] /home/meren $ wget
ftp://ftp.sleepycat.com/releases/db.1.85.tar.gz
(...)
100%[>] 270,953 17.13K/sETA 00:00
20:21:09 (15.25 KB/s) - `db.1.85.tar.gz' saved [270,953]
[EMAIL PROTECTED] /home/meren $ file db.1.85.tar.gz
db.1.85.tar.gz: gzip compressed data, from Unix
[EMAIL PROTECTED] /home/meren $ python
>>> tar = tarfile.open("db.1.85.tar.gz", "r:gz")
>>> for tarinfo in tar:
... print tarinfo.name
...
db.1.85
db.1.85/btree
db.1.85/btree/Makefile.inc
db.1.85/btree/bt_close.c
db.1.85/btree/bt_conv.c
db.1.85/btree/bt_debug.c
db.1.85/btree/bt_delete.c
db.1.85/btree/btree.h
db.1.85/btree/bt_get.c
db.1.85/btree/bt_open.c
(...)
>>> for tarinfo in tar:
... tar.extract(tarinfo)
...
>>> Ctrl + D
[EMAIL PROTECTED] /home/meren $ ls db*
db.1.85
db.1.85.tar.gz
[EMAIL PROTECTED] /home/meren $ file db.1.85
db.1.85: empty
[EMAIL PROTECTED] /home/meren $ cat db.1.85
[EMAIL PROTECTED] /home/meren $
->8->8->8->8---
Also this file is extracting with the same result too:
ftp://ftp.linux.org.tr/pub/mirrors/gentoo/distfiles/ncompress-4.2.4.tar.gz
This thing is very rarely happening, but it is
happening. Also, I could extract these archives
properly via the native 'tar' binary..
Thanks in advance,
Ciao.
--
>Comment By: A. Murat EREN (meren)
Date: 2005-10-26 12:35
Message:
Logged In: YES
user_id=718263
thanks for the patch, it is just working.
--
Comment By: Lars Gustäbel (gustaebel)
Date: 2005-10-26 10:31
Message:
Logged In: YES
user_id=642936
I submitted patch #1338314 which fixes this problem.
tarfile.py is aware of these "buggy" archives, but the
workaround did not work anymore.
Thank you for your report, especially for the vast number of
test archives.
--
Comment By: A. Murat EREN (meren)
Date: 2005-10-25 18:58
Message:
Logged In: YES
user_id=718263
no files attached.. sorry, here it is..
--
Comment By: A. Murat EREN (meren)
Date: 2005-10-25 18:36
Message:
Logged In: YES
user_id=718263
I figured out that this is a very pesky problem.
The problem is coming from the tar archives themselves.
Simply, the "tarinfo.isdir()" check in the library returns
false for the directories and they are extracting like a
regular file.. How did they create these fool archives I
don't know, and I couldn't reproduce similar buggy archives
by myself. It would be nice to know.
Because of the problem is not in the python library, it is
very difficult to implement an efficent workaround to
properly extract this kind of buggy tar archives. I'm going
to attach a dirty workaround to show the idea (is anyone
reading these reports?)..
Ciao..
--
Comment By: A. Murat EREN (meren)
Date: 2005-10-24 18:19
Message:
Logged In: YES
user_id=718263
more examples:
ftp://ftp.porcupine.org/pub/security/portmap_5beta.tar.gz
ftp://ftp.porcupine.org/pub/security/tcp_wrappers_7.6.tar.gz
additionally, the same problem appearing when trying to extract
these archives with the "ark" (yet another kde tool, just a simple
front-end for the tar command. interesting, isn't it).
Ciao.
--
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1336623&group_id=5470
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Feature Requests-1110010 ] 'attrmap' function, attrmap(x)['attname'] == x.attname
Feature Requests item #1110010, was opened at 2005-01-26 11:28
Message generated for change (Comment added) made by gregsmith
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1110010&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Gregory Smith (gregsmith)
Assigned to: Nobody/Anonymous (nobody)
Summary: 'attrmap' function, attrmap(x)['attname'] == x.attname
Initial Comment:
One of the side effects of the new-style classes is that
objects don't necessarily have __dict__ attached to them.
It used to be possible to write things like
def __str__(self):
return "Node %(name)s, %(nlinks)d links,
active: %(active)s" % self.__dict__
... but this doesn't work if the class doesn't have a
__dict__. Even if does, I'm not sure it will always get
members from base classes.
There is a 'vars' function; you could put 'vars(self)'
in the above instead of self.__dict__, but it still
doesn't work if
the class doesn't have a __dict__.
I can see different solutions for this:
(1) change the 'string %' operator so that it allows
%(.name)s, leading to a getattr() on the right-side
argument
rather than a getitem.
return "Node %(.name)s, %(.nlinks)d links,
active: %(.active)s" % self
(2) Make a builtin like vars, but which works when the
object doesn't have a __dict__ I.e. attrmap(x) would
return a mapping which is bound to x, and reading
attrmap(x)['attname'] is the same as
getattr(x,'attname'). Thus
return "Node %(name)s, %(nlinks)d links,
active: %(active)s" % attrmap(self)
This attrmap() function can be implemented in pure
python, of course.
I originally thought (1) made a lot of sense, but (2) seems
to work just as well and doesn't require changing much.
Also, (1) allows cases like "%(name)s %(.name2)s",
which are not very useful, but are very likely to be
created by
accident; whereas in (2) you are deciding on the right
of the '%' whether you are naming attributes or
providing mapping keys.
I'm not sure it's a good idea change 'vars' to have
this behaviour, since vars(x).keys() currently works in
a predictable way when vars(x) works; whereas
attrmap(x).keys() may not be complete, or possible,
even when attrmap(x) is useful. I.e. when x has a
__getattr__ defined.
On the other hand, vars(x) doesn't currently do much at
all, so maybe it's possible to enhance it like this
without breaking anything.
The motivation for this came from the "%(name)s" issue,
but the attrmap() function would be useful in other
places e.g.
processdata( infile, outfile, **attrmap(options))
... where options might be obtained from optparse, e.g.
Or, an attrmap can be used with the new Templates:
string.Template('Node $name').substitute( attrmap(node))
Both of these examples will work with vars(), but only
when the object actually has __dict__. This is why
I'm thinking it may make sense to enhance vars: some
code may be broken by the change; but other code,
broken by new-style classes, may be unbroken by this
change.
The proxy could be writable, so that
attrmap(x)['a'] = y
is the same as
x.a = y
.. which could have more uses.
A possible useful (possibly weird) variation: attrmap
accepts 1 or more parameters, and the resulting
proxy is bound to all of them. when attrmap(x,y,z)['a']
is done, the proxy will try x.a, y.a, z.a until one of
them doesn't raise AttributeError. So it's equivalent
to merging dictionaries. This would be useful
in the %(name)s or Template cases, where you want
information from several objects.
--
>Comment By: Gregory Smith (gregsmith)
Date: 2005-10-26 14:14
Message:
Logged In: YES
user_id=292741
To josiahcarlson: Yes, I can add __getitem__ to any class.
But I don't see why I should make a whole bunch of classes
act like dictionaries, only so that I can pass them to
single API (like string.Template) which requires a
dictionary. A class should have a dictionary behaviour if it
makes sense for that class; if I have to add it
specifically, and only, to make it work well in one
particular situation, that's a kluge. If I have to add dict
behaviour to a whole bunch of classes to make them all work
in that one situation, that's a serious kluge. And entirely
unintuitive, to borrow a phrase. When you encounter this
situation, IMO, it's clear that a little 'pipe-fitting' like
this is the best solution.
Furthermore, and to underline the above, what if I have a
class which already has a __getitem__ interface for it's own
purposes? Chances are this behaves in a way which makes it
useless for harvesting general attributes from the object;
the object always has a getattr interface, however.
Since ALL objects have a getattr interface, and
[ python-Bugs-1338995 ] CVS webbrowser.py (1.40) bugs
Bugs item #1338995, was opened at 2005-10-26 16:08 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1338995&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Greg Couch (gregcouch) Assigned to: Nobody/Anonymous (nobody) Summary: CVS webbrowser.py (1.40) bugs Initial Comment: There are two calls to _safequote that are only made on darwin, aka, Mac OS X. That function is missing. And the UnixBrowser is missing an & in the "simpler command" and thus causes python to hang until the browser is exited (if it wasn't running already). -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1338995&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1339007 ] shelve.Shelf.__del__ throws exceptions
Bugs item #1339007, was opened at 2005-10-26 16:30
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1339007&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Geoffrey T. Dairiki (dairiki)
Assigned to: Nobody/Anonymous (nobody)
Summary: shelve.Shelf.__del__ throws exceptions
Initial Comment:
Python calls an object's __del__ method, even if the
objects __init__ method was unsuccessful (i.e. __init__
terminated via exception.)
Because of this, if there is an error while attempting
to open a Shelf, the destructor is still called, even
though the Shelf is not fully constructed. The code in
shelve.Shelf does not check for this possibility, and
therefor triggers an AttributeError.
Here is an example script which should reproduce the error.
===Begin test.pyl
import shelve
try:
shelf = shelve.open('non-existant-file', flag='r')
except:
pass
===End test.py=
For me, the above script produces the message:
Exception exceptions.AttributeError: "DbfilenameShelf
instance has no attribute 'writeback'" in ignored
I can reproduce this using either Python 2.4.1 or
2.3.5. I took a quick look at the current CVS version
of shelve.py, and it appears to suffer the same problem.
--
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1339007&group_id=5470
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Bugs-1339045 ] Threading misbehavior with lambdas
Bugs item #1339045, was opened at 2005-10-27 02:21 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1339045&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Threads Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Maciek Fijalkowski (fijal) Assigned to: Nobody/Anonymous (nobody) Summary: Threading misbehavior with lambdas Initial Comment: suppose i write: def f(x): print x() for i in range(3): f ( lambda : i ) I got 0,1,2 But when I write for i in range(3): thread . start_new_thread ( f , ( lambda : i ) ) I got 2,2,2 Probably I don't get well design principles, but isn't it against thread consistency? (as long as threads does not interact with each other, interlace doesn't matter). -- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1339045&group_id=5470 ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[ python-Feature Requests-1110010 ] 'attrmap' function, attrmap(x)['attname'] == x.attname
Feature Requests item #1110010, was opened at 2005-01-26 08:28
Message generated for change (Comment added) made by josiahcarlson
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1110010&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Gregory Smith (gregsmith)
Assigned to: Nobody/Anonymous (nobody)
Summary: 'attrmap' function, attrmap(x)['attname'] == x.attname
Initial Comment:
One of the side effects of the new-style classes is that
objects don't necessarily have __dict__ attached to them.
It used to be possible to write things like
def __str__(self):
return "Node %(name)s, %(nlinks)d links,
active: %(active)s" % self.__dict__
... but this doesn't work if the class doesn't have a
__dict__. Even if does, I'm not sure it will always get
members from base classes.
There is a 'vars' function; you could put 'vars(self)'
in the above instead of self.__dict__, but it still
doesn't work if
the class doesn't have a __dict__.
I can see different solutions for this:
(1) change the 'string %' operator so that it allows
%(.name)s, leading to a getattr() on the right-side
argument
rather than a getitem.
return "Node %(.name)s, %(.nlinks)d links,
active: %(.active)s" % self
(2) Make a builtin like vars, but which works when the
object doesn't have a __dict__ I.e. attrmap(x) would
return a mapping which is bound to x, and reading
attrmap(x)['attname'] is the same as
getattr(x,'attname'). Thus
return "Node %(name)s, %(nlinks)d links,
active: %(active)s" % attrmap(self)
This attrmap() function can be implemented in pure
python, of course.
I originally thought (1) made a lot of sense, but (2) seems
to work just as well and doesn't require changing much.
Also, (1) allows cases like "%(name)s %(.name2)s",
which are not very useful, but are very likely to be
created by
accident; whereas in (2) you are deciding on the right
of the '%' whether you are naming attributes or
providing mapping keys.
I'm not sure it's a good idea change 'vars' to have
this behaviour, since vars(x).keys() currently works in
a predictable way when vars(x) works; whereas
attrmap(x).keys() may not be complete, or possible,
even when attrmap(x) is useful. I.e. when x has a
__getattr__ defined.
On the other hand, vars(x) doesn't currently do much at
all, so maybe it's possible to enhance it like this
without breaking anything.
The motivation for this came from the "%(name)s" issue,
but the attrmap() function would be useful in other
places e.g.
processdata( infile, outfile, **attrmap(options))
... where options might be obtained from optparse, e.g.
Or, an attrmap can be used with the new Templates:
string.Template('Node $name').substitute( attrmap(node))
Both of these examples will work with vars(), but only
when the object actually has __dict__. This is why
I'm thinking it may make sense to enhance vars: some
code may be broken by the change; but other code,
broken by new-style classes, may be unbroken by this
change.
The proxy could be writable, so that
attrmap(x)['a'] = y
is the same as
x.a = y
.. which could have more uses.
A possible useful (possibly weird) variation: attrmap
accepts 1 or more parameters, and the resulting
proxy is bound to all of them. when attrmap(x,y,z)['a']
is done, the proxy will try x.a, y.a, z.a until one of
them doesn't raise AttributeError. So it's equivalent
to merging dictionaries. This would be useful
in the %(name)s or Template cases, where you want
information from several objects.
--
Comment By: Josiah Carlson (josiahcarlson)
Date: 2005-10-26 23:16
Message:
Logged In: YES
user_id=341410
gregsmith:
class attrmap:
def __getitem__(self, key):
return getattr(self, key)
Use that as a mixin with any class you want to offer
attribute mapping support to. As in...
class foo(..., attrmap):
...
Now your class has attribute mapping support. Amazing!
As for the neatness of __dict__, I have used it before, but
when I need to rely on using __dict__, I make sure my object
has one (generally by not creating or using __slots__).
--
Comment By: Gregory Smith (gregsmith)
Date: 2005-10-26 11:14
Message:
Logged In: YES
user_id=292741
To josiahcarlson: Yes, I can add __getitem__ to any class.
But I don't see why I should make a whole bunch of classes
act like dictionaries, only so that I can pass them to
single API (like string.Template) which requires a
dictionary. A class should have a dictionary behaviour if it
makes sense for that class; if I have to add it
specifically, and only, to make it work well in one
particular situation, that
