Kevin Watters wrote:
> I'm tracking down a memory leak in my app--and I think it's boiling
> down to a virtual method on one of my classes that has an extra
> reference, one not coming from any Python object.
Hi Kevin, hi Phil, hi all,
Okay, Kevin, you MUST be reading my mind because, first thing I did this
morning after yesterday evening's work was fire up my email client and
start a message entitled "Serious memory leak in PyQt?".
I can confirm the issue and, perhaps, provide a little more data.
The core issue is the way Python handles references around bound
methods.
Let obj.meth() be a method on an object. By default, the reference count
to the bound method is zero, and the reference count to the object is,
exactly, the number of external references you have to that object.
I am not sure how it is that the refcount to the bound method is zero by
default. I suspect it may have to do with the way methods are
implemented in Python; i.e. the method is not on the object but on
the /class/, and the method is only bound to the /object/ at call time.
Let us now extract the bound method under an external label:
m = obj.meth
The method object returned by the implicit getattr is /bound/: it comes
with a closure that contains the object the method is bound to, and
that object's refcount thus increases by one.
And here's the issue: what PyQt (well, SIP, really) seems to do with
virtual methods that are reimplemented in Python, is grab a reference
to the bound method, and /never let it go/.
Hence, your bound method's refcount never returns to zero. Hence,
your /object's/ refcount never returns to zero. And you get a memory
leak.
This is an issue because all the event management in Qt is done with
virtual methods.
Example code:
---[ Code ]------------------------------------------------------------
import sys
from PyQt4 import QtGui
app = QtGui.QApplication( sys.argv )
class MyWidget( QtGui.QWidget ):
def resizeEvent( s, e ):
print "In resizeEvent."
return QtGui.QWidget.resizeEvent( s, e )
def __del__( s ):
print "%s deleted." % s
class MyOtherWidget( QtGui.QWidget ):
def nonVirtualMethod( s ):
print "All's good."
def __del__( s ):
print "%s deleted." % s
## Let us now test all the possible cases.
obj1 = MyOtherWidget()
obj1.nonVirtualMethod()
del obj1 ## No virtual method, no problem.
obj2 = MyWidget()
del obj2 ## Virtual method never used, no problem.
obj3 = MyWidget()
obj3.show() ## resizeEvent is bound by SIP.
app.processEvents()
obj3.close()
del obj3 ## ... But never released. Memory leak.
-----------------------------------------------------------------------
Usually this wouldn't be a huge issue, but when you are creating and
destroying lots of widgets, suddenly it's a thorny problem.
Phil, may we please discuss possible ways to fix this? It might be as
simple as using weak references to the bound methods instead of direct
references.
Arguably, 'simple' is perhaps something of a misnommer, because Python's
weakrefs don't handle bound methods well at all (the weak reference
dies the moment the bound method goes out of scope).
Fortunately, that can be worked around reasonably simply:
---[ Code ]------------------------------------------------------------
import weakref
class WeakCallableRef:
def __init__( s, fn, callback=None ):
assert callable( fn )
s._methname = None
s._objref = None
s._fnref = None
s._dead = False
s._callback = callback
obj = getattr( fn, "im_self", None )
if obj: ## fn is a bound method
s._objref = weakref.ref( obj, s.markDead )
s._methname = fn.im_func.func_name
else: ## fn is a static method or a plain function
s._fnref = weakref.ref( fn, s.markDead )
def markDead( s, ref ):
s._dead = True
if s._callback: s._callback( s )
def __call__( s ):
if s._dead: return None
if s._objref: ## bound method
return getattr( s._objref(), s._methname, None )
else:
return s._fnref()
-----------------------------------------------------------------------
This class behaves like weakref.ref, only for callables.
Might it be possible to have it used by PyQt for virtual methods?
Thanks,
-- S.
_______________________________________________
PyQt mailing list PyQt@riverbankcomputing.com
http://www.riverbankcomputing.com/mailman/listinfo/pyqt
