Dear Paul,
> Cannot reproduce the failure on Cython 0.19.1. If you are using an
> earlier version, please try that instead.
We also used Cython 0.19.1. If the unit test runs through, this means that
the memory leak is there. I'm sorry for the ambiguity. Please change
the lines
if not cause_leak:
n_memory_views_expexted -= 1
into just
n_memory_views_expexted -= 1
Then the test will fail if the memory leak exists.
I also just noticed that
unittest.main()
doesn't work in Cython files. The tests are just ignored. Please call the
tests this way instead:
python -m unittest memoryview_leak
For your convenience, here's the updated code:
-------- File memoryview_leak.pyx
---------------------------------------------------------------
"""
Cython module exposing memory leaks due to memoryview attributes in cdef
classes in Cython 0.19.1
Compile::
python setup.py build_ext --inplace
Run::
python -m unittest memoryview_leak
"""
from __future__ import print_function, division
import unittest
import gc
import numpy as np
cdef class LeakTestSuper:
"""
Superclass for the leak test in :class:`LeakTest1` and :class:`LeakTest2`.
"""
pass
cdef class LeakTest1(LeakTestSuper):
"""
Class for memory leak testing.
Holds a memoryview which is extracted from an :class:`numpy.ndarray`
at initialization.
This class avoids the leak by resetting the memoryview to None in
the method :meth:`__dealloc__`.
``__del__()`` wouldn't work, because it is not supported in Cython's
`cdef` classes.
"""
cdef double[:] _s
def __init__(self):
self._s = np.empty(2)
def __dealloc__(self):
self._s = None
cdef class LeakTest2(LeakTestSuper):
"""
Class for memory leak testing.
Holds an array that is allocated at initialization.
This class does not avoid the leak and thus exposes the problem with
memoryviews in Cython.
"""
cdef double[:] _s
def __init__(self):
self._s = np.empty(2)
class TestLeakTest(unittest.TestCase):
def n_objects_by_type(self, typename):
return len([
obj for obj in gc.get_objects()
if type(obj).__name__ == typename])
def test_gc(self):
# Make sure to clear all old memoryviews (circularly referenced).
gc.collect()
# Now there mustn't be any memoryviews left.
n_memory_views_expexted = 0
for cause_leak in [False, True, False, True]:
# Create an object of LeakTest1 or LeakTest2 allocating a memoryview
# internally.
leaktest = LeakTest2() if cause_leak else LeakTest1()
# Check the number of allocated memory views.
n_memory_views_expexted += 1
n_memory_views = self.n_objects_by_type('memoryview')
self.assertEqual(n_memory_views, n_memory_views_expexted)
# Delete the reference to leaktest and let the garbage collector do
# its thing.
del leaktest
gc.collect()
# Check for leaks by counting the memoryviews again.
if True: # not cause_leak:
n_memory_views_expexted -= 1
n_memory_views = self.n_objects_by_type('memoryview')
self.assertEqual(n_memory_views, n_memory_views_expexted)
-----------------------------------------------------------------------------------------------------
Here's the corresponding setup file:
-------- File setup.py
----------------------------------------------------------------------------
from __future__ import print_function, division
from distutils.core import setup
from Cython.Build import cythonize
setup(
name = "memoryview_leak",
ext_modules = cythonize('memoryview_leak.pyx'),
)
---------------------------------------------------------------------------------------------------------
Best regards,
Bruno Daniel
From: cython-devel
[cython-devel-bounces+bruno.daniel=blue-yonder....@python.org] on behalf of
cython-devel-requ...@python.org [cython-devel-requ...@python.org]
Sent: Wednesday, September 25, 2013 12:00 PM
To: cython-devel@python.org
Subject: cython-devel Digest, Vol 32, Issue 10
Send cython-devel mailing list submissions to
cython-devel@python.org
To subscribe or unsubscribe via the World Wide Web, visit
https://mail.python.org/mailman/listinfo/cython-devel
or, via email, send a message with subject or body 'help' to
cython-devel-requ...@python.org
You can reach the person managing the list at
cython-devel-ow...@python.org
When replying, please edit your Subject line so it is more specific
than "Re: Contents of cython-devel digest..."
Today's Topics:
1. Re: Memory leak of memoryview attributes in cdef subclasses
(Pauli Virtanen)
----------------------------------------------------------------------
Message: 1
Date: Tue, 24 Sep 2013 20:43:14 +0300
From: Pauli Virtanen <p...@iki.fi>
To: cython-devel@python.org
Subject: Re: [Cython] Memory leak of memoryview attributes in cdef
subclasses
Message-ID: <l1siva$rg5$1...@ger.gmane.org>
Content-Type: text/plain; charset=ISO-8859-1
Hi,
18.09.2013 16:14, Daniel, Bruno kirjoitti:
> We encountered memory leaks when using memoryview attributes in cdef
> subclasses
> in Cython code. They can be avoided by adding a dealloc method setting the
> value
> of the memoryview attribute to None. The problem does not occur in topmost
> cdef classes.
>
> Here's an example: (See the module docstring on how to compile and run the
> example.)
Cannot reproduce the failure on Cython 0.19.1. If you are using an
earlier version, please try that instead.
One bug with memory leaks in cdef classes was fixed in 0.18 IIRC.
--
Pauli Virtanen
------------------------------
Subject: Digest Footer
_______________________________________________
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel
------------------------------
End of cython-devel Digest, Vol 32, Issue 10
********************************************
_______________________________________________
cython-devel mailing list
cython-devel@python.org
https://mail.python.org/mailman/listinfo/cython-devel
