#16022: Cyclic reference in FieldFile causes memory usage to grow considerably
-------------------------------------+-------------------------------------
Reporter: Gustavo Narea | Owner: nobody
Type: | Status: new
Cleanup/optimization |
Component: Database layer | Version: 1.1
(models, ORM) |
Severity: Normal | Resolution:
Keywords: memory leak | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Carlton Gibson):
* type: Bug => Cleanup/optimization
* stage: Unreviewed => Accepted
Comment:
Hi Joshua.
Thanks for the reproduce project — very nice.
I'm going to accept this for review as a potential optimisation, since
memory use is lower in your example using the `weakref` there, and, as you
say, it doesn't break any tests.
I **don't** think it's an actual memory leak per se — rather a question of
the (somewhat high) level at which Python's garbage collector chooses to
kick in.
You have a loop in your test-cases:
{{{
for i in range(100):
# print(process.memory_full_info().rss / 1024 ** 2)
new = Leak.objects.get(id=leak.id)
extra = {"f": new.f}
}}}
If you uncomment the `print()` we see the memory usage rising, in the
first unpatched case:
{{{
...
653.0
663.015625
673.03125
683.046875
693.0625
703.09375
713.109375
723.125
733.140625
743.15625
753.171875
763.1875
773.203125
783.21875
793.234375
...
}}}
If you then though `import gc` and force the garbage collector to run,
you'll see that the memory use remains steady:
{{{
...
152.453125
152.453125
152.453125
152.453125
152.453125
152.453125
152.453125
152.453125
152.453125
152.453125
152.453125
152.453125
152.453125
152.453125
152.453125
152.453125
152.453125
...
}}}
As such it's not a **leak** — it's perfectly collectable — which is inline
with Aymeric's previous observation:
> ... the memory curve climbed a little bit at the beginning, then
stabilized.
The GC does eventually kick-in — but **yes** no reason why we can't see
about bringing that down by removing the reference loop.
Do you fancy making a PR with your suggested change, so we can get some
eyes on it?
Thanks.
--
Ticket URL: <https://code.djangoproject.com/ticket/16022#comment:11>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/0107018021ed0357-98d2fa64-3aa6-4176-8472-eb6f1205a7ff-000000%40eu-central-1.amazonses.com.
