#36701: ModelState objects create reference cycles that require a gc pass to
free
-------------------------------------+-------------------------------------
Reporter: Patryk Zawadzki | Type: Bug
Status: new | Component: Database
| layer (models, ORM)
Version: 5.2 | Severity: Normal
Keywords: memory gc | Triage Stage:
modelstate fields_cache | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Disclaimer: it's impossible for pure Python code to truly leak memory (in
the sense that valgrind would detect), however it's quite easy to create
structures that effectively occupy memory for a long time because they
require the deepest (generation 2) garbage collection cycle to collect and
that happens very rarely. In addition to that, the more such structures
aggregate, the more expensive the garbage collection cycle becomes,
because it effectively stops the entire interpreter to do its job and it
can take seconds. On top of that, it's entirely possible for a container
to run out of memory before the garbage collection happens and we (Saleor
Commerce) see containers being terminated by the kernel OOM killer due to
high memory pressure where most of that memory is locked by garbage.
Each model instance includes a `ModelState` object that in turn contains
references to other model instances. It's possible for those to be cyclic.
In fact, the simplest cyclic case is a `OneToOneField` that links the
states of both objects to the opposite side as soon as the relationship is
traversed.
1. When `foo.bar` is evaluated, the `ForwardManyToOneDescriptor` fetches
the related `Bar` object and sets `foo._state.fields_cache["bar"]` to the
retrieved instance (through a call to `Field.set_cached_value`).
2. If the relation is not `multiple` (so a one-to-one), it also sets
`bar._state.fields_cache["foo"]` to the `foo` object on the `Bar`
instance.
While `OneToOneField` is the easiest way to create such a cycle, it's also
possible to create them through manual assignment (`foo.bar = bar`,
`bar.baz = baz`, `baz.foo = foo`), although one might argue that a
manually created cycle is the fault and therefore concern of the user.
The easiest way to solve this is to break the reference cycle by
implementing a finalizer (`__del__` method) on either the base `Model`
class to remove the state (`del self._state`), or the `ModelState` class
to remove the field cache (`self.fields_cache.clear()`).
--
Ticket URL: <https://code.djangoproject.com/ticket/36701>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion visit
https://groups.google.com/d/msgid/django-updates/0107019a3aa90fc9-09f6de5d-8b0a-4204-9826-5d1fe2b0c2b7-000000%40eu-central-1.amazonses.com.
