#35677: Unexpected behaviour of Prefetch with queryset filtering on a through
model
-------------------------------------+-------------------------------------
Reporter: David Glenck | Owner: (none)
Type: Bug | Status: new
Component: Database layer | Version: 5.1
(models, ORM) |
Severity: Normal | Resolution:
Keywords: Prefetch, | Triage Stage: Accepted
prefetch_related, many-to-many |
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Simon Charette):
I think there might be a way to solve the annotate issue as well but it's
not trivial.
When `annotate` is used `sql.Query.used_aliases` doesn't get populated
{{{#!python
>>>
Subscription.objects.annotate(active=F('status__is_active')).query.used_aliases
set()
}}}
This means that even if the subsequent `filter()` wasn't clearing it (it
does since `filter_is_sticky` is `False` by then) by the time it makes its
way to the prefetching logic there is nothing left in `used_alias` that
can be reused.
The only solution I can think of is to add the filter while allowing all
aliases to be reused
{{{#!diff
diff --git a/django/db/models/fields/related_descriptors.py
b/django/db/models/fields/related_descriptors.py
index 5356e28d22..7b003d02cd 100644
--- a/django/db/models/fields/related_descriptors.py
+++ b/django/db/models/fields/related_descriptors.py
@@ -112,7 +112,8 @@ def _filter_prefetch_queryset(queryset, field_name,
instances, db):
if high_mark is not None:
predicate &= LessThanOrEqual(window, high_mark)
queryset.query.clear_limits()
- return queryset.filter(predicate)
+ queryset.query.add_q(predicate, reuse_all_aliases=True)
+ return queryset
class ForwardManyToOneDescriptor:
@@ -1172,7 +1173,6 @@ def get_prefetch_querysets(self, instances,
querysets=None):
# Make sure filters are applied in a "sticky" fashion to
reuse
# multi-valued relationships like direct filter() calls
against
# many-to-many managers do.
- queryset.query.filter_is_sticky = True
queryset = _filter_prefetch_queryset(
queryset, self.query_field_name, instances, db
)
diff --git a/django/db/models/sql/query.py b/django/db/models/sql/query.py
index aef3f48f10..0945aa9198 100644
--- a/django/db/models/sql/query.py
+++ b/django/db/models/sql/query.py
@@ -1602,7 +1602,7 @@ def build_filter(
def add_filter(self, filter_lhs, filter_rhs):
self.add_q(Q((filter_lhs, filter_rhs)))
- def add_q(self, q_object):
+ def add_q(self, q_object, reuse_all_aliases=False):
"""
A preprocessor for the internal _add_q(). Responsible for doing
final
join promotion.
@@ -1616,7 +1616,11 @@ def add_q(self, q_object):
existing_inner = {
a for a in self.alias_map if self.alias_map[a].join_type ==
INNER
}
- clause, _ = self._add_q(q_object, self.used_aliases)
+ if reuse_all_aliases:
+ can_reuse = set(self.alias_map)
+ else:
+ can_reuse = self.used_aliases
+ clause, _ = self._add_q(q_object, can_reuse)
if clause:
self.where.add(clause, AND)
self.demote_joins(existing_inner)
}}}
Ideally we wouldn't reuse all aliases though, only the ones that we know
for sure we want to reuse for `_filter_prefetch_queryset(field_name)`,
which can be obtained through a combination of `names_to_path` and
comparison to `alias_map`.
--
Ticket URL: <https://code.djangoproject.com/ticket/35677#comment:9>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/0107019196a0499b-f8302df7-01d7-4bc9-bb04-8abf8aac7b44-000000%40eu-central-1.amazonses.com.
