#34978: Annotating through an aggregate with RawSQL() raises 1056 "Can't group
on"
on MySQL/MariaDB.
-------------------------------------+-------------------------------------
Reporter: Matthew Somerville | Owner: Simon
| Charette
Type: Bug | Status: assigned
Component: Database layer | Version: 4.2
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Simon Charette):
See #26602 ''Provide a way to manage grouping with RawSQL'' which was
closed because of a justified use case.
I would say that the same can be said about this ticket as the reported
problem can be fixed in multiple ways with the provided ORM interfaces.
The most obvious and non-invasive one is to use `Min(RawSQL(...))` instead
{{{#!python
annotate(
best_date=Min(
RawSQL(
'IFNULL(productions_place.press_date,
IF(productions_place.end_date!="", productions_place.end_date,
productions_place.start_date))',
(),
),
)
)
}}}
a second one is to use expressions entirely
{{{#!python
annotate(best_date=Min(Coalesce("press_date", Case(When(end_date="",
then=F("start_date")), default=F("end_date"))))
}}}
a third one, assuming the user wants to stick to `IF` and `IFNULL`
{{{#!python
from django.db.models import Func
class If(Func):
function = "IF"
class IfNull(Func):
function = "IFNULL"
annotate(best_date=Min(
IfNull("press_date", If(end_date="", "start_date", "end_date"))
))
}}}
-----
> This one's interesting because we can't dig into RawSQL to determine
whether it's valid in a GROUP BY or not.
That's right David, it's the crux of the issue.
The reason why 041551d716b69ee7c81199eee86a2d10a72e15ab broke the reported
use that is that prior to this change the ORM supported a non-standard
feature of MySQL [https://dev.mysql.com/doc/refman/8.0/en/sql-mode.html
disabled in recent versions] that allowed grouping solely by the primary
key of the first entry in `FROM`. It's important to note that using
`RawSQL` to aggregate was only working on MySQL due to this feature and
never worked on any of the other backends that follow the functional
dependency detection in `GROUP BY` clauses as specified by the `SQL:1999`
standard.
The nail in the coffin of this feature was went it was discovered that it
had a peculiarity when dealing with subqueries #31331 that would have
required a significant amount of work to get working.
I could see us go three ways about dealing with this issue
1. Revert the changes made in 041551d716b69ee7c81199eee86a2d10a72e15ab
while making `allows_group_by_pk` based on the absence of
`ONLY_FULL_GROUP_BY`. Note that this won't resolve the aggregation over
the annotation of a dependant subquery but will restore the usage of
`RawSQL` for aggregation on MySQL only when `ONLY_FULL_GROUP_BY` is
disabled.
2. 1 + adjustments to the `allows_group_by_pk` to special case dependant
subquery annotations
3. Adjust the 4.2 existing release notes about this change to let them
know that this version of Django removed support for doing `RawSQL`
aggregations on MySQL and that they should use proper expressions instead
going forward.
Due to lack of demonstration that some aggregates or window function
cannot be expressed using the advanced ORM primitives, that
non-`ONLY_FULL_GROUP_BY` model is a non standard MySQL feature that is not
enabled by default since 8.0, and that this change happen to standardize
the usage of `RawSQL` for aggregations on all backends I'd be inclined to
go with 3.
--
Ticket URL: <https://code.djangoproject.com/ticket/34978#comment:4>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/0107018bed53dd79-516330ee-b1cd-4e47-921f-109770bcb1c8-000000%40eu-central-1.amazonses.com.
