#34539: `get_prep_value` no longer called for JSONField
-------------------------------------+-------------------------------------
Reporter: Julie Rymer | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 4.2
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Julie Rymer):
My usecase is the following:
I have a db json field that I use to keep data in the following format
`[{id: str, label: str, length: Decimal, height: Decimal, width: Decimal,
weight: Decimal}]`
For easier manipulation and validation, I have a dataclass containing
those properties. I created a custom JSONField to convert to/from a list
of that dataclass to a simple python dict to be inserted in db.
From the [https://docs.djangoproject.com/en/4.2/howto/custom-model-fields
/#converting-values-to-python-objects doc]
> If your custom Field class deals with data structures that are more
complex than strings, dates, integers, or floats, then you may need to
override from_db_value() and to_python().
And so I overrode `from_db_value()` and `to_python()`
Further along the [https://docs.djangoproject.com/en/4.2/howto/custom-
model-fields/#converting-python-objects-to-query-values doc]
> Since using a database requires conversion in both ways, if you override
from_db_value() you also have to override get_prep_value() to convert
Python objects back to query values.
And so I overrode `get_prep_value()`. But `get_prep_value()` is never
called, alas. This behaviour is not consistent from what is expected.
I know that this doc is about `Field.get_prep_value()` and not
`JSONField.get_prep_value()`, but unless I missed something, there is no
specific documentation for `JSONField.get_prep_value()`, nor forewarning
in `Field.get_prep_value()`, so the obvious guess is that this doc apply
to `JSONField.get_prep_value()` as well as any other Field class.
I think the question to solve is
- do we want to add a warning that the documented rules for customisation
does not apply to some Field classes and to document the missing rules, or
- should we harmonize instead all Field classes that don't follow the
rules yet and make their behaviour consistent with current documentation.
I am more in favour of the second option since it does seems to be less of
a change effort and more practical for the end-user.
You can find my custom Field implementation here:
{{{
from __future__ import annotations
import decimal
import json
from dataclasses import dataclass, fields
from decimal import Decimal
from django.core import exceptions
from django.db import models
from django.utils.translation import gettext_lazy as _
@dataclass(repr=True, eq=True, unsafe_hash=True)
class ProductPackage:
id: str
length: Decimal # in cm
height: Decimal # in cm
width: Decimal # in cm
weight: Decimal # in kg
label: str = ""
@staticmethod
def from_dict(package_json: dict) -> ProductPackage:
return ProductPackage(
id=package_json.get("id"),
length=Decimal(package_json["length"]),
height=Decimal(package_json["height"]),
width=Decimal(package_json["width"]),
weight=Decimal(package_json["weight"]),
label=package_json.get("label", ""),
)
def to_dict(self) -> dict:
return dict((field.name, getattr(self, field.name)) for field in
fields(self))
@classmethod
def from_list(cls, packages_json: list[dict]) -> list:
return list(
cls.from_dict(package_json) for package_json in (packages_json
or [])
)
def __copy__(self):
return self.copy()
def copy(self):
return self.__class__(**self.to_dict())
class PackageJSONField(models.JSONField):
"""
ProductPackage class will make package dimension more practical to use
and this class also handle Decimal so they can be
serialized/deserialized from JSON
"""
description = "A JSON field to store a list of ProductPackage"
default_error_messages = {
**models.JSONField.default_error_messages,
"invalid_packages": _("Value must be valid list of packages."),
"invalid_decimal": _("Package dimensions must be valid decimal
strings."),
}
def from_db_value(self, value, expression, connection) ->
list[ProductPackage]:
packages_json = super().from_db_value(value, expression,
connection)
return ProductPackage.from_list(packages_json)
def to_python(self, value):
if value is None:
return value
if isinstance(value, ProductPackage):
return value
try:
if isinstance(value, list):
return ProductPackage.from_list(value)
return json.loads(value, cls=self.decoder)
except json.JSONDecodeError:
raise exceptions.ValidationError(
self.error_messages["invalid"],
code="invalid",
params={"value": value},
)
except TypeError:
raise exceptions.ValidationError(
self.error_messages["invalid_packages"],
code="invalid_packages",
params={"value": value},
)
except decimal.InvalidOperation:
raise exceptions.ValidationError(
self.error_messages["invalid_decimal"],
code="invalid_decimal",
params={"value": value},
)
def get_prep_value(self, value: list[ProductPackage | dict]):
if value is None:
return value
if any(isinstance(package, dict) for package in value):
value = ProductPackage.from_list(value)
value = list(
{
"id": package.id,
"label": package.label,
"length": str(package.length),
"height": str(package.height),
"width": str(package.width),
"weight": str(package.weight),
}
for package in value
)
return super().get_prep_value(value)
def get_db_prep_value(self, value, connection, prepared=False):
# the override I ended up doing for the current issue
if isinstance(value, models.expressions.Value) and isinstance(
value.output_field, models.JSONField
):
value = value.value
elif hasattr(value, "as_sql"):
return value
return connection.ops.adapt_json_value(self.get_prep_value(value),
self.encoder)
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/34539#comment:9>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/01070187e67a30d2-9d95a9e7-eeb8-48c1-b409-8f971f8bdb75-000000%40eu-central-1.amazonses.com.
