#36371: JSONField.from_db_value crashes when DB returns parsed JSON despite
KeyTransform guard
-------------------------------------+-------------------------------------
Reporter: Mason Pitts | Type: Bug
Status: new | Component: Database
| layer (models, ORM)
Version: 5.2 | Severity: Normal
Keywords: jsonfield, | Triage Stage:
from_db_value, double-decoding, | Unreviewed
psycopg3, cx_oracle, python- |
oracledb |
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
In Django 5.2, the default implementation of JSONField.from_db_value()
only skips double-decoding when the ORM expression is a KeyTransform on a
non-string. However, many modern database drivers (e.g. PostgreSQL
psycopg3, Oracle DB_TYPE_JSON via cx_Oracle 8.1+/python-oracledb) will
automatically deserialize JSON columns into native Python types (dict,
list) before Django sees them. Since from_db_value() still unconditionally
calls json.loads() in most cases, you get:
{{{
TypeError: the JSON object must be str, bytes or bytearray, not list
}}}
even though the value is already a valid Python object.
Here is the current code below as of 5/6/2025.
{{{
def from_db_value(self, value, expression, connection):
if value is None:
return value
# Some backends (SQLite at least) extract non-string values in
their
# SQL datatypes.
if isinstance(expression, KeyTransform) and not isinstance(value,
str):
return value
try:
return json.loads(value, cls=self.decoder)
except json.JSONDecodeError:
return value
}}}
Here is a potential solution that attempts to return value if it is a
Python type.
{{{
def from_db_value(self, value, expression, connection):
if value is None:
return None
# Decode binary data first.
if isinstance(value, (bytes, bytearray)):
value = value.decode()
# If value isn’t a string at this point, the driver already gave
us
# a native Python type (dict, list, bool, int, float, ...).
if not isinstance(value, str):
return value
try:
return json.loads(value, cls=self.decoder)
except json.JSONDecodeError:
return value
}}}
Steps to reproduce:
1. Define a model with a models.JSONField().
2. Use a database and driver combination that natively decodes JSON
columns—for example:
- PostgreSQL with psycopg3
- Oracle 21c+ JSON column type with cx_Oracle 8.1+ or python-
oracledb in thin mode
* I encountered this problem using oracle_db in thin mode.
3. Query the model in the Django admin or via MyModel.objects.all().
4. Observe the traceback raising a TypeError when json.loads() is fed a
list or dict.
Version information:
Django: 5.2
Python: 3.12.10
Affected drivers/backends:
PostgreSQL with psycopg3
Oracle 21c+ with cx_Oracle 8.1+ / python-oracledb in thin mode
--
Ticket URL: <https://code.djangoproject.com/ticket/36371>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion visit
https://groups.google.com/d/msgid/django-updates/01070196a603383b-ea113402-7a25-4144-aa67-93aec22987bc-000000%40eu-central-1.amazonses.com.
