#33460: Change SQLite backend to generate INSERT statements using VALUES
instead of
UNION.
-------------------------------------+-------------------------------------
Reporter: Keryn | Owner: Keryn Knight
Knight |
Type: | Status: assigned
Cleanup/optimization |
Component: Database | Version: dev
layer (models, ORM) | Keywords: sqlite orm
Severity: Normal | bulk_create batch
Triage Stage: | Has patch: 1
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
This is a ticket for tracking this
[https://github.com/django/django/pull/15354 pre-emptive PR].
Django's minimum supported version of SQLite is currently documented as
3.9.
Using `INSERT INTO ... VALUES (...)` instead of `INSERT INTO ... SELECT
UNION ALL ...` for multiple rows (i.e. `bulk_create`) was added to SQLite
in [http://www.sqlite.org/releaselog/3_7_11.html 3.7.11] released in 2012.
[https://www.sqlite.org/releaselog/3_8_8.html SQLite 3.8.8], released in
2015, further changed it so that `VALUES(...)` was not subject to the
`SQLITE_LIMIT_COMPOUND_SELECT` value, which should subsequently allow for
improvements in batch sizes (e.g. investigating changes to
`django.db.backends.sqlite3.operations.DatabaseOperations.bulk_batch_size`
and
`django.db.backends.sqlite3.features.DatabaseFeatures.max_query_params`,
the latter of which supports `32766 ` values instead of `999` as of
[https://www.sqlite.org/releaselog/3_32_0.html 3.32.0])
The documentation for current SQLite (3.37.2) says, of using
`VALUES(...)`:
> The phrase "VALUES(expr-list)" means the same thing as "SELECT expr-
list". The phrase "VALUES(expr-list-1),...,(expr-list-N)" means the same
thing as "SELECT expr-list-1 UNION ALL ... UNION ALL SELECT expr-list-N".
Both forms are the same, except that the number of SELECT statements in a
compound is limited by SQLITE_LIMIT_COMPOUND_SELECT whereas the number of
rows in a VALUES clause has no arbitrary limit.
> There are some restrictions on the use of a VALUES clause that are not
shown on the syntax diagrams:
> A VALUES clause cannot be followed by ORDER BY.
> A VALUES clause cannot be followed by LIMIT.
I have run the test suite against `3.37.2` and whatever the CI versions
are (I've not checked) ... so far so good.
--
Ticket URL: <https://code.djangoproject.com/ticket/33460>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/052.267f6d917e18c188dd60a63b4c5e7f48%40djangoproject.com.
