#35975: New feature: faster fixture loading via loaddata command
-------------------------------+-----------------------------------------
Reporter: JorisBenschop | Type: Uncategorized
Status: new | Component: Uncategorized
Version: 5.1 | Severity: Normal
Keywords: loaddata | Triage Stage: Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+-----------------------------------------
As per [https://forum.djangoproject.com/t/feature-proposal-faster-fixture-
loading-via-loaddata-command/36972/4 this forum discussion], I have
created a patch to improve load times for the loaddata command under some
circumstances.
Currently the “loaddata” management command uses the obj.save() method for
each deserialized object within a fixture. This function first tries an
UPDATE statement and, if that fails, tries an INSERT statement.
I propose to add two optional flags to the loaddata command:
–force-insert adds the “force_insert=True” to the save() method. This
reduces load time by ~50% but has an increased risk of the upload failing
in case the record already exists.
–bulk_create: This option groups records in the fixture by the model,
and insert the group as a single bulk_create statement. For large
fixtures, I have achieved a 1000-fold improvement in loading time. It has
a number of risks (already described in the bulk_create section), notably
that it skips some routines covered in the save() method of the model.
Both these flags are supposed to be run by people who know that above-
mentioned issues will not be a problem for them, and should not be enabled
by default.
--
Ticket URL: <https://code.djangoproject.com/ticket/35975>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion visit
https://groups.google.com/d/msgid/django-updates/01070193975152a8-e158f025-d3a8-4da9-8c29-32a4d3d7a0e5-000000%40eu-central-1.amazonses.com.
