#35904: Speed up fixture loading by adding options bulk insert/create
-----------------------------------+--------------------------------------
Reporter: JorisBenschop | Owner: (none)
Type: New feature | Status: new
Component: Testing framework | Version: 5.0
Severity: Normal | Resolution:
Keywords: | Triage Stage: Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-----------------------------------+--------------------------------------
Changes (by JorisBenschop):
* has_patch: 0 => 1
* resolution: wontfix =>
* status: closed => new
Old description:
> I noticed that loading many fixtures is fairly inefficient, in that it
> tries an update and then an insert for each record. I would propose an
> additional option to the command line (—bulk-insert) that only runs the
> insert queries, but also detects if subsequent items in the fixtures are
> from the same model, so they may be inserted using bulk_insert. If there
> is genuine interest in this feature i will develop it and submit. I have
> this running in my own codebase already.
> Please also indicate if there is no interest, and i spend my energy
> elsewhere =)
New description:
As per this forum discussion, I have created a patch to improve load
times for the loaddata command under some circumstances.
Currently the “loaddata” management command uses the obj.save() method for
each deserialized object within a fixture. This function first tries an
UPDATE statement and, if that fails, tries an INSERT statement. By using
the --force_insert a reduction of 50% of queries is achieved.
A second option is to use bulk_create for insertion of multiple records.
This improves insertion speed by (n-1/n), or ~99% for insertion of 100
records.
These options are not meant to cover each use case, and therefore are set
to optional.
--
--
Ticket URL: <https://code.djangoproject.com/ticket/35904#comment:3>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion visit
https://groups.google.com/d/msgid/django-updates/010701939b83a78d-3c0665be-8daa-496e-95f4-bc7c49ef1bde-000000%40eu-central-1.amazonses.com.
