#31202: Bulk update suffers from poor performance with large numbers of models
and
columns
-------------------------------------+-------------------------------------
Reporter: Tom Forbes | Owner: Tom
Type: | Forbes
Cleanup/optimization | Status: assigned
Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by jerch):
Recently stumbled again over the perf issue with `bulk_update` in my
computedfields library. And to my surprise - it is even much slower than
applying flat `update` calls to a bunch of objects. Since I got complaints
from users I added a `flat_update` to my django-fast-update package (still
in PR: https://github.com/netzkolchose/django-fast-update/pull/25).
Here are some perf numbers to digest:
{{{
(env) ➜ example git:(drop_bulk_update) ✗ DBENGINE=sqlite ./manage.py perf
10 instances
bulk_update: 0.05896315574645996
fast_update: 0.002216744422912598
flat_update: 0.006779623031616211
100 instances
bulk_update: 0.5576666355133056
fast_update: 0.01461637020111084
flat_update: 0.06025240421295166
1000 instances
bulk_update: 5.557345986366272
fast_update: 0.13169689178466798
flat_update: 0.5667753458023072
10000 instances
bulk_update: 60.25784921646118
fast_update: 1.262794852256775
flat_update: 5.409043550491333
(env) ➜ example git:(drop_bulk_update) ✗ DBENGINE=postgres ./manage.py
perf
10 instances
bulk_update: 0.05660002231597901
fast_update: 0.004314589500427246
copy_update: 0.007386398315429687
flat_update: 0.017342567443847656
100 instances
bulk_update: 0.5351178169250488
fast_update: 0.01944422721862793
copy_update: 0.017321300506591798
flat_update: 0.13300635814666747
1000 instances
bulk_update: 5.777743911743164
fast_update: 0.1764376640319824
copy_update: 0.10273151397705078
flat_update: 1.348728108406067
10000 instances
bulk_update: 100.42072892189026
fast_update: 1.751427412033081
copy_update: 0.6608501672744751
flat_update: 13.84923231601715
}}}
(Perf test case is also in the PR.)
What sticks out is the fact, that `bulk_update` is even so much worse than
plain `update` chaining. Which makes me wonder what was its actual use
case in first place and reason to add it to django core?
--
Ticket URL: <https://code.djangoproject.com/ticket/31202#comment:19>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion visit
https://groups.google.com/d/msgid/django-updates/010701986c634d8f-6510480b-2365-4961-a4fd-2662f5302721-000000%40eu-central-1.amazonses.com.
