Hello all,
I’d love for some feedback on an idea I’ve been mulling around lately,
namely adding a bulk_save method to Dango.
A somewhat common pattern for some applications is to loop over a list of
models, set an attribute and call save on them. This unfortunately can
issue a lot of database queries which can be a significant slowdown. You
can work around this by using ‘.update()’ in some cases, but not all.
It seems it would be possible to use a CASE statement in SQL to handle
bulk-updating many rows with differing values. For example:
SomeModel.object.filter(id__in=[1,2]).update(
some_field=Case(
When(id=1, then=Value('Field value for ID=1')),
When(id=2, then=Value('Field value for ID=2'))
)
)
I’ve made a ticket for this here:
https://code.djangoproject.com/ticket/29037
I managed to get a 70x performance increase using this technique on a
fairly large table, and it seems it could be applicable to many projects
just like bulk_create.
The downsides to this is that it can produce very large SQL statements when
updating many rows (I had MySQL complain about a 10MB statement once), but
this can be overcome with batching and other optimisations (i.e the same
values can use WHEN id IN (x, y, z) rather than 3 individual WHEN
statements).
I’m imagining an API very similar to bulk_create, but spend any time on a
patch I thought I would ask if anyone have any feedback on this suggestion.
Would this be a good addition to Dango?
--
You received this message because you are subscribed to the Google Groups
"Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/CAFNZOJOmmRDzZv9jMDxnp3-Wp%3Dg5F1dR_Gga3d51kARKGbrrzQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
