#33636: BulkProcessMixin on models.Model
-------------------------------------+-------------------------------------
Reporter: Myung Eui Yoon | Owner: Myung Eui
| Yoon
Type: New feature | Status: assigned
Component: Database layer | Version: 4.0
(models, ORM) |
Severity: Normal | Resolution:
Keywords: bulk, model | Triage Stage:
| Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Myung Eui Yoon):
* has_patch: 0 => 1
Old description:
> {{{
> """
> class BulkyModel(models.Model, ModelBulkProcessMixin):
> id = models.BigAutoField(primary_key=True)
> name = models.CharField(max_length=10, null=False)
>
> With ModelBulkProcessMixin , We could minimize memory usage.
> Without ModelBulkProcessMixin, We shold maintain bulk array size
> up to 100_000
> or manually maintain arraysize up to batch_size like 10_000
>
> if len(chunked_list)>10_000:
> Model.objects.bulk_create(chunked_list)
>
> and check remain in list again at the end.
>
> if len(chunked_list)>0:
> Model.objects.bulk_create(chunked_list)
> """
>
> names = [f"name-{num}" for num in range(100_000)]
>
> with BulkyModel.gen_bulk_create(batch_size=10_000) as bulk:
> for name in names:
> bulk.add(BulkyModel(name=name))
>
> self.assertEqual(100_000, BulkyModel.objects.all().count())
> }}}
New description:
{{{
"""
class BulkyModel(models.Model, ModelBulkProcessMixin):
id = models.BigAutoField(primary_key=True)
name = models.CharField(max_length=10, null=False)
With ModelBulkProcessMixin , We could minimize memory usage.
Without ModelBulkProcessMixin, We shold maintain bulk array size
up to 100_000
or manually maintain arraysize up to batch_size like 10_000
if len(chunked_list)>10_000:
Model.objects.bulk_create(chunked_list)
and check remain in list again at the end.
if len(chunked_list)>0:
Model.objects.bulk_create(chunked_list)
"""
names = [f"name-{num}" for num in range(100_000)]
with BulkyModel.gen_bulk_create(batch_size=10_000) as bulk:
for name in names:
bulk.add(BulkyModel(name=name))
self.assertEqual(100_000, BulkyModel.objects.all().count())
}}}
https://github.com/django/django/pull/15577
--
Comment:
Issued Pull Request
https://github.com/django/django/pull/15577
--
Ticket URL: <https://code.djangoproject.com/ticket/33636#comment:2>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/0107018018f2f058-4e8357b6-6b78-4cab-825f-f45221a3e5dd-000000%40eu-central-1.amazonses.com.
