#35041: DATA_UPLOAD_MAX_MEMORY_SIZE causes a confusing error when not an integer
-------------------------------------+-------------------------------------
Reporter: dtasev | Owner: nobody
Type: Bug | Status: new
Component: File | Version: 4.2
uploads/storage |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
Description changed by dtasev:
Old description:
> When trying to POST to a FileField or ImageField, an error will be shown
> when trying to save the object (even without specifying a file) if
> DATA_UPLOAD_MAX_MEMORY_SIZE is not an integer.
>
> To replicate:
>
> {{{
> # in settings.py
> DATA_UPLOAD_MAX_MEMORY_SIZE = 4e7
>
> # in models.py
> from django.db import models
> class FileHolder(models.Model):
> file = models.FileField(upload_to="files", blank=True, null=True)
>
> # in admin,py
> from django.contrib import admin
> admin.site.register(FileHolder, EfasNewsAdmin)
> }}}
>
> Make migrations & migrate, then go to the Admin view of the model, create
> a new instance and save. There is no need to specify any file for upload,
> the error will be shown, here is a stacktrace with Django 4.2.8 and
> Python 3.11.6
>
> {{{
> Traceback (most recent call last):
> File "/usr/local/lib/python3.11/site-
> packages/django/core/handlers/exception.py", line 55, in inner
> response = get_response(request)
> ^^^^^^^^^^^^^^^^^^^^^
> File "/usr/local/lib/python3.11/site-
> packages/django/core/handlers/base.py", line 197, in _get_response
> response = wrapped_callback(request, *callback_args,
> **callback_kwargs)
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> File "/usr/local/lib/python3.11/site-
> packages/django/contrib/admin/options.py", line 688, in wrapper
> return self.admin_site.admin_view(view)(*args, **kwargs)
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> File "/usr/local/lib/python3.11/site-
> packages/django/utils/decorators.py", line 130, in _wrapper_view
> result = middleware.process_view(request, view_func, args, kwargs)
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> File "/usr/local/lib/python3.11/site-
> packages/django/middleware/csrf.py", line 470, in process_view
> self._check_token(request)
> File "/usr/local/lib/python3.11/site-
> packages/django/middleware/csrf.py", line 373, in _check_token
> request_csrf_token = request.POST.get("csrfmiddlewaretoken", "")
> ^^^^^^^^^^^^
> File "/usr/local/lib/python3.11/site-
> packages/django/core/handlers/wsgi.py", line 93, in _get_post
> self._load_post_and_files()
> File "/usr/local/lib/python3.11/site-packages/django/http/request.py",
> line 373, in _load_post_and_files
> self._post, self._files = self.parse_file_upload(self.META, data)
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> File "/usr/local/lib/python3.11/site-packages/django/http/request.py",
> line 321, in parse_file_upload
> return parser.parse()
> ^^^^^^^^^^^^^^
> File "/usr/local/lib/python3.11/site-
> packages/django/http/multipartparser.py", line 123, in parse
> return self._parse()
> ^^^^^^^^^^^^^
> File "/usr/local/lib/python3.11/site-
> packages/django/http/multipartparser.py", line 235, in _parse
> data = field_stream.read(size=read_size)
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> File "/usr/local/lib/python3.11/site-
> packages/django/http/multipartparser.py", line 465, in read
> return b"".join(parts())
> ^^^^^^^^^^^^^^^^^
> File "/usr/local/lib/python3.11/site-
> packages/django/http/multipartparser.py", line 460, in parts
> emitting = chunk[:remaining]
> ~~~~~^^^^^^^^^^^^
> TypeError: slice indices must be integers or None or have an __index__
> method
> }}}
>
> Changing DATA_UPLOAD_MAX_MEMORY_SIZE = 4e7 to DATA_UPLOAD_MAX_MEMORY_SIZE
> = 40000000 and repeating the steps will remove the error.
>
> As far as I can tell Django doesn't do type checking of the value of the
> settings, at least I couldn't get other settings to fail due to invalid
> types. If a type check during the "system check" step is not possible
> then an additional type assertion would be good so that a better error
> message can be shown as only integer works.
>
> I have only tested this through Django Admin, but the error happens
> inside django.http so perhaps it can be replicated via other ways of
> POST-ing.
>
> Related links:
> - StackOverflow question where I found the solution first:
> https://stackoverflow.com/questions/48865441/django-admin-typeerror-on-
> any-post-request
New description:
When trying to POST to a FileField or ImageField, an error will be shown
when trying to save the object (even without specifying a file) if
DATA_UPLOAD_MAX_MEMORY_SIZE is not an integer.
To replicate:
{{{
# in settings.py
DATA_UPLOAD_MAX_MEMORY_SIZE = 4e7
# in models.py
from django.db import models
class FileHolder(models.Model):
file = models.FileField(upload_to="files", blank=True, null=True)
# in admin,py
from django.contrib import admin
admin.site.register(FileHolder, admin.ModelAdmin)
}}}
Make migrations & migrate, then go to the Admin view of the model, create
a new instance and save. There is no need to specify any file for upload,
the error will be shown, here is a stacktrace with Django 4.2.8 and Python
3.11.6
{{{
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-
packages/django/core/handlers/exception.py", line 55, in inner
response = get_response(request)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-
packages/django/core/handlers/base.py", line 197, in _get_response
response = wrapped_callback(request, *callback_args,
**callback_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-
packages/django/contrib/admin/options.py", line 688, in wrapper
return self.admin_site.admin_view(view)(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-
packages/django/utils/decorators.py", line 130, in _wrapper_view
result = middleware.process_view(request, view_func, args, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-
packages/django/middleware/csrf.py", line 470, in process_view
self._check_token(request)
File "/usr/local/lib/python3.11/site-
packages/django/middleware/csrf.py", line 373, in _check_token
request_csrf_token = request.POST.get("csrfmiddlewaretoken", "")
^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-
packages/django/core/handlers/wsgi.py", line 93, in _get_post
self._load_post_and_files()
File "/usr/local/lib/python3.11/site-packages/django/http/request.py",
line 373, in _load_post_and_files
self._post, self._files = self.parse_file_upload(self.META, data)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/django/http/request.py",
line 321, in parse_file_upload
return parser.parse()
^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-
packages/django/http/multipartparser.py", line 123, in parse
return self._parse()
^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-
packages/django/http/multipartparser.py", line 235, in _parse
data = field_stream.read(size=read_size)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-
packages/django/http/multipartparser.py", line 465, in read
return b"".join(parts())
^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-
packages/django/http/multipartparser.py", line 460, in parts
emitting = chunk[:remaining]
~~~~~^^^^^^^^^^^^
TypeError: slice indices must be integers or None or have an __index__
method
}}}
Changing DATA_UPLOAD_MAX_MEMORY_SIZE = 4e7 to DATA_UPLOAD_MAX_MEMORY_SIZE
= 40000000 and repeating the steps will remove the error.
As far as I can tell Django doesn't do type checking of the value of the
settings, at least I couldn't get other settings to fail due to invalid
types. If a type check during the "system check" step is not possible then
an additional type assertion would be good so that a better error message
can be shown as only integer works.
I have only tested this through Django Admin, but the error happens inside
django.http so perhaps it can be replicated via other ways of POST-ing.
Related links:
- StackOverflow question where I found the solution first:
https://stackoverflow.com/questions/48865441/django-admin-typeerror-on-
any-post-request
--
--
Ticket URL: <https://code.djangoproject.com/ticket/35041#comment:1>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/0107018c6e187d83-f4bb3849-6b68-4684-a28c-e1815a2a0830-000000%40eu-central-1.amazonses.com.
