#36439: Auth hashing blocks event loop if using asyncio
-------------------------------------+-------------------------------------
Reporter: Robert Aistleitner | Owner: (none)
Type: | Status: new
Cleanup/optimization |
Component: contrib.auth | Version: 5.2
Severity: Normal | Resolution:
Keywords: async, auth, | Triage Stage: Accepted
asyncio, performance |
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Roelzkie):
Replying to [comment:6 Simon Charette]:
> There appears to be a few things wrong with your current benchmark.
>
> First you don't provide which hasher you used and and your setup steps
most importantly whether or not you first set a password for the users you
are testing against. If you didn't (e.g. you only set a username with an
invalid password) then it's highly likely all underlying `verify_password`
calls never get to the point of hashing `user.username` and then
performing a constant time compare which are the expensive operations CPU
wise that would benefit from executing in a thread pool.
>
> Secondly the way you're iterating over each awaitable serially in a loop
prevents any concurrent scheduling execution from taking place thus you
most likely wouldn't notice if the event loop was blocked as you're only
processing one task at a time. You'd want to buffer up futures and await
them all so they can step on each others toes a bit if you hope to catch
any interference between them.
Hello, my bad, you are right. I'm executing the code serially with
awaitables. I'm putting them all into a task group, and the gap in
performance is huge with a `ThreadPoolExecutor`.
This is for 100 users only, and I'm using the default
`PBKDF2PasswordHasher` hasher. I will also check other hashers later on:
1. **Current:** 14138 function calls (14120 primitive calls) in 24.844
seconds
2. **With sync_to_async:** 46823 function calls (45895 primitive calls) in
24.615 seconds
3. **With ThreadPoolExecutor(8):** 45522 function calls (43874 primitive
calls) in 5.565 seconds
{{{#!python
import os
from datetime import datetime
import django
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "mysite.settings")
django.setup()
# --
import asyncio
import cProfile
from django.contrib.auth.models import User
async def main():
profiler = cProfile.Profile()
profiler.enable()
async with asyncio.TaskGroup() as tg:
async for user in User.objects.all()[:100]:
tg.create_task(user.acheck_password(user.username))
profiler.disable()
profiler.print_stats(sort='cumulative')
if __name__ == "__main__":
asyncio.run(main())
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/36439#comment:7>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion visit
https://groups.google.com/d/msgid/django-updates/01070197bfa5f413-31b3bd6f-12d8-4959-981f-996dc3448653-000000%40eu-central-1.amazonses.com.
