#33099: Improve performance of import_string().
-------------------------------------+-------------------------------------
Reporter: Mariusz Felisiak | Owner:
Type: | piaoxue1949
Cleanup/optimization | Status: assigned
Component: Utilities | Version: 3.2
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Keryn Knight):
Here's some back of the napkin cprofile output on my laptop, in case
`piaoxue1949` is unable to provide before/after based on the proposed
changes:
Before, as of `46c8df640cfed5dd525ac1bcf5ad7e57b7ff2571`:
{{{
In [2]: get_backends()
Out[2]:
[<django.contrib.auth.backends.ModelBackend at 0x105307e50>,
<django.contrib.auth.backends.AllowAllUsersModelBackend at 0x1053bf790>,
<django.contrib.auth.backends.RemoteUserBackend at 0x105307940>,
<django.contrib.auth.backends.AllowAllUsersRemoteUserBackend at
0x105307790>]
In [3]: %timeit get_backends()
45.2 µs ± 1.02 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)
In [4]: %prun for _ in range(10000): get_backends()
1940003 function calls in 0.911 seconds
Ordered by: internal time
ncalls tottime percall cumtime percall filename:lineno(function)
80000 0.119 0.000 0.228 0.000 <frozen
importlib._bootstrap>:166(_get_module_lock)
40000 0.118 0.000 0.718 0.000 <frozen
importlib._bootstrap>:1002(_find_and_load)
80000 0.079 0.000 0.100 0.000 <frozen
importlib._bootstrap>:87(acquire)
80000 0.077 0.000 0.097 0.000 <frozen
importlib._bootstrap>:112(release)
80000 0.058 0.000 0.089 0.000 <frozen
importlib._bootstrap>:58(__init__)
80000 0.054 0.000 0.080 0.000 <frozen
importlib._bootstrap>:185(cb)
40000 0.034 0.000 0.243 0.000 <frozen
importlib._bootstrap>:203(_lock_unlock_module)
40000 0.032 0.000 0.856 0.000
module_loading.py:15(import_string)
160000 0.030 0.000 0.030 0.000 {built-in method
_thread.allocate_lock}
40000 0.029 0.000 0.800 0.000
__init__.py:109(import_module)
40000 0.028 0.000 0.194 0.000 <frozen
importlib._bootstrap>:156(__enter__)
40000 0.024 0.000 0.761 0.000 <frozen
importlib._bootstrap>:1018(_gcd_import)
... many more lines, snipped for brevity
}}}
and after, using the `cached_import` to peek directly into `sys.modules`
{{{
In [3]: %timeit get_backends()
3.78 µs ± 24.1 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
In [4]: %prun for _ in range(10000): get_backends()
260003 function calls in 0.115 seconds
Ordered by: internal time
ncalls tottime percall cumtime percall filename:lineno(function)
40000 0.024 0.000 0.061 0.000
module_loading.py:15(import_string)
10000 0.023 0.000 0.105 0.000
__init__.py:24(_get_backends)
40000 0.018 0.000 0.028 0.000
module_loading.py:8(cached_import)
40000 0.017 0.000 0.078 0.000 __init__.py:20(load_backend)
40000 0.010 0.000 0.010 0.000 {built-in method
builtins.getattr}
40000 0.010 0.000 0.010 0.000 {method 'rsplit' of 'str'
objects}
1 0.005 0.005 0.115 0.115 <string>:1(<module>)
40000 0.004 0.000 0.004 0.000 {method 'append' of 'list'
objects}
10000 0.004 0.000 0.109 0.000 __init__.py:37(get_backends)
1 0.000 0.000 0.115 0.115 {built-in method
builtins.exec}
1 0.000 0.000 0.000 0.000 {method 'disable' of
'_lsprof.Profiler' objects}
}}}
that's the whole list. Not stripped for brevity.
--
Ticket URL: <https://code.djangoproject.com/ticket/33099#comment:2>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/065.d1a7bb6943fd18893a5780a6c8195365%40djangoproject.com.
