#35651: The number of Redis connections has been continuously increasing.
-------------------------------------+-------------------------------------
Reporter: gojuukaze | Owner: (none)
Type: Bug | Status: new
Component: Core (Cache system) | Version: 5.0
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by gojuukaze):
* summary: django redis cache not really using connection pooling => The
number of Redis connections has been continuously increasing.
Old description:
> I'm using `django.core.cache.backends.redis.RedisCache` as a backend for
> redis, and I recently observed that the number of Redis connections has
> been continuously increasing.
>
> After debugging, I found that whenever the cache method is called, the
> `RedisCacheClient `class is reinitialized, and in the `__init__` function
> of `RedisCacheClient`, the connection pool is set to empty.
>
> This causes the `_get_connection_pool` method of `RedisCacheClient` to
> always create a new connection pool instead of using the existing one.
>
> {{{
> class RedisCacheClient:
> def __init__(
> self,
> servers,
> serializer=None,
> pool_class=None,
> parser_class=None,
> **options,
> ):
> import redis
>
> self._lib = redis
> self._servers = servers
> self._pools = {} # << === set pools
>
> def _get_connection_pool(self, write):
> index = self._get_connection_pool_index(write)
>
> #
> # self._pools is is always empty.
> #
>
> if index not in self._pools:
> self._pools[index] = self._pool_class.from_url(
> self._servers[index],
> **self._pool_options,
> )
> return self._pools[index]
> }}}
>
> One solution is to put `_pools` outside of `__init__` , for example：
>
> {{{
> class RedisCacheClient:
>
> # init pool
> _pools = {}
>
> def __init__(
> self,
> servers,
> serializer=None,
> pool_class=None,
> parser_class=None,
> **options,
> ):
> import redis
>
> self._lib = redis
> self._servers = servers
> ## self._pools={}
>
> }}}
>
> By the way, I am using Django 5.0.7 and running it in asynchronous mode.
New description:
I'm using `django.core.cache.backends.redis.RedisCache` as a backend for
redis, and I recently observed that the number of Redis connections has
been continuously increasing.
After debugging, I found that whenever the cache method is called, the
`RedisCacheClient `class is reinitialized, and in the `__init__` function
of `RedisCacheClient`, the connection pool is set to empty.
This causes the `_get_connection_pool` method of `RedisCacheClient` to
always create a new connection pool and a new connection instead of using
the existing one.
{{{
class RedisCacheClient:
def __init__(
self,
servers,
serializer=None,
pool_class=None,
parser_class=None,
**options,
):
import redis
self._lib = redis
self._servers = servers
self._pools = {} # << === set pools
def _get_connection_pool(self, write):
index = self._get_connection_pool_index(write)
#
# self._pools is is always empty.
#
if index not in self._pools:
self._pools[index] = self._pool_class.from_url(
self._servers[index],
**self._pool_options,
)
return self._pools[index]
}}}
One solution is to put `_pools` outside of `__init__` , for example：
{{{
class RedisCacheClient:
# init pool
_pools = {}
def __init__(
self,
servers,
serializer=None,
pool_class=None,
parser_class=None,
**options,
):
import redis
self._lib = redis
self._servers = servers
## self._pools={}
}}}
By the way, I am using Django 5.0.7 and running it in asynchronous mode.
--
--
Ticket URL: <https://code.djangoproject.com/ticket/35651#comment:1>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/010701910d6d8d7b-ba0c9bcd-9be6-40c8-a1ea-010c3685a523-000000%40eu-central-1.amazonses.com.
