> Hello. > > I've been investigating some memory usage issues with the Gevent plugin > and have noticed an issue. Whenever I enable the Gevent plugin and perform > some requests to my app, which in turn uses Pycassa (thrift) to interact > with a Cassandra server, the memory usage starts increasing with each > request. The strange thing is, if I disable the Gevent plugin but still > monkey patch everything in the python application startup, the problem > goes away. > > These are some numbers I obtained with objgraph regarding the most common > object types in memory after some requests were made to the app: > > GEVENT plugin: > > builtin_function_or_method 50266 > io 49066 > timer 49051 > tuple 14923 > function 11888 > dict 3536 > cell 2806 > list 2067 > weakref 1891 > wrapper_descriptor 1363 > getset_descriptor 1338 > type 1303 > method_descriptor 926 > member_descriptor 524 > module 504 > property 455 > classobj 283 > set 206 > instancemethod 192 > ColumnDef 155 > > > Single process mode but still Gevent monkey patched: > > function 11868 > tuple 4866 > dict 3672 > cell 2809 > list 2130 > weakref 1836 > wrapper_descriptor 1363 > getset_descriptor 1334 > type 1266 > builtin_function_or_method 1208 > method_descriptor 926 > member_descriptor 524 > module 499 > property 455 > classobj 279 > set 206 > instancemethod 189 > ColumnDef 155 > staticmethod 146 > FieldDescriptor 126 > > > I've tracked down those 3 types (builtin_function_or_method, io and timer) > that appear most in memory and the first seems to be a "switch" function. > Probably Gevent related... "io" is used by TSocket in the thrift code and > "timer" I could not find the back references... > > This seems to be a problem in the Gevent plugin of uWSGI, since if I > disable it but still use Gevent the problem goes away, but as there are so > many components interacting here it's hard to know for sure. Does anyone > have a clue about this? > > Thanks, > André > _______________________________________________ > uWSGI mailing list > [email protected] > http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi >
Try running with gevent mode + 2 async core (--gevent 2) and start blasting the system. If the memory constantly grows someone is leaking, otherwise you are simply experimenting huge object allocation on the multiple uwsgi/gevent cores/greenlets -- Roberto De Ioris http://unbit.it _______________________________________________ uWSGI mailing list [email protected] http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
