Re: [Tutor] How to debug a memory leak in a wsgi application?

2017-12-13 Thread James Chapman 
Why pymalloc? I presume this means you're using ctypes which means I have
more questions.

If you're allocating your own blocks of memory then you need to free them
too. IE, does each call to pymalloc have a corresponding call to pyfree?

Is the overhead of pythons built in malloc really a problem?

Are you changing pointers before you've freed the corresponding block of
memory?

There are many ways to create a memory leak, all of them eliminated by
letting python handle your memory allocations.

But, back to your original question, check out "valgrind".

HTH

--
James

On 6 December 2017 at 16:23, Etienne Robillard  wrote:

> Hi Alan,
>
> Thanks for the reply. I use Debian 9 with 2G of RAM and precompiled Python
> 2.7 with pymalloc. I don't know if debugging was enabled for this build and
> whether I should enable it to allow memory profiling with guppy... My
> problem is that guppy won't show the heap stats for the uWSGI master
> process. However I have partially resolved this issue by enabling
> --reload-on-rss 200 for the uwsgi process.  Previously, the htop utility
> indicated a 42.7% rss memory usage for 2 uWSGI processes. I have restarted
> the worker processes with SIGINT signal. Now my uwsgi command line looks
> like:
>
> % uwsgi --reload-on-rss 200 --gevent 100 --socket localhost:8000
> --with-file /path/to/file.uwsgi --threads 2 --processes 4 --master
> --daemonize /var/log/uwsgi.log
>
> My framework is Django with django-hotsauce 0.8.2 and werkzeug. The web
> server is nginx using uWSGI with the gevent pooling handler.
>
> Etienne
>
> Le 2017-12-06 à 10:00, Alan Gauld via Tutor a écrit :
>
>> On 06/12/17 09:21, Etienne Robillard wrote:
>>
>>> Hi
>>>
>>> I think my wsgi application is leaking and I would like to debug it.
>>>
>>> What is the best way to profile memory usage in a running wsgi app?
>>>
>> This is probably a bit advanced for the tutor list, you might
>> get a better response on the main Python list.
>>
>> But to get a sensible answer you need to provide more data:
>> What OS and Python version?
>> What toolset/framework are you using?
>> What measurements lead you to suspect a memory leak?
>>
>>
>>
> --
> Etienne Robillard
> tkad...@yandex.com
> https://www.isotopesoftware.ca/
>
> ___
> Tutor maillist  -  Tutor@python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor
>
___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

Re: [Tutor] How to debug a memory leak in a wsgi application?

2017-12-13 Thread Etienne Robillard 

Hi James,

Thank for your reply. Are you suggesting that under Linux the malloc() 
glibc library call is more memory efficient than using pymalloc?


Best regards,

Etienne


Le 2017-12-13 à 12:27, James Chapman a écrit :
Why pymalloc? I presume this means you're using ctypes which means I 
have more questions.


If you're allocating your own blocks of memory then you need to free 
them too. IE, does each call to pymalloc have a corresponding call to 
pyfree?


Is the overhead of pythons built in malloc really a problem?

Are you changing pointers before you've freed the corresponding block 
of memory?


There are many ways to create a memory leak, all of them eliminated by 
letting python handle your memory allocations.


But, back to your original question, check out "valgrind".

HTH

--
James

On 6 December 2017 at 16:23, Etienne Robillard > wrote:


Hi Alan,

Thanks for the reply. I use Debian 9 with 2G of RAM and
precompiled Python 2.7 with pymalloc. I don't know if debugging
was enabled for this build and whether I should enable it to allow
memory profiling with guppy... My problem is that guppy won't show
the heap stats for the uWSGI master process. However I have
partially resolved this issue by enabling --reload-on-rss 200 for
the uwsgi process.  Previously, the htop utility indicated a 42.7%
rss memory usage for 2 uWSGI processes. I have restarted the
worker processes with SIGINT signal. Now my uwsgi command line
looks like:

% uwsgi --reload-on-rss 200 --gevent 100 --socket localhost:8000
--with-file /path/to/file.uwsgi --threads 2 --processes 4 --master
--daemonize /var/log/uwsgi.log

My framework is Django with django-hotsauce 0.8.2 and werkzeug.
The web server is nginx using uWSGI with the gevent pooling handler.

Etienne

Le 2017-12-06 à 10:00, Alan Gauld via Tutor a écrit :

On 06/12/17 09:21, Etienne Robillard wrote:

Hi

I think my wsgi application is leaking and I would like to
debug it.

What is the best way to profile memory usage in a running
wsgi app?

This is probably a bit advanced for the tutor list, you might
get a better response on the main Python list.

But to get a sensible answer you need to provide more data:
What OS and Python version?
What toolset/framework are you using?
What measurements lead you to suspect a memory leak?



-- 
Etienne Robillard

tkad...@yandex.com 
https://www.isotopesoftware.ca/ 

___
Tutor maillist  - Tutor@python.org 
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor





--
Etienne Robillard
tkad...@yandex.com
https://www.isotopesoftware.ca/

___
Tutor maillist  -  Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor