Hi uWSGI,
This strange error killed one of our workers:
[pid: 20851|app: 0|req: 48986/51858] 172.16.1.69 () {40 vars in 1650 bytes}
[Tue Oct 21 07:07:03 2014] GET
/tracking/load-details?guid=cceca015-af92-4047-96f9-08ac70945a82&id=20170025
=> generated 383 bytes in 9 msecs (HTTP/1.1 302) 4 headers in 570 bytes (1
switches on core 1)
[pid: 20851|app: 0|req: 48986/51859] 172.16.1.69 () {50 vars in 1663 bytes}
[Tue Oct 21 07:06:53 2014] POST /tracking/tracking-priority-board-data =>
generated 35314 bytes in 11098 msecs (HTTP/1.1 200) 3 headers in 436 bytes
(1 switches on core 0)
[pid: 20851|app: 0|req: 48987/51860] 172.16.1.69 () {50 vars in 1701 bytes}
[Tue Oct 21 07:07:11 2014] POST /tracking/tracking-priority-board-data =>
generated 233237 bytes in 1071 msecs (HTTP/1.1 200) 3 headers in 435 bytes
(1 switches on core 1)
[pid: 20851|app: 0|req: 48988/51861] 172.16.1.69 () {50 vars in 1761 bytes}
[Tue Oct 21 07:07:22 2014] POST /tracking/tracking-priority-board-data =>
generated 34070 bytes in 733 msecs (HTTP/1.1 200) 3 headers in 461 bytes (1
switches on core 0)
[pid: 20851|app: 0|req: 48989/51862] 172.16.1.69 () {50 vars in 1830 bytes}
[Tue Oct 21 07:07:43 2014] POST /tracking/tracking-priority-board-data =>
generated 38109 bytes in 735 msecs (HTTP/1.1 200) 3 headers in 461 bytes (1
switches on core 1)
[pid: 20851|app: 0|req: 48990/51863] 172.16.1.69 () {42 vars in 1596 bytes}
[Tue Oct 21 07:07:51 2014] GET
/tracking/update-status/8c5cd457-8678-4757-956b-51a80a0ef071?_=1413893126339
=> generated 7946 bytes in 6664 msecs (HTTP/1.1 200) 3 headers in 460 bytes
(1 switches on core 0)
uwsgi: pthread_mutex_lock.c:62: __pthread_mutex_lock: Assertion
`mutex->__data.__owner == 0' failed.
DAMN ! worker 4 (pid: 20851) died, killed by signal 6 :( trying respawn ...
Respawned uWSGI worker 4 (new pid: 38722)
[pid: 20848|app: 0|req: 2710/51864] 172.16.1.69 () {50 vars in 1701 bytes}
[Tue Oct 21 07:08:13 2014] POST /tracking/tracking-priority-board-data =>
generated 233237 bytes in 1271 msecs (HTTP/1.1 200) 3 headers in 435 bytes
(1 switches on core 0)
[pid: 38722|app: 0|req: 48993/51865] 172.16.1.69 () {50 vars in 1772 bytes}
[Tue Oct 21 07:08:24 2014] POST /tracking/tracking-priority-board-data =>
generated 34070 bytes in 2844 msecs (HTTP/1.1 200) 3 headers in 461 bytes
(1 switches on core 0)
What happened?
Some background: For most people a worker going down isn't the end of the
world, but our app uses threads which acquire locks through Memcached (we
use a load balancer and multiple app servers), so if a request gets killed
before it can release the lock, that lock stays held forever; e.g. our
(albeit experimental) app has been out of commission since Tuesday.
Incidentally, I'm all ears if you have suggestions on handling that more
resiliently :)
We're using uWSGI 2.0.1 and Ubuntu 12.04.3, both of which are a little out
of date, so maybe that's the first thing we should do. In Ubuntu I think
glibc is called libc6, and that version is 2.15-0ubuntu10.5.
Here's uname -a: Linux bridge-dev-web01 3.8.0-29-generic
#42~precise1-Ubuntu SMP Wed Aug 14 16:19:23 UTC 2013 x86_64 x86_64 x86_64
GNU/Linux
Thanks,
Matt Phipps
_______________________________________________
uWSGI mailing list
[email protected]
http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
