Thanks Robert, I will try this. This next part is a little off topic, but related: As all the main purpose of this function is to make DB calls and incrementing certain fields in a row on our database, would there be a possible way to use redis (or some other queue) as a temporary database where I use mules to consume keys from redis whose values are hashes of the incremented fields we will eventually be updating our main database with? This way, rather than updating our database for each increment, we update our redis temp store for each increment and have the mules consume the accumulated data resulting in less writes to the main database.
On Fri, May 17, 2013 at 10:24 AM, <[email protected]> wrote: > Send uWSGI mailing list submissions to > [email protected] > > To subscribe or unsubscribe via the World Wide Web, visit > http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi > or, via email, send a message with subject or body 'help' to > [email protected] > > You can reach the person managing the list at > [email protected] > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of uWSGI digest..." > > > Today's Topics: > > 1. Re: Random HTTP Close on 413 Errors (Roberto De Ioris) > 2. Re: Python Decorators (Roberto De Ioris) > 3. Re: asynchronous function calls (Roberto De Ioris) > 4. Mulefunc decorator (Adroaldo de Andrade) > 5. Re: Failing client request on RPC server (Roberto De Ioris) > 6. HTTP router and http-keepalive (Andr? Cruz) > 7. Re: Failing client request on RPC server (dr??ek mr??ek) > 8. Re: Failing client request on RPC server (dr??ek mr??ek) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 17 May 2013 12:32:03 +0200 > From: "Roberto De Ioris" <[email protected]> > To: "uWSGI developers and users list" <[email protected]> > Subject: Re: [uWSGI] Random HTTP Close on 413 Errors > Message-ID: > <[email protected]> > Content-Type: text/plain;charset=utf-8 > > > > We've noticed that when making calls that return a 413 http status code > we > > sometimes see an error with the response from uwsgi. It seems random and > > we cannot figure out what causes this exactly. The only pattern we've > > noticed is some machines usually get an error, and others usually do not. > > We're using uwsgi 1.9.6 on CentOS 5 built without libcap devel. Has > > anyone seen this or can anyone explain what's going on? > > > > Thanks, > > Charles > > > Hi, if you have nginx on front, 413 errors are managed by itself. uWSGI > should not even get the request (check its logs). > > Regarding the trunctaed output, unless uWSGI has generated something > faking nginx, i have no idea why curl is closing the connection before > receiving the whole output > > > > > Below is an example bad call using curl: > > * About to connect() to greencde.dev port 8001 (#0) > > * Trying 127.0.0.1... > > * Connected to greencde.dev (127.0.0.1) port 8001 (#0) > >> POST /creative/upload_file HTTP/1.1 > >> User-Agent: curl/7.29.0 > >> Host: greencde.dev:8001 > >> Accept: */* > >> Cookie: > > > access_token=d29bcab1a2c72a29d0b1ab9fe9e1416cce7af08dd5d500d7958180460564d7534d41752194f8d10d50d20119af5c101fa99b8469b3271cb4d9c8e31ab65d4a6a05193f9b4 > >> Content-Length: 14625352 > >> Expect: 100-continue > >> Content-Type: multipart/form-data; > > boundary=----------------------------9247cf3763f8 > >> > > < HTTP/1.1 413 REQUEST ENTITY TOO LARGE > > < Content-Type: text/html > > < Content-Length: 174 > > * HTTP error before end of send, stop sending > > < > > > > > > Request Entity Too Large > > > > Requests cannot be larger than 10752 kb > > * Closing connection 0 > > > > > > -- > > Charles Law Software Developer > > Watch how we make online advertising simple: http://bit.ly/Ent_vid > > www.openx.com <http://www.openx.org/> | follow us on: > > Twitter<http://twitter.com/openx> > > Facebook <http://www.facebook.com/OpenX> > > LinkedIn<http://www.linkedin.com/company/openx/products> > > _______________________________________________ > > uWSGI mailing list > > [email protected] > > http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi > > > > > -- > Roberto De Ioris > http://unbit.it > > > ------------------------------ > > Message: 2 > Date: Fri, 17 May 2013 12:40:12 +0200 > From: "Roberto De Ioris" <[email protected]> > To: "uWSGI developers and users list" <[email protected]> > Subject: Re: [uWSGI] Python Decorators > Message-ID: > <[email protected]> > Content-Type: text/plain;charset=utf-8 > > > > So this means the websocket runs in its own Greenlet and I ask it if it > > has produced result and if not I check redis for messages. > > > > more or less, tha main greenlet (the one spawned by uWSGI) is attached to > the websocket and wait for it. Instead of calling uwsgi.websocket_recv() > (that will wait for a message), you have to wait (in gevent) for > uwsgi.connection_fd() that is the websocket file descriptor. > > In the mean time another greenlet (spawned by you) wait for redis. > > On no-traffic, both the greenlet are stopped, as soon as one of the two > socket (redis or websocket) is ready the correspondign greenlet will be > woke up. > > This approach requires an additional queue to pass infos from the redis > greenlet to the websocket one, so i am not a big fan of it. > > The best approach (for me) is waiting in the same greenlet both for redis > and the websocket. If you can report the redis part of your code i will > show you how to integrate it with websockets. In pseudo-code is something > like that: > > fd_websocket = uwsgi.connection_fd() > fd_redis = redis_socket > > while True: > fd = wait_for([fd_websocket, fd_redis]) > if fd == fd_websocket: > msg = uwsgi.websocket_recv() > redis_send(msg) > elif fd = fd_redis: > msg = redis_read > uwsgi.websocket_send(msg) > > > -- > Roberto De Ioris > http://unbit.it > > > ------------------------------ > > Message: 3 > Date: Fri, 17 May 2013 12:45:32 +0200 > From: "Roberto De Ioris" <[email protected]> > To: "uWSGI developers and users list" <[email protected]> > Subject: Re: [uWSGI] asynchronous function calls > Message-ID: > <[email protected]> > Content-Type: text/plain;charset=utf-8 > > > > Hey guys, > > I'm sure this is a basic question, but what is the most efficient way to > > make an asynchronous function call when handling at http request. > > I want the function call to be offloaded somewhere else as to have no > > effect on the speed of the response. I use the spooler for another part > > of > > my application (its been great) but I'm not sure if its too light weight > > for this part (based on nothing I guess). The function call will happen > > 100s of times a second and I want something that i know can handle it. > Is > > the spooler the correct choice (maybe uping my spooler processes) or do I > > need something more heavy duty? > > > > Thanks, > > Tony > > > > The spooler is too slow for such a load (unless you do not need to process > those functions reasonably fast). > > Maybe mules are best suited (check the @mulefunc decorator), but be sure > to tune their internal buffer as they will need to manage lot of packets > (if you note choosing a good value for the internal buffer is impossible, > use something like redis as the queue and then use a pool of mules as > consumers) > > -- > Roberto De Ioris > http://unbit.it > > > ------------------------------ > > Message: 4 > Date: Fri, 17 May 2013 09:23:51 -0300 > From: Adroaldo de Andrade <[email protected]> > To: [email protected] > Subject: [uWSGI] Mulefunc decorator > Message-ID: <[email protected]> > Content-Type: text/plain; charset=us-ascii > > Hi > > I'm using uWSGI 1.4.9 and Python 2.7.2, for Mac (64bit), and I made a > several consecutive calls to a function that is using @mulefunc decorator > to make some asynchronous tasks. My setup is for 3 workers and 1 mule. > > Well, it should have execute the function a bunch of times, but it is > executing just twice, not the amount of times the function was called. > > First I thought that I ran out mule queue space (64Kb), but each call is > only using 114 bytes, so that is not the issue. > > Code example: > > # config file > > [uwsgi] > master = true > chdir = %d/../server > home = %d/../environment > env = LANG=en_US.UTF-8 > env = LC_ALL=en_US.UTF-8 > env = LC_CTYPE=en_US.UTF-8 > env = DJANGO_SETTINGS_MODULE=production_settings > module = wsgi_production:application > socket = 127.0.0.1:3035 > processes = 3 > mule = 1 > uid = 1000 > gid = 1000 > enable-threads = true > single-interpreter = true > die-on-term = true > harakiri = 60 > harakiri-verbose = true > post-buffering = 8192 > max-requests = 1000 > vacuum = true > logto = /var/log/uwsgi/server.log > # Import mule funcs > import = foobar.bar > > # foo.py > > from foobar.bar import bar_work > > def async_barr(request): > for i in range(0, 10): > bar_work(i) > > > # bar.py > > from uwsgidecorators import mulefunc > > @mulefunc > def bar_work(wait): > print "** Calling mule! %d" % int(wait) > time.sleep(int(wait)) > > In the case it should call bar_work ten times, but it call just twice. > Adding more mules dint solve the problem. > > Thank you. > > ------------------------------ > > Message: 5 > Date: Fri, 17 May 2013 17:16:20 +0200 > From: "Roberto De Ioris" <[email protected]> > To: "uWSGI developers and users list" <[email protected]> > Subject: Re: [uWSGI] Failing client request on RPC server > Message-ID: > <[email protected]> > Content-Type: text/plain;charset=utf-8 > > > > Good to know it is possible :) Maybe it will come in handy for some > > purposes :-) > > > > Thanks. > > Tom > > > > In 1.9.11 a couple of wrappers for the rpc subsystem have been added: > > https://github.com/unbit/uwsgi-docs/blob/master/Changelog-1.9.11.rst > > please do not ask me to support SOAP :) > > -- > Roberto De Ioris > http://unbit.it > > > ------------------------------ > > Message: 6 > Date: Fri, 17 May 2013 17:00:37 +0100 > From: Andr? Cruz <[email protected]> > To: uWSGI developers and users list <[email protected]> > Subject: [uWSGI] HTTP router and http-keepalive > Message-ID: <[email protected]> > Content-Type: text/plain; charset=iso-8859-1 > > Hello. > > I'm trying to setup HTTP keepalive and, although my application appears to > send all the right headers, the connection is always closed when the client > issues requests with a body, like POSTs or PUTs. GETs keep the connection > open. > > Example (even with a content-length of 0 this breaks the connection): > > Connected to localhost. > Escape character is '^]'. > POST /XXX HTTP/1.1 > User-Agent: curl/7.24.0 (x86_64-apple-darwin12.0) libcurl/7.24.0 > OpenSSL/0.9.8r zlib/1.2.5 > Host: localhost:8000 > Accept: */* > Content-Length: 0 > Expect: 100-continue > > HTTP/1.1 200 OK > Date: Fri, 17 May 2013 15:52:23 GMT > Vary: Accept-Language, Cookie > Content-Length: 413 > Content-Type: application/json > Content-Language: pt > Connection: Keep-Alive > > {"bytes": 0, "thumb_exists": false, "rev": > "c3dcd9eb-bf09-11e2-9267-406c8f001c8a", "modified": "Fri, 17 May 2013 > 15:52:23 +0000", "cursor": > "action_upload,event_c3dc7f2e-bf09-11e2-95e7-406c8f001c8a,d1508fef-7bec-4641-a2ed-571abf52606b_c3dcfef3-bf09-11e2-8c83-406c8f001c8a", > "path": "/11232", "is_dir": false, "icon": "page_white.gif", "root": > "dropbox", "mime_type": "application/octet-stream", "size": "0 > bytes"}Connection closed by foreign host. <----------- > > > I have added to my config: > > add-header: Connection: Keep-Alive > http-keepalive: 1 > http-auto-chunked: 1 > > On another note, I've noticed that by connecting to the http socket and > sending just carriage returns, my WSGI application is invoked with invalid > arguments. The connection is broken after the first corrupted request. In > the case of Django, it blows up with: > > Traceback (most recent call last): > File > "/Users/andre/work/penv/discosite/lib/python2.7/site-packages/django/core/handlers/wsgi.py", > line 245, in __call__ > request = self.request_class(environ) > File > "/Users/andre/work/penv/discosite/lib/python2.7/site-packages/django/core/handlers/wsgi.py", > line 147, in __init__ > self.method = environ['REQUEST_METHOD'].upper() > KeyError: u'REQUEST_METHOD' > > > Is this permitted by the WSGI spec? If I send a GET first, and the > connection is kept alive, I'm allowed to just keep sending carriage > returns, generating errors, and the connection is not broken. > > Best regards, > Andr? > > ------------------------------ > > Message: 7 > Date: Fri, 17 May 2013 18:27:10 +0200 > From: dr??ek mr??ek <[email protected]> > To: uWSGI developers and users list <[email protected]> > Subject: Re: [uWSGI] Failing client request on RPC server > Message-ID: > <CAFYutdgkcCFvU=-RfHiWWfVpMi=DZMchUmgi= > [email protected]> > Content-Type: text/plain; charset="iso-8859-1" > > Thank you for implementing. It's working well :) > > When using this xmlrpc-plugin I assume all input params must be strings. > > Tom > > > On Fri, May 17, 2013 at 5:16 PM, Roberto De Ioris <[email protected]> > wrote: > > > > > > Good to know it is possible :) Maybe it will come in handy for some > > > purposes :-) > > > > > > Thanks. > > > Tom > > > > > > > In 1.9.11 a couple of wrappers for the rpc subsystem have been added: > > > > https://github.com/unbit/uwsgi-docs/blob/master/Changelog-1.9.11.rst > > > > please do not ask me to support SOAP :) > > > > -- > > Roberto De Ioris > > http://unbit.it > > _______________________________________________ > > uWSGI mailing list > > [email protected] > > http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi > > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: < > http://lists.unbit.it/pipermail/uwsgi/attachments/20130517/419511d7/attachment-0001.html > > > > ------------------------------ > > Message: 8 > Date: Fri, 17 May 2013 19:26:09 +0200 > From: dr??ek mr??ek <[email protected]> > To: uWSGI developers and users list <[email protected]> > Subject: Re: [uWSGI] Failing client request on RPC server > Message-ID: > <CAFYutdgPsYCCvF=2zKE+EwBBcA= > [email protected]> > Content-Type: text/plain; charset="utf-8" > > If so maybe it's slight inconvenience - only string params - but never mind > in case of param structure a la JSON. > > I like that type-rich approach described by Aarni's in example ( > https://gist.github.com/akx/5585146) is also possible. > > Tom > > > > On Fri, May 17, 2013 at 6:27 PM, dr??ek mr??ek <[email protected]> wrote: > > > Thank you for implementing. It's working well :) > > > > When using this xmlrpc-plugin I assume all input params must be strings. > > > > Tom > > > > > > On Fri, May 17, 2013 at 5:16 PM, Roberto De Ioris <[email protected] > >wrote: > > > >> > >> > Good to know it is possible :) Maybe it will come in handy for some > >> > purposes :-) > >> > > >> > Thanks. > >> > Tom > >> > > >> > >> In 1.9.11 a couple of wrappers for the rpc subsystem have been added: > >> > >> https://github.com/unbit/uwsgi-docs/blob/master/Changelog-1.9.11.rst > >> > >> please do not ask me to support SOAP :) > >> > >> -- > >> Roberto De Ioris > >> http://unbit.it > >> _______________________________________________ > >> uWSGI mailing list > >> [email protected] > >> http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi > >> > > > > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: < > http://lists.unbit.it/pipermail/uwsgi/attachments/20130517/e5e66c6c/attachment.html > > > > ------------------------------ > > _______________________________________________ > uWSGI mailing list > [email protected] > http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi > > > End of uWSGI Digest, Vol 44, Issue 21 > ************************************* >
_______________________________________________ uWSGI mailing list [email protected] http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
