> > > On 10/19/2014 10:09 AM, Roberto De Ioris wrote: >>> On 10/18/2014 11:40 PM, Roberto De Ioris wrote: >>>>> Hello all, >>>>> >>>>> Hopefully I'm not asking an question that's been answered many times >>>>> before; I've been unable to find anything via Google. >>>>> >>>>> Short version: >>>>> When I attempt to stream data to a uwsgi server running a JSONRPC >>>>> application, every other request fails with the error 'Error writing >>>>> request body to server', even if the data is< 100B. This effect >>>>> where >>>>> the first request succeeds, the second fails, third succeeds, etc. is >>>>> completely reproducible and constant. When I set up the client so >>>>> that >>>>> it buffers the request and then sends it all in one piece everything >>>>> works normally. Also, if I start the server via >>>>> wsgiref.simple_server.make_server.serve_forever() everything works >>>>> normally. >>>>> >>>>> More details: >>>>> I'm currently testing with Java code; the error can be toggled on by >>>>> setting one of >>>>> http://docs.oracle.com/javase/7/docs/api/java/net/HttpURLConnection.html#setFixedLengthStreamingMode%28int%29 >>>>> http://docs.oracle.com/javase/7/docs/api/java/net/HttpURLConnection.html#setChunkedStreamingMode%28int%29 >>>>> >>>>> Also, putting nginx in front of the server fixes the problem, >>>>> presumably >>>>> because nginx buffers the entire request before sending it to uwsgi. >>>>> >>>>> Interestingly, if I sleep for at least 5s prior to each request, the >>>>> problem is resolved. At 4s every second request fails. >>>>> >>>>> The uwsgi command line is: >>>>> uwsgi --master --processes 20 --cheaper 4 \ >>>>> --http :10000 --http-timeout 600 --pidfile pyservice.pid >>>>> --daemonize uwsgi.log \ >>>>> --wsgi-file PyLogServer.py >>>>> >>>>> My uwsgi version is pretty old, 1.4.4. Ubuntu is 12.04. >>>>> >>>>> Any ideas what might be causing this problem / debugging suggestions? >>>>> Are there any more details I could provide that might be useful? >>>>> >>>>> Thanks, Gavin >>>>> >>>>> >>>> What you mean for "streaming ? >>>> >>>> Because if you mean chunked body it is not supported by WSGI specs >>>> (albeit >>>> uWSGI has an api for it) >>> Well... I mean whatever Java does under the hood when I call one of the >>> two methods linked above. Unfortunately I don't know the exact details, >>> but it probably is chunking the body. >>> >>>> Btw, you should post uWSGI logs for both successful requests and >>>> failed >>>> ones. And if you can previde an example client it would help in >>>> understanding what is going on. >>> I made a SSCCE: >>> client: http://pastebin.com/cNFgRMLY >>> server: http://pastebin.com/y2AYkScy >>> >>> client output is: >>> _______________________ >>> http://localhost:10000 >>> 0 >>> 200 OK >>> 0123456789 >>> 1 >>> java.net.SocketException: Unexpected end of file from server >>> 2 >>> 200 OK >>> 0123456789 >>> 3 >>> java.net.SocketException: Unexpected end of file from server >>> 4 >>> 200 OK >>> 0123456789 >>> 5 >>> java.net.SocketException: Unexpected end of file from server >>> Sleep: 0, failures 3/6 >>> _______________________ >>> >>> server logs: >>> ________________________ >>> *** Starting uWSGI 1.4.4 (64bit) on [Sun Oct 19 09:50:49 2014] *** >>> compiled with version: 4.6.3 on 23 January 2013 15:54:59 >>> os: Linux-3.2.0-23-generic #36-Ubuntu SMP Tue Apr 10 20:39:51 UTC 2012 >>> nodename: *snip* >>> machine: x86_64 >>> clock source: unix >>> pcre jit disabled >>> detected number of CPU cores: 7 >>> current working directory: *snip* >>> writing pidfile to pyservice.pid >>> detected binary path: /usr/local/bin/uwsgi >>> your processes number limit is 160617 >>> your memory page size is 4096 bytes >>> detected max file descriptor number: 10000 >>> lock engine: pthread robust mutexes >>> uWSGI http bound on :10000 fd 4 >>> uwsgi socket 0 bound to TCP address 127.0.0.1:46592 (port >>> auto-assigned) >>> fd 3 >>> Python version: 2.7.3 (default, Aug 1 2012, 05:25:23) [GCC 4.6.3] >>> *** Python threads support is disabled. You can enable it with >>> --enable-threads *** >>> Python main interpreter initialized at 0x22e8a80 >>> your server socket listen backlog is limited to 100 connections >>> mapped 1520232 bytes (1484 KB) for 20 cores >>> *** Operational MODE: preforking *** >>> WSGI app 0 (mountpoint='') ready in 0 seconds on interpreter 0x22e8a80 >>> pid: 4005 (default app) >>> *** uWSGI is running in multiple interpreter mode *** >>> spawned uWSGI master process (pid: 4005) >>> spawned uWSGI worker 1 (pid: 4006, cores: 1) >>> spawned uWSGI worker 2 (pid: 4007, cores: 1) >>> spawned uWSGI worker 3 (pid: 4008, cores: 1) >>> spawned uWSGI worker 4 (pid: 4009, cores: 1) >>> spawned uWSGI http 1 (pid: 4010) >>> [pid: 4009|app: 0|req: 1/1] 127.0.0.1 () {32 vars in 427 bytes} [Sun >>> Oct >>> 19 09:50:56 2014] POST / => generated 10 bytes in 0 msecs (HTTP/1.1 >>> 200) >>> 1 headers in 39 bytes (1 switches on core 0) >>> [pid: 4008|app: 0|req: 1/2] 127.0.0.1 () {32 vars in 427 bytes} [Sun >>> Oct >>> 19 09:50:56 2014] POST / => generated 10 bytes in 0 msecs (HTTP/1.1 >>> 200) >>> 1 headers in 39 bytes (1 switches on core 0) >>> [pid: 4009|app: 0|req: 2/3] 127.0.0.1 () {32 vars in 427 bytes} [Sun >>> Oct >>> 19 09:50:56 2014] POST / => generated 10 bytes in 0 msecs (HTTP/1.1 >>> 200) >>> 1 headers in 39 bytes (1 switches on core 0) >>> ______________________________ >>> >>> Note that there's nothing in the logs for the failed requests. >>> >>> When I run the server with wsgiref I get: >>> _______________________ >>> >> >> Basically both wsgiref and uWSGI generates the same output (10 bytes). >> My >> suspect is that the java client is complaining about the fact the the >> output is not fully http/1.1 compliant. >> >> Can you try adding Connection: close header in the response ? > That did the trick. I can toggle the error on and off by changing the py > response headers to: > response_headers = [ > ('Connection', 'close'), > ('content-length', str(len(request_body)))] > > Interestingly, calling > conn.disconnect() > on the java side changes nothing. > > Would you mind educating me (and any other interested readers) regarding > what's actually going on here? I read up on the Connection: close header > value (which I've never needed before) but it's not clear to me here why > it's necessary. > > I suppose another question is whether this solution will work in the > case the request body is actually chunked; my assumption in the test > case above is that there's only one chunk. > > Thanks again, > Gavin > >
>From what i can understand, the java client is making a keep-alive connection, so at the first request it gets the response but holds the connection open. The second request is issued but the server (uWSGI) has already closed the connection (and the java app complains). The third request reopen the connection and so on. Keep-alive mode requires perfect-framing in responses, so you need to set Connection, Content-Length, Content-encoding and whatever you need. Nginx and Apache are smart about it and completes uWSGI generated responses (generally transforming the output in chunked mode). Instead the uWSGI http router or the native http support requires you to configure how your app will behave. Generally adding Connection: close is more than enough (you can even instruct uWSGI to do it automatically with --add-header "Connection: close") Regarding input chunked encoding it is something not covered by WSGI, but uWSGI exposes an api for chunked input: http://uwsgi-docs.readthedocs.org/en/latest/Chunked.html so you can eventually use it -- Roberto De Ioris http://unbit.it _______________________________________________ uWSGI mailing list [email protected] http://lists.unbit.it/cgi-bin/mailman/listinfo/uwsgi
