Hello people, I'm forwarding a bug filed against Bottle (which I'm maintaining in Debian) by a fellow DD. You can see it here: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=584827
---8<--- In http://bottle.paws.de/docs/tutorial.html it says: File Objects and Streams Bottle passes everything that has a read() method (file objects) to the wsgi.file_wrapper provided by your WSGI server implementation. This wrapper should use optimised system calls (sendfile on UNIX) to transfer the file contents. @route('/file') def get_file(): return open('some/file.txt','r') However, in the code we have this: # Cast Files into iterables if hasattr(out, 'read') and 'wsgi.file_wrapper' in request.environ: out = request.environ.get('wsgi.file_wrapper', lambda x, y: iter(lambda: x.read(y), ''))(out, 1024*64) # Handle Iterables. We peek into them to detect their inner type. try: out = iter(out) first = out.next() while not first: first = out.next() except StopIteration: return self._cast('') except HTTPResponse, e: first = e except Exception, e: first = HTTPError(500, 'Unhandled exception', e, format_exc(10)) if isinstance(e, (KeyboardInterrupt, SystemExit, MemoryError))\ or not self.catchall: raise # These are the inner types allowed in iterator or generator objects. if isinstance(first, HTTPResponse): return self._cast(first) if isinstance(first, StringType): return itertools.chain([first], out) if isinstance(first, unicode): return itertools.imap(lambda x: x.encode(response.charset), itertools.chain([first], out)) return self._cast(HTTPError(500, 'Unsupported response type: %s'\ Bottle seems to be extracting the first element from the iterable in order to look at what the iterator returns and possibly add transparent recoding and whatnot, then it returns a chain iterator with the first element it took out and the rest. Now, the WSGI server, even if it supports the file_wrapper optimisation[1], will never be able to see that we are returning a file_wrapper, so the optimisation will never trigger, in the face of what Bottle's documentation says. [1] http://www.python.org/dev/peps/pep-0333/#optional-platform-specific-file-handling --->8--- A second message from him says: ---8<--- On Sun, Jun 06, 2010 at 11:33:20PM +0100, Enrico Zini wrote: > Now, the WSGI server, even if it supports the file_wrapper > optimisation[1], will never be able to see that we are returning a > file_wrapper, so the optimisation will never trigger, in the face of > what Bottle's documentation says. The same also defeats the idea that in WSGI, if you return an object that has a close() method, it will be called at the end of the request. In this case, Bottle will wrap our returned generator (which has a close method) in one that doesn't. Goodbye cleanup after streaming ends. A possible solution could be, if hasattr(out, 'read'), to skip the casting altogether: # Cast Files into iterables if hasattr(out, 'read'): if 'wsgi.file_wrapper' in request.environ: out = request.environ.get('wsgi.file_wrapper', lambda x, y: iter(lambda: x.read(y), ''))(out, 1024*64) return out And by the way, I see no reason in providing a default value to environ.get in that piece of code, since one line above we are checking that the value actually exists. So, much more simple and readable, and with only one environ lookup: # Cast Files into iterables if hasattr(out, 'read'): wrapper = request.environ.get('wsgi.file_wrapper', None) if wrapper: out = wrapper(out) return out This also lets the WSGI server decide the default value for the blocksize instead of hardcoding an arbitrary one. --->8--- Please keep the bug CCed in your replies. Kindly, David -- . ''`. Debian developer | http://wiki.debian.org/DavidPaleino : :' : Linuxer #334216 --|-- http://www.hanskalabs.net/ `. `'` GPG: 1392B174 ----|---- http://deb.li/dapal `- 2BAB C625 4E66 E7B8 450A C3E1 E6AA 9017 1392 B174
signature.asc
Description: PGP signature
