Re: streaming patches reloaded
yeah, this is really a huge wart , what Django does here - holds everything in memory - works only for the simplest cases thus makes django unsuited for a whole slew of web applications too bad that in a news environment uploading larger files is not a typical use case, that's why this fix takes so long to be applied, folks, just please take care of this already, what you have in place is fundamentally wrong --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: Convince us to continue using setuptools
Never really understood why you needed this in the first place. The automatic setuptools installation has caused me some grief as it installs an obsolete version ... I forgot the exact specifics but it took me quite a while to track down that installing django was causing all kinds of strange problems with other python modules. I think it is fair to say that having the situation where installing django will also covertly install setuptools is greatly undesirable. As many have pointed out, using a MANIFEST.in provides a very simple means to use file globbing for files and directories. If that was the main reason for using setuptools you'd have a straightforward solution right away. i. ps. Nowadays I don't ever install it, I just simply add the django folder to my python path and have not had any problems. I would not treat django as a python module anymore, installing into the python site libraries makes keeping track of different releases very difficult. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: Call for comment: Forms/manipulator replacement, take 1
>{{ form.message.errors }}
>Message: {{ form.message }}
A very common use case is to format/highlight the error message or
input area upon error. I can already see myself extending the Form
class to include something of the sorts
{{ form.message.errors }}
Message: {{ form.message }}
where the css would be selected based on the state or valid/invalid.
i.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/django-developers
-~--~~~~--~~--~--~---
Re: New newforms code
Adrian Holovaty wrote: > Have you guys checked out the Widget class / capability in newforms? > That's the part that lets you define how a field should be output as > HTML. I'm definitely interested in hearing whether/how that doesn't > let you do what you want to do. Yes it worked out well. But it was quite a head scratcher, it takes a while to form a clear mental picture of how the widget is tied to the input field. In the documentation right at the top of the page, where the Widget, Field and Form is listed it should state that Fields are rendered via Widgets and that you can override the default widget by passing a new one in the constructor. That was the piece of information that it was unclear to me for the longest of time. You might also want to add a submit widget, I had to make my own for a case where there were multiple buttons on a page. After all the submit is a valid input widget as well. Also I had to roll my own FloatField (exactly the same as the IntegerField only with float). In general I found this documentation harder to follow that the other docs. It seems to be inverted. As it stands today it starts detailing the forms and how data is bound to them before telling about the fields and widgets. Yet a form consists of fields and fields use widgets. If it started explaining the widgets, then fields followed by forms it might make an easier read. best, i. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Django with CherryPy 3.0
There is an easy way to turn on multi-threading for the default server, see the diff below: Index: basehttp.py === --- basehttp.py (revision 4183) +++ basehttp.py (working copy) @@ -7,10 +7,16 @@ been reviewed for security issues. Don't use it for production use. """ -from BaseHTTPServer import BaseHTTPRequestHandler, HTTPServer +from BaseHTTPServer import BaseHTTPRequestHandler from types import ListType, StringType import os, re, sys, time, urllib +import BaseHTTPServer, SocketServer +class HTTPServer(SocketServer.ThreadingMixIn, BaseHTTPServer.HTTPServer): +def __init__(self, server_address, RequestHandlerClass=None): +BaseHTTPServer.HTTPServer.__init__(self, server_address, RequestHandlerClass ) +print '*** MixinHTTPServer ***' + __version__ = "0.1" __all__ = ['WSGIServer','WSGIRequestHandler','demo_app'] --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Django with CherryPy 3.0
Jeremy Dunck wrote: > That said, most of the time, this is good stuff. Perhaps a management > option to turn it off? My understanding is that no part of basehttp.py is used when Django is deployed via modpython. If that is correct then this addition would make no difference in a production environment. I've researched this mainly because I needed this functionality in the test server, used it for a month or so and it works surprisingly well. I did just for fun a little experiment and it seems to outperform the paste http server as well as cherrypy even under severe load. Potential pitfall: no limit on the number of threads (this could be easily added though), so if you were to use it as a production server (which of course you shouldn't) then you could be DOS'ed simply by having clients connect and not disconnect. Istvan --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
a simple patch for streaming uploads
There is ticket (and a patch) for enabling streaming uploads, but it
seemed a little complicated and until it gets accepted (or reworked)
here is nice workaround to do the same. See patch at the end.
It is backward compatible, the current behavior stays the same, you
can use still request.FILES to get the dictionary keyed by file names.
When you first access request.FILES it loads everything into memory.
Now for large files you can use request.STREAM to get a file handle on
the input. Note that in this latter case you'll need to do some simple
parsing to get your file.
The patch itself is only 5 lines, but I've changed a few method names
to be more consistent.
Istvan
Index: wsgi.py
===
--- wsgi.py (revision 4455)
+++ wsgi.py (working copy)
@@ -5,6 +5,7 @@
from django import http
from pprint import pformat
from shutil import copyfileobj
+import tempfile
try:
from cStringIO import StringIO
except ImportError:
@@ -108,12 +109,13 @@
def _load_post_and_files(self):
# Populates self._post and self._files
if self.method == 'POST':
+buffer = self.STREAM.read()
if self.environ.get('CONTENT_TYPE',
'').startswith('multipart'):
header_dict = dict([(k, v) for k, v in
self.environ.items() if k.startswith('HTTP_')])
header_dict['Content-Type'] =
self.environ.get('CONTENT_TYPE', '')
-self._post, self._files =
http.parse_file_upload(header_dict, self.raw_post_data)
+self._post, self._files =
http.parse_file_upload(header_dict, buffer )
else:
-self._post, self._files =
http.QueryDict(self.raw_post_data), datastructures.MultiValueDict()
+self._post, self._files = http.QueryDict( buffer ),
datastructures.MultiValueDict()
else:
self._post, self._files = http.QueryDict(''),
datastructures.MultiValueDict()
@@ -152,9 +154,10 @@
self._load_post_and_files()
return self._files
-def _get_raw_post_data(self):
+def _get_post_stream(self):
try:
-return self._raw_post_data
+self._stream.seek(0)
+return self._stream
except AttributeError:
buf = StringIO()
try:
@@ -162,18 +165,19 @@
content_length =
int(self.environ.get('CONTENT_LENGTH', 0))
except ValueError: # if CONTENT_LENGTH was empty string
or not an integer
content_length = 0
-safe_copyfileobj(self.environ['wsgi.input'], buf,
size=content_length)
-self._raw_post_data = buf.getvalue()
-buf.close()
-return self._raw_post_data
+
+self._stream = tempfile.TemporaryFile()
+safe_copyfileobj(self.environ['wsgi.input'],
self._stream, size=content_length)
+self._stream.seek(0)
+return self._stream
GET = property(_get_get, _set_get)
POST = property(_get_post, _set_post)
COOKIES = property(_get_cookies, _set_cookies)
FILES = property(_get_files)
REQUEST = property(_get_request)
-raw_post_data = property(_get_raw_post_data)
-
+STREAM = property(_get_post_stream)
+
class WSGIHandler(BaseHandler):
def __call__(self, environ, start_response):
from django.conf import settings
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---
Re: django documentation - editing
On Mar 25, 8:06 pm, Malcolm Tredinnick <[EMAIL PROTECTED]> wrote: > I fear that one day I will achieve that and will then be asked to leave > the safety of the monastery and journey the world righting injustice in > episodic television format... that is quite funny ... one more reason to read this newsgroup i. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Newforms: colon after label
I think the OP is correct, if you want the label to end with a colon ... then add the colon to the label. It doesn't get any simpler than that. Any other solution just gets in the way. i. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Streaming uploads to temp files
> But I have concerns about performance in the common case. File upload is inherently a slow process strictly limited by the connection speed so it is not clear that any kind of performace problems would be noticable at all (even if there were such problems) Loading files into memory is *extremely* limiting, there are great many applications, photo or music albums or rich web applications that need file upload. The last thing one wants is to have to worry about balooning up the memory footprint because of something as trivial as uploading a file. No sane application slurps up files of unkown size, why would a generic web framework do so? > rare to allow multi-tens-of-megabytes uploads from Joe Public The question rather is why make Django *incapable* of dealing with files without worrying of what a user might choose to upload? When the solution is rather trivial? And by the way, thanks for the nice patch Ivan. I'm (re)writing an application to Django and would have never done so had there not be this hope that this issue will be settled properly. Istvan --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: HttpResponse with file-like objects
Two observations: 1. It is hard to believe that Django does not already do this ... it is somewhat depressing to see that trivial behavior such as streaming output was never implemented 2. I'm commenting here as an outsider, but looking at the patch it seems that the _is_string attribute is unnecessary, all it seems to be used only to forbid certain methods, like writing, but why not let them happen? i. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: HttpResponse with file-like objects
but it is not an arbitraty iterator one is writing to, it is the the HttpResponse, why couldn't it handle new content being added just because some content is present as an interator? once it exhausts the original iterator it could continue on (if new content was added after it) or why not let the iterator decide whether it can be written to and raise an exception on its own if it cannot be done. hardcoding an exception on the truth value of the ._is_string attribute feels very ad-hoc --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: HttpResponse with file-like objects
> just can't imagine any practical application of this... suppose one wants to return content created by two generators ... they could wrap it themselves into a single one but why not write it both to the response and let that iterate thru each, if your code would support that it would end up being simpler, with no special casing for strings ... win-win situation --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: Connections not being closed in m-r
> I don't understand how the traceback module ends up being None > Adding a check for None will make the warning message go away fixing up code to make a mysterious error message go away is vey very suboptimal (ok ... so I've done I it myself a few times as well ...) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers -~--~~~~--~~--~--~---
Re: Django 100% threadsafe with DB?
Django is 0% threadsafe (as in nada, null or zilch) it is not supposed to be run that way, but if you must keep locking around every operation. i. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Django 100% threadsafe with DB?
On Sep 25, 10:58 pm, Graham Dumpleton <[EMAIL PROTECTED]> wrote: > ensure there own code is multithread safe, but now again someone is > saying that Django itself is not multithread safe. :-( Well, I was just repeating the what I heard from the developers. Not so long ago a proposal was made that involved a minimal change - but it would have made the builtin webserver multithreaded ( of course your know about his as you were involved as well). This was rejected on the grounds that multithreaded operations might not work correctly. See this: http://code.djangoproject.com/ticket/3357 Istvan. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
