Re: Feature proposal: Test client form value extraction
Hi, On Thu, Aug 27, 2009 at 03:28:03PM -0100, Joshua Russo wrote: > On Thu, Aug 27, 2009 at 11:54 AM, Joshua Russo wrote: > Ok, so I found that the way I was 'casting' the response object didn't work. > Is > there no way to cast an instance of a base class to a child class in Python? > > What I did was to create a class method on my child class that takes the > current response instance and creates a copy of it using my new response > object. > > http://dpaste.com/hold/86193/ > > Does this look appropriate, or is there a better way to do this? This should work: response.__class__ = TestHttpResponse -Forest -- Forest Bond http://www.alittletooquiet.net http://www.pytagsfs.org signature.asc Description: Digital signature
Re: Feature proposal: Test client form value extraction
Hi, On Thu, Aug 27, 2009 at 07:42:24PM -0100, Joshua Russo wrote: > On Thu, Aug 27, 2009 at 6:22 PM, Forest Bond > wrote: > > Hi, > > On Thu, Aug 27, 2009 at 03:28:03PM -0100, Joshua Russo wrote: > > On Thu, Aug 27, 2009 at 11:54 AM, Joshua Russo > wrote: > > Ok, so I found that the way I was 'casting' the response object didn't > work. Is > > there no way to cast an instance of a base class to a child class in > Python? > > > > What I did was to create a class method on my child class that takes the > > current response instance and creates a copy of it using my new response > > object. > > > > http://dpaste.com/hold/86193/ > > > > Does this look appropriate, or is there a better way to do this? > > This should work: > > response.__class__ = TestHttpResponse > > > Nope, it resets the status_code. That's what I tried at first. Okay, you'd have to copy the class attributes, too. Or, you could just define a sub-class on-the-fly: class _TestHttpResponse(TestHttpResponse, response.__class__): pass response.__class__ = _TestHttpResponse I haven't tested this. -Forest -- Forest Bond http://www.alittletooquiet.net http://www.pytagsfs.org signature.asc Description: Digital signature
Re: Status of branch soc2009/http-wsgi-improvements? (and ticket 2131)
Hi,
On Wed, Feb 10, 2010 at 11:09:29AM -0800, mrts wrote:
> On Feb 10, 5:24 pm, Tom Evans wrote:
> > On Wed, Feb 10, 2010 at 3:09 PM, Jari Pennanen
> > wrote:
> > > Hi!
> >
> > > I was wondering what is the status of branch branches/soc2009/http-
> > > wsgi-improvements
> > > (http://github.com/django/django/tree/soc2009/http-wsgi-improvements
> > > )? I'm personally interested one bug it fixes, mainly ticket #2131
> > > (http://code.djangoproject.com/ticket/2131)
>
> You don't need that fix to use efficient file serving.
>
> Just use an empty response and set the X-Sendfile header manually if
> using
> Apache. If not, check your server documentation for the right header
> name.
>
> E.g.:
>
> response = HttpResponse()
> response['X-Sendfile'] = '/full/path/to/file'
> response['Content-Type'] = 'some/content_type'
> response['Content-Length'] = os.stat('/full/path/to/file').st_size
> response['Content-Disposition'] = 'attachment; filename="foo"'
> return response
This only works for use cases where you have an on-disk file to serve. There
are plenty of situations where this may not be the case:
* Cloud storage services (e.g. Rackspace Cloud Files)
* Large data sets (e.g. data exports to CSV, XML)
Etc.
Thanks,
Forest
--
Forest Bond
http://www.alittletooquiet.net
http://www.pytagsfs.org
signature.asc
Description: Digital signature
[RFC] Fixing middleware/view decorator handling of content iterators
Hi,
Django allows an iterator to be passed as response content when instantiating an
HttpResponse. However, doing so causes problems with the following classes and
functions:
UpdateCacheMiddleware:
Caches the response object using the configured cache backend, but most
iterators cannot be pickled.
CommonMiddleware:
Calculates a value for the ETag header, causing the iterator to be drained
prematurely.
GzipMiddleware:
Calculates the response content length, causing the iterator to be drained
prematurely.
ConditionalGetMiddleware
Calculates the response content length and sets the Content-Length header,
causing the iterator to be drained prematurely.
patch_response_headers
Calculates a value for the ETag header, causing the iterator to be drained
prematurely.
Some of these problems are discussed in the following tickets:
#6027: FileWrapper iterator drained by GzipMiddleware before content can be
returned
#6527: A bug in HttpResponse with iterators
#12214: never_cache decorator breaks HttpResponse with iterator content
Attached is a proof-of-concept patch illustrating how I'd like to tackle this
issue.
Roughly, my approach is as follows:
* Introduce a ResponseContent class with subclasses MutableResponseContent and
IteratorResponseContent to limit operations on the response content depending
on the type of the value provided (string type or iterator).
* Forbid premature draining of the content iterator via response.content by only
evaluating the content iterator if accessed via iter(response) and raising an
exception if it is accessed via response.content.
* Modify middleware/view decorator code that accesses response.content to check
response.content_obj.is_readable() before trying to read the content.
This approach should be backwards compatible, with the exception that if a
middleware or view decorator does inappropriately access response.content, a
TypeError will be raised. Previously, the content iterator would be prematurely
drained and an empty response would be returned. Since the previous behavior
was not usable, it seems unlikely anyone would be depending on it.
I have a few questions:
* Does this approach make sense to others? If so, I'll add a few tests and look
at how the docs might need updating.
* Is this type of change too invasive for 1.2?
* Should I create a "master" ticket to deal with this issue and continue
development work there? The other tickets focus on specific symptoms of this
problem rather than the root cause more generally.
Any comments are appreciated.
Thanks,
Forest
--
Forest Bond
http://www.alittletooquiet.net
http://www.pytagsfs.org
=== modified file 'django/http/__init__.py'
--- django/http/__init__.py 2010-01-23 23:13:00 +
+++ django/http/__init__.py 2010-02-15 20:24:31 +
@@ -1,5 +1,6 @@
import os
import re
+from cStringIO import StringIO
from Cookie import BaseCookie, SimpleCookie, CookieError
from pprint import pformat
from urllib import urlencode
@@ -296,6 +297,107 @@
class BadHeaderError(ValueError):
pass
+def iter_string_chunked(s, chunk_size):
+for i in range(0, len(s), chunk_size):
+yield s[i:i+chunk_size]
+
+class ResponseContent(object):
+charset = None
+
+def __init__(self, value, charset):
+self.charset = charset
+self.set_value(value)
+
+def __unicode__(self):
+raise TypeError('unicode conversion not supported by %r' % type(self))
+
+def __str__(self):
+raise TypeError(
+ 'byte string conversion not supported by %r' % type(self)
+)
+
+def set_value(self, value):
+raise NotImplementedError()
+
+def is_readable(self):
+raise NotImplementedError()
+
+def close(self):
+pass
+
+def write(self, data):
+raise TypeError('file-like interface not supported by %r' % type(self))
+
+def flush(self):
+raise TypeError('file-like interface not supported by %r' % type(self))
+
+def tell(self):
+raise TypeError('file-like interface not supported by %r' % type(self))
+
+class MutableResponseContent(ResponseContent):
+_buffer = None
+_chunk_size = 102400
+
+def __init__(self, *args, **kwargs):
+self._buffer = StringIO()
+super(MutableResponseContent, self).__init__(*args, **kwargs)
+
+def __unicode__(self):
+return self._buffer.getvalue().decode(self.charset)
+
+def __str__(self):
+return self._buffer.getvalue()
+
+def __iter__(self):
+return iter_string_chunked(str(self), self._chunk_size)
+
+def set_value(self, value):
+if not isinstance(value, basestring):
+raise TypeError(type(value))
+if isinstance(value, unicode):
+value = value.encode(self.charset)
+self._buffer.seek(0)
+self._buffer.truncate()
+self._buffer.write(value)
+
+def is_readable(
Re: [RFC] Fixing middleware/view decorator handling of content iterators
Hi, On Mon, Feb 15, 2010 at 03:27:27PM -0600, Jeremy Dunck wrote: > On Mon, Feb 15, 2010 at 3:16 PM, Forest Bond > wrote: > ... > > * Forbid premature draining of the content iterator via response.content by > > only > > evaluating the content iterator if accessed via iter(response) and raising > > an > > exception if it is accessed via response.content. > ... > > * Is this type of change too invasive for 1.2? > > I assume you mean 1.3, since 1.2 is in feature freeze. ... and so this change would be disallowed if considered a new feature, but it's purpose is to fix bugs, although it is a non-trivial change. > Anyway, the response.content attribute is documented and client code > may depend on it, making this a backwards-compatibility issue. > > http://docs.djangoproject.com/en/dev/ref/request-response/#django.http.HttpResponse.content Yes, I did address this in my message. Accessing response.content if an iterator was passed in would result in an empty response, which doesn't seem like behavior that is worth preserving in the name of backwards compatibility. On the other hand, the following pattern is the exception (this would not produce an empty response): response.content = some_function(response.content) I think that pattern is the primary backwards compatibility concern. > For GZip support of iterators and similar large in-flight values, I > think spooling to memory and disk, like file upload does, might be a > reasonable solution, but still, .content is just a string, so it'll be > difficult to keep compatibility. If backwards compatibility for this case must be maintained, my approach would need to be adjusted to either evaluate the iterator and store the result (as you suggest) or to keep the current behavior WRT response.content and just fix the offending middleware/view decorators (i.e. use my patch but not throw an exception when response.content is read). If we do take the evaluate-and-store approach, I'd want to give the user some control over this behavior so that he could disable it if the content was going to be quite large or otherwise inappropriate for temporary storage. Thanks for your feedback! -Forest -- Forest Bond http://www.alittletooquiet.net http://www.pytagsfs.org signature.asc Description: Digital signature
Re: [RFC] Fixing middleware/view decorator handling of content iterators
Hi, Thanks for your response, Russ. Let me put my use case out there to ground my responses a bit. I'm using Rackspace Cloud Files, a cloud storage service, to store large files (several hundred megabytes up to a few gigabytes in size). I need to serve these files to authenticated users via a view. (There are probably other ways to expose the files, but this is by far the simplest and most obvious. I'd rather not open the discussion up to include rearchitecting the design to side step the iterator issue.) Doing this requires being able to stream the data from the storage service to the user. Reading all data from the storage service and then sending it to the user would not work as the user's connection would time out. I don't really care to cache the data on the web server, anyway. The point of the storage service is to offload the storage of these large files so that the web server does not need to accommodate them. I don't want to be worried about the web server running out of disk space if it has to handle too many simultaneous requests for files. On Thu, Feb 18, 2010 at 08:00:22PM +0800, Russell Keith-Magee wrote: > I acknowledge that this fixes the problem (in the sense that it makes > the errors caused by multiple iteration go away), but I'm not sure > it's the right solution. Disabling Etags, content-length coding, > gziped content and so on seems like throwing the baby out with the > bathwater. These features are not disabled, of course, they are just not automatically calculated. If a view needs to return an iterator but it wants these things, it should be able to calculate Etag and Content-Length headers itself easily enough. For instance, Cloud Files provides an Etag and size for each file that is stored. These can be accessed without retrieving the file and are trivial to pass along in the HTTPResponse from my view. > Of course, the simplest way to do this would be to just evaluate the > iterator as soon as it is provided -- which kind of negates any > benefit of using an iterator in the first place. So -- I suppose the > bigger question that needs to be asked is what exactly is the use case > for an iterable response? I mean, I understand the general benefit of > using iterators and generators instead of rolled out lists, but what > is the use case in a HTTP context? ... and I think this hits the nail on the head. Why would I be looking to use an iterator unless I was dealing with a data stream that was going to take some time to deliver? In other words, does it ever make sense to use an iterator except in those situations where capturing the response content before sending it to the client would be prohibitive? In my mind, the answer to that question is "no". However, I do see that enforcing a single iteration may seem restrictive for the general case. If that is the consensus, then I think there should be some way to trigger streaming-only, iterate-once handling. Perhaps one of the following options makes sense: * StreamingHttpResponse(content = iterator) * HttpResponse(content = iterator, streaming = True) * HttpResponse(content = StreamingResponseContent(iterator)) Then, for the general case, we could capture the data from the iterator and store it on disk or in memory so that it can be accessed arbitrarily in middleware, etc. Frankly, storing it in memory makes sense to me given that if there's a lot of data you'll want to use the streaming version to avoid connection timeouts. Maybe there's some use-case I'm not thinking of, though. > > * Is this type of change too invasive for 1.2? > > I'm inclined to say yes. I know this is addressing a number of > specific tickets, so it isn't strictly a feature addition, but those > issues have existed for a long time, and the change you're proposing > has a conceptually wide footprint - I'd rather see changes like this > treated like a feature, so they get the benefit of review through > multiple prereleases, etc. Makes sense to me. > > * Should I create a "master" ticket to deal with this issue and continue > > development work there? The other tickets focus on specific symptoms of > > this > > problem rather than the root cause more generally. > > This would be a good idea. Given that there is a specific course of > action on the table, it's worth having a ticket to track the evolution > of the idea. By way of comparison - this is exactly what Ben Firshman > did with class-based Syndication views in ticket #12403; the ticket > addressed a dozen other tickets, but provided a single place to > coordinate review of the specific solution. Okay, I'll create a ticket. > Make sure you reference any other existing tickets, but don't close > those tickets as duplicates - that way, if we find
Re: django + memcached: problem with limit of 1 mb
Hi, On Thu, Feb 18, 2010 at 03:17:32PM -0500, Alex Gaynor wrote: > On Thu, Feb 18, 2010 at 3:12 PM, Jeremy Dunck wrote: > > On Thu, Feb 18, 2010 at 1:34 PM, ramu...@gmail.com > > wrote: > >> Does anyone know why django can't keep a data bigger, than 1 mb in > >> memcached ? > >> > >> This is no big deal to cut big data to 1mb pieces before setting and > >> merge this pieces after getting from memcached. And this algorithm can > >> work transparently for users. > > > > I think Ramusus is asking for the low-level caching framework to split > > serialized data into 1MB chunks before storing in a bunch of keys if > > the data is larger than 1MB. > > > > He's not asking why there's a limit, he's asking for Django to work > > around the limit. > > > > I think the general reasoning for a 1MB limit is that it places bounds > > on access/storage times for a given key, and avoids an antipattern of > > storing really large objects in cache. I've wished for special-case > > side-stepping of the 1MB limit before myself, but I'd hate for it to > > be the default. > > I don't see how that would be possible, sure storing it would be easy, > but how do you retrieve it? Either you have to always try the > "part-2" key (unreasonable overhead fro 99% of the use cases) or you > have to store a tuple of (blocks, data) in the first slot, that's not > a huge runtime overhead, but it feels like a large code-complexity > overhead for a very rare usecase (I had no idea memcached even had > these limits, and I use it pretty extensively). Such operations would also not be atomic. Concurrent attempts to update cache would be problematic. Thanks, Forest -- Forest Bond http://www.alittletooquiet.net http://www.pytagsfs.org signature.asc Description: Digital signature
Re: Fixing middleware/view decorator handling of content iterators
e.disabled_middleware` (list or tuple). Any installed > middleware specified there should be bypassed when applying response > middleware. Django could then ship with an `HttpStreamingResponse` > class that disables any included middleware that is known not to work > with generator content, and this could easily be subclassed and > updated by users who need to disable any 3rd party middleware to > create a streaming response. Okay, I think "disabled_middleware" is a bad name because it doesn't actually imply that all middleware should be disabled. Anyway, it seems like middleware could just check if isinstance(response, StreamingHttpResponse). I look forward to your feedback. Thanks, Forest -- Forest Bond http://www.alittletooquiet.net http://www.pytagsfs.org signature.asc Description: Digital signature
Re: Fixing middleware/view decorator handling of content iterators
Hi, On Sat, Feb 20, 2010 at 03:46:52PM -0500, Forest Bond wrote: > Then there is the question of how to trigger handling as a streaming response. > Either we require a view to explicitly request it by using a > StreamingHttpResponse instead of a normal HttpResponse, or we assume that if > the > view returns an iterator, it wants a streaming response. > > Personally, I think that any view that returns an iterator must want a > streaming > response. Otherwise, why would it have returned an iterator? I can't think > of > any other reason to do that. However, I have no objection to requiring views > use StreamingHttpResponse explicitly. I should note that in order to maintain backwards compatibility, passing an iterator to HttpResponse should deliver at least the current level of support for streaming the response. That's an argument in support of enabling streaming behavior for a response if an iterator is passed in (i.e. not requiring views to use StreamingHttpResponse to get streaming behavior). -Forest -- Forest Bond http://www.alittletooquiet.net http://www.pytagsfs.org signature.asc Description: Digital signature
Re: Fixing middleware/view decorator handling of content iterators
Hi,
Sorry for the volume of messages, but I had one more note to add. ;)
On Sat, Feb 20, 2010 at 03:46:52PM -0500, Forest Bond wrote:
> Here are the use cases that have been identified:
>
> 1. A content iterator was passed for no apparent reason. The view could just
>as easily have returned HttpResponse(''.join(iterator)) with no ill
> effects.
>I assume this use-case is what people have in mind when they talk about
>evaluating the iterator and storing the content in memory or on disk; doing
>this with large responses could cause a connection timeout.
If it wasn't already clear, I don't think that this use case is one that should
be supported. I included it because it seems that there is some expectation
that it is (as evidenced by the proposal that iterators be evaluated and the
result stored in memory or on disk).
-Forest
--
Forest Bond
http://www.alittletooquiet.net
http://www.pytagsfs.org
signature.asc
Description: Digital signature
Re: Fixing middleware/view decorator handling of content iterators
Hi, On Sun, Feb 21, 2010 at 01:28:23AM -0800, Tai Lee wrote: > On Feb 21, 7:46 am, Forest Bond wrote: > > Okay, I think "disabled_middleware" is a bad name because it doesn't > > actually > > imply that all middleware should be disabled. Anyway, it seems like > > middleware > > could just check if isinstance(response, StreamingHttpResponse). > > I'm not sure I follow your reasoning here. If you just don't like the > name, that's a bike shed argument. I'm proposing that > `disabled_middleware` be a list or tuple of middleware that should NOT > be called on the response, because the author of the `HttpResponse` > subclass that defined `disabled_middleware` knows that those > middleware are not needed or will not work as intended. Sorry, I badly misread your message. Now that I've reread it I see that the name is perfectly appropriate. Thanks, Forest -- Forest Bond http://www.alittletooquiet.net http://www.pytagsfs.org signature.asc Description: Digital signature
Re: Fixing middleware/view decorator handling of content iterators
xcept KeyError:
pass
else:
if content_length < 20:
# This might be implemented using itertools.tee
add_streaming_response_to_cache(response)
else:
add_normal_response_to_cache(response)
Obviously this is an oversimplification (I ignore never_cache, etc.), but I
think the point stands. Simply knowing whether or not the response is streaming
and whether or not it has a known size is sufficient for knowing whether or not
you should cache it.
(If you wanted to get fancy you could handle local caching of streaming
responses without a known size–you could wrap the content iterator with a
generator function that determines the final size of the content and possibly
caches it.)
> What I'm driving at here is getting to the core of duck typing. As the
> developer of FooMiddleware, I don't really care whether I get a
> HttpResponse, a StreamingHTTPResponse, or something else entirely --
> what I care about is whether I can iterate over the content, modify
> the content, ask for a content length, or whatever it is that my
> middleware needs to do. The capabilities is more important than the
> specific type -- especially when the capabilities are subject to
> change (such as Forrest's example where Etags and content lengths can
> be provided, as long as GZipping is disabled).
I completely agree that middleware should be able to inspect response objects to
employ duck-typing-style checks. But I don't think that a formalized
capabilities-based interface is the only way to do this, and I think that
capabilities ("can I do X") are only half of the picture (the other half being
"should I do X"), so even if response capabilities are known, middleware is
still going to have to infer a lot based on what headers are already set.
But if someone can come up with a use case where "has header bar?" and
isinstance(response, StreamingHTTPResponse) are insufficient and a
capabilities-style interface would work, I will gladly accept that interface.
Thanks,
Forest
--
Forest Bond
http://www.alittletooquiet.net
http://www.pytagsfs.org
signature.asc
Description: Digital signature
Re: Fixing middleware/view decorator handling of content iterators
Hi, On Mon, Feb 22, 2010 at 12:39:40AM +, Luke Plant wrote: > On Sunday 21 February 2010 23:07:35 Forest Bond wrote: > > > Simply knowing whether or > > not the response is streaming and whether or not it has a known > > size is sufficient for knowing whether or not you should cache it. > > I can't find it now, but someone definitely had a use case where they > definitely wanted local caching *and* a streaming response (it was > views that took a long time to generate a small amount of actual > content - they outputted an occasional HTML comment to stop the > connection from timing out). So I don't think the equation you make > stands - we have to leave this to the developer to tell us. Caching streamed responses doesn't require a capabilities-style interface. The CacheMiddleware could do this regardless of interface and either way would have to wrap the content iterator in a generator. Also, the view developer can already explicitly request non-caching behavior using never_cache. This is the established way for the developer to specify which behavior he wants. > Implementing Russell's proposal could be as simple as: > >if getattr(response, 'can_modify_content', True): ># go ahead and modify content > > This: > - is backwards compatible > - is easy to use - just add the attribute to your response, >or even create a decorator for the view > - has far better duck typing properties than using isinstance. >This is really important for people who may have developed >their own HttpResponse subclass. > > (That's not necessarily a final API, I'm just illustrating the > advantages over isinstance). I don't think the discussion is over using attributes versus isinstance. I'd be happy with a response.is_streaming attribute instead of using isinstance(response, StreamingHttpResponse). If there is disagreement, it is between the following two view points: * We require a set of abstract, discrete response "capabilities" to implement middleware response handling semantics. * There are two kinds of responses: "streamed" and "non-streamed". Everything else is just headers. As Russ noted, the real challenge in implementing a capabilities-based interface is in deciding what the capabilities should be. Deciding what capabilities should be supported and how they should be interpreted by middleware is much more difficult than deciding how they should be accessed. For instance, can you describe the semantics of this hypothetical can_modify_content capability? How should middleware interpret it? What use case in particular does it address? And even if a response has "can_modify_content" set to True, how will the middleware know whether the content should be modified using string operations or by wrapping an iterator? Is that a separate capability? Thanks, Forest -- Forest Bond http://www.alittletooquiet.net http://www.pytagsfs.org signature.asc Description: Digital signature
Re: [Django Code] #3944: django.core.urlresolvers.reverse_helper doesn't support full regex syntax
On Fri, Apr 06, 2007 at 03:17:41AM -, Django Code wrote: > Parsing regexps is hardly dangerous. Let's keep some perspective. Pardon my tone, I suppose "dangerous" might be a bit strong of a word. > It is difficult and if you have a better way to do reverse matching, please > post a patch. Well, I think that the current approach can probably work with 90% of the regexes it will encounter in the wild, but the other 10% will be difficult, if not impossible, to deal with well. This is a largely a consequence of a few things: * Regular expressions are designed for text-matching, and, consequently, don't fully capture the intent of URL designers. This inevitably leads to ambiguity in some cases. * Writing a complete & correct ad-hoc regular expression parser in pure Python is probably not ideal for this particular scenario. Clearly, if the parser is not at least mostly complete & correct, it will encounter regexes that it can't deal with well. I'll consider my second point self-evident. An example illustrates the first point well: ^things/eating/things(?:/([a-z]+)(?:/(?:([0-9]+*$ URL's that would match: things/eating/things/zebras which would show an unknown quantity of zebras eating an immeasurably numerous wide variety of things things/eating/things/zebras/589 which would show 589 zebras eating an immeasurably numerous wide variety of things things/eating/things/zebras/turkeys which would show an unknown quantity of zebras eating an equally unknown number of turkeys things/eating/things/zebras/turkeys/25 which would show an unknown quantity of zebras eating 25 turkeys things/eating/things/zebras/589/turkeys/25 which would show 589 zebras eating 25 turkeys Now I'm not saying that this is the most useful case. And, yes, the URL could be re-written so that it would probably work with the reverse resolver. But wouldn't it be more straight-forward for the URL designer to explicitly specify how to construct a URL that matches the pattern? Something like this: lambda args: ( 'things/eating/things%s' % ''.join([ '/%s' % str(a) for a in args if a ])) This is a slight violation of DRY, but it should be an optional addition to the url conf. It would only be necessary if the URL designer's regular expression is complex enough to break the reverse resolver. In thinking about it, this example may not be an ideal illustration. Maybe I can do better. My point rests on the assumption that, no matter how good django gets at parsing regular expressions, it will always be possible to write one that works perfectly for forward resolution, but breaks for reverse resolution. That's acceptable, I think, but django should probably provide a means by which the designer can work around it, without changing his regex. Consider the | operators, look-ahead operators, etc... Not easy to deal with without a full regular expression implementation with a grammar and what-not. > The functionality requested here is a dupe of #2977. But I'm sure 2977 isn't as exciting to read. As a result, I maintain that this ticket continues to offer value to the django community. :) -Forest signature.asc Description: Digital signature
Re: [Django Code] #3944: django.core.urlresolvers.reverse_helper doesn't support full regex syntax
On Fri, Apr 06, 2007 at 11:26:07PM +1000, Malcolm Tredinnick wrote: > On Fri, 2007-04-06 at 09:08 -0400, Forest Bond wrote: > > My point rests on the assumption that, no matter how good django gets at > > parsing regular expressions, it will always be possible to write one that > > works perfectly for forward resolution, but breaks for reverse resolution. > > That's acceptable, I think, but django should probably provide a means by > > which the designer can work around it, without changing his regex. > > You already can work around this restriction when you need to -- and > easily. Using reverse() and permalink() are not compulsory! If there is > some reg-exp that cannot be reversed for some reason, then instead of > adding extra code to support another syntax, we just ask you to write > exactly a method like you propose and call it directly. It's not like > calling functions is hard, even in templates. > > > Consider the | operators, look-ahead operators, etc... Not easy to deal > > with without a full regular expression implementation with a grammar and > > what-not. > > I'm unconvinced that look-ahead operators are needed for URL parsing. > What's a real-world example that cannot be done just as easily without > them? Again, I'll point to #2977's enhancements and claim they're pretty > much sufficient. Ok, I think you are correct. Thank you. -Forest signature.asc Description: Digital signature
SortedDict.__repr__
Hi,
This is more-or-less a problem with Python's dict implementation, IMO. It's
really nothing more than a convenience thing, but for SortedDict, repr(d)
doesn't indicate the proper key order. It seems that the standard dict.repr
does not iterate over the dict like I would expect it to.
Attached is a patch that re-implements __repr__ using an iterator over the
object. Thus, SortedDict's are displayed properly in the Python shell.
-Forest
Index: django/utils/datastructures.py
===
--- django/utils/datastructures.py (revision 4936)
+++ django/utils/datastructures.py (working copy)
@@ -99,6 +99,14 @@
obj.keyOrder = self.keyOrder
return obj
+def __repr__(self):
+'''Returns string representation of d. The Python default for dicts
+doesn't seem to use keys(). We just re-implement it as an iterator
+over self so that the representation appears to have the correct order.
+'''
+return '{%s}' % ', '.join(
+ '%s: %s' % (repr(k), repr(self[k])) for k in self)
+
class MultiValueDictKeyError(KeyError):
pass
signature.asc
Description: Digital signature
Re: SortedDict.__repr__
On Sun, Apr 08, 2007 at 12:16:00PM +0800, Russell Keith-Magee wrote: > On 4/7/07, Forest Bond <[EMAIL PROTECTED]> wrote: > > > Attached is a patch that re-implements __repr__ using an iterator over the > > object. Thus, SortedDict's are displayed properly in the Python shell. > > Hi Forest, > > The best way to make sure this isn't forgotten is to get it into the > ticket system (http://code.djangoproject.com). Search the ticket > archive to see if this problem has been reported before; if it hasn't, > open a new ticket. Then attach your patch to the ticket. Ok, sorry and thanks. http://code.djangoproject.com/attachment/ticket/3964 -Forest signature.asc Description: Digital signature
Re: Streaming HttpResponse revisted. Any core devs, please take a look if you can :)
Hi Tai, On Mon, Aug 20, 2012 at 08:39:07AM -0700, Tai Lee wrote: > I'd like to re-visit the discussion surrounding #7581 [1], a ticket about > streaming responses that is getting quite long in the tooth now, which Jacob > finally "accepted" 11 months ago (after a long time as DDN) and said that it > is > clear we have to do *something*, but *what* remains to be seen. Since the last time this came up I've wondered if the simplest solution might be to make it easier to integrate raw WSGI apps into a site. I still am not 100% sure it's the right approach but I thought it was worth mentioning. The argument for this is that the use cases for streaming responses are usually corner cases that are well outside of Django's sweet spot. They go against the grain of the Django's flexible request/response handling (middleware in particular). But they would be trivial to handle in a raw WSGI app. If Django made it easy to hand off requests to a WSGI app (i.e. mount the app somewhere in the URLconf and perhaps expose authentication and/or other session data) the framework could get out of the way and let the WSGI app have more direct control over response handling. I think there are probably other scenarios where easy WSGI integration would be beneficial, too. For what it's worth... Thanks, Forest -- Forest Bond http://www.alittletooquiet.net http://www.rapidrollout.com signature.asc Description: Digital signature
[RFC] HttpResponse: freeze API, read_once
= 'gzip' response['Content-Length'] = str(len(response.content)) return response Shortcomings The biggest problem with the approach I've proposed is that it is not sufficiently granular to handle the case where I don't want the response content modified (e.g. compressed) but I *do* want to return HTTP 304 Not Modified where appropriate. That's the only use case I can think of right now that is not addressed by my proposal. I have a few ideas about this but I'm curious to see what ideas other people have and I think it can be addressed later. I am more than happy to provide an implementation of the above, but I'd like to get an idea of what chance it has of being accepted before I start working on it. I am also happy to maintain it outside of Django svn for a while so that people can test it out. Feedback would be appreciated. Thanks, Forest -- Forest Bond http://www.alittletooquiet.net http://www.pytagsfs.org signature.asc Description: Digital signature
Re: [RFC] HttpResponse: freeze API, read_once
Hi Santiago,
On Mon, Sep 13, 2010 at 03:58:12PM -0300, Santiago Perez wrote:
> The biggest problem with the approach I've proposed is that it is not
> sufficiently granular to handle the case where I don't want the response
> content
> modified (e.g. compressed) but I *do* want to return HTTP 304 Not Modified
> where
> appropriate. That's the only use case I can think of right now that is
> not
> addressed by my proposal. I have a few ideas about this but I'm curious
> to
> see
> what ideas other people have and I think it can be addressed later.
>
>
> Another missing use case is when you have streaming content that you don't
> want
> to load in memory but you still want it to be modified (compressed for
> instance) in ways that can be done during the streaming.
Good point.
But it can be added fairly easily if we also stipulate that with iterator
content the iterator itself is available as attribute "content_iterator".
def compress_content(response):
if response.header_is_frozen('Content-Encoding') \
or response.header_is_frozen('Content-Length') \
or response.content_is_frozen():
return response
if response.read_once:
response.content_iterator =
compress_iterator(response.content_iterator)
# We cannot know the response length.
if 'Content-Length' in response:
del response['Content-Length']
else:
response.content = compress_string(response.content)
response['Content-Length'] = str(len(response.content))
response['Content-Encoding'] = 'gzip'
return response
Thanks,
Forest
--
Forest Bond
http://www.alittletooquiet.net
http://www.pytagsfs.org
signature.asc
Description: Digital signature
Tickets #12979, #11557
Hi, Does anyone with commit access have a few minutes to look over the following tickets? * http://code.djangoproject.com/ticket/12979 * http://code.djangoproject.com/ticket/11557 Thanks, Forest -- Forest Bond http://www.alittletooquiet.net http://www.pytagsfs.org signature.asc Description: Digital signature
Re: [RFC] HttpResponse: freeze API, read_once
Hi Luke, On Tue, Sep 14, 2010 at 01:22:37AM +0100, Luke Plant wrote: > you wrote: > > > > I'd like to return to the problem of middleware, view decorators, etc. > > affecting > > responses in negative ways. > > Thanks so much for all your hard work and persistence on this issue. Thanks. Persistence is one of my few good traits. ;) > Although I haven't gone through this in detail, I think this is arriving > at a good solution now. I massively prefer the explicit 'freeze' API on > HttpResponse to subclasses that do different things. It's no silver bullet, but it's probably good enough to cover the majority of use cases without making things complicated. I'm curious to hear what people think. > It would be extremely useful to me if you could go through the use cases > linked on the previous thread and see if your solution addressed them > all. I'll get an e-mail out tomorrow with a good overview of each use case I know of and how well the freeze API would deal with them. > > I am more than happy to provide an implementation of the above, but I'd > > like to > > get an idea of what chance it has of being accepted before I start working > > on > > it. I am also happy to maintain it outside of Django svn for a while so > > that > > people can test it out. > > From my perspective that would be great. One thing that would be > brilliant whatever the eventual solution would be a set of use cases > encoded into some tests - I suspect that whatever solution/API is chosen > it would be worth doing the tests first so that you can see if it is > going to work in practice, and so that other people can see how to use > it. That's a great idea. Probably step one then would be to write up some test cases and make sure we all agree about those. Once I hear what people have to say about the proposed API, I'll start on that. Thanks, Forest -- Forest Bond http://www.alittletooquiet.net http://www.pytagsfs.org signature.asc Description: Digital signature
Re: [RFC] HttpResponse: freeze API, read_once
Hi,
I was re-reading this and wanted to clarify a few points.
On Mon, Sep 13, 2010 at 01:34:55PM -0400, Forest Bond wrote:
> Response Freezing
> =
>
> To address #1 and #2, I'd like to propose a response "freezing" API. The
> basic
> idea is that I can freeze a particular header expect that that header will not
> be modified by middleware or view decorators. I can also freeze the response
> content to indicate that the content should not be modified.
>
> The relevant functions would be:
>
> * HttpResponse.freeze_header(header)
> * HttpResponse.freeze_all_headers()
I actually don't think freeze_all_headers is necessary.
> Iterator Protection
> ===
>
> To address #3, we need a way to protect responses with iterator content from
> being drained prematurely. For this, I propose a boolean attribute on
> HttpResponse instances called "read_once". This will default to False to
> preserve backwards compatibility, but if it is set to True, an exception
> should
> be raised when the content is read. Middleware can inspect response.read_once
> to see if it is okay to read the response content.
I just realized that this paragraph is inconsistent with my compress_content
example below, which wouldn't work if an exception is raised when the content is
read.
Perhaps we can simply say that if read_once is True, middleware can access the
response content via attribute content_iterator only. I'd like to raise an
exception if it is accessed directly via attribute content just so we can weed
out the common case where a middleware has not been updated to check read_once.
> Examples
>
>
> def compress_content(response):
> # Note that we don't need to check response.read_once here since we are
> # replacing the content with a non-iterator. If the user didn't want us
> # to do this, he would have frozen the content.
> if response.header_is_frozen('Content-Encoding') \
> or response.header_is_frozen('Content-Length') \
> or response.content_is_frozen():
> return response
> response.content = compress_string(response.content)
> response['Content-Encoding'] = 'gzip'
> response['Content-Length'] = str(len(response.content))
> return response
Thanks,
Forest
--
Forest Bond
http://www.alittletooquiet.net
http://www.pytagsfs.org
signature.asc
Description: Digital signature
Re: [RFC] HttpResponse: freeze API, read_once
Hi, On Tue, Sep 14, 2010 at 01:22:37AM +0100, Luke Plant wrote: > It would be extremely useful to me if you could go through the use cases > linked on the previous thread and see if your solution addressed them > all. Sorry, I'll have to get this out tomorrow. I ended up being a little short on time today. For what it's worth, looking at the use cases again has been instructive and I think the proposed API can be simplified some and still be able to address each of them. Thanks, Forest -- Forest Bond http://www.alittletooquiet.net http://www.pytagsfs.org signature.asc Description: Digital signature
[RFC] HttpResponse: streaming, freeze API (v2)
http_date(last_modified)
# Make sure our ETag and Last-Modified headers stay the same.
response.freeze_header('ETag')
response.freeze_header('Last-Modified')
# Prevent compression.
response.freeze_header('Content-Encoding')
# Prevent caching.
response['Cache-Control'] = 'no-cache'
response.freeze_header('Cache-Control')
return response
With Compression
I know that some people want to stream large files and have them compressed
on-the-fly, so in that case:
* If we know Last-Modified and ETag values, they can be provided.
* Content-Length is unknown.
* Content should be compressed.
The same caveats as in the previous example apply to caching, Last-Modified, and
ETag.
View code::
# Assume content_type and last_modified come from the storage service.
response = HttpResponse(content = iterator, content_type = content_type)
response.streaming = True
response['ETag'] = etag
response['Last-Modified'] = http_date(last_modified)
# Make sure our ETag and Last-Modified headers stay the same.
response.freeze_header('ETag')
response.freeze_header('Last-Modified')
# Prevent caching.
response['Cache-Control'] = 'no-cache'
response.freeze_header('Cache-Control')
return response
Streaming Long Responses Without Timing Out
---
Parameters:
* Content is an iterator that must not be converted into a string because that
would cause the HTTP connection to timeout.
* Content-Length is unknown.
* Compression is undesirable since rechunking could result in longer delays
between chunks, possibly leading to a timeout.
This is the only use case I've seen that can really benefit from content
freezing, just in case some middleware would wrap the content iterator in such a
way as to reduce the frequency with which content chunks are sent.
View code::
response = HttpResponse(content = iterator)
response.streaming = True
response.freeze_content()
return response
But in practice, the most likely source of content modification for a streaming
response is middleware implementing compression, so this would probably work
just as well.
View code::
response = HttpResponse(content = iterator)
response.streaming = True
# Prevent compression.
response.freeze_header('Content-Encoding')
return response
In either case, the response will be cached (although note that the cache
middleware will need to be modified to support caching streaming responses as
discussed above).
Are there use cases I've missed?
Thanks,
Forest
--
Forest Bond
http://www.alittletooquiet.net
http://www.pytagsfs.org
signature.asc
Description: Digital signature
Re: HttpResponse: freeze API, read_once
Hi Tai,
On Tue, Sep 14, 2010 at 09:25:36PM -0700, Tai Lee wrote:
> I'm going to try and preempt a possible objection that has been raised
> in previous discussions on this subject.
>
> Won't this change require a lot of repetitive logic to be shifted into
> middleware functions? All middleware authors will now need to inspect
> the response and find out if they can make the changes they want to
> make.
Middleware will have to be changed to support streaming responses, yes. But the
checks that middleware have to do on the response are fairly trivial.
> If an old middleware hasn't been updated and a developer tries to
> freeze a header that the old middleware needs to alter, an exception
> will be raised. The only option for the developer in this case would
> be to alter the old 3rd party middleware themselves?
Or they can fix it by subclassing::
class FixedMiddleware(BrokenMiddleware):
def process_response(self, request, response):
if response.header_is_frozen('X-My-Header'):
return response
return super(FixedMiddleware, self).process_response(request,
response)
That's not a huge penalty for someone that really need streaming responses, I
wouldn't think. It also wouldn't take that much for the author of the 3rd party
middleware to implement the fix properly.
Thanks,
Forest
--
Forest Bond
http://www.alittletooquiet.net
http://www.pytagsfs.org
signature.asc
Description: Digital signature
Re: [RFC] HttpResponse: streaming, freeze API (v2)
Hi Santiago, On Wed, Sep 15, 2010 at 04:34:42PM -0300, Santiago Perez wrote: > In the cases where you want to prevent server-side caching but are ok > with client-side caching, couldn't you freeze the 'Cache-Control' > header to another value and set all necessary headers to get the > caching you want? Would that work? I guess Cache-Control: private would mostly work, but even then proxy caches would not cache the response. It'd still be an improvement. Thanks, Forest -- Forest Bond http://www.alittletooquiet.net http://www.pytagsfs.org signature.asc Description: Digital signature
Re: [ANN] Security releases and Django 1.3 beta 1
Hi, On Wed, Dec 22, 2010 at 10:40:06PM -0600, James Bennett wrote: > Tonight we've released Django 1.3 beta 1, as well as Django 1.2.4 and > Django 1.1.3 to correct a pair of security issues. FYI, the new releases have not been tagged in svn. Thanks, Forest -- Forest Bond http://www.alittletooquiet.net http://www.pytagsfs.org signature.asc Description: Digital signature
Re: Single Table Inheritance
Hi Jordan, On Tue, Mar 29, 2011 at 08:40:28AM -0700, Jordan MacDonald wrote: > I'm sure this subject has been beaten to death, but I haven't found an > answer to a simple scenario and I'm wondering why this hasn't been > addressed before. > > I have three models, structured like so: > > Document > -Presentation > -Spreadsheet > > Document is never instantiated on its own; a prime candidate for an > abstract base class. However, there are times where I want to list/ > search across all documents, and I'd like to be able to write > Document.objects.all(). I'd then like to be able to iterate over this > enumerable and have each object cast to its proper class. > > This is something accomplished with single table inheritance in Rails; > why don't we have the equivalent in Django? > > I know I could just use the Document class and have a type field, but > then I have to do all of the type checking legwork manually. I was > hoping Django would handle normalizing/denormalizing as part of the > ORM. In essence, creating its own type field automatically in the back- > end and casting each object to the appropriate class based on the > string value in this field. > > Does anyone know why this isn't available? Is there an equally > efficient method of modeling this approach of which I am unaware? I haven't used it, but maybe you should look into django_polymorphic: http://bserve.webhop.org/django_polymorphic/ Thanks, Forest -- Forest Bond http://www.alittletooquiet.net http://www.pytagsfs.org signature.asc Description: Digital signature
Re: RFC: Django 1.0 roadmap and timeline
Hi, On Jun 11, 10:03 pm, "Jacob Kaplan-Moss" <[EMAIL PROTECTED]> wrote: > * Must-haves: features that, if not completed, are worth delaying the > release. That is, if the work on this list is not completed by a > release date, we'll push the date. I think that this is a must-have: #285 -- WSGI SCRIPT_NAME and PATH_INFO stuff Incidentally, so did Adrian. The text is copied from his post with subject "Django 1.0 features -- the definitive list". http://groups.google.com/group/django-developers/browse_thread/thread/b4c237ad76f9eeca It's in important change and is backwards-incompatible. Thanks, Forest --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: RFC: Django 1.0 roadmap and timeline
On Jun 12, 8:51 am, "Marty Alchin" <[EMAIL PROTECTED]> wrote: > On Thu, Jun 12, 2008 at 7:06 AM, Forest Bond <[EMAIL PROTECTED]> wrote: > > I think that this is a must-have: > > > #285 -- WSGI SCRIPT_NAME and PATH_INFO stuff > > Then you'll be glad to know that it's #3 the list of "Must-have > features" in Jacob's email, just a bit below the portion you quoted. Indeed. Don't know how, but I mis-read that. Sorry. -Forest --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Filter on is-a test
Hi, I have a simple implementation that allows filtering based on whether or not a particular row represents an instance of a particular model class. This is only useful with concrete model inheritance. For example: class Transaction(Model): ... class Invoice(Transaction): ... class Payment(Transaction): ... # Assume for the sake of example that I define several other Transaction # sub-classes and that I have data in the database to query. invoice_transactions = Transaction.objects.isa(Invoice) payment_transactions = Transaction.objects.isa(Payment) invoices_and_payments = invoice_transactions | payment_transactions Is this useful to others? I don't love the syntax. I think the ideal implementation would provide access to this through a filter keyword argument rather than a special manager/queryset method: Transaction.objects.filter(__isa = Invoice) That would allow more complex filtering (although I don't have an implementation for this yet): Transaction.objects.filter(related_model__isa = Foo) Is there any opposition to this kind of feature? If it seems worth having, I'll go ahead an create a ticket, and eventually attach a patch (if we can agree on what the syntax should look like). Thanks, Forest -- Forest Bond http://www.alittletooquiet.net http://www.pytagsfs.org signature.asc Description: Digital signature
Re: Filter on is-a test
Hi Russell, On Mon, Jul 27, 2009 at 08:05:45AM +0800, Russell Keith-Magee wrote: > On Sun, Jul 26, 2009 at 10:22 PM, Forest Bond > wrote: > > Hi, > > > > I have a simple implementation that allows filtering based on whether or > > not a > > particular row represents an instance of a particular model class. This is > > only > > useful with concrete model inheritance. > > > > For example: > > > > class Transaction(Model): > > ... > > > > class Invoice(Transaction): > > ... > > > > class Payment(Transaction): > > ... > > > > # Assume for the sake of example that I define several other Transaction > > # sub-classes and that I have data in the database to query. > > > > invoice_transactions = Transaction.objects.isa(Invoice) > > payment_transactions = Transaction.objects.isa(Payment) > > > > invoices_and_payments = invoice_transactions | payment_transactions > > > > Is this useful to others? > > > > I don't love the syntax. I think the ideal implementation would provide > > access > > to this through a filter keyword argument rather than a special > > manager/queryset > > method: > > > > Transaction.objects.filter(__isa = Invoice) > > > > That would allow more complex filtering (although I don't have an > > implementation > > for this yet): > > > > Transaction.objects.filter(related_model__isa = Foo) > > > > Is there any opposition to this kind of feature? If it seems worth having, > > I'll > > go ahead an create a ticket, and eventually attach a patch (if we can agree > > on > > what the syntax should look like). > > I don't have any objection to the feature in principle. I can think of > a few places in my own code where filtering on subclass could be > useful. > > However, before we start arguments on API syntax, the part of your > proposal that is missing is how this will translate as SQL. Any ideas > how this would work? How the joins/where clauses introduced by the isa > operator would interact with other joins? This is a very crude implementation that I'm using right now. It only works for a single layer of inheritance. class IsaQuerySet(QuerySet): def isa(self, other_model): clone = self._clone() if other_model is self.model: return clone other_table = other_model._meta.db_table self_column = self.model._meta.pk.name other_column = other_model._meta.parents[self.model].column self_alias = clone.query.get_initial_alias() other_alias = clone.query.join( (self_alias, other_table, self_column, other_column), promote = True, ) clone.query.where.add( (other_alias, other_column, None, 'isnull', None), 'AND', ) return clone Basically, we outer join the sub-class's table and filter out rows where the join gives NULL pk's for that table. I think that is sufficient for this simple case. I haven't spent a lot of time thinking about the more complex cases (more than one level of inheritance, filtering across relations), but I don't think it would be much more complicated. -Forest -- Forest Bond http://www.alittletooquiet.net http://www.pytagsfs.org signature.asc Description: Digital signature
Re: Filter on is-a test
Hi Alex, On Sun, Jul 26, 2009 at 08:42:26PM -0400, Alex Gaynor wrote: > On Sun, Jul 26, 2009 at 8:05 PM, Russell > Keith-Magee wrote: > > > > On Sun, Jul 26, 2009 at 10:22 PM, Forest Bond > > wrote: > >> Hi, > >> > >> I have a simple implementation that allows filtering based on whether or > >> not a > >> particular row represents an instance of a particular model class. This > >> is only > >> useful with concrete model inheritance. > >> > >> For example: > >> > >> class Transaction(Model): > >> ... > >> > >> class Invoice(Transaction): > >> ... > >> > >> class Payment(Transaction): > >> ... > >> > >> # Assume for the sake of example that I define several other Transaction > >> # sub-classes and that I have data in the database to query. > >> > >> invoice_transactions = Transaction.objects.isa(Invoice) > >> payment_transactions = Transaction.objects.isa(Payment) > >> > >> invoices_and_payments = invoice_transactions | payment_transactions > >> > >> Is this useful to others? > >> > >> I don't love the syntax. I think the ideal implementation would provide > >> access > >> to this through a filter keyword argument rather than a special > >> manager/queryset > >> method: > >> > >> Transaction.objects.filter(__isa = Invoice) > >> > >> That would allow more complex filtering (although I don't have an > >> implementation > >> for this yet): > >> > >> Transaction.objects.filter(related_model__isa = Foo) > >> > >> Is there any opposition to this kind of feature? If it seems worth > >> having, I'll > >> go ahead an create a ticket, and eventually attach a patch (if we can > >> agree on > >> what the syntax should look like). > > > > I don't have any objection to the feature in principle. I can think of > > a few places in my own code where filtering on subclass could be > > useful. > > > > However, before we start arguments on API syntax, the part of your > > proposal that is missing is how this will translate as SQL. Any ideas > > how this would work? How the joins/where clauses introduced by the isa > > operator would interact with other joins? > > > > Yours, > > Russ Magee %-) > > > > > > > > > As far as I'm aware this is already possible by querying on the > implicit reverse 1-1 relation that exists. For example: > > Place.objects.exclude(restaurant__isnull=True) > > would remove all instances of Places who aren't also restaurants. The problem with this is that I have to know all of the existing sub-classes that I don't want. These aren't necessarily exclusively defined by a single app, so adding a new app in the site settings could break existing apps. It's also a little awkward, IMO. -Forest -- Forest Bond http://www.alittletooquiet.net http://www.pytagsfs.org signature.asc Description: Digital signature
Re: Filter on is-a test
Hi Honza, On Mon, Jul 27, 2009 at 01:59:39AM +0200, Honza Král wrote: > you can already do this yourself - just add a FK to ContentType to > Transaction and in .save() method assign it the correct value: > > self.content_type = ContentType.objects.get_for_model(self) > > Then you can easily do this query yourself without any need for Django > to know about your inheritance patterns. (and burdening people who > don't care about this with the tiny overhead). I did think about this style of implementation, but it seemed redundant to have a column store the type of transaction when it is already in the database by way of having a corresponding row in the sub-class's table. -Forest -- Forest Bond http://www.alittletooquiet.net http://www.pytagsfs.org signature.asc Description: Digital signature
Re: Filter on is-a test
Hi, On Mon, Jul 27, 2009 at 04:50:17PM +0200, Honza Král wrote: > On Mon, Jul 27, 2009 at 5:13 AM, Forest Bond > wrote: > > Hi Honza, > > > > On Mon, Jul 27, 2009 at 01:59:39AM +0200, Honza Král wrote: > >> you can already do this yourself - just add a FK to ContentType to > >> Transaction and in .save() method assign it the correct value: > >> > >> self.content_type = ContentType.objects.get_for_model(self) > >> > >> Then you can easily do this query yourself without any need for Django > >> to know about your inheritance patterns. (and burdening people who > >> don't care about this with the tiny overhead). > > > > I did think about this style of implementation, but it seemed redundant to > > have > > a column store the type of transaction when it is already in the database by > > way of having a corresponding row in the sub-class's table. > > Yes, it is denormalization of the data already present in the > database. But the amount of OUTER JOINS you would have to do in some > cases make the proper normalized approach very slow and generally not > nice. I think it's a single OUTER JOIN for each is-a test. That doesn't seem excessive to me. Am I not thinking about this correctly? Thanks, Forest -- Forest Bond http://www.alittletooquiet.net http://www.pytagsfs.org signature.asc Description: Digital signature
Re: Filter on is-a test
Hi, On Mon, Jul 27, 2009 at 08:05:45AM +0800, Russell Keith-Magee wrote: > On Sun, Jul 26, 2009 at 10:22 PM, Forest Bond > wrote: > > invoice_transactions = Transaction.objects.isa(Invoice) > > payment_transactions = Transaction.objects.isa(Payment) > > > > invoices_and_payments = invoice_transactions | payment_transactions > > However, before we start arguments on API syntax, the part of your > proposal that is missing is how this will translate as SQL. Any ideas > how this would work? How the joins/where clauses introduced by the isa > operator would interact with other joins? Regarding syntax: I thought back to what sent me down the explicit-is-a path, and I remembered that the first thing I tried was this: Transaction.objects.filter(payment__id__isnull = False) I was hoping that this would produce SQL similar to this simplified query: SELECT * FROM transaction LEFT OUTER JOIN payment ON payment.id = transaction.id WHERE payment.id IS NOT NULL But instead Django delivered the decidedly unhelpful: SELECT * FROM transaction WHERE id IS NOT NULL Which, of course, gave me the full set of transactions. Django appears to collapse the sub-class relationship in the where clause. There may be other scenarios where this behavior is unexpected. [I also recall that Django permitted creation of conflicting related object accessors. I had a ForeignKey(Payment) on the Invoice model, so Transaction instances had a payment accessor that could have referred to either Transaction->Invoice->ForeignKey(Payment) or Transaction->Payment. This is a bug, I guess; I'll create a ticket when I have time.] Perhaps the best thing to do would be to simply make the payment__id__isnull syntax work as I had originally expected. Thoughts? Thanks, Forest -- Forest Bond http://www.alittletooquiet.net http://www.pytagsfs.org signature.asc Description: Digital signature
Re: HTTP Errors
Hi, On Mon, Jul 27, 2009 at 03:10:59PM +0200, Jonas Obrist wrote: > I don't quite understand why I can raise a Http404 but not for example a > Http403. > > I think one should be able to either raise all Http error codes (4xx, > 5xx) or none. And I'd prefer being able to raise them as Exceptions This has come up before (I think several times). I used to be in the exceptions-for-any-error-response camp, but now I actually wonder why Http404 needs to be a special case at all. I think I'd be perfectly happy just returning HttpResponseNotFound. I eventually came to realize that the need to raise exceptions to trigger responses may indicate suboptimal layering in my view functions. It probably means that some helper function is confused and thinks that it is a view function (in fact, the code in the helper function probably got ripped out of a view function). Keeping HTTP-related logic in the view function itself alleviates the need for a Http403 exception. -Forest -- Forest Bond http://www.alittletooquiet.net http://www.pytagsfs.org signature.asc Description: Digital signature
Re: HTTP Errors
Hi, On Mon, Jul 27, 2009 at 09:50:31PM +0400, Ivan Sagalaev wrote: > > Marc Fargas wrote: > > You could write an Exception middleware that takes care of processing > > your "special" exception. Bu anyway, note that you are willing is to > > give "response" so the right thing to do would be returning a response > > object > > The latter is not very convenient. Often the code that makes a decision > about forbidding something is buried deep into model layer and can't > easily return an HttpResponse. So I believe that making a special > exception and converting it into an HttpResponseForbidden in a > middleware is a good way to do this in current Django code. But I can > really see the value of making it a part of Django even if just for the > sake of consistency with Http404. BTW other HTTP errors in my practice > don't work by this pattern so I don't propose making an exception for > everything >= 400. But your model code shouldn't have HTTP-related bits buried in it. That's a layering violation. That code should raise a more generic exception that your view code converts to an HTTP response. -Forest -- Forest Bond http://www.alittletooquiet.net http://www.pytagsfs.org signature.asc Description: Digital signature
Re: Deleting an object deletes objects with nullable foreignkeys pointing at it
Hi, On Mon, Jul 27, 2009 at 12:34:22PM -0700, Stephen Sundell wrote: > I have revision 10865, and use postgresql. I've noticed that with > models like this: > > class Edition(models.Model): > name = models.CharField(max_length=25) > > class Email(models.Model) > edition = models.ForeignKey(Edition,null=True,blank=True) > > If I delete an edition (edition.delete()), it deletes all the emails > associated with it. I thought it was just supposed to set the foreign > key to null. Am I wrong in my thinking or doing something incorrect? You must set the foreign key fields to NULL manually before deleting. -Forest -- Forest Bond http://www.alittletooquiet.net http://www.pytagsfs.org signature.asc Description: Digital signature
Re: Proposal: post_create signal
On Thu, May 31, 2007 at 09:16:07AM +1200, Robert Coup wrote: > Ben Schwarze wrote: > > Due to this process, I was looking for a way fetch a signal after the > > creation of an object. > > Currently Django has the pre_save and post_save signals that will be > > send before and after saving the object. But there seems to be no way > > to determine whether an object has been just created or updated. > > > Why not connect to pre_save and check whether the primary key is None, > similar to the accepted way to check for creation if you override > Model.save()? The most obvious reason, of course, is that you don't know at that point whether the save is successful or not. -Forest signature.asc Description: Digital signature
Should regroup return a QuerySet?
Hi,
Currently, the regroup template tag returns a list of dicts that look something
like this:
[
{'grouper': 'thegrouper', 'list': [obj1, obj2, obj3]},
...
]
This is fine & correct behavior, and the tag works for both QuerySet's and lists
as inputs. However, if a QuerySet is passed in, it would be nice to get a
QuerySet out as the 'list' value. That way, custom methods on the QuerySet
could still be used in the template. For instance, in my particular
application, I would like to be able to do this:
{% regroup daily_usages by extension as daily_usages_by_extension %}
{% for daily_usages_for_extension in daily_usages_by_extension %}
{% with daily_usages_for_extension.list.usage_summary as extension_summary %}
{{daily_usages_for_extension.grouper}}
{{extension_summary.incoming_quantity}}
{{extension_summary.incoming_duration}}
{{extension_summary.outgoing_quantity}}
{{extension_summary.outgoing_duration}}
{{extension_summary.other_quantity}}
{% endwith %}
{% endfor %}
I can temporarily overcome this obstacle by using another custom QuerySet
method:
{% for usages_for_extension in daily_usages.group_by_extension %}
{% with usages_for_extension.list.usage_summary as extension_summary %}
{{usages_for_extension.grouper}}
{{extension_summary.incoming_quantity}}
{{extension_summary.incoming_duration}}
{{extension_summary.outgoing_quantity}}
{{extension_summary.outgoing_duration}}
{{extension_summary.other_quantity}}
{% endwith %}
{% endfor %}
Clearly, however, the more general solutions to this would be more ideal. It
seems like other folks might also enjoy this feature?
(It occurs to me now that I could have written an alternative implementation of
the regroup template tag (instead of using my custom QuerySet method).)
What do people think about this?
-Forest
signature.asc
Description: Digital signature
Re: Should regroup return a QuerySet?
On Thu, Jun 07, 2007 at 10:38:52AM -0400, Forest Bond wrote:
> Hi,
>
> Currently, the regroup template tag returns a list of dicts that look
> something
> like this:
>
> [
> {'grouper': 'thegrouper', 'list': [obj1, obj2, obj3]},
> ...
> ]
>
> This is fine & correct behavior, and the tag works for both QuerySet's and
> lists
> as inputs. However, if a QuerySet is passed in, it would be nice to get a
> QuerySet out as the 'list' value. That way, custom methods on the QuerySet
> could still be used in the template.
It also occurs to me that if regroup handled QuerySets as a special case,
large-ish datasets could be regrouped in a much more efficient manner, since the
filter method could be used instead of iterating and testing for equality.
-Forest
signature.asc
Description: Digital signature
Re: Should regroup return a QuerySet?
On Thu, Jun 07, 2007 at 01:54:18PM -0700, Ilya Semenov wrote:
>
> > It also occurs to me that if regroup handled QuerySets as a special case,
> > large-ish datasets could be regrouped in a much more efficient manner,
> > since the
> > filter method could be used instead of iterating and testing for equality.
>
> First, there will be no perfomance gain at all.
>
> Consider the current regroup running at most 1 query:
> SELECT * FROM app_model;
>
> The proposed behaviour would run 1+N queries, first for getting
> different grouping values:
> SELECT DISTINCT(grouper_field) FROM app_model;
> and then a separate query for each of the grouping values:
> SELECT * FROM app_model WHERE grouper_field = 'grouping_value_1';
> SELECT * FROM app_model WHERE grouper_field = 'grouping_value_2';
> ...
> SELECT * FROM app_model WHERE grouper_field = 'grouping_value_N';
> In most conditions that would give more overhead than iterating and
> testing for equality in Python.
I'm not convinced this is overwhelmingly true, although it may be true for many
of the common use cases. Certainly, in some significant number of cases (large
data set, small number of grouping values), the database would be quicker, no?
The performance comparision is highly dependent on the particulars of the setup,
though. If the database connection is slow, you are certainly right. If not,
do you think that perhaps it might be close? Isn't it more common for the
database connection to be fast, rather than slow?
There is an obvious, although not incredibly common, use case in which the
current implementation would be decidedly slower: when the grouped lists are
then sliced (assuming a mildly large dataset):
{% for object in grouped.list|slice:"0:10" %}
{% endfor %}
This is more performant since it leverages the laziness of QuerySets, and it
might represent a reasonably common situation: "Show the user a list of the top
10 items for each of the following grouper fields."
Of course, most current implementations of that use case are probably explicitly
making N different queries prior to template evaluation. It's certainly
possible that doing the grouping at template-time is not something many people
would choose to do.
> Second, in multi-threaded environment that would possibly lead to
> consitency failures. Consider a concurrent thread changing
> grouper_field for some entities. That could lead to the same entity
> being shown up twice in two groups.
Hmm. Good point. I hadn't thought of it. In my application, the field I'm
grouping by does not change once it has been set, so it's a non-issue.
Your second point above confirms for me that the standard regroup tag should not
handle QuerySets as special cases. However, it does strike me as useful to have
the QuerySet-aware variant around, caveats and all. I suppose it may not be
useful enough for core. Any opinion on that?
-Forest
signature.asc
Description: Digital signature
Re: Should regroup return a QuerySet?
On Thu, Jun 07, 2007 at 04:11:41PM -0500, Adrian Holovaty wrote: > Aside from these very good reasons, it wouldn't be a good idea to > couple the template system to the database layer. The template system > shouldn't know anything about QuerySets, or Models, or Managers, or > any other Django-model-specific phenomena. Ok, I definately agree here. (Side thought: is it really that bad for the template system to leverage QuerySet functionality without requiring it? It's obviously not as bad as depending on it.) I just sent another email in which I ponder the utility of having a second tag, similar to regroup, but explicitly designed to handle QuerySets. What about django.db having a templatetags.py? :) Or maybe django.contrib.dbtemplatetags -Forest signature.asc Description: Digital signature
Re: Shared memory across processes
On Wed, Jun 27, 2007 at 02:41:15PM -, Donny wrote: > > On Jun 26, 11:28 am, "Marty Alchin" <[EMAIL PROTECTED]> wrote: > > The biggest hurdle to dbsettings at this point is that it caches > > settings in a standard Python module, which only exists within a > > single process. This was all well and good while developing it, but > > people are now starting to try it under mod_python, which (depending > > on the Apache MPM) creates multiple processes, each with their own > > dbsettings cache. This is fine except when editing settings, since it > > can only update the cache in the current process. > > Stop me if you've heard this: > > This is not a bug, it's a feature. > > Really, though, that's what it is. mod_python is chosen for deployment > environments where a person does _NOT_ want a check to occur for > updates because it impacts performance, instead choosing to push > updates to the server explicitly. An easy way to do this for > developers is a command like this: > > apache[2]ctl graceful|restart > > Do NOT do your development under mod_python! That is _NOT_ what it's > intended for. I think you've misunderstood Marty. He is not talking about modifying settings.py, he is talking about his dbsettings django app, which stores some parameters in the database. These parameters are intended to be changed by non-developers, and shouldn't require an apache restart to take effect. -Forest signature.asc Description: Digital signature
mod_python: Why use req.uri, not req.path_info?
Hi, I was trying to set up a site at a URL that is not at the root of the web server. I am using mod_python, and my apache config looks similar to that included with the relevant django documentation, which I'll duplicate here: [from http://www.djangoproject.com/documentation/modpython/] SetHandler python-program PythonHandler django.core.handlers.modpython SetEnv DJANGO_SETTINGS_MODULE mysite.settings PythonDebug On That documentation correctly adds: "Restart Apache, and any request to /mysite/ or below will be served by Django. Note that Django’s URLconfs won’t trim the “/mysite/” — they get passed the full URL." So, the problem is that I have to update my urls.py to include the /mysite/ prefix in all URLs. This is clearly not very DRY. For me, the biggest pain is that I'm actually trying to deploy the same project at two different URLs (with different host parts, too). (I have reasons to do this, although they aren't incrediblly relevevant, so I won't go into that). Since my urls.py has to be so aware of my apache config, I can't do things the way I want to and use the apache config to enable & disable access to different application fragments under each base URL. All of this could be fixed by simply changing ModPythonRequest (django/core/handlers/modpython.py) to use req.path_info instead of req.uri. Is there any reason not to do that (aside from breaking backwards compatibility)? If a change like this is in order, I assume it will be made prior to 1.0. What do people think about this? I didn't get much from a Google search, but it seems hard to believe that it hasn't come up yet in the past. If this can be solved using some crazy middleware-type solution, I'd be open to that. It doesn't seem ideal to me for the current behavior to be the default, though. -Forest signature.asc Description: Digital signature
Re: mod_python: Why use req.uri, not req.path_info?
On Tue, Jul 10, 2007 at 04:13:44PM -0400, Max Battcher wrote:
>
> On 7/10/07, Forest Bond <[EMAIL PROTECTED]> wrote:
> > So, the problem is that I have to update my urls.py to include the /mysite/
> > prefix in all URLs.
> >
> > ...
> >
> > If this can be solved using some crazy middleware-type solution, I'd be
> > open to
> > that. It doesn't seem ideal to me for the current behavior to be the
> > default,
> > though.
>
> Fixing the URLs redundancy is easy, create a new mysite_urls.py and
> mysite_settings.py for the site at /mysite/. In mysite_settings put
> the ROOT_URLCONF to mysite_urls.py and mysite_urls just needs the
> simple url pattern ('^mysite/', include('urls.py')) to match the new
> "global" prefix. You can do the same for other sites or it would be
> easy enough to use a more generic prefix regex and share the same
> prefix matching urls and trade off a little regex efficiency... Each
> site should have its own settings anyway, at the very least for
> separate SITE_IDs. (Settings is just python, so you can easily
> inherit some base settings with a line like ``from base_settings
> import *``.)
Yes, that would work, and I've done similar things for other situations, but I'm
really just having a hard time seeing why django even cares about /mysite/ at
all.
The solution you mention here requires me to add two python source files to my
tree, and in the end, if I want to move my app to a different URL, I still have
to change both the apache configuration and mysite_urls.py. It just shouldn't
be that complicated, should it?
I figure I should just have to change the apache configuration, which is where
django instances are mapped to URLs. urls.py is where django views are mapped
to sub-urls of the main django instance, right?
-Forest
signature.asc
Description: Digital signature
Re: mod_python: Why use req.uri, not req.path_info?
On Tue, Jul 10, 2007 at 04:25:17PM -0400, Marty Alchin wrote: > > On 7/10/07, Forest Bond <[EMAIL PROTECTED]> wrote: > > All of this could be fixed by simply changing ModPythonRequest > > (django/core/handlers/modpython.py) to use req.path_info instead of req.uri. > > Is there any reason not to do that (aside from breaking backwards > > compatibility)? > > I'm certainly no expert in this area, but keep in mind that mod_python > is just one available option for deploying Django. There would have to > be some equivalent in the other situations, such as FastCGI, in order > for this to be taken seriously. Is this kind of change relevant for all of the other methods? I have not used any of the others. Certainly, if a change like this is made, the other handlers would need to be updated for consistency. I agree with you there. If there was some way to tell apache/mod_python to pass the URL sans the prefix in to django, I would simply do that. I have not been able to find any such thing. But really, the prefix is a deployment detail that I should be able to make my application agnostic to. I don't have to include my domain name in my urls.py, why should I have to include a prefix? Maybe this behavior could be determined by something in settings.py. -Forest signature.asc Description: Digital signature
Re: mod_python: Why use req.uri, not req.path_info?
On Tue, Jul 10, 2007 at 09:40:49PM -, [EMAIL PROTECTED] wrote: > On Jul 10, 5:02 pm, Forest Bond <[EMAIL PROTECTED]> wrote: > > Is this kind of change relevant for all of the other methods? I have not > > used > > any of the others. Certainly, if a change like this is made, the other > > handlers > > would need to be updated for consistency. I agree with you there. > > > > If there was some way to tell apache/mod_python to pass the URL sans the > > prefix > > in to django, I would simply do that. I have not been able to find any such > > thing. > > > > But really, the prefix is a deployment detail that I should be able to make > > my > > application agnostic to. I don't have to include my domain name in my > > urls.py, > > why should I have to include a prefix? > > > > Maybe this behavior could be determined by something in settings.py. > > > > -Forest > > > > signature.asc > > 1KDownload > > Your applications should be root url agnostic. Each app should have > its own urls.py and the root site urls.py (and its accompanying > settings.py) should have the site specific information including the > root. Then the apps are included in the site specific root urls.py Pardon me for mis-speaking. I meant that I felt my site-specific urls.py should be root URL agnostic. Actually, I am pushing for that URL prefix to only be mentioned in the apache config. While I appreciate your assistance in helping me do things how you would do them, I am really hoping someone can answer my question: why use req.uri instead of req.path_info? Is there a good reason? > You might want to check out this thread for more information: > http://groups.google.com/group/django-developers/browse_thread/thread/e9d92bceab679f6a/# A quick scan of that thread doesn't turn up much relevant info. What am I looking for there? > Here is an example of having the root urls.py get its bases from > settings.py (though not a very good one): > https://svn.python.org/conference/django/trunk/pycon/urls.py Thanks; I won't be needing further assistance with Python, however. -Forest signature.asc Description: Digital signature
Re: Patch vs checkin
On Wed, Jul 11, 2007 at 09:12:14AM -0500, Jacob Kaplan-Moss wrote: > Probably the best way to host an "unofficial" branch would be by using > Tom's Mercurial mirror of Django trunk: > http://hg.korpios.com/django.trunk/. Since you're on svn, you might also try svk. -Forest signature.asc Description: Digital signature
Re: Patch vs checkin
On Wed, Jul 11, 2007 at 09:43:15PM +0700, Ben Ford wrote: > I'm already happily using SVK, and was originally planning to use it to > remotely host the multi-db branch on my virtual server... However the only > info > I could find on achieving this was incorrect and I'm not sure if it is even > possible to split repos (onto 2 different machine), which is what I'd > originally planned... > I currently have all of the changes I've made to the branch hosted in an SVK > repo, but I have no way to share those changes with interested parties - > except > by way of patches, which seems inelegant! > Do you know of anyway to achieve this using SVK..? Well, I've actually only used svk very briefly to check it out, but I seem to remember that the model is something like this: * Mirror remote svn repository locally. * Make a branch of the mirror. * Make a checkout of the branch. * Make changes to the checkout. * Commit to the local branch. * Merge the local branch into the local mirror; this effectively pushes the changes to the remote svn repository. See here: http://svk.bestpractical.com/view/UsingSVKAsARepositoryMirroringSystem It's on of those svn-ish things that sounds incredibly impossible if you haven't done it before, but probably isn't all that bad. :) -Forest signature.asc Description: Digital signature
Re: mod_python: Why use req.uri, not req.path_info?
On Wed, Jul 11, 2007 at 09:27:43PM -, [EMAIL PROTECTED] wrote: > > This snippit replaces the request.path. My question is, how does this > work with sitemap, absolute rul's and reverse url lookups? doesn't it > break all of those, or am I missing something? More importantly (for me), this snippeet has no chance of solving my problem :) -Forest signature.asc Description: Digital signature
Re: mod_python: Why use req.uri, not req.path_info?
On Thu, Jul 12, 2007 at 08:45:33AM +0700, Yuri Baburov wrote:
> Can't do experiments now, but my admin interface (django trunk) began
> to work right after this middleware addition, that meant reverse url
> were resolved somehow.
> Snippet exactly change req.uri to req.script_name+req.uri, that gives
> req.path. Am I right? If I'm not, you can change it to set
> request.path to what you need.
Yeah, I think my urls.py situation would be resolved by setting request.path to
req.path_info, but that would break {% url %} and admin pages for me, I believe,
although I haven't put too much thought into it yet.
Really, as Malcolm & Graham pointed out, this is something that Django needs a
little help with...
Thanks for pointing out where the hack is best placed, though :)
-Forest
signature.asc
Description: Digital signature
Re: Patch vs checkin
On Sat, Jul 14, 2007 at 05:19:57PM -0700, Brian Harring wrote: > On Fri, Jul 13, 2007 at 11:28:27PM -0500, Gary Wilson wrote: > > > > Brian Harring wrote: > > > On Wed, Jul 11, 2007 at 09:38:54AM -0500, Deryck Hodge wrote: > > >> For example: > > >> > > >> mkdir repos > > >> cd repos > > >> bzr init > > >> svn co http://code.djangoproject.com/svn/django/trunk/ django_trunk > > >> bzr add django_trunk > > >> bzr commit > > >> > > >> Then you can bzr branch from there to have as many branches as needed. > > > Doing that makes merging a fairly major PITA: alternative- > > > > What do you mean by this? Merging from others' branches? Merging to > > svn repo? I've been using bazaar lately for my work on Django, but have > > only really used it on my local machine. > > Pardon the delay, been rather busy. If you're just importing vcs > trunk in as a single rev in bzr, you're losing data along the way- > specifically, losing each rev from the branch you're merging. This > limits the merging ability (and limits weave merging a bit), and makes > things a pita if you're trying to identify why something changed. > > Basically, keep the rev granularity- importing vcs head, and commiting > that as a rev creates problems long term; simplest example is that if > I try to merge your branch into mine, it can't easily tell which revs > are shared between our branches on the trunk imports. If it's the > only option you have, sure, go that route, but personally I've found > it winds up wasting my time :) I think Deryck is importing into bzr and then sending a patch back upstream. Ancestry is a non-issue for that use-case. -Forest signature.asc Description: Digital signature
Re: None != Null?
On Mon, Jul 16, 2007 at 09:24:07AM -0700, David Cramer wrote: > > Is there a specific reason that myfield=None doesn't translate to > myfield__isnull=True in the database backend? myfield = None translates into the SQL expression myfield = NULL. This is not really that useful, since that clause will always evaluate to false, but it is very correct. Transforming myfield = None to SQL myfield IS NULL seems a little magical to me. If that were done, though, I'd be in favor of dropping __isnull entirely, since it would no longer be uniquely useful. -Forest signature.asc Description: Digital signature
Re: None != Null?
On Tue, Jul 17, 2007 at 06:55:39AM -0400, Ned Batchelder wrote: > I'm in the camp that thinks making foo=None useful is a good thing. To the > python programmer who is our audience, this has a very clear semantic. > Especially when you take the scenario of using one object's values to create a > query for objects like it. Having to special case None seems arbitrary and > confusing. > > Michael, could you help us out by elaborating on what code you have that you > would have to re-write? __isnull would be deprecated but wouldn't go away. > With the current semantics, you couldn't have any foo=None uses in your code > (since it means nothing useful), so how could changing its meaning to > something > useful break your code? Well, I hate to make it even more of a special case, but would something like this make sense: foo__isnull = True foo IS NULL (deprecated) foo__exact = None foo = NULL foo = None foo IS NULL Or does that break consistency in an even worse way (since foo = x is no longer the same as foo__exact = x)? The only reason I suggest it is that it does keep "foo = NULL" in the realm of producable SQL expressions, while making the common case "foo = None" a little more intuitive for Python folks. It may not be that great an idea, either. -Forest -- Forest Bond http://www.alittletooquiet.net signature.asc Description: Digital signature
Re: I need fix "non proper sql server support" in the next 3 weeks. What is next?
On Tue, Jul 31, 2007 at 09:58:57PM -, mamcx wrote: > I have a requeriment to support Sql Server in a outsourced contract. > Is a must, a requeriment for delivery. So, I can't wait anymore for > free open source hacking... You know, I never could figure out why someone who is benefiting from an open-source project would write to that project's mailing list (with poor grammar, no less) and make a comment like this one. While you're here, you might as well try understanding this open-source thing. You clearly got part of it: if you want something done, and don't want to pay someone else to do it, do it yourself. It's the other part (which constitutes advice that Schwarzenegger himself might have offered) that you seem to have missed: stop whining and write some code. -Forest -- Forest Bond http://www.alittletooquiet.net signature.asc Description: Digital signature
Re: I need fix "non proper sql server support" in the next 3 weeks. What is next?
On Tue, Jul 31, 2007 at 09:09:41PM -0300, Julio Nobrega wrote: > He's asking how to contribute with code he's going to write. If you > weren't in such a hurry to give the standard zealot answer, perhaps > you could have seen it. I doubt very much the "standard zealot answer" would include reference to Schwarzenneger. :) None-the-less, I apologize for misinterpreting what appeared to me to be a sneering jab at open source, amounting to little more than biting the hand that feeds (independent of what was being offered). Sorry, mamcx. -Forest -- Forest Bond http://www.alittletooquiet.net signature.asc Description: Digital signature
Re: Ticket #3700: XhtmlDegraderMiddleware
On Wed, Aug 01, 2007 at 11:29:44AM -0500, Jacob Kaplan-Moss wrote: > Of course, what we really need is a CPAN-for-Django-bits-thingie... I > wish there was a middle ground between "included" and "not." Surely, you were thinking of Cheese Shop, right? :) Seriously, though, how many special-purpose package management systems do we need? -Forest -- Forest Bond http://www.alittletooquiet.net signature.asc Description: Digital signature
Re: How submit a patch with files not in subversion?
On Thu, Aug 02, 2007 at 01:38:00AM -, mamcx wrote: > The point is that I don't have write permissions on the subversion > tree... This tells subversion to start caring about file myfile.py: svn add myfile.py After you've done that, this will include myfile.py in your diff: svn diff You don't need write access to the subversion repository to do this. -Forest -- Forest Bond http://www.alittletooquiet.net signature.asc Description: Digital signature
Re: Autoescaping: good time?
On Fri, Aug 03, 2007 at 01:09:28PM -0500, Jacob Kaplan-Moss wrote: > On 8/3/07, Marty Alchin <[EMAIL PROTECTED]> wrote: > > I'd add .py to that list, if it gets done this way. > > Wait - are you using Django templates to generate *Python*? > > That's... just... *sick*. No, that's just meta-programming, bash-style ;) -Forest -- Forest Bond http://www.alittletooquiet.net signature.asc Description: Digital signature
Re: How much test case must print runtest?
On Mon, Aug 06, 2007 at 10:34:32AM -0600, Ian Kelly wrote: > > On 8/6/07, mamcx <[EMAIL PROTECTED]> wrote: > > > > Ok, I do CD to the folder where test is, then run: > > > > runtests.py --settings test > > > > test.py is a settings file with the new backend and db config. You might have a name collision with your settings file and another critical module somewhere. Perhaps you should rename your settings file to testmssql.py? -Forest -- Forest Bond http://www.alittletooquiet.net signature.asc Description: Digital signature
Re: Python 3
On Sun, Sep 02, 2007 at 10:04:35AM -0500, Jacob Kaplan-Moss wrote: > > but have any of the developers tried running django on Python 3? > > Python 3.0a1 (py3k, Aug 31 2007, 13:09:02) > [GCC 4.0.1 (Apple Computer, Inc. build 5367)] on darwin > Type "help", "copyright", "credits" or "license" for more information. > >>> import django > >>> import django.template > Traceback (most recent call last): > File "", line 1, in > File "django/template/__init__.py", line 296 > except TemplateSyntaxError, e: > ^ > SyntaxError: invalid syntax > >>> import django.db > Traceback (most recent call last): > File "", line 1, in > File "django/db/__init__.py", line 12 > except ImportError, e: > ^ > SyntaxError: invalid syntax > >>> import django.forms > Traceback (most recent call last): > File "", line 1, in > File "django/forms/__init__.py", line 1, in > from django.oldforms import * > File "django/oldforms/__init__.py", line 29 > raise KeyError, "Field %s not found\n%s" % (field_name, repr(self.fields)) > ^ > SyntaxError: invalid syntax > > Yeah, it's gonna be a long road... Well, these SyntaxError's are easily corrected using the automated code conversion too, I think. The road may not be unbearably long. These just need parentheses: raise MyError(arg1, arg2) -Forest -- Forest Bond http://www.alittletooquiet.net signature.asc Description: Digital signature
Re: Decoration and aliasing
Hi, On Mon, Oct 08, 2007 at 10:49:14AM -0500, Jacob Kaplan-Moss wrote: > On 10/7/07, Jeremy Dunck <[EMAIL PROTECTED]> wrote: > > I note that this is a (perhaps subtle-enough) backwards-incompatible change. > > Can you say a bit more about how this is backwards-incompatible? I'm > having a dense morning, and can't seem to see any breakage here (which > is a good thing!) I think Jeremy was simply pointing out that object attributes are being permanently changed and somebody, somewhere may be relying on them currently. In this particular case that seems unlikely, of course. -Forest -- Forest Bond http://www.alittletooquiet.net signature.asc Description: Digital signature
Re: Model post_save doesn't play well with Model.save overriding
On Wed, Nov 14, 2007 at 08:55:09AM -0500, Marty Alchin wrote: > On Nov 14, 2007 12:21 AM, Malcolm Tredinnick <[EMAIL PROTECTED]> wrote: > > Jeremy's still got a valid problem, though, but I'm not sure > > reintroducing pre_save() and post_save() is a necessary step yet. > > Hopefully there's something else we can do. > > I wonder if there's a way to modify PyDispatcher (since we're > distributing our own copy anyway) so that it allows dispatching of a > particular signal to be suspended temporarily. That way, dispatched > signals might go into a queue, perhaps and get sent in order when the > signal is reactivated. Then, Jeremy's code might look something like > this: > > from django.dispatch import dispatcher > from django.db.models import signals > > class MyModel(Model): > def save(self): > queue = dispatcher.queue(signals.post_save) > super(MyModel, self).save() > > #extra work here > > queue.dispatch() > > def even_more(sender,instance): > #stuff that depends on extra work having already occurred > > dispatcher.connect(signals.post_save, even_more_stuff) This is neat and all, but I don't think at actually solves the problem at hand. If save is not overridden, when does queue.dispatch() get called? The only way I can think of overcoming this is to dynamically modify self.save when it is accessed (by overriding __getattr__). The signals are fired from code that is dynamically appended to the save method. This guarrantees that the signal is fired at the very end of the save call: class Model(object): def save(self, ...): ... def __getattr__(self, name): value = super(Model, self).__getattr__(name) if name == 'save': def save(self, *args, **kwargs): retval = value(self, *args, **kwargs) send_postsave_signal() return retval return save retun value ---- This adds some overhead, of course, but it does do (roughly) what is being asked. -Forest -- Forest Bond http://www.alittletooquiet.net signature.asc Description: Digital signature
Re: Model post_save doesn't play well with Model.save overriding
Hi, > On Nov 14, 2007 9:37 AM, Forest Bond <[EMAIL PROTECTED]> wrote: > > This is neat and all, but I don't think at actually solves the problem at > > hand. > > If save is not overridden, when does queue.dispatch() get called? > > If save isn't overridden, queue.dispatch() wouldn't be called, because > the queue wouldn't even exist. Signals would just be dispatched the > same way they normally are. Even most overrides of save() still > wouldn't need to deal with the queue, it'd only be necessary if > someone actively needed to delay dispatches in code they don't > control. Ah, I wasn't following that signals are only queued if the queue is explicitly engaged. Now I get it. > > The only way I can think of overcoming this is to dynamically modify > > self.save > > when it is accessed (by overriding __getattr__). The signals are fired from > > code that is dynamically appended to the save method. This guarrantees > > that the > > signal is fired at the very end of the save call: > > You're right, this adds extra overhead, and I don't think it's > necessary, to be honest. Yeah, using the queue does imply some risk > that somebody might set up a queue and not dispatch it, or that they > might forget to use the queue in the first place if they need it, but > it's not the responsibility of the framework to prevent people from > making coding mistakes. Down that path is Java. Nah, I'm not after protecting programmers from themselves, I just missed a bit of sophistication in your queue. I'm not entirely convinced the overhead is all that terrible, especially given that: * Database writes are less frequent than reads. * Anybody overriding save is probably adding more overhead than this anyway. Maybe we could only engage this slight trickery if we detect that save has been overridden by a sub-class of Model? Then, if save hasn't been touched, there is no additional overhead. -Forest -- Forest Bond http://www.alittletooquiet.net On Wed, Nov 14, 2007 at 09:48:37AM -0500, Marty Alchin wrote: signature.asc Description: Digital signature
Re: Django 1.0 features -- the definitive list
Hi, On Fri, Nov 30, 2007 at 08:12:14AM -0600, Malcolm Tredinnick wrote: > On Fri, 2007-11-30 at 08:33 -0500, Forest Bond wrote: > > On Fri, Nov 30, 2007 at 12:33:31AM -0600, Adrian Holovaty wrote: > > > * #285 -- WSGI SCRIPT_NAME and PATH_INFO stuff > > > > Aren't there SCRIPT_NAME/PATH_INFO/etc. problems with mod_python, too? > > It'd be > > nice if django 1.0-based apps could be moved to different relative mount > > points > > without changing .py files at all. Or was this resolved when I wasn't > > looking? > > It's all the same issue. This is one of the things I'm going to review > and commit during the sprint. I was hoping you'd say that. Thanks! -Forest -- Forest Bond http://www.alittletooquiet.net signature.asc Description: Digital signature
Re: Django 1.0 features -- the definitive list
On Fri, Nov 30, 2007 at 12:33:31AM -0600, Adrian Holovaty wrote: > * #285 -- WSGI SCRIPT_NAME and PATH_INFO stuff Aren't there SCRIPT_NAME/PATH_INFO/etc. problems with mod_python, too? It'd be nice if django 1.0-based apps could be moved to different relative mount points without changing .py files at all. Or was this resolved when I wasn't looking? -Forest -- Forest Bond http://www.alittletooquiet.net signature.asc Description: Digital signature
Re: DB API - the limiting syntax, is it magic?
Hi,
On Tue, Dec 04, 2007 at 10:18:11AM -0600, Adrian Holovaty wrote:
> On Dec 4, 2007 12:57 AM, Gary Wilson <[EMAIL PROTECTED]> wrote:
> > But what would the situation be with a new limit() method...
> >
> > objects = MyModel.objects.filter(site=1)
> > first_one = objects.limit(1)
> > do_something_special(first_one)
> > return render_to_response('t.html', {'objects': objects, 'first':
> > first_one})
> >
> > If limit() returns a new QuerySet, you're still going to have two queries.
>
> No -- I guess I didn't explain myself well enough. In this case, I
> wouldn't use limit(). I have two goals:
>
> * Retrieve all objects in the table.
> * Do something special with the first one (once the whole list has
> been retrieved).
>
> The ideal API would look like this, and it would only run a single query:
>
> objects = MyModel.objects.filter(site=1)
> first_one = objects[0]
Isn't that sort of wanting it both ways -- wanting querysets to be both lazy and
not? Surely if the caller wants the objects fetched *now*, he should have to
indicate that, right? Is this not a reasonable way to do that?
objects = list(MyModel.objects.filter(site=1))
first_one = objects[0]
Maybe another method would make that read nicer:
objects = MyModel.objects.filter(site=1).fetch()
first_one = objects[0]
-Forest
--
Forest Bond
http://www.alittletooquiet.net
signature.asc
Description: Digital signature
Re: DB API - the limiting syntax, is it magic?
Hi, Replying to self. On Tue, Dec 04, 2007 at 11:25:42AM -0500, Forest Bond wrote: > objects = MyModel.objects.filter(site=1).fetch() > first_one = objects[0] Incidentally, qs.fetch() need not be equivalent to list(qs). It could just trigger qs to cache the results of the query and return qs itself. That would ensure that the query is only executed once, unless further manipulations are attempted. Not that that is necessarily the right thing to do. -Forest -- Forest Bond http://www.alittletooquiet.net signature.asc Description: Digital signature
Re: Fixing app_label
Hi, On Fri, Dec 07, 2007 at 07:25:03PM +1100, Malcolm Tredinnick wrote: > (or stop using app_loader, which is something I've been tending towards more > and more lately). And how does one do that? -Forest -- Forest Bond http://www.alittletooquiet.net signature.asc Description: Digital signature
Re: Fixing app_label
Hi, On Fri, Dec 07, 2007 at 10:56:09AM -0500, Empty wrote: > On Dec 7, 2007 7:09 AM, Forest Bond <[EMAIL PROTECTED]> wrote: > > On Fri, Dec 07, 2007 at 07:25:03PM +1100, Malcolm Tredinnick wrote: > > > (or stop using app_loader, which is something I've been tending towards > > > more and more lately). > > > > And how does one do that? > > > > Just remove 'django.template.loaders.app_directories.load_template_source', > from the TEMPLATE_LOADERS list in your settings.py module. > > Then all of your templates would be found via the file system template loader. Oh, right. Misunderstood the original statement. Sorry about that. -Forest -- Forest Bond http://www.alittletooquiet.net signature.asc Description: Digital signature
Backward-incompatible change to APPEND_SLASH behavior
Hi, I ran into a backwards-incompatible change related to changing behavior of APPEND_SLASH: http://code.djangoproject.com/changeset/6852 I had some URL patterns in my urlconf that matched URLs with and without a trailing slash. Previously, these would always get redirected to the version with the trailing slash. However, after the change, the middleware would detect that I was handling the URL without the trailing slash and, consequently, not redirect at all. I've updated my code to be compatible, however, I felt that this should be added to http://code.djangoproject.com/wiki/BackwardsIncompatibleChanges. I didn't feel authoritative enough on the subject to add it myself. Thanks, Forest -- Forest Bond http://www.alittletooquiet.net signature.asc Description: Digital signature
