Add a "split" field to a model causes problems
Hi all,
This one stung me today. Basically as part of an event (calendar) app, I
have functionality for splitting a series of events into two at a given
timestamp. The details aren't particularly relevant, but the key thing is
that I had a method called "split" on one of the models.
This was working fine until I tried to add a ManyToManyField to the same
model, at which point it through a rather ugly error during initialisation
of the apps.
django/db/models/fields/related.py line 56, in add_lazy_relation
> app_label, model_name = relation.split(".")
> TypeError: unbound method split() must be called with RecurringEvent
> instance as first argument (got str instance instead)
Because I had no knowledge of the inner workings here, this error wasn't
helpful until pdb told me that `relation` was an instance of my model. The
issue is that in this point in the code, relation can either be a model or
a string 'app_label.model_name', so it's using the presence of a split
function to determine which it is, me having a split function on my model
breaks this check. Having a non-callable 'split' causes a similar problem,
just with a different exception.
Long story short, adding a 'split' method to my model caused the
inner-workings of Django to break, I think the documentation should provide
a list of attribute names that can't be added to a Django model without
breaking things, though this is the first time I've ever come across such a
problem so it might be a one-off.
Regards,
Andrew Ingram
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/django-developers/-/0A23Ye0xgcQJ.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Re: Streaming HttpResponse revisted. Any core devs, please take a look if you can :)
On 23 loka, 00:51, Aymeric Augustin wrote: > Hello, > > While I'm working on HttpResponse, I'd like to resolve two related tickets. > Here's my plan for discussion. > > #6527 is about normalizing how HttpResponse deals with iterators. For more > background, see part 1 of the first email of this thread. > > Let's take an HttpResponse instantiated with an iterator. > > It's possible not to access the content until the WSGI server iterates on the > response. This allows creating streaming responses in Django 1.4. But it's > incompatible with most middleware. > > Most often middleware will access the content before the response is finally > rendered. The first access to response.content works, but subsequent accesses > return an empty string. > > The fix is easy: consume the content of the iterator and save it in a list > for further use. This can be done at two points: > > a) In __init__. This is a simple solution advocated by Malcolm in several > tickets and it decreases the complexity of HttpResponse (a worthy goal). > > It's backwards incompatible because it breaks the streaming scenario > described above. We'd have to raise a deprecation warning when the iterator > is consumed before the final iteration by the WSGI server, tell people to use > StreamingHttpResponse instead, and make the change after two releases. > > b) When required. This is the solution implemented in this > patch:https://code.djangoproject.com/attachment/ticket/6527/6527-v3.patch > Even if the target is a), this is a useful workaround for the length of the > deprecation period. I vote consume on __init__ after deprecation period. During the deprecation period, cache on first access. The behaviour of (b) "stream except if accessed" doesn't seem useful. If you need streaming response, then use HttpStreamingResponse, if not, then you don't get streaming response ever. How do we actually deprecate this? Would we raise deprecation for any iterator passed to HttpResponse? > Finally, I've closed five or six other tickets that were fixed at the same > time as #7581 or didn't add anything to the two tickets described above; > please tell me if I've missed something. Thanks for getting streaming responses committed. I tried to work on this feature but my HTTP/WSGI skills weren't good enough to actually commit anything... - Anssi -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: finding the source of "interesting" SQL queries
So the straw man solution we've implemented for now looks like this.
(Background: we have a settings.THREADLOCALS construct which serves in a
capacity similar to Ned's global requests; also, our monkeypatch_method
decorator does what you probably think it does.) First, we've a tiny bit of
middleware:
from django.conf import settings
class RequestURIMiddleware:
"""
Set the request URI in THREADLOCALS so that it can be used to comment
SQL queries with what triggered them.
"""
def process_request(self, request):
settings.THREADLOCALS.request_uri = request.build_absolute_uri()
And then we've monkeypatched the cursor method on BaseDatabaseWrapper like
so:
from django.conf import settings
from django.db.backends import BaseDatabaseWrapper, util
from patchers import monkeypatch_method
import traceback
class CommentingCursorWrapper(util.CursorDebugWrapper):
def execute(self, sql, *args, **kwargs):
"""
Before sending to the DB, this adds a comment to the SQL with notes
about the query's origin
"""
try:
if getattr(settings, 'SQL_COMMENTS_ALWAYS_TRACEBACK', False):
raise AttributeError
# THREADLOCALS.request_uri is (usually) populated by
RequestURIMiddleware
origin_comment = ' /* Originated from request to
{0}'.format(settings.THREADLOCALS.request_uri)
except AttributeError:
# If no URI available (e.g., Celery task), report the first
non-Django point in the call stack:
tb = reversed(traceback.format_stack()[:-1]) # walk it bottom-up,
excluding this frame
for frame in tb:
if 'django' not in frame:
origin_comment = ' /* Originated
at{0}'.format(frame.split('\n')[0])
break
origin_comment = origin_comment.replace('%', '%%')
origin_comment = origin_comment.replace('*/', '\*\/')
sql += origin_comment + ' */'
return self.cursor.execute(sql, *args, **kwargs)
@monkeypatch_method(django.db.backends.BaseDatabaseWrapper)
def cursor(self, *args, **kwargs):
return CommentingCursorWrapper(cursor._original_cursor_function(self,
*args, **kwargs), self)
So in short, we comment the SQL with the URI when it's available, and fall
back to the (presumably more expensive to gather) traceback information
when it's not.
This is just the monkeypatch we're throwing in to solve our immediate
problem, but it'd be easy enough to convert into a patch. Before we do, any
thoughts/criticisms of this approach?
Best,
Marty
http://www.mapmyfitness.com/profile/woodlee/
On Monday, October 22, 2012 11:23:47 PM UTC-5, Matt McClure wrote:
>
> Thanks, Russell. That's similar to the approach we were thinking of
> implementing. Hopefully we'll have a straw man to share shortly.
>
> Matt
>
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/django-developers/-/-X0AFXuzDv8J.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Re: Add a "split" field to a model causes problems
On Tue, Oct 23, 2012 at 6:52 PM, Andrew Ingram wrote:
> Hi all,
>
> This one stung me today. Basically as part of an event (calendar) app, I
> have functionality for splitting a series of events into two at a given
> timestamp. The details aren't particularly relevant, but the key thing is
> that I had a method called "split" on one of the models.
>
> This was working fine until I tried to add a ManyToManyField to the same
> model, at which point it through a rather ugly error during initialisation
> of the apps.
>
> django/db/models/fields/related.py line 56, in add_lazy_relation
>> app_label, model_name = relation.split(".")
>> TypeError: unbound method split() must be called with RecurringEvent
>> instance as first argument (got str instance instead)
>
>
> Because I had no knowledge of the inner workings here, this error wasn't
> helpful until pdb told me that `relation` was an instance of my model. The
> issue is that in this point in the code, relation can either be a model or
> a string 'app_label.model_name', so it's using the presence of a split
> function to determine which it is, me having a split function on my model
> breaks this check. Having a non-callable 'split' causes a similar problem,
> just with a different exception.
>
> Long story short, adding a 'split' method to my model caused the
> inner-workings of Django to break, I think the documentation should provide
> a list of attribute names that can't be added to a Django model without
> breaking things, though this is the first time I've ever come across such a
> problem so it might be a one-off.
>
> Hi Andrew,
Thanks for that gnarly bug report. I'm a little surprised that the 'split'
name causes a problem here; based on my knowledge of Django's internals, I
wouldn't have thought that the situation you describe shouldn't be a
problem. My gut reaction tells me that the actual problem is somewhere else
in your model definition, causing the wrong object instance to be passed
into the 'relation' variable; the 'split' method is a symptom, rather than
the actual problem. However, I may be wrong.
If you can open a bug report with a complete working (or rather, broken)
example of the problem, we can dig into it more. If you're right that the
split method is the problem, we should certainly document it; if the
problem lies somewhere else, there's possibly some better validation that
we need to do.
Yours,
Russ Magee %-)
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Re: Add a "split" field to a model causes problems
On Tue, Oct 23, 2012 at 5:11 PM, Russell Keith-Magee <
russ...@keith-magee.com> wrote:
>
>
> On Tue, Oct 23, 2012 at 6:52 PM, Andrew Ingram wrote:
>
>> Hi all,
>>
>> This one stung me today. Basically as part of an event (calendar) app, I
>> have functionality for splitting a series of events into two at a given
>> timestamp. The details aren't particularly relevant, but the key thing is
>> that I had a method called "split" on one of the models.
>>
>> This was working fine until I tried to add a ManyToManyField to the same
>> model, at which point it through a rather ugly error during initialisation
>> of the apps.
>>
>> django/db/models/fields/related.py line 56, in add_lazy_relation
>>> app_label, model_name = relation.split(".")
>>> TypeError: unbound method split() must be called with RecurringEvent
>>> instance as first argument (got str instance instead)
>>
>>
>> Because I had no knowledge of the inner workings here, this error wasn't
>> helpful until pdb told me that `relation` was an instance of my model. The
>> issue is that in this point in the code, relation can either be a model or
>> a string 'app_label.model_name', so it's using the presence of a split
>> function to determine which it is, me having a split function on my model
>> breaks this check. Having a non-callable 'split' causes a similar problem,
>> just with a different exception.
>>
>> Long story short, adding a 'split' method to my model caused the
>> inner-workings of Django to break, I think the documentation should provide
>> a list of attribute names that can't be added to a Django model without
>> breaking things, though this is the first time I've ever come across such a
>> problem so it might be a one-off.
>>
>> Hi Andrew,
>
> Thanks for that gnarly bug report. I'm a little surprised that the 'split'
> name causes a problem here; based on my knowledge of Django's internals, I
> wouldn't have thought that the situation you describe shouldn't be a
> problem. My gut reaction tells me that the actual problem is somewhere else
> in your model definition, causing the wrong object instance to be passed
> into the 'relation' variable; the 'split' method is a symptom, rather than
> the actual problem. However, I may be wrong.
>
> If you can open a bug report with a complete working (or rather, broken)
> example of the problem, we can dig into it more. If you're right that the
> split method is the problem, we should certainly document it; if the
> problem lies somewhere else, there's possibly some better validation that
> we need to do.
>
> Yours,
> Russ Magee %-)
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>
No this is a bug in Django, I see it.
https://github.com/django/django/blob/master/django/db/models/fields/related.py#L51
Basically some genius (very likely me), decided we shouldn't type check for
string or model, and just try to call split.
It's an easy enough fix. Please file a bug and if there's no patch I'll do
it tonight.
Alex
--
"I disapprove of what you say, but I will defend to the death your right to
say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
