Re: auth.User: The abstract base class idea
I agree on both this point and your future point about using a migration tool such as South (and South is truly the only one I can think of that will do this painlessly). We definitely suffer from these User model problems at our Company as does everyone I think that really uses Django's authentication system for that. These suggestions seem the most sound all things considered and the original list of points makes a strong argument for it. I would like to see this sooner than later. How many more releases go by? Hopefully its in by 1.5 ,,, On Mar 20, 12:00 pm, Alex Ogier wrote: > > This is pretty much exactly what django-primate does, except it includes a > > special metaclass to allow > > overriding of default fields. > > Yeah, that is the basic idea. I think that permissions and associated > code should be moved from UserBase to live directly on auth.User or in > a mixin of their own (they are mostly orthogonal to authentication, > yes?) so that writing one's own base user is easier, but the > monkey-patch looks like it is implemented in a nice way. So long as > writing your own base is easy enough, the extra special-case metaclass > and field overriding isn't strictly necessary, and can wait until > explicitly overriding fields has proper support in core (something I > think will be generally useful). > > -Alex Ogier -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
model query get_or_create() thread safety and uniqueness -- proposal for transactional commit before final get
Greetings Developers, Today at work I fixed a thread safety issue in our code whereby two threads were calling get_or_create on the same URL which has a uniqueness constraint. It was in some cases raising an IntegrityError in sentry even after I converted our atomically incorrect code to use get_or_create proving to me it must be two different threads performing the same action. Let me elaborate ... Django's get_or_create does a Get, failing that a create, failing that a rollback and another get. For some reason in MySQL, and to me conceptually the code roll back to prior to the commit when the other thread wins in the create block and saves the object, it still doesn't exist when the other thread tries to get it a final time, because I believe it rolled back to the earlier pre-save savepoint. Here is the final part of the get_or_create: obj = self.model(**params) sid = transaction.*savepoint*(using=self.db) obj.save(force_insert=True, using=self.db) transaction.savepoint_commit(sid, using=self.db) return obj, True except *IntegrityError* as e: transaction.savepoint_rollback(sid, using=self.db) exc_info = sys.exc_info() try: return self.get(**lookup), False except self.model.DoesNotExist: * # Re-raise the IntegrityError* with its original traceback. six.reraise(*exc_info) The question for me is: *Why wouldn't we want this to be this way?*: obj = self.model(**params) sid = transaction.*savepoint*(using=self.db) obj.save(force_insert=True, using=self.db) transaction.savepoint_commit(sid, using=self.db) return obj, True except *IntegrityError* as e: transaction.savepoint_rollback(sid, using=self.db) exc_info = sys.exc_info() try: *transaction.commit()* # To refresh the DB view for thread safety return self.get(**lookup), False # ... Get succeeds now in a thread unsafe world except self.model.DoesNotExist: * # Re-raise the IntegrityError* with its original traceback. six.reraise(*exc_info) *Also ---* On the Django Website, The Django Downloads page Django gtihub tarball master is a 1.4 zip file. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: model query get_or_create() thread safety and uniqueness -- proposal for transactional commit before final get
Hi Amymeric,
That makes sense about the different (and default) MySQL transactional mode.
I don't know about the Django 1.6 transaction changes yet, but I am curious
of it will break my managed transaction blocks in existing Django 1.3.x
code.
For example, to handle this get_or_create issue I outlined, I do something
like the following:
var = None # initialize the variable
try:
var, c = self.model.objects.get_or_create(url=url)
except Exception, e:
log.error("Exception in get_or_create: {0}".format(e))
transaction.commit()
vars = self.model.objects.filter(url=url)
var = vars[0]
...
transaction.commit()
return var
Will this still work in Django 1.6 for our use case assuming our MySQL
configuration stays the same default as it has been? ...
Thanks for your replies,
On Thursday, April 4, 2013 11:23:43 PM UTC-4, Matteius wrote:
>
>
> Greetings Developers,
>
> Today at work I fixed a thread safety issue in our code whereby two
> threads were calling get_or_create on the same URL which has a uniqueness
> constraint. It was in some cases raising an IntegrityError in sentry even
> after I converted our atomically incorrect code to use get_or_create
> proving to me it must be two different threads performing the same action.
> Let me elaborate ...
>
> Django's get_or_create does a Get, failing that a create, failing that a
> rollback and another get. For some reason in MySQL, and to me
> conceptually the code roll back to prior to the commit when the other
> thread wins in the create block and saves the object, it still doesn't
> exist when the other thread tries to get it a final time, because I believe
> it rolled back to the earlier pre-save savepoint.
>
> Here is the final part of the get_or_create:
>
> obj = self.model(**params)
> sid = transaction.*savepoint*(using=self.db)
> obj.save(force_insert=True, using=self.db)
> transaction.savepoint_commit(sid, using=self.db)
> return obj, True
> except *IntegrityError* as e:
> transaction.savepoint_rollback(sid, using=self.db)
> exc_info = sys.exc_info()
> try:
> return self.get(**lookup), False
> except self.model.DoesNotExist:
> * # Re-raise the IntegrityError* with its original
> traceback.
> six.reraise(*exc_info)
>
>
> The question for me is: *Why wouldn't we want this to be this way?*:
>
> obj = self.model(**params)
> sid = transaction.*savepoint*(using=self.db)
> obj.save(force_insert=True, using=self.db)
> transaction.savepoint_commit(sid, using=self.db)
> return obj, True
> except *IntegrityError* as e:
> transaction.savepoint_rollback(sid, using=self.db)
> exc_info = sys.exc_info()
> try:
> *transaction.commit()* # To refresh the DB view for
> thread safety
> return self.get(**lookup), False # ... Get succeeds
> now in a thread unsafe world
> except self.model.DoesNotExist:
> * # Re-raise the IntegrityError* with its original
> traceback.
> six.reraise(*exc_info)
>
>
> *Also ---* On the Django Website, The Django Downloads page Django
> gtihub tarball master is a 1.4 zip file.
>
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
Re: model query get_or_create() thread safety and uniqueness -- proposal for transactional commit before final get
So in Django 1.6, commit_manually() is deprecated and then removed in Django 1.8. I'm not sure I totally get the change to require everyone use atomic(), and I'm not sure it is clear yet what changes I'll need to make to make to convert the code to use atomic. It does appear instead of doing plain commits() you'll be doing transaction savepoint commits and rollbacks. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
settings.py should have way to hide key/values even when debug set True
I think it would be really useful to have a way (possibly a decorator such as @hide_setting) such as to protect deployed sites when they switch over to debug mode. To me this would be a most useful setting to have, especially when protecting secret key settings. Please be advised. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: settings.py should have way to hide key/values even when debug set True
I was thinking the decorator could apply to whatever setting is declared directly following it. If declared, the setting simply wouldn't show up. However I could add SECRET or PASSWORD to my token sensitive settings just as well. On Jan 31, 7:00 pm, Russell Keith-Magee wrote: > On Tue, Feb 1, 2011 at 12:12 AM, Horst Gutmann wrote: > > On Mon, Jan 31, 2011 at 5:02 PM, Matteius wrote: > >> I think it would be really useful to have a way (possibly a decorator > >> such as @hide_setting) such as to protect deployed sites when they > >> switch over to debug mode. To me this would be a most useful setting > >> to have, especially when protecting secret key settings. Please be > >> advised. > > > If I remember correctly, settings starting with SECRET_ are not shown > > on the debug page. > > Not completely correct. There is a list of settings that won't be > displayed -- anything that contains the text > SECRET, PASSWORD, PROFANITIES_LIST, or SIGNATURE will be replaced > with asterisks. It's not an unreasonable suggestion that this list > should be user-configurable. > > As for the original proposal - I'm not sure how a decorator would help > here. What is the decorator being applied to? What would the decorator > indicate? How would that information be exposed to the debug view? > > Yours, > Russ Magee %-) -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Is Django cache.add atomic --AKA Free from race conditions?
If I have multiple requests hit the cache.add at the same time, is it
guaranteed that only one cache.add will return success (True) once and
all other requests are guaranteed to be failures, even in the case of
race conditions?
cache.add('add_Grade', 'Some Value')
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Re: Is Django cache.add atomic --AKA Free from race conditions?
which I am guessing it is not ...
On Feb 2, 4:57 pm, Alex Gaynor wrote:
> On Wed, Feb 2, 2011 at 5:55 PM, Matteius wrote:
> > If I have multiple requests hit the cache.add at the same time, is it
> > guaranteed that only one cache.add will return success (True) once and
> > all other requests are guaranteed to be failures, even in the case of
> > race conditions?
>
> > cache.add('add_Grade', 'Some Value')
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Django developers" group.
> > To post to this group, send email to django-developers@googlegroups.com.
> > To unsubscribe from this group, send email to
> > django-developers+unsubscr...@googlegroups.com
> > .
> > For more options, visit this group at
> >http://groups.google.com/group/django-developers?hl=en.
>
> It is IFF the underlying implementation is atomic.
>
> Alex
>
> --
> "I disapprove of what you say, but I will defend to the death your right to
> say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
> "The people's good is the highest law." -- Cicero
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Re: Is Django cache.add atomic --AKA Free from race conditions?
Well OK so it would appear as though memcached and the python localmem
both use locking and memcached support object level atomicity. Is
this the goal of Django Transactions to provide this natively with
atomic model saves, or will I have to wrap my model saves with a keyed
value to establish a lock on the user/model to prevent duplicate
actions ...
On Feb 2, 10:21 pm, Matteius wrote:
> which I am guessing it is not ...
>
> On Feb 2, 4:57 pm, Alex Gaynor wrote:
>
>
>
> > On Wed, Feb 2, 2011 at 5:55 PM, Matteius wrote:
> > > If I have multiple requests hit the cache.add at the same time, is it
> > > guaranteed that only one cache.add will return success (True) once and
> > > all other requests are guaranteed to be failures, even in the case of
> > > race conditions?
>
> > > cache.add('add_Grade', 'Some Value')
>
> > > --
> > > You received this message because you are subscribed to the Google Groups
> > > "Django developers" group.
> > > To post to this group, send email to django-developers@googlegroups.com.
> > > To unsubscribe from this group, send email to
> > > django-developers+unsubscr...@googlegroups.com
> > > .
> > > For more options, visit this group at
> > >http://groups.google.com/group/django-developers?hl=en.
>
> > It is IFF the underlying implementation is atomic.
>
> > Alex
>
> > --
> > "I disapprove of what you say, but I will defend to the death your right to
> > say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
> > "The people's good is the highest law." -- Cicero
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Re: Is Django cache.add atomic --AKA Free from race conditions?
Nevermind, it would seem as though transactions are only for rolling
back or preventing database changes in the case of failures, and
provide no other kind of constraints. I'm writing an article on
enforcing uniqueness constraints in Django.
On Feb 3, 1:19 am, Matteius wrote:
> Well OK so it would appear as though memcached and the python localmem
> both use locking and memcached support object level atomicity. Is
> this the goal of Django Transactions to provide this natively with
> atomic model saves, or will I have to wrap my model saves with a keyed
> value to establish a lock on the user/model to prevent duplicate
> actions ...
>
> On Feb 2, 10:21 pm, Matteius wrote:
>
>
>
> > which I am guessing it is not ...
>
> > On Feb 2, 4:57 pm, Alex Gaynor wrote:
>
> > > On Wed, Feb 2, 2011 at 5:55 PM, Matteius wrote:
> > > > If I have multiple requests hit the cache.add at the same time, is it
> > > > guaranteed that only one cache.add will return success (True) once and
> > > > all other requests are guaranteed to be failures, even in the case of
> > > > race conditions?
>
> > > > cache.add('add_Grade', 'Some Value')
>
> > > > --
> > > > You received this message because you are subscribed to the Google
> > > > Groups
> > > > "Django developers" group.
> > > > To post to this group, send email to django-developers@googlegroups.com.
> > > > To unsubscribe from this group, send email to
> > > > django-developers+unsubscr...@googlegroups.com
> > > > .
> > > > For more options, visit this group at
> > > >http://groups.google.com/group/django-developers?hl=en.
>
> > > It is IFF the underlying implementation is atomic.
>
> > > Alex
>
> > > --
> > > "I disapprove of what you say, but I will defend to the death your right
> > > to
> > > say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
> > > "The people's good is the highest law." -- Cicero
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Re: contributing to django as a part of master thersis
Russell hit the nail on the head with his initial reply. I don't have time to read all the rest of this, but definitely and good luck on your masters. Congratulations on getting into a program. On Feb 10, 12:29 am, Russell Keith-Magee wrote: > On Thu, Feb 10, 2011 at 5:45 AM, Thomas Adamcik wrote: > > On Wed, Feb 9, 2011 at 12:00 PM, Russell Keith-Magee > > wrote: > >> Another interesting, but more complicated opportunity in this area > >> would be asynchronous database calls [1], which some databases (by > >> which I mean, Postgres) support. > > >> [1]http://initd.org/psycopg/docs/advanced.html#asynchronous-support > > >> This sort of interface would allow you to push multiple queries onto > >> the database, continue processing in your view, and only retrieve the > >> data when it's actually required. This exploits the lazy evaluation of > >> querysets by using the period while the query is being lazy to > >> actually issue and compute the query result. > > >> Implementing asynchronous database access isn't a trivial task. But > >> then, you said this was for your Masters, so non-trivial work should > >> be what you are looking for :-) > > > I've been toying with a similar idea, though I never got around > > writing any code. The API I envisioned was basically > > Model.objects.async(), where the async() call should immediately fire > > the async query, if some other code tries to use the queryset before > > it is ready we simply block like a regular queryset would. > > Model.objects.filter(...).async() should of course also work. > > I had a slightly different color for the bikeshed, but the idea was > about the same -- provide a "this query is really finished, so you can > execute it now" marker that would kickstart the execution process, and > use query iteration as the point at which you either return the > result, or wait until the async call returns. > > The added bonus of this approach is that it can easily be a no-op for > backends that don't support the behavior. > > > Based on experiments with the ORM I've done to get INET/CIDR support > > for Postgres, I was planing on implementing this as an external > > project using a custom models.Manager + query.Queryset. > > I'll be interested to see what you come up with. If it turns out you > need something to change in Django's ORM internals in order to make > this happen, let us know. > > Yours, > Russ Magee %-) -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Inline debugging
What I want to see (and possibly the closest I've seen it with was a video on the django_command_extensions) is an interactive debugger built into my test-dev server so I can have break pts, switch logic values in place, etc. dynamically on the test-dev server. -Matteius On Feb 10, 1:10 am, Sameer Rahmani wrote: > Hi Russel > > at first thanks for consideration. I post that patch just because > providing a little background about idea for other, also i know that > that patch is not well written code. So it will be nice to talk about > the main idea not about that patch. > > I think hooks are a useful for development. If django provide a hook > facility (i think django does not have such thing right now, but i'm not > sure) something like elisp hooks, django development process will > improve. > > but at last i think runserver can have some hooks that allow users to > change that by his/her choices but django can have its own inline > debugging tools build upon those hooks internally. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
ValueQuerySet should be serializable into json but isn't
I want to optimize my json call and protect data by doing something
like:
assignments = Assignment.objects.values('id',
'name').filter(course=enrollment.course)
payload = serializers.serialize("json", assignments,
ensure_ascii=False)
But my experience and others have been:
(' dict object has no attribute __meta').
The only option I see is to manually encode a json string and bypass
the serialization?
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Getting Slammed Hard on Foreign Keys in the admin due to __unicode__
I have a very small test database on my test development virtualbox environment and so while I've been adding in custom AJAX for dynamic ForeignKey filters (in the case of Enrollment's Assignment list) I noticed that because I'm using the Django Debug Toolbar. Well on these pages with only ~28 Enrollments I get hit up with 72 queries and the page takes 780 ms to load. Well this could probably be solved by adding select_related() to the lookups on foreign key fields or at least something. This is terrible default behavior, but I can't really ship an admin interface to the client without showing the relationship in the system. This is definitely a case where the admin needs improvement. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Getting Slammed Hard on Foreign Keys in the admin due to __unicode__
No the hits are with the initial page load before any AJAX gets
called, my AJAX code is 2 DB hits. I am not claiming select_related
should be default, but somehow it should be an option for the admin
change form. Here is why, here is an example of the worst cases:
You'll see two different ForeignKey fields, each one has a separate
unicode method (see below): I hope you can see how the default
behavior with such a pattern creates 73 individual admin queries by
default, and the only way to fix this supported by Django is to get
rid of those discriptive unicode methods, or as you say, to manually
override admin to use select_related on these cases. Has very little
to do with optimizing at the ORM level, and has everything to do with
the Django Admin as I originally asses.
class Grade(models.Model):
# Database Fields
return_date = models.DateTimeField(auto_now_add=True)
enrollment = models.ForeignKey(Enrollment,
verbose_name='Student Enrollment',
related_name='grade_enrollment')
assignment = models.ForeignKey(Assignment,
verbose_name='Assignment to Grade',
related_name='grade_assignment')
points_earned = models.DecimalField('Points Earned', max_digits=5,
decimal_places=2)
returned_files = ProtectedFileField(upload_to=get_grade_path,
max_length=250, null=True, blank=True)
comments = models.TextField('Comments', null=True, blank=True,
help_text="Form accepts XHTML but use it mindfully--The view
should not be altered!")
def __unicode__(self):
""" Default unicode string describing the Grade. """
return (self.enrollment.__unicode__() + " - " +
self.assignment.name)
# EndDef
class Enrollment(models.Model):
def __unicode__(self):
""" Default unicode string describing the Enrollment. """
return (self.student.username + " - " + self.course.name)
# EndDef
class Assignment(models.Model):
def __unicode__(self):
""" Default unicode string describing the Assignment. """
return (self.course.department.name + " - " + self.course.name
+ " - " + self.name)
# EndDef
On Feb 14, 12:20 am, Stephen Burrows
wrote:
> Though it's a little hard to tell from your post, this sounds more
> like it's definitely a case where your code needs improvement. From
> what I understand, the database hits are coming from your custom AJAX,
> not from the admin's default behavior. Also, you already know the
> solution: use select_related. If you are saying that django ORM should
> make select_related=True the default - no. That would be bad. Every
> query for a model with a foreign key would be bloated by the
> additional joins and selects, whether or not any of the data on the fk
> was being used. Overall efficiency might actually decrease.
> To put it another way, "Premature optimization is the root of all
> evil."
> Best,
> Stephen
>
> On Feb 13, 10:35 pm, Matteius wrote:
>
>
>
> > I have a very small test database on my test development virtualbox
> > environment and so while I've been adding in custom AJAX for dynamic
> > ForeignKey filters (in the case of Enrollment's Assignment list) I
> > noticed that because I'm using the Django Debug Toolbar. Well on
> > these pages with only ~28 Enrollments I get hit up with 72 queries and
> > the page takes 780 ms to load. Well this could probably be solved by
> > adding select_related() to the lookups on foreign key fields or at
> > least something. This is terrible default behavior, but I can't
> > really ship an admin interface to the client without showing the
> > relationship in the system. This is definitely a case where the admin
> > needs improvement.
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Re: ValueQuerySet should be serializable into json but isn't
Yes and I am stating that a ValueQuerySet seems related enough to
QuerySet that in Django default behavior should allow them to work
with Django serializers, and I'm not the only one to have assumed it
would! Using Django serializes you'll find that a ValueQuerySet is
missing attribute _meta when it goes to walk the model class def.
Well the reality is in AJAX situations we often don't want to send the
whole model and so being able to serialize a ValueQuerySet is a very
useful feature that Django is lacking for simplifying the query:
Well, we also have:
def enrollment_assignments(request, enrollment_id):
""" AJAX call to get list of assignments for enrollment_id. """
if request.is_ajax(): # Fetch Assignment List
enrollment = Enrollment.objects.get(id__exact=enrollment_id)
assignments = Assignment.objects.values('id',
'name').filter(course=enrollment.course).order_by('name')
payload = json.dumps([a for a in assignments])
else:
payload = "Sorry, this URL is for AJAX calls only."
return HttpResponse(payload,
content_type = 'application/javascript;
charset=utf8')
# End Def
Also I must have filled out the Django EuroCon 2011 Amsterdam form 4
or 5 times and I still haven't received any E-mails about this. I
probably should have stayed with RAILS 4 years ago, but oh no Python
and Django Reinhardt .
Well and Django has really fixed many of my initial concerns, I really
want to go, there is a bunch of stuff I missed in Amsterdam the last
time I went in 2007. I would also like to give a presentation of
django-classcomm Http://classcomm.googlecode.com/ and http://classcomm.net/
as well as working on the Django development sprint following the
conference.
-Matteius
On Feb 13, 8:11 pm, Russell Keith-Magee
wrote:
> On Mon, Feb 14, 2011 at 9:55 AM, Matteius wrote:
> > I want to optimize my json call and protect data by doing something
> > like:
>
> > assignments = Assignment.objects.values('id',
> > 'name').filter(course=enrollment.course)
> > payload = serializers.serialize("json", assignments,
> > ensure_ascii=False)
>
> > But my experience and others have been:
>
> > (' dict object has no attribute __meta').
>
> > The only option I see is to manually encode a json string and bypass
> > the serialization?
>
> This question is very close to being a "How To" question.
> Django-developers is for discussing the development of Django itself.
> If you want to ask how to do something, you should ask on
> django-users.
>
> However -- just in case this was a misphrased request for discussion
> of a feature proposal: Django's serializers are not universal
> serializers. They are specifically for serializing full querysets --
> i.e., collections of full objects. If you want to serialize other
> structures, you need to use simplejson.dumps natively.
>
> Yours,
> Russ Magee %-)
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Re: Getting Slammed Hard on Foreign Keys in the admin due to __unicode__
Thanks, you got me going on the right path.
On Feb 14, 12:31 pm, Jacob Kaplan-Moss wrote:
> On Mon, Feb 14, 2011 at 11:41 AM, Matteius wrote:
> > No the hits are with the initial page load before any AJAX gets
> > called, my AJAX code is 2 DB hits. I am not claiming select_related
> > should be default, but somehow it should be an option for the admin
> > change form.
>
> Have you tried supplying the admin a queryset with select_related()?
>
> That is::
>
> class GradeAdmin(admin.ModelAdmin):
> def queryset(self, request):
> return super(GradeAdmin,
> self).queryset(request).select_related('enrollment', 'assignment')
>
> (This is documented
> here:http://docs.djangoproject.com/en/dev/ref/contrib/admin/#django.contri...).
>
> This should work. If the admin doesn't respect the select_related
> call, then that's a bug.
>
> Jacob
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Re: Getting Slammed Hard on Foreign Keys in the admin due to __unicode__
OK this suggestion has gotten me part of the way and certainly improved the state of the admin. However, I'm still getting linear dependence in the admin now in the case of inline editing. I have Assignments, Instructor and Mentors all with a ForeignKey to Course. Well for the example of a new Course's edit page now has all these as inlines. Well if I have a course with 4 assignments and 1 instructor thats 25 DB queries. We need a similar way to limit queries with inline objects in the Django admin. Sincerely, Matteius -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Getting Slammed Hard on Foreign Keys in the admin due to __unicode__
OK got it down to 15 using another def queryset in the Assignment admin.TabularInline. I'm still not sure why it is executing the following query 5 times, but it is (half the total time of the total queries): 0.56SELECT EXPLAIN Toggle Stacktrace SELECT `auth_user`.`id`, `auth_user`.`username`, `auth_user`.`first_name`, `auth_user`.`last_name`, `auth_user`.`email`, `auth_user`.`password`, `auth_user`.`is_staff`, `auth_user`.`is_active`, `auth_user`.`is_superuser`, `auth_user`.`last_login`, `auth_user`.`date_joined` FROM `auth_user` On Feb 16, 12:50 am, Matteius wrote: > OK this suggestion has gotten me part of the way and certainly > improved the state of the admin. However, I'm still getting linear > dependence in the admin now in the case of inline editing. I have > Assignments, Instructor and Mentors all with a ForeignKey to Course. > Well for the example of a new Course's edit page now has all these as > inlines. Well if I have a course with 4 assignments and 1 instructor > thats 25 DB queries. We need a similar way to limit queries with > inline objects in the Django admin. > > Sincerely, > Matteius -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Getting Slammed Hard on Foreign Keys in the admin due to __unicode__
I just thought because I did surgery on my own neck once I'd be able to go to the guilde of surgeons. On Feb 16, 7:12 pm, Luke Plant wrote: > On Wed, 2011-02-16 at 11:36 -0800, Matteius wrote: > > OK got it down to 15 using another def queryset in the Assignment > > admin.TabularInline. I'm still not sure why it is executing the > > following query 5 times, but it is (half the total time of the total > > queries): > > We are very definitely into django-users territory, not > django-developers here. > > Thanks, > > Luke > > -- > A former CEO: "Some of you think that only half of the Board of > Directors do all the work and the rest do nothing, actually the > reverse is true." (True Quotes From Induhviduals, Scott Adams) > > Luke Plant ||http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Support for MySQL `INSERT ... ON DUPLICATE KEY UPDATE ...`. proposed and code example
*Greetings Django Developers,* I am opening this thread based on my research and existing coding toward a solution of this issue, whereby Django today does not support effective usage of the MySQL specific construct `INSERT ... ON DUPLICATE KEY UPDATE ...`. First some background: last month I solved a master-master MySQL replication problem we were having at my place of work whereby certain tables had records that were colliding during fail over events. The issue originated from our optimistic collection of data, except that in fail over with master-master, we found it was possible to collect the same data in both centers and then the UNIQUE constrain on the keys would blow up. I solved the problem, by opening a mysql.py and writing two methods, which utilize the Django internals, to essentially extend the INSERT construct, and to generate what is essentially a valid Upsert construct. It utilizes the MySQL functionality outlined here: http://dev.mysql.com/doc/refman/5.0/en/insert-on-duplicate.html I now have the support of the development team to work towards a Django patch that which would adapt this functionality in the most transparent way possible to the ORM, in order to add further support for this MySQL specific feature. There are two calling patterns which are important here: 1. The update idempotent field pattern, where we simply pass in the name of the primary key such that we assign the key to be equal to the key – essentially, if the record does exist, don’t truly update anything about it. 2. The update all fields pattern, where by if the original INSERT clause failed due to duplicate key it will intend to go ahead with updating all fields, as if the original INSERT had worked out of the box. The difference between the two, and why this is important, is because if we are creating a dummy record at some point (such as a FK constraint but prior to knowing how we want to fill out the record) then we don’t want this INSERT to collide and take out the creation of the real data within the record. As a result we have two ways to call this. I hope this is about to become more clear as I share the code sample which I wish to adopt into a patch, such that I might fulfill my dream of becoming a Django contributor. I now have permission to adapt this code to be a suitable patch for Django to adopt support of this MySQL specific feature. So I am sharing it now here in its 37 lines of glory. https://gist.github.com/matteius/fff39563d1d8ddfc7168 I am looking for general feedback as well as some specific feedback: 1. Right now how I actually execute this custom logic is slightly ugly: I check if the settings DATABASE engine is MySql, and we call the specialized logic, otherwise I use the regular model.save() because it means the code is being run from unit tests on a sqlitedb. 2. To me it seems somewhat similar to update_or_create, but is still MySQL specific and has the calling patterns mentioned above. Is there a way this might be a well suited MySQL specific override to a pattern such as `update_or_create` or is there another pattern more suited? 3. When we look at a database specific feature such as a MySQL statement such as this, what kind advice or guidance can be given around unit testing a database specific feature? My only though about this so far is it might be possible to test the semantic construct being generated, and to even mock out the sql execution in the test. – this brings me to a sub-question I’d like to know more about Django … 3.b.) What DB powers the Django unit tests (sqllite?) and Does Django ever mock out function calls in the test—I often use the mock library but I doubt that is being used here. 4.) I hope this is well received, I think supporting this DB specific features that helps solve problems that arise out of more complicated configurations is important to Django maturing as a product. I had found a couple older items out there referencing the lack of support, but this solution to the problem is technically my own and I think could be well adapted for Django. Since this is a MySQL specific feature, it is not totally clear to me how this best fits within the scope of the ORM. I wrote here first rather than opening a ticket, because I read the guidelines. Despite attending the DjangoCon 2013 sprints, I am still not technically a contributor to Django—this is my best shot yet. -- You received this message because you are subscribed to the Google Groups "Django developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/C
