Re: On returning appropriate subclass instances when querying a model with subclasses

[Carl Meyer]

On Aug 23, 9:14 pm, Andrea Zilio  wrote:
> So the answer to the question "Can I get the right instances with one
> single and only DB query" seems to be:
> "No, you cannot do so without touching the Django orm code".
>
> Am I right?

Actually, I think the ORM's extensibility mechanisms are sufficient
that you could provide your QuerySet.select_subclasses() method, which
would perform a single query with n LEFT JOIN's, with custom QuerySet/
Query subclasses. Writing it would certainly require a non-trivial
level of familiarity with the ORM internals, and you might have to use
private APIs - but I don't think you'd have to modify Django code.

Carl
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: An issue with ticket system

[Carl Meyer]

On Sep 18, 6:48 pm, Alex Gaynor  wrote:
> If you press the original format button you will see that the patch is
> still there, trac just can't display all diffs (it's a very special
> piece of software).

Specifically, if you're obsessive about avoiding minor breakages, you
can try to avoid the "No newline at end of file" marker in your diffs;
pretty sure that's what causes Trac to choke.

Carl
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Revisiting #8245: admin.py errors under runserver cause app to disappear

[Carl Meyer]

Hi all,

Today I filed #11957 [1], and because it is questioning an earlier
bugfix, I was asked to bring up the issue for discussion here.

Under the dev server, if an error occurs while importing an app's
admin.py, the error will show up on the first request, and then
disappear. On all subsequent requests (until the dev server is
restarted), the admin will load normally, but without that app or its
models. Even if the admin.py is fixed, the app will be missing until
the server is manually restarted. I've observed the confusion this
causes to beginners working through part 2 of the tutorial: they learn
that under most conditions the development server does not require a
restart, and when they hit this situation there is no clue that
restarting the dev server is the necessary fix.

This confusing behavior is a consequence of the fix chosen for #8245
[2] and applied in r9680 [3]. In order to avoid spurious
AlreadyRegistered errors masking the real error in an admin.py, a
global LOADING flag was introduced that causes admin.autodiscover() to
bail out if an earlier run did not finish successfully. The unintended
side effect is that once autodiscover() has encountered an error in an
admin.py under the dev server, it will perpetually bail until the
server is manually restarted (runserver auto-reloading does not help
here, because an admin.py that has failed to import is not in
sys.modules, so fixing the error in it doesn't trigger a restart).

There was an alternative patch attached to #8245 by jarrow that
instead rolls back all changes to the admin's model registry if an
error is encountered during autodiscover (thus avoiding spurious
AlreadyRegistered exceptions, but without the need for a global flag).
If I roll back r9680 and apply jarrow's patch instead, both #8245 and
the issue I describe above are solved.

Can anyone shed light on why jarrow's patch was rejected for #8245 (no
reasons are provided in the ticket)? Is there any reason not to switch
to that approach now in order to get more predictable behavior from
admin.py errors under runserver?

Thanks,

Carl

[1] http://code.djangoproject.com/ticket/11957
[2] http://code.djangoproject.com/ticket/8245
[3] http://code.djangoproject.com/changeset/9680
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Re: Revisiting #8245: admin.py errors under runserver cause app to disappear

[Carl Meyer]

Anyone have thoughts on this? It would be really nice to get this
fixed so admin.py errors don't break the runserver, and I think the
updated patch on #11957 accomplishes this without any breakage.

Carl

On Sep 28, 4:56 pm, Carl Meyer  wrote:
> Today I filed #11957 [1], and because it is questioning an earlier
> bugfix, I was asked to bring up the issue for discussion here.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---

Deleting related objects in the admin: patch for #6191

[Carl Meyer]

Hey all,

During the Chicago sprint a few weeks ago I put together a patch for
#6191: the admin's "the following related objects will be deleted"
list in some cases omits items that actually are deleted (oops!). The
admin has its own routine for collecting related objects for
notification, separate from the ORM code that collects the objects for
deletion. This situation seemed ripe for further discrepancies down
the road, so rather than just fixing the immediate bug in the admin
routine I opted to make the admin notification reuse the ORM's
collect_sub_objects() method, so the same logic that is used to
actually delete the objects is used for the notification.

I'd love to get this fix in for 1.2 if possible, since it involves
silent loss of data. It's a bug fix, so can go in post-beta; but it's
also a relatively large patch, so it seems like getting more eyes on
it sooner rather than later would be good. Patch works well for me, I
added a number of new tests and tests are passing (well, two unrelated
admin_views tests currently fail for me with or without the patch).
Anyone up for taking a look?

Thanks,

Carl

[1] http://code.djangoproject.com/ticket/6191

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Deleting related objects in the admin: patch for #6191

[Carl Meyer]

On Jan 26, 7:24 pm, Russell Keith-Magee 
wrote:
> I'll put it on my list. It will probably still be a 'post 1.2-beta'
> activity, but I'll try to look at it at the start of the bugfixing
> phase.

Great, thanks. Looking forward your feedback.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: dbsettings, and user configurable app settings

[Carl Meyer]

On Feb 27, 1:53 pm, stuff4ash  wrote:
> b) easy way to register, override, configurations for apps.
> a settings.py in an app folder that is "magically" loaded the same way
> other things are would suffice, but there has to be a way to:

I've never understood the desire for more magical behavior here. My
apps have a settings.py that imports django.conf.settings and provides
any necessary defaults for unset settings; the rest of my app imports
from there. Works great, easy to look and see what settings my app
consumes and what defaults they have. I don't see what advantages a
more complex system would provide. Example here:
http://bitbucket.org/carljm/django-adminfiles/src/tip/adminfiles/settings.py

> -override and prioritize settings, as sometimes you have to override
> things.

I'm not sure what this means. In my setup, project-level settings
always override app-level defaults; which is as it should be, I think.

> -mark defaults, get defaults from other apps, and mark some settings
> as obligatory.

I use getattr(settings, 'SOME_SETTING', 'default') for defaults. If I
want to get a default from some other app, I can import it and do so.
If a setting is obligatory, I raise an import-time
ImproperlyConfiguredError.

> i have a django mingus blog now. when i upgrade mingus, and the many
> many many apps it has, how do i know without reading all the docs, if
> there is a new setting that i should set or change?

If it's a required setting, you should get an error when you try to
run the project without it (I sure hope you're testing the upgrade
somewhere other than your production server...). If it's optional, why
should you be bothered with it unless you already know you want it? If
you really want to know all the available settings, that's what docs
are for.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: dbsettings, and user configurable app settings

[Carl Meyer]

On Mar 1, 9:40 am, stuff4ash  wrote:
> I am not necesary advocating magic. But the need for a more robust (or
> lets say, any) configuration management exists for many.

"Configuration management" is a fuzzy phrase that can mean many things
to many people. I think it'd be more helpful to focus on specific
problems and possible solutions. My critique was specifically of the
suggestion you made for an automatic import of app-level settings.py,
which introduces magical behavior for (AFAICS) no gain. I've certainly
had the need for something like dbsettings or livesettings, though
I've often addressed it with a separate app/model rather than trying
to shoehorn django.conf.settings into a role it wasn't intended for. I
also agree that for projects with many reusable apps (which describes
all of mine), split settings are less headache to manage; I like
Transifex' conf-directory solution and have proposed it for Pinax.

It's not that the status quo is perfect and no improvement is
possible; just that specific suggestions with concrete use-cases are
more useful than handwavy claims about the need for some major new
subsystem whose purpose and specific features are not clear.

> I am very familiar with your setup, and its the best one in use I have
> seen. but it has several shortcomings.

So what shortcomings are relevant to you?

> one of the few times that I wished django was more plonish.

Which means what, in this case?

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Long-running tests

[Carl Meyer]

On Apr 17, 2:58 am, Paul McMillan  wrote:
> For the particular test case in modeltests/fixtures/models.py, we
> might be able to improve test time by tearing the fixtures down after
> verifying them, rather than doing a full flush. This will be easier to
> do after converting from a doctest to a unittest.

Unless I'm missing something, converting from doctest to unittest
itself will solve the problem (at least for transactional databases),
as Django's TestCase will roll back the transaction and make flush/
teardown unnecessary.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Delete order of generic relation models

[Carl Meyer]

Hi Ryan,

Moving discussion here off the Trac ticket.

On Apr 25, 5:44 pm, Ryan  wrote:
> A decent workaround is for my post_delete hook to pass
> force_update=True and catch DatabaseError when saving the article, in
> case it has already been deleted.  But I think it would be nice if
> django just deleted the generically-related objects first so I didn't
> have to worry about this.
>
> I've opened a ticket and submitted a patch so generically-related
> children are deleted before their parents:
>
> http://code.djangoproject.com/ticket/13416
>
> Does anyone see any problems this could cause?  Thanks.

On the ticket I said: 'It seems like "the right thing" in your
specific use case, but it causes much more serious problems in other
cases. Making the change you propose significantly increases the
likelihood of a circular dependency in deletion if the generically-
related objects have any other relations. The purpose of the
"nullable" arg is to tell the deletion algorithm whether it is
_possible_ to delete these objects later, in order to avoid circular
dependency. It is, in fact, possible to delete generic relations
later; therefore setting nullable True is the right thing to do.'

And you said: 'How hard would it be to check that the generically-
related objects have no other relations before setting nullable=True?'

To which the answer is, "somewhat complicated (there are a variety of
kinds of relationships it might have, as you can see in the
implementation of _collect_sub_objects), but possible."

But it's the wrong question. The issue isn't "how hard would it be to
make this little change in the current implementation," because what
you're really asking for is a new contract from Django regarding
cascade-deletion order to simplify your signal-handling code. So the
real question is "should Django introduce a brand-new guarantee for
end-user code regarding cascade-deletion-order?" Remembering that this
contract then has to be maintained in future implementations of
cascade deletion which may be quite different from the current
implementation (see #13067 and #7539).

Even the current behavior with non-nullable FKs deleted before their
parent is not something your code should rely on; it's an undocumented
implementation detail in order to make the current version of cascade-
deletion work on databases that enforce referential integrity. As Russ
notes on the ticket, if you set up an FK cycle you can quickly see
Django violate that ordering (as it has no choice).

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Class based generic views in 1.3?

[Carl Meyer]

On Jun 2, 6:20 pm, Ben Firshman  wrote:
> Yeah, this idea came up at the sprints, but it's a little too magic for my 
> tastes.

I dunno... is __new__ really more magic than having a __call__ method
that magically copies the instance you call it on? That certainly
breaks my expectations.

Django doesn't shy away from "magic" in other areas where it makes for
a usable API. The things that happen to model and form fields
certainly violate the initial expectations of a Python programmer; but
it turns out that most people are happy to write readable models and
don't care what's happening under the hood, and those who do care can
learn a little something about metaclasses. Seems to me that __new__
is a relatively beautiful solution to a thorny problem: it lets people
use class views in all the same ways they use function views while
maintaining thread-safety, it maintains the "callable that returns
HttpResponse" contract, and most people won't ever see or care about
the "magic" that makes it work.

+1 for __new__.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposal: Improvements for django.forms

[Carl Meyer]

Hey Petr,

On Jun 3, 11:32 am, "petr.marhoun"  wrote:
> OK, more details: If fieldsets would be defined, fields would be
> concatenation of fields from individual fieldsets. So methods
> as_table, as_ul and as_li and "for field in form" would work as now.
> But if you want to really use fieldsets, you have to iterate fieldsets
> in your templates.

Great! I was also planning to propose form fieldsets but hadn't gotten
around to it yet. I have an external app, django-form-utils[1], that
already implements form fieldsets in very much the way you propose
(although with a few differences). The app has seen some use and
people seem happy with it; I would be happy to port this
implementation to a core patch, with a few changes.

One thing form-utils does differently from your proposal is that it
defines each fieldset as a two-tuple (like admin fieldsets), with a
name as the first element and the dictionary of additional info as the
second element. One advantage of having a fieldset "name" is that it
also allows accessing fieldsets individually in the template, for more
fine-grained layout control (thanks Rob Hudson for adding this to form-
utils).

Just to give a full summary of the fieldsets API I would propose
(which is not exactly the same as what's currently in form-utils; I'd
like to correct some mistakes :> ):

class PersonForm(forms.Form):
name = forms.CharField()
age = forms.IntegerField()
nickname = forms.CharField()

class Meta:
fieldsets = [('main', {'fields': ['name', 'age']}),
 ('extra', {'fields': ['nickname'],
 'legend': 'Extra info',
 'classes': ['aside']})]

(If "fieldsets" is provided for a ModelForm, it takes priority over
any "fields" or "exclude" definitions: the fields included in the
fieldsets are the ones included in the form, whether you iterate over
fieldsets or just fields.)

Iterating over the form directly yields BoundFields just as it does
now:

{% for field in form %}

Iterating over form.fieldsets yields Fieldset objects. Fieldset
objects can be iterated over to yield BoundFields:

{% for fieldset in form.fieldsets %}
{% for field in fieldset %}

Fieldsets can also be accessed individually by name on form.fieldsets:

{% for field in form.fieldsets.main %}

And fields can be accessed directly by name on the "fields" attribute
of a Fieldset (to avoid clashes between form field names and other
Fieldset attribute names):

{% form.fieldsets.main.fields.age %}

Fieldset objects also have the attributes "name", "legend", and
"classes" (which is still an iterable, not collapsed to a string, so
template authors can check {% if "blah" in fieldset.classes %} if
needed.

I am proposing the "classes" option, again parallel to current admin
fieldsets, in lieu of any additional options like "attrs" or
"template". There's a slippery slope where almost any info _could_
theoretically be provided for the template author by the Python form
author; I think it makes sense to keep it to a simple list of class
names, as the most basic, generic way to distinguish "types" of
fieldsets. I think pretty much any use case can be handled via this
mechanism, since you can write custom template tags if needed to
provide reusable display logic. (I'd be open to the "attrs" option,
but am quite opposed to "template": form definitions seem like
entirely the wrong place to embed template names.)

I would of course assume that a patch for fieldsets in forms would
also involve porting the admin's fieldsets to use this feature.

FWIW, I think it would be much preferable to separate discussion of
orthogonal features into separate threads, even if they are all form-
related. Makes it much easier to follow the discussion.

Thanks!

Carl

 [1] http://bitbucket.org/carljm/django-form-utils

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposal: Improvements for django.forms

[Carl Meyer]

On Jun 3, 8:57 am, Russell Keith-Magee 
wrote:
> 5. Parameters "template" and "attrs" for forms, formsets, fieldsets,
> forms, (bound) fields and widgets
> 6. Support for (X)HTML5 and legacy HTML
>
> I strongly suspect that these two may be covered by something I
> discussed with Jacob at DjangoCon. Broadly, the proposal is to
> deprecate the {{ field }} and {{ form }} approach to template
> rendering in favor of a template tag {% render field %} and  {% render
> form %}; these template tags would allow you to customize the way that
> individual fields, fieldsets and forms are rendered, including
> controlling doctype compliance.
>
> If you want to override the way fields or forms are rendered, you will
> be able to subclass the base render template tag, override the
> rendering behavior, and re-register the tag (or register it under a
> different name).

Great! This one is a high priority for me (my designer keeps harping
on me about that markup he can't control), and I am happy to help make
it happen. Looking forward to seeing the results of your tinkering. If
those tags could optionally take a template name as argument, I think
that would cover the vast majority of customization use-cases without
any need for subclassing or re-registering the template tag.

Personally I'd love to also see the default widgets rendered by
templates, so a normal template override of a predictably-named
builtin template is all that's needed to customize the rendering of a
given widget. But there might be performance issues there if the
caching template loader isn't used, and that isn't enabled by default.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposal: Improvements for django.forms

[Carl Meyer]

Hi Petr,

On Jun 3, 5:48 pm, "petr.marhoun"  wrote:
> I have some notes to forms-utils:
> - The code says: "if legend is None: legend = name". I dislike it - I
> want to have possibility not to set legend. I think that if legend is
> not set it should be "None" in fieldset.

Yes, this is one of the form-utils mistakes that I would want to
correct in the Django-core version :-) Note I didn't mention anything
about auto-setting legend in the proposal here.

> - It seems that it is not possible to say if form has fieldsets
> (fieldsets method even construct fieldsets if meta attribute is not
> set) - I think it is useful to know it in templates.

Yes, form.fieldsets should behave appropriately with __nonzero__ so
you can say {{ if form.fieldsets }}.

> Here I would prefer to be minimalistic. For example, widgets have
> "attrs", not "classes", so it is more consistent to use "attrs". And I
> have proposed "attrs" in another proposal because I think that there
> is general problem - how to customize parts of form rendering without
> writing of whole templates? So I would start with items "fields" and
> "legend", other items could be added later.

That's fine with me. I don't personally have a strong opinion about
"classes" vs "attrs."

> It is not so simple. Our proposals are that "fields" are list of field
> names. But in admin "fields" are list of items - and each item can be
> field name or list of field names. So I think that generally it is not
> possible to port admin's fieldsets to these proposals' fieldsets -
> proposals' fieldsets are subset of admin's fieldsets. (And syntax from
> admin seems to be too complex for Django core.)

I don't think it's simple, but I do think it's doable to build the
admin functionality on top of the core functionality as outlined
above, without duplicating code or bringing extra admin complexity
into core. It probably will require the admin code to have pretty
intimate knowledge of the core code, but that's nothing new :-)

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposal: Improvements for django.forms

[Carl Meyer]

On Jun 3, 5:48 pm, "petr.marhoun"  wrote:
> - It seems that it is not possible to say if form has fieldsets
> (fieldsets method even construct fieldsets if meta attribute is not
> set) - I think it is useful to know it in templates.

I spoke too quickly here: this is a relevant API question. In form-
utils, if you define no fieldsets it builds a default one for you
automatically with all the fields in it, so len(form.fieldsets) is
always at least 1. The advantage is that this allows templates that
expect fieldsets to always work, but I'm not sure it's actually a good
idea. Open to thoughts.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: LOGIN_URL, LOGOUT_URL, LOGIN_REDIRECT_URL and SCRIPT_NAME.

[Carl Meyer]

On Jul 7, 7:11 pm, Graham Dumpleton 
wrote:
[snip]
> web application, to be a well behaved WSGI citizen, should honour
> SCRIPT_NAME setting as supplied by the server, and ensure that ways
> are provided such that everything in the users code, including
> configuration, urls or settings files, can be expressed relative to
> the URL that the application is mounted at, thereby avoiding as much
> as possible any need for a user to modify their code base when
> deploying to a new environment at a different location in URL
> namespace.

IMO one root problem here (which has been discussed before) is that
settings.py conflates "deployment config" with "application config"
and thus tends to lead to fuzzy thinking about which is which (which
causes problems especially for larger organizations with separate
teams for development and ops). Application config should not have to
change between different deployments of the same codebase; deployment
config almost certainly will.

The status quo makes LOGIN_URL (and friends) themselves a mishmash of
deployment config and application config; i.e. the initial
"application mount point" portion is deployment config, and the rest
is clearly application config (it would only change if the URLconf
changes). This seems like a bad thing to me: the ops team shouldn't
have to understand the internal layout of the application's urls in
order to deploy it correctly.

I would rather see LOGIN_URL et al as purely application config, in
which case they should automatically respect the URL mount point. The
problem, as Russ points out, is that if this mount point is learned at
runtime from the WSGI request, it adds complication to code outside
the request cycle that needs to know where the application is mounted.
But that exact same problem already exists for the site hostname!
Django punts this issue to the developer and/or contrib.sites.
Wouldn't it be most sensible to treat the URL mount point similarly to
hostname, since they are really just two pieces of the same data: the
deployed root URL of the application? (Practically speaking, this
could mean a new field in contrib.sites.Site, or allowing/condoning
"example.com/something" as a value for the domain field, and extending
RequestSite to do the same by checking SCRIPT_NAME).

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: LOGIN_URL, LOGOUT_URL, LOGIN_REDIRECT_URL and SCRIPT_NAME.

[Carl Meyer]

Hi Russ,

On Jul 10, 8:24 am, Russell Keith-Magee 
wrote:
> On Sat, Jul 10, 2010 at 12:49 AM, Carl Meyer  wrote:
> > Wouldn't it be most sensible to treat the URL mount point similarly to
> > hostname, since they are really just two pieces of the same data: the
> > deployed root URL of the application? (Practically speaking, this
> > could mean a new field in contrib.sites.Site, or allowing/condoning
> > "example.com/something" as a value for the domain field, and extending
> > RequestSite to do the same by checking SCRIPT_NAME).
>
> I'm not sure I understand exactly what you're describing here. Lets
> say we modify Site, and I have the mount point data described in my
> Site.objects.get_current() instance; are you proposing that anywhere
> that you use LOGIN_URL, we should be prepending site.mount_point (or
> whatever the bikeshed looks like)?

Actually, on further reflection I retract this suggestion entirely.
Conceptually the host domain and the script prefix could be considered
part of the "same thing," but practically speaking they are quite
different, given the need for every internal link URL to include the
script prefix, but not the host domain.

Re-reading this thread, the named URL pattern solution seems eminently
reasonable:

- LOGIN_URL and friends can all be set to either the name of a URL
pattern, or to an absolute URL for backwards compatibility (this could
be documented as deprecated, and perhaps also raise a
PendingDeprecationWarning?).
- document that set_script_prefix is required for scripts outside the
request-response cycle that want to generate URLs, if they have a
script prefix that isn't / (since get_script_prefix returns / by
default).

As Alex points out, the needed code is already present in the
redirect() shortcut view, though if we want absolute URLs to raise
PendingDeprecationWarning we wouldn't be able to just reuse that view.

If this seems OK I'm willing to reopen #8906 and attach a patch that
does this.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposal: Revised form rendering

[Carl Meyer]

Hi Russ,

First of all, thanks very much for this proposal! Form rendering has
been a major pain point for us (thus the existence of
django-form-utils), and improving it is tops on my 1.3 wishlist. I
will also be at DjangoCon and eager to sprint in this area.

Django has a really good template system. With template inheritance,
includes, and overrides, it allows the designer I work with a
remarkable amount of flexibility in producing exactly the HTML he
wants, without repeating himself. He's not a programmer (and thus
doesn't follow django-developers), but he works full-time with the
Django template system and loves it. His (frequent) complaints to me
about Django form rendering are always the same: "Why is this markup
generated deep inside Python code, and why can't I override a template
to fix it? I thought we'd moved on from PHP!"

I think it's possible to solve the problems you're aiming at here
without so much new Python machinery, just by delegating more to
templates (as Eric and Danny have already mentioned briefly). HTML is
the native language of web designers, and I'm firmly convinced that
the less we hide their HTML behind Django-specific abstractions, the
more usable the system will be.

I'll first outline in brief what I think a fully template-oriented
approach might look like, address some possible concerns with it, and
follow that with some specific comments on your proposal.

A fully template-based approach to form rendering
=

You have a form object in your template. You render it using a simple
"render_form" filter:

{{ form|render_form }}

"render_form" just renders a default form rendering template (at some
sensible template path, the bikeshed could be painted
"django/forms/default.html" or some such), with the form object itself
passed into the context.

This template just does the usual iterating over form fields, and the
default one provided by Django could mimic form.as_table(). That
default template could of course be overridden for any given project,
and the "render_form" filter also accepts as argument an alternative
template to use for a particular form:

{{ form|render_form:"path/to/my/form_template.html" }}

So far, this approach is workable in current Django. This is what
django-form-utils does, and what we do in all our projects.

The next step is to have the default widgets render themselves using
templates found at a standard path
(e.g. "django/forms/widgets/textinput.html"). This template would be
passed the boundfield and widget, and so would have access to all the
information it needs to render itself correctly.

There would also be a "render_field" template filter, that would
optionally accept an arbitrary widget template path:

{{ form.fieldname|render_field:"path/to/my/widget_template.html" }}

Normally this would only be used inside a form rendering template, but
if you wanted to skip the "render_form" filter and render your form
directly using this filter repeatedly (or inside a loop), that's
possible too.

All of the problems you identified as targets are now trivially
solvable, just by giving the designer direct access to the template
system, without all kinds of magical abstractions in between. Form
layouts can be fully controlled in templates. Widget markup can be
fully controlled in templates (and of course can validate to whatever
doctype you want it to, without any special machinery for that).

The last target you identified, allowing widget-related JS to be
rendered in a block at the bottom of the template, can be solved
simply with another filter and set of templates:

{{ form|render_form_js }}

Which would look for templates such as
"django/forms/widgets/js/textinput.html" for each widget in the form,
rendering nothing for a widget if that template doesn't exist. This
filter could also optionally accept a path to a template containing
form-wide JS. These templates would probably just contain script tags,
either linking to an external JS file or containing inline JS. (IMO,
collating these into a single script tag is out of scope and better
handled by a dedicated solution like django-compressor; though the
possibility of referencing a single form-wide JS template opens the
door to other possible solutions).

Advantages
--

- As mentioned, HTML is the native language for a web designer: this
  exposes it directly and simply.

- Similarity to existing approach: this is a less radical
  change. Rather than introducing a new template tag that does all
  kinds of magic behind the scenes, and several new abstractions
  (renderers, chrome) to boot, this just leverages existing, proven,
  well-understood tools, and the resulting templates look more similar
  to existing form-rendering templates.

- No need for special-casing things like doctypes, adding attributes
  directly to a widget-generated HTML input tag. Everything is in
  templates, everything is equally customizable via that familiar
  route.

Possibl

Re: Proposal: Revised form rendering

[Carl Meyer]

Hi Russ,

On Jul 13, 12:11 pm, Russell Keith-Magee 
wrote:
> I have exactly 0 mad skillz as a front end guy, but I know that rich,
> pretty ajax widgets et al are necessary for a good user experience. My
> goal, as a backend guy with limited frontend skills, was to find a way
> to get the people who *did* have frontend skills to be able to package
> their widgets in such a way that I could reuse them.

Indeed, I share that goal.

> Now, the Media framework has been largely a flop in this regard
> (although we may be able to use bits of it in a support role in this
> new form work). However, my desire for a widget library is unquenched
> :-)

>From my perspective as a programmer, I think the Media framework is
brilliant. It just has one gigantic Achilles heel: it requires
touching Python code.

Honestly, I think if we're serious about wanting a vibrant ecosystem
of Django-enabled frontend widgets, design goal #1 should be that
creating a widget requires touching zero Python code. And I think
that's possible to achieve.

> My concern about doing this entirely in templates is that it makes the
> process of sharing a widget library a little more difficult. If it's
> just code, then it just needs to be in the PYTHONPATH. If there are
> templates too, then the templates need to be somewhere that they can
> be accessed.

So I think this is very much a programmer's perspective, almost
humorously so :-) "Just code" may seem easy to you, but from the
perspective of at least one experienced Django-template designer I've
talked to, being able to do it entirely in templates makes the idea of
sharing a widget library conceivable, as opposed to almost impossibly
intimidating if its "just code."

The app_directories template loader is installed by default. Getting
templates "somewhere they can be accessed" means creating a directory
on the PYTHONPATH with an empty models.py, an empty __init__.py, and a
"templates/" subdirectory, then putting that in INSTALLED_APPS. That's
it. How is that harder than following exactly the same steps for an
app that includes Python code? From the perspective of a non-
programmer, it's much easier.

> There's also more to a widget in this context than just the template
> -- for example, there's nominating the javascript triggers that the
> widget requires.

Yep, I had a lengthy conversation with Idan in IRC yesterday about how
to approach this. I touched on it in my proposal above briefly (it can
still just be handled with templates), but didn't discuss details like
the fact that a given widget's "JS/media" template should only be
included once on a page, regardless of how many times that widget was
used on the page, even in multiple forms. Which means that the
"render_form_js" filter probably actually needs to be a tag, so it can
accept an arbitrary number of form objects and render the widget JS
once for every widget in every form.

> I'm not saying that these problems can't be overcome -- just that it's
> a complexity that I want to make sure is covered.

Indeed. I think they can be overcome, but I fully agree that the
template-based proposal needs more fleshing-out to make sure it covers
all these use cases adequately.

> The filter syntax doesn't hit the 'render the form completely standard
> except for this one change on this one field' use case. However,
> there's nothing about the 'template' aspects of your proposal that
> need to be bound to filters; they could be equally accomodated in a
> template tag.

I agree that tag vs filter is an orthogonal question, and I'm not
opposed to converting them to tags. I agree that it may be necessary
in order to add an extra optional argument or two to handle some use
cases.

The default-except-for-this-one-field use case is actually quite
doable under the proposal as written; you'd just create a derivative
form-rendering template that inherits your default one, and just
override a small block of it to add a bit of conditional logic to
render a certain field differently, otherwise deferring to
block.super. That's DRY and reasonably easy, but I if render_form were
converted to a tag it might be possible to add some optional arguments
to it to make it even easier.

The other improvement I would probably want to make to the API as I
first proposed it would be a way to point a given form-render or field-
render at a given directory of widget-templates, with the individual
template still picked out by widget-name.

> My manifestation of this problem is slightly different -- it's the
> issue of how to ship a widget library that can be used anywhere. I
> suppose one argument here is that you just need to advertise your
> chrome with doctype support notes.
>
> Alternatively, since this is a template language, you could always
> introduce an {% if %} block and render conditionally on doctype...

Exactly. I can imagine a number of different ways a reusable widget
library might approach this problem: simply advertise support for a
certain doctype (don't di

Re: Proposal: Revised form rendering

[Carl Meyer]

Hi Tai,

On Jul 13, 8:42 pm, Tai Lee  wrote:
> I wouldn't like to see widget libraries being packaged up with
> templates including conditional templates based on doctype. This seems
> messy, not friendly to override in your own templates, and effectively
> combining what should be multiple separate templates into one.

I agree with you that separate template trees is much better than
conditionals. Also, if using conditionals, where does the value come
from that the conditional switches on? Part of my idea is that there
should be no need for some magic "use this doctype" value to be passed
around.

> Wouldn't a mechanism that allows the path from which templates are
> loaded to be changed benefit both widget libarary authors and the
> designers who are implementing those widget libraries? A widget
> library author would include and advertise templates for each doctype
> they support, e.g. "html4/mywidgetlibrary/forms/datetime.html" and
> "xhtml1/mywidgetlibrary/forms/datetime.html", then the designer would
> use:

Your {% templatedirs %} tag is an intriguing idea in its own right,
and certainly could intersect with template-based form rendering, but
I'd prefer to keep the two proposals orthogonal.

> This gives a lot of flexibility and allows widget library authors to
> support multiple doctypes or flavours (not just specific to doctypes),
> and designers to pick and choose (or even override at the project
> level) the widget templates according to their doctype or preferred
> flavour, without Django needing any specific machinery for doctypes or
> designers needing to edit any Python code.

I totally agree with this. I think very similar results can be
achieved in the form-rendering system just with an optional parameter
to a "form/render_form" tag that specifies a "base path" for finding
template widgets for this form render, as I mentioned above. I think
it's better to keep the form-rendering proposal specific to forms, and
evaluate {% templatedirs %} separately as a feature proposal on its
own merits.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposal: Revised form rendering

[Carl Meyer]

On Jul 14, 4:15 pm, SmileyChris  wrote:
> We also need d) Media hooks for a single widget - whether this can be
> done in only the template layer is a tricky problem...
>
> I'm still struggling with how you'd be able to render all the media at
> the top of your template for the form. I guess you'd have to define
> any field widget overrides in that block, before rendering the form
> media.

Yeah, this is tricky. For JS generally it shouldn't be rendered until
the bottom of the template, for page-speed reasons, but the reverse is
true for CSS. I don't think there's yet been a proposal in this thread
for how to enable rendering widget media before the form is rendered,
both proposals so far would have you specifying/overriding chrome (or
widget templates) only at form-render time.

Obviously it'd also be good to be able to split CSS from JS, like the
Media framework allows.

One half-baked idea: what about annotating the form object itself with
information about customizations you've made: paths to form-rendering
templates and widget templates? So you could use the same
customization syntax when rendering the form or rendering the form
media, whichever you do first, and it would remember those
customizations for later?

This introduces a little more cleverness than I'd prefer, but if the
only other option is repeating every customization twice... I'm also
not sure how this would interact with the idea that the "form media
render" tag should accept multiple form objects, so even if you use
the same widget/chrome in multiple forms on a page, its media still
only gets rendered once.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: memcached-based-cache - timeout=0 does not work as intended by memcached

[Carl Meyer]

On Jul 23, 12:45 pm, Jacob Kaplan-Moss  wrote:
> On Fri, Jul 23, 2010 at 8:17 AM, Tobias McNulty  
> wrote:
> > The only concern in that ticket seems to be that 0 means different things
> > for different cache backends.
> > There may have been some effort towards making them all behave the same when
> > 0 is passed.
> > Personally I prefer the approach of not messing with the value at all, and
> > passing it straight to the configured cache backend.  You don't want Django
> > messing with your .extra() or .raw(), so why should it try to magically
> > alter the parameters you pass to your cache backend?
>
> I agree.

It's not obvious to me why .extra or .raw are the appropriate analogy
here, instead of the rest of the ORM API, which does attempt to
present the same semantics regardless of backend.

The argument that "you know what cache backend your project uses" does
not apply to the significant ecosystem of caching-related reusable
Django code: johnny-cache, cache-machine, cachebot, and more.

Last week I uploaded a patch to #6447 that maintains cross-backend
parity of allowed cache keys (without any performance hit on
memcached) by making sure the other backends will reject the same keys
memcached will reject. You approved this approach on IRC, Jacob, so
given the discrepancy I'm just curious whether it is in fact a design
goal to have semantic parity between the backends shipped with Django.

If it is, the memcached backend should probably still be changed to
the native behavior (for speed reasons, at least; the less code in the
memcached backend, the better), but it may be worth considering
emulating that same behavior in the other backends.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: memcached-based-cache - timeout=0 does not work as intended by memcached

[Carl Meyer]

On Jul 26, 10:02 am, Tobias McNulty  wrote:
> If such reusable code needs to cache things forever, which sounds perfectly
> reasonable to me, I'd still rather not co-opt "0" to mean "forever" in all
> cases.  Each backend that supports caching with no timeout could easily
> offer a class attribute, such as "TIMEOUT_FOREVER", that indicated how to
> set the timeout in these cases.

I'm not sure I see the problem with co-opting 0, since it already has
(or should have) that meaning on the most commonly-used backend, and
doesn't really have any other sensible meaning as a cache timeout
value. But the proposed class attribute is fine too.

> Rejecting a particular subset of cache keys is also not the greatest analogy
> to co-opting configuration values. :-)

The common thread, of course, is making it possible to write reusable
caching code without special-casing particular backends.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Model-level validation

[Carl Meyer]

On Thu, Sep 29, 2022 at 6:19 PM Curtis Maloney  wrote:

> On Thu, 29 Sep 2022, at 14:29, Aaron Smith wrote:
>
> Why doesn't Django validate Models on save()?
>
>
> The short answer is: backwards compatibility.
>
> Model level validation hasn't always been a thing, with Django initially
> depending primarily on Form validation.
>
> Since it hadn't _always_ been there, it wasn't possible to introduce it,
> enforce it, and not break most apps out there.
>
> There was so much code written that generally assumed it could call
> `save()` and not have to catch validation errors.
>
> For what it's worth, I'm all in favor of making it run on `save()` ...
> updating the documentation and preparing the community is going to be a
> mammoth task, however. A safe course through this will take some very
> careful planning.
>

Another factor that should be considered is that the Django ORM gives you
plenty of ways to update your database (eg bulk updates on a queryset) that
clearly cannot materialize and validate every object in the queryset. So is
it better to consistently say “the ORM doesn’t validate, forms do,” or
better to say “the ORM will validate on Model.save() but not in various
other cases, so you still can’t really rely on the ORM to enforce model
validation invariants consistently.”

Carl

>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAGLapGA7NCkNrBNe7LQmTOzbj%2Bpa3xm7_tEPnBa7eZbUsq1xGg%40mail.gmail.com.

Re: load template tag library

[Carl Meyer]

On Nov 16, 12:40 pm, ptone  wrote:
> On Nov 16, 1:12 am, Roald  wrote:
> > Can anybody explain why template tag libraries are loaded from
> > *inside* a template? The more I work with them, the more I get the
> > feeling that specifying which template tags are available in a
> > template should be specified in the view-code (or more general: the
> > thing that loads/renders the template).
>
> Such as a TemplateEngine...?
>
> This seems a good candidate feature 
> for:https://code.djangoproject.com/ticket/17093

I fully agree with Luke, Stephen, and Russell that {% load %} tags in
the template are how templatetags should be loaded and made available
for use in that template, simply for reasons of clarity and
explicitness.

I do, however, think that the tight coupling we currently have between
INSTALLED_APPS and "what templatetag libraries are available for
load", while a reasonable default for an integrated framework, is an
unfortunate restriction on use of the template system, which I hope
can be lifted by #17093.

And I also don't agree that efforts to use the template language in a
"sandboxed" way are doomed to failure: if you can instantiate a
template instance with an explicitly-limited set of templatetag
libraries (which #17093 would allow), and you can control exactly what
data is put into the template context, I think you _can_ make
templates safe for untrusted use. Obviously it requires care.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: load template tag library

[Carl Meyer]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 11/17/2011 02:51 PM, Luke Plant wrote:
> It's pretty easy to produce a DOS attack using only builtin template
> tags and filters, and a completely empty context e.g.:
> 
> {% for a in "xxx"|make_list %}
>   {% for a in "xxx"|make_list %}
>  {# etc #}
>   {% endfor %}
> {% endfor %}
> 
> I'm sure there must be other ways to do this, and there may well be
> different other types of flaws. I guess it depends on what you mean by
> 'safe', but we certainly haven't built the template system with this in
> mind.

Yes, good point. I'd expect there are a number of ways to do something
along these lines; you'd have to render the template in a thread with a
short timeout or something.

I personally wouldn't try to use the template language with truly
untrusted input; I might (and have) with
potentially-incompetent-trusted-user input.

I guess my real point is just that I think the core request, to be able
to lock down what template tag libraries can be loaded in a template, is
a reasonable one, and "just document the restriction!" is not sufficient
reason to reject it.

Carl
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk7FhHUACgkQ8W4rlRKtE2exUQCgiQIioRlS6gaX8gLbGvtJVEbD
ilcAoIGM4PbeegMwPZE5jnWT7CQyORtb
=iGOW
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: New feature: Defining fieldsets in form class (Ticket #17301)

[Carl Meyer]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi Mikołaj,

Thanks for your work on contributing to Django!

I am a bit skeptical about defining form fieldsets in Python code, as
they have no effect on server-side interpretation or validation of form
data, they are purely an HTML/presentational issue. I think that in
general we should be moving form presentation entirely into templates
rather than adding more presentation-related attributes into Python
code. There's already work begun on this via a Google Summer of Code
project (see #15667 and #16922); there are some barriers to overcome
(particularly performance issues with the template engine), but I think
it's still the right approach for the long term. And the more
form-presentation controls we add in Python code, the more difficult
this transition becomes (because we end up with two conflicting methods
of defining a forms layout).

However, I can see some justification for simple fieldset definitions in
the form definition, as a high-level way to group fields. This can allow
more flexibility in defining custom reusable form layouts (via
templates) that depend on having such field groupings. And it's clear
that others want this feature: I won't stand in the way of it if another
core developer wants to champion its inclusion.

However, I feel quite strongly that even if we add this, we should NOT
add the additional "classes" and "fields in a row" features that you
mention. Fieldsets themselves are defensible as logical "structure" of
the form - these additional features are getting into the arena of
trying to fully define HTML layout of a form in Python code, which I
think is fundamentally the wrong approach, and makes it more difficult
for non-Python-programmer template authors to work with Django forms.
Why should forms be an exception to the general rule that HTML and
presentation belong in templates?

I see the admin as the exception here, rather than as a model for how
the forms library should work. The admin's purpose is to create generic
customizable CRUD screens for arbitrary models; this is necessarily
going to involve defining the available axes of presentation
customization, and making it easy to do those particular customizations.
That's a very specific use case; it means trading off flexibility for
convenience in very simple customizations. That's an appropriate
tradeoff for the admin to make in building on top of django.forms; it's
not an appropriate tradeoff for django.forms, IMO.

In sum: +0 to fieldsets in Python code, -1 to classes and rows defined
in Python code.

Carl
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk7TwyAACgkQ8W4rlRKtE2eWiACgnT5afZXt7skGy66pIa4zDTrM
kKkAoOoM/VIvJNPFMk0eaROh2DYT7/gv
=SdfG
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Allowing models to influence QuerySet.update

[Carl Meyer]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 11/29/2011 02:58 PM, Kääriäinen Anssi wrote:
> """
> H, that is not ideal behavior. You mean QuerySet.delete() calls
> the signal for each deleted object, rather than doing a delete at the
> database level?
> """

These two things aren't mutually exclusive. The actual delete is still
done in one query per table, not one query per object. post_delete is
sent, but the delete() method on the instance is not called.

QuerySet.delete() has called the post_delete signal for each deleted
object ever since queryset-refactor was merged into trunk in Apr 2008,
at least. (I didn't bother trying to track it prior to that, it may well
go back much farther.)

Since the db-optimization doc that Luke linked was created in Jan 2010,
the claim there (that no model signals are sent by either
QuerySet.update() or QuerySet.delete()) has been inaccurate ever since
it was added.

> This might not be exactly accurate, but I think it goes something like this:
>   - Fetch all the to-be deleted objects (one query)
>   - Check if there are cascades for those objects, fetch the cascades (one 
> query per cascaded Model class?)
>   - Send pre_delete signals for all deleted instances
>   - Do the delete as one query for the to-be deleted objects, and then one 
> query(?) per cascade Model class
>   - Send post_delete signals

Yes, this is an accurate summary. Prior to r14507, it did quite a few
more queries than that (one query per related model class per
to-be-deleted-object, rather than just one query per related model class).

> Now, this is not that inefficient - but it would be a good
> optimization to NOT fetch the instances if there are no listeners for
> pre/post delete signals and there are no cascades (or all cascades are
> DO_NOTHING). Even if there are cascades, you could fetch just PKs of the
> to-be deleted models (even that is not actually needed, as you can use
> joins)

Yes, these would be possible optimizations (in the no-listeners case).
They would break the admin's delete-confirmation screen, so we'd have to
be willing to accept a regression in functionality there, or else
duplicate the delete-cascade logic in the admin (leaving open more
possibilities for the delete-confirmation screen to not tell the truth),
in order to make these optimizations.

In the case of many cascade-deleted objects, bulk delete performance in
1.3 is already much better than it was in any previous Django release.
If something even closer to the metal is needed (besides the ability to
write raw SQL deletes), I'd prefer to see new raw-bulk-delete API added
rather than QuerySet.delete() made less functional in order to be
faster. I think there might even be a ticket for that already...

Carl
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk7VXSwACgkQ8W4rlRKtE2eHugCfaC9OXnm4klDvrrdSUttZEknW
tVwAnRA2nqbcDjK3R4NUfnDb0GxmA1Vo
=jsy0
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Allowing models to influence QuerySet.update

[Carl Meyer]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 11/29/2011 11:32 PM, Anssi Kääriäinen wrote:
> I should not have said that this would be a good optimization, and
> used the possible optimizations wording. I haven't got any performance
> problems when using delete in 1.3. So from my point
> of view the above mentioned optimization is mostly a theoretical
> possibility. I did not mean the 1.3 version of deletion is bad at all.

No worries, I wasn't offended, just trying to be clear about the
situation ;-)

> If dependent object collection could be made a public API (with the
> possibility of fetching all dependencies to depth N, not just cascaded
> dependencies), it would be a nice feature for sure. If just the
> cascades part would be made public, it would be great, too. Showing
> "these objects will be deleted" confirmation page is not a feature
> only admin needs.

True enough. Making to-be-deleted collection a public API would just
require some documentation, I think - the API is already there and the
admin uses it.

Carl
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk7Wjj8ACgkQ8W4rlRKtE2fg5wCgqPcLgW0zr4yN8CkK+huEBoJf
YGoAn2x6TaO6Me6bb6yHK2v834LtkWIR
=nQ2M
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: PUT and post data

[Carl Meyer]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi Thibault,

Thanks for checking in on this ticket. It's helpful to get a clear
resolution rather than leaving things hanging in an unclear state.

On 12/01/2011 11:14 AM, Thibault Jouannic wrote:
> I'm looking for some news about the ticket #12635.
> 
> https://code.djangoproject.com/ticket/12635
> 
> Today, it's still impossible to easily access post data when sending a
> PUT request. Since I'm trying to build a restful web service using
> django, this is quite a problem.
> 
> It's been told in another thread to use the `request.raw_post_data`
> attribute, however I cannot think of a valid reason for why put data
> could'nt be has easily accessible as post data.

The valid reason is outlined by Malcolm Tredinnick in a thread linked
from that ticket [1], and by Ian Clelland just now. request.POST is a
special-case for a request body submitted with the
"x-www-form-urlencoded" or "multipart/form-data" content types. This is
common for POST because these content types are used by web browsers to
submit HTML forms.

However, web browsers do not submit PUT requests from HTML forms (and
they aren't likely to ever do so, as that proposed HTML5 feature was
removed [2]). Therefore, there's no reason to think form-urlencoded is
any more common than any other content type for PUT requests (in fact,
in my experience it's rare to find a web service that expects
form-urlencoded; JSON and XML are much more common). So there is no
reason to privilege form-urlencoded by adding a special case for it.

If you do want your web service to accept form-urlencoded or
multipart/form-data PUT requests, you can still use Django's QueryDict
and/or MultiPartParser to do the heavy lifting for you.

> I see in the ticket that the 1.4 milestone was deleted. Was the ticket
> abandoned?
> 
> I've seen a lot of discussions and tickets about rest, and I know it's
> a complex issue, but since I'm kinda new here, it's hard to know what
> is the current state of the problem.

The ticket was in "Design Decision Needed" state, which isn't very clear
- - it means the core developers haven't yet made a decision on whether it
needs to be addressed or not.

I've now closed the ticket wontfix, based on the above reasoning.
Hopefully that helps clarify its status.

Carl


 [1]
http://groups.google.com/group/django-developers/browse_thread/thread/771238a95ceb058e/
 [2] http://www.w3.org/TR/2010/WD-html5-diff-20101019/#changes-2010-06-24
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk7X5l0ACgkQ8W4rlRKtE2cAQACgmo9k/hCLtCK+fgvChJzqRSV7
9WQAoLox8Ok0EuqUSprxl0uG2e78zVpN
=gvqm
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Deprecate change pk + save behavior for 1.4

[Carl Meyer]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi Anssi,

On 12/02/2011 06:54 PM, Kääriäinen Anssi wrote:
> I think I will pursuit the immutable PK approach, and see how it
> works (back to square one). BTW are there -1 calls on this approach,
> or the pk change tracking in general?

I haven't been fully following this thread, but I will say that I'm not
yet convinced that the ORM behavior should be changed such that saving
an instance with a modified PK updates the row rather than saving a new
instance. The two foremost concerns:

1. Backwards compatibility. I think this was mentioned, but I didn't see
a clear plan put forward. We certainly can't just break existing code
that is using PK-modification or PK-clearing as a cloning technique.

2. Foreign keys. Do you plan to introduce a full update-cascade
framework similar to the delete-cascade one, to handle the case where
there are other models with FKs pointing to the model whose PK you are
about to modify? How will you handle databases that aren't able to
suspend constraint checking until the end of a transaction?

I could be convinced otherwise, but at this point my feeling is that the
current behavior is a reasonably good fit for the character of the
Django ORM: simple and (not quite anymore, but still almost) stateless.
For better or worse, a PK is what uniquely identifies a model instance
as corresponding to a particular table row. If you change the PK, your
instance no longer corresponds to the same row.

Carl
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk7ZhwgACgkQ8W4rlRKtE2ejtwCgkkH0+hX92NPQPFfhpJjkxB3B
UWEAn3ae30eKxRdkqWmqYYs4/ka4FY1G
=7VB1
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Deprecate change pk + save behavior for 1.4

[Carl Meyer]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 12/02/2011 07:29 PM, Kääriäinen Anssi wrote:
> At this point this is not the idea. The idea is to just disallow this 
> (assuming
> multicolumn PK firstname, lastname):
> 
> user = User(firstname = 'Jack', lastname = 'Smith')
> user.save()
> user.firstname = 'John'
> user.save()
> 
> Current behavior will leave Jack Smith in the DB, and save John Smith as
> new object. I my opinion it is too easy to clone the object accidentally.

Did we gain support for multi-column PK recently and I missed it? Or is
this just a hypothetical example assuming such support is added in the
future?

> The idea would be to raise exception from the second save (deprecation
> warning at this point). A way to get the current behavior is needed too, but
> the user should explicitly request that.

Hmm, I'm not so sure this is really a problem, or enough of a problem to
warrant making people change currently-working code. If you use
non-sequence PKs (and usually you shouldn't), you should be aware of
what you're doing.

If the real problem is the way the admin exposes the possibility of
accidental cloning to end users, maybe we should be looking at a simpler
admin-level fix (like not making PK fields editable by default)?

Carl

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk7ZkYIACgkQ8W4rlRKtE2fwGgCeOan3naNFeAddJHFCP0XGp7wh
LLgAn2g9MUPrafpE+h9yz4kWtQOnZht2
=ID4P
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Deprecate change pk + save behavior for 1.4

[Carl Meyer]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 12/03/2011 10:18 AM, Carl Meyer wrote:
> So what about admin users who are currently relying on this behavior as
> a way to clone objects with natural PKs (given that the
> save-and-add-another button is useless with natural PKs unless you can
> explicitly give the value for the new row)?

Er, I meant save-as-new, of course.

Carl
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk7aWnMACgkQ8W4rlRKtE2dI0QCgq3JDRwTGT6D2li3V9jw7IJuJ
F20AoJTKWP9vQuHNVzo7Us/Vw/mey5os
=om2k
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Deprecate change pk + save behavior for 1.4

[Carl Meyer]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 12/03/2011 02:13 AM, Anssi Kääriäinen wrote:
> Admin should be fixed [#2259]. Making PK fields non-editable in
> ModelForms would be good, too. Is it OK to consider the current
> ModelForm behavior a bug, or will it need a deprecation cycle?

I think it would need a deprecation cycle. People certainly could be
making use of the current behavior (and there would need to be a way to
get the PK field back into the form, e.g. by explicitly listing it in
"fields"). Excluding PK from ModelForm by default isn't clearly fixing a
bug, it's more protecting people from unintuitive behavior.

(Note that in admin we have the option to display it read-only by
default; ModelForm has no such feature, we'd have to exclude it entirely
by default).

> In my opinion admin getting this wrong is a strong indication about
> how non-intuitive the API is.

That's a fair point.

> It is OK that Django doesn't support primary key updates. But if it
> doesn't support update of PK, then lets error out on primary key
> change. Current behavior of .save() is actually: "save the object to
> DB, except when PK has changed, then do a clone". That is a bad API
> for natural primary keys.
> 
> About breaking current code: my intention is to have a flag to .save()
> which would allow current code to work. .save(clone=True) would get
> current behavior back. Setting the PK to None and doing a save will
> work for AutoField PKs.

So what about admin users who are currently relying on this behavior as
a way to clone objects with natural PKs (given that the
save-and-add-another button is useless with natural PKs unless you can
explicitly give the value for the new row)?

Carl
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk7aWf0ACgkQ8W4rlRKtE2dOowCfRvvpxE3dCAou2Ag/NGzl1t94
8PQAn33d9xdfpEQfcORUAkMER+PEvh1B
=Rhc7
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Python 3 port - all tests now pass on 2.5.4, 2.6.2, 2.7.2 and 3.2.2 with the same codebase

[Carl Meyer]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi Vinay,

On 12/05/2011 04:41 AM, Vinay Sajip wrote:
[snip]
> containing " IN ()", which the backend rejects as a syntax error]. I'm
> running the full suite now, but I don't expect it to finish before the
> end of the day :-(

That sounds orders of magnitude slower than running the tests under
Postgres here - and I haven't done any of the pg-config tweaking others
have mentioned (not even turning fsync off). Which leads me to think
that either there is some particularly unusual Postgres slowness on your
machine, or there's a major performance regression with the 3.x port and
Postgres (which, if it's that significant, would block merging the port).

Maybe if you run the tests in 2.x under Postgres, with trunk instead of
the 3.x-port, you could observe the comparative speed on your same machine?

Carl
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk7c4AwACgkQ8W4rlRKtE2chPQCeNPuFvnqgAjDTlvNwPZmQArNN
5EYAoLRvkznDey6Y9fQ7lVwhrLGRq9u7
=BUc4
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Python 3 port - all tests now pass on 2.5.4, 2.6.2, 2.7.2 and 3.2.2 with the same codebase

[Carl Meyer]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 12/05/2011 08:10 AM, Vinay Sajip wrote:
> Not too bad for a first complete pass. Over two and a half hours to
> run, and that' s running from /dev/shm (IIUC effectively RAMdisk) and
> with fsync = off. All else set to default values. Time to investigate
> Jeremy's link to Frank Wiles' post ...

I think first investigating any difference in performance between trunk
and your py3k branch makes more sense than heading straight to more
Postgres performance tweaks.

Carl
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk7c4wUACgkQ8W4rlRKtE2djswCfaf/NDuIyO5SRgMfmTG/JBUif
83gAn3xrbtwWrgEeBdsM9vgdPYCoSgIh
=fbQj
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Feature proposal: models.CALL_DELETE or effective equivalent

[Carl Meyer]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 12/05/2011 08:31 AM, Yo-Yo Ma wrote:
> While I agree that signals allow for neat decoupling, they aren't as
> DRY or quick as a simple field kwarg. Imagine you have 8 models that
> have custom delete methods. Which is easier: A) avstracting the
> functionality of each of the 8 delete methods into helper functions
> and writing signals for all 8, or B) typing on_delete=CALL_DELETE? How
> will this question be answered, objectively?

Clearly the latter. But "what is easier to type" is not, of course, the
only relevant consideration. A couple notes:

1. Implementing the proposed on_delete handler would require a major
reworking of how on_delete handlers function. Currently, they allow for
modifying the list of objects to be deleted, but not changing how the
delete actually happens. To make the latter possible, while still doing
things correctly w.r.t. deletion ordering in the presence of FKs, etc,
would be far from a trivial patch.

2. The proposal would have abysmal performance characteristics on a
large delete. Thus, it would have to come with big red warnings in the
docs. I'm generally averse to adding features that require big red
warnings in the docs.

Carl
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk7c6ioACgkQ8W4rlRKtE2d2mgCgq0bOAHbMj4b9qbAeeeTi9VUx
PFYAoNiYP5+g9r/EbNiy6qj17oKlPmfP
=TblP
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Python 3 port - all tests now pass on 2.5.4, 2.6.2, 2.7.2 and 3.2.2 with the same codebase

[Carl Meyer]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 12/05/2011 11:56 AM, Jeremy Dunck wrote:
> Carl, to be clear, if py3 pg is slower, but py2 pg is still in line w/
> baseline, would that block merge to trunk?  It's quite possible the
> problem lies in psycopg, but I think merging (with caveat emptor on
> the py3/pg combination) would still be good.

Yeah, I agree. If there's no regression on py2 with the branch (which
seems to be the case given Anssi's rough data), then slow py3 is
certainly better than no py3. Though if its really ten-fold-plus slower,
that would make py3/pg pretty much unusable, depending whether its
something that's particularly bad in the test suite or that would affect
typical code similarly. This is of course all uninformed hypothetical
speculation :-)

Carl
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk7dFnQACgkQ8W4rlRKtE2cKmQCeJWaMv8xQ+OERPYGl6bZ7tv8H
vzcAn3zz0W26OxpDCoPf+tF/oOY33gh5
=qR3p
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: PUT and post data

[Carl Meyer]

Fwiw, for the last number of months we've been telling people that pull 
requests are fine in lieu of uploaded patch, but the pull request still needs 
to be linked from a Trac ticket (so Trac remains the single source of truth).

Carl
-- 
Sent from my Android phone with K-9 Mail. Please excuse my brevity.

Adrian Holovaty  wrote:

On Wed, Dec 7, 2011 at 8:25 AM, Thibault Jouannic  wrote:
> I've read the submitting patches doc, but is a git pull request
> accepted?

For now, no. We'll be moving to GitHub in the near future, but
currently we're using Subversion/Trac. Upload a patch via the ticket
tracker (code.djangoproject.com).

Adrian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.


-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: PUT and post data

[Carl Meyer]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 12/07/2011 07:25 AM, Thibault Jouannic wrote:
> https://github.com/django/django/pull/87
> 
> Hope I did everything right.
> 
> I've read the submitting patches doc, but is a git pull request
> accepted? Otherwise, Should I upload a patch in the closed ticket, or
> open a new one?

Oh, just noticed this question. Generally it's best to err on the side
of opening a new ticket when a previous one has been closed, unless it's
exactly the same issue and the closing was simply in error. In this
case, it's not the same thing, as we're adding documentation of the
current behavior, not adding request.PUT as the previous ticket requested.

Thanks for the documentation patch!

Carl
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk7f4KIACgkQ8W4rlRKtE2f6TACgqQGglLk4rpiMF+lkhnfQwKLs
dMYAnjuGZG8i/aLVltQxlcCtoJU+/Cu1
=QStu
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Django-nonrel patches

[Carl Meyer]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Mmm, this reads to me like a bit of a mis-understanding. Hinges on what
Jonas means by "before the patch gets accepted." The phrase is slightly
confusing because we have an Accepted state in Trac, but that applies to
tickets, in concept, not to specific patches.

Declining to do additional patch work before the _ticket_ is marked
Accepted is reasonable and consistent with our usual process. Marking
the ticket accepted only means that the general idea is accepted in
principle, it does not validate the specific approach in any existing patch.

Docs and tests are, of course, required before any patch is actually
committed. In my reading of this thread, I think that's already understood.

Aymeric mentioned that tests and/or docs would clarify the intent of
some of the tickets, which isn't clear enough for him to even accept or
decline the ticket in concept. In that case, I think it would make sense
to close the ticket "needsinfo" with specific comments on what about the
feature request is not clear enough to evaluate. (Patch review shouldn't
even be necessary in order to move a ticket from Unreviewed to Accepted
or closed, IMO - the ticket description itself ought to provide clear
enough motivation for that).

Carl
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk7hRg8ACgkQ8W4rlRKtE2flDACdFPDijhay53JXoFHAIobjFCRP
0FwAoI81yprWk359Tzb3XQKSsEng8P+a
=4YyI
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Python 3 port - all tests now pass on 2.7.2 and 3.2.2 with the same codebase

[Carl Meyer]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 12/10/2011 03:35 PM, Joe & Anne Tennies wrote:
> So, with the current plan to add 3.x support to 1.5 and the plan to
> remove 2.5.x support in 1.5. Should all the exception handling blocks be
> converted to "except MyExceptionClass as e:" style notation then?

Seems to me the plan to drop Python 2.5 support after Django 1.4, and
merge Python 3 support then, is pretty solidly confirmed. In which case
there's no use keeping around 2.5 hacks in the Python 3 branch. So I'd
say yes, for sure.

Carl
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk7j5VwACgkQ8W4rlRKtE2es3ACeJojvDcQ4G4FE6elvizdHmfIe
Le0An2LLMgRd6d0uAdy4/PjYpMWvZD+N
=qd4x
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

notice: change in Trac emails behavior

[Carl Meyer]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

I don't believe this has ever been announced here, and it should be: As
of the last week of November 2011, the "always_notify_updater" Trac
setting on code.djangoproject.com was switched to False (as a result of
user complaints; see https://code.djangoproject.com/ticket/16763).

This means that you must explicitly use the CC field if you wish to be
notified by email of updates to a ticket; you will no longer be notified
automatically simply because you have commented on the ticket.

It seems to me (and others in IRC agreed) that given this change it
would be better for the "add me to CC" checkbox to be checked by default
when commenting on a ticket. This preserves something a bit closer to
the previous behavior (which many contributors preferred), while still
permitting an obvious and easy opt-out. If anyone knows exactly how to
make this happen, post instructions and someone with Trac admin access
can make it happen.

Carl
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk8CIaUACgkQ8W4rlRKtE2f0SgCeKFcxok8cijjG6gYGylqaAqxe
zxQAoJsnOSawCpCFWfN+TNqJS/gWrY8C
=hbcg
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: RFC: Query Methods

[Carl Meyer]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi Zachary,

On 01/02/2012 10:55 PM, Zachary Voase wrote:
> At the moment it's very easy to add methods to individual models, just
> by putting a method on the model class itself which accepts 'self'.
> However, defining business logic on collections of records is
> currently pretty complicated — you have to create at least a custom
> Manager subclass, and if you want to chain those methods together
> you'll need a QuerySet subclass. An example of the desired behaviour,
> and the steps required for it, is shown in [1].
> 
> I originally created django-qmixin [2] to tackle this problem; you can
> define methods which will be present on the manager and all querysets
> produced therefrom, including on relations. However, the django-qmixin
> approach of creating a mixin and then including that on the Manager
> class object doesn't really gel with the rest of Django core. I've
> worked out two approaches which are easier for novices to understand,
> and match the idioms of the rest of Django. They both involve adding a
> @models.querymethod decorator, which would be applied to methods which
> operate on collections of records (i.e. querysets). It's an analog to
> Python's @classmethod.
> 
> The first approach [3] involves adding these querymethods to the model
> class itself; the second [4] adds them to a manager subclass (but
> without the trouble of the QuerySet subclass). I prefer the former,
> for simplicity, but you may believe otherwise.
> 
> I'm working on a proof-of-concept implementation, but I feel it's more
> important to agree on the interface and rationale beforehand. Any
> thoughts?

Thanks for working on this. I think the decorator API you've selected is
a pretty good one for this; I agree with Russell that the methods belong
on a custom Manager, not on the model itself.

There will be some tricky points in the implementation. In particular,
QuerySets must remain pickleable (so they can be cached), which is
problematic if you are dynamically creating the QuerySet subclass. As
Donald Stufft points out on the ticket, we previously had an approach in
django-model-utils (manager_from) that was based on dynamically-created
QuerySet subclasses, and we moved away from it because it broke pickling
of QuerySets. There may be another way around this; I didn't look at the
problem in depth.

I have to say, this all feels to me a bit like polishing a turd; adding
ever-more complex hacks with metaclasses, dynamic subclasses, etc., all
to paper over what is really a more fundamental design problem. To wit:
a Manager is no different from a QuerySet in practice - it's just a sort
of "starter" queryset accessible from the model. If in practice people
pretty much always want to pair a custom Manager with a custom QuerySet
that has the same extra methods (and that is my experience), I think
it's telling us that those shouldn't be two separate classes at all.

I can't help but wonder if it would be possible to solve the problem
more thoroughly and remove the need for magic decorators by eliminating
the Manager class entirely and making it possible to simply assign a
QuerySet to a model, perhaps with a helper method:

   objects = manager(PersonQuerySet.filter(active=True))

I'm sure there are devils in the details here, too, but on cursory
inspection of the Manager code, it seems like it might be doable.

Carl
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk8DPlYACgkQ8W4rlRKtE2e+TQCffZZmaqKRc4Hyn3reUzYj6Cga
ja8AoNyIutFq4GuUfTgn5RgBMQaW/rJk
=AIQa
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: RFC: Query Methods

[Carl Meyer]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 01/03/2012 10:42 AM, Alex Gaynor wrote:
> I haven't analyzed your suggestion in detail Carl, but there is a good
> reason for managers to exist:

> querysets cache internally on iteration, managers are not directly
> evaluable, but were they you could end up doing something like:

> for instance in Model.objects:
>   pass

> and you'd cache those results in global state.

Indeed, that's of those devils in the details, but it's not
insurmountable. Clearly, either the Model metaclass or the hypothetical
"manager" function would need to do something to the provided QuerySet
to make it not iterable directly. This could be a pretty simple flag on
QuerySet, it doesn't require a separate class.

Carl
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk8DQEsACgkQFRcxmeyPUXI5DQCffoTuvSuJ84HlQq1F8SGv6yRX
SFMAniE3FhaGH5eVlrgtSy3se/g2/4Nj
=lqm7
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposal: Logout user when they change their password.

[Carl Meyer]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 01/08/2012 03:57 PM, Arnoud van Heuvelen wrote:
[snip]
> 2) Build a relation table that links the sessions to the users. (Or
> use a CSV in the user object.) This could be done completely in the
> auth package. Downside is that we have to create a new table. Upside
> is again, that we could invalidate sessions on password change.
[snip]
> My favourite is option 2 for now. It seems the most resource friendly,
> even though we need to save more data. I don't know what Django's
> policy on adding tables is though.

This is not an option because the database session backend is only one
of many session backends.

Carl
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk8LD1IACgkQ8W4rlRKtE2eHmgCbBx/7FlOHfi32Cr1pNWa4mKsW
Ca0An1rB+T9fwFwdlHzEmwj/PxTxIAd3
=Mj2V
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Proposal: Logout user when they change their password.

[Carl Meyer]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 01/09/2012 09:01 AM, Carl Meyer wrote:
> This is not an option because the database session backend is only one
> of many session backends.

Oh, never mind - I misunderstood the proposal. This wouldn't be
dependent on using the database session backend if its done entirely in
the auth app and based on the session key. It would, however, require a
database check on every session access, meaning the value of using a
faster session backend would be lost. Thus I still think it's not
feasible, and (if anything is to be done here) a solution like #3 that
stores extra data in the session itself is preferable.

Carl
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk8LEGQACgkQ8W4rlRKtE2faiwCfUhQg9MouUnkGpYWec7WI1VJL
C7UAnjgEIiAgruPSPnlEZfucyRWmxbV3
=Jpx9
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: RFC: Query Methods

[Carl Meyer]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi Anssi,

On 01/10/2012 04:14 AM, Anssi Kääriäinen wrote:
> However, I would like it if you could mix methods from different
> managers. Maybe add qs.set_manager(manager_name), and so you could do:
> SomeModel.objects.order_by('id').some_manager_method().set_manager('other_manager').some_manager_method_in_other_manager().
> set_manager would simply change the qs.manager. This way you could
> change the manager which is looked for chainable methods, and all
> chainable methods would be available from all managers.
> 
> I hoverer do like the idea that you have all the methods defined in
> any manager directly available in the queryset, without the need to
> use set_manager. This is easy to achieve.
> 
> Thoughts?

Having access to any manager method from any queryset, regardless of
which manager that queryset came from, seems to me like a highly
surprising anti-feature, not a benefit. Can you outline a realistic use
case for it?

Carl
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk8McRgACgkQ8W4rlRKtE2fAlwCgq6JcsmkEyUR4djK8bPr49Uf0
lWIAmwdn2oAVqrv7GXph/qePaCYCuVVp
=GL8K
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Caching back-refernces on one-to-one fields

[Carl Meyer]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 01/19/2012 11:02 AM, Jeremy Dunck wrote:
> On Thu, Jan 19, 2012 at 5:14 PM, Adrian Holovaty  wrote:
>> john = Author.objects.get(name='john')
>> books = list(john.book_set.all())
>>
>> # Does a database query, but should be smart enough to simply return john.
>> books[0].author
>> """
>>
>> I'm pretty sure there's a long-standing ticket for this, but I'm not
>> sure which one it is. Shai, does your solution approach this in a way
>> that can solve the issue for ForeignKeys as well?
> 
> This one seems much more likely to be a breaking change - it isn't
> that unusual to do something like:
> 
> special_book = Book.objects.get(...)
> special_book.fiddly_bit = True
> # note no save
> 
> for author in Author.objects.all():
> for book in author.books.all():
>if book.fiddly_bit:
>book.related_books.add(special_book)
> 
> The semantics of this change under a new caching layer.
> 
> If the intention is that it's really the same object (and not a new
> instance of the same db object retrieved from a caching layer) then
> this introduces actions at a distance.
> 
> This is ye olde Django ticket:
> https://code.djangoproject.com/ticket/17
> 
> And for context: ;-)
> http://blogs.tedneward.com/2006/06/26/The+Vietnam+Of+Computer+Science.aspx

I don't think Adrian is proposing anything as extensive as #17. What
he's proposing (IIUC) wouldn't change the semantics of your sample code
at all. All it would do is prepopulate the FK field on the results of a
reverse-FK query, so the innermost "book.author" here doesn't need to do
a query:

for author in Author.objects.all():
for book in author.books.all():
book.author

There wouldn't be any action-at-a-distance with other Book instances.

Carl
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk8YXbgACgkQ8W4rlRKtE2fWPQCfXUp1NVmTaNvwZVPpyOsFVAtP
+UsAoJqJ5l0nz+98L+Z+dJ8rdPe5DeM8
=BTg6
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Why recommend to include project name in settings?

[Carl Meyer]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi Chris,

On 02/10/2012 05:36 AM, Chris Wilson wrote:
>> But "mysite" is a package, and Django wants us to be able to import
>> "mysite.settings" and "mysite.urls" (in fact, the latter is the
>> default ROOT_URLCONF setting). In order to make that possible,
>> setup_environ does some magic... This bit of clever causes a boat-load
>> of problems...
> 
> 
> 
> 
> My question is, why exactly Django wants us to be able to import
> "mysite.settings" and "mysite.urls"? Wouldn't it be much simpler and
> better if we could just import settings and urls?
> 
> As an experiment, I've removed the "setup_environ(settings_mod)" from my
> manage.py, which adds the project's parent directory to the PYTHONPATH,
> and it seems to work so far. I think that mod_wsgi uses an environment
> without this addition as well, leading to tickets like this:
> 
> 
> 
> 
> Is there any reason not to drop the project name part of
> DJANGO_SETTINGS_MODULE and ROOT_URLCONF, to reduce the risk of
> confusion, pollution, double imports and bad practice app coding in Django?

If you follow that quoted thread to the end, you'll find that this
problem has already been resolved in trunk :-) See
https://docs.djangoproject.com/en/dev/releases/1.4/#updated-default-project-layout-and-manage-py
for the release-notes summary.

For the default project layout, I chose to resolve it in favor of having
an outer "project package", as this namespaces the project better and
avoids potential top-level namespace clashes with generic names like
"settings" and "urls". However, it's easy enough to adjust the layout
and imports for your own project to eliminate that project package and
just use "import settings" and "import urls", if you prefer that.

Carl
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk81NPMACgkQ8W4rlRKtE2fulACfQRFyBOBEdB5slItKesanxCr0
lXoAoM4i5LFMn3L2DOYqqyyOr+6WJ6hL
=HWsX
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Why recommend to include project name in settings?

[Carl Meyer]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 02/10/2012 08:25 AM, Chris Wilson wrote:
> On Fri, 10 Feb 2012, Carl Meyer wrote:
>>> Is there any reason not to drop the project name part of
>>> DJANGO_SETTINGS_MODULE and ROOT_URLCONF, to reduce the risk of
>>> confusion, pollution, double imports and bad practice app coding in
>>> Django?
>>
>> If you follow that quoted thread to the end, you'll find that this
>> problem has already been resolved in trunk :-) See
>> https://docs.djangoproject.com/en/dev/releases/1.4/#updated-default-project-layout-and-manage-py
>> for the release-notes summary.
> 
> Thanks, I had seen the patch on Trac and noticed that it was being
> resolved in the direction of the "status quo", by enforcing the presence
> of a top-level directory/module/package. This is the opposite of what I
> was suggesting: entirely removing the top-level thing. I was curious
> about the reasons for this decision, and whether the (apparently
> simpler) alternative had been considered and rejected, and if so why?

The status quo, as you noted, was "confusion, pollution, double imports
and bad practice app coding". Resolving this was the important step;
which direction it was resolved is a minor issue in comparison.

I did consider the alternative, but only briefly, as I think namespacing
is clearly better practice. In the lengthy django-developers thread
about the issue, nobody raised or advocated the non-namespaced option
(AFAIR).

>> For the default project layout, I chose to resolve it in favor of
>> having an outer "project package", as this namespaces the project
>> better and avoids potential top-level namespace clashes with generic
>> names like "settings" and "urls". However, it's easy enough to adjust
>> the layout and imports for your own project to eliminate that project
>> package and just use "import settings" and "import urls", if you
>> prefer that.
> 
> Can you tell me more about why "potential top-level namespace clashes
> with generic names like "settings" and "urls"" would be a problem?

If things were set up in the way you are advocating, your Django
project's settings.py and urls.py would _be_ top-level modules, and they
would either mask or be masked by other top-level modules on sys.path
named "settings" or "urls", depending on sys.path ordering. Namespacing
the project is more explicit about what you are importing, and avoids
this possibility.

> We can always import .settings and .urls if we we want the top-level
> modules, can't we?

I can't figure out what you are suggesting here. "import .module" is not
legal syntax. "from . import module" is the opposite of importing a
top-level module; it's explicitly requesting a module relative to the
current package instead of a top-level module. In fact, if we followed
your suggestion and eliminated the project package, "from . import
settings" or "from . import urls" would be impossible; you can never
import top-level modules that way, Python will give you a "ValueError:
Attempted relative import in non-package."

Carl
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk81PMEACgkQ8W4rlRKtE2dgMACaAtfLhyjH4U3HBrotUtzg1FhO
cM0An2uKcQDpNsdz+89cROBOsRXeDwsY
=8K6l
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Why recommend to include project name in settings?

[Carl Meyer]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 02/10/2012 08:47 AM, Adam "Cezar" Jenkins wrote:
> On Fri, Feb 10, 2012 at 9:50 AM, Carl Meyer  <mailto:c...@oddbird.net>> wrote:
> 
> "import .module"
> 
> 
> Not arguing one way or another, but the above has always worked for me
> from a Python standpoint.

Could you clarify what has worked for you about "import .module"? Every
way I try it, it's just a syntax error.

Carl
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk81PnMACgkQ8W4rlRKtE2c/OACfco31wdU7M3E40uIU7renmee7
CvQAoKWDiOAxJ2BuLiw1RcmsmdE6/Y5w
=MQVQ
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Discussion on Ticket #16304

[Carl Meyer]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

On 02/11/2012 10:13 AM, j4nu5 wrote:
> I have written a patch for ticket #16304 (https://
> code.djangoproject.com/ticket/16304).

Thanks for the patch.

> The ticket adds support for the HTML5 input placeholder attribute
> (WHAT Working Group placeholder description -
> http://www.whatwg.org/specs/web-apps/current-work/multipage/common-input-element-attributes.html#the-placeholder-attribute),
> in Forms as well as ModelForms.

I agree with the previous commenter that HTML widget attributes do not
belong in model field definitions. Making it possible to achieve
arbitrary modifications to ModelForm output HTML without ever overriding
a widget in the ModelForm is a bad design goal; there's a reason the
ModelForm (and widgets) exist as an intermediary layer.

> I was wondering if a larger undertaking can be taken to make Django
> HTML5 aware. Options may include support for new input types like tel,
> email etc. which currently are rendered simply as text and ModelForms/
> Forms have to manually override widgets to achieve desired results.

There is an existing ticket with patch to support HTML5 input types
(#16630). This is also addressed by the template-based-widgets patch in
#15667. I think the latter is the best direction, but it is blocked by
slowness of the template engine.

> P.S. The submitted patch lacks tests. I could not figure out how to
> write tests for the UI. If that is acceptable, can core devs please
> accept the patch?

Patches without tests won't be committed. There are many existing
examples of tests that test form rendering and HTML output; for
instance, most of the tests in regressiontests/forms/.

Carl
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk84VFsACgkQ8W4rlRKtE2fqbgCfZF+38wR0kbsdLnQ2oM5JBfxn
E0UAoOlnRgDVRXmuEaVPU2WsOe3C74Jh
=yR2q
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Django 1.4 beta 1 released

[Carl Meyer]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi Reinout,

On 02/16/2012 03:35 PM, Reinout van Rees wrote:
> Partially related question: several tickets have a pull request on
> github instead of an svn patch. Is that enough? I assume a real svn
> patch is better?

A link on the ticket to a github branch or pull request is fine in lieu
of an uploaded patch. I don't know that this is formally documented
anywhere, but it's been the actual practice for a year or more now.

Carl
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk89i/sACgkQ8W4rlRKtE2dGUACfRoX9eOdRQR9vBQFHMSUnWMYW
aPUAnjxqQlqefZDDmgmhJ19bkJ3Mt2wA
=0Zak
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Class-based view decorators

[Carl Meyer]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hello Tobia,

Thanks for taking the time to propose these additions. Comments below:

On 02/17/2012 11:04 AM, Tobia Conforto wrote:
> The first is an updated version of method_decorator, made to accept
> additional arguments and pass them to the original decorator:
> 
> class MyView(View): @method_decorator(never_cache) 
> @method_decorator(login_required, login_url='/my-login') 
> @method_decorator(permission_required, 'thing.change_thing') def
> dispatch(self, request, *args, **kwargs): ...

Is this really any better than
@method_decorator(permission_required("thing.change_thing"))? The latter
seems more explicit/clearer, and allows method_decorator to stay simpler.

> The second is class_view_decorator, to apply classic view decorators
> to the newer class-based views:
> 
> @class_view_decorator(never_cache) 
> @class_view_decorator(login_required, login_url='/my-login') 
> @class_view_decorator(permission_required, 'thing.change_thing') 
> class MyView(View): ...

Unfortunately, class decorators that create and return a dynamic
subclass cause infinite recursion if super() is used in a method of the
decorated class, because the name given as the first argument to super()
will actually refer to the subclass at runtime. This problem, and
alternative options, were discussed extensively in a thread here a few
months ago.

Carl
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk8+qN0ACgkQ8W4rlRKtE2fHmACfT+a+9GGAuGx11fhMnXp3PDBT
62MAn0deRbjzwfXhl8ro/ZcfS8oVQR+r
=zM3e
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Django 1.4: call for release blockers

[Carl Meyer]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi Anssi,

On 02/18/2012 05:37 AM, Anssi Kääriäinen wrote:
> The second one is about .iterator() and .prefetch_related()
> interoperability. They are mutually exclusive (you need to fetch all
> the objects before you can do prefetch. However, .iterator() says do
> not fetch all the objects in one go...). There are basically three
> ways forward: raise an error, favor .iterator() (current behavior) or
> favor .prefetch_related().
> 
> I think each one has some merit, and this is mostly bike-shedding.
> However, if you have good reasons for one of the options it would be
> good to hear them now. I would prefer either error or doing the
> prefetch even when iterator is used. As I don't recall ever actually
> using .iterator() in production I have no strong opinions. The ticket
> is #17668.

I don't have a real strong opinion here either, but I think this rarity
of usage of .iterator() is an argument for favoring it. In my
experience, if you are bothering to use .iterator() it's because you
really know you need it; you are fetching a ton of items and loading
them all at once might run your server out of memory. Whereas the
consequence of ignoring prefetch_related is just extra SQL queries -
bad, but less likely to be catastrophic.

That's all kind of handwavy, but in the absence of good hard reasons in
either direction, it's the best I've got :-)

Really, silently ignoring either one gives me a bit of a bad taste. I
understand Luke's reasoning for not raising an error; you might not
control the whole queryset construction chain, and unpredictable errors
are bad. But I wish there were some way we could alert the developer of
the situation.

I guess anyone who really needs the prefetch_related is probably
monitoring their SQL queries, and is likely to notice the extra SQL
queries pretty quickly.

Carl
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk9BJeUACgkQ8W4rlRKtE2fcjgCbBudTE6UjRCTRashf6mZ5PNzI
toUAnRIatUAfQVw33mlbAp6W3Mx0ofHg
=UCkj
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Django 1.4: call for release blockers

[Carl Meyer]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



On 02/19/2012 09:40 AM, Carl Meyer wrote:
> On 02/18/2012 05:37 AM, Anssi Kääriäinen wrote:
>> The second one is about .iterator() and .prefetch_related()
>> interoperability. They are mutually exclusive (you need to fetch all
>> the objects before you can do prefetch. However, .iterator() says do
>> not fetch all the objects in one go...). There are basically three
>> ways forward: raise an error, favor .iterator() (current behavior) or
>> favor .prefetch_related().
> 
>> I think each one has some merit, and this is mostly bike-shedding.
>> However, if you have good reasons for one of the options it would be
>> good to hear them now. I would prefer either error or doing the
>> prefetch even when iterator is used. As I don't recall ever actually
>> using .iterator() in production I have no strong opinions. The ticket
>> is #17668.
> 
> I don't have a real strong opinion here either, but I think this rarity
> of usage of .iterator() is an argument for favoring it. In my
> experience, if you are bothering to use .iterator() it's because you
> really know you need it; you are fetching a ton of items and loading
> them all at once might run your server out of memory. Whereas the
> consequence of ignoring prefetch_related is just extra SQL queries -
> bad, but less likely to be catastrophic.
> 
> That's all kind of handwavy, but in the absence of good hard reasons in
> either direction, it's the best I've got :-)

Actually, another reason to favor .iterator is that it's a terminal
clause. So in any situation where iterator and prefetch_related are both
used, iterator must have been used last. So favoring .iterator favors
the most recent request, and in the hypothetical "you got the queryset
from some other library" situation, favors the integrating developer's
request.

Carl
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk9BJ3gACgkQ8W4rlRKtE2djBwCeIQiWCfmazZ2clfkkrCcEqV6C
g48An3q2IowQsgW5Q2zaYkDsNuY4BQRM
=SeV6
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Revisiting multiline tags

[Carl Meyer]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 02/17/2012 11:04 PM, Glenn Washburn wrote:
> I'd like to reopen discussion on the multiline tag issue (see:
> https://code.djangoproject.com/ticket/8652) which was closed 3 three
> years ago as "won't fix".  The last comment notes that this won't
> happen as the decision has been made many times.  But there is no link
> to any discussion on this topic, so I can only assume mtredinnick's
> point two is the supposed reason for not fixing this.

Here's a discussion linked from #3888 in which Malcolm lays out in brief
why multiline tags have been rejected:
https://groups.google.com/forum/?fromgroups#!topic/django-developers/A17TJWd3YJU

Personally I'd be -0 at worst on multiline tags, but I also don't see
any compelling use-cases. I think tags on a single line are
significantly easier to parse visually.

> Its interesting to note that
> https://code.djangoproject.com/ticket/3888, which is referenced in the
> issue, is also closed as "won't fix", but appears to be allowed in
> 1.3.1.  So there is a precedent for "won't fix" issues being included.

There is indeed precedent for wontfix decisions being reversed, but this
isn't one. The lexing bug revealed in #3888 that caused multiline {# #}
to sorta kinda work in a broken way had its own ticket opened (#4164)
and was fixed. In any recent Django, including 1.3.1 and 1.4 beta,
multiline {# #} does not work at all.

Carl
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk9BKtIACgkQ8W4rlRKtE2fnIACg0TxHHzG3rFKbRSkloeZWopmQ
3h8AoNEfygapod43tk3o6ypN2RKLAcGo
=9Xhx
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Revisiting multiline tags

[Carl Meyer]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 02/19/2012 10:20 AM, Torsten Bronger wrote:
> I've made the same observations as in the parallel posting: I18n
> becomes awkward with single-line tags.  We have dozens of lines like
> 
> {% blocktrans with originator=entry.originator|get_really_full_name:"mailto" 
> link=entry.get_metadata.link task_id=entry.task.id 
> process_name=entry.task.process_class|contenttype_name %}

Acknowledged. That would be a lot nicer split across multiple lines.

Carl
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk9BM/AACgkQ8W4rlRKtE2cXkACgkRuL11edpgUB7MfRyPjuOyS7
jkAAoL4JW+CDFsryZb+UkjCizKxlMYs4
=U4+9
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Revisiting multiline tags

[Carl Meyer]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi Glenn,

On 02/19/2012 07:06 PM, Glenn Washburn wrote:
> Very brief I might add.  He only alludes to one technical reason
> saying: "error trapping can occur much earlier".  I'm trying to
> understand what is meant by this as well.  I don't see how my proposed
> change affects error handling in anyway.  What problems are caused by
> my change?  I'm actually very interested in this as I'm likely going to
> be using this.

Not sure; it seems the "many discussions" of this happened before I was
around.

> I hope I'm not being too demanding by desiring a documented list of
> reasons why this is a bad idea.  I've search the groups and not found
> anything significant related to this.  Where was it discussed an are
> there logs? (was it on irc?)

I'm interested as well; I don't see obvious issues with the approach in
your patch. It may in fact be that template lexing has changed enough in
the last few years to make this simpler now than it was then.

> Is there a way we can move forward on this?

Technically speaking, the way forward would be to open a ticket and
attach a patch with tests and docs (in any case, I would do this rather
than consider reopening #3888 - it's too muddled with reference to
possible specific handling of comments). More importantly, though, given
the history here, would be to get a comment from a core developer more
familiar than I am with that history; and perhaps one with a stronger
opinion on the issue. So posting here was the right thing to do - now
the right thing is probably to wait for a bit.

Carl
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk9Btq4ACgkQ8W4rlRKtE2dfMACcCzIFAPiXQLbnEx5NyE4D6zDG
SyAAnjzHSKxJWLSgiiuSN1pyznjgzcuY
=cJ44
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Should the settings.py template include USE_TZ = True?

[Carl Meyer]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 02/20/2012 01:59 PM, Aymeric Augustin wrote:
> The main reason for enabling time zone support by default in new
> projects (through the settings.py template) was to store UTC in the
> database (on SQLite, MySQL and Oracle; PostgreSQL always does that
> anyway).
> 
> This decision was certainly skewed by my background in enterprise
> software, where reliable handling of datetimes is a no brainer. But
> this isn't the most common use case for Django.
> 
> Python doesn't make it very easy to deal with time zones, so forcing
> that concept on developers isn't friendly. The "right way" to do
> things is impractical, and there isn't that much space for
> improvement.

Can you give more specifics here? Exactly how much harder is it to work
with USE_TZ = True than USE_TZ = False for a new Django developer,
presuming they don't really care about timezones at this point and just
want to do things more or less right so as not to box themselves in a
corner if they want to add real timezone support in the future?

I guess I think there's some value in setting people on the right path
earlier rather than later, but it really depends on exactly how much
harder that path is.

Carl
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk9CugQACgkQ8W4rlRKtE2fr6wCg2/cXMn/bHL/p6Cg5YDzZmPNe
AasAoKWeKEnDnWvcuYZR3EsqRsMlRfnB
=Sjc9
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Re-posting of previous thread: ``QuerySet.update`` not updating ``auto_now=True`` fields

[Carl Meyer]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 02/20/2012 05:11 PM, Yo-Yo Ma wrote:
> I understand that ``QuerySet.update`` issues an UPDATE SQL statement,
> but Django's code could be modified to include each ``auto_now=True``
> field on a model in the UPDATE statement, setting the value to
> ``datetime.now()`` as it does when you use ``Model.save``.

There is already a ticket for this (#15566).

Carl
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk9C5YkACgkQ8W4rlRKtE2dlyQCcCXYg8kWqVda8SWDOcehpCNXc
MIUAoIl018Rwf+b0hdxq0YI2uw46vepc
=Njjf
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: django-admin.py startproject creating duplicate settings.py files

[Carl Meyer]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 02/20/2012 08:31 PM, Yo-Yo Ma wrote:
> I have trunk installed from last night, and this is actual terminal
> output (except for the stuff omitted on the left of the $):
> 
> (my_venv) myusername$ django-admin.py startproject foobarz
> (my_venv) myusername$ ls foobarz/
> __init__.py   foobarz manage.py   settings.py urls.py
> (my_venv) myusername$ ls foobarz/foobarz/
> __init__.py   settings.py urls.py wsgi.py
> 
> 
> I opened both settings.py files, and they are indeed identical files.
> Is this intentional? I was interested in the new manage.py format, so
> I thought I'd adjust my app to use it and whatever other new layout
> features, but clearly this isn't correct.

Nope, it's not correct, and I can't reproduce it either. I get just a
"manage.py" in the outer directory, and settings/urls/wsgi in the
package. My guess is that you already had an old-style startproject in
that "foobarz" directory or something. Try it again, and make sure the
target directory doesn't exist. Also try cleaning your Django repo of
untracked files; it's possible you somehow got extra files in the
conf/project_template directory that don't belong there?

Carl
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk9DFf4ACgkQ8W4rlRKtE2cY7QCgvT5GMOuWLAgjl6QI3R/cBd6P
Py8AoIQaqmi11fOfx67mz+kzl8bi95iv
=zEND
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: django-admin.py startproject creating duplicate settings.py files

[Carl Meyer]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1


On 02/20/2012 09:19 PM, Yo-Yo Ma wrote:
> (myvenv)My-Names-MacBook-Pro:dev myusername$ mkdir test_startproject
> (myvenv)My-Names-MacBook-Pro:dev myusername$ cd test_startproject/
> (myvenv)My-Names-MacBook-Pro:test_startproject myusername$ ls -a
> . ..
> (myvenv)My-Names-MacBook-Pro:test_startproject myusername$ django-
> admin.py startproject testing123
> (myvenv)My-Names-MacBook-Pro:test_startproject myusername$ ls
> testing123
> (myvenv)My-Names-MacBook-Pro:test_startproject myusername$ ls
> testing123/
> __init__.py   manage.py   settings.py testing123  urls.py
> (myvenv)My-Names-MacBook-Pro:test_startproject myusername$ ls
> testing123/testing123/
> __init__.py   settings.py urls.py wsgi.py
> (myvenv)My-Names-MacBook-Pro:test_startproject myusername$

Sorry, no idea. You could stick some "pdb.set_trace()" into the copy
function and see where it's getting those files from, I guess.

Carl
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk9DKFYACgkQ8W4rlRKtE2ciuwCg6ofAId96AwlvEkZunujznLiA
eGsAnjWKVXRR4pwPzWRBMPx6M7n3/V1V
=04ia
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: ModelFormSet validation fails when initial queryset contains objects not in the default manager's queryset

[Carl Meyer]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi Tom,

On 02/20/2012 10:58 PM, Tom wrote:
> I have a ModelForm and a ModelFormSet. I initialize my ModelFormSet with
> a queryset containing objects from a custom manager that is not the
> model's default manager. My formset does not validate, because
> ModelForms with instances that are objects in my queryset but not in the
> model's default manager's queryset don't have valid ids. 
> 
> My code: http://dpaste.org/d6V9y/
> 
> I think I'm seeing this because the add_fields method of
> BaseModelFormSet has the line
> (659) qs=self.model._default_manager.get_query_set().
> I can get my code to work by subclassing BaseModelFormset and setting
> form.fields[self._pk_field.name].queryset= self.get_queryset() (thanks
> to some help in irc).
> My question is why the default manager's queryset is used instead of
> self.get_queryset().

On initial inspection I'd say you're right; it seems like
self.get_queryset() should be used there rather than assuming the
default manager's queryset. Can't be sure until I look into it more
closely - the next step would be to file a ticket at
code.djangoproject.com, ideally with a patch containing a failing test
case integrated into Django's own test suite.

Thanks for the report!

Carl
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk9Du3EACgkQ8W4rlRKtE2cPdwCeLId9F+FYEPGvuq6vpp+I
Pn4AoMUeSvU8msQ22lGgneeUA+/g58b7
=Tv81
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: ModelFormSet validation fails when initial queryset contains objects not in the default manager's queryset

[Carl Meyer]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi Ramiro,

On 02/21/2012 08:05 AM, Ramiro Morales wrote:
> On Tue, Feb 21, 2012 at 2:58 AM, Tom  wrote:
>> Hi everyone,
> 
> Please start threads like this to the django-users mailing list
> as this list is dedicated exclusively to discussions about
> development of Django.

It seems to me you may have mis-read Tom's message. It seems perfectly
on-topic to me; he's not asking how to fix his problem, he's asking
about the design of BaseModelFormSet itself.

Carl
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk9DvEkACgkQ8W4rlRKtE2eofgCeMgBwlfF3u4KaZLexAex/jdPZ
VK8An33E/ENnb0tHeCytk4qLoBN3/eUe
=P5Fo
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Discrepancy involving choices with ModelForm and Form

[Carl Meyer]

Hi Stuart,

On 02/21/2012 05:29 PM, Stuart Laughlin wrote:
> It seems to me there is a discrepancy between ModelForm and Form when
> it comes to fields with choices. When I create a ModelForm from a
> model having a CharField with choices set, the resultant form contains
> a select with a blank choice included by default. On the other hand,
> when I create a Form and give it a ChoiceField, the resultant form
> contains a select *without* a blank choice included by default.
> 
> In other words, I can't think of why this form field declaration...
> 
> my_form_field = forms.ChoiceField(choices=MY_CHOICES)
> 
> ...shouldn't result in a Select widget with a blank choice included.
> And that goes double if I set required=False.

When you are creating a ChoiceField in a Form directly, you are
specifying exactly what the choices will be *for the form field*. It's
important that you can get full control at some level - if a blank
option were automatically added to ChoiceFields, how would you propose
that a developer remove it if they didn't want it?

A ModelForm, on the other hand, is an automated translation layer from a
Model to a Form, so it has some leeway to make its best guess as to how
you'd want your model represented as a form. In this case, that usually
includes adding a blank option (but not always: if the field is required
and has a default value, no blank option is added). And if you don't
like its "best guess", you always have the option to override things
manually at the Form layer.

In any case, this would be a backwards-incompatible change, so it would
need to be a clearcut bugfix to even be considered. I don't think it's
an improvement at all, let alone a clearcut bugfix.

> I searched for a ticket along these lines but didn't find anything.
> I'm not an experienced django contributor, so I'm posting here hoping
> for some feedback to verify that I'm on the right track (or not), i.e.
> is this something that I ought to create a ticket for (and some tests
> and a patch)?

I think not, but thanks for taking the time to make the proposal!

Carl



signature.asc
Description: OpenPGP digital signature

Re: HttpRequest.read(), HttpRequest.body, HttpRequest.raw_post_data

[Carl Meyer]

Hi Tom,

On 02/23/2012 06:07 AM, Tom Christie wrote:
> I see that as of 1.4 `HttpRequest.raw_post_data` is being marked as
> "pending deprecation", and `HttpRequest.body` has been introduced.
> This still leaves us with two ways to get at the request body - `.body`
> and `.read()`
> 
> If we're going to make the change of marking `raw_post_data` as deprecated,
> wouldn't it make sense to only have a single interface to getting the
> request body,
> rather than two different interfaces which behave in subtly different ways?
> 
> For example, we could also mark '.read()' as pending deprecation, and
> just have 'body' be a stream-like object.

Well, the reason .read() exists is because a design decision was made
long ago that HttpRequest objects should provide a file-like interface
(thus also .readline(), .readlines(), and .xreadlines()). Short of
reversing that decision (which isn't going to happen, given its long
history), deprecating .read() is not an option. And I wouldn't be in
favor of removing .body, which is the obvious HTTP-oriented name for the
request body.

I don't really see the presence of both as a problem. The "body"
attribute is the primary, and simplest, way to just get the full body of
the response. "read" is present only so that a request can be treated
like a file object.

Carl



signature.asc
Description: OpenPGP digital signature

Re: Improving docs for User.is_authenticated()

[Carl Meyer]

Hi Tom,

On 02/23/2012 08:41 AM, Tom Evans wrote:
> I don't like this function that much. It doesn't actually check
> whether users are authenticated - which is to say, they have presented
> credentials which we have accepted and authorized them to use to the
> site. Instead it always returns true. is_not_anonymous_user() may be a
> better name.
> 
> User.is_authenticated() is documented like so:
> 
> """
> is_authenticated()
> Always returns True. This is a way to tell if the user has been
> authenticated. This does not imply any permissions, and doesn't check
> if the user is active - it only indicates that the user has provided a
> valid username and password.
> 
> 
> This is misleading, as it doesn't actually indicate that the user has
> provided a valid username and password, since it always returns True.
[snip]
> Obviously, this function cannot change in behaviour or name, so I
> suggest altering the docs, dropping the clause about indicating that
> the user has provided username and password to make it clearer what
> this method does.

I agree with you on all counts: the method is poorly named, it can't be
changed now (and is not worth a deprecation process), but the docs
should be less misleading. Can you file a ticket for this?

Carl



signature.asc
Description: OpenPGP digital signature

Re: Improving docs for User.is_authenticated()

[Carl Meyer]

On 02/23/2012 09:42 AM, Luke Granger-Brown wrote:
> It does prove that they've authenticated, in that request.user will
> contain an AnonymousUser if they're not logged in, which overrides this
> method to always return False. If they are, then they'll get their
> actual user, which will return True.

But request.user is far from the only circumstance in which you might
have a User instance. And all User instances have this method, not just
the one attached to a request by the auth middleware. The method name is
only accurate in the latter case.

Carl



signature.asc
Description: OpenPGP digital signature

Re: HttpRequest.read(), HttpRequest.body, HttpRequest.raw_post_data

[Carl Meyer]

On 02/23/2012 09:40 AM, Tom Christie wrote:
>> a design decision was made long ago that HttpRequest objects should
> provide a file-like interface (thus also .readline(), .readlines(), and
> .xreadlines())
> 
> Wouldn't having .read() .readline(), .readlines(), and .xreadlines() all
> on `request.body` provide a slightly cleaner interface, whilst still
> staying in line with that design decision?

Maybe, maybe not. For simple cases, it's awfully nice that request.body
is just a string rather than a file-like object.

In any case, no point in arguing whether it's marginally better or not -
there's no way it's enough better to justify the massive
backwards-incompatible change for everyone currently using a request as
a file-like object.

Carl



signature.asc
Description: OpenPGP digital signature

Re: Revisiting multiline tags

[Carl Meyer]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 02/23/2012 11:29 PM, Russell Keith-Magee wrote:
> This thread contains 6 people expressing support for this change, and
> 2 against (a BDFL, a core developer) -- and you can add me to the -0

FWIW, I'd forgotten how painful the single-line restriction was the last
time I had to work on internationalized templates using blocktrans. The
presented use cases have me thoroughly convinced that this is an
unreasonable restriction on template authors, and I'd be +1 on lifting it.

Carl
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)

iEYEARECAAYFAk9HrEkACgkQ8W4rlRKtE2fRmgCgoSknqISpYN+zQbqksdgAnQBg
+ToAoNcB27Gr/S2+JZEYzq+p70eh83R9
=Z3lD
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: django 1.4b + mysql + boolean fields

[Carl Meyer]

Hello Tomasz,

On 02/24/2012 03:00 AM, Tomasz Kloc wrote:
> I've changed my database from postgresql to mysql. I have never used
> mysql in django projects before, so it was surprising to me when i saw
> 0/1 values instead of True/False in boolean fields. It wasn't an issue
> until i upgraded django from 1.3.1 to 1.4b.  After that, all boolean
> fields in admin interface are checked (in edit mode).

If I am not mistaken, this was already reported in ticket #17747
(https://code.djangoproject.com/ticket/17747). It had been closed as
"worksforme" because the original reporter didn't realize the problem
was specifically related to use of MySQL (and, it seems, the MySQL GIS
backend). Thanks for the additional information!

Carl



signature.asc
Description: OpenPGP digital signature

Re: call_command; add *.pyc and *.pyo files to commands list

[Carl Meyer]

On 02/29/2012 09:07 PM, bhuztez wrote:
> IMHO, no extra support for them is required. Simply add support for
> PEP 302 importers, Django will find management commands in pyc/pyo
> files.
> 
> I improved patch for #14087 weeks ago, which add support for PEP 302
> importers. I am just wondering why the core team did not even bother
> to review it.

I looked over your patch briefly; I don't have time for a full review
right now (probably after 1.4 is released), but in general I think the
direction you're taking is the right one.

The way Django currently finds management commands is pretty poor,
because it makes all kinds of assumptions about the filesystem layout of
Python packages that aren't valid (which is why it doesn't support
zipped eggs, namespace packages, etc.) Adding direct support for pyc
files into Django is a non-starter, because it takes us even further
down the bad-assumptions road (pyc files are an internal CPython
implementation detail, and in Python 3 they've already been moved into a
__pycache__ subdirectory). Your approach, which moves away from the
invalid assumptions and towards using real Python APIs (like
pkgutil.iter_modules) is, I think, the right one.

Carl



signature.asc
Description: OpenPGP digital signature

Re: Pretty Django model instaces updating

[Carl Meyer]

Hi Kee,

On 03/01/2012 11:24 AM, Kee wrote:
> This helper could be used as an instance method in
> django.core.models.Model class:
>  https://gist.github.com/1951748

Thanks for the suggestion. I agree that that's a useful pattern and one
I frequently use myself. The problem is that the model instance
namespace is a precious resource, and using up any more of it than we
already do can easily lead to backwards-compatibility problems. In this
case, I don't think turning a one-liner into a shorter one-liner
justifies using up more of that namespace.

Carl



signature.asc
Description: OpenPGP digital signature

Re: Newline stripping in templates: the dnl way

[Carl Meyer]

On 03/01/2012 01:28 PM, Leon Matthews wrote:
> It looks like a lot of work looks went into this patch, why did it stall?

Same reason any ticket stalls - it seems that nobody felt strongly
enough about it to put the time into reviewing and thoroughly testing
the patch and marking it Ready for Checkin. If you'd like to see it in
(post 1.4 at this point, of course), feel free to do that!

Carl



signature.asc
Description: OpenPGP digital signature

Re: Pretty Django model instaces updating

[Carl Meyer]

On 03/01/2012 09:46 PM, Tai Lee wrote:
> This is easy to achieve in your own code. Create your own BaseModel
> class that all your models subclass. Instead of using
> `.filter().update()`, do this:
> 
> def update(self, **kwargs):
>   """
>   Updates the fields specified in `kwargs` and then call `save()`.
>   """
>   if kwargs:
>   for k, v in kwargs.iteritems():
>   setattr(self, k, v)
>   self.save(force_update=True)
> 
> This will still fire the `save` signals, and your model instance will
> have the new values as well as having them saved to the database.
> 
> I think it is also easier than assigning values to each field and then
> calling `.save(only_fields[field1, field2])`, which I don't think is
> any better than filter().update(). If `save(only_fields=...)` were to
> only write specified fields to the database, this might cause
> unexpected side effects with `save` signals?

The "only_fields" kwarg suggestion comes from ticket 4102, and it is
primarily intended as a performance optimization for large models by
only sending the values of certain fields to the database at all,
something that your proposed code does not do.

Carl



signature.asc
Description: OpenPGP digital signature

Re: New feature: Templated email messages

[Carl Meyer]

Hello Artem,

On 03/02/2012 03:04 PM, Артем Рыжов wrote:
> I'm working as a web developer for many years. One of the most frequent tasks 
> is sending email messages. And template system is very usefull for this task 
> as for web pages. It's strange, but I have not found builtin support for 
> email 
> templates on many platforms that I used early. So I have had to write own 
> tool 
> for this purpose. This is a simple task, but it's like reinventing the wheel. 
> Same for Django. It's annoying to use the template class directly, but I have 
> not found any builtin shortcut function. I've found several third-party 
> solutions, but I did not like them and created my own (again) :) It is based 
> on class django.core.mail.Message and provides very similar interface. There 
> is also templated version of send_mail shortcut (but as I understand it's 
> going to become a method of Message class soon).

Thanks for your proposal and offered contribution.

I think a couple of reasons this is not ready for core are right in your
message - it is easy to implement outside of core, and there are a
number of different third-party implementations, and no clear community
consensus on the Right Way to do it. The place for this evolution to
happen is outside of core, in third-party packages that are as easy for
developers to use as a "pip install".

If your implementation is really the best solution to a very common
need, it should become very popular, with many developers asserting that
they rely on it in every project. At that point, it becomes possible to
open the discussion of whether it belongs in core (and you'll note that
even with very popular extensions we tend to be quite conservative about
adding things to core if they can be done externally).

So I appreciate the offer, but I don't think the time is right to add
this to Django core.

Carl



signature.asc
Description: OpenPGP digital signature

Re: Django participation in Google Summer of Code 2012

[Carl Meyer]

On 03/08/2012 08:19 AM, Łukasz Rekucki wrote:
> On 8 March 2012 16:43, Aymeric Augustin
>  wrote:
>> PEP 414 was accepted a few days ago. It's designed to make it easier to
>> support 2.6, 2.7, 3.3+ on the same codebase.
>>
>> I hope we'll take advantage of this new feature in Django; however, that
>> means a large update (if not a reboot) of the py3k branch.
>>
> 
> Why would you want to do that, when the py3k is already working with
> unicode_literals ? That's a step backwards.

I agree with Łukasz; I support PEP 414 in general, but I'm not at all
convinced that it makes sense to use it for Django's port, given that we
already have a working port using the unicode_literals approach.

Carl



signature.asc
Description: OpenPGP digital signature

Re: [1.4] Pass "obj" parameter to get_inline_instances

[Carl Meyer]

Hi Yohan.

On 03/08/2012 07:04 AM, Yohan Boniface wrote:
> Some great changes have been made on the admin for this release. One of
> them is the add of the "get_inline_instances" [1], that gives the
> possibility to change the inlines list at runtime.
> 
> I've a little suggestion on this. It could be really more powerful if to
> this method get passed the "obj" parameter, like for example it's done
> for get_fieldsets [2].
> 
> For what I can see, this seems to be a very soft change.
> 
> The function is called three times in options.py.
> One is in get_formsets [3], and get_formsets itself has the "obj"
> parameter.
> Another is in the add_view [4], and here obj could have its default
> value, None, as we are in add view and so no object exists.
> The last is in the change_view [5], which retrieve the "obj" from its
> own parameters.

I think this is a sensible change.

> Do you think this change is enough minor to be added in the 1.4 release
> process?

Unfortunately, no. This is a new feature, which means it was too late to
add it as soon as the beta was released, let alone after the RC. (After
the RC we're hesitant to even fix bugs unless they are release-blocking
bugs).

This is too bad, because once 1.4 is released there will have to be
consideration of backwards-compatibility for existing
"get_inline_instances" methods that don't accept the "obj" parameter.
This may tip the scales towards considering this not worth doing,
depending what compelling use cases can be presented.

Feel free to file a bug for this and it can be revisited after 1.4 is
released.

Carl



signature.asc
Description: OpenPGP digital signature

Re: Django participation in Google Summer of Code 2012

[Carl Meyer]

Hi Ian,

On 03/08/2012 08:40 AM, Ian Clelland wrote:
> On Thursday, March 8, 2012, Aymeric Augustin
>  > wrote:
>> PEP 414 was accepted a few days ago. It's designed to make it easier
> to support 2.6, 2.7, 3.3+ on the same codebase.
> 
> (finishes reading)
> Ugh. I'm starting to expect "PEP 499: In a last-ditch effort to
> encourage developers to adopt Python 3, it is declared that Python 3.9
> will be exactly identical, in syntax and semantics, to Python 2.6.7"

Let's please not rehash this discussion here, it was already beaten to
death on the python-dev mailing list.

>> I hope we'll take advantage of this new feature in Django; however,
> that means a large update (if not a reboot) of the py3k branch.
> 
> Sarcasm aside, the only thing that this PEP does is force us to consider
> the minimum "starting version" of Python 3 that we want to support. If
> we declare that Django 1.5 will run on Python 3.2+, then we can ignore
> this PEP, and continue to use the u() and b() functions in the py3k
> branch, until 3.2 support is one day deprecated.

Note that the current version of Vinay's port (since it was updated to
account for dropping Python 2.5 support post-1.4) does not use u() and
b() functions, it uses "from __future__ import unicode_literals". This
means that the only place string wrappers are needed is in the few
places where a "native string" is needed ("str" on both Python 2 and 3).
Vinay posted numbers to python-dev indicating that there is no
measurable performance overhead in the current port.

> On the other hand, if we can say that Django 1.5 will only support
> Python 3.3+, then there will never be a version of Django that needs to
> run on 3.2, and we can remove those functions from the py3k branch. Not
> really a reboot, just one transformation that can now be removed.
> (probably giving us a few % speed improvement)
> 
> I don't think we should base the decision on this PEP, though. More
> important factors are probably the release schedule for 3.3 (August?)
> and how that meshes with the 1.5 development cycle, as well as vendor
> support for, and availability of, Python 3.3. Right now, there are a lot
> of people with access to long-term vendor-supported Python 3
> environments, but nobody has 3.3. If we say that you can't upgrade to
> Python 3 until you can get a full 3.3 environment, how will that affect
> the usefulness of support in Django 1.5?

I think the fact that the next Ubuntu LTS will include Python 3.2, and
it won't be possible to make use of PEP 314 on Python 3.2 without
install-time-transformation hacks, is a strong argument in favor of
sticking with a working unicode_literals port rather than reworking it
to use PEP 314.

Carl



signature.asc
Description: OpenPGP digital signature

Re: Making sure Web APIs built with Django don't have CSRF vulnerabilities.

[Carl Meyer]

Hi Tom,

Thanks for raising this.

On 03/14/2012 11:20 AM, Tom Christie wrote:
[snip]
> One idea to mitigating this in Django core that I've considered would be
> introducing a '@csrf_defered' decorator, that would act like
> '@csrf_exempt', but wrap request.session in a lazily evaluated
> '@csrf_protect'.  That'd give developers an easy way to generically do
> "Don't CSRF protect this view unless/until it accesses session
> information.".  API frameworks could then use '@csrf_defered' instead of
> '@csrf_exempt', and know that they'll get the correct
> behavior whether the API uses session authentication or not.

This would be specific to Django's session implementation, so it would
tie the CSRF protection to use of django.contrib.sessions, which is
something we've otherwise avoided (in the new CSRF implementation). It
also wouldn't help anyone who has their own session implementation or
cookie-based authentication of some sort.

Practically speaking, I think this might be ok and would cover the
majority of real cases. But at the very least it means that this
decorator should live in contrib.sessions, not in the core CSRF code.

Carl



signature.asc
Description: OpenPGP digital signature

Re: authentication by email

[Carl Meyer]

Hi Daniel,

On 03/15/2012 09:24 AM, Daniel Sokolowski wrote:
> Why can we not just increase the length limit on the username field?,
> Why can't we just throw a validation error if data entered is to long
> and the schema has not been updated? I think the answer yes we can and
> easily.

I don't mean to pick on you specifically, but to me this is an excellent
example of a casual assertion of something we can "easily" do that
doesn't hold up under real examination (for instance, if you tried to
write the patch to actually do it).

How do you propose to "throw a validation error" if "the schema has not
been updated"? How do we know whether it's been updated? Are you
proposing that Django should introspect the users database table every
time it starts up in order to check the length of the username field?
Where would you put this introspection check? Have you considered the
effects this would have on every user of contrib.auth (both those who do
and don't run the schema migration)? And the effect on Django
development (needing to run the tests with both an "old" and "new" table
to ensure that the backwards-compatibility handling is tested?)

Carl



signature.asc
Description: OpenPGP digital signature

Re: authentication by email

[Carl Meyer]

On 03/15/2012 09:41 AM, Daniel Sokolowski wrote:
> Yes it clearly would, however I see two possible solutions to make it
> more friendly:
> 
> 1. We provide MySQL, Sqlite3, PostgreSQL instructions on how to do it -
> I mean the actual commands to execute.
> 2. Catch the DB error and throw a form validation warning for those that
> didn't.

Catching the database error won't work - some databases won't raise an
error, they'll just silently truncate the value, potentially leading to
all sorts of nasty data corruption problems and confusing bugs down the
road.

Carl



signature.asc
Description: OpenPGP digital signature

Re: [1.4] SECRET_KEY deprecation is confusing...

[Carl Meyer]

On 03/15/2012 07:52 AM, Nick Pope wrote:
> There is a problem in 1.4rc2 where a missing SECRET_KEY causes Django to
> refuse to start.
> 
> According to the current version of the release notes on the
> website: *In Django 1.4, starting Django with an empty SECRET_KEY will
> raise a DeprecationWarning. In Django 1.5, it will raise an exception
> and Django will refuse to start.*
> *
> *
> This doesn't make sense...  It currently raises DeprecationWarning which
> is an exception which causes Django to fail to start.  To trigger a
> deprecation warning while allowing execution to continue you need to use
> warnings.warn().
> 
> As things stand you are essentially implementing the behaviour for both
> 1.4 *and* 1.5!

Indeed; thanks for the report. I've now fixed it to use warnings.warn,
as it should have originally.

Carl



signature.asc
Description: OpenPGP digital signature

Re: authentication by email

[Carl Meyer]

Hi Daniel,

On 03/15/2012 11:48 AM, Daniel Sokolowski wrote:
> The issue here is that django auth is limited, and restrictive and needs
> hacks to make it use emails as usernames, we can agree on that yes? 

Certainly.

> We
> can also agree that contrib.auth2 with LFK is a complex undertaking far
> into the future?. 

It is not a simple undertaking. How far into the future it is depends
entirely on whether someone who wants to see it happen takes it upon
themselves to make it happen.

> Can we also agree that the 30 character limitation on
> the username ought to be increased?

I am not sure whether this should happen as a separate step or not. In
an ideal world, we would have a longer username field. In the real
world, we have to balance the benefit against the cost, and requiring a
schema migration from practically every Django installation on the
planet would IMO be the most significant backwards-incompatible change
Django has ever shipped, at least since Django 1.0. It is not at all
clear to me that the status quo, bad as it is, is worse than this cure.

The cost would be mitigated somewhat if we had schema migrations in
core, which would make the upgrade instructions much simpler and more
foolproof.

The cost would be reduced even more if we simply shipped this as part of
a larger customizable-auth change (which will probably require at least
a deprecation path itself).

So, in my opinion, your energies would be more productively directed
towards helping make one of those latter two things happen. But feel
free to work on a proposal to simply change the field length. If you can
provide a patch with a really compelling backwards-compatibility story,
my mind is certainly open to change.

Carl



signature.asc
Description: OpenPGP digital signature

Re: Looking for some insight into a tiny piece of the django codebase

[Carl Meyer]

Hi Tim,

On 03/19/2012 05:17 PM, Tim Diggins wrote:
> I'm wondering what the lines at the end of django/db/models/
> deletion.py do:
> 
> for model, instances in self.data.iteritems():
> for instance in instances:
> setattr(instance, model._meta.pk.attname, None)
> 
> They're at the end of the delete method from Collector class,
> https://code.djangoproject.com/browser/django/trunk/django/db/models/deletion.py#L277
> 
> They came in in the original version of this file in
> https://code.djangoproject.com/changeset/14507 by carljm.

carljm, at your service :P

> I just don't quite get why this is needed (and I'm working on a django
> mod that is stopping declared immutable fields from being mutated --
> sometimes throwing an exception) - I could make sure that it always
> silently fails, but I'm wondering why it exists at all.
> 
> My guess would normally be something to do with freeing up garbage
> collection or something similar, but I can't see that with something
> that is 99% likely to be an integer.

It's there because some of those instances may be ones the user has a
reference to, and this cleans up their state so if the user should
happen to save one of them again after deleting it, the ORM will save it
as a new object rather than trying to re-save with the previous PK
(which can cause trouble in the common case of auto-increment PKs).

Carl



signature.asc
Description: OpenPGP digital signature

Re: Contrib backwards-compatibility question

[Carl Meyer]

Hi Alex,

On 03/22/2012 11:26 AM, Alex Ogier wrote:
> Is it considered a backwards incompatibility to import
> django.conf.settings at the top level in a contrib models.py file
> (specifically, contrib.admin)? I know it's possible to do so, for
> example contrib.comments.models imports it, but it could cause circular
> dependencies for any project that in turn imports
> django.contrib.admin.models in their settings file. It's not a common
> situation, probably, and it's an unfortunate restriction for many
> reasons. So if I were to propose a patch that did so, would it get shut
> down for that reason?

No, I'm pretty sure importing django.db.models (which any models.py file
is bound to do, even if indirectly) already imports settings, so this
should have no impact. There are areas where we try to avoid
module-level imports of settings if possible, but models.py files aren't
one of them.

In any case, importing anything from Django or your project in your
settings.py file is strongly discouraged, precisely because it tends to
lead to circular import problems.

Carl



signature.asc
Description: OpenPGP digital signature

Re: Django 1.4 default database required?

[Carl Meyer]

On 04/02/2012 09:35 AM, Optimus Paul wrote:
> I've been running Django for quite a while without a "database", we
> use MongoDB, and it has worked well for us.  We upgraded to 1.4 and
> found that suddenly a default database is required.  Is there a reason
> for this?  Or is this a bug?

Preston has the right solution for this - delete your DATABASES setting
entirely. I think it would be best if an empty DATABASES setting had the
same effect, especially since that's how it was in 1.3 - if you'd be
willing to file a ticket for that, that'd be great.

> We get the error when importing  django.shortcuts.render_to_response,
> which doesn't seem like the kind of place a database connection would
> even be required.

render_to_response doesn't require a database, but there are other
functions in django.shortcuts that do (e.g. get_object_or_404), and the
error is triggered on any import of django.db.*

Carl



signature.asc
Description: OpenPGP digital signature

Re: auth.user refactor: the profile aproach

[Carl Meyer]

On 04/02/2012 06:35 PM, Jacob Kaplan-Moss wrote:
> I've written up a proposal for how *I* would like to address refactoring
> auth.user: https://gist.github.com/2245327.

+1 from me.

One minorish nit: I think that "in the face of ambiguity, refuse the
temptation to guess" should apply equally to reading or writing
user.data, and that there shouldn't be an awkward discrepancy between
reading and writing. Thus, if you've got multiple profiles with
overlapping attributes (the less-common case), you need to be explicit
with AUTH_PROFILES in order to either read or write the overlapping keys
via user.data (but writing is then allowed). In the common case of no
overlapping keys, of course, you don't have to worry about AUTH_PROFILES.

Carl



signature.asc
Description: OpenPGP digital signature

Re: auth.user refactor: the profile aproach

[Carl Meyer]

On 04/03/2012 08:28 AM, Alex Ogier wrote:
> I have written up a little bit about the alternate proposal that I made
> a while ago, Solution 2a
> from https://code.djangoproject.com/wiki/ContribAuthImprovements
> 
> In addition to other arguments, there is a point-by-point breakdown of
> what I feel are the weaknesses in Jacob's proposal.
> 
> You can find it at  https://gist.github.com/2289395 

Since I weighed in with a +1 on the linked-profiles approach, I should
say that I also find this proposal pretty compelling, and would be happy
to see us go either way. After reflecting on it a bit I think on balance
I prefer this proposal, for the following reasons:

1. Requires zero migration from people with existing projects who are
happy with the existing contrib.auth.User model. I think this is a big
plus: all the other proposals require every single installed Django
codebase to perform a migration.

2. Introduces fewer new-and-different one-off concepts into Django (the
data__ ORM special case, the automatic select_related special case, the
automatic collection of profile fields into user.data...). This also
translates into significantly less implementation work and less new
code, which is another big plus - we want something that'll actually get
done and not introduce a bunch of new code we need to maintain.

By the way, I took the liberty of removing from the wiki page the
references to models/settings circular dependencies, because AFAIK the
statements made about it on the wiki page were simply incorrect.
Importing settings does _not_ immediately loop through INSTALLED_APPS
and load every models file (that only happens if you run a management
command that performs model validation, and even then it happens only
after settings are fully loaded). And "from django.db import models"
itself imports settings, so if there were such a circular dependency
problem, every models.py in existence would already be suffering from
it. I frequently refer to settings in models.py and it does not cause a
problem. If anyone can provide sample code demonstrating that this is
actually a problem, feel free to correct me!

Carl



signature.asc
Description: OpenPGP digital signature

Re: auth.user refactor: the profile aproach

[Carl Meyer]

On 04/03/2012 02:34 PM, Daniel Sokolowski wrote:
> Correct me if I’m wrong but both LFK or a swappable user model approach
> like your fail to address issue of extensibility. If today my project is
> authorizing with username and password and tomorrow I wish to add
> OpenAuth then my User model is replaced, whereas with Jacobs approach I
> add another profile model, yes? What about auth apps, you could only use
> one, with profiles many could co exist; one for Facebook, Twitter, etc.

You would add whatever authorization fields you need to your single user
model. If you add new kinds of authorization that require new fields on
the user model, you'd do a schema migration (just like with any other
modification to any other model).

A third-party app that wants to provide, say, Facebook integration,
could either do its own model with a one-to-one to user (nothing is
going to suddenly prevent that approach, it just won't have any extra
sugar around it), or could provide an abstract base class that your user
model can inherit from that includes the needed fields, or could just
document "to use this app, your user model must have these fields: ..."
and let you do it yourself.

Carl



signature.asc
Description: OpenPGP digital signature

#18094: signals, model inheritance, and proxy models

[Carl Meyer]

Hi all,

There's a discussion ongoing on ticket #18094
(https://code.djangoproject.com/ticket/18094) that has enough potential
back-compat implications that it seems worth getting feedback here.

Currently, when you delete a concrete-inheritance child model instance,
pre_delete and post_delete signals are sent both for the child and
parent models.

In contrast, when you delete a proxy model instance, the pre/post_delete
signals are sent only for the proxy model class, never for the parent
concrete class.

This is problematic. Imagine a reusable tagging app that has a Tag
model, and attaches a pre_delete signal handler for Tag instances. In my
usage of that app, I use my own TagProxy subclass. When I delete a
TagProxy instance, a Tag is in fact deleted, but the app's pre_delete
signal handler never fires. This violates the reasonable assumption that
subclassing doesn't change superclass behavior except where it
explicitly overrides it.

So we'd like to make the obvious fix, which is to fire the
pre/post_delete signals for every superclass of the instance(s) you
delete, regardless of whether they are parents via concrete or proxy
inheritance.

This raises the question of consistency with pre/post_save and
pre/post_init, which are currently both sent only once, for the exact
class being saved/initialized (thus already inconsistent with delete
signals, which are also sent for concrete-inheritance parents). The same
Tag scenario above would apply equally to save and init signals: the
tagging app should be able to assume that if it registers a save or init
signal handler for Tag, it will get called whenever a Tag is saved or
initialized, even if that's happening via a Tag subclass.

So it seems that perhaps we should also fix the save and init signals to
be fired for each superclass. Is this an acceptable change from a
backwards-compatibility perspective? In the common case of registering a
signal handler using "sender=", it should be a clear win: signal
handlers will now execute when you'd expect them to. But in the case of,
say, a generic audit-trail handler that listens for all post_save (no
sender specified), it will now get what could seem like duplicate
signals for inherited models (one for each superclass of the instance
being saved).

Thoughts?

(The other approach that might come to mind for fixing this would be to
move the fix into the signals framework itself, so that it executes
signal handlers registered for signal=Foo anytime a signal is fired for
anything that passes issubclass(Foo), rather than requiring Foo
identity. This is appealing, and would solve the potential "duplicate
signals" problem with a generic receiver, but is a much more invasive
change in the fundamental semantics of the signals framework, so I don't
think it's a realistic option.)

Carl



signature.asc
Description: OpenPGP digital signature

Re: #18094: signals, model inheritance, and proxy models

[Carl Meyer]

On 04/12/2012 10:52 AM, Jeremy Dunck wrote:
> On Thu, Apr 12, 2012 at 9:31 AM, Carl Meyer  wrote:
>> There's a discussion ongoing on ticket #18094
>> (https://code.djangoproject.com/ticket/18094) that has enough potential
>> back-compat implications that it seems worth getting feedback here.
> 
> Small note, I'll try to respond to the whole thing later.  This ticket
> exists on the topic, too:
> https://code.djangoproject.com/ticket/9318

Thanks for pointing that out! I've closed #18094 as a duplicate of
#9318, as they really are addressing the same issue. #9318 is currently
taking the approach I'd dismissed as "not a realistic option" of fixing
the signals framework itself to be smart about sender subclasses. It may
be that I dismissed that option too quickly.

Carl



signature.asc
Description: OpenPGP digital signature

Re: #18094: signals, model inheritance, and proxy models

[Carl Meyer]

On 04/12/2012 11:02 AM, Javier Guerra Giraldez wrote:
> in my mental model, there are three types of inheritance:
> 
> concrete: two tables, deletion should fire two signals, one for the
> child record and one for the parent record.
> 
> abstract: there's no parent table, deletion should fire one signal,
> for the child record; unless the parent overrides the delete() method
> and chooses to send a custom signal
> 
> proxy: there's no child table.  kind of an 'abstract child', instead
> of 'abstract parent'.  deletion should fire one signal, for the
> _parent_ record (which is the only real record), unless the child
> overrides the delete() method and chooses to send a custom signal.
> 
> 
> IOW, i think the existing signals are database-related and should be
> fired only for the concrete part(s).  if the abstract part wants to,
> it can send custom signals.

I agree that signals should not be fired for abstract models. Proxy
models are a rather different case, though - they exist precisely in
order to be able to augment/supplement Python-level model behavior, and
signals are a part of that behavior. It would seem unfortunate to me if
you couldn't attach signal handlers specifically to a proxy subclass of
a model.

Also, firing signals only for the concrete superclass of a proxy model
would be a much worse backwards-incompatible change than any of the
others discussed. Signal handlers (for delete, save, or init) registered
for a proxy model class specifically currently work - with this change
they would just suddenly stop firing.

> second idea:
> 
> define three different signals: one for concrete members, one for
> abstract, and one for both.
> 
> - non-inheritance deletion:
> - send concrete-deleted and common-deleted
> 
> - concrete inheritance deletion (both are concrete):
> - child: concrete-deleted and common-deleted
> - parent: concrete-deleted and common-deleted
> 
> - abstract inheritance (parent is abstract):
> - child: concrete-deleted and common-deleted
> - parent: abstract-deleted and common-deleted
> 
> - proxy inheritance (child is abstract):
> - child: abstract-deleted and common-deleted
> - parent: concrete-deleted and common-deleted
> 
> but i think that's overkill

Yes, I think this is too much complexity compared to the other solutions.

Carl



signature.asc
Description: OpenPGP digital signature

Re: #18094: signals, model inheritance, and proxy models

[Carl Meyer]

On 04/12/2012 11:02 AM, Javier Guerra Giraldez wrote:
> IOW, i think the existing signals are database-related and should be
> fired only for the concrete part(s).  if the abstract part wants to,
> it can send custom signals.

Also, it isn't really true that the model signals are strictly tied to
database activity; they are tied to events on Python model objects. One
of the three signals under discussion is the pre/post_init signal, which
fires anytime a model instance is instantiated, even if no database
query or database change occurred.

Carl



signature.asc
Description: OpenPGP digital signature

Re: extra files in startproject (was: Django is not a serious framework, really)

[Carl Meyer]

Hi Jason,

On 04/11/2012 06:10 AM, Jason Ma wrote:
> Hi,
> I download and tried to use the Django 1.4 yesterday. I am a dummy
> and I just follow the official document, but When I just start a
> project.
> I found that it is what I see from my computer:
> 
> jason@jason-pc:~/workspace/hunqing$ tree .
> .
> ├── hunqing
> │   ├── __init__.py
> │   ├── __init__.pyc
> │   ├── settings.py
> │   ├── settings.pyc
> │   ├── urls.py
> │   ├── urls.pyc
> │   ├── wsgi.py
> │   └── wsgi.pyc
> ├── __init__.py
> ├── manage.py
> ├── settings.py
> └── urls.py
> 
> but what doc say?
> mysite/
> manage.py
> mysite/
> __init__.py
> settings.py
> urls.py
> wsgi.py

Others have commented on the pyc files, but I don't think that's really
the point here. The point is that there is an extra __init__.py,
settings.py, and urls.py in the outer directory next to manage.py that
should not be there.

This is definitely a bug, and it's one that I've already seen reported
several times, but it is not a bug in Django. With a clean installation
of Django 1.4, the tutorial steps work as advertised. This bug occurs if
somehow your installation of Django 1.4 is "layered" on top of an
installation of Django 1.3, without the 1.3 installation ever having
been removed. I'm not sure how this happens, but my best guess is that
it could happen if you are using a very old version of pip (like, pip
0.3 era, before pip gained uninstall support). You're most likely to be
using an old pip if you are using a Linux distribution's packaged pip;
those tend to be quite outdated.

If you are interested in helping to solve this issue (as opposed to, for
instance, trollish hyperbole), it would be very helpful to know what
operating system and version you are using, and how you installed Django
1.4 (i.e the exact commands you used).

Thanks!

Carl



signature.asc
Description: OpenPGP digital signature

Re: #18094: signals, model inheritance, and proxy models

[Carl Meyer]

On 04/12/2012 12:43 PM, Anssi Kääriäinen wrote:
> It is important that pre/post init signals will not get more expensive
> than they currently are. Even now they can give around 100% overhead
> to model.__init__(). And this is in a case where the currently
> initialized model has no signals attached at all - it is enough that
> _some_ model in the project has pre or post init signals attached. I
> don't believe signaling cost is severe for .save and .delete as those
> operations are doing much heavier work than __init__ anyways, but for
> init the cost is big enough already. If at all possible, make them
> cheaper.

So this is an argument against firing the init signals multiple times,
for each superclass, but it's not an argument against changing the
signal framework to include superclass receivers, as proposed in #9318;
that would not change the performance characteristics for the
no-receivers situation.

(I see that this choice also has implications for the performance
improvement patch in #16679, but it looks like we have a workable option
there either way).

> In addition, just changing the signals to fire for every parent class
> is pretty much as backwards incompatible as it gets: imagine you have
> post_save signal which increments a counter in a related model on
> create (posts in a thread for example). You have also a proxy model,
> and you have attached the same signal to the proxy model, too, because
> that is what you have to do to get the signal fire when you save the
> proxy model. After the change you will get the counter incremented two
> times for create, and you have a data corrupting bug. This might not
> be a common pattern, but for example Django's use of pre and post init
> signals follows the pattern of attaching to each subclass individually
> and these signals would thus fire multiple times.

Yes, this is similar to the audit-trail handler I mentioned, and I agree
that it is a significant backwards-compatibility problem for either option.

So perhaps we do need the signal inheritance behavior to be opt-in when
connecting the signal handler. I think I'd like to see a deprecation
path so that eventually the inheritance behavior is the default, and you
have to opt out of it, as I think in most cases it's the desired behavior.

Carl



signature.asc
Description: OpenPGP digital signature

Re: extra files in startproject

[Carl Meyer]

Hi Alex,

On 04/12/2012 01:07 PM, Alex Ogier wrote:
> Maybe it would be worth experimenting with various combinations of
> django 1.x django-admin.py executables with django 1.4 libraries? Maybe
> if django-admin.py is a symlink into a 1.3 tree but django 1.4 is on the
> search path this stuff could crop up?

Sorry, I wasn't clear - I already know what the immediate cause of the
problem is, and it's easy to reproduce manually.

"startproject" takes the actual files in django/conf/project_template/
as the template for the new project. If an installation of Django 1.4 is
installed over top of an installation of Django 1.3 (and I mean
literally on top of, in the same filesystem tree), then the project
template files that were moved to a subdirectory in 1.4 (__init__.py,
urls.py, settings.py) are found in both locations. So even if it's
purely the 1.4 startproject code running, it'll install these extra
files, because it finds them there in the project template.

The open question is just how this situation occurs in the first place.
In other words, which particular buggy installer or installation
technique is causing an overlaid installation like that, so we can warn
people away from it and better advise them how to fix it.

Carl



signature.asc
Description: OpenPGP digital signature

Re: extra files in startproject

[Carl Meyer]

On 04/12/2012 03:16 PM, Aymeric Augustin wrote:
> This problem occurs at least when you run "setup.py install" before
> and after the commit that introduced the new project layout.
> 
> Some people who had the habit of running "setup.py install" from a
> git clone to keep up-to-date with the development version reported
> the problem. (Just to be 100% clear — this technique doesn't work
> because it doesn't remove .py or .pyc files that are removed from
> Django.) As a result, I added a warning to the docs in r17636.

Ah, this makes sense. Thanks!

> Most likely, installing 1.4 with "setup.py install", as explained in
> our docs [1], has the same result when 1.3 was previously installed
> in the same fashion.

Yes, it would.

> I suggest we add a warning to this section of the docs: "if a
> previous version of Django is installed, go delete it manually in
> your site-packages directory" (unless there's a better method to
> remove a Python package?)

The "better way" would normally be "pip uninstall". Unfortunately, since
Django's setup.py uses pure distutils (not setuptools), installing
Django with "python setup.py install" does not record any metadata along
with the installation, making an automated uninstall impossible. So the
only workable technique is to manually remove the "django" directory
from site-packages.

I agree that we should add a documentation warning about this. I think
the current warning is not quite in the right place, as it follows the
section on installing via a pth file or symbolic link. "setup.py
install" should not be used in that case, but not for reasons of the
above bug, rather simply because it's not needed. So (IMO) it's
confusing to combine the warnings.

I've filed #18115 to track this.

Carl



signature.asc
Description: OpenPGP digital signature