Re: Using AbstractBaseUser without django.contrib.auth

[Dan Watson]

On Thursday, April 2, 2015 at 2:27:51 AM UTC-4, Marc Tamlyn wrote:
>
> Moving them into another module won't make much difference as their 
> definition requires Permission and Group and therefore they'd still need to 
> import Permission and Group. We'd need an "AbstractAbstractBaseUser" which 
> would be silly.
>
>
 AbstractBaseUser doesn't require any models from contrib.auth (as opposed 
to AbstractUser, which inherits PermissionsMixin), and I'm fine with 
leaving PermissionsMixin in models.py since it does require Group and 
Permission.

Of course, there is no requirement to use AbstractBaseUser for your custom 
> model at all, though this does result in some otherwise unnecessary code 
> duplication. I would say your choice is either two empty tables in your 
> database, or code duplication between Django's source and your custom user. 
> Personally I'd prefer the former.
>

Very true, but it seems like AbstractBaseUser was designed to be used by 
systems not wanting Django's permission structure. It's even documented[1] 
how to make AbstractBaseUser subclasses work with the admin. I can live 
with either wart (copying code or empty tables) if need be, just wanted to 
explore some alternatives. I'll open a ticket.

[1] 
https://docs.djangoproject.com/en/1.8/topics/auth/customizing/#custom-users-and-django-contrib-admin

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/297c3eb4-7832-424a-a269-4af471eae23a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Using AbstractBaseUser without django.contrib.auth

[Marc Tamlyn]

Apologies, I was confusing abstract base user and abstract user there.
Seems your proposal should work. Have you opened a ticket?

On 2 April 2015 at 14:28, Dan Watson  wrote:

> On Thursday, April 2, 2015 at 2:27:51 AM UTC-4, Marc Tamlyn wrote:
>>
>> Moving them into another module won't make much difference as their
>> definition requires Permission and Group and therefore they'd still need to
>> import Permission and Group. We'd need an "AbstractAbstractBaseUser" which
>> would be silly.
>>
>>
>  AbstractBaseUser doesn't require any models from contrib.auth (as opposed
> to AbstractUser, which inherits PermissionsMixin), and I'm fine with
> leaving PermissionsMixin in models.py since it does require Group and
> Permission.
>
> Of course, there is no requirement to use AbstractBaseUser for your custom
>> model at all, though this does result in some otherwise unnecessary code
>> duplication. I would say your choice is either two empty tables in your
>> database, or code duplication between Django's source and your custom user.
>> Personally I'd prefer the former.
>>
>
> Very true, but it seems like AbstractBaseUser was designed to be used by
> systems not wanting Django's permission structure. It's even documented[1]
> how to make AbstractBaseUser subclasses work with the admin. I can live
> with either wart (copying code or empty tables) if need be, just wanted to
> explore some alternatives. I'll open a ticket.
>
> [1]
> https://docs.djangoproject.com/en/1.8/topics/auth/customizing/#custom-users-and-django-contrib-admin
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/297c3eb4-7832-424a-a269-4af471eae23a%40googlegroups.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAMwjO1FAdmDoOmH5Gh2O26Obwz879di9vodKCHu73hWKbZ53Kg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Two phase cleaning of models/forms

[Tom Evans]

On Wed, Mar 25, 2015 at 9:37 AM, Thomas Güttler  wrote:
> Up to now the validation/cleaning of a attribute can not access the
> value of an other attribute.
>
> The work around is to override the clean() method of the form/model.
>
> A team mate and I talked about it, and we asked ourself, if
> a two phase cleaning would help here.
>
> Our idea:
>
>  - The first clean phase works like the current implementation
>
>  - The second clean phase is optional and does nothing except the user
> implements it.
>
>  - In the second clean phase your are allowed to peek into
>the results of the first clean phase.
>The clean method for the second phase would need to get the other values
> passed in.

How is this different from the existing behaviour of clean_FOO(),
followed by clean()? Is that not "clean phase 1" and "clean phase 2"?

On Wed, Mar 25, 2015 at 2:14 PM, Preston Timmons
 wrote:
> There are times when I've definitely wanted this feature. Particularly, when
> multiple fields on a form have this type of constraint. Putting all the
> logic in the clean method gets convoluted.

You can place each multi field constraint to be tested in to its own
method and call them from clean() if it gets too unwieldy.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFHbX1LMrvA4zE2yt%3DhLV-SW%2B6dzP8paUDZR2qfWc%3Dr2-TAcuA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Two phase cleaning of models/forms

[Preston Timmons]

Since I think this can be a useful feature, I'll explain why.

One use case is for validating address forms. We deal with a lot of
them with varying levels of validation based on country, state, zip
code, etc. Sometimes, multiple sets of address fields appear on the
same form. We can't simply reuse the fields without worrying about
disparate validation routines specified on the form in addition to the
fields. This leads to a meticulous set of mixins. It gets the job done,
but I don't think that's great api.

If fields had a second clean method that was called with the form
data, we could do something simpler like:

state1 = StateField(country_field="country1")
state2 = StateField(country_field="country2")

The field would then look like:

StateField():

def __init__(self, country_field=None):
self.country_field = country_field

def clean_full(self, cleaned_data):
if self.country_field:
do_something(cleaned_data[country_field])

Yes, clean_FOO() can be made to work. No, it doesn't make reusing
fields with complex validation in multiple forms and contexts easy,
especially if the field names may need to differ in certain forms.

Preston

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/473fc160-dd08-4899-90c2-36589cafb264%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Two phase cleaning of models/forms

[Carl Meyer]

Hi Preston,

On 04/02/2015 10:15 AM, Preston Timmons wrote:
> Since I think this can be a useful feature, I'll explain why.
> 
> One use case is for validating address forms. We deal with a lot of
> them with varying levels of validation based on country, state, zip
> code, etc. Sometimes, multiple sets of address fields appear on the
> same form. We can't simply reuse the fields without worrying about
> disparate validation routines specified on the form in addition to the
> fields. This leads to a meticulous set of mixins. It gets the job done,
> but I don't think that's great api.
> 
> If fields had a second clean method that was called with the form
> data, we could do something simpler like:
> 
> state1 = StateField(country_field="country1")
> state2 = StateField(country_field="country2")
> 
> The field would then look like:
> 
> StateField():
> 
> def __init__(self, country_field=None):
> self.country_field = country_field
> 
> def clean_full(self, cleaned_data):
> if self.country_field:
> do_something(cleaned_data[country_field])
> 
> Yes, clean_FOO() can be made to work. No, it doesn't make reusing
> fields with complex validation in multiple forms and contexts easy,
> especially if the field names may need to differ in certain forms.

As I said earlier in the thread, I think it's entirely possible to build
this, in a fully generic and reusable way, on top of current Django core
as a third-party add-on. It could even have the exact API you propose.
You just need a `Form` subclass with a `clean` method that loops through
the fields on the form and calls the `clean_full` method (if it exists)
on each of them.

I think the "frequency of need" vs "added complexity" calculation does
not come out in favor of adding this to Django core, but I don't see any
reason it can't exist as a third-party utility.

I also find it a mild violation of field encapsulation for fields to
know about other fields on the same form. In my mind, the right level of
abstraction for cases like this is usually to build a reusable
`AddressForm`, which does cross-field validation at the form level, and
then reuse that alongside other forms. (In general, I think users of
Django's forms library often don't take enough advantage of the fact
that a single HTML form doesn't need to be a single Django `Form` class.)

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/551D6FAE.507%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Using AbstractBaseUser without django.contrib.auth

[Dan Watson]

On Thursday, April 2, 2015 at 10:38:48 AM UTC-4, Marc Tamlyn wrote:
>
> Apologies, I was confusing abstract base user and abstract user there. 
> Seems your proposal should work. Have you opened a ticket?
>

I have:

https://code.djangoproject.com/ticket/24564
https://github.com/django/django/pull/4438

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/a46911df-7bfa-477c-a2cd-3e5b0462880f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Two phase cleaning of models/forms

[Preston Timmons]

Yep, I think you're right now. Given existing solutions there's not warrant
enough to add another built-in validation hook.

Preston

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/1dd15f29-271a-46ca-b041-e77f48c0a567%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Fate of sql* management commands

[Aymeric Augustin]

On 30 mars 2015, at 23:10, Carl Meyer  wrote:

> So it is not true that the Python models are the canonical
> representation of your schema, and the SQL DDL is just a translation of
> them. In fact, your migrations are the (only) canonical representation
> of your schema, which may include things (specialized indices, views,
> SQL functions, triggers...) that don't have any direct representation in
> the Python models at all. The Python models just need to be in-sync
> enough to generate the correct SQL for queries at run-time, but they
> often will not fully represent your schema.

While I understand this argument in theory, I'm not sure we can stop there in
practice. I know that anecdote != data but here's my story and what I think it
means for Django.

I started a moderately complex Django-based e-commerce project about one year
ago. We switched to Django 1.7 right after it was released and went live a few
days later. We made the most basic use of migrations: add a model here, run
makemigrations, commit; change a field there, run makemigrations, commit.

We did only one complicated thing. We moved our custom user model because of a
circular import problem. (Pro-tip #1: put your custom user model in an app
that doesn't depend on anything else. Perhaps we should document that.) We
papered over the resulting mess by squashing migrations and life was good.
(Pro-tip #2: if you change AUTH_USER_MODEL, throw your migrations history away
and regenerate it. It's easy. We should definitely document how to do that.)

A few weeks ago, we ran into issues I can't describe exactly when generating
or applying new migrations. I'm sorry, I don't remember the details because
I'm not monitoring development very closely. Judging by how much time had been
spent on the issue and how messy it looked, I decided to simply scratch our
migrations history, which we should have done earlier. (See pro-tip #2.)

I wanted to check that the resulting schema matched what we had in production
in order to make sure that I wouldn't introduce inconsistencies. On my
machine, I emptied all migrations folders, ran makemigrations, created a fresh
database, ran migrate, and dumped the schema. Then I grabbed a dump of the
production schema. The dumps were about 11 000 lines long. The diff produced
by apgdiff -- a great tool -- was 2 000 lines long.

Much of the diff was noise created by non-deterministic index and constraint
names. I'm almost sure we fixed that in 1.8. I hacked a script to renormalize
hashes and was still left with significant differences. Oops.

These differences happen when Django produces a different schema:

- if you simply create a model
- if you make a series of changes that result in the same model

These differences appear not only when you reset migrations like I did, but
also if you squash migrations, since the squashed migration is pretty much a
fresh migration that declares that it replaces a bunch of previous migrations.
The history of migrations runs a series of alterations. The squashed migration
runs a simple creation.

Here are the differences I found and couldn't explain by problems in our code:

- Sequences aren't handled correctly when changing an AutoField into an
  IntegerField. I filed #24533. This could result in different behavior in
  dev/test and production, which I find dangerous. I could probably present it
  as a crashing or data loss issue.

- Some unique indexes on OneToOne fields were missing. This may be a variant
  of #24163. I haven't investigated it fully. This is about as bad as the
  previous one.

- Many varchar_pattern_ops indexes were missing in production. See #23954.
  This could result in performance issues.

- I had a constraint generated twice locally, once in prod. I'm not sure why.

Given that basic use of makemigrations & migrate can result in significant
errors in the resulting database schema, in the current state of migrations,
we cannot promote them as the single way to manage the database schema. The
risk of not creating the expected schema -- or, worse, not having created the
expected schema in the past -- is simply too high. Every project that ever
used 1.7 or 1.7.1 suffers from such issues.

Therefore I think we must document how our users with basic needs, like
myself, can obtain the reference database schema corresponding to their
models, assuming that they never used RunSQL or that they know what they
did with it.

I’m arguing that there’s a practical need here and practicality beats purity.

-- 
Aymeric.




-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://group

Re: Fate of sql* management commands

[Marc Tamlyn]

As far as I'm aware, we have some code paths which still work in 1.9 which
generate tables directly from the models. We use this when running Django's
own test suite, and it is also used now by djangobench. I haven't looked
into exactly how to turn this into logged SQL rather than run SQL but it
should be possible.

I think we should document such a tool as a debugging tool rather than a
development tool, but it would be useful nonetheless. Something like
`sqlmigrate --from-clean` might be appropriate.

Marc
On 2 Apr 2015 21:50, "Aymeric Augustin" 
wrote:

> On 30 mars 2015, at 23:10, Carl Meyer  wrote:
>
> > So it is not true that the Python models are the canonical
> > representation of your schema, and the SQL DDL is just a translation of
> > them. In fact, your migrations are the (only) canonical representation
> > of your schema, which may include things (specialized indices, views,
> > SQL functions, triggers...) that don't have any direct representation in
> > the Python models at all. The Python models just need to be in-sync
> > enough to generate the correct SQL for queries at run-time, but they
> > often will not fully represent your schema.
>
> While I understand this argument in theory, I'm not sure we can stop there
> in
> practice. I know that anecdote != data but here's my story and what I
> think it
> means for Django.
>
> I started a moderately complex Django-based e-commerce project about one
> year
> ago. We switched to Django 1.7 right after it was released and went live a
> few
> days later. We made the most basic use of migrations: add a model here, run
> makemigrations, commit; change a field there, run makemigrations, commit.
>
> We did only one complicated thing. We moved our custom user model because
> of a
> circular import problem. (Pro-tip #1: put your custom user model in an app
> that doesn't depend on anything else. Perhaps we should document that.) We
> papered over the resulting mess by squashing migrations and life was good.
> (Pro-tip #2: if you change AUTH_USER_MODEL, throw your migrations history
> away
> and regenerate it. It's easy. We should definitely document how to do
> that.)
>
> A few weeks ago, we ran into issues I can't describe exactly when
> generating
> or applying new migrations. I'm sorry, I don't remember the details because
> I'm not monitoring development very closely. Judging by how much time had
> been
> spent on the issue and how messy it looked, I decided to simply scratch our
> migrations history, which we should have done earlier. (See pro-tip #2.)
>
> I wanted to check that the resulting schema matched what we had in
> production
> in order to make sure that I wouldn't introduce inconsistencies. On my
> machine, I emptied all migrations folders, ran makemigrations, created a
> fresh
> database, ran migrate, and dumped the schema. Then I grabbed a dump of the
> production schema. The dumps were about 11 000 lines long. The diff
> produced
> by apgdiff -- a great tool -- was 2 000 lines long.
>
> Much of the diff was noise created by non-deterministic index and
> constraint
> names. I'm almost sure we fixed that in 1.8. I hacked a script to
> renormalize
> hashes and was still left with significant differences. Oops.
>
> These differences happen when Django produces a different schema:
>
> - if you simply create a model
> - if you make a series of changes that result in the same model
>
> These differences appear not only when you reset migrations like I did, but
> also if you squash migrations, since the squashed migration is pretty much
> a
> fresh migration that declares that it replaces a bunch of previous
> migrations.
> The history of migrations runs a series of alterations. The squashed
> migration
> runs a simple creation.
>
> Here are the differences I found and couldn't explain by problems in our
> code:
>
> - Sequences aren't handled correctly when changing an AutoField into an
>   IntegerField. I filed #24533. This could result in different behavior in
>   dev/test and production, which I find dangerous. I could probably
> present it
>   as a crashing or data loss issue.
>
> - Some unique indexes on OneToOne fields were missing. This may be a
> variant
>   of #24163. I haven't investigated it fully. This is about as bad as the
>   previous one.
>
> - Many varchar_pattern_ops indexes were missing in production. See #23954.
>   This could result in performance issues.
>
> - I had a constraint generated twice locally, once in prod. I'm not sure
> why.
>
> Given that basic use of makemigrations & migrate can result in significant
> errors in the resulting database schema, in the current state of
> migrations,
> we cannot promote them as the single way to manage the database schema. The
> risk of not creating the expected schema -- or, worse, not having created
> the
> expected schema in the past -- is simply too high. Every project that ever
> used 1.7 or 1.7.1 suffers from such issues.
>
> Therefore I think we must document how our user