Django 3.1 Released

[Mariusz Felisiak]

Details are available on the Django project weblog:

https://www.djangoproject.com/weblog/2020/aug/04/django-31-released/

--
You received this message because you are subscribed to the Google Groups "Django 
developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/001b3256-3624-26f8-a3c9-6ff1cb348eba%40gmail.com.

FK field caching behavior change between 1.11.x and 2.x

[Gert Burger]

(initially posted to 
https://forum.djangoproject.com/t/fk-field-caching-behavior-change-between-1-11-x-and-2-x/3151
 
on 26 June but this mailing list might be more appropriate)

Hi

Whilst upgrading a codebase from 1.11.x to 2.0/2.2 I noticed a weird change 
in behavior of FK fields when copying model instances.

At the bottom of the post there is a testcase that succeeds on 1.11.x and 
fails on 2.x
I think the commit that changed the behavior is 
bfb746f983aa741afa3709794e70f1e0ab6040b5

So my question is two fold:

   1. Is the behavior in >=2.0 correct? It seems quite unexpected.
   2. What is the recommended way to clone a model instance? To date we 
   have been using copy() in a similar fashion to the test without issue. 
   deepcopy seems to work fine in >=2.0 but we haven’t done too much 
   testing yet.

Test (placed in tests/model_fields/test_field_caching_change.py):

-code--

import copy
from django.test import TestCase

from .models import Bar, Foo


class ForeignKeyCachingBehaviorTest(TestCase):
def test_copy(self):
foo1 = Foo.objects.create(a='foo1', d=1)
foo2 = Foo.objects.create(a='foo2', d=2)
bar1 = Bar.objects.create(a=foo1, b='bar1')

bar2 = copy.copy(bar1)

bar2.pk = None
bar2.a = foo2

# bar2 points to foo2
self.assertEqual(bar2.a, foo2)
self.assertEqual(bar2.a.id, bar2.a_id)

# bar1 is unchanged and must still point to foo1
# These fail on Django >= 2.0
self.assertEqual(bar1.a, foo1)
self.assertEqual(bar1.a.id, bar1.a_id)

-code--

and executed that via:

python3.6 tests/runtests.py --parallel 1 model_fields

Regards
Gert Burger

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/7b043b5d-dec2-4479-9c7c-a20b588693acn%40googlegroups.com.

Re: FK field caching behavior change between 1.11.x and 2.x

[charettes]

Hello Gert, that seems a bit surprising to me and was likely not a desired 
change.

Could you bisect the exact commit that caused the regression[0]? That would 
certainly help determining the action to take here.

Thanks,
Simon

[0] 
https://docs.djangoproject.com/en/3.1/internals/contributing/triaging-tickets/#bisecting-a-regression

Le mardi 4 août 2020 à 08:10:00 UTC-4, gertb...@gmail.com a écrit :

> (initially posted to 
> https://forum.djangoproject.com/t/fk-field-caching-behavior-change-between-1-11-x-and-2-x/3151
>  
> on 26 June but this mailing list might be more appropriate)
>
> Hi
>
> Whilst upgrading a codebase from 1.11.x to 2.0/2.2 I noticed a weird 
> change in behavior of FK fields when copying model instances.
>
> At the bottom of the post there is a testcase that succeeds on 1.11.x and 
> fails on 2.x
> I think the commit that changed the behavior is 
> bfb746f983aa741afa3709794e70f1e0ab6040b5
>
> So my question is two fold:
>
>1. Is the behavior in >=2.0 correct? It seems quite unexpected.
>2. What is the recommended way to clone a model instance? To date we 
>have been using copy() in a similar fashion to the test without issue. 
>deepcopy seems to work fine in >=2.0 but we haven’t done too much 
>testing yet.
>
> Test (placed in tests/model_fields/test_field_caching_change.py):
>
> -code--
>
> import copy
> from django.test import TestCase
>
> from .models import Bar, Foo
>
>
> class ForeignKeyCachingBehaviorTest(TestCase):
> def test_copy(self):
> foo1 = Foo.objects.create(a='foo1', d=1)
> foo2 = Foo.objects.create(a='foo2', d=2)
> bar1 = Bar.objects.create(a=foo1, b='bar1')
>
> bar2 = copy.copy(bar1)
>
> bar2.pk = None
> bar2.a = foo2
>
> # bar2 points to foo2
> self.assertEqual(bar2.a, foo2)
> self.assertEqual(bar2.a.id, bar2.a_id)
>
> # bar1 is unchanged and must still point to foo1
> # These fail on Django >= 2.0
> self.assertEqual(bar1.a, foo1)
> self.assertEqual(bar1.a.id, bar1.a_id)
>
> -code--
>
> and executed that via:
>
> python3.6 tests/runtests.py --parallel 1 model_fields
>
> Regards
> Gert Burger
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/c284e35e-ae02-44e5-b44d-4994eff9b248n%40googlegroups.com.

Re: FK field caching behavior change between 1.11.x and 2.x

[Gert Burger]

Hi Simon,

I think the commit is bfb746f983aa741afa3709794e70f1e0ab6040b5 "Refs #16043
-- Refactored internal fields value cache".

Cheers

On Tue, 4 Aug 2020 at 15:52, charettes  wrote:

> Hello Gert, that seems a bit surprising to me and was likely not a desired
> change.
>
> Could you bisect the exact commit that caused the regression[0]? That
> would certainly help determining the action to take here.
>
> Thanks,
> Simon
>
> [0]
> https://docs.djangoproject.com/en/3.1/internals/contributing/triaging-tickets/#bisecting-a-regression
>
> Le mardi 4 août 2020 à 08:10:00 UTC-4, gertb...@gmail.com a écrit :
>
>> (initially posted to
>> https://forum.djangoproject.com/t/fk-field-caching-behavior-change-between-1-11-x-and-2-x/3151
>> on 26 June but this mailing list might be more appropriate)
>>
>> Hi
>>
>> Whilst upgrading a codebase from 1.11.x to 2.0/2.2 I noticed a weird
>> change in behavior of FK fields when copying model instances.
>>
>> At the bottom of the post there is a testcase that succeeds on 1.11.x and
>> fails on 2.x
>> I think the commit that changed the behavior is
>> bfb746f983aa741afa3709794e70f1e0ab6040b5
>>
>> So my question is two fold:
>>
>>1. Is the behavior in >=2.0 correct? It seems quite unexpected.
>>2. What is the recommended way to clone a model instance? To date we
>>have been using copy() in a similar fashion to the test without
>>issue. deepcopy seems to work fine in >=2.0 but we haven’t done too
>>much testing yet.
>>
>> Test (placed in tests/model_fields/test_field_caching_change.py):
>>
>> -code--
>>
>> import copy
>> from django.test import TestCase
>>
>> from .models import Bar, Foo
>>
>>
>> class ForeignKeyCachingBehaviorTest(TestCase):
>> def test_copy(self):
>> foo1 = Foo.objects.create(a='foo1', d=1)
>> foo2 = Foo.objects.create(a='foo2', d=2)
>> bar1 = Bar.objects.create(a=foo1, b='bar1')
>>
>> bar2 = copy.copy(bar1)
>>
>> bar2.pk = None
>> bar2.a = foo2
>>
>> # bar2 points to foo2
>> self.assertEqual(bar2.a, foo2)
>> self.assertEqual(bar2.a.id, bar2.a_id)
>>
>> # bar1 is unchanged and must still point to foo1
>> # These fail on Django >= 2.0
>> self.assertEqual(bar1.a, foo1)
>> self.assertEqual(bar1.a.id, bar1.a_id)
>>
>> -code--
>>
>> and executed that via:
>>
>> python3.6 tests/runtests.py --parallel 1 model_fields
>>
>> Regards
>> Gert Burger
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/c284e35e-ae02-44e5-b44d-4994eff9b248n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAEQ1D3nzSBnygSBMJnJOiWFWPVQt8gwMo96qjMbUEYwi3UkdeQ%40mail.gmail.com.

Exception [IntegrityError] - violates not-null constraint in email field

[Francisco André]

Hey guys.

I'm developing a new project and creating a custom user model, extending 
the AbstractBaseUser class, but when add a new user in admin interface or 
modelform I get the error: IntegrityError at /null value in column "email" 
violates not-null constraint.

The field email is necessary and I inform in the form, it's used to login 
too.

It's necessary create a "custom modelform" too? I followed many  tutorials 
by web, including documentation. Links bellow.

https://docs.djangoproject.com/en/3.0/topics/auth/customizing/#custom-users-admin-full-example
https://simpleisbetterthancomplex.com/tutorial/2016/07/22/how-to-extend-django-user-model.html#abstractbaseuser

And, this my gist code of model, form and admin file.

https://gist.github.com/fandrefh/ff6d8bbe047654174aa699072589539a

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/34e04b84-52d5-4cf0-9667-501969ea2370o%40googlegroups.com.

Re: FK field caching behavior change between 1.11.x and 2.x

[charettes]

It was likely overlooked by the patch.

Looks like Model.__copy__ should make sure to make a deep-copy of 
self._state now that fields are cached in self._state.fields_cache.

Using copy.deepcopy will circumvent the issue but I feel like copy.copy is 
common enough pattern that we should maintain compatibility for it. After 
all, the fact the _state_ of the model is now stored in _state.fields_cache 
is no more than an implementation detail and it would be normal to expect 
shallow copying to work in this case. On the other hand the fact this has 
been happening since Django 2.0 and has just been reported and that shallow 
copies won't work correctly for prefetched relationships are both 
non-negligible argument for a wontfix since this pattern was never 
explicitly documented.

There is a precedent for a similar undocumented pattern though [0].

Thoughts?

Simon

[0] 
https://github.com/django/django/commit/dee5896b55385d2c66d8c1b8386604868f7fc6b4

Le mardi 4 août 2020 à 10:48:43 UTC-4, gertb...@gmail.com a écrit :

> Hi Simon,
>
> I think the commit is bfb746f983aa741afa3709794e70f1e0ab6040b5 "Refs 
> #16043 -- Refactored internal fields value cache".
>
> Cheers
>
> On Tue, 4 Aug 2020 at 15:52, charettes  wrote:
>
>> Hello Gert, that seems a bit surprising to me and was likely not a 
>> desired change.
>>
>> Could you bisect the exact commit that caused the regression[0]? That 
>> would certainly help determining the action to take here.
>>
>> Thanks,
>> Simon
>>
>> [0] 
>> https://docs.djangoproject.com/en/3.1/internals/contributing/triaging-tickets/#bisecting-a-regression
>>
>> Le mardi 4 août 2020 à 08:10:00 UTC-4, gertb...@gmail.com a écrit :
>>
>>> (initially posted to 
>>> https://forum.djangoproject.com/t/fk-field-caching-behavior-change-between-1-11-x-and-2-x/3151
>>>  
>>> on 26 June but this mailing list might be more appropriate)
>>>
>>> Hi
>>>
>>> Whilst upgrading a codebase from 1.11.x to 2.0/2.2 I noticed a weird 
>>> change in behavior of FK fields when copying model instances.
>>>
>>> At the bottom of the post there is a testcase that succeeds on 1.11.x 
>>> and fails on 2.x
>>> I think the commit that changed the behavior is 
>>> bfb746f983aa741afa3709794e70f1e0ab6040b5
>>>
>>> So my question is two fold:
>>>
>>>1. Is the behavior in >=2.0 correct? It seems quite unexpected.
>>>2. What is the recommended way to clone a model instance? To date we 
>>>have been using copy() in a similar fashion to the test without 
>>>issue. deepcopy seems to work fine in >=2.0 but we haven’t done too 
>>>much testing yet.
>>>
>>> Test (placed in tests/model_fields/test_field_caching_change.py):
>>>
>>> -code--
>>>
>>> import copy
>>> from django.test import TestCase
>>>
>>> from .models import Bar, Foo
>>>
>>>
>>> class ForeignKeyCachingBehaviorTest(TestCase):
>>> def test_copy(self):
>>> foo1 = Foo.objects.create(a='foo1', d=1)
>>> foo2 = Foo.objects.create(a='foo2', d=2)
>>> bar1 = Bar.objects.create(a=foo1, b='bar1')
>>>
>>> bar2 = copy.copy(bar1)
>>>
>>> bar2.pk = None
>>> bar2.a = foo2
>>>
>>> # bar2 points to foo2
>>> self.assertEqual(bar2.a, foo2)
>>> self.assertEqual(bar2.a.id, bar2.a_id)
>>>
>>> # bar1 is unchanged and must still point to foo1
>>> # These fail on Django >= 2.0
>>> self.assertEqual(bar1.a, foo1)
>>> self.assertEqual(bar1.a.id, bar1.a_id)
>>>
>>> -code--
>>>
>>> and executed that via:
>>>
>>> python3.6 tests/runtests.py --parallel 1 model_fields
>>>
>>> Regards
>>> Gert Burger
>>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-develop...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/c284e35e-ae02-44e5-b44d-4994eff9b248n%40googlegroups.com
>>  
>> 
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/0b5d48d0-5594-40d6-9dce-80995f8635ecn%40googlegroups.com.

Re: FK field caching behavior change between 1.11.x and 2.x

[Alexander Hill]

I reckon stick with your first instinct SImon.

I don't think using copy.copy needs to be an explicitly documented pattern.
It's a heavily-used part of the standard library, and the objects Django
provides should work with it as well as they can. The behaviour is
surprising and buggy at face value: changing the value of an attribute on
one object should not change the value of that attribute on another!

Alex

On Wed, Aug 5, 2020 at 7:43 AM charettes  wrote:

> It was likely overlooked by the patch.
>
> Looks like Model.__copy__ should make sure to make a deep-copy of
> self._state now that fields are cached in self._state.fields_cache.
>
> Using copy.deepcopy will circumvent the issue but I feel like copy.copy is
> common enough pattern that we should maintain compatibility for it. After
> all, the fact the _state_ of the model is now stored in _state.fields_cache
> is no more than an implementation detail and it would be normal to expect
> shallow copying to work in this case. On the other hand the fact this has
> been happening since Django 2.0 and has just been reported and that shallow
> copies won't work correctly for prefetched relationships are both
> non-negligible argument for a wontfix since this pattern was never
> explicitly documented.
>
> There is a precedent for a similar undocumented pattern though [0].
>
> Thoughts?
>
> Simon
>
> [0]
> https://github.com/django/django/commit/dee5896b55385d2c66d8c1b8386604868f7fc6b4
>
> Le mardi 4 août 2020 à 10:48:43 UTC-4, gertb...@gmail.com a écrit :
>
>> Hi Simon,
>>
>> I think the commit is bfb746f983aa741afa3709794e70f1e0ab6040b5 "Refs
>> #16043 -- Refactored internal fields value cache".
>>
>> Cheers
>>
>> On Tue, 4 Aug 2020 at 15:52, charettes  wrote:
>>
>>> Hello Gert, that seems a bit surprising to me and was likely not a
>>> desired change.
>>>
>>> Could you bisect the exact commit that caused the regression[0]? That
>>> would certainly help determining the action to take here.
>>>
>>> Thanks,
>>> Simon
>>>
>>> [0]
>>> https://docs.djangoproject.com/en/3.1/internals/contributing/triaging-tickets/#bisecting-a-regression
>>>
>>> Le mardi 4 août 2020 à 08:10:00 UTC-4, gertb...@gmail.com a écrit :
>>>
 (initially posted to
 https://forum.djangoproject.com/t/fk-field-caching-behavior-change-between-1-11-x-and-2-x/3151
 on 26 June but this mailing list might be more appropriate)

 Hi

 Whilst upgrading a codebase from 1.11.x to 2.0/2.2 I noticed a weird
 change in behavior of FK fields when copying model instances.

 At the bottom of the post there is a testcase that succeeds on 1.11.x
 and fails on 2.x
 I think the commit that changed the behavior is
 bfb746f983aa741afa3709794e70f1e0ab6040b5

 So my question is two fold:

1. Is the behavior in >=2.0 correct? It seems quite unexpected.
2. What is the recommended way to clone a model instance? To date
we have been using copy() in a similar fashion to the test without
issue. deepcopy seems to work fine in >=2.0 but we haven’t done too
much testing yet.

 Test (placed in tests/model_fields/test_field_caching_change.py):

 -code--

 import copy
 from django.test import TestCase

 from .models import Bar, Foo


 class ForeignKeyCachingBehaviorTest(TestCase):
 def test_copy(self):
 foo1 = Foo.objects.create(a='foo1', d=1)
 foo2 = Foo.objects.create(a='foo2', d=2)
 bar1 = Bar.objects.create(a=foo1, b='bar1')

 bar2 = copy.copy(bar1)

 bar2.pk = None
 bar2.a = foo2

 # bar2 points to foo2
 self.assertEqual(bar2.a, foo2)
 self.assertEqual(bar2.a.id, bar2.a_id)

 # bar1 is unchanged and must still point to foo1
 # These fail on Django >= 2.0
 self.assertEqual(bar1.a, foo1)
 self.assertEqual(bar1.a.id, bar1.a_id)

 -code--

 and executed that via:

 python3.6 tests/runtests.py --parallel 1 model_fields

 Regards
 Gert Burger

>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Django developers (Contributions to Django itself)" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to django-develop...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/django-developers/c284e35e-ae02-44e5-b44d-4994eff9b248n%40googlegroups.com
>>> 
>>> .
>>>
>> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubs