Re: Presenting DCP, a compatibility layer for Django (feedback welcome)

[Pascal Chambon]

Hi Dmitry,

thanks for the alternative way, however it seems more a *complement *than a
replacement of DCP.

Indeed, your "rewriting" approach updates a codebase to support the latest
django API, but it raises a number of issues :

- how could it be applied to django reusable apps, installed via pip ? Does
it mean a package hook should be used to automatically transform code on
"pip install" for django-related modules, as setuptools allows it for the
2to3 case ? Or a special import hook for django ?
- it doesn't ensure backwards compatibility of the updated module, so
unless every django-app maintainers use that system to stick to the latest
django version, conflicts would remain when handling big projects.
- imo it can only handle trivial changes (renames mostly) ; when features
disappear, or are replaced by much different approaches, unless this
rewritter itself injects big shims, it wouldn't work. For example, the
SortedDict, in which keys could be inserted at arbitrary positions, has
been replaced by stdlib OrderedDict which is "readonly", so the code around
needs to be rethought accordingly.

That being said, I guess everyone would love it, if they could upgrade
THEIR codebase semi-automatically, instead of doing mass regex
search/replaces.
There are plenty of AST modifiers for python (
https://github.com/berkerpeksag/astor, or others, pytest does some nifty
ast hacking too...), else a regex based django-command would already be
nice too.

regards,
Pascal Chambon





2017-01-22 22:38 GMT+01:00 Dmitry Gladkov :

> Hi,
>
> Making Django upgrades less painful is a great goal, but I believe the
> patching Django and restoring removed functionality is not the right
> solution. JavaScript world has the same problem with far more frequent
> major compatibility breakages and they solve it with automatic codebase
> upgrade tools like jscodeshift [1]. This approach uses AST transformation
> similar to what 2to3 does, but with configurable transformation rules.
> There's also another project written in Python [2] that implements simpler
> and more general, but less reliable regex-based approach.
>
> I believe adding codebase upgrade rules for each major Django release and
> giving users ability to apply them by running something like  ./manage.py
> upgrade_from 1.7 after Django upgrade will greatly simplify upgrading of
> large Django projects.
>
> [1] https://github.com/facebook/jscodeshift
> [2] https://github.com/facebook/codemod
>
> --
> Best wishes,
> Dmitry Gladkov
>
> On 22 January 2017 at 20:53, Pascal Chambon 
> wrote:
>
>> Hi,
>>
>> @James Pc - thanks for the support, if you happen to miss some fixers in
>> DCP and don't have the opportunity to contribute them, please open an issue
>> so that I have a look
>>
>> @Tim Graham & James James Bennett - from what I sum up, the new policy
>> simply extends the delay between breaking changes, and so the overall life
>> expectancy of django reusable apps, but other limitations remain as is.
>> I've detailed the misc benefits that a compatibility layer would bring, I
>> just hope not too many people needing big compatibility will fail to learn
>> about django-compat or DCP, and complicate their life with handmade shims
>> or useless forks.
>>
>> regards,
>> Pascal
>>
>> 2017-01-16 10:48 GMT+01:00 James Bennett :
>>
>>> On Sun, Jan 15, 2017 at 1:09 PM, Pkl  wrote:
>>>
 My bad, if people are guaranteed 2 x 24-month cycles before a feature
 gets removed, it's already much better. However, the same pattern as
 previously appears in docs : "each feature release will continue to have a
 few documented backwards incompatibilities where a deprecation path isn’t
 possible or not worth the cost.". I might be paranoid, but I foresee lots
 of dependency breakages in the future, if incompatibilities continue to be
 introduced at developer's will History proved even seemingly harmless
 django modifications (ex. import aliases removed) broke external code,
 sometimes forcing "commit reverts" in django code.

>>>
>>> Just to clarify, the future plan -- beginning with Django 2.0, the next
>>> release after 1.11, which is about to feature-freeze -- is:
>>>
>>> 1. Releases go in a cycle of X.0, X.1, X.2, (X+1).0. So, for example,
>>> 2.0, then 2.1, then 2.2, then 3.0.
>>> 2. Each X.2 is an LTS.
>>> 3. Code which runs with no deprecation warnings on an LTS will also run
>>> (though may raise deprecation warnings) on the next LTS.
>>> 4. Thus, any time you run on an LTS with no deprecation warnings, you
>>> know you're safe to upgrade to the next LTS. And once you clear any new
>>> deprecation warnings on the new LTS, you know you're clear to upgrade to
>>> the one after that.
>>>
>>> Regarding backwards-incompatible changes in general: they do happen, but
>>> they also follow the guidelines in the API stability document. When they
>>> occur, it's because there's a security issue or larger bug being solved.
>>> Additionally, many of the s

Re: Presenting DCP, a compatibility layer for Django (feedback welcome)

[Raphaël Barrois]

Hi Pascal,

I'm unsure as to what problem you're trying to solve; if I understand 
correctly, this project aims at making a "new"
Django behave like an "old" one.


I see a few possible use cases in the conversation:

* Running a legacy application (without new development) beyond the upstream 
support schedule (namely Django)
* Developing libraries (or reusable apps) that need to support several Django 
versions at once
* Using libraries that don't support (yet/anymore) your chosen Django version

In my opinion, the Django project has no reason to try to support the first 
one: those projects are basically
requesting longer security support from both Django and a set of third-party 
libraries; this requires time from the
maintainers of those projects.
If the project has decided to *not* provided longer-term-support, it is often 
for good reason, mostly "nobody is paying
developers enough for that additional support to be viable — if developers are 
paid at all".


For the second part, as a library developer, I'm indeed interested in a 
compatibility layer that would allow me to use
the *new* Django APIs in older versions; not the other way around.
This way, I can benefit immediately from improvements in the Django codebase 
and prepare my code for future upgrades —
akin to Python's ``from __future__ import ...```.
However, such a feature would need to provide a *wrapper* around Django's APIs 
and not modify it: other code in
the project is likely to rely on the documented behaviour.


For the third part, I'm unsure how you'd handle various libraries having 
different Django target versions?





On Mon, 23 Jan 2017 13:09:35 +0100
Pascal Chambon  wrote:

> Hi Dmitry,
> 
> thanks for the alternative way, however it seems more a *complement *than a
> replacement of DCP.
> 
> Indeed, your "rewriting" approach updates a codebase to support the latest
> django API, but it raises a number of issues :
> 
> - how could it be applied to django reusable apps, installed via pip ? Does
> it mean a package hook should be used to automatically transform code on
> "pip install" for django-related modules, as setuptools allows it for the
> 2to3 case ? Or a special import hook for django ?
> - it doesn't ensure backwards compatibility of the updated module, so
> unless every django-app maintainers use that system to stick to the latest
> django version, conflicts would remain when handling big projects.
> - imo it can only handle trivial changes (renames mostly) ; when features
> disappear, or are replaced by much different approaches, unless this
> rewritter itself injects big shims, it wouldn't work. For example, the
> SortedDict, in which keys could be inserted at arbitrary positions, has
> been replaced by stdlib OrderedDict which is "readonly", so the code around
> needs to be rethought accordingly.
> 
> That being said, I guess everyone would love it, if they could upgrade
> THEIR codebase semi-automatically, instead of doing mass regex
> search/replaces.
> There are plenty of AST modifiers for python (
> https://github.com/berkerpeksag/astor, or others, pytest does some nifty
> ast hacking too...), else a regex based django-command would already be
> nice too.
> 
> regards,
> Pascal Chambon
> 
> 
> 
> 
> 
> 2017-01-22 22:38 GMT+01:00 Dmitry Gladkov :
> 
> > Hi,
> >
> > Making Django upgrades less painful is a great goal, but I believe the
> > patching Django and restoring removed functionality is not the right
> > solution. JavaScript world has the same problem with far more frequent
> > major compatibility breakages and they solve it with automatic codebase
> > upgrade tools like jscodeshift [1]. This approach uses AST transformation
> > similar to what 2to3 does, but with configurable transformation rules.
> > There's also another project written in Python [2] that implements simpler
> > and more general, but less reliable regex-based approach.
> >
> > I believe adding codebase upgrade rules for each major Django release and
> > giving users ability to apply them by running something like  ./manage.py
> > upgrade_from 1.7 after Django upgrade will greatly simplify upgrading of
> > large Django projects.
> >
> > [1] https://github.com/facebook/jscodeshift
> > [2] https://github.com/facebook/codemod
> >
> > --
> > Best wishes,
> > Dmitry Gladkov
> >
> > On 22 January 2017 at 20:53, Pascal Chambon 
> > wrote:
> >  
> >> Hi,
> >>
> >> @James Pc - thanks for the support, if you happen to miss some fixers in
> >> DCP and don't have the opportunity to contribute them, please open an issue
> >> so that I have a look
> >>
> >> @Tim Graham & James James Bennett - from what I sum up, the new policy
> >> simply extends the delay between breaking changes, and so the overall life
> >> expectancy of django reusable apps, but other limitations remain as is.
> >> I've detailed the misc benefits that a compatibility layer would bring, I
> >> just hope not too many people needing big compatibility will fail to learn
> >> about djang

Marking flushed sessions as not modified

[Andrew Godwin]

An interesting bug has turned up in Channels where someone is trying to
remove a session using .flush(), but our decorator sees that it's been
modified and then re-saves it, thinking it needs to persist data.

What are people's opinions on changing flush() to also set modified=False
on sessions? Is there a bad side-effect of this I haven't considered?

Andrew

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFwN1up%3D6JwN_H3FF7r31FWXzfV5ZdLUXAFfsJyiGYDfqnDvDw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Renaming the postgresql_psycopg2 backend

[Tim Graham]

The 'django.db.backends.postgresql' alias was added in Django 1.9 along 
with import shims in the old location: 
django.db.backends.postgresql_psycopg2 [0]. I'd like to remove the import 
shims at some point [1] (but keep the two line shim in django/db/utils.py 
that allows using DATABASES 'ENGINE': 
'django.db.backends.postgresql_psycopg2' indefinitely, given the minimal 
cost). Do you think subclassing and extending the built-in backend is a 
common enough use case that it's worth formally deprecating the 
postgresql_psycopg2 module rather than simply removing it in Django 2.0?

[0] 
https://github.com/django/django/commit/bcb4fe0012f8f869348ea83f5a35706f4545c44a
[1] https://github.com/django/django/pull/7938

On Sunday, July 26, 2015 at 4:21:14 AM UTC-4, Baptiste Mispelon wrote:
>
> I finally got around to creating a ticket for this: 
> https://code.djangoproject.com/ticket/25175
>
> On 07/15/2015 03:30 PM, Collin Anderson wrote:
>
> We definitely should make sure postgresql_psycopg2 still works as 
> expected. As a data point, Heroku uses "postgres" instead of "postgresql" 
> in their DATABASE_URL scheme. Maybe we could support all three? :) 
>
> On Wednesday, July 15, 2015 at 9:17:30 AM UTC-4, Baptiste Mispelon wrote: 
>>
>> Hi everyone, 
>>
>> After starting a new project recently and failing three times to type 
>> the name of the postgres backend correctly, I was wondering if there's 
>> really any value in keeping around this strange name. 
>>
>>  From what I understand, it's a historical artifact from a time when 
>> there was more than one postgres backend. 
>>
>>
>> Could we at least make it an alias of a more simply named "postgresql" 
>> backend? 
>>
>>
>> Thanks, 
>> Baptiste 
>>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-develop...@googlegroups.com .
> To post to this group, send email to django-d...@googlegroups.com 
> .
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit 
> 
> https://groups.google.com/d/msgid/django-developers/3ab4f912-fc25-4859-bd8d-5651e2c68e93%40googlegroups.com
> .
> For more options, visit https://groups.google.com/d/optout.
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/416e2f39-8c05-4db7-975f-a5dc16380598%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Marking flushed sessions as not modified

[Tim Graham]

I'm not sure offhand. Could you provide a link to the issue so we could 
look at the details?

On Monday, January 23, 2017 at 1:45:48 PM UTC-5, Andrew Godwin wrote:
>
> An interesting bug has turned up in Channels where someone is trying to 
> remove a session using .flush(), but our decorator sees that it's been 
> modified and then re-saves it, thinking it needs to persist data.
>
> What are people's opinions on changing flush() to also set modified=False 
> on sessions? Is there a bad side-effect of this I haven't considered?
>
> Andrew
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/2ae6fbc5-171f-4647-9604-29fc6833ee0a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Marking flushed sessions as not modified

[Andrew Godwin]

Here's the Channels issue underlying it:
https://github.com/django/channels/issues/477

For quick reference, the @channel_session decorator is something that wraps
a consumer, loading a session on the way in and saving it on the way out if
it changed, similar to how session middleware might work.

Andrew

On Mon, Jan 23, 2017 at 11:57 AM, Tim Graham  wrote:

> I'm not sure offhand. Could you provide a link to the issue so we could
> look at the details?
>
>
> On Monday, January 23, 2017 at 1:45:48 PM UTC-5, Andrew Godwin wrote:
>>
>> An interesting bug has turned up in Channels where someone is trying to
>> remove a session using .flush(), but our decorator sees that it's been
>> modified and then re-saves it, thinking it needs to persist data.
>>
>> What are people's opinions on changing flush() to also set modified=False
>> on sessions? Is there a bad side-effect of this I haven't considered?
>>
>> Andrew
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/django-developers/2ae6fbc5-171f-4647-9604-
> 29fc6833ee0a%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFwN1urXGAPcaOdxcJnxe4%3D8XNLS8kRo0J37WEOub9b4%3D-6haw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Renaming the postgresql_psycopg2 backend

[Adam Johnson]

For reference the lines doing the fixup in django/db/utils.py:
https://github.com/django/django/blob/master/django/db/utils.py#L108

I'm fine with simply removing it rather than putting it through the
deprecation process - I don't think subclassing database backends is a big
enough usecase, given that it's not even documented.

On 23 January 2017 at 19:50, Tim Graham  wrote:

> The 'django.db.backends.postgresql' alias was added in Django 1.9 along
> with import shims in the old location: django.db.backends.postgresql_psycopg2
> [0]. I'd like to remove the import shims at some point [1] (but keep the
> two line shim in django/db/utils.py that allows using DATABASES 'ENGINE':
> 'django.db.backends.postgresql_psycopg2' indefinitely, given the minimal
> cost). Do you think subclassing and extending the built-in backend is a
> common enough use case that it's worth formally deprecating the
> postgresql_psycopg2 module rather than simply removing it in Django 2.0?
>
> [0] https://github.com/django/django/commit/bcb4fe0012f8f869348ea83f5a3570
> 6f4545c44a
> [1] https://github.com/django/django/pull/7938
>
> On Sunday, July 26, 2015 at 4:21:14 AM UTC-4, Baptiste Mispelon wrote:
>>
>> I finally got around to creating a ticket for this:
>> https://code.djangoproject.com/ticket/25175
>>
>> On 07/15/2015 03:30 PM, Collin Anderson wrote:
>>
>> We definitely should make sure postgresql_psycopg2 still works as
>> expected. As a data point, Heroku uses "postgres" instead of "postgresql"
>> in their DATABASE_URL scheme. Maybe we could support all three? :)
>>
>> On Wednesday, July 15, 2015 at 9:17:30 AM UTC-4, Baptiste Mispelon wrote:
>>>
>>> Hi everyone,
>>>
>>> After starting a new project recently and failing three times to type
>>> the name of the postgres backend correctly, I was wondering if there's
>>> really any value in keeping around this strange name.
>>>
>>>  From what I understand, it's a historical artifact from a time when
>>> there was more than one postgres backend.
>>>
>>>
>>> Could we at least make it an alias of a more simply named "postgresql"
>>> backend?
>>>
>>>
>>> Thanks,
>>> Baptiste
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to django-develop...@googlegroups.com.
>> To post to this group, send email to django-d...@googlegroups.com.
>> Visit this group at http://groups.google.com/group/django-developers.
>> To view this discussion on the web visit
>> 
>> https://groups.google.com/d/msgid/django-developers/3ab4f912
>> -fc25-4859-bd8d-5651e2c68e93%40googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/django-developers/416e2f39-8c05-4db7-975f-
> a5dc16380598%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Adam

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAMyDDM2w-tOF4bkMdQ6mUvRbvjg-A7sT2-NDUAYL7458Y4_KQQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: changing the default MySQL isolation level to READ COMMITTED

[Adam Johnson]

Sorry for the radio silence, I've been busy at work.

After re-reading things and thinking a bit more about it, I think Tim's
suggestion of just changing the default in a release is probably the right
idea. The warning + check does seem very bothersome; as Aymeric says, this
is very far from the concerns of most Django users. We might still want a
system check for if the user is using repeatable-read as a guard against
the bugs that have been seen.

Anyway presumably it's too late to make the change in 1.11 because feature
freeze means feature freeze, so I guess we target this at 2.0 now.

On 13 January 2017 at 15:44, Tim Graham  wrote:

> In the end, I don't use MySQL but if a similar change were made for
> PostgreSQL, I would find the current approach more annoying (bothering me
> about changing defaults over two release cycles) than cautious.
>
> I'm also uncertain that duplicating a deprecation warning in a system
> check is a good precedent to set. Adam, is the current implementation what
> you envisioned when you made your comment and what do you see as the
> advantages of that?
>
> Aymeric, I assume "I’d love if this fix made it into 1.10" was a typo for
> 1.11. Anyway, the "Allowed transaction isolation level to be chosen on
> MySQL" commit isn't controversial. I'll try to get at least that in.
>
> On Friday, January 13, 2017 at 10:11:26 AM UTC-5, Adam Johnson wrote:
>>
>>  aside from some very performance-sensitive websites that already worked
>>> around Django’s bugs
>>
>>
>> Tbf it's true I already added READ-COMMITTED at work
>>
>>  I’d love if this fix made it into 1.10
>>
>>
>>
>>
>>
>> On 13 January 2017 at 15:05, Aymeric Augustin <
>> aymeric.au...@polytechnique.org> wrote:
>>
>>> Hello,
>>>
>>> I think there’s been little feedback because this issue is very, very
>>> far from the concerns of most Django users — aside from some very
>>> performance-sensitive websites that already worked around Django’s bugs and
>>> will add the configuration line needed to keep whatever the isolation level
>>> they chose.
>>>
>>> If I was trying to merge that change, I think I’d just document the
>>> backwards incompatibility and call it a day. But as everyone knows, I’m
>>> taking backwards compatibility less seriously than most.
>>>
>>> Given the uncertainty around the consequences of this change, I don’t
>>> think Shai’s approach is out of place, even though it creates an overhead
>>> for every Django project using MySQL.
>>>
>>> Since Shai is leading the effort and considering the general lack of
>>> feedback on this issue, I think it’s fair to let him make the final call
>>> and keep the deprecation path if he thinks that’s safer.
>>>
>>> Regardless, I’d love if this fix made it into 1.10, let’s not delay it
>>> because we’re worried of being too cautious :-)
>>>
>>> --
>>> Aymeric.
>>>
>>>
>>> On 13 Jan 2017, at 14:52, Tim Graham  wrote:
>>>
>>> I guess three days is too little time to get a consensus on this. At
>>> this point I'm thinking to defer this from 1.11.
>>>
>>> On Wednesday, January 11, 2017 at 9:50:27 AM UTC-5, Patryk Zawadzki
>>> wrote:

 To add some context, REPEATABLE READ can break get_or_create among
 other things (not sure how many invalid tickets Django gets related to
 that).

>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Django developers (Contributions to Django itself)" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to django-develop...@googlegroups.com.
>>> To post to this group, send email to django-d...@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/django-developers.
>>> To view this discussion on the web visit https://groups.google.com/d/ms
>>> gid/django-developers/42212944-d7e7-4c33-8c34-8e235a7d515a%
>>> 40googlegroups.com
>>> 
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Django developers (Contributions to Django itself)" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to django-develop...@googlegroups.com.
>>> To post to this group, send email to django-d...@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/django-developers.
>>> To view this discussion on the web visit https://groups.google.com/d/ms
>>> gid/django-developers/2D7B4E2A-30EF-4201-9272-C5430825E123%
>>> 40polytechnique.org
>>> 
>>> .
>>>
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>
>>
>> --
>> Adam
>>
> --
> You received this message because you are subscribed t

Re: changing the default MySQL isolation level to READ COMMITTED

[Adam Johnson]

Sorry for my radio/email silence, I had a lot of things to do at work and 
wanted to think this through a bit more.

Looking again at the warning + check implementation, it is a bit messy. My 
goal with the check was to get a message in front of users who need it, but 
I suppose the release notes are good enough for this. As Aymeric says, this 
is very far from most Django users' concerns; it would be bothersome for 
most users to have to change some configuration. So I think we can get away 
with changing the isolation level in a release without the pre-warning 
message. Potentially we could still have a system check that returns a 
warning if using repeatable-read, because it can lead to all the bugs we've 
seen.

Presumably it's too late to make the change in 1.11 (or is it...?) so 2.0 
it is.

On Friday, January 13, 2017 at 3:44:31 PM UTC, Tim Graham wrote:
>
> In the end, I don't use MySQL but if a similar change were made for 
> PostgreSQL, I would find the current approach more annoying (bothering me 
> about changing defaults over two release cycles) than cautious.
>
> I'm also uncertain that duplicating a deprecation warning in a system 
> check is a good precedent to set. Adam, is the current implementation what 
> you envisioned when you made your comment and what do you see as the 
> advantages of that?
>
> Aymeric, I assume "I’d love if this fix made it into 1.10" was a typo for 
> 1.11. Anyway, the "Allowed transaction isolation level to be chosen on 
> MySQL" commit isn't controversial. I'll try to get at least that in.
>
> On Friday, January 13, 2017 at 10:11:26 AM UTC-5, Adam Johnson wrote:
>>
>>  aside from some very performance-sensitive websites that already worked 
>>> around Django’s bugs
>>
>>
>> Tbf it's true I already added READ-COMMITTED at work
>>
>>  I’d love if this fix made it into 1.10
>>
>>
>>
>>  
>>
>> On 13 January 2017 at 15:05, Aymeric Augustin <
>> aymeric.au...@polytechnique.org> wrote:
>>
>>> Hello,
>>>
>>> I think there’s been little feedback because this issue is very, very 
>>> far from the concerns of most Django users — aside from some very 
>>> performance-sensitive websites that already worked around Django’s bugs and 
>>> will add the configuration line needed to keep whatever the isolation level 
>>> they chose.
>>>
>>> If I was trying to merge that change, I think I’d just document the 
>>> backwards incompatibility and call it a day. But as everyone knows, I’m 
>>> taking backwards compatibility less seriously than most.
>>>
>>> Given the uncertainty around the consequences of this change, I don’t 
>>> think Shai’s approach is out of place, even though it creates an overhead 
>>> for every Django project using MySQL.
>>>
>>> Since Shai is leading the effort and considering the general lack of 
>>> feedback on this issue, I think it’s fair to let him make the final call 
>>> and keep the deprecation path if he thinks that’s safer.
>>>
>>> Regardless, I’d love if this fix made it into 1.10, let’s not delay it 
>>> because we’re worried of being too cautious :-)
>>>
>>> -- 
>>> Aymeric.
>>>
>>>
>>> On 13 Jan 2017, at 14:52, Tim Graham  wrote:
>>>
>>> I guess three days is too little time to get a consensus on this. At 
>>> this point I'm thinking to defer this from 1.11.
>>>
>>> On Wednesday, January 11, 2017 at 9:50:27 AM UTC-5, Patryk Zawadzki 
>>> wrote:

 To add some context, REPEATABLE READ can break get_or_create among 
 other things (not sure how many invalid tickets Django gets related to 
 that).

>>>
>>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "Django developers (Contributions to Django itself)" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to django-develop...@googlegroups.com.
>>> To post to this group, send email to django-d...@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/django-developers.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/django-developers/42212944-d7e7-4c33-8c34-8e235a7d515a%40googlegroups.com
>>>  
>>> 
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>>
>>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "Django developers (Contributions to Django itself)" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to django-develop...@googlegroups.com.
>>> To post to this group, send email to django-d...@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/django-developers.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/django-developers/2D7B4E2A-30EF-4201-9272-C5430825E123%40polytechnique.org
>>>  
>>> 

Re: changing the default MySQL isolation level to READ COMMITTED

[Adam Johnson]

(I sent basically the same email twice because it looked to me like the
first was lost, didn't even show in my outbox :/)

On 23 January 2017 at 21:15, Adam Johnson  wrote:

> Sorry for the radio silence, I've been busy at work.
>
> After re-reading things and thinking a bit more about it, I think Tim's
> suggestion of just changing the default in a release is probably the right
> idea. The warning + check does seem very bothersome; as Aymeric says, this
> is very far from the concerns of most Django users. We might still want a
> system check for if the user is using repeatable-read as a guard against
> the bugs that have been seen.
>
> Anyway presumably it's too late to make the change in 1.11 because feature
> freeze means feature freeze, so I guess we target this at 2.0 now.
>
> On 13 January 2017 at 15:44, Tim Graham  wrote:
>
>> In the end, I don't use MySQL but if a similar change were made for
>> PostgreSQL, I would find the current approach more annoying (bothering me
>> about changing defaults over two release cycles) than cautious.
>>
>> I'm also uncertain that duplicating a deprecation warning in a system
>> check is a good precedent to set. Adam, is the current implementation what
>> you envisioned when you made your comment and what do you see as the
>> advantages of that?
>>
>> Aymeric, I assume "I’d love if this fix made it into 1.10" was a typo for
>> 1.11. Anyway, the "Allowed transaction isolation level to be chosen on
>> MySQL" commit isn't controversial. I'll try to get at least that in.
>>
>> On Friday, January 13, 2017 at 10:11:26 AM UTC-5, Adam Johnson wrote:
>>>
>>>  aside from some very performance-sensitive websites that already worked
 around Django’s bugs
>>>
>>>
>>> Tbf it's true I already added READ-COMMITTED at work
>>>
>>>  I’d love if this fix made it into 1.10
>>>
>>>
>>>
>>>
>>>
>>> On 13 January 2017 at 15:05, Aymeric Augustin <
>>> aymeric.au...@polytechnique.org> wrote:
>>>
 Hello,

 I think there’s been little feedback because this issue is very, very
 far from the concerns of most Django users — aside from some very
 performance-sensitive websites that already worked around Django’s bugs and
 will add the configuration line needed to keep whatever the isolation level
 they chose.

 If I was trying to merge that change, I think I’d just document the
 backwards incompatibility and call it a day. But as everyone knows, I’m
 taking backwards compatibility less seriously than most.

 Given the uncertainty around the consequences of this change, I don’t
 think Shai’s approach is out of place, even though it creates an overhead
 for every Django project using MySQL.

 Since Shai is leading the effort and considering the general lack of
 feedback on this issue, I think it’s fair to let him make the final call
 and keep the deprecation path if he thinks that’s safer.

 Regardless, I’d love if this fix made it into 1.10, let’s not delay it
 because we’re worried of being too cautious :-)

 --
 Aymeric.


 On 13 Jan 2017, at 14:52, Tim Graham  wrote:

 I guess three days is too little time to get a consensus on this. At
 this point I'm thinking to defer this from 1.11.

 On Wednesday, January 11, 2017 at 9:50:27 AM UTC-5, Patryk Zawadzki
 wrote:
>
> To add some context, REPEATABLE READ can break get_or_create among
> other things (not sure how many invalid tickets Django gets related to
> that).
>

 --
 You received this message because you are subscribed to the Google
 Groups "Django developers (Contributions to Django itself)" group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to django-develop...@googlegroups.com.
 To post to this group, send email to django-d...@googlegroups.com.
 Visit this group at https://groups.google.com/group/django-developers.
 To view this discussion on the web visit https://groups.google.com/d/ms
 gid/django-developers/42212944-d7e7-4c33-8c34-8e235a7d515a%4
 0googlegroups.com
 
 .
 For more options, visit https://groups.google.com/d/optout.


 --
 You received this message because you are subscribed to the Google
 Groups "Django developers (Contributions to Django itself)" group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to django-develop...@googlegroups.com.
 To post to this group, send email to django-d...@googlegroups.com.
 Visit this group at https://groups.google.com/group/django-developers.
 To view this discussion on the web visit https://groups.google.com/d/ms
 gid/django-developers/2D7B4E2A-30EF-4201-9272-C5430825E123%4
 0polytechnique.org
 

Re: Renaming the postgresql_psycopg2 backend

[Raffaele Salmaso]

On Mon, Jan 23, 2017 at 8:50 PM, Tim Graham  wrote:

> but keep the two line shim in django/db/utils.py that allows using
> DATABASES 'ENGINE': 'django.db.backends.postgresql_psycopg2'
> indefinitely, given the minimal cost
>
I prefer to remove all references, and have an explicit error on
unavailable backend.

-- 
| Raffaele Salmaso
| https://salmaso.org
| https://bitbucket.org/rsalmaso
| https://github.com/rsalmaso

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CABgH4JtdLCo8ofZKGtnuJY97JM6ebYz29Sa7sGri-9RXk3G5JQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Renaming the postgresql_psycopg2 backend

[Claude Paroz]

Le lundi 23 janvier 2017 20:50:03 UTC+1, Tim Graham a écrit :
>
> (...) Do you think subclassing and extending the built-in backend is a 
> common enough use case that it's worth formally deprecating the 
> postgresql_psycopg2 module rather than simply removing it in Django 2.0?
>

I am generally strongly against simply dropping non-internal stuff. In 
those cases, I would be for a short deprecation period (without pending 
deprecation) so as at least those who follow the good practice of checking 
their test suite is deprecation-free on Django n before upgrading to Django 
n + 1 can have a good experience.

Claude

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/1d97196b-877f-4d79-8e74-97b1c38d7557%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.