Re: Renaming the postgresql_psycopg2 backend

[Josh Smeaton]

I remembered a thing from Michaels talk at #DUTH. Let me present a use case 
for subclassing a backend:

https://github.com/opbeat/django-postgres-readonly/blob/master/django_postgres_readonly/base.py

I think if we end up favouring immediate deprecation, we could proactively 
find and inform backend maintainers that would be affected. Their users 
might not be so appreciative though. I think Claude's position is a good 
one. A short deprecation period at a minimum.

For my own interest I just did a quick search on PyPI for `django postgres` 
and found the following (having various levels of django version support):

https://github.com/2degrees/django-postgres-delete-cascade/blob/master/django_postgres_delete_cascade/base.py
https://github.com/kennethreitz/django-postgrespool/blob/master/django_postgrespool/base.py
https://github.com/jdelic/django-postgresql-setrole/blob/master/postgresql_setrole/__init__.py
 
(this one imports from the new location!)

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/1755d125-ee5e-402f-bbd2-c6edb4c720c6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Presenting DCP, a compatibility layer for Django (feedback welcome)

[Pascal Chambon]

Hi Raphaël ,

the goal of DCP is to "unite" the behaviour of different django versions,
so that all the code needed by a project (misc. pypi dependencies, as well
as custom project modules) may work together, whatever the exact django
version each of these module actually targets. That's closer to your 3).

At the moment, available fixers are all for backwards compatibility, so the
idea is to install the latest django version (with latest security
features), and be able to run all kinds of dependencies with it, even if
they haven't yet been updated. If fixers for forward compatibility got
contributed, people could at the contrary remain on an old django version,
yet begin to use new APIs and features. As far as I've seen, most changes
can be patched in a way which allows both the old and new behaviour to work
at the same time (that's what happens during the PendingDeprecation period
of a change, by the way).

However, note that DCP is mainly aimed at *project maintainers*, eg. a
specific website or saas. When developing a pluggable apps or library, you
can't expect all your users to install and activate DCP in their own
project, so a compatibility toolkit like "django-compat" (
https://github.com/arteria/django-compat) is more relevant - at least as
long as no DCP-like system is built into django.

regards,
Pascal




2017-01-23 13:37 GMT+01:00 Raphaël Barrois :

> Hi Pascal,
>
> I'm unsure as to what problem you're trying to solve; if I understand
> correctly, this project aims at making a "new"
> Django behave like an "old" one.
>
>
> I see a few possible use cases in the conversation:
>
> * Running a legacy application (without new development) beyond the
> upstream support schedule (namely Django)
> * Developing libraries (or reusable apps) that need to support several
> Django versions at once
> * Using libraries that don't support (yet/anymore) your chosen Django
> version
>
> In my opinion, the Django project has no reason to try to support the
> first one: those projects are basically
> requesting longer security support from both Django and a set of
> third-party libraries; this requires time from the
> maintainers of those projects.
> If the project has decided to *not* provided longer-term-support, it is
> often for good reason, mostly "nobody is paying
> developers enough for that additional support to be viable — if developers
> are paid at all".
>
>
> For the second part, as a library developer, I'm indeed interested in a
> compatibility layer that would allow me to use
> the *new* Django APIs in older versions; not the other way around.
> This way, I can benefit immediately from improvements in the Django
> codebase and prepare my code for future upgrades —
> akin to Python's ``from __future__ import ...```.
> However, such a feature would need to provide a *wrapper* around Django's
> APIs and not modify it: other code in
> the project is likely to rely on the documented behaviour.
>
>
> For the third part, I'm unsure how you'd handle various libraries having
> different Django target versions?
>
>
>
>
>
> On Mon, 23 Jan 2017 13:09:35 +0100
> Pascal Chambon  wrote:
>
> > Hi Dmitry,
> >
> > thanks for the alternative way, however it seems more a *complement
> *than a
> > replacement of DCP.
> >
> > Indeed, your "rewriting" approach updates a codebase to support the
> latest
> > django API, but it raises a number of issues :
> >
> > - how could it be applied to django reusable apps, installed via pip ?
> Does
> > it mean a package hook should be used to automatically transform code on
> > "pip install" for django-related modules, as setuptools allows it for the
> > 2to3 case ? Or a special import hook for django ?
> > - it doesn't ensure backwards compatibility of the updated module, so
> > unless every django-app maintainers use that system to stick to the
> latest
> > django version, conflicts would remain when handling big projects.
> > - imo it can only handle trivial changes (renames mostly) ; when features
> > disappear, or are replaced by much different approaches, unless this
> > rewritter itself injects big shims, it wouldn't work. For example, the
> > SortedDict, in which keys could be inserted at arbitrary positions, has
> > been replaced by stdlib OrderedDict which is "readonly", so the code
> around
> > needs to be rethought accordingly.
> >
> > That being said, I guess everyone would love it, if they could upgrade
> > THEIR codebase semi-automatically, instead of doing mass regex
> > search/replaces.
> > There are plenty of AST modifiers for python (
> > https://github.com/berkerpeksag/astor, or others, pytest does some nifty
> > ast hacking too...), else a regex based django-command would already be
> > nice too.
> >
> > regards,
> > Pascal Chambon
> >
> >
> >
> >
> >
> > 2017-01-22 22:38 GMT+01:00 Dmitry Gladkov :
> >
> > > Hi,
> > >
> > > Making Django upgrades less painful is a great goal, but I believe the
> > > patching Django and restoring removed functionality is not the 

Re: Renaming the postgresql_psycopg2 backend

[Adam Johnson]

Ok fair point, I agree it should go through the short deprecation process.
It's also very small and not that bothersome to keep around.

On 24 January 2017 at 10:46, Josh Smeaton  wrote:

> I remembered a thing from Michaels talk at #DUTH. Let me present a use
> case for subclassing a backend:
>
> https://github.com/opbeat/django-postgres-readonly/blob/
> master/django_postgres_readonly/base.py
>
> I think if we end up favouring immediate deprecation, we could proactively
> find and inform backend maintainers that would be affected. Their users
> might not be so appreciative though. I think Claude's position is a good
> one. A short deprecation period at a minimum.
>
> For my own interest I just did a quick search on PyPI for `django
> postgres` and found the following (having various levels of django version
> support):
>
> https://github.com/2degrees/django-postgres-delete-
> cascade/blob/master/django_postgres_delete_cascade/base.py
> https://github.com/kennethreitz/django-postgrespool/blob/master/
> django_postgrespool/base.py
> https://github.com/jdelic/django-postgresql-setrole/
> blob/master/postgresql_setrole/__init__.py (this one imports from the new
> location!)
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/django-developers/1755d125-ee5e-402f-bbd2-
> c6edb4c720c6%40googlegroups.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Adam

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAMyDDM24wfNDWkPzqFrxhJx28ZPq%2BLN0MxcMU4KXC4xmGLE2PQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Removing and renaming Django's Python 2 related helpers

[Aymeric Augustin]

Hello,

Django is earning a lot of goodwill from its well-defined deprecation policy. 
It was recently improved to allow pluggable apps to work without import shims 
and without deprecation warnings from one LTS to the next. I don’t know the 
exact details but I believe that's the intent.

Combined with a thorough documentation of backwards-incompatible changes, it 
helps many developers put up with changes, even those they don’t understand, 
disagree with, or require significant work to adapt applications.

For this reason, in the context of the transition from Python 2 to Python 3, 
which is very backwards incompatible for people who haven’t been able to 
anticipate it, I’m in favor of a generous application of the deprecation policy.

I’m +1 on deprecating rather than removing utilities that were mentioned in the 
documentation, notably python_2_unicode_compatible, which was a public API 
until it got merged into six and we started using six’ version.

I’m +0 on deprecating rather than removing modules that a developer of a 
pluggable app would reasonably use for Python 2 compatibility, such as 
django.utils.lru_cache or django.utils.six.

I’d just remove less visible functions such as django.utils._os.abspathu, 
provided they clearly fall in the “private API” bucket.

Best regards,

-- 
Aymeric.



> On 21 Jan 2017, at 21:55, Tim Graham  wrote:
> 
> As we worked on removing Python 2 compatibility code from master this week 
> [0], we collected a number of import shims and functions that are only needed 
> for code that wants to support Python 2 [1].
> 
> So far there is django.utils.six, as well as some undocumented things:
> django.utils.lru_cache
> django.utils._os.abspathu, upath, npath
> django.utils.decorators.available_attrs
> django.utils.encoding.python_2_unicode_compatible
> I'm advocating to remove the undocumented things in Django 3.0 (released Dec. 
> 2019) or later without a deprecation. By that time, I hope third-party apps 
> won't support Python 2 either and so part of adding Django 3.0 compatibility 
> will include formally dropping support for Python 2 (if not done already) and 
> removing usage of this stuff (a fairly easy find/replace, hopefully).
> 
> Others have advocated to deprecate all these things, but I don't see much 
> advantage. If I were maintaining an app, I'd rather be able to use import 
> shims without warnings until Python 2 is gone. What's your preference?
> 
> A similar concern applies to django.utils.http's urlquote() and 
> urlquote_plus() (documented) and urlunquote() and urlunquote_plus() 
> (undocumented). Claude has a PR that deprecates them [4] which I think is 
> fine considering they are half documented.
> 
> A related issue is the naming of the force_text() / force_str() and 
> smart_text() / smart_str() functions. Aymeric proposed a PR to change all 
> force_text() usage in Django to force_str() (they function identically on 
> Python 3) [2], however, I feel this might create some confusion. For example, 
> when backporting to 1.11 and earlier, we'll have to change force_str() to 
> force_text() for Python 2 compatibility. Third-party apps might also be 
> confused on which function to use. Claude proposed a similar that changes 
> ugettext() and ungettext() to gettext() / ngettext() [3]. Would it be less 
> confusing to consider these changes when Python 2 is no longer supported by 
> any version of Django?
> 
> [0] https://code.djangoproject.com/ticket/23919
> [1] https://code.djangoproject.com/ticket/27753
> [2] https://github.com/django/django/pull/7913
> [3] https://github.com/django/django/pull/7916
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to django-developers@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/django-developers 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/08121d30-345b-4fe3-964e-5f19578b4bc9%40googlegroups.com
>  
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://

Re: changing the default MySQL isolation level to READ COMMITTED

[Anssi Kääriäinen]

For reference there are some discussions about this 
in https://code.djangoproject.com/ticket/13906. The short version of the 
discussions is that MySQL is very interesting when it comes to transaction 
isolation levels and things like SELECT FOR UPDATE...

I think it would be ok to allow changing the isolation level, set the 
default to read committed, and document that if you use some other 
isolation level you are on your own. We could of course allow using 
different isolation levels on other databases, too. PostgreSQL's true 
serializable transactions for example are extremely nice for some 
application domains.

  -Anssi

On Friday, January 13, 2017 at 5:44:31 PM UTC+2, Tim Graham wrote:
>
> In the end, I don't use MySQL but if a similar change were made for 
> PostgreSQL, I would find the current approach more annoying (bothering me 
> about changing defaults over two release cycles) than cautious.
>
> I'm also uncertain that duplicating a deprecation warning in a system 
> check is a good precedent to set. Adam, is the current implementation what 
> you envisioned when you made your comment and what do you see as the 
> advantages of that?
>
> Aymeric, I assume "I’d love if this fix made it into 1.10" was a typo for 
> 1.11. Anyway, the "Allowed transaction isolation level to be chosen on 
> MySQL" commit isn't controversial. I'll try to get at least that in.
>
> On Friday, January 13, 2017 at 10:11:26 AM UTC-5, Adam Johnson wrote:
>>
>>  aside from some very performance-sensitive websites that already worked 
>>> around Django’s bugs
>>
>>
>> Tbf it's true I already added READ-COMMITTED at work
>>
>>  I’d love if this fix made it into 1.10
>>
>>
>>
>>  
>>
>> On 13 January 2017 at 15:05, Aymeric Augustin <
>> aymeric.au...@polytechnique.org> wrote:
>>
>>> Hello,
>>>
>>> I think there’s been little feedback because this issue is very, very 
>>> far from the concerns of most Django users — aside from some very 
>>> performance-sensitive websites that already worked around Django’s bugs and 
>>> will add the configuration line needed to keep whatever the isolation level 
>>> they chose.
>>>
>>> If I was trying to merge that change, I think I’d just document the 
>>> backwards incompatibility and call it a day. But as everyone knows, I’m 
>>> taking backwards compatibility less seriously than most.
>>>
>>> Given the uncertainty around the consequences of this change, I don’t 
>>> think Shai’s approach is out of place, even though it creates an overhead 
>>> for every Django project using MySQL.
>>>
>>> Since Shai is leading the effort and considering the general lack of 
>>> feedback on this issue, I think it’s fair to let him make the final call 
>>> and keep the deprecation path if he thinks that’s safer.
>>>
>>> Regardless, I’d love if this fix made it into 1.10, let’s not delay it 
>>> because we’re worried of being too cautious :-)
>>>
>>> -- 
>>> Aymeric.
>>>
>>>
>>> On 13 Jan 2017, at 14:52, Tim Graham  wrote:
>>>
>>> I guess three days is too little time to get a consensus on this. At 
>>> this point I'm thinking to defer this from 1.11.
>>>
>>> On Wednesday, January 11, 2017 at 9:50:27 AM UTC-5, Patryk Zawadzki 
>>> wrote:

 To add some context, REPEATABLE READ can break get_or_create among 
 other things (not sure how many invalid tickets Django gets related to 
 that).

>>>
>>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "Django developers (Contributions to Django itself)" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to django-develop...@googlegroups.com.
>>> To post to this group, send email to django-d...@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/django-developers.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/django-developers/42212944-d7e7-4c33-8c34-8e235a7d515a%40googlegroups.com
>>>  
>>> 
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>>
>>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "Django developers (Contributions to Django itself)" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to django-develop...@googlegroups.com.
>>> To post to this group, send email to django-d...@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/django-developers.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/django-developers/2D7B4E2A-30EF-4201-9272-C5430825E123%40polytechnique.org
>>>  
>>> 
>>> .
>>>
>>> For more options, visit https://groups.google.com/d/optout

Re: Marking flushed sessions as not modified

[Tim Graham]

The change you proposed might be acceptable, although I'm a bit nervous 
about breaking backwards-compatibility in some unforeseen use case. I 
wonder if modifying the condition in the
channel_session decorator to: if session.modified and not 
session.is_empty(): would solve the issue? This seems similar to what the 
SessionMiddleware does.

On Monday, January 23, 2017 at 2:59:59 PM UTC-5, Andrew Godwin wrote:
>
> Here's the Channels issue underlying it: 
> https://github.com/django/channels/issues/477
>
> For quick reference, the @channel_session decorator is something that 
> wraps a consumer, loading a session on the way in and saving it on the way 
> out if it changed, similar to how session middleware might work.
>
> Andrew
>
> On Mon, Jan 23, 2017 at 11:57 AM, Tim Graham  > wrote:
>
>> I'm not sure offhand. Could you provide a link to the issue so we could 
>> look at the details?
>>
>>
>> On Monday, January 23, 2017 at 1:45:48 PM UTC-5, Andrew Godwin wrote:
>>>
>>> An interesting bug has turned up in Channels where someone is trying to 
>>> remove a session using .flush(), but our decorator sees that it's been 
>>> modified and then re-saves it, thinking it needs to persist data.
>>>
>>> What are people's opinions on changing flush() to also set 
>>> modified=False on sessions? Is there a bad side-effect of this I haven't 
>>> considered?
>>>
>>> Andrew
>>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-develop...@googlegroups.com .
>> To post to this group, send email to django-d...@googlegroups.com 
>> .
>> Visit this group at https://groups.google.com/group/django-developers.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/2ae6fbc5-171f-4647-9604-29fc6833ee0a%40googlegroups.com
>>  
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/41a2b00e-a40a-4b45-9936-741cc58d2aed%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Marking flushed sessions as not modified

[Andrew Godwin]

Yes, that's probably the best short-term fix and will keep it working with
older versions - I'll put that in as a fix for this now.

I agree it's potentially dangerous, but then I also think the current
behaviour is a bit misleading. I'll dig around some third party apps and
see if I can see examples of how people call it.

Andrew

On Tue, Jan 24, 2017 at 6:50 AM, Tim Graham  wrote:

> The change you proposed might be acceptable, although I'm a bit nervous
> about breaking backwards-compatibility in some unforeseen use case. I
> wonder if modifying the condition in the
> channel_session decorator to: if session.modified and not
> session.is_empty(): would solve the issue? This seems similar to what the
> SessionMiddleware does.
>
> On Monday, January 23, 2017 at 2:59:59 PM UTC-5, Andrew Godwin wrote:
>>
>> Here's the Channels issue underlying it: https://github.com/django/
>> channels/issues/477
>>
>> For quick reference, the @channel_session decorator is something that
>> wraps a consumer, loading a session on the way in and saving it on the way
>> out if it changed, similar to how session middleware might work.
>>
>> Andrew
>>
>> On Mon, Jan 23, 2017 at 11:57 AM, Tim Graham  wrote:
>>
>>> I'm not sure offhand. Could you provide a link to the issue so we could
>>> look at the details?
>>>
>>>
>>> On Monday, January 23, 2017 at 1:45:48 PM UTC-5, Andrew Godwin wrote:

 An interesting bug has turned up in Channels where someone is trying to
 remove a session using .flush(), but our decorator sees that it's been
 modified and then re-saves it, thinking it needs to persist data.

 What are people's opinions on changing flush() to also set
 modified=False on sessions? Is there a bad side-effect of this I haven't
 considered?

 Andrew

>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Django developers (Contributions to Django itself)" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to django-develop...@googlegroups.com.
>>> To post to this group, send email to django-d...@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/django-developers.
>>> To view this discussion on the web visit https://groups.google.com/d/ms
>>> gid/django-developers/2ae6fbc5-171f-4647-9604-29fc6833ee0a%
>>> 40googlegroups.com
>>> 
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/django-developers/41a2b00e-a40a-4b45-9936-
> 741cc58d2aed%40googlegroups.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFwN1upG9J5y2UQT-mGJebGxesUjPA0MEn%3D4hSncAu5wEQ9rVw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Renaming the postgresql_psycopg2 backend

[Tim Graham]

Okay, I updated the PR to use a deprecation. I'd rather not complicate 
things with an accelerated deprecation.

On Tuesday, January 24, 2017 at 6:39:32 AM UTC-5, Adam Johnson wrote:
>
> Ok fair point, I agree it should go through the short deprecation process. 
> It's also very small and not that bothersome to keep around.
>
> On 24 January 2017 at 10:46, Josh Smeaton  > wrote:
>
>> I remembered a thing from Michaels talk at #DUTH. Let me present a use 
>> case for subclassing a backend:
>>
>>
>> https://github.com/opbeat/django-postgres-readonly/blob/master/django_postgres_readonly/base.py
>>
>> I think if we end up favouring immediate deprecation, we could 
>> proactively find and inform backend maintainers that would be affected. 
>> Their users might not be so appreciative though. I think Claude's position 
>> is a good one. A short deprecation period at a minimum.
>>
>> For my own interest I just did a quick search on PyPI for `django 
>> postgres` and found the following (having various levels of django version 
>> support):
>>
>>
>> https://github.com/2degrees/django-postgres-delete-cascade/blob/master/django_postgres_delete_cascade/base.py
>>
>> https://github.com/kennethreitz/django-postgrespool/blob/master/django_postgrespool/base.py
>>
>> https://github.com/jdelic/django-postgresql-setrole/blob/master/postgresql_setrole/__init__.py
>>  
>> (this one imports from the new location!)
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-develop...@googlegroups.com .
>> To post to this group, send email to django-d...@googlegroups.com 
>> .
>> Visit this group at https://groups.google.com/group/django-developers.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/1755d125-ee5e-402f-bbd2-c6edb4c720c6%40googlegroups.com
>>  
>> 
>> .
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> -- 
> Adam
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/aeefd658-7abc-42dd-bbb0-97c64c818037%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Removing and renaming Django's Python 2 related helpers

[Sam Willis]

Hi,

An alternative option with 'six' is to replace it with an alias of six 
propper (not vendored), something like this:

# django/utils/six.py
try:
from six import *
except ImportError:
raise ImportError((
"`django.utils.six` is deprecated, install six from pypi "
"(https://pypi.python.org/pypi/six) instead. After installing 
it will be available at "
"`django.utils.six` until Django V2.xx."))
except:
import warnings
warnings.warn((
"The `django.utils.six` alias is deprecated and will be removed 
in Django V2.xx.",
DeprecationWarning, 2
)

That way it removes the need for it to be maintained in core but ensures 
that other Django apps that are supporting py2 and 3 don't break.

Sam

On Tuesday, January 24, 2017 at 11:53:01 AM UTC, Aymeric Augustin wrote:
>
> Hello,
>
> Django is earning a lot of goodwill from its well-defined deprecation 
> policy. It was recently improved to allow pluggable apps to work without 
> import shims and without deprecation warnings from one LTS to the next. I 
> don’t know the exact details but I believe that's the intent.
>
> Combined with a thorough documentation of backwards-incompatible changes, 
> it helps many developers put up with changes, even those they don’t 
> understand, disagree with, or require significant work to adapt 
> applications.
>
> For this reason, in the context of the transition from Python 2 to Python 
> 3, which is very backwards incompatible for people who haven’t been able to 
> anticipate it, I’m in favor of a generous application of the deprecation 
> policy.
>
> I’m +1 on deprecating rather than removing utilities that were mentioned 
> in the documentation, notably python_2_unicode_compatible, which was a 
> public API until it got merged into six and we started using six’ version.
>
> I’m +0 on deprecating rather than removing modules that a developer of a 
> pluggable app would reasonably use for Python 2 compatibility, such 
> as django.utils.lru_cache or django.utils.six.
>
> I’d just remove less visible functions such as django.utils._os.abspathu, 
> provided they clearly fall in the “private API” bucket.
>
> Best regards,
>
> -- 
> Aymeric.
>
>
>
> On 21 Jan 2017, at 21:55, Tim Graham > 
> wrote:
>
> As we worked on removing Python 2 compatibility code from master this week 
> [0], we collected a number of import shims and functions that are only 
> needed for code that wants to support Python 2 [1].
>
> So far there is django.utils.six, as well as some undocumented things:
>
>- django.utils.lru_cache 
>- django.utils._os.abspathu, upath, npath 
>- django.utils.decorators.available_attrs
>- django.utils.encoding.python_2_unicode_compatible
>
> I'm advocating to remove the undocumented things in Django 3.0 (released 
> Dec. 2019) or later without a deprecation. By that time, I hope third-party 
> apps won't support Python 2 either and so part of adding Django 3.0 
> compatibility will include formally dropping support for Python 2 (if not 
> done already) and removing usage of this stuff (a fairly easy find/replace, 
> hopefully).
>
> Others have advocated to deprecate all these things, but I don't see much 
> advantage. If I were maintaining an app, I'd rather be able to use import 
> shims without warnings until Python 2 is gone. What's your preference?
>
> A similar concern applies to django.utils.http's urlquote() and 
> urlquote_plus() (documented) and urlunquote() and urlunquote_plus() 
> (undocumented). Claude has a PR that deprecates them [4] which I think is 
> fine considering they are half documented.
>
> A related issue is the naming of the force_text() / force_str() and 
> smart_text() / smart_str() functions. Aymeric proposed a PR to change all 
> force_text() 
> usage in Django to force_str() (they function identically on Python 3) [2], 
> however, I feel this might create some confusion. For example, when 
> backporting to 1.11 and earlier, we'll have to change force_str() to 
> force_text() for Python 2 compatibility. Third-party apps might also be 
> confused on which function to use. Claude proposed a similar that changes 
> ugettext() 
> and ungettext() to gettext() / ngettext() [3]. Would it be less confusing 
> to consider these changes when Python 2 is no longer supported by any 
> version of Django?
>
> [0] https://code.djangoproject.com/ticket/23919
> [1] https://code.djangoproject.com/ticket/27753
> [2] https://github.com/django/django/pull/7913
> [3] https://github.com/django/django/pull/7916
>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-develop...@googlegroups.com .
> To post to this group, send email to django-d...@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/django-developers.
> To v

Re: Methodology for increasing the number of PBKDF2 iterations

[Martin Koistinen]

Updated the sheet with more recent GPU pricing.

On Thursday, January 19, 2017 at 1:19:57 PM UTC-5, Martin Koistinen wrote:
>
> All, I've converted my worksheet into a Google Docs Sheet here: 
> https://docs.google.com/spreadsheets/d/16_KdYAW03sb86-w_AFFnM79IaTWQ7Ugx4T0VMfGteTM/edit?usp=sharing
>
> Note that it isn't really editable here. You'll need to make a copy into 
> your own account or download into a local spreadsheet to tweak for your 
> system and security policy.
>
> Comments and suggestions are welcome and if appropriate, I'll make edits 
> accordingly.
>
>
> On Wednesday, January 18, 2017 at 12:32:55 PM UTC-5, Martin Koistinen 
> wrote:
>>
>> Tim, I've sent you a model I've assembled recently for your review. I'll 
>> work towards making it more user-friendly (I.e., NOT in Apple Numbers 
>> format) and share it here for the whole community.
>>
>> But for here and now, I would at the very least assume that the cost of 
>> a brute-force attack on password hashes falls over time inversely 
>> proportional to Moore's Law. Its a naive approach, but is a fairly 
>> reasonable one. So, to compensate, we should be doubling the number of 
>> iterations every 18-24 months, or perhaps at a minimum, raise them by 
>> sqrt(2) annually ~= +40% each year.
>>
>> +40%/year assumes that the starting point of 100,000 is OK for Q1 2017 
>> (this will not be true for every project) and it assumes that Moore's Law 
>> is evenly "applied" over time (its not).
>>
>>
>> On Wednesday, January 18, 2017 at 10:25:46 AM UTC-5, Tim Graham wrote:
>>>
>>> I increased the iterations to 100,000 on master (targeting Django 2.0). 
>>> It would be nice to determine a guideline for how to determine future 
>>> increases.
>>>
>>> On Monday, January 16, 2017 at 12:55:25 PM UTC-5, Martin Koistinen wrote:

 Tobias,

 Thanks for the comprehensive benchmarking and summary of the situation! 
 I agree on all points, but I'd like to add, that we should err on the side 
 of high iterations for the simple fact that most developers would sooner 
 accept the risk of a DoS long before the risk of compromised user accounts.

 Also, if a developer is experienced/motivated enough to *lower* the 
 hash iterations, s/he'll be more likely to also be experienced/motivated 
 enough to put other controls in place to compensate.


 Best,
 - Martin

 On Sunday, January 15, 2017 at 5:45:02 PM UTC-5, Tobias McNulty wrote:
>
> I'm not sure the DoS concern is really something that can be addressed 
> here. Regardless of the number of iterations we choose, POSTing to the 
> login form will always be a target, unless it's appropriately protected 
> (i.e., with some combination of rate limiting, recaptcha, and/or 
> something 
> at the network level). A run-of-the-mill cloud server that doesn't limit 
> access to the Python app in some way is simply never going to be a match 
> for a malicious person with a laptop, let alone a more sophisticated 
> attack.
>
> I created a tox.ini 
> 
>  to 
> run Martin's benchmark with multiple Django & Python versions. A couple 
> notes:
>
>- I ran this several times on Circle CI using Ubuntu 12.04 
> 
>with Python 2.7.7, 3.3.3, 3.4.3, and 3.5.0, and Ubuntu 14.04 
> 
>with 2.7.12, 3.3.6, 3.4.4, and 3.5.2. To view the results, expand the 
> "tox" 
>section under the "Test" header.
>- All results are what one would expect: Python 2.7.7 and Python 
>3.3.x are ~3-4x slower than Python 2.7.8+ and Python 3.4+, and there 
> are no 
>inexplicably slow outliers, like the official Python 3.5.2 installer 
> for OS 
>X.
>
> My local results are as follows:
>
>- Ubuntu 16.04 w/a Core i5 @ 3.50GHz:
>   - 62-65ms for 100,000 iterations
>   - 100-106ms for 165,000 iterations
>- Mac OS 10.12, Core i5 @ 2.7GHz:
>   - 117-120ms for 100,000 iterations
>   - 195-203ms for 165,000 iterations
>
> I really don't know how we can pick a number that'll work for 
> everyone, but I'm all for setting it high and allowing people to decrease 
> the number of iterations or, better yet, switch to the hasher that the 
> docs recommend everyone use anyway 
> 
>  
> (Argon2). If we define 100-120ms as acceptable performance, 100k would 
> seem 
> reasonable based on the results above and posted elsewhere in this thread.
>
> Martin, FWIW, I can confirm that the Python 3.5.2 installer from 
> python.org demonstrates the same