Simplify authentication backend interface

2018-11-10 Thread Tobias Bengfort
I feel like the interface for authentication backends is unnecessarily
complex: Basically, you only need authenticate() and has_perm(), but
currently the interface also includes get_group_permissions(),
get_all_permissions(), and has_module_perms().

The architecture is like this: User inherits from PermissionMixin which
implements the methods get_group_permissions(), get_all_permissions(),
has_perm(), has_perms(), and has_module_perms().

All of these methods basically just call the corresponding methods on
the authentication backends, if available. has_perms() is the only
exception, as it is just a shortcut to call has_perm() multiple times.

I believe that has_perm() is vastly more important than
get_*_permissions(). Still, I can understand that the latter exist. What
I find confusing is that there is no get_user_permissions() in this
interface. Whithout its counterpart, get_group_permissions() seems
pretty much useless to me. Also note that the separation between user
and group permissions may not be applicable with custom backends.

Another issue is that developers can easily end up with inconsistent
backends: There is no guarantee that get_all_permissions() will return a
superset of get_group_permissions(). There is also no guarantee that
has_perm() will be equivalent to `perm in get_all_permissions()`.

Then there is the issue of has_module_perms(). As far as I understand,
this should have been called has_app_perms(). It is used in
contrib.admin where it makes a lot of sense, but I don't really see how
it could be useful anywhere else.

I looked at some popular backends to see how they are implemented:

The default ModelBackend implements get_user_permissions() and
get_group_permissions(). get_all_permissions() joins their results and
has_perm() and has_module_perms() call get_all_permissions(). The
results of get_all_permissions() are cached.

django-guardian works pretty much the same. However, even though the
mechanism is the same, get_group_permissions() is not exposed in the
interface. has_module_perms() is missing.

django-rules only implements has_perm(). get_*_permissions() are
missing. has_module_perms() exists, but it is just an alias to
has_perm().

I guess the benefits of simplifying the interface do not justify
breaking changes for most of these issues. However, I think there are
two changes that could significantly improve the situation:

-   Either add get_user_permissions() or remove get_group_permissions()
-   Add default implementations for get_all_permissions() and
has_perm(), either in PermissionMixin or in a new BaseBackend class.

Any thoughts?

tobias

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/9eb25389-d59e-1f6c-4b50-d1c7a986f923%40posteo.de.
For more options, visit https://groups.google.com/d/optout.


Re: Allow skipping CSRF check for Referer header

2018-11-10 Thread Florian Apolloner
Wouldn't one alternative be checking the Origin header? It appears though 
that all browsers support it with the sad exception that it is still behind 
a feature flag in Firefox. :/ 
(https://bugzilla.mozilla.org/show_bug.cgi?id=1424076)

On Saturday, November 10, 2018 at 1:03:08 AM UTC+1, Adam Johnson wrote:
>
> I also discovered a similar problem recently when deploying the 
> "Referrer-Policy" header using James Bennett's library: 
> https://django-referrer-policy.readthedocs.io/en/stable/ . Initially I 
> opted for 'no-referrer' as I figured it was the most secure, but since this 
> check is only done on HTTPS-enabled sites, it wasn't discovered that all 
> forms were broken until production where HTTPS is used.
>
> I think the existence of Referrer-Policy bolsters the argument for this 
> option, since some users might want to use 'no-referrer'. And in fact the 
> current check presumes the Referrer Policy is left at the default, 
> no-referrer-when-downgrade , and I'm pretty sure other values than 
> no-referrer break it as well.
>
> +1 from me (I wonder if it could even be automatic if HSTS is enabled?)
>
> On Fri, 9 Nov 2018 at 21:31, Aaron Hill > 
> wrote:
>
>> Currently, Django's CSRF middleware will reject any 'non-safe' HTTPS 
>> request that lacks a Referer header: ​
>> https://github.com/django/django/blob/22e8ab02863819093832de9f771bf40a62a6bd4a/django/middleware/
>> csrf.py#L242
>>
>> However, some users may prevent their browsers from sending the Referer 
>> header, due to privacy concerns. These users are unable to submit 
>> 'non-safe' requests (e.g. POST requests) on HTTPS-enabled Django-powered 
>> website that use CSRF protection.
>>
>> For some websites, checking the Referer header may provide no added 
>> security benefit. For example, an HSTS-preloaded website which controls all 
>> of its subdomains has nothing to gain from this check - there are no 
>> untrusted subdomains which can mount an attack, and HSTS prevents an HTTP 
>> MITM attack.
>>
>>
>> To allow these websites to provide more flexibility to their users, 
>> Django should support disabling this CSRF Referer check. This could be 
>> done through a new setting, e.g. ' CSRF_REFERER_CHECK' (defaulting to 
>> 'True' to avoid breaking existing sites).
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-develop...@googlegroups.com .
>> To post to this group, send email to django-d...@googlegroups.com 
>> .
>> Visit this group at https://groups.google.com/group/django-developers.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/bcc04352-cd39-485a-83ad-49d0608d6ccd%40googlegroups.com
>>  
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
> -- 
> Adam
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/5c89439f-1f3c-44ff-99a6-708ae5c42684%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Pluggable secret key backend

2018-11-10 Thread Andreas Pelme
Hi,

settings.SECRET_KEY can be used for sessions, password resets, form wizards and
other cryptographic signatures via the signing APIs. Changing SECRET_KEY means
that all of those will be invalidated and the users will be affected in weird
ways without really knowing what happened. (Why am I logged out? Where did my
form submission go? Why does not this password reset link work?). This is
desirable in case the key is compromised and swift action must be taken.

There are other situations when it would be nice to change the SECRET_KEY when
this sudden invalidation is not desirable:

- When someone leaves a project/company that had access to the production
  system. After SSH keys/login credentials is revoked the developer could
  potentially have a copy of the secret key. It is essentially a backdoor with
  full remote access. It would be wise to rotate the key in those cases.

- Periodic and automatic rotations of keys to make it less useful in the
  future.

The current situation of a single SECRET_KEY makes key rotation impractical. If
you run a busy site with active users 24/7, there is never a nice time to
change the SECRET_KEY.

A solution for this problem would be sign new secrets with a new key while
still allow signatures made with the old key to be considered valid at the same
time. Changing keys and having a couple of hours of overlap where signatures
from both keys are accepted would mitigate most of the user facing problems
with invalidating sessions, password reset links and form wizard progress.

You could do this today by implementing your own session backend, message
storage backend and password reset token generator but that is cumbersome and
does not work across reusable apps that directly use low level Django signing
APIs unless they too provide hooks to provide your own secret.

I propose a pluggable project wide secret key backend
(settings.SECRET_KEY_BACKEND maybe?) with an API something like:

class SecretKeyBackend:
  def get_signing_key(self): …
  def get_verification_keys(self): ...

The default (and backward compatible) backend would then be implemented as
something like:

class SecretKeySettingsBackend:
  def get_signing_key(self):
return settings.SECRET_KEY
  def get_verification_keys(self):
return [settings.SECRET_KEY]

django.core.signing.Signer.{sign,unsign} would need to be updated to use this
backend instead of directly using settings.SECRET_KEY.

That would solve the problem project wide and work across any third party
application that uses django.core.signing directly.

This would open the door for third party secrets backend packages that
retrieves keys from systems such as Hashicorp Vault, AWS Secrets Manager,
Google Cloud KMS, Docker Secrets etc.

Having a method that retrieves the key would allow changes to secret key during
run time instead of relying on a hard coded setting would allow the key to
change without restarting the server process.

Would something like this be worth pursuing? Could it be designed in som other
way? I could not find any previous discussion/tickets on this and thought it
would be a good idea to discuss it here before opening a ticket or making an
attempt at a PR. :)

Cheers,

Andreas


-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/20D8A2BD-BC9C-4F02-9038-044687165DE9%40pelme.se.
For more options, visit https://groups.google.com/d/optout.


Re: Pluggable secret key backend

2018-11-10 Thread ludovic coues
I don't see how this would work.

For example the session. You take the user cookie. You try to validate with
your secret key. That doesn't work because the current key is the new one.

With a custom cookie backend, you could check if the old secret could
validate the cookie. But you need to change your cookie backend to handle
the case of multiple secret key. And all third party session backend need
to update.


On Sat, Nov 10, 2018, 11:12 Andreas Pelme  Hi,
>
> settings.SECRET_KEY can be used for sessions, password resets, form
> wizards and
> other cryptographic signatures via the signing APIs. Changing SECRET_KEY
> means
> that all of those will be invalidated and the users will be affected in
> weird
> ways without really knowing what happened. (Why am I logged out? Where did
> my
> form submission go? Why does not this password reset link work?). This is
> desirable in case the key is compromised and swift action must be taken.
>
> There are other situations when it would be nice to change the SECRET_KEY
> when
> this sudden invalidation is not desirable:
>
> - When someone leaves a project/company that had access to the production
>   system. After SSH keys/login credentials is revoked the developer could
>   potentially have a copy of the secret key. It is essentially a backdoor
> with
>   full remote access. It would be wise to rotate the key in those cases.
>
> - Periodic and automatic rotations of keys to make it less useful in the
>   future.
>
> The current situation of a single SECRET_KEY makes key rotation
> impractical. If
> you run a busy site with active users 24/7, there is never a nice time to
> change the SECRET_KEY.
>
> A solution for this problem would be sign new secrets with a new key while
> still allow signatures made with the old key to be considered valid at the
> same
> time. Changing keys and having a couple of hours of overlap where
> signatures
> from both keys are accepted would mitigate most of the user facing problems
> with invalidating sessions, password reset links and form wizard progress.
>
> You could do this today by implementing your own session backend, message
> storage backend and password reset token generator but that is cumbersome
> and
> does not work across reusable apps that directly use low level Django
> signing
> APIs unless they too provide hooks to provide your own secret.
>
> I propose a pluggable project wide secret key backend
> (settings.SECRET_KEY_BACKEND maybe?) with an API something like:
>
> class SecretKeyBackend:
>   def get_signing_key(self): …
>   def get_verification_keys(self): ...
>
> The default (and backward compatible) backend would then be implemented as
> something like:
>
> class SecretKeySettingsBackend:
>   def get_signing_key(self):
> return settings.SECRET_KEY
>   def get_verification_keys(self):
> return [settings.SECRET_KEY]
>
> django.core.signing.Signer.{sign,unsign} would need to be updated to use
> this
> backend instead of directly using settings.SECRET_KEY.
>
> That would solve the problem project wide and work across any third party
> application that uses django.core.signing directly.
>
> This would open the door for third party secrets backend packages that
> retrieves keys from systems such as Hashicorp Vault, AWS Secrets Manager,
> Google Cloud KMS, Docker Secrets etc.
>
> Having a method that retrieves the key would allow changes to secret key
> during
> run time instead of relying on a hard coded setting would allow the key to
> change without restarting the server process.
>
> Would something like this be worth pursuing? Could it be designed in som
> other
> way? I could not find any previous discussion/tickets on this and thought
> it
> would be a good idea to discuss it here before opening a ticket or making
> an
> attempt at a PR. :)
>
> Cheers,
>
> Andreas
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers  (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/20D8A2BD-BC9C-4F02-9038-044687165DE9%40pelme.se
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAEuG%2B

Re: Pluggable secret key backend

2018-11-10 Thread Adam Johnson
Hi Andreas

I like your proposal, moving to a backend is an elegant way of solving both
the immediate problem and opening up the other possibilities you mentioned.

I think it would also be nice to have an "out of the box" way of rotating
the key, without needing to implement a custom backend. Perhaps a second
setting OLD_SECRET_KEYS that may contain a list of old keys that are
returned for verification too? Or we could allow SECRET_KEY to be a
list/tuple, and if so, sign with the first and verify with all of them.

Adam

On Sat, 10 Nov 2018 at 10:12, Andreas Pelme  wrote:

> Hi,
>
> settings.SECRET_KEY can be used for sessions, password resets, form
> wizards and
> other cryptographic signatures via the signing APIs. Changing SECRET_KEY
> means
> that all of those will be invalidated and the users will be affected in
> weird
> ways without really knowing what happened. (Why am I logged out? Where did
> my
> form submission go? Why does not this password reset link work?). This is
> desirable in case the key is compromised and swift action must be taken.
>
> There are other situations when it would be nice to change the SECRET_KEY
> when
> this sudden invalidation is not desirable:
>
> - When someone leaves a project/company that had access to the production
>   system. After SSH keys/login credentials is revoked the developer could
>   potentially have a copy of the secret key. It is essentially a backdoor
> with
>   full remote access. It would be wise to rotate the key in those cases.
>
> - Periodic and automatic rotations of keys to make it less useful in the
>   future.
>
> The current situation of a single SECRET_KEY makes key rotation
> impractical. If
> you run a busy site with active users 24/7, there is never a nice time to
> change the SECRET_KEY.
>
> A solution for this problem would be sign new secrets with a new key while
> still allow signatures made with the old key to be considered valid at the
> same
> time. Changing keys and having a couple of hours of overlap where
> signatures
> from both keys are accepted would mitigate most of the user facing problems
> with invalidating sessions, password reset links and form wizard progress.
>
> You could do this today by implementing your own session backend, message
> storage backend and password reset token generator but that is cumbersome
> and
> does not work across reusable apps that directly use low level Django
> signing
> APIs unless they too provide hooks to provide your own secret.
>
> I propose a pluggable project wide secret key backend
> (settings.SECRET_KEY_BACKEND maybe?) with an API something like:
>
> class SecretKeyBackend:
>   def get_signing_key(self): …
>   def get_verification_keys(self): ...
>
> The default (and backward compatible) backend would then be implemented as
> something like:
>
> class SecretKeySettingsBackend:
>   def get_signing_key(self):
> return settings.SECRET_KEY
>   def get_verification_keys(self):
> return [settings.SECRET_KEY]
>
> django.core.signing.Signer.{sign,unsign} would need to be updated to use
> this
> backend instead of directly using settings.SECRET_KEY.
>
> That would solve the problem project wide and work across any third party
> application that uses django.core.signing directly.
>
> This would open the door for third party secrets backend packages that
> retrieves keys from systems such as Hashicorp Vault, AWS Secrets Manager,
> Google Cloud KMS, Docker Secrets etc.
>
> Having a method that retrieves the key would allow changes to secret key
> during
> run time instead of relying on a hard coded setting would allow the key to
> change without restarting the server process.
>
> Would something like this be worth pursuing? Could it be designed in som
> other
> way? I could not find any previous discussion/tickets on this and thought
> it
> would be a good idea to discuss it here before opening a ticket or making
> an
> attempt at a PR. :)
>
> Cheers,
>
> Andreas
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers  (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/20D8A2BD-BC9C-4F02-9038-044687165DE9%40pelme.se
> .
> For more options, visit https://groups.google.com/d/optout.
>


-- 
Adam

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.goog

Re: Allow skipping CSRF check for Referer header

2018-11-10 Thread Adam Johnson
I would think that feature flag rules it out for a long time?

On Sat, 10 Nov 2018 at 09:52, Florian Apolloner 
wrote:

> Wouldn't one alternative be checking the Origin header? It appears though
> that all browsers support it with the sad exception that it is still behind
> a feature flag in Firefox. :/ (
> https://bugzilla.mozilla.org/show_bug.cgi?id=1424076)
>
> On Saturday, November 10, 2018 at 1:03:08 AM UTC+1, Adam Johnson wrote:
>>
>> I also discovered a similar problem recently when deploying the
>> "Referrer-Policy" header using James Bennett's library:
>> https://django-referrer-policy.readthedocs.io/en/stable/ . Initially I
>> opted for 'no-referrer' as I figured it was the most secure, but since this
>> check is only done on HTTPS-enabled sites, it wasn't discovered that all
>> forms were broken until production where HTTPS is used.
>>
>> I think the existence of Referrer-Policy bolsters the argument for this
>> option, since some users might want to use 'no-referrer'. And in fact the
>> current check presumes the Referrer Policy is left at the default,
>> no-referrer-when-downgrade , and I'm pretty sure other values than
>> no-referrer break it as well.
>>
>> +1 from me (I wonder if it could even be automatic if HSTS is enabled?)
>>
>> On Fri, 9 Nov 2018 at 21:31, Aaron Hill  wrote:
>>
>>> Currently, Django's CSRF middleware will reject any 'non-safe' HTTPS
>>> request that lacks a Referer header: ​
>>> https://github.com/django/django/blob/22e8ab02863819093832de9f771bf40a62a6bd4a/django/middleware/
>>> csrf.py#L242
>>>
>>> However, some users may prevent their browsers from sending the Referer
>>> header, due to privacy concerns. These users are unable to submit
>>> 'non-safe' requests (e.g. POST requests) on HTTPS-enabled Django-powered
>>> website that use CSRF protection.
>>>
>>> For some websites, checking the Referer header may provide no added
>>> security benefit. For example, an HSTS-preloaded website which controls all
>>> of its subdomains has nothing to gain from this check - there are no
>>> untrusted subdomains which can mount an attack, and HSTS prevents an HTTP
>>> MITM attack.
>>>
>>>
>>> To allow these websites to provide more flexibility to their users,
>>> Django should support disabling this CSRF Referer check. This could be
>>> done through a new setting, e.g. ' CSRF_REFERER_CHECK' (defaulting to
>>> 'True' to avoid breaking existing sites).
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Django developers (Contributions to Django itself)" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to django-develop...@googlegroups.com.
>>> To post to this group, send email to django-d...@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/django-developers.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/django-developers/bcc04352-cd39-485a-83ad-49d0608d6ccd%40googlegroups.com
>>> 
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>
>> --
>> Adam
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/5c89439f-1f3c-44ff-99a6-708ae5c42684%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>


-- 
Adam

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAMyDDM3UcU38Xn%3D9PMhUMTKTfgGPY%3D77UiY8XG05S9CP9-%2BQDA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


Re: skipping elidable migrations

2018-11-10 Thread Adam Johnson
Do you have an example? If you're using Django's default testing framework,
it normally creates a fresh database, so from the moment a new data
migration is written it would be tested with the empty database scenario.

Afaiu it is possible to write RunPython operations in a way that no-ops on
empty databases, e.g. starting with the equivalent of "if not
Model.object.exists(): return." It is also possible with RunSQL on database
backends that support conditional logic around SQL statements,  or they can
be easily converted to RunPython using a function that uses
cursor.execute(). And if a project develops problems with past migrations,
it's always possible to (carefully) edit them to include such conditions.

On Fri, 9 Nov 2018 at 16:11, Dan Watson  wrote:

> Hi All,
>
> I was wondering if anyone had any thoughts on an option to the "migrate"
> command (--skip-elidable?) that would skip running elidable migrations. The
> use case here is that data migrations that build up over time may act on
> certain assumptions (existing tables/data) that may not be true when
> migrating a new database. It seems that since they were explicitly marked
> as able to be deleted when squashing, they would be safe to not run when
> creating a new database. Maybe we don't go so far as to make this the
> default behavior when migrating a fresh database, but an option would be
> nice. I realize you could simply squash your migrations, but that's not
> without penalty of code churn, testing, etc. especially when your existing
> migration graph is otherwise fine (and performant).
>
> If there's some consensus about this being worthwhile, or at least no
> strong objections to it, I can take a stab at the implementation.
>
> Regards,
> Dan
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/fba7a58a-444b-4c90-b139-151580423366%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>


-- 
Adam

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAMyDDM0JUEWSBJVgEnhrdXttitV%2BAqAkWoMzeSB%2BHh7mDsPbWA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


Re: Pluggable secret key backend

2018-11-10 Thread Andreas Pelme



> On 10 Nov 2018, at 13:00, ludovic coues  wrote:
> 
> I don't see how this would work.
> 
> For example the session. You take the user cookie. You try to validate with 
> your secret key. That doesn't work because the current key is the new one.
> 
> With a custom cookie backend, you could check if the old secret could 
> validate the cookie. But you need to change your cookie backend to handle the 
> case of multiple secret key. And all third party session backend need to 
> update.


I propose that we make the low level django.core.signing aware of multiple 
keys. Everything that is already using django.core.signing such as signed 
cookies, sessions and password reset tokens would need *not* need to change.

Cheers,
Andreas

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/5ED0D5A2-AC77-4231-824C-2EDDD7F2A903%40pelme.se.
For more options, visit https://groups.google.com/d/optout.


Re: Pluggable secret key backend

2018-11-10 Thread Andreas Pelme
On 10 Nov 2018, at 13:29, Adam Johnson  wrote:
> 
> Hi Andreas
> 
> I like your proposal, moving to a backend is an elegant way of solving both 
> the immediate problem and opening up the other possibilities you mentioned.

Thanks Adam, I am glad you like the proposal. :)

> I think it would also be nice to have an "out of the box" way of rotating the 
> key, without needing to implement a custom backend. Perhaps a second setting 
> OLD_SECRET_KEYS that may contain a list of old keys that are returned for 
> verification too? Or we could allow SECRET_KEY to be a list/tuple, and if so, 
> sign with the first and verify with all of them.

Agreed, I will add something like that then! :)

Cheers,
Andreas

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/A16A11DF-1439-46EF-BF0D-85C483F53608%40pelme.se.
For more options, visit https://groups.google.com/d/optout.


Re: Allow skipping CSRF check for Referer header

2018-11-10 Thread Florian Apolloner
Not neccessarily, one could still use the Origin header in cases where 
software strips the Referer and if you set the Referrer-Policy to 
same-origin you shouldn't have problems with firefox either.

On Saturday, November 10, 2018 at 1:42:41 PM UTC+1, Adam Johnson wrote:
>
> I would think that feature flag rules it out for a long time?
>
> On Sat, 10 Nov 2018 at 09:52, Florian Apolloner  > wrote:
>
>> Wouldn't one alternative be checking the Origin header? It appears though 
>> that all browsers support it with the sad exception that it is still behind 
>> a feature flag in Firefox. :/ (
>> https://bugzilla.mozilla.org/show_bug.cgi?id=1424076)
>>
>> On Saturday, November 10, 2018 at 1:03:08 AM UTC+1, Adam Johnson wrote:
>>>
>>> I also discovered a similar problem recently when deploying the 
>>> "Referrer-Policy" header using James Bennett's library: 
>>> https://django-referrer-policy.readthedocs.io/en/stable/ . Initially I 
>>> opted for 'no-referrer' as I figured it was the most secure, but since this 
>>> check is only done on HTTPS-enabled sites, it wasn't discovered that all 
>>> forms were broken until production where HTTPS is used.
>>>
>>> I think the existence of Referrer-Policy bolsters the argument for this 
>>> option, since some users might want to use 'no-referrer'. And in fact the 
>>> current check presumes the Referrer Policy is left at the default, 
>>> no-referrer-when-downgrade , and I'm pretty sure other values than 
>>> no-referrer break it as well.
>>>
>>> +1 from me (I wonder if it could even be automatic if HSTS is enabled?)
>>>
>>> On Fri, 9 Nov 2018 at 21:31, Aaron Hill  wrote:
>>>
 Currently, Django's CSRF middleware will reject any 'non-safe' HTTPS 
 request that lacks a Referer header: ​
 https://github.com/django/django/blob/22e8ab02863819093832de9f771bf40a62a6bd4a/django/middleware/
 csrf.py#L242

 However, some users may prevent their browsers from sending the Referer 
 header, due to privacy concerns. These users are unable to submit 
 'non-safe' requests (e.g. POST requests) on HTTPS-enabled Django-powered 
 website that use CSRF protection.

 For some websites, checking the Referer header may provide no added 
 security benefit. For example, an HSTS-preloaded website which controls 
 all 
 of its subdomains has nothing to gain from this check - there are no 
 untrusted subdomains which can mount an attack, and HSTS prevents an HTTP 
 MITM attack.


 To allow these websites to provide more flexibility to their users, 
 Django should support disabling this CSRF Referer check. This could be 
 done through a new setting, e.g. ' CSRF_REFERER_CHECK' (defaulting to 
 'True' to avoid breaking existing sites).

 -- 
 You received this message because you are subscribed to the Google 
 Groups "Django developers (Contributions to Django itself)" group.
 To unsubscribe from this group and stop receiving emails from it, send 
 an email to django-develop...@googlegroups.com.
 To post to this group, send email to django-d...@googlegroups.com.
 Visit this group at https://groups.google.com/group/django-developers.
 To view this discussion on the web visit 
 https://groups.google.com/d/msgid/django-developers/bcc04352-cd39-485a-83ad-49d0608d6ccd%40googlegroups.com
  
 
 .
 For more options, visit https://groups.google.com/d/optout.

>>>
>>>
>>> -- 
>>> Adam
>>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-develop...@googlegroups.com .
>> To post to this group, send email to django-d...@googlegroups.com 
>> .
>> Visit this group at https://groups.google.com/group/django-developers.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/5c89439f-1f3c-44ff-99a6-708ae5c42684%40googlegroups.com
>>  
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
> -- 
> Adam
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/d79cf158-c3da-402e-a3a7-5260e7

Re: Pluggable secret key backend

2018-11-10 Thread Dan Davis
Maybe a LoFi way to accomplish this is just to make sure that the
SECRET_KEY is cast to bytes() before use.   That way, a non-bytes object
placed there during settings will be asked to convert it to bytes before
use.   I use the same trick with an internal module that retrieves database
passwords from a web service.   For cx_Oracle, I only had to implement
__str__, but for PostgreSQL, MySQL, and pyodbc to SQL Server I eventually I
collected many other string methods to be duck typed as a string.

The same trick might work today with SECRET_KEY, depending on how it is
used.   If anyone ever does a check that isinstance(settings.SECRET_KEY,
bytes), then we'd have problems, but if Django has the discipline to
iterate it, get its length, and cast it to bytes before use, then it would
be OK.

On Sat, Nov 10, 2018 at 9:36 AM Andreas Pelme  wrote:

> On 10 Nov 2018, at 13:29, Adam Johnson  wrote:
> >
> > Hi Andreas
> >
> > I like your proposal, moving to a backend is an elegant way of solving
> both the immediate problem and opening up the other possibilities you
> mentioned.
>
> Thanks Adam, I am glad you like the proposal. :)
>
> > I think it would also be nice to have an "out of the box" way of
> rotating the key, without needing to implement a custom backend. Perhaps a
> second setting OLD_SECRET_KEYS that may contain a list of old keys that are
> returned for verification too? Or we could allow SECRET_KEY to be a
> list/tuple, and if so, sign with the first and verify with all of them.
>
> Agreed, I will add something like that then! :)
>
> Cheers,
> Andreas
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers  (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/A16A11DF-1439-46EF-BF0D-85C483F53608%40pelme.se
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFzonYbcFn2h0-g8nk9Bj1fgprUU0AFLxph9L_1H%2B0KEMLZ1%3DQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


Re: Proposal to extend support for object permission

2018-11-10 Thread Tobias Bengfort
On 08/11/2018 12:00, Carlton Gibson wrote:
> Perhaps you could put docs changes you'd make in a third PR (or
> ticket if you like, to discuss the outline)? (Maybe one PR with three
> commits makes it easier to review as a whole.)

I created a pull request with some changes, mostly related to documentation:

  https://github.com/django/django/pull/10636

This is still pretty rough. I feel like I overdid it. But I tried to
split it into different commits so that the individual changes can be
cherry-picked.

tobias

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/8139ec3c-963a-dddc-ee17-cf6fcb167434%40posteo.de.
For more options, visit https://groups.google.com/d/optout.


Fellow Reports - November 2018

2018-11-10 Thread Tim Graham


Week ending November 10, 2018

Triaged

---

https://code.djangoproject.com/ticket/29921 - Default BooleanField.required 
different from Django 2.0 when using choices (invalid)

https://code.djangoproject.com/ticket/29928 - TestCase doesn't check for 
foreign key constraints when using sqlite (accepted)

https://code.djangoproject.com/ticket/29932 - QuerySet.difference() after 
intersection() returns incorrect results on SQLite (accepted)

https://code.djangoproject.com/ticket/29930 - View only permissions on 
stacked inline result in error (accepted)

Authored

--

https://github.com/django/django/pull/10622 - Fixed #29934 -- Added 
sqlparse as a required dependency.

Reviewed/committed

--

https://github.com/django/django/pull/7528 - Fixed #25251 -- Made data 
migrations available in TransactionTestCase when using --keepdb.

https://github.com/django/django/pull/10603 - Fixed #29917 -- Stopped 
collecting ModelAdmin.actions from base ModelAdmins.

https://github.com/django/django/pull/10595 - Fixed #29908 -- Fixed setting 
of foreign key after related set access if ForeignKey uses to_field.

https://github.com/django/django/pull/10406 - Refs #29641 -- Refactored 
database schema constraint creation code.
https://github.com/django/django/pull/10632 - Fixed #29940 -- Recommended 
using the ORM rather than raw SQL.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/d65c4b6c-9a64-4709-ae9d-d267285d85a9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: Allow querying JSONField with F objects #29769

2018-11-10 Thread Mani S
Hi devs,
Any update on this? Would like to know your thoughts




Thanks,
Mani

On Sun, Oct 28, 2018 at 1:28 AM Mani S  wrote:

> Django's F objects does not perform JSON lookups. A ticket has been raised
> for the same https://code.djangoproject.com/ticket/29769
>
> I have written a customer expression for querying JSONField and
> * it works!*
>
> class KeyTextTransformFactory:
>
> def __init__(self, key_name):
> self.key_name = key_name
>
> def __call__(self, *args, **kwargs):
> return KeyTextTransform(self.key_name, *args, **kwargs)
>
> class JSONF(F):
>
> def resolve_expression(self, query=None, allow_joins=True, reuse=None, 
> summarize=False, for_save=False):
> rhs = super().resolve_expression(query, allow_joins, reuse, 
> summarize, for_save)
>
> field_list = self.name.split(LOOKUP_SEP)
> for name in field_list[1:]:
> rhs = KeyTextTransformFactory(name)(rhs)
> return rhs
>
> It is necessary to include Cast in rhs,
> Sample.objects.filter(jsonfield__lookup__value=Cast(JSONF('value'),
> IntegerField()))
>
>
> Comment ref: https://code.djangoproject.com/ticket/29769#comment:5
>
>
> Note: This is my first contribution in Django source code, any
> comments/suggestions would help me learn the process better.
>
>
>
>
>
> Thank you,
> Mani
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/88f52ca0-ebd9-4139-b88e-9f21292b7603%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAMRk_KbkxayLiY8Zv_SJNNfBvQajmDoSKZwWQFc_YQyZGnj9yA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


Re: Pluggable secret key backend

2018-11-10 Thread Aymeric Augustin
Hello,

I think this is a great idea.

As suggested by others, an even better default implementation would be:

class SecretKeysBackend:

def get_signing_key(self):
if isinstance(settings.SECRET_KEY, (list, tuple)):
return settings.SECRET_KEY[0]
else:
return settings.SECRET_KEY

def get_verification_keys(self):
if isinstance(settings.SECRET_KEY, (list, tuple)):
return settings.SECRET_KEY
else:
return [settings.SECRET_KEY]

Once Django is updated to take advantage of this feature, hat would make key 
rotation practical for every Django user!

(And it seems easier to adjust the semantics of SECRET_KEY than to introduce a 
SECRET_KEYS settings.)

Best regards,

-- 
Aymeric.



> On 10 Nov 2018, at 11:12, Andreas Pelme  wrote:
> 
> Hi,
> 
> settings.SECRET_KEY can be used for sessions, password resets, form wizards 
> and
> other cryptographic signatures via the signing APIs. Changing SECRET_KEY means
> that all of those will be invalidated and the users will be affected in weird
> ways without really knowing what happened. (Why am I logged out? Where did my
> form submission go? Why does not this password reset link work?). This is
> desirable in case the key is compromised and swift action must be taken.
> 
> There are other situations when it would be nice to change the SECRET_KEY when
> this sudden invalidation is not desirable:
> 
> - When someone leaves a project/company that had access to the production
>  system. After SSH keys/login credentials is revoked the developer could
>  potentially have a copy of the secret key. It is essentially a backdoor with
>  full remote access. It would be wise to rotate the key in those cases.
> 
> - Periodic and automatic rotations of keys to make it less useful in the
>  future.
> 
> The current situation of a single SECRET_KEY makes key rotation impractical. 
> If
> you run a busy site with active users 24/7, there is never a nice time to
> change the SECRET_KEY.
> 
> A solution for this problem would be sign new secrets with a new key while
> still allow signatures made with the old key to be considered valid at the 
> same
> time. Changing keys and having a couple of hours of overlap where signatures
> from both keys are accepted would mitigate most of the user facing problems
> with invalidating sessions, password reset links and form wizard progress.
> 
> You could do this today by implementing your own session backend, message
> storage backend and password reset token generator but that is cumbersome and
> does not work across reusable apps that directly use low level Django signing
> APIs unless they too provide hooks to provide your own secret.
> 
> I propose a pluggable project wide secret key backend
> (settings.SECRET_KEY_BACKEND maybe?) with an API something like:
> 
> class SecretKeyBackend:
>  def get_signing_key(self): …
>  def get_verification_keys(self): ...
> 
> The default (and backward compatible) backend would then be implemented as
> something like:
> 
> class SecretKeySettingsBackend:
>  def get_signing_key(self):
>return settings.SECRET_KEY
>  def get_verification_keys(self):
>return [settings.SECRET_KEY]
> 
> django.core.signing.Signer.{sign,unsign} would need to be updated to use this
> backend instead of directly using settings.SECRET_KEY.
> 
> That would solve the problem project wide and work across any third party
> application that uses django.core.signing directly.
> 
> This would open the door for third party secrets backend packages that
> retrieves keys from systems such as Hashicorp Vault, AWS Secrets Manager,
> Google Cloud KMS, Docker Secrets etc.
> 
> Having a method that retrieves the key would allow changes to secret key 
> during
> run time instead of relying on a hard coded setting would allow the key to
> change without restarting the server process.
> 
> Would something like this be worth pursuing? Could it be designed in som other
> way? I could not find any previous discussion/tickets on this and thought it
> would be a good idea to discuss it here before opening a ticket or making an
> attempt at a PR. :)
> 
> Cheers,
> 
> Andreas
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers  (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/20D8A2BD-BC9C-4F02-9038-044687165DE9%40pelme.se.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from thi