Proposal: security enhancements

[James Bennett]

I've written this up in pseudo-DEP format partly for ease of
organization, and partly because I'm unsure whether it would require a
DEP. Right now I'm just throwing it out here as a proposal, and
offering to work on implementing it; if you have questions, concerns,
or suggestions for things to add to it, please let me know :)


Introduction
-

Django has acquired a reputation for being helpful with/good at
"security". Which has been a focus since at least 1.0; Django goes out
of its way to protect you out of the box against common security
problems, and includes a lot of tools geared toward making it easy to
do the right thing when building apps.

But security is a Red Queen problem; we can't ever pause and rest,
since as soon as we do we'll fall behind.

In light of which, I'd like to propose a few security-related
improvements/additions, and offer to put in the time to help implement
them. My baseline goal here is to set things up such that it's
possible, using only things that ship with Django itself and an
SSL-enabled hosting service, to score an A+ rating on
securityheaders.com. Beyond that, some stretch goals are possible.


Content Security Policy


CSP[1] is one of the more important defenses available against XSS and
a few other types of attacks. Django does not currently have
out-of-the-box support for it; the usual solution is to install
django-csp[2], configure it and set up its middlware to emit the CSP
header.

I'd like to see support for CSP built in to Django. This could be done
by integrating django-csp, which is open source and uses a compatible
license, though there'd need to be some updates to its documentation
(which seems to lag a bit behind what the code supports) and it might
also be good to switch over to using dictionary-based configuration.


Referrer-Policy
-

The Referrer-Policy header[3] is relatively new, but gaining support
in major browsers. It helps enhance both security and privacy.

I wrote a package earlier this year[4] that supports sending the
Referrer-Policy header, and BSD-licensed it. It would be relatively
easy to integrate this into Django.


Subresource integrity
-

Subresource integrity[5] allows the author of an HTML document to
supply the expected hashes of resources (scripts, stylesheets, images,
etc.) referenced in the document. This helps to harden against not
only local breaches, but also breaches of third-party scripts and
CDNs.

I don't have a clear vision yet of how it would work, but I'd like to
see the form media system and the staticfiles app grow support for
SRI. In the case of form media, the output already generates the full
HTML and could include the hash. In the case of the {% static %}
template tag, I'm open to suggestions for how it could
generate/include the hash. The admin should use this by default.


CORS
-

Cross-Origin Resource Sharing is increasingly important on the modern
web, and Django does not currently support setting CORS headers. There
is an open-source package for CORS in Django[6], which could be
integrated.


rel="noopener"
-

Setting rel="noopener" on links which will open in new tabs/windows is
an easy way to head off certain types of surprising attacks[7]. Like
SRI, I don't yet have a great plan for how this could be supported
easily/automatically in Django, but I'd like to give it a try.


Improved system-check integration
---

Currently the security system-check is OK, but still a bit
minimal. I'd like to add support for checking at least everything I've
proposed adding above, and probably several more items. In my ideal
world, a service like securityheaders.com could be replicated by the
security check.

In my magical stretch-goal land, I'd also figure out a way to support
the pyup safety library[8] to scan for a requirements file and any
dependencies in setup.py, and warn if known-insecure versions are
specified.


Improved security documentation
-

Right now we have OK documentation on security, in that we have an
overview[9] listing the broad categories of things Django can protect
you against and pointing to their documentation.

I'd like to expand on that by including a rundown of where to learn
about common security issues, how to go beyond the basic protective
mechanisms Django includes today, and automated scanning/security
check tools like securityheaders, Mozilla Observatory, pyup,
requires.io, etc.


[1] https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP
[2] https://pypi.org/project/django_csp/
[3]
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Referrer-Policy
[4] https://pypi.org/project/django-referrer-policy/
[5]
https://developer.mozilla.org/en-US/docs/Web/Security/Subresource_Integrity
[6] https://pypi.org/project/django-cors-headers/
[7] https://mathiasbynens.github.io/rel-noopene

Re: Proposal: security enhancements

[Jacob Kaplan-Moss]

Great ideas, James. I totally agree we shouldn't rest on our laurels, and
love the goal of pushing things forwards. Overall, I'm not sure a DEP is
needed: each of these things is fairly small and tightly scoped, can be
implemented on its own, and provides value independent of the whole. That
seems like a scenario where just a bunch of loosely-related PRs makes the
most sense. Added bonus: many of these things would be fairly easy pickings
for a new contributor. If you wanted, you could delegate/coordinate some of
this, and help us get more folks involved as a bonus.

Some comments on specifics:

On Tue, May 1, 2018 at 12:28 AM, James Bennett 
wrote:

> Content Security Policy
>

CSP is a tricky one. On the one hand, it's a tremendously effective defense
against XSS. But, it's pretty tricky to get right: I've seen several sites
struggle with a proper CSP config for years. It tends to be beyond the
grasp of your classic one- or two-person dev team. When you get it wrong,
it totally hoses your site.

Most complex sites find they need to operate in report-only mode for a
while and analyze the data before switching to enforce. And that requires a
good reporting/analysis mechanism (something like report-uri.com, or a
local equivalent).

So all that to say: I highly support exploring this more, but it could be
easy to turn CSP into a foot-gun. I don't think it's as easy as just
shipping django-csp and calling it a day; we'd need to make sure it's not
going to cause more problems than it solves.


> Referrer-Policy
>

+1, this seems like a no-brainer.


> Subresource integrity
>

The jury seems to still be out on the value of SRI (at least, in my corner
of the security community). It has some serious problems with dynamic
assets, and with externally-hosted tools like Google Analytics. I'm not
convinced that the current spec is fully-baked enough for us to support.

The admin's a special case since we tightly control what's shipped there;
SRI-for-the-admin would be a nice, if incremental improvement. Preventing
injection attacks in the admin is a very good thing :)

CORS
>

 Yup, another no-brainer.

rel="noopener"
>

I'm not sure I get this one, might need to see what you come up with to
understand the effect.


> In my magical stretch-goal land, I'd also figure out a way to support
>
the pyup safety library[8] to scan for a requirements file and any
> dependencies in setup.py, and warn if known-insecure versions are
> specified.
>

This seems entirely doable! Of course, grappling with the various options
for dependency management might make this.. less fun (https://xkcd.com/1987/
).

Jacob

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAK8PqJG-xF6zJXwrm-SBifUgTUtjd8_rrn_-2rO8AaDthbL9Jw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Fellow Reports -- April 2018

[Carlton Gibson]

Hi all, 

Calendar Week 17 -- ending 27th April.

Reviewed Patches on:

https://code.djangoproject.com/ticket/8936  -- Add view (read-only) 
permission to admin
https://code.djangoproject.com/ticket/29353 -- _middleware_chain is not set 
for StaticFilesHandler
https://code.djangoproject.com/ticket/28687 -- Add a 'Not Empty' option to 
admin's related filter

Triaged:

https://code.djangoproject.com/ticket/29361 -- 
django.contrib.flatpages.urls does not have an app_name attribute (wontfix)


Kind Regards,

Carlton

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/67f52335-1a88-46cf-9843-12c696a44b9f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Django bugfix releases: 2.0.5 and 1.11.13

[Tim Graham]

Details are available on the Django project weblog:
https://www.djangoproject.com/weblog/2018/may/01/bugfix-releases/

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAD-rxRD7sevj%3Dm5_gVLaNe25TaL8mN1V%3Dx_MPFBUPkwL2d5FnQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Fellow Reports - April 2018

[Tim Graham]

Week ending April 28, 2018

Triaged

---

https://code.djangoproject.com/ticket/29354 - ordering inherited from 
abstract superclass does not get applied to admin inlines (invalid)

https://code.djangoproject.com/ticket/29366 - startproject --template 
option doesn't support http authentication, neither through .netrc file 
(duplicate)

https://code.djangoproject.com/ticket/29365 - the reverse function does not 
work in the current application, which was included in the URL using 
namespace (needsinfo)

Reviewed/committed

--

https://github.com/django/django/pull/9891 - Fixed #28312 -- Made 
ModelChoiceIterator.__len__() more memory-efficient.

https://github.com/django/django/pull/9892 - Refs #28312 -- Added an 
optimized __bool__() to ModelChoiceIterator.

https://github.com/django/django/pull/9893 - Fixed #29349 -- Doc'd the 
default behavior of router.allow_relation().

https://github.com/django/django/pull/7610 - Fixed #27480 -- Added 
cache.touch().

https://github.com/django/django/pull/9902 - Fixed #29367 -- Fixed model 
state on objects with a primary key created with QuerySet.bulk_create().
https://github.com/django/django/pull/9600 - Fixed #29015 -- Added an 
exception if the PostgreSQL database name is too long.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/a090374c-9c31-45dd-9347-74f0abc23983%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Thoughts on diff-based migrations with django?

[djrobstep]

Bump! Is there a documented way to generate the from-scratch creation SQL 
for the currently defined models? If not, is it possible to add something 
that spits this out? Happy to take a look at implementing this.


On Sunday, 25 February 2018 13:48:27 UTC+11, djrobstep wrote:
>
> Yes, I don't think it would make sense to include in django directly. 
> Because the diffing approach works at the database directly, below the 
> layer of the models, the two parts don't really need to be integrated. All 
> that is really required for django to support this as an option is to 
> provide a documented way to emit the from-scratch creation SQL for the 
> currently defined models.
>
> As far as I can tell there used to be an admin command available for this 
> in version 1.x but it was removed? Is there already another supported way 
> to do this?
>
> Incidentally, I've used this approach on some larger projects with 
> diverging branches and it worked pretty well. You do have to set some 
> things up differently tho - keep meaning to write a blog post or something 
> about this. The need to have access to a production schema is an additional 
> hurdle, but well worth it for the additional testability and robustness, in 
> my experience.
>
> On Wednesday, 21 February 2018 18:44:30 UTC+11, Andrew Godwin wrote:
>>
>> Hi,
>>
>> This was an approach we deliberately avoided during development of the 
>> current migration system - it works poorly with diverging feature branches 
>> and requires that your developers have access to production schema at all 
>> times (and additionally that you don't have divergent production/staging/QA 
>> deploys, like many environments do).
>>
>> I think it would be a fine addition as an optional third-party tool, but 
>> it's not something that really makes sense to include in Django at this 
>> point I think.
>>
>> Andrew
>>
>> On Mon, Feb 19, 2018 at 10:05 PM, djrobstep  wrote:
>>
>>> Hi all,
>>>
>>> Some thoughts on schema migrations that may interest you. By way of 
>>> background, I'm the author of migra (https://github.com/djrobstep/migra), 
>>> a schema comparison/diff tool for Postgres.
>>>
>>> I initially wrote this tool because I wanted to be able to generate 
>>> migration scripts automatically, without needing historical migration files.
>>>
>>> It means that you can do things like sync your database to your models 
>>> mostly automatically, and also explicitly test for a matching schema. 
>>> Instead of a chain of migration files you only ever need to keep track of 
>>> one, containing any pending changes).
>>>
>>> I've used this approach with success on a number of projects now, and it 
>>> seems to work pretty well. I also talked about this approach to migrations 
>>> at the most recent PostgresOpen (
>>> https://www.youtube.com/watch?v=xr498W8oMRo).
>>>
>>> The downside of this approach is that it's slightly more operationally 
>>> complex (you need have a temporary copy of your production schema available 
>>> for comparison purposes, typically a --schema-only dump file) and also that 
>>> it's postgres-only at this point. It's also conceptually quite different to 
>>> the typical Django/Rails style of migrations which represents a barrier to 
>>> entry.
>>>
>>> In spite of this, I think doing migrations this way has some compelling 
>>> advantages, that are relevant to users of Django.
>>>
>>> I'm interested in your opinions on this topic as Django developers. What 
>>> do you think? Is it something worth offering as an option for managing 
>>> Django migrations? If your feelings are positive, I'm happy to take a look 
>>> at extending the code accordingly.
>>>
>>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "Django developers (Contributions to Django itself)" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to django-develop...@googlegroups.com.
>>> To post to this group, send email to django-d...@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/django-developers.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/django-developers/5f84157b-071e-42c3-90dd-6f9e3df48ab3%40googlegroups.com
>>>  
>>> 
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/896fc8