date:20150831

Hi, as far as I know, we haven't banned anyone in Trac. We did receive a 
similar report in IRC last week (maybe it was you). Could you create a 
ticket in the code.djangoproject.com issue tracker with any more details? 
https://github.com/django/code.djangoproject.com/issues

On Monday, August 31, 2015 at 7:29:24 AM UTC-4, Serhiy Int wrote:
>
> I'm getting this always when I try to use Github Login on 
> code.djangoproject.com
>
> AFAIU I was banned there, am I right? But I never received any 
> notification. What can I do to get me unbanned?
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/81491e55-6608-4009-914d-c9d2f15d9c8d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Keeping apps without migrations?

In the absence of a more sophisticated implementation, here's a possible 
simple solution: https://github.com/django/django/pull/5212

Maybe we can come up with a better name for the migrate option (currently 
--run-syncdb).

On Friday, August 28, 2015 at 3:08:26 AM UTC-4, Markus Holtermann wrote:
>
> Hey Anssi,
>
> thanks for the ideas. A few comments inline
>
> On Friday, August 28, 2015 at 3:25:34 PM UTC+10, Anssi Kääriäinen wrote:
>>
>> Could we allow applications to have fake migration files? I'm thinking 
>> of something like operations.AllModelsUpdated(). After this operation 
>> is ran, Django will use the present models for that app. You could 
>> place this as your last migration, and run direct SQL in previous 
>> migrations (or use some completely different mechanism) . Direct SQL 
>> without model state changes will be fast. The last migration would 
>> supposedly be fast, as it only needs to construct up to date images of 
>> the current models. 
>>
>
> That is something you can already do by using `SeparateDatabaseAndState` 
> https://docs.djangoproject.com/en/1.8/ref/migration-operations/#django.db.migrations.operations.SeparateDatabaseAndState.
>  
> Use RunSQL for the Database part and CreateModel for the state part.
>
> The problem is that I'm not at all sure how this will mix with 
>> migrations from other applications, so this proposal is perhaps 
>> technically impossible to do. But if this is feasible, then it should 
>> be possible to mix migrations and manual database changes pretty much 
>> seamlessly. This one would work much better if we had project level 
>> migrations - implementing a migration operation that tells Django that 
>> all of the project's models are up to date is almost trivial. 
>>
>
> I don't see how project level migrations would make that easier. Unless 
> you have only changes to the data in the database Django won't change any 
> SQL and will happily reuse the latest models up to that state.
>
> I believe a fake migration would work pretty well for the use case 
>> where you want to use manual database control for your project, but 
>> use 3rd party applications with migrations. 
>>
>> Another idea I've been thinking is that maybe we could have snapshots 
>> for migrations. The reason I believe this would work is that building 
>> the model state is expensive, but in many cases it is completely 
>> redundant to build the model state from scratch. 
>>
>> The snapshots would essentially do the following operation: after 
>> migrations X, Y and Z are applied the model state will be as presented 
>> in this migration. The snapshots would only contain model state, not 
>> any operations. When you run manage.py migrate, the app state builder 
>> will be able to skip to latest snapshot that has all dependencies 
>> already applied, and then continue building model state from there. 
>> When performance begins to degrade, you can create a new snapshot. The 
>> snapshots would probably have to be project level to work properly. 
>> This would seriously speed up migrations, except for the case where 
>> you run the migrations against fresh database. 
>>
>
> You are only half right: the performance problem is the *rendering* of the 
> models from the model states. However I don't see how having checkpoints 
> would help, reusing the state. It would help to not need to create the 
> state up until that point, but the performance issue there got resolved I 
> think. The main problem is models having 'object references' to other 
> models (`models.ForeignKey(User)` vs `models.ForeignKey('auth.User')`). In 
> order to have consistent references between two model classes (i.e. 
> `ModelA._meta.get_field('fk_to_modelb').related_model is ModelB`) all 
> related models have to be rerendered once one changes: 
> https://github.com/django/django/blob/master/django/db/migrations/state.py#L32
>  
>
>>  - Anssi 
>>
>
> /Markus
>  
>
>> On Fri, Aug 28, 2015 at 3:18 AM, Andrew Godwin  
>> wrote: 
>> > I think I agree with Markus that we do not yet have a good enough 
>> > performance story to make migrations mandatory. If you want to ditch 
>> them 
>> > and do your own database management, we should let you - which we could 
>> do 
>> > with some kind of MIGRATIONS_MODULES opt-out that basically sets 
>> everything 
>> > to pseudo-unmanaged, but I see no reason not to have the syncdb-like 
>> > solution keep working as it at least runs off of SchemaEditor now. 
>> > 
>> > Andrew 
>> > 
>> > On Thu, Aug 27, 2015 at 5:14 PM, Markus Holtermann 
>> >  wrote: 
>> >> 
>> >> The general benefit of *not* having migrations is the incredible 
>> better 
>> >> performance to setup the tables. Especially for larger projects this 
>> is 
>> >> something we should at least keep in mind (see 
>> >> https://code.djangoproject.com/ticket/24743). I spent a fair bit of 
>> time on 
>> >> the issue and have not yet found a satisfying solution. 
>> >> 
>> >> Supporting apps without migrations would esse

Re: Keeping apps without migrations?

As far as how well apps without migrations would work compared to 1.8, a 
couple things come to mind:

1. There's no management commands to get the SQL for these apps.
2. There's no more support for initial data/custom SQL.

On Monday, August 31, 2015 at 10:48:25 AM UTC-4, Tim Graham wrote:
>
> In the absence of a more sophisticated implementation, here's a possible 
> simple solution: https://github.com/django/django/pull/5212
>
> Maybe we can come up with a better name for the migrate option (currently 
> --run-syncdb).
>
> On Friday, August 28, 2015 at 3:08:26 AM UTC-4, Markus Holtermann wrote:
>>
>> Hey Anssi,
>>
>> thanks for the ideas. A few comments inline
>>
>> On Friday, August 28, 2015 at 3:25:34 PM UTC+10, Anssi Kääriäinen wrote:
>>>
>>> Could we allow applications to have fake migration files? I'm thinking 
>>> of something like operations.AllModelsUpdated(). After this operation 
>>> is ran, Django will use the present models for that app. You could 
>>> place this as your last migration, and run direct SQL in previous 
>>> migrations (or use some completely different mechanism) . Direct SQL 
>>> without model state changes will be fast. The last migration would 
>>> supposedly be fast, as it only needs to construct up to date images of 
>>> the current models. 
>>>
>>
>> That is something you can already do by using `SeparateDatabaseAndState` 
>> https://docs.djangoproject.com/en/1.8/ref/migration-operations/#django.db.migrations.operations.SeparateDatabaseAndState.
>>  
>> Use RunSQL for the Database part and CreateModel for the state part.
>>
>> The problem is that I'm not at all sure how this will mix with 
>>> migrations from other applications, so this proposal is perhaps 
>>> technically impossible to do. But if this is feasible, then it should 
>>> be possible to mix migrations and manual database changes pretty much 
>>> seamlessly. This one would work much better if we had project level 
>>> migrations - implementing a migration operation that tells Django that 
>>> all of the project's models are up to date is almost trivial. 
>>>
>>
>> I don't see how project level migrations would make that easier. Unless 
>> you have only changes to the data in the database Django won't change any 
>> SQL and will happily reuse the latest models up to that state.
>>
>> I believe a fake migration would work pretty well for the use case 
>>> where you want to use manual database control for your project, but 
>>> use 3rd party applications with migrations. 
>>>
>>> Another idea I've been thinking is that maybe we could have snapshots 
>>> for migrations. The reason I believe this would work is that building 
>>> the model state is expensive, but in many cases it is completely 
>>> redundant to build the model state from scratch. 
>>>
>>> The snapshots would essentially do the following operation: after 
>>> migrations X, Y and Z are applied the model state will be as presented 
>>> in this migration. The snapshots would only contain model state, not 
>>> any operations. When you run manage.py migrate, the app state builder 
>>> will be able to skip to latest snapshot that has all dependencies 
>>> already applied, and then continue building model state from there. 
>>> When performance begins to degrade, you can create a new snapshot. The 
>>> snapshots would probably have to be project level to work properly. 
>>> This would seriously speed up migrations, except for the case where 
>>> you run the migrations against fresh database. 
>>>
>>
>> You are only half right: the performance problem is the *rendering* of 
>> the models from the model states. However I don't see how having 
>> checkpoints would help, reusing the state. It would help to not need to 
>> create the state up until that point, but the performance issue there got 
>> resolved I think. The main problem is models having 'object references' to 
>> other models (`models.ForeignKey(User)` vs 
>> `models.ForeignKey('auth.User')`). In order to have consistent references 
>> between two model classes (i.e. 
>> `ModelA._meta.get_field('fk_to_modelb').related_model is ModelB`) all 
>> related models have to be rerendered once one changes: 
>> https://github.com/django/django/blob/master/django/db/migrations/state.py#L32
>>  
>>
>>>  - Anssi 
>>>
>>
>> /Markus
>>  
>>
>>> On Fri, Aug 28, 2015 at 3:18 AM, Andrew Godwin  
>>> wrote: 
>>> > I think I agree with Markus that we do not yet have a good enough 
>>> > performance story to make migrations mandatory. If you want to ditch 
>>> them 
>>> > and do your own database management, we should let you - which we 
>>> could do 
>>> > with some kind of MIGRATIONS_MODULES opt-out that basically sets 
>>> everything 
>>> > to pseudo-unmanaged, but I see no reason not to have the syncdb-like 
>>> > solution keep working as it at least runs off of SchemaEditor now. 
>>> > 
>>> > Andrew 
>>> > 
>>> > On Thu, Aug 27, 2015 at 5:14 PM, Markus Holtermann 
>>> >  wrote: 
>>> >> 
>>> >> The general benefit

Erroneous HTTPS CSRF Referrer Failure

I understand why the Referrer check for secure requests is in place.
What is currently preventing cross-domain API requests is that the check
is not configurable. I'm talking specifically about requests when
`request.is_secure()` returns `True` and an unsafe but specifically
cross-origin request is being made.

What currently exists is a simple referrer check.

```
good_referer = 'https://%s/' % request.get_host()
if not same_origin(referer, good_referer):
reason = REASON_BAD_REFERER % (referer, good_referer)
return self._reject(request, reason)
```

Let's discuss two domains, `api.project.com` and
`dashboard.project.com`. Both are served over HTTPS. Both are fronted
with a reverse proxy that performs TLS termination.

Inside the Django settings normally one would configure:

```python
# Read the special header set by the reverse proxy that indicates a
secure request

SECURE_PROXY_SSL_HEADER = ("HTTP_FORWARDED", "https")

# Set the cookies on the root domain, allows api-dev.project.com to be
configured

CSRF_COOKIE_DOMAIN = ".project.com"
SESSION_COOKIE_DOMAIN = ".project.com"

# Configure hosts we allow
ALLOWED_HOSTS = [
"api.project.com"
]
```

Now a `POST` request comes in to the API from a user on
`dashboard.project.com/posts/new`. Django sees the following:

* `POST not in ('GET', 'HEAD', 'OPTIONS', 'TRACE')`, this is an
"unsafe" request

* `request.is_secure()` is `True`
* `Host` header is `api.project.com`
* Referrer is `dashboard.project.com/posts/new`

At this point, the `same_origin(referer, good_referer)` check fails on
[middleware/csrf.py:159](https://github.com/django/django/blob/8047e3666b0b50bb04e6f16c2a4fb21ddfd5713f/django/middleware/csrf.py#L159)
and the entire request breaks. This check is not configurable and does
not allow any hooks to bypass or alter behavior short of turning CSRF
protection off entirely. This check is not present on non-secure
requests and can cause surprise and confusion as to this new behavior
when secure requests are enabled.

### What are some possible solutions?

Why not configure the Host header when sending the request?

Causes all absolute reversed URLs to use that Host header which is
obviously incorrect.

What about serving behind the same reverse proxy and provide path
based routing?

Path based routing is a separate discussion but it introduces cache
problems and configuration problems. Nevertheless, Django should not be
required to serve on the same `Host` as the originator with zero
configuration.

Why not write some custom middleware that does your own CSRF checks?

Not DRY, complicated, prone to failure. Would prefer to not roll my own
security code for obvious reasons.

What are you doing now to solve this issue?

Added a small piece of middleware the changes the `Referer` header to a
"`good_referer`".

### Suggested Solution

Allow specific referrer hosts to be accepted via the settings. The
change could be as simple as:

```
good_referers = [ "https://%s/"; % host for host in
settings.ALLOWED_HOSTS ]
if not any([same_origin(referer, good_referer) for good_referer in
good_referers]):

...
```

Alternatively introduce a new setting that bypasses this secure referer
check. I've outlined the common scenario where this check fails to
provide any usefulness. While I believe this check provides additional
security under ideal and simple use cases I don't believe Django's
security should be relegated to the simple case.

I'd greatly welcome any comments or feedback on this matter, including
alternative suggestions to resolving the issue that don't require making
changes in Django core.

Best,

-Josh

***
[me](http://kehn.us) | [@joshkehn](https://twitter.com/joshkehn)

--
You received this message because you are subscribed to the Google Groups "Django
developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/F09FAEDA-3FF6-4339-9CC4-EF89649F31A5%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Erroneous HTTPS CSRF Referrer Failure

Hi Josh,

On 08/31/2015 09:38 AM, Joshua Kehn wrote:
> I understand why the Referrer check for secure requests is in place.
> What is currently preventing cross-domain API requests is that the check
> is not configurable. I'm talking specifically about requests when
> |request.is_secure()| returns |True| and an unsafe but specifically
> cross-origin request is being made.

I'm not sure what you mean by "unsafe but specifically cross-origin
request" here. I think the point is that the request is in fact safe,
because it's coming from an approved CORS source, but there's no way to
tell the CSRF middleware that.

> What currently exists is a simple referrer check.
> 
> |good_referer = 'https://%s/' % request.get_host() if not
> same_origin(referer, good_referer): reason = REASON_BAD_REFERER %
> (referer, good_referer) return self._reject(request, reason) |
[snip detailed example]
>  This check is not configurable and does
> not allow any hooks to bypass or alter behavior short of turning CSRF
> protection off entirely. This check is not present on non-secure
> requests and can cause surprise and confusion as to this new behavior
> when secure requests are enabled.
> 
> 
>   What are some possible solutions?
[snip unfeasible solutions]
> What are you doing now to solve this issue?
> 
> Added a small piece of middleware the changes the |Referer| header to a
> "|good_referer|".

I've had this same problem in a CORS-accessible API scenario, and I've
used the same solution. Ideally the middleware should verify that the
request is a valid CORS request, from an approved host, before patching
the referer.

Note that the most popular CORS-headers package for Django that I'm
aware of (https://github.com/ottoyiu/django-cors-headers) even includes
a middleware to patch the referer to get around this problem:
https://github.com/ottoyiu/django-cors-headers/blob/master/corsheaders/middleware.py#L26

>   Suggested Solution
> 
> Allow specific referrer hosts to be accepted via the settings. The
> change could be as simple as:
> 
> |good_referers = [ "https://%s/"; % host for host in
> settings.ALLOWED_HOSTS ] if not any([same_origin(referer, good_referer)
> for good_referer in good_referers]): ... |

I don't think just piggybacking on `ALLOWED_HOSTS` like this is
sufficient (though it would be a small improvement on the current
situation). `ALLOWED_HOSTS` is meant to be a list of host names that are
aliases for the _current_ server. It should not also include remote
servers that are authorized for CORS requests to this server. So in your
example, `dashboard.project.com` should _not_ be included in
`ALLOWED_HOSTS` on `api.project.com`, so this fix wouldn't help.

> Alternatively introduce a new setting that bypasses this secure referer
> check.

I don't think there should be a setting to disable referer checking
altogether, but I do think there should be a way to make CORS and the
referer check work together without having to monkey-patch the request
referer.

The simplest approach would be a setting which is a list of valid CSRF
referer hosts. The problem with this is that it doesn't imply any
validation of the request CORS headers, but maybe that's OK. If you're
declaring "I trust this particular remote host not to CSRF my users,"
that's probably sufficient.

Alternatives might include just adding CORS support to Django directly,
or providing a more generic hook for third-party CORS packages to take
over the referer check themselves.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/55E4824B.4090802%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.

signature.asc
Description: OpenPGP digital signature

Re: Erroneous HTTPS CSRF Referrer Failure

On 31 Aug 2015, at 12:35, Carl Meyer wrote:

I'm not sure what you mean by "unsafe but specifically cross-origin
request" here. I think the point is that the request is in fact safe,
because it's coming from an approved CORS source, but there's no way
to

tell the CSRF middleware that.

Yes, exactly. However I avoided using the term CORS because (1) it is
not supported in Django core and (2) specifically has use around
`OPTIONS` requests and ensuring preflight browser requests succeed.

I've had this same problem in a CORS-accessible API scenario, and I've
used the same solution. Ideally the middleware should verify that the
request is a valid CORS request, from an approved host, before
patching

the referer.

Exactly except the referrer should not be patched. The referrer may be
used upstream in other code, what should change is the CSRF check.

Note that the most popular CORS-headers package for Django that I'm
aware of (https://github.com/ottoyiu/django-cors-headers) even
includes

a middleware to patch the referer to get around this problem:
https://github.com/ottoyiu/django-cors-headers/blob/master/corsheaders/middleware.py#L26

I'm using that one at present, I did not know the middleware had that.
Thanks!

Re: Erroneous HTTPS CSRF Referrer Failure

Hi Josh,

I think it would make sense to just add a `CSRF_ALLOWED_REFERERS`
setting, defaulting to `None` (which would give the current behavior of
requiring a match with the `Host` header). If set, it would be a list of
valid referer hosts. Documentation needs to be extremely clear that you
should only include hosts that are under your control, or you trust
completely, to this setting.

Anyone else see a problem with that that I'm missing?

You up for filing a ticket and maybe a patch/pull-request too?

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/55E48D02.8000301%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Erroneous HTTPS CSRF Referrer Failure

Anyone else see a problem with that that I'm missing?

I think this sounds fine.

You up for filing a ticket and maybe a patch/pull-request too?

Absolutely.

Thanks

--jk

***
[me](http://kehn.us) | [@joshkehn](https://twitter.com/joshkehn)

On 31 Aug 2015, at 13:21, Carl Meyer wrote:

Hi Josh,

I think it would make sense to just add a `CSRF_ALLOWED_REFERERS`
setting, defaulting to `None` (which would give the current behavior
of
requiring a match with the `Host` header). If set, it would be a list
of
valid referer hosts. Documentation needs to be extremely clear that
you

should only include hosts that are under your control, or you trust
completely, to this setting.

Anyone else see a problem with that that I'm missing?

You up for filing a ticket and maybe a patch/pull-request too?

Carl

--
You received this message because you are subscribed to a topic in the
Google Groups "Django developers (Contributions to Django itself)"
group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/django-developers/eQeaNzSlSbw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to
django-developers@googlegroups.com.

Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/55E48D02.8000301%40oddbird.net.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Django
developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/C181054B-2D7D-45DB-93DE-12EE20CC4C47%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Erroneous HTTPS CSRF Referrer Failure

A couple follow-up thoughts:

On 08/31/2015 11:22 AM, Joshua Kehn wrote:
> On 31 Aug 2015, at 13:21, Carl Meyer wrote:
> I think it would make sense to just add a |CSRF_ALLOWED_REFERERS|
> setting, defaulting to |None| (which would give the current behavior of
> requiring a match with the |Host| header). If set, it would be a list of
> valid referer hosts. Documentation needs to be extremely clear that you
> should only include hosts that are under your control, or you trust
> completely, to this setting.

1) Maybe `CSRF_TRUSTED_REFERERS` is a better name, to emphasize the
implied trust.

2) If it's set, a match with the Host header (or maybe with any host in
`ALLOWED_HOSTS`) should still be allowed, so you aren't forced to
duplicate `ALLOWED_HOSTS` inside `CSRF_TRUSTED_REFERERS`.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/55E48E0E.3020302%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Erroneous HTTPS CSRF Referrer Failure


On 31 Aug 2015, at 13:25, Carl Meyer wrote:


A couple follow-up thoughts:

On 08/31/2015 11:22 AM, Joshua Kehn wrote:

On 31 Aug 2015, at 13:21, Carl Meyer wrote:
 I think it would make sense to just add a |CSRF_ALLOWED_REFERERS|
 setting, defaulting to |None| (which would give the current behavior 
of
 requiring a match with the |Host| header). If set, it would be a 
list of
 valid referer hosts. Documentation needs to be extremely clear that 
you

 should only include hosts that are under your control, or you trust
 completely, to this setting.


1) Maybe `CSRF_TRUSTED_REFERERS` is a better name, to emphasize the
implied trust.


That name does sound better.

2) If it's set, a match with the Host header (or maybe with any host 
in

`ALLOWED_HOSTS`) should still be allowed, so you aren't forced to
duplicate `ALLOWED_HOSTS` inside `CSRF_TRUSTED_REFERERS`.


So the check here would look something like (excuse any typos, I'm not 
writing this in an editor):


```
allowed_hosts = list(settings.ALLOWED_HOSTS) + 
list(settings.CSRF_TRUSTED_REFERERS)

if "*" in allowed_hosts:
# Skip further checks since Django has been configured to allow any 
host.

else:
good_referers = ["https://{0}".format(host) for host in allowed_hosts]
if not any([same_origin(referer, good_referer) for good_referer in 
good_referers]):

# Reject CSRF referer mismatch
```

I would imagine that the `"*"` host would be allowed in 
`CSRF_TRUSTED_REFERERS` just like it is in `ALLOWED_HOSTS`?


The next thought would be a separate extension but worth explorting: 
Should Django then enforce CSRF referrer checks outside of secure 
requests if we have a setting specifically for it?


--jk

***
[me](http://kehn.us) | [@joshkehn](https://twitter.com/joshkehn)

--
You received this message because you are subscribed to the Google Groups "Django 
developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/6F8CD7D1-DD97-4CF2-BC80-78C7C589EC9B%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Erroneous HTTPS CSRF Referrer Failure

On 08/31/2015 11:37 AM, Joshua Kehn wrote:
> 2) If it's set, a match with the Host header (or maybe with any host in
> |ALLOWED_HOSTS|) should still be allowed, so you aren't forced to
> duplicate |ALLOWED_HOSTS| inside |CSRF_TRUSTED_REFERERS|.
> 
> So the check here would look something like (excuse any typos, I'm not
> writing this in an editor):
> 
> |allowed_hosts = list(settings.ALLOWED_HOSTS) +
> list(settings.CSRF_TRUSTED_REFERERS) if "*" in allowed_hosts: # Skip
> further checks since Django has been configured to allow any host. else:
> good_referers = ["https://{0}".format(host) for host in allowed_hosts]
> if not any([same_origin(referer, good_referer) for good_referer in
> good_referers]): # Reject CSRF referer mismatch |
> 
> I would imagine that the |"*"| host would be allowed in
> |CSRF_TRUSTED_REFERERS| just like it is in |ALLOWED_HOSTS|?

No, I don't think `*` should be allowed in `CSRF_TRUSTED_REFERERS`; I
don't think there is any scenario in which that is a safe or reasonable
configuration.

And I think that the fact that it's allowed in `ALLOWED_HOSTS` might be
a reason to just stick to "Host header or CSRF_TRUSTED_REFERERS", and
leave ALLOWED_HOSTS out of it.

> The next thought would be a separate extension but worth explorting:
> Should Django then enforce CSRF referrer checks outside of secure
> requests if we have a setting specifically for it?

No, that would be a backwards-incompatible change, and the REFERER check
offers zero additional security in the HTTP case, because HTTP is
wide-open to MITM attacks regardless.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/55E49551.7030402%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Erroneous HTTPS CSRF Referrer Failure

Is this related or duplicate to https://code.djangoproject.com/ticket/24496? 
That ticket has a patch that got stalled a bit, but might be worth reviving 
first in case this new one causes it to go stale.

On Monday, August 31, 2015 at 1:37:45 PM UTC-4, Joshua Kehn wrote:
>
> On 31 Aug 2015, at 13:25, Carl Meyer wrote:
>
> A couple follow-up thoughts:
>
> On 08/31/2015 11:22 AM, Joshua Kehn wrote:
>
> On 31 Aug 2015, at 13:21, Carl Meyer wrote:
> I think it would make sense to just add a |CSRF_ALLOWED_REFERERS|
> setting, defaulting to |None| (which would give the current behavior of
> requiring a match with the |Host| header). If set, it would be a list of
> valid referer hosts. Documentation needs to be extremely clear that you
> should only include hosts that are under your control, or you trust
> completely, to this setting.
>
> 1) Maybe CSRF_TRUSTED_REFERERS is a better name, to emphasize the
> implied trust.
>
> That name does sound better.
>
> 2) If it's set, a match with the Host header (or maybe with any host in
> ALLOWED_HOSTS) should still be allowed, so you aren't forced to
> duplicate ALLOWED_HOSTS inside CSRF_TRUSTED_REFERERS.
>
> So the check here would look something like (excuse any typos, I'm not 
> writing this in an editor):
>
> allowed_hosts = list(settings.ALLOWED_HOSTS) + 
> list(settings.CSRF_TRUSTED_REFERERS)
> if "*" in allowed_hosts:
> # Skip further checks since Django has been configured to allow any host.
> else:
> good_referers = ["https://{0}".format(host) for host in allowed_hosts]
> if not any([same_origin(referer, good_referer) for good_referer in 
> good_referers]):
> # Reject CSRF referer mismatch
>
> I would imagine that the "*" host would be allowed in 
> CSRF_TRUSTED_REFERERS just like it is in ALLOWED_HOSTS?
>
> The next thought would be a separate extension but worth explorting: 
> Should Django then enforce CSRF referrer checks outside of secure requests 
> if we have a setting specifically for it?
>
> --jk
> --
>
> me  | @joshkehn 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/f9ed52f3-28a1-4918-a62f-a76fba06e3b5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Erroneous HTTPS CSRF Referrer Failure

On 31 Aug 2015, at 13:56, Carl Meyer wrote:

> No, I don't think `*` should be allowed in `CSRF_TRUSTED_REFERERS`; I
> don't think there is any scenario in which that is a safe or reasonable
> configuration.
>
> And I think that the fact that it's allowed in `ALLOWED_HOSTS` might be
> a reason to just stick to "Host header or CSRF_TRUSTED_REFERERS", and
> leave ALLOWED_HOSTS out of it.

I would agree. I'll draft a ticket and post here once I have.

> No, that would be a backwards-incompatible change, and the REFERER check
> offers zero additional security in the HTTP case, because HTTP is
> wide-open to MITM attacks regardless.
>

Good points, agreed.

I'll get the ticket in and work on a patch later today for review

Thanks

--jk

***
[me](http://kehn.us) | [@joshkehn](https://twitter.com/joshkehn)

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/B66A3D89-FCA5-4541-82E1-7A3A0C646C1F%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Erroneous HTTPS CSRF Referrer Failure


On 31 Aug 2015, at 14:02, Tim Graham wrote:

Is this related or duplicate to 
https://code.djangoproject.com/ticket/24496?
That ticket has a patch that got stalled a bit, but might be worth 
reviving

first in case this new one causes it to go stale.


Looks related.

If we decide to go with Troy Grosfield's suggestion of adding a 
CSRF_WHITELIST_ORIGINS setting (which I like), I can document that 
instead.


Sounds very similar to what we've discussed here.

--jk

***
[me](http://kehn.us) | [@joshkehn](https://twitter.com/joshkehn)

--
You received this message because you are subscribed to the Google Groups "Django 
developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/507B756F-4CDC-4AC8-9CA3-DCE9007A53FD%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Ticket #25328 - LiveServerTestCase with HTTPS - opinions?

Could you explain what types of tests require https (or otherwise expand on 
the rationale for adding this)? If the answer is that we don't want to have 
a separate "test settings" with all the SSL settings disabled 
(CSRF_COOKIE_SECURE, SECURE_HSTS_SECONDS, etc.) then I understand this, 
however, there was previous discussion about adding SSL support to 
runserver met with mixed reaction [1] and it seems a little funny to me 
that we'd including a test SSL server and not a runserver solution.

[1] 
https://groups.google.com/d/topic/django-developers/PgBcSEiUdw0/discussion

On Monday, August 31, 2015 at 6:58:33 AM UTC-4, Jakub Gocławski wrote:
>
> Hi,
>
> Ticket: https://code.djangoproject.com/ticket/25328
>
> I made a proposal to include a LiveServerTestCase, which runs an HTTPS 
> server instead of standard HTTP server. I've been asked to get more 
> feedback for this idea.
>
> If my refactor is merged I can release the test case as a separate app. 
> But including it in Django itself would encourage to write HTTPS-only 
> applications and to write Selenium tests for them.
> I've seen recent moves to encourage various security "good habits" into 
> Django, like integrating "security header" in 1.8 (
> https://docs.djangoproject.com/en/1.8/ref/middleware/#module-django.middleware.security).
>  
> Let's Enrypt (which should be released in November) will futher ease 
> enabling HTTPS.
> Testing HTTPS-enabled site in Django should be just as easy as testing a 
> standard HTTP site.
>
> Do you think it's a good idea to add it to Django core? Or should I 
> release it as as a separate app?
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/91570cfc-0e36-4c6b-ba68-82c123dea894%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Erroneous HTTPS CSRF Referrer Failure

On 08/31/2015 12:09 PM, Joshua Kehn wrote:
> On 31 Aug 2015, at 14:02, Tim Graham wrote:
> 
> Is this related or duplicate to
> https://code.djangoproject.com/ticket/24496?
> That ticket has a patch that got stalled a bit, but might be worth
> reviving
> first in case this new one causes it to go stale.
> 
> Looks related.
> 
> If we decide to go with Troy Grosfield's suggestion of adding a
> CSRF_WHITELIST_ORIGINS setting (which I like), I can document that
> instead.
> 
> Sounds very similar to what we've discussed here.

Yes, CSRF_WHITELIST_ORIGINS is the same feature discussed here (but I
think CSRF_TRUSTED_REFERERS -- or CSRF_TRUSTED_ORIGINS if we don't like
reproducing the RFC-codified mis-spelling of "referrer" -- is a better
name).

This solution is more powerful than just using CSRF_COOKIE_DOMAIN, since
it also allows for separate-domain CORS situations in addition to
cross-subdomain requests. So I would consider this to be a good fix for
#24496; I don't think we need another ticket.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/55E49BD3.4050605%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: OpenPGP digital signature

Re: Improving MSSQL and Azure SQL support on Django

By "we" do you mean the engineers at Microsoft who will be working on this? 
Will they be doing any planning for this before the meet up in October? I'd 
like to know more specifics about the agenda and goals for the workshop.

Also, I wanted to make sure my question about a videoconferencing option 
for Django developers interested in participating in the discussion at that 
time but unable to travel to Seattle didn't get lost.

On Tuesday, August 25, 2015 at 7:53:48 PM UTC-4, Meet Bhagdev wrote:
>
>
> Absolutely agree with Tim here. We need to start exploring all the cool 
> open sourced third party adapters. Borrowing/adopting them definitely seems 
> like the way to go about things instead of re inventing the wheel. I tried 
> doing some research and came across the following: 
>
> 1. Django-mssql 
>
> 2. Django-pymssql 
>
> 3. Django-pyodbc-azure 
>
> 4. Django-pyodbc 
>
>
> Am I missing any? 
>
> Best,
> Meet
>
> On Monday, August 24, 2015 at 11:12:44 AM UTC-7, Tim Graham wrote:
>>
>> I guess the first step is to identify which third-party backend(s) we'll 
>> target to adopt officially (or at least borrow from heavily). For example, 
>> will we need separate backends for MSSQL and Azure? (Knowing nothing about 
>> the landscape myself, this question could be nonsensical.) Is this 
>> discussion something that should happen before the October summit? It seems 
>> to me the face-to-face time will likely be more productive if we have some 
>> of the high-level details ironed out.
>>
>> By the way, is videoconferencing an option for Django developers 
>> interested in participating in the discussion at that time but unable to 
>> travel to Seattle?
>>
>> On Saturday, August 22, 2015 at 5:53:37 PM UTC-4, Shai Berger wrote:
>>>
>>> On Saturday 22 August 2015 13:28:31 Aymeric Augustin wrote:
>>>
>>> > 
>>>
>>> > There isn’t such a clear story for running Django on Linux. This led 
>>> me to
>>>
>>> > write https://github.com/aaugustin/django-pymssql. Alternatives 
>>> include
>>>
>>> > https://github.com/denisenkom/django-sqlserver and
>>>
>>> > https://github.com/lionheart/django-pyodbc.
>>>
>>>  
>>>
>>> There's also django-pyodbc-azure 
>>> , a fork of 
>>> django-pyodbc (actually, the current django-pyodbc is also a fork of the 
>>> original project, which has been discontinued). I took the liberty to 
>>> forward the message to that project.
>>>
>>>  
>>>
>>> Shai.
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/25afe4a4-34d8-44c9-b3e0-c4e85b5c6b5f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Making the test suite run faster

Aymeric, did you envision any changes to Django's CI setup? Currently we 
run 1 Jenkins executor per CPU, so I don't know that adding parallelization 
would have any benefit? (We are already using all 8 CPUs when we're running 
8 concurrent builds from the matrix.)  If not, then I wonder how we can 
ensure that Django's test suite continues to work well in parallel.

On Monday, August 31, 2015 at 4:11:29 AM UTC-4, Aymeric Augustin wrote:
>
> Hi Tino,
>
> 2015-08-31 8:35 GMT+02:00 Tino de Bruijn >:
>
>> - What happens when two SerializeMixin tests try to lock the same file? 
>> Does one wait for the other (probably not), or is a lockfile exception 
>> raised?
>>
>
> The second one waits for the first one to complete. This happens all the 
> time because conflicting tests are often defined close to one another and 
> the parallel test runner runs tests in order.
>  
>
>> - How does this work in combination with the --keepdb flag
>>
>
> All clones of the test database are kept.
>
> There's one edge case: if you run with --parallel-num=2 --keepdb and then 
> --parallel-num=4 --keepdb, the test runner will crash because clones 3 and 
> 4 won't exist. I don't think that's a big problem.
>
> -- 
> Aymeric.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/4c827092-f990-4ea8-9960-dd4b86fbe980%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Ticket #23242 -- Approximate date_hierarchy

This seems okay at first glance, though I wonder if an enum-like object 
might be better than magic int/boolean values. Something like:

class ApproximateWith(object):
NONE = 0
YEARS = 1
MONTHS = 2
DAYS = 3

Do you think a separate ModelAdmin attribute better than allowing something 
like:

date_hierarchy = ('pub_date', ApproximateWith.YEARS)

Values that can be string or tuple have contributed to bugs in the past, 
but I thought it might be worth proposing.

On Saturday, August 29, 2015 at 4:50:20 PM UTC-4, Gavin Wahl wrote:
>
> I'm going to fix ticket #23242 by adding an option to approximate the 
> date_hierarchy options instead of calculating them exactly. The 
> implementation is straightforward but I'm not sure what the interface 
> should be. 
>
> The basic idea is instead of doing `SELECT DISTINCT date_trunc('year', 
> created_at) FROM table`, then a date_trunc('month', created_at) to get the 
> list of months, and so on, you can find the range with `SELECT 
> date_trunc('year', max(created_at)), date_trunc('year', min(creeated_at))` 
> and just assume that every year/month/day in between is filled in.
>
> How should this feature be enabled? Should it be possible to approximate 
> the year list, but once you've selected the year switch to exact? I'm 
> thinking something like:
>
> - approximate_date_hierarchy = False # default
> - approximate_date_hierarchy = 1 # approximate years only
> - approximate_date_hierarchy = 2 # approximate months and years
> - approximate_date_hierarchy = 3 or True # approximate days, months, and 
> years
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/9152eb8c-7c47-4a35-a89b-278fd0aa2fc8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Making the test suite run faster

2015-08-31 Thread Aymeric Augustin

On 31 août 2015, at 20:41, Tim Graham  wrote:

> Aymeric, did you envision any changes to Django's CI setup?

Glad you asked :-) I have some thoughts but no definitive opinion.

> Currently we run 1 Jenkins executor per CPU, so I don't know that adding 
> parallelization would have any benefit? (We are already using all 8 CPUs when 
> we're running 8 concurrent builds from the matrix.)  

Indeed, we’re already parallelizing by running simultaneously tests against 
multiple databases. So we aren’t going to make order-of-magnitude gains.

The last processes of a given build may complete a bit faster if they’re run in 
parallel mode as they will make better use of the server’s resources once there 
are fewer active executors than CPUs. So the completion time for a matrix build 
may be slightly lower, unless the increased concurrency slows tests down 
sufficiently to negate this effect.

If increased concurrency is a problem, half the number of executors running 
tests with --parallel-num=2 should still complete faster, because it “packs” 
CPU usage more efficiently towards the end of the matrix build. It’s hard to 
tell what’s going to happen without trying.

> If not, then I wonder how we can ensure that Django's test suite continues to 
> work well in parallel.

Perhaps we could make the --parallel option the default in runtests.py. You’d 
have to pass --parallel-num=1 to disable it. This would help occasional 
sprinters unfamiliar with the option. They could just run ./runtests.py, melt 
their laptops, and not even have time for a coffee break.

-- 
Aymeric.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/3292A158-8F48-4742-BCC8-295BAD935BCE%40polytechnique.org.
For more options, visit https://groups.google.com/d/optout.

Re: Improving MSSQL and Azure SQL support on Django

2015-08-31 Thread Meet Bhagdev

Hi Tim,
Thanks for bringing it up. Yes, we will have developers who will engage 
with Django developers (like yourself) during the workshop. 

The goals of the October workshop are to:

1)  Get to know each other and begin building a relationship

2)  Get in a room with Microsoft developers and discuss the current 
landscape 

3)  Work on half day coding sprint(s) with Microsoft developers to get 
started with contributions

4)  Establish a plan for how Microsoft can best contribute to Django, 
and ensure we have great integration between Azure SQL and MSSQL and Django 
by maintaining our contributions


The key takeaway is that we want to contribute to existing solutions to 
improve the Django and MSSQL/Azure SQL story. To do so we want to 
understand the current landscape, the gaps, and the next steps to make this 
happen (the right way).


We are currently in the planning stages and would love to get feedback. 
What do you think about the goals mentioned above? Is there anything you 
like to add/remove?


We can definitely make attending via Skype an option for attendees unable 
to make it in person. 


Best,

Meet


On Monday, August 31, 2015 at 11:33:40 AM UTC-7, Tim Graham wrote:
>
> By "we" do you mean the engineers at Microsoft who will be working on 
> this? Will they be doing any planning for this before the meet up in 
> October? I'd like to know more specifics about the agenda and goals for the 
> workshop.
>
> Also, I wanted to make sure my question about a videoconferencing option 
> for Django developers interested in participating in the discussion at that 
> time but unable to travel to Seattle didn't get lost.
>
> On Tuesday, August 25, 2015 at 7:53:48 PM UTC-4, Meet Bhagdev wrote:
>>
>>
>> Absolutely agree with Tim here. We need to start exploring all the cool 
>> open sourced third party adapters. Borrowing/adopting them definitely seems 
>> like the way to go about things instead of re inventing the wheel. I tried 
>> doing some research and came across the following: 
>>
>> 1. Django-mssql 
>>
>> 2. Django-pymssql 
>>
>> 3. Django-pyodbc-azure 
>>
>> 4. Django-pyodbc 
>>
>>
>> Am I missing any? 
>>
>> Best,
>> Meet
>>
>> On Monday, August 24, 2015 at 11:12:44 AM UTC-7, Tim Graham wrote:
>>>
>>> I guess the first step is to identify which third-party backend(s) we'll 
>>> target to adopt officially (or at least borrow from heavily). For example, 
>>> will we need separate backends for MSSQL and Azure? (Knowing nothing about 
>>> the landscape myself, this question could be nonsensical.) Is this 
>>> discussion something that should happen before the October summit? It seems 
>>> to me the face-to-face time will likely be more productive if we have some 
>>> of the high-level details ironed out.
>>>
>>> By the way, is videoconferencing an option for Django developers 
>>> interested in participating in the discussion at that time but unable to 
>>> travel to Seattle?
>>>
>>> On Saturday, August 22, 2015 at 5:53:37 PM UTC-4, Shai Berger wrote:

 On Saturday 22 August 2015 13:28:31 Aymeric Augustin wrote:

 > 

 > There isn’t such a clear story for running Django on Linux. This led 
 me to

 > write https://github.com/aaugustin/django-pymssql. Alternatives 
 include

 > https://github.com/denisenkom/django-sqlserver and

 > https://github.com/lionheart/django-pyodbc.

  

 There's also django-pyodbc-azure 
 , a fork of 
 django-pyodbc (actually, the current django-pyodbc is also a fork of the 
 original project, which has been discontinued). I took the liberty to 
 forward the message to that project.

  

 Shai.

>>>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/0b70b91e-00d8-4188-8644-e48286b8a571%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Improving MSSQL and Azure SQL support on Django

To arrive at the best solution, I think a lot of discussion needs to happen 
on this mailing list before October. As for me, besides the name of some 
existing packages that offer SQL Server/Azure support, I know very little 
about the current landscape so I really wouldn't have anything to offer in 
a discussion, but there are others on this list who definitely do.

The existing Microsoft database backends have come out of specific 
developer needs and I think it would maximize success if your team were 
personally invested  in the backend by building a Django application backed 
by SQL Server/Azure. If I were a Microsoft engineer responsible for this 
project, I would start by building a small Django application to solve some 
problem that my team has. Then I would test it out with all the third-party 
backends for Microsoft databases previously mentioned to get a feel for how 
they work. I should learn enough to at least participate in a discussion 
with other Django developers about the direction of this project. Ideally, 
I would be able to learn enough to write up a Django Enhancement Proposal 
[1] that summarizes the landscape and describes what an "official solution" 
should look like.

If you are relying on members of the Django team to do all this upfront 
work, then please tell us so we can see if someone will do it. If you are 
expecting to get all this done in a couple days in Seattle, this doesn't 
seem feasible to me. However, if we have a working proposal by 
mid-September that the community can review and give feedback on, then I 
think our time together in October could be productive. I don't know 
anything about the background of your team, but it seems pretty unlikely 
that any sort of coding sprint would be useful unless the work and learning 
I described happens well before the workshop.

[1] https://github.com/django/deps/

On Monday, August 31, 2015 at 3:07:28 PM UTC-4, Meet Bhagdev wrote:
>
> Hi Tim,
> Thanks for bringing it up. Yes, we will have developers who will engage 
> with Django developers (like yourself) during the workshop. 
>
> The goals of the October workshop are to:
>
> 1)  Get to know each other and begin building a relationship
>
> 2)  Get in a room with Microsoft developers and discuss the current 
> landscape 
>
> 3)  Work on half day coding sprint(s) with Microsoft developers to get 
> started with contributions
>
> 4)  Establish a plan for how Microsoft can best contribute to Django, 
> and ensure we have great integration between Azure SQL and MSSQL and Django 
> by maintaining our contributions
>
>
> The key takeaway is that we want to contribute to existing solutions to 
> improve the Django and MSSQL/Azure SQL story. To do so we want to 
> understand the current landscape, the gaps, and the next steps to make this 
> happen (the right way).
>
>
> We are currently in the planning stages and would love to get feedback. 
> What do you think about the goals mentioned above? Is there anything you 
> like to add/remove?
>
>
> We can definitely make attending via Skype an option for attendees unable 
> to make it in person. 
>
>
> Best,
>
> Meet
>
>
> On Monday, August 31, 2015 at 11:33:40 AM UTC-7, Tim Graham wrote:
>>
>> By "we" do you mean the engineers at Microsoft who will be working on 
>> this? Will they be doing any planning for this before the meet up in 
>> October? I'd like to know more specifics about the agenda and goals for the 
>> workshop.
>>
>> Also, I wanted to make sure my question about a videoconferencing option 
>> for Django developers interested in participating in the discussion at that 
>> time but unable to travel to Seattle didn't get lost.
>>
>> On Tuesday, August 25, 2015 at 7:53:48 PM UTC-4, Meet Bhagdev wrote:
>>>
>>>
>>> Absolutely agree with Tim here. We need to start exploring all the cool 
>>> open sourced third party adapters. Borrowing/adopting them definitely seems 
>>> like the way to go about things instead of re inventing the wheel. I tried 
>>> doing some research and came across the following: 
>>>
>>> 1. Django-mssql 
>>>
>>> 2. Django-pymssql 
>>>
>>> 3. Django-pyodbc-azure 
>>>
>>> 4. Django-pyodbc 
>>>
>>>
>>> Am I missing any? 
>>>
>>> Best,
>>> Meet
>>>
>>> On Monday, August 24, 2015 at 11:12:44 AM UTC-7, Tim Graham wrote:

 I guess the first step is to identify which third-party backend(s) 
 we'll target to adopt officially (or at least borrow from heavily). For 
 example, will we need separate backends for MSSQL and Azure? (Knowing 
 nothing about the landscape myself, this question could be nonsensical.) 
 Is 
 this discussion something that should happen before the October summit? It 
 seems to me the face-to-face time will likely be more productive if we 
 have 
 some of the hi

Re: Adding more repr() methods

Thanks, that was my intuition but just wanted to double check.

On Sunday, August 30, 2015 at 9:53:02 AM UTC-4, Ned Batchelder wrote:
>
> The eval criterion rarely seems like a useful rule of thumb to me.  Does 
> anyone actually use this to make objects?  I think the useful rule is, a 
> repr should be an unambiguous representation of the object that is useful 
> in debugging.  Certainly the "eval" rule is sufficient for the 
> "unambiguous" rule, but it isn't necessary.  I would rather have something 
> readable than something evalable.
>
> --Ned.
>
> On 8/29/15 1:05 PM, Tim Graham wrote:
>
> On the pull request, Fábio has raised the argument, "the __repr__ method 
> should be unambiguous and eval(repr(obj)) == obj".
>
> Is this something we should strive for?
>
> On Friday, August 28, 2015 at 9:18:04 PM UTC-4, Josh Smeaton wrote: 
>>
>> Right, I agree with you in the case then. But I think the general idea of 
>> more/improved repr methods is a good one.
>>
>> On Friday, 28 August 2015 23:02:25 UTC+10, Tim Graham wrote: 
>>>
>>> The repr for MessageMiddleware includes settings.MESSAGE_STORAGE.
>>>
>>> On Thursday, August 27, 2015 at 8:59:37 PM UTC-4, Josh Smeaton wrote: 

 Generally, I'm a fan of adding repr methods. I don't think we should 
 run around finding all the places they might be useful and adding them, 
 but 
 if someone is to put forward patches where they've noticed a benefit, we 
 should encourage that. 

 I'm not sure I fully understand your concerns though. It seems to me 
 that the repr methods in that PR only include things relating to the class 
 being observed. Did I miss something?

 Cheers

 On Friday, 28 August 2015 01:16:02 UTC+10, Tim Graham wrote: 
>
> I'd like to ask for other opinions on this pull request which adds 
> more __repr__() methods: https://github.com/django/django/pull/5059
>
> Some concerns:
>
> * Adding things that aren't part of the class (e.g. settings) seems 
> questionable to me.
>
> * There is a chance to leak sensitive data in __repr__ as done in 
> #22990  (doesn't seems 
> to be a concern with the changes here, but is something to be mindful of).
>
>
> Thanks!
>
 -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-develop...@googlegroups.com .
> To post to this group, send email to django-d...@googlegroups.com 
> .
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit 
> 
> https://groups.google.com/d/msgid/django-developers/3f26bb3a-5c3d-4057-a788-92cde6768afb%40googlegroups.com
> .
> For more options, visit https://groups.google.com/d/optout.
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/1b03f2c6-7b97-45f8-ac07-5d8820f3f793%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Improving MSSQL and Azure SQL support on Django

2015-08-31 Thread Russell Keith-Magee

Hi Tim, Meet

I know this is very late to be mentioning, but one idea that worth
raising: DjangoCon US is next week (in Austin). Thursday and Friday
are coding sprints, where there will be many people (including a good
chunk of the core team) looking for projects to hack on.

Microsoft is already a sponsor of DjangoCon itself; if we can get some
engineers from the Azure team (assuming there aren't already some
coming), we can start some of this prep work.

Yours,
Russ Magee %-)

On Tue, Sep 1, 2015 at 3:46 AM, Tim Graham  wrote:
> To arrive at the best solution, I think a lot of discussion needs to happen
> on this mailing list before October. As for me, besides the name of some
> existing packages that offer SQL Server/Azure support, I know very little
> about the current landscape so I really wouldn't have anything to offer in a
> discussion, but there are others on this list who definitely do.
>
> The existing Microsoft database backends have come out of specific developer
> needs and I think it would maximize success if your team were personally
> invested  in the backend by building a Django application backed by SQL
> Server/Azure. If I were a Microsoft engineer responsible for this project, I
> would start by building a small Django application to solve some problem
> that my team has. Then I would test it out with all the third-party backends
> for Microsoft databases previously mentioned to get a feel for how they
> work. I should learn enough to at least participate in a discussion with
> other Django developers about the direction of this project. Ideally, I
> would be able to learn enough to write up a Django Enhancement Proposal [1]
> that summarizes the landscape and describes what an "official solution"
> should look like.
>
> If you are relying on members of the Django team to do all this upfront
> work, then please tell us so we can see if someone will do it. If you are
> expecting to get all this done in a couple days in Seattle, this doesn't
> seem feasible to me. However, if we have a working proposal by mid-September
> that the community can review and give feedback on, then I think our time
> together in October could be productive. I don't know anything about the
> background of your team, but it seems pretty unlikely that any sort of
> coding sprint would be useful unless the work and learning I described
> happens well before the workshop.
>
> [1] https://github.com/django/deps/
>
>
> On Monday, August 31, 2015 at 3:07:28 PM UTC-4, Meet Bhagdev wrote:
>>
>> Hi Tim,
>> Thanks for bringing it up. Yes, we will have developers who will engage
>> with Django developers (like yourself) during the workshop.
>>
>> The goals of the October workshop are to:
>>
>> 1)  Get to know each other and begin building a relationship
>>
>> 2)  Get in a room with Microsoft developers and discuss the current
>> landscape
>>
>> 3)  Work on half day coding sprint(s) with Microsoft developers to get
>> started with contributions
>>
>> 4)  Establish a plan for how Microsoft can best contribute to Django,
>> and ensure we have great integration between Azure SQL and MSSQL and Django
>> by maintaining our contributions
>>
>>
>> The key takeaway is that we want to contribute to existing solutions to
>> improve the Django and MSSQL/Azure SQL story. To do so we want to understand
>> the current landscape, the gaps, and the next steps to make this happen (the
>> right way).
>>
>>
>> We are currently in the planning stages and would love to get feedback.
>> What do you think about the goals mentioned above? Is there anything you
>> like to add/remove?
>>
>>
>> We can definitely make attending via Skype an option for attendees unable
>> to make it in person.
>>
>>
>> Best,
>>
>> Meet
>>
>>
>> On Monday, August 31, 2015 at 11:33:40 AM UTC-7, Tim Graham wrote:
>>>
>>> By "we" do you mean the engineers at Microsoft who will be working on
>>> this? Will they be doing any planning for this before the meet up in
>>> October? I'd like to know more specifics about the agenda and goals for the
>>> workshop.
>>>
>>> Also, I wanted to make sure my question about a videoconferencing option
>>> for Django developers interested in participating in the discussion at that
>>> time but unable to travel to Seattle didn't get lost.
>>>
>>> On Tuesday, August 25, 2015 at 7:53:48 PM UTC-4, Meet Bhagdev wrote:


 Absolutely agree with Tim here. We need to start exploring all the cool
 open sourced third party adapters. Borrowing/adopting them definitely seems
 like the way to go about things instead of re inventing the wheel. I tried
 doing some research and came across the following:

 1. Django-mssql

 2. Django-pymssql

 3. Django-pyodbc-azure

 4. Django-pyodbc


 Am I missing any?

 Best,
 Meet

 On Monday, August 24, 2015 at 11:12:44 AM UTC-7, Tim Graham wrote:
>
> I guess the first step is to identify which third-party b

Re: Erroneous HTTPS CSRF Referrer Failure