Re: integrating django-secure

2014-08-28 Thread Tim Graham 
The settings for the SecurityMiddleware as they appear in django-secure are:

SECURE_HSTS_SECONDS=0
SECURE_HSTS_INCLUDE_SUBDOMAINS=False
SECURE_CONTENT_TYPE_NOSNIFF=False
SECURE_BROWSER_XSS_FILTER=False
SECURE_SSL_REDIRECT=False
SECURE_SSL_HOST=None
SECURE_REDIRECT_EXEMPT=[]

Yo-Yo, would be helpful to say *why* you are -1 so we can take that into 
consideration.

On Thursday, August 28, 2014 2:45:07 AM UTC-4, Yo-Yo Ma wrote:
>
> +1 on basically adding the functionality of checksecure to the default 
> validation.
>
> -1 to adding the middleware.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/1903db9f-8ea9-4303-9d99-8431c7a4976d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Improvement to objects.get_or_create and objects.update_or_create

2014-08-28 Thread Tom Evans 
On Wed, Aug 27, 2014 at 10:19 PM, Benjamin Scherrey
 wrote:
> I don't believe the functionality is backwards incompatible at all unless
> I'm missing something. The new behavior of automatically selecting the
> optimal search field (prioritized by pk first then by any discovered field
> marked as unique) would only occur if the 'default' parameter was None.

So, if I passed in kwargs={'pk': 7, 'foo': 'bar', 'wibble': 'quuz'},
with "foo" and "wibble" being non-indexed fields, then the query that
would be run by get_or_create to determine whether the item exists
would differ in your new version.

This means it is not backwards compatible, as if I passed in those
arguments to get_or_create(), then I expect one of three things to
happen:

  1) the object returned to have those values
  2) an object is newly created that has those values
  3) an error occurs because an object could not be created that has
those values.

The proposed change breaks that contract, eg it would simply return
whatever item has the pk 7, regardless of 'foo' or 'wibble'
attributes.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFHbX1J-EL%3DPV2x9dZevZzcaWG%2BE30O18nf3gFrLQKjFoK1O9A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Improvement to objects.get_or_create and objects.update_or_create

2014-08-28 Thread Babatunde Akinyanmi 
On 27 Aug 2014 22:19, "Benjamin Scherrey"  wrote:
>

>
> You're right, it wasn't hard to implement as external code as I've
already done so. The main reason why I propose it for inclusion into
Django's codebase is because the new behavior is actually what I would have
expected these two functions to do and, so far, individuals that I've
polled said the same thing. When they discover what the generated SQL
actually looks like they're quite surprised.
>

+1.

I agree on the 2 points. I'm not expert enough to comment on the backwards
compatibility though.

>
> -- Ben
>
>
> On Thu, Aug 28, 2014 at 3:55 AM, Marc Tamlyn 
wrote:
>>
>> This would be fairly significantly backwards incompatible, and any
heuristic based code I think belongs in user space.
>>
>> It should not be too problematic to write as an external module, if it
gains a lot of traction we can reconsider.
>>
>> Worth also noting that the json implementation in contrib.postgres is
being delayed until at least postgres 9.4 and will use the jsonb data type
which does have an equality operator.
>>
>>
>> On 27 August 2014 21:04, Benjamin Scherrey  wrote:
>>>
>>> Apologies for the cross-post. I imagine this is actually where this
proposal belongs. Would anyone be interested in getting an implementation
of this for consideration for incorporation into some upcoming or back
ported release of Django?
>>>
>>>   -- Ben Scherrey
>>>
>>> -- Forwarded message --
>>> From: Benjamin Scherrey 
>>> Date: Mon, Aug 25, 2014 at 4:58 PM
>>> Subject: Improvement to objects.get_or_create and
objects.update_or_create
>>> To: django-users 
>>>
>>>
>>> Just want to run an idea by the list for a feature improvement for the
oft-used convenience functions get_or_create and update_or_create. I'll
just talk about get_or_create for now on but everything I saw going forward
applies to both.
>>>
>>> It's a common idiom to populate a dictionary and simply pass it
straight on to the function when creating or updating objects. While
get_or_create takes a named parameter, defaults, as the content to
store/update and the normal parameters are used for both search and
content. The problem is that quite often you just have a dictionary of
content you want to throw into the model table and don't want to separate
it out into search content vs. default content. If you don't separate it
out you get a big nasty where clause on the select that tries to compare
every bit of content for the item.
>>>
>>> For the most part this behavior is just a nuisance and probably
sub-optimal db query. However, with the new json data type it will actually
cause the code to break as you can't pass in a json structure as a search
value in a where clause. Your app will break.
>>>
>>> My proposal, which I've written in my own code, is to have
get_or_create automatically select the most appropriate data to use for the
search clause and then to make the rest "default" data to be applied to the
found or newly created object. It first tries to see if the primary key
field is present in the data structure. If that's not found then it goes
through the model fields and looks for any field designated as unique in
the data structure. The result is a cleaner and faster SQL request and a
prevention of the above mentioned error condition. If no primary key or
unique field is present in the data structure then behavior will continue
as currently implemented.
>>>
>>> Is this a change that would be welcome in the django project? If so I'd
 be happy to create a pull request that implements this policy and
supporting unit test coverage.
>>>
>>> thanx,
>>>
>>>   -- Ben
>>>
>>> --
>>> Chief Systems Architect Proteus Technologies
>>> Chief Fan Biggest Fan Productions
>>> Personal blog where I am not your demographic.
>>>
>>> This email intended solely for those who have received it. If you have
received this email by accident - well lucky you!!
>>>
>>> --
>>> You received this message because you are subscribed to the Google
Groups "Django developers" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
an email to django-developers+unsubscr...@googlegroups.com.
>>> To post to this group, send email to django-developers@googlegroups.com.
>>> Visit this group at http://groups.google.com/group/django-developers.
>>> To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/CAHN%3D9D5-9GyYAKVn4j0xB7tvvDtUtYFFMrUxc2UxPm%3DvgebA3w%40mail.gmail.com
.
>>>
>>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>> --
>> You received this message because you are subscribed to the Google
Groups "Django developers" group.
>> To unsubscribe from this group and stop receiving emails from it, send
an email to django-developers+unsubscr...@googlegroups.com.
>> To post to this group, send email to django-developers@googlegroups.com.
>> Visit this group at http://groups.google.com/group/django-developers.
>> To view this discussion on the web visit
https://gro

Re: integrating django-secure

2014-08-28 Thread Tim Graham 
I've implemented the ability to register "deployment checks" by adding 
deploy=True to register: @register("tag_name", deploy=True). These checks 
are only run if you pass the --deploy flag to check. So in development you 
can run `manage.py check --deploy --settings=settings_prod` to check your 
production settings file. Running these checks automatically if DEBUG is 
False would likely give them better visibility, but I don't see an easy way 
of disabling them when testing if we did that.

Regarding settings, would it be preferable to move them into a single 
dictionary setting called something like SECURITY_MIDDLEWARE_CONFIG?

On Thursday, August 28, 2014 6:27:40 AM UTC-4, Tim Graham wrote:
>
> The settings for the SecurityMiddleware as they appear in django-secure 
> are:
>
> SECURE_HSTS_SECONDS=0
> SECURE_HSTS_INCLUDE_SUBDOMAINS=False
> SECURE_CONTENT_TYPE_NOSNIFF=False
> SECURE_BROWSER_XSS_FILTER=False
> SECURE_SSL_REDIRECT=False
> SECURE_SSL_HOST=None
> SECURE_REDIRECT_EXEMPT=[]
>
> Yo-Yo, would be helpful to say *why* you are -1 so we can take that into 
> consideration.
>
> On Thursday, August 28, 2014 2:45:07 AM UTC-4, Yo-Yo Ma wrote:
>>
>> +1 on basically adding the functionality of checksecure to the default 
>> validation.
>>
>> -1 to adding the middleware.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/5069271f-8a21-4fdd-921f-ee2baa11b45b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: integrating django-secure

2014-08-28 Thread Florian Apolloner 


On Thursday, August 28, 2014 2:44:08 PM UTC+2, Tim Graham wrote:
>
> Regarding settings, would it be preferable to move them into a single 
> dictionary setting called something like SECURITY_MIDDLEWARE_CONFIG?
>

Definitely.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/00fdf211-18d2-46f8-bca0-56de5ff37e36%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: integrating django-secure

2014-08-28 Thread Michael Manfre 
On Thu, Aug 28, 2014 at 8:44 AM, Tim Graham  wrote:

> I've implemented the ability to register "deployment checks" by adding
> deploy=True to register: @register("tag_name", deploy=True). These checks
> are only run if you pass the --deploy flag to check. So in development you
> can run `manage.py check --deploy --settings=settings_prod` to check your
> production settings file. Running these checks automatically if DEBUG is
> False would likely give them better visibility, but I don't see an easy way
> of disabling them when testing if we did that.
>
> Regarding settings, would it be preferable to move them into a single
> dictionary setting called something like SECURITY_MIDDLEWARE_CONFIG?
>

Yes. It is much nicer having a collection of related settings in a dict.


>
> On Thursday, August 28, 2014 6:27:40 AM UTC-4, Tim Graham wrote:
>>
>> The settings for the SecurityMiddleware as they appear in django-secure
>> are:
>>
>> SECURE_HSTS_SECONDS=0
>> SECURE_HSTS_INCLUDE_SUBDOMAINS=False
>> SECURE_CONTENT_TYPE_NOSNIFF=False
>> SECURE_BROWSER_XSS_FILTER=False
>> SECURE_SSL_REDIRECT=False
>> SECURE_SSL_HOST=None
>> SECURE_REDIRECT_EXEMPT=[]
>>
>> Yo-Yo, would be helpful to say *why* you are -1 so we can take that into
>> consideration.
>>
>
-1 on having to subclass a middleware to define its settings. Doing that
seems like the wrong approach and prevents having settings consolidated in
a single place. This would make it more difficult to have different
settings for different environments. This would likely result in a snippet
that subclasses the middleware to look in settings.


>
>> On Thursday, August 28, 2014 2:45:07 AM UTC-4, Yo-Yo Ma wrote:
>>>
>>> +1 on basically adding the functionality of checksecure to the default
>>> validation.
>>>
>>> -1 to adding the middleware.
>>>
>>  --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/5069271f-8a21-4fdd-921f-ee2baa11b45b%40googlegroups.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAGdCwBtC-eBxk0D1Wka5FKjaNg9_dxpWwsb6m84Er3Q%3D%2BfsdAw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Improvement to objects.get_or_create and objects.update_or_create

2014-08-28 Thread Collin Anderson 
This is a bit magical. It means adding or removing a unique constraint 
could have additional unintended side effects.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/3276b4f8-1f89-4ee8-a1ec-bbffc901aa57%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Improvement to objects.get_or_create and objects.update_or_create

2014-08-28 Thread Benjamin Scherrey 
You are. of course, correct Tom. I've never once used it that way but that
is the published api/contract as you stated. Possibly it should have
insisted that the programmer designate which terms are search terms
separate from those which are content only rather than making it optional.
But my proposal does indeed result in a very different semantic ultimately.
Thanx for the feedback. Guess I'll keep wrapping it outside of the core
code.

-- Ben


On Thu, Aug 28, 2014 at 5:35 PM, Tom Evans  wrote:

> On Wed, Aug 27, 2014 at 10:19 PM, Benjamin Scherrey
>  wrote:
> > I don't believe the functionality is backwards incompatible at all unless
> > I'm missing something. The new behavior of automatically selecting the
> > optimal search field (prioritized by pk first then by any discovered
> field
> > marked as unique) would only occur if the 'default' parameter was None.
>
> So, if I passed in kwargs={'pk': 7, 'foo': 'bar', 'wibble': 'quuz'},
> with "foo" and "wibble" being non-indexed fields, then the query that
> would be run by get_or_create to determine whether the item exists
> would differ in your new version.
>
> This means it is not backwards compatible, as if I passed in those
> arguments to get_or_create(), then I expect one of three things to
> happen:
>
>   1) the object returned to have those values
>   2) an object is newly created that has those values
>   3) an error occurs because an object could not be created that has
> those values.
>
> The proposed change breaks that contract, eg it would simply return
> whatever item has the pk 7, regardless of 'foo' or 'wibble'
> attributes.
>
> Cheers
>
> Tom
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/CAFHbX1J-EL%3DPV2x9dZevZzcaWG%2BE30O18nf3gFrLQKjFoK1O9A%40mail.gmail.com
> .
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Chief Systems Architect Proteus Technologies 
Chief Fan Biggest Fan Productions 
Personal blog where I am not your demographic
.

This email intended solely for those who have received it. If you have
received this email by accident - well lucky you!!

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAHN%3D9D6CbRJccYZwRDgGv6wfoy5RaU%3D8_PTLyJ3DC%3D26S1uZQw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: integrating django-secure

2014-08-28 Thread Carl Meyer 
On 08/28/2014 12:45 AM, Yo-Yo Ma wrote:
> +1 on basically adding the functionality of checksecure to the default 
> validation.
> 
> -1 to adding the middleware.

This doesn't make sense to me. Half the checks that checksecure performs
are related to whether you've turned on functionality from the
middleware. If you don't have the middleware, then you can't add those
checks either; how can you add a check for "do you have this setting set
which doesn't do anything?"

I do have some questions about how to "group" middleware; i.e. does it
make sense to have a single SecurityMiddleware (like that in
django-secure) that does a bunch of different things depending on
settings toggles? Or separate middleware for each individual feature,
following the lead of XFrameOptionsMiddleware (a django-secure feature
which has already been independently added to Django)? Or going the
other direction, just forget SecurityMiddleware and add all the various
togglable bits of functionality to CommonMiddleware?

I don't have a clear idea what the best approach is here, just raising
it as a question.

Thanks Tim for taking on this project!

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/53FFADFD.4040609%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.

Re: integrating django-secure

2014-08-28 Thread Carl Meyer 
On 08/28/2014 06:44 AM, Tim Graham wrote:
> I've implemented the ability to register "deployment checks" by adding
> deploy=True to register: @register("tag_name", deploy=True). These
> checks are only run if you pass the --deploy flag to check. So in
> development you can run `manage.py check --deploy
> --settings=settings_prod` to check your production settings file.
> Running these checks automatically if DEBUG is False would likely give
> them better visibility, but I don't see an easy way of disabling them
> when testing if we did that.

This makes sense to me. I don't like the fact that we assume in some
places that "DEBUG off means production", and I would not prefer to add
another such assumption here.

> Regarding settings, would it be preferable to move them into a single
> dictionary setting called something like SECURITY_MIDDLEWARE_CONFIG?

Sure.

FWIW, catching up on the thread, I fully agree with Russ that this
should be in core, not contrib, and I agree with Aymeric and Russ that
global config is better done via settings than via subclassing middleware.

Carl

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/53FFAEF9.5030800%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.

Re: Support byte range requests in django.views.static.serve

2014-08-28 Thread mdj2 
I made a ticket in the hopes of drawing more attention to the patch:

https://code.djangoproject.com/ticket/23382

On Sunday, April 13, 2014 8:30:37 PM UTC-7, md...@pdx.edu wrote:
>
> Is the Django community interested in supporting HTTP range requests in 
> django.views.static.serve 
> ?
>
> The primary benefit I see is that it makes files served up for  and 
>  "seek-able" with the django server. This generally isn't a problem 
> for small files (except in Chrome 
> ), 
> but becomes an issue for larger ones.
>
> Werkzeug has a function that parses the range header 
> ,
>  
> which I used to support range requests in a Django application. I estimate 
> that robust support for HTTP range requests would cost <300 lines of code, 
> plus tests.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/bd2ee88a-12df-4cad-8b4e-b8c26f5c77e2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: integrating django-secure

2014-08-28 Thread Tim Graham 
Thank-you all for the feedback. The PR now uses a dictionary for the 
settings. A couple drawbacks to this approach:
* It requires some logic in django/conf/__init__.py to allow users to 
override select keys in the setting while preserving the other defaults.
* The docs and error messages will have to be much more verbose (e.g. the 
'HSTS_INCLUDE_SUBDOMAINS' key in SECURITY_MIDDLEWARE_CONFIG rather than 
simply 'SECURE_HSTS_INCLUDE_SUBDOMAINS').
There have been some other movements to consolidate settings into a dict 
(e.g. #22734 for email settings) so it seems like we prefer this approach 
despite the drawbacks.

As far as splitting up the middleware, I had similar thoughts as well. I 
don't see a need to split up SecurityMiddleware, although it does seem 
slightly inconsistent to have other middleware like XFrameOptionsMiddleware 
which could just as easily be combined in SecurityMiddleware (as it was in 
django-secure). Splitting it up would require adding more checks (one for 
each miiddleware) which I wouldn't be wild about.

On Thursday, August 28, 2014 6:37:03 PM UTC-4, Carl Meyer wrote:
>
> On 08/28/2014 06:44 AM, Tim Graham wrote: 
> > I've implemented the ability to register "deployment checks" by adding 
> > deploy=True to register: @register("tag_name", deploy=True). These 
> > checks are only run if you pass the --deploy flag to check. So in 
> > development you can run `manage.py check --deploy 
> > --settings=settings_prod` to check your production settings file. 
> > Running these checks automatically if DEBUG is False would likely give 
> > them better visibility, but I don't see an easy way of disabling them 
> > when testing if we did that. 
>
> This makes sense to me. I don't like the fact that we assume in some 
> places that "DEBUG off means production", and I would not prefer to add 
> another such assumption here. 
>
> > Regarding settings, would it be preferable to move them into a single 
> > dictionary setting called something like SECURITY_MIDDLEWARE_CONFIG? 
>
> Sure. 
>
> FWIW, catching up on the thread, I fully agree with Russ that this 
> should be in core, not contrib, and I agree with Aymeric and Russ that 
> global config is better done via settings than via subclassing middleware. 
>
> Carl 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/b7a82555-36f9-44df-be1b-3651043d8a35%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.