Re: request to reopen ticket #24522: --random test option

[chris.j...@gmail.com]

For those that would rather it be opt-out, remember that we can always make 
it opt-out later, once we have more experience with the feature.

Regarding the suggestion to have `--random 0` mean not to use randomizing, 
I also think that could be confusing. 0 is also a valid integer seed, and 
it seems nice to have the option of passing 0 as a simple seed if one 
wants. We could use --no-random for disabling if we ever wanted to make the 
feature opt-out in the future.

--Chris



On Wednesday, March 10, 2021 at 4:42:08 AM UTC-8 Mariusz Felisiak wrote:

> I usually agree with new features being opt-in, but perhaps this case is 
>> different?
>>
>> If I had tests that are breaking if executed randomly, I’d want to know 
>> about it yesterday. IOW, I’m having difficulty imagining a scenario where 
>> the user would be thankful for not activating this feature by default. So 
>> personally, I’d like to see an opt-out setting for this in settings.py.
>>
>> /$0.02 
>> Fran
>>
>
> `--reverse`  will catch 95% of test isolation issues for you. It's highly 
> more likely that running tests in reverse order will catch isolation issue 
> for you than running them in a non-deterministic order. 
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/3ca3cbeb-ac37-457c-bcb2-5d6585e8ac99n%40googlegroups.com.

Re: Rethink (?) how we handle security headers.

[chris.j...@gmail.com]

I just came across this thread after seeing a related PR. Rather than a 
"get_security_header()" function, couldn't Django make the setting an 
object that is accessed by the user like a dict, but internally (underneath 
the hood) it tries from a user-defined dict and then falls back to a value 
in a Django-defined dict of defaults?

--Chris

On Sunday, January 24, 2021 at 1:08:24 PM UTC-8 timog...@gmail.com wrote:

> If we go with something like:
>
> SECURITY_HEADERS = {
> "REFERRER_POLICY": "same-origin",
> "HSTS": {
> "PRELOAD": True,
> "SECONDS": 1_000_000,
>  },
> }
>
> should we have an empty dictionary as the default or a dictionary with 
> defaults for all settings? 
>
> A drawback of an empty dictionary is that any code that wants to retrieve 
> a setting has to do something like 
> settings.SECURITY_HEADERS.get('REFERRER_POLICY', '') which 
> means the fallback value is duplicated every place the setting is 
> consulted.  Maybe a way to mitigate that is to add a 
> "get_security_header()" function that retrieves from django.conf.settings 
> (user-defined settings) and falls back to some defaults.
>
> A drawback of a fully-populated default is that overriding it either 
> requires users to redefine the entire dictionary or writing code like:
>
> from django.conf.global_settings import SECURITY_HEADERS
> SECURITY_HEADERS['REFERRER_POLICY'] = '...'
>
> If users redefine the entire dictionary, then adding new SECURITY_HEADERS 
> options in future Django versions will require something like 
> "get_security_header()" since we won't be able to assume the key is there 
> in projects upgrading from older versions.
>
> I'm not so sure the benefit of consolidating to a single setting is worth 
> this additional complexity.
> On Tuesday, January 19, 2021 at 8:27:58 AM UTC-5 Adam Johnson wrote:
>
>> I don't see the need for adding a setting to add headers to outgoing 
>> responses. A middleware to do so is a handful of lines:
>>
>> class SecurityHeadersMiddleware:
>> def __init__(self, get_response):
>> self.get_response = get_response
>>
>> def __call__(self, request):
>> response = self.get_response(request)
>> response["Cross-Origin-Embedder-Policy"] = "require-corp"
>> return response
>>
>> Also there are many non-security related headers one might want to add, 
>> so having the ability to add them in a setting called SECURITY_HEADERS 
>> could be confusing for maintenance.
>>
>> On Tue, 19 Jan 2021 at 12:02, Tim Graham  wrote:
>>
>>> I guess there have been a few different proposals here, but I imagined 
>>> SECURITY_HEADERS would allow setting any security-related header without 
>>> making changes in Django. If the setting is more than header/value pairs, 
>>> then it seems this won't be the case.
>>>
>>> On Tuesday, January 19, 2021 at 4:39:54 AM UTC-5 Adam Johnson wrote:
>>>
 I'd imagined the setting would be more like a roll-up of the existing 
 settings, so no need for parsing:

 SECURITY_HEADERS = {
 "REFERRER_POLICY": "same-origin",
 "HSTS": {
 "PRELOAD": True,
 "SECONDS": 1_000_000,
  },
 }


 On Tue, 19 Jan 2021 at 01:30, Tim Graham  wrote:

> The proposal seems to be a setting of the form:
>
> SECURITY_HEADERS = {
> 'Strict-Transport-Security': 
> 'max-age=60; includeSubDomains; preload',
> ...
> }
>
> Currently we have system checks to suggest 
> setting SECURE_HSTS_SECONDS, SECURE_HSTS_PRELOAD, etc. Do you envision 
> trying to keep these checks? It seems it'll be more complicated to parse 
> and validate arbitrary string values instead of individual settings.
>
> It seems some headers may still need special handling in 
> SecurityMiddleware. For example, 'Strict-Transport-Security' is only set 
> if 
> request.is_secure().
> On Friday, August 21, 2020 at 12:53:33 PM UTC-4 Adam Johnson wrote:
>
>> A single SECURITY_HEADERS (or perhaps SECURITY) setting sounds good. 
>> Would love to get CORS into core too.
>>
>> The Opener-Policy ticket has been marked as someday/maybe because the 
>> header is still not widely supported.
>>
>> On Thu, 20 Aug 2020 at 00:02, James Bennett  
>> wrote:
>>
>>> While I think Adam's right that adding one or two new settings
>>> wouldn't be a problem, I do worry about the ongoing proliferation, 
>>> and
>>> it's a thing that I keep wanting to try to find the time to work on
>>> but never actually succeed at.
>>>
>>> Separate from the suggestion of a generic way to add headers on every
>>> response, I've been leaning for a while toward the idea of
>>> consolidating the security-header settings the way we've already
>>> consolidated database and template settings. We can paint the 
>>> bikeshed
>>> whatever color, but suppose for sake of an example nam

Re: Rethink (?) how we handle security headers.

[chris.j...@gmail.com]

On Friday, March 26, 2021 at 1:16:30 AM UTC-7 Adam Johnson wrote:

> That would be counter to how all current dict based settings work, so I 
> think it would be too surprising. 
>

I see. The django.conf.settings object should only include the raw settings 
data and not auxiliary objects that can depend on that data. In that case, 
my suggestion can be taken as an alternative API to a 
"get_security_header()" function. It can be an dict-like object imported 
from a module as opposed to being accessed through django.conf.settings. 
One advantage of a dict-like object over a single function is that has the 
ability to grow helper methods over time in addition to the dict-like 
accesses. That could be useful in different contexts. A precedent for this 
is the django.db.connections object, which is a ConnectionHandler object 
constructed from settings and that supports some dict-like operations.

--Chris
 

>
> On Fri, 26 Mar 2021 at 05:35, chris.j...@gmail.com  
> wrote:
>
>> I just came across this thread after seeing a related PR. Rather than a 
>> "get_security_header()" function, couldn't Django make the setting an 
>> object that is accessed by the user like a dict, but internally (underneath 
>> the hood) it tries from a user-defined dict and then falls back to a value 
>> in a Django-defined dict of defaults?
>>
>> --Chris
>>
>>
>> On Sunday, January 24, 2021 at 1:08:24 PM UTC-8 timog...@gmail.com wrote:
>>
>>> If we go with something like:
>>>
>>> SECURITY_HEADERS = {
>>> "REFERRER_POLICY": "same-origin",
>>> "HSTS": {
>>> "PRELOAD": True,
>>> "SECONDS": 1_000_000,
>>>  },
>>> }
>>>
>>> should we have an empty dictionary as the default or a dictionary with 
>>> defaults for all settings? 
>>>
>>> A drawback of an empty dictionary is that any code that wants to 
>>> retrieve a setting has to do something like 
>>> settings.SECURITY_HEADERS.get('REFERRER_POLICY', '') which 
>>> means the fallback value is duplicated every place the setting is 
>>> consulted.  Maybe a way to mitigate that is to add a 
>>> "get_security_header()" function that retrieves from django.conf.settings 
>>> (user-defined settings) and falls back to some defaults.
>>>
>>> A drawback of a fully-populated default is that overriding it either 
>>> requires users to redefine the entire dictionary or writing code like:
>>>
>>> from django.conf.global_settings import SECURITY_HEADERS
>>> SECURITY_HEADERS['REFERRER_POLICY'] = '...'
>>>
>>> If users redefine the entire dictionary, then adding new 
>>> SECURITY_HEADERS options in future Django versions will require something 
>>> like "get_security_header()" since we won't be able to assume the key is 
>>> there in projects upgrading from older versions.
>>>
>>> I'm not so sure the benefit of consolidating to a single setting is 
>>> worth this additional complexity.
>>> On Tuesday, January 19, 2021 at 8:27:58 AM UTC-5 Adam Johnson wrote:
>>>
>>>> I don't see the need for adding a setting to add headers to outgoing 
>>>> responses. A middleware to do so is a handful of lines:
>>>>
>>>> class SecurityHeadersMiddleware:
>>>> def __init__(self, get_response):
>>>> self.get_response = get_response
>>>>
>>>> def __call__(self, request):
>>>> response = self.get_response(request)
>>>> response["Cross-Origin-Embedder-Policy"] = "require-corp"
>>>> return response
>>>>
>>>> Also there are many non-security related headers one might want to add, 
>>>> so having the ability to add them in a setting called SECURITY_HEADERS 
>>>> could be confusing for maintenance.
>>>>
>>>> On Tue, 19 Jan 2021 at 12:02, Tim Graham  wrote:
>>>>
>>>>> I guess there have been a few different proposals here, but I imagined 
>>>>> SECURITY_HEADERS would allow setting any security-related header without 
>>>>> making changes in Django. If the setting is more than header/value pairs, 
>>>>> then it seems this won't be the case.
>>>>>
>>>>> On Tuesday, January 19, 2021 at 4:39:54 AM UTC-5 Adam Johnson wrote:
>>>>>
>>>>>> I'd imagined the se

Re: APPEND_SLASH behavior

[chris.j...@gmail.com]

On Thursday, May 6, 2021 at 10:32:59 AM UTC-7 f.apo...@gmail.com wrote:

> I took a quick glance (literally just that) at the pull requests. I do 
> like the one that offers a way to abort early inside a prefix -- this is a 
> nice optimization and as well might open interesting options for 
> specialized catch all views. I am not convinced about the backtracking PR, 
> which problems does this solve in reality? What does this mean for features 
> like atomic requests -- it is still just one request after all…
>

I'm the one that filed the "abort early" ticket. I was curious about the 
other one though after reading some of the discussion. With the suggested 
work-around of having a view call other views, would a view be able to 
continue URL resolution in that case? To avoid the problems with the bad 
interaction with the request machinery, another approach that came to mind 
would be to allow inserting / including a function at any point in the 
URLconf. The function would return whether the pattern should be skipped or 
claimed. That would have the advantage of taking place before any view is 
started.

--Chris



>>>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/941a7a20-e5a4-4450-a9e8-1871e2fb1c91n%40googlegroups.com.

template API to list templates

[chris.j...@gmail.com]

Hi,

I recently had a need to iterate over all templates in a code base for a 
development task. I was wondering how people felt about adding a "list all 
templates"-like function to the template backend and loader API's. Such a 
function could also accept a subdirectory to limit the templates returned.

I think such a function could be implemented fairly easily. Currently, I 
don't think it's possible to do so without relying on template API 
internals, and backend and loader-specific logic.

--Chris

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/628b2ea3-c39e-4a14-b17f-21e154cb793fn%40googlegroups.com.

Re: The certificate for code.djangoproject.com expired on 7/4/2021.

[chris.j...@gmail.com]

Maybe someone can file an issue about it to address the same thing 
happening in the future. Perhaps 
here: https://github.com/django/code.djangoproject.com/issues

--Chris

On Sunday, July 4, 2021 at 9:14:20 AM UTC-4 Jason Johns wrote:

> Ahh, I see the issue.
>
> This screenshot is for Summer of Code 2021, at 
> https://code.djangoproject.com/wiki/SummerOfCode2021.  You can see the 
> cert is not valid before 0431 eastern
>
> [image: Screen Shot 2021-07-04 at 9.10.01 AM.png]
>
> This screenshot is for https://docs.djangoproject.com/en/3.2/:
>
> [image: Screen Shot 2021-07-04 at 9.09.14 AM.png]
>
> Looks like the SoC URL cert didn't get applied before this morning, and 
> users who tried to access that URL would have gotten Certificate Expired 
> errors.
> On Sunday, July 4, 2021 at 9:07:55 AM UTC-4 Jason Johns wrote:
>
>> FWIW, a user at pyslackers said they got a 503 about the same time about 
>> 50 minutes after Asif's initial message.  Then about two hours later (about 
>> an hour before Adam's reply), someone else responded that they weren't able 
>> to replicate.  Maybe there was a delay in applying the cert rotation?  I'm 
>> not able to easily see when the cert was generated.
>>
>> On Sunday, July 4, 2021 at 5:38:01 AM UTC-4 Adam Johnson wrote:
>>
>>> Hi Asif
>>>
>>> I'm not seeing a problem - the certificate I get expires in October.
>>>
>>> [image: Screenshot 2021-07-04 at 10.36.15.png]
>>>
>>> Thanks,
>>>
>>> Adam
>>>
>>> On Sun, 4 Jul 2021 at 06:34, Asif Saif Uddin  wrote:
>>>

 Websites prove their identity via certificates, which are valid for a 
 set time period. The certificate for code.djangoproject.com expired on 
 7/4/2021.
  
 Error code: SEC_ERROR_EXPIRED_CERTIFICATE
  
 View Certificate

 https://code.djangoproject.com/wiki/SummerOfCode2021

 Asif

 -- 
 You received this message because you are subscribed to the Google 
 Groups "Django developers (Contributions to Django itself)" group.
 To unsubscribe from this group and stop receiving emails from it, send 
 an email to django-develop...@googlegroups.com.
 To view this discussion on the web visit 
 https://groups.google.com/d/msgid/django-developers/539fc003-f4de-4dae-8ace-d5103c19f44en%40googlegroups.com
  
 
 .

>>>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/4ab92cf4-04e0-4e30-a088-ec13ce3bc3d9n%40googlegroups.com.

Re: Recognising Contributions

[chris.j...@gmail.com]

Related to this discussion, what's the current policy / practice around 
linking to tickets from the release notes? It looks the release notes for 
patch releases already link to tickets, e.g.
https://docs.djangoproject.com/en/dev/releases/3.2.6/

However, for feature release, it looks like the tickets aren't linked to, 
e.g.
https://docs.djangoproject.com/en/dev/releases/3.2/
https://docs.djangoproject.com/en/dev/releases/4.0/

Is the difference deliberate? Is it currently being done manually for patch 
releases -- is that why it's being done / can only be done there?

--Chris


On Wednesday, August 18, 2021 at 3:37:21 AM UTC-4 carlton...@gmail.com 
wrote:

> Hey David. 
>
> Thanks for the follow-up here. 
>
> I think at least for 4.0 we should focus on adding callouts/recognition to 
> the release blog post, rather than the release notes. 
>
> For one, release notes only contain new features, and we want to call out 
> all the different contributions that don’t fall under that. The bug fixes, 
> the docs changes, the translations teams, triage and review folks, and so 
> on… — there’s much more than just new features. It would be nice to call 
> out the new contributors (and the old too, but especially the new) — I 
> think we can (and have time) to pull together something good in the more 
> discursive format the release announcement allows. 
>
> For another, more narrowly practical point, I’d like a test-bed before we 
> change the release note process, which is part of our daily workflow. Doing 
> it in the release announcement is a one-shot opportunity to try ideas and 
> see how it goes. If there are elements we then want to bring into the 
> release notes going forward then that’s open. (But softly, softly… being 
> the thought.) 
>
> Kind Regards,
>
> Carlton
>
>
> On 18 Aug 2021, at 09:20, smi...@gmail.com  wrote:
>
> Hi All,
>
> Just coming back to this again (time flies), although we've got a while 
> until 4.0 is released so no rush here.
>
> I've got a few different thoughts here:
>
> *Data*
>
> I had a look at the various tools discussed above to see if any give us 
> what we need. While on that journey I came accross git's `shortlog`. 
> Apparently git use this to create their release notes, there's a few 
> options here but something like the following will give the list of 
> contributors (and number of commits) since 3.2.
>
> What I like about this is it "coalesce together commits by the same 
> person". The other tools above would have resulted in a number of different 
> entries for many folk as they use different email addresses (or their 
> GitHub "email" gets used). 
>
> $ git shortlog 3.2..main -n -s --group=author 
> --group=trailer:co-authored-by
>
> I'm not quite sure how git then get to new and returning users, but 
> presumably that could be a fairly short script to work out the new names 
> since 3.2. 
>
> *Release Notes*
>
> Adding names and tickets seemed to receive a positive response earlier in 
> the year. So the question here, is what format? 
>
> Python uses a couple of different formats
> 1)  :  : Patch by 
> 2)  (Contributed by  in )
>
> I think the main thing is to choose something, does anyone have a 
> preference, either one of the above styles or something different? (My vote 
> would be for the second option above)
>
> [1] https://docs.python.org/3/whatsnew/changelog.html
> [2] https://docs.python.org/3/whatsnew/3.8.html
>
> *Events*
>
> Responding to Tom's point above, I think we'd want to reach out to one of 
> the organisers of an event to see if folk would be interested in exploring 
> this further. It's far more complex than "just" adding some text to a 
> release note. However, given that it's a gathering of Django folk it seems 
> like an opportunity to do _something_. 
>
> While I've seen what recognition can look like at corporate events those 
> tend to rely on being Exclusive (think VIP areas et al), rather than 
> Inclusive. I suspect for a Django conference/event we'd want to do 
> something quite different. 
>
> Kind Regards
>
> David
> On Wednesday, 7 July 2021 at 21:52:14 UTC+1 t...@carrick.eu wrote:
>
>> This is something I've been thinking about a bit as well.
>>
>> Mostly I think adding authors to the release notes is probably the best 
>> bang for buck in terms of recognition. This is what I was mostly thinking 
>> about myself. The release notes are (I believe) very widely read, 
>> especially in comparison with anything on GitHub.
>>
>> The other suggestions are, I think, good and worthwhile, but probably not 
>> as impactful.
>>
>> I am interested / curious about your last point. I think adding some 
>> recognition to in person events might be nice, but I'm not sure what it 
>> would look like in practice.
>>
>> Tom
>>
>> On Mon, 5 Jul 2021 at 17:38, 'Adam Johnson' via Django developers 
>> (Contributions to Django itself)  wrote:
>>
>>> I'm all for exposing names in more places.
>>>
>>> Linking through to PR's from the release notes would

Re: Recognising Contributions

[chris.j...@gmail.com]

Just to expand on my question a little bit with a comment, I think it would 
help to make the ticket corresponding to a release note easier to find 
because it would give people an easier way to answer the question, "what is 
the reasoning / background / discussion for this change?" Currently, there 
isn't really a convenient way to do this. One method that comes to mind is 
using git-blame on the source file corresponding to the release notes, and 
then clicking through to the commit, then PR, then ticket, but that method 
is roundabout and requiring several steps. Another might be searching the 
tracker for keywords, but that method can be hit-or-miss. To address the 
readability / "beauty" concern, maybe there's an even less obtrusive way to 
link to the ticket than including the ticket number after the sentence in 
parentheses (e.g. like adding a symbol after the final period that acts a 
hyperlink and/or making the hyperlink appear only on hover).

However, this is separate from the recognition issue, so perhaps off-topic 
for this thread.

--Chris


On Wednesday, August 18, 2021 at 3:22:17 PM UTC-4 timog...@gmail.com wrote:

> I started including ticket numbers in patch releases because there aren't 
> too many changes there and every change has a release note. I think the 
> ticket numbers are useful to quickly identify the cause of some change or 
> regression in a patch release where upgrades should generally be safer and 
> therefore may have less testing than with a major release upgrade.
>
> I think linking to tickets and/or mentioning authors in major releases 
> notes would detract from readability (but maybe just beauty), and of 
> course, not every commit (like bug fixes) in major releases requires a 
> release note.
>
> As an infrequent contributor these days, I'm not seeking any more 
> recognition from my contributions, nor would more recognition encourage me 
> to contribute more. I'd rather the Django team spend their efforts on 
> building software than on publicity.
> On Wednesday, August 18, 2021 at 2:20:25 PM UTC-4 chris.j...@gmail.com 
> wrote:
>
>> Related to this discussion, what's the current policy / practice around 
>> linking to tickets from the release notes? It looks the release notes for 
>> patch releases already link to tickets, e.g.
>> https://docs.djangoproject.com/en/dev/releases/3.2.6/
>>
>> However, for feature release, it looks like the tickets aren't linked to, 
>> e.g.
>> https://docs.djangoproject.com/en/dev/releases/3.2/
>> https://docs.djangoproject.com/en/dev/releases/4.0/
>>
>> Is the difference deliberate? Is it currently being done manually for 
>> patch releases -- is that why it's being done / can only be done there?
>>
>> --Chris
>>
>>
>> On Wednesday, August 18, 2021 at 3:37:21 AM UTC-4 carlton...@gmail.com 
>> wrote:
>>
>>> Hey David. 
>>>
>>> Thanks for the follow-up here. 
>>>
>>> I think at least for 4.0 we should focus on adding callouts/recognition 
>>> to the release blog post, rather than the release notes. 
>>>
>>> For one, release notes only contain new features, and we want to call 
>>> out all the different contributions that don’t fall under that. The bug 
>>> fixes, the docs changes, the translations teams, triage and review folks, 
>>> and so on… — there’s much more than just new features. It would be nice to 
>>> call out the new contributors (and the old too, but especially the new) — I 
>>> think we can (and have time) to pull together something good in the more 
>>> discursive format the release announcement allows. 
>>>
>>> For another, more narrowly practical point, I’d like a test-bed before 
>>> we change the release note process, which is part of our daily workflow. 
>>> Doing it in the release announcement is a one-shot opportunity to try ideas 
>>> and see how it goes. If there are elements we then want to bring into the 
>>> release notes going forward then that’s open. (But softly, softly… being 
>>> the thought.) 
>>>
>>> Kind Regards,
>>>
>>> Carlton
>>>
>>>
>>> On 18 Aug 2021, at 09:20, smi...@gmail.com  wrote:
>>>
>>> Hi All,
>>>
>>> Just coming back to this again (time flies), although we've got a while 
>>> until 4.0 is released so no rush here.
>>>
>>> I've got a few different thoughts here:
>>>
>>> *Data*
>>>
>>> I had a look at the various tools discussed above to see if any give us 
>>> what we need. While on that journey I