Re: Accessibility standards & contribution guidelines for Django

[Carlton Gibson]

Hi Thibaud. 

Thanks for this, and for your efforts here generally! 🥇

> ...a long-overdue follow up...

This is the only bit I can't agree with 😀 — ticking along, 
slow-and-steady, is realistic/sustainable/desirable/... 🙂

You mention a lot of things, so that'll take a little while to absorb — 
links to click, things to read. (If you have stronger opinions on "We 
should do X, using tool Y" then that's worth putting forward.)

The thought that comes up is, what can we do on the documentation front? We 
have guides/how-tos on security, testing, and so on. Could we add similar 
for accessibility? 

* These are the standards to follow
* Here's some of the basics of making sure your Django site is accessible. 
* Here are some tools you can use to test/ensure that. 

(Lots of this is in your email here.) 

I've read various articles on this over the years, but don't feel confident 
on it at all. I think a trusted guide in the Django docs would be an 
awesome resource. (As a Django user I want a guide in the docs so that I 
can make my site accessible from the start.) 

That doesn't directly address your specific questions. 

Kind Regards,

Carlton



-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/7d5db2ae-5a53-4649-838c-72cf42a122d4n%40googlegroups.com.

Re: Removal of USE_L10N setting

[Carlton Gibson]

Hey Claude, thanks for this. 

This seems like a win to me... a nice clean-up. 

I have to say, I never once set `USE_L10N = False` — I don't imagine many
people do, but my only concern would be whole swaths of folks that depend on
this… — do such exist in numbers? 🤔

This comments seemed pertinent: 

> Le dimanche 13 juin 2021 à 01:09:26 UTC+2, re...@net a écrit :
> > I have run into bugs because template authors forgot to unlocalize 
> > values, in particular database IDs. These bugs often go unnoticed for 
> > quite a while because they only show up once you reach 1000 database 
> > rows. Forgetting to localize a value would typically be a much more 
> > obvious bug. 
> 
> I understand this concern. I wonder if that could be solved separately, 
for example by:
>  - returning int subclasses from AutoField's to_python (a bit like 
str/SafeString for handling escaping)
>  - not localizing those integers, unless use_l10n is forced (True)
> Does this look like a plan (even without touching USE_L10N)?

So we escape HTML by default, because that's the right thing to do. 

Same here, we already localize by default (in reality, because the project 
template has been that way ≈forever), again because it's the right 
behaviour. 

We `unlocalize` where that's not helpful. (But maybe there could be nicer 
API 
there.) 

Here we're just removing a whole load of codepaths for handling a unusual 
use-case. (Again, assuming it's as minority as we think). 

>From the PR: 

> ...if needed, you can simulate the
> old ``USE_L10N = False`` behavior by defining those format settings in 
your
> settings file. 
> - docs/releases/4.0.txt

Maybe we could give an example there (🤔) but there's a fallback. 

So I think +1. 

Kind Regards,

Carlton

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/97b2de49-bced-469c-9f1c-2af736be9e79n%40googlegroups.com.

Re: A base compression middleware class

[Tom Carrick]

It's also worth noting that the security issue mentioned in the docs[1]
makes it unsuitable for many (most?) Django projects, unfortunately, and
brotli is also susceptible to this attack.

It's probably not something I'd be keen on encouraging the use of, though I
also think the idea in principle is a good one.

[1]
https://docs.djangoproject.com/en/3.2/ref/middleware/#django.middleware.gzip.GZipMiddleware

Tom

On Tue, 15 Jun 2021 at 02:16, Curtis Maloney  wrote:

> Hi Illia,
>
> I like the idea here, and your design looks sensible at first blush, but I
> feel conflicted.
>
> As much as I like the idea of allowing more flexibility and
> future-proofing this middleware,
> AIUI the gzip middleware is generally discouraged, as it's typically more
> efficient to have your web server do this work.
>
> I'd be more tempted to deprecate GzipMiddleware in core, and move this
> feature into a 3rd party app.
>
> An additional hook for your interface: a function to test if this
> compressor should be used on this response. For instance, the gzip
> middleware has a minimum size below which it's deemed not worth the effort
> to try to compress.
>
> --
> Curtis
>
>
> On Tue, 15 Jun 2021, at 04:49, Illia Volochii wrote:
>
> Hi all,
>
> There is `GZipMiddleware` that compresses response bodies using gzip.
> https://github.com/django/django/blob/main/django/middleware/gzip.py
>
> But there are other algorithms supported by browsers (e.g., Brotli).
> At the moment, if somebody wants to add support for any of them in a
> Django project, one has to create a new middleware class redefining
> all the logic processing responses.
>
> I propose simplifying the process by creating
> `BaseCompressionMiddleware`. Its subclasses would just need to define
> a tuple of supported compressors.
>
> The tuple elements could be instances of such a data class:
> ```
> @dataclass
> class Compressor:
> name: str
> compress_string_func: Callable[[bytes], bytes]
> compress_sequence_func: Optional[
> Callable[[Iterable[bytes]], Iterator[bytes]]
> ] = None
>
> def is_accepted_by(self, accept_encoding):
> return bool(re.search(rf'\b{self.name}\b', accept_encoding))
> ```
>
>
> And the class could select a compressor using such code:
> ```
> @classmethod
> def select_compressor(cls, request, response):
> ae = request.META.get('HTTP_ACCEPT_ENCODING', '')
> for compressor in cls.compressors:
> if (
> compressor.is_accepted_by(ae)
> and (not response.streaming or
> compressor.compress_sequence_func)
> ):
> return compressor
> ```
>
>
> Note, in the example `compress_string_func` is required and
> `compress_sequence_func` is optional. I think that
> `compress_string_func` would usually be a library function, and
> `compress_sequence_func` would require some custom logic. And we
> should not force users to define the latter.
>
> `GZipMiddleware` could be changed to
> ```
> gzip_compressor = Compressor(
> name='gzip',
> compress_string_func=compress_string,
> compress_sequence_func=compress_sequence,
> )
>
> class GZipMiddleware(BaseCompressionMiddleware):
> compressors = (gzip_compressor,)
> ```
>
>
> And someone wanting to support both Brotli and gzip could define:
> ```
> import brotli
> from django.middleware.gzip import gzip_compressor
>
> brotli_compressor = Compressor(
> name='br',
> compress_string_func=brotli.compress,
> )
>
> class CompressionMiddleware(BaseCompressionMiddleware)
> compressors = (brotli_compressor, gzip_compressor)
> ```
>
>
> Please let me know what you think about such an enhancement.
> I will be happy to create a pull request if this is accepted.
>
> Thanks,
> Illia
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/49535d7e-758c-4acf-a9da-831ccfa1d17bn%40googlegroups.com
> 
> .
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/c4cef215-7cae-4c6b-a9b2-7b4785b3a9bf%40www.fastmail.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Djan

Re: Accessibility standards & contribution guidelines for Django

[Sarah A]

Hi folks,

I'm interested in accessibility lately, I discovered a tool that can 
probably be added to the list: pa11y . It was used for 
automated tests.
I don't have enough experience to give my opinion but @Thibaud you might 
have an opinion on it.

I wanted to check the issue on password UX but maybe I can help on others 
things related to this before. Tell me what do you think.

Best regards,

Sarah

Le mardi 15 juin 2021 à 10:44:07 UTC+2, carlton...@gmail.com a écrit :

> Hi Thibaud. 
>
> Thanks for this, and for your efforts here generally! 🥇
>
> > ...a long-overdue follow up...
>
> This is the only bit I can't agree with 😀 — ticking along, 
> slow-and-steady, is realistic/sustainable/desirable/... 🙂
>
> You mention a lot of things, so that'll take a little while to absorb — 
> links to click, things to read. (If you have stronger opinions on "We 
> should do X, using tool Y" then that's worth putting forward.)
>
> The thought that comes up is, what can we do on the documentation front? 
> We have guides/how-tos on security, testing, and so on. Could we add 
> similar for accessibility? 
>
> * These are the standards to follow
> * Here's some of the basics of making sure your Django site is accessible. 
> * Here are some tools you can use to test/ensure that. 
>
> (Lots of this is in your email here.) 
>
> I've read various articles on this over the years, but don't feel 
> confident on it at all. I think a trusted guide in the Django docs would be 
> an awesome resource. (As a Django user I want a guide in the docs so that I 
> can make my site accessible from the start.) 
>
> That doesn't directly address your specific questions. 
>
> Kind Regards,
>
> Carlton
>
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/67e57058-7c97-4e77-876a-513b498703b9n%40googlegroups.com.

Re: Accessibility standards & contribution guidelines for Django

[Tobias Bengfort]

Hi Thibaud,

thanks for the follow up. +1 on basically everything you wrote!

On 15/06/2021 02.59, Thibaud Colas wrote:
This distinction between "sites built with Django" and the "Django 
admin" highlights an important point, which is that for lots of sites 
the Django admin would be what accessibility standards call an authoring 
tool. There is a separate standard for authoring tools that builds upon 
WCAG 2.1: ATAG 2.0. I think we should also explicitly aim for compliance 
with ATAG 2.0 wherever possible, as it has a lot of useful guidelines we 
could follow for the Django admin. For Django itself, there might be 
very little to do to implement those guidelines, but for the wider 
ecosystem of CMS-like features in Django it would make a clear 
difference if Django was clearly defined as an authoring tool. WCAG 
itself is a very thin baseline as standards go, so anything that goes 
beyond it is well worth considering in my opinion.


I am not completely sure about the definition of an "authoring tool", 
but it might make sense to consider django as a whole as such an 
authoring tool, not just the Django admin. ATAG A. (Make the authoring 
tool user interface accessible) would then only apply to Django admin, 
but ATAG B. (Support the production of accessible content) would apply 
to everything.



  * *Which automated or semi-automated testing tools to use when testing
changes to Django*. My go-to is anything Axe
-based, as it’s ubiquitous, and has very
few false positives. In particular with the Accessibility Insights
 browser extension.


AFAIK the accessibility checker in chrome's lighthouse is also based on 
Axe. As it is already built into a very popular browser this might lower 
the threshold for testing even further.


tobias

--
You received this message because you are subscribed to the Google Groups "Django 
developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/7756471f-bfd0-42c4-3811-d355d5d6d0d4%40posteo.de.

Re: `Model.validate_unique` excluding partial unique constraint

[charettes]

FWIW I thought I'd give a timeboxed shot at 2. to make sure I don't send 
you towards a deep rabbit hole and it seems pretty straightforward!

https://github.com/django/django/compare/main...charettes:query-empty-model

Le lundi 14 juin 2021 à 03:09:35 UTC-4, gaga...@gmail.com a écrit :

> Thanks, it clears things a lot.
>
> I'll try my hand at it when I'll have some more time available.
>
> Le jeudi 10 juin 2021 à 06:00:17 UTC+2, charettes a écrit :
>
>> Alright so here's for a few hints about I believe things should be done.
>>
>> First things first Lookup must be made a subclass of Expression which is 
>> being worked on[0].
>>
>> Ideally Q would also be made a subclass of Expression but that's likely a 
>> big can of worms so I'd focus on implementing it for Q only at first.
>>
>> Now for the compiler part. Things are bit tricky here as these 
>> expressions are not going to be bound to a model/table and most of the 
>> sql.Query and resolve_expression machinery revolves around the availability 
>> of a Query.model: models.Model property. I think there's two solutions here:
>>
>> 1. Adapt sql.Query so it can be *unbounded* meaning that it's .model 
>> property type would change from models.Model to Optional[models.Model]. 
>> 2. Follow the sql.RawQuery route and define a new sql.UnboundQuery class 
>> that *looks* like a Query but doesn't allow any form of column references 
>> or JOINs or filters (WHERE).
>>
>> In both cases the Query like object should prevent any form of column 
>> references and JOINs with a comprehensible error messages (e.g. in 
>> resolve_ref 
>> and setup_join if go with 1.). I have a feeling 2. is easier to 
>> implement but 1. seems like it could be a much more rewarding experience 
>> for you and the community as you'll have to special case a couple of long 
>> lived assumptions in django.db.models.sql.
>>
>> Depending on whether you choose the 1. or 2. you'll have to implement a 
>> way for database backends to specify how to *wrap* the provided expression 
>> in a SELECT statement. Most databases won't require any special casing but 
>> I know that Oracle will require the addition of a trailing "DUAL" clause 
>> (SELECT ... FROM DUAL)[1] and possibly some special casing of expressions 
>> such as exists but there's already pre-existing logic for that[2]. If you 
>> go with 1. this can be done by returning a backend specific string in 
>> SQLCompiler.get_from_clause when self.query.alias_map is empty[3].
>>
>> In the end the call stack should be (assuming 1. is followed):
>>
>> Q.check(self, using):
>>   query = Query()
>>   query.add_annotations(self, '_check')
>>   query.set_values('_check')
>>   compiler = query.get_compiler(using=db)
>>   result = compiler.execute_sql(SINGLE)
>>   return bool(result[0])
>>
>> I hope this clears things up a bit!
>>
>> Cheers,
>> Simon
>>
>> [0] https://github.com/django/django/pull/14494
>> [1] https://docs.oracle.com/cd/B19306_01/server.102/b14200/queries009.htm
>> [2] 
>> https://github.com/django/django/blob/ed3af3ff4b3cfb72de598f1b39a1028ba3826efb/django/db/models/expressions.py#L382-L389
>> [3] 
>> https://github.com/django/django/blob/ed3af3ff4b3cfb72de598f1b39a1028ba3826efb/django/db/models/sql/compiler.py#L797-L810
>>
>> Le mercredi 2 juin 2021 à 11:36:02 UTC-4, gaga...@gmail.com a écrit :
>>
>>> Thanks for the thorough answer. I also realize now that it worked in my 
>>> app only because of another side effect when my instance was saved..
>>>
>>> I started to take a look at the ORM part where the check method should 
>>> be implemented as I'm not used to it. Shouldn't .check() be implemented on 
>>> Q and not on Expression? Or are you including Lookup / Q in it?
>>>
>>> Then I'd guess it's just a matter of calling as_sql() from each part and 
>>> assemble them. Everythings we need seems to be done in Query and we can't 
>>> use it as it has to be linked to a model, so we would have to redo it? 
>>> Although as_sql needs a compiler which itself needs a query. I admit I'm a 
>>> bit lost in all those classes, everything seems to be too much linked to 
>>> the models to do something without one.
>>>
>>> If you have any more hints as to where I should look, thanks again.
>>> Le mercredi 2 juin 2021 à 00:33:12 UTC+2, charettes a écrit :
>>>
 Hello there,

 Partial unique constraints are currently not supported during 
 validation for reasons described in this ticket[0].

 For example (inspired by this Github comment[1]), if you define the 
 following model

 class Article(models.Model):
 slug = models.CharField(max_length=100)
 deleted_at = models.DateTimeField(null=True)

 class Meta:
 constraints = [
 UniqueConstraint('slug', condition=Q(deleted_at=None), 
 name='unique_slug'),
 ]

 Then validate_unique must perform the following query to determine if 
 the constraint is violated

 SELECT NOT (%(deleted_at

Re: `Model.validate_unique` excluding partial unique constraint

[charettes]

I meant 1. in my previous email where sql.Query.model is allowed to be 
None. The tests happen to pass on SQLite, MySQL, and Postgres.

Le mardi 15 juin 2021 à 20:02:28 UTC-4, charettes a écrit :

> FWIW I thought I'd give a timeboxed shot at 2. to make sure I don't send 
> you towards a deep rabbit hole and it seems pretty straightforward!
>
> https://github.com/django/django/compare/main...charettes:query-empty-model
>
> Le lundi 14 juin 2021 à 03:09:35 UTC-4, gaga...@gmail.com a écrit :
>
>> Thanks, it clears things a lot.
>>
>> I'll try my hand at it when I'll have some more time available.
>>
>> Le jeudi 10 juin 2021 à 06:00:17 UTC+2, charettes a écrit :
>>
>>> Alright so here's for a few hints about I believe things should be done.
>>>
>>> First things first Lookup must be made a subclass of Expression which is 
>>> being worked on[0].
>>>
>>> Ideally Q would also be made a subclass of Expression but that's likely 
>>> a big can of worms so I'd focus on implementing it for Q only at first.
>>>
>>> Now for the compiler part. Things are bit tricky here as these 
>>> expressions are not going to be bound to a model/table and most of the 
>>> sql.Query and resolve_expression machinery revolves around the availability 
>>> of a Query.model: models.Model property. I think there's two solutions here:
>>>
>>> 1. Adapt sql.Query so it can be *unbounded* meaning that it's .model 
>>> property type would change from models.Model to Optional[models.Model]. 
>>> 2. Follow the sql.RawQuery route and define a new sql.UnboundQuery class 
>>> that *looks* like a Query but doesn't allow any form of column references 
>>> or JOINs or filters (WHERE).
>>>
>>> In both cases the Query like object should prevent any form of column 
>>> references and JOINs with a comprehensible error messages (e.g. in 
>>> resolve_ref 
>>> and setup_join if go with 1.). I have a feeling 2. is easier to 
>>> implement but 1. seems like it could be a much more rewarding experience 
>>> for you and the community as you'll have to special case a couple of long 
>>> lived assumptions in django.db.models.sql.
>>>
>>> Depending on whether you choose the 1. or 2. you'll have to implement a 
>>> way for database backends to specify how to *wrap* the provided expression 
>>> in a SELECT statement. Most databases won't require any special casing but 
>>> I know that Oracle will require the addition of a trailing "DUAL" clause 
>>> (SELECT ... FROM DUAL)[1] and possibly some special casing of expressions 
>>> such as exists but there's already pre-existing logic for that[2]. If you 
>>> go with 1. this can be done by returning a backend specific string in 
>>> SQLCompiler.get_from_clause when self.query.alias_map is empty[3].
>>>
>>> In the end the call stack should be (assuming 1. is followed):
>>>
>>> Q.check(self, using):
>>>   query = Query()
>>>   query.add_annotations(self, '_check')
>>>   query.set_values('_check')
>>>   compiler = query.get_compiler(using=db)
>>>   result = compiler.execute_sql(SINGLE)
>>>   return bool(result[0])
>>>
>>> I hope this clears things up a bit!
>>>
>>> Cheers,
>>> Simon
>>>
>>> [0] https://github.com/django/django/pull/14494
>>> [1] 
>>> https://docs.oracle.com/cd/B19306_01/server.102/b14200/queries009.htm
>>> [2] 
>>> https://github.com/django/django/blob/ed3af3ff4b3cfb72de598f1b39a1028ba3826efb/django/db/models/expressions.py#L382-L389
>>> [3] 
>>> https://github.com/django/django/blob/ed3af3ff4b3cfb72de598f1b39a1028ba3826efb/django/db/models/sql/compiler.py#L797-L810
>>>
>>> Le mercredi 2 juin 2021 à 11:36:02 UTC-4, gaga...@gmail.com a écrit :
>>>
 Thanks for the thorough answer. I also realize now that it worked in my 
 app only because of another side effect when my instance was saved..

 I started to take a look at the ORM part where the check method should 
 be implemented as I'm not used to it. Shouldn't .check() be implemented on 
 Q and not on Expression? Or are you including Lookup / Q in it?

 Then I'd guess it's just a matter of calling as_sql() from each part 
 and assemble them. Everythings we need seems to be done in Query and we 
 can't use it as it has to be linked to a model, so we would have to redo 
 it? Although as_sql needs a compiler which itself needs a query. I admit 
 I'm a bit lost in all those classes, everything seems to be too much 
 linked 
 to the models to do something without one.

 If you have any more hints as to where I should look, thanks again.
 Le mercredi 2 juin 2021 à 00:33:12 UTC+2, charettes a écrit :

> Hello there,
>
> Partial unique constraints are currently not supported during 
> validation for reasons described in this ticket[0].
>
> For example (inspired by this Github comment[1]), if you define the 
> following model
>
> class Article(models.Model):
> slug = models.CharField(max_length=100)
> deleted_at = models.DateTimeField(null=True)
>>>