Re: Fellow Reports - November 2019

[Mariusz Felisiak]

Week ending November 17, 2019.

*Triaged:*
https://code.djangoproject.com/ticket/30968 - Use github actions for 
linting. (duplicate)
https://code.djangoproject.com/ticket/30970 - Waiting for enhancement in 
"inspectdb" command. (duplicate)
https://code.djangoproject.com/ticket/30972 - USERNAME_FIELD 
UniqueConstraint option (duplicate)
https://code.djangoproject.com/ticket/30977 - Moved site variables 
assignment outside of the loop in PasswordResetForm.save(). (accepted)
https://code.djangoproject.com/ticket/30966 - Migration crashes due to 
foreign key issue, depending on otherwise irrelevant order on MySQL. 
(accepted)
https://code.djangoproject.com/ticket/30978 - Missing configuration step 
with custom filters. (invalid)
https://code.djangoproject.com/ticket/30976 - ManyToManyField doesn't JOIN 
when target object has a ForeignKey with the same name as a default query 
name. (invalid)
https://code.djangoproject.com/ticket/30979 - Unreliable variable value 
(invalid)
https://code.djangoproject.com/ticket/30982 - django.db.backends.postgresql 
is not one of available backends. (needsinfo)
https://code.djangoproject.com/ticket/30981 - Admin changelist crashes when 
using F() and OrderBy() expressions in the ModelAdmin's admin_order_field. 
(accepted)
https://code.djangoproject.com/ticket/30980 - admin.E130 should specify 
which actions' names were duplicated. (accepted)
https://code.djangoproject.com/ticket/23433 - Django installs 
/usr/bin/django-admin and /usr/bin/django-admin.py (accepted)
https://code.djangoproject.com/ticket/30985 - Error when adding a group in 
admin - "The database backend does not accept 0 as a value for AutoField". 
(worksforme)
https://code.djangoproject.com/ticket/30984 - Cache culling. (invalid)
https://code.djangoproject.com/ticket/30986 - Using conditional RawSQL's in 
QuerySet.filter() crashes on Oracle. (created)
https://code.djangoproject.com/ticket/30987 - Add PositiveBigIntegerField. 
(accepted)

*Reviewed/committed:*
https://github.com/django/django/pull/12054 - Fixed #30958 -- Used a 
clearer example in the Cast() docs.
https://github.com/django/django/pull/11853 - Fixed #30977 -- Optimized 
PasswordResetForm.save() a bit.
https://github.com/django/django/pull/11912 - Fixed #30252 -- Clarified 
need to reopen forms.ImageField.image file to access raw image data.
https://github.com/django/django/pull/12048 - Fixed #30967 -- Fixed 
TrigramTest failures on PostgreSQL 12+.
https://github.com/django/django/pull/11886 - Fixed #30405 -- Fixed source 
code mismatch crash in ExceptionReporter.
https://github.com/django/django/pull/11948 - Fixed #30828 -- Added how to 
remove/insert many-to-many relations in bulk to the database optimization 
docs.
https://github.com/django/django/pull/12050 - Fixed #30971 -- Prevented 
Query.resolve_lookup_value() from coercing list values to tuples.
https://github.com/django/django/pull/12030 - Fixed #29916 -- Added 
lower_inc, lower_inf, upper_inc, and upper_inf lookups for RangeFields.
https://github.com/django/django/pull/12049 - Fixed #29808 -- Fixed initial 
migration detection when identifiers are case-insensitive.

*Authored:*
https://github.com/django/django/pull/12058 - [2.2.x] Added Python 3.8 
compatibility in Django 2.2.x.
https://github.com/django/django/pull/12059 - Doc'd Python 3.8 
compatibility in Django 2.2.x.
https://github.com/django/django/pull/12063 - Fixed random 
auth_tests.test_tokens.TokenGeneratorTest.test_10265 failures.
https://github.com/django/django/pull/12068 - Fixed #30986 -- Fixed 
queryset crash when filtering against boolean RawSQL expressions on Oracle.

Best regards,
Mariusz

>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/03be96bd-a28e-4a6a-ae4a-bec00edfd978%40googlegroups.com.

Re: Deprecate HttpRequest.is_ajax

[Jure Erznožnik]

Sorry for barging in like this, but this is actually a problem I have 
been dealing with quite a bit lately, so:


In my work I very often have to decide, depending on what's calling, 
what the rendered output might be. Ultimately I went with DRF and its 
content negotiation, though even that one - as implemented - sometimes 
isn't sufficient.


See, the problem sometimes isn't that you request JSON and then get 
JSON, request HTML and get HTML. You also have to cater for exceptions. 
Maybe a 4xx would return additional objects to insert into the DOM while 
a 200 would be fine with a JSON or even without data. What about 500?


I'm currently handling this with custom headers and the caller (the 
browser) tells the server what kind of outputs it can handle in 
different types of output.


The server then performs the branching at certain code points, 
specifically the ones mentioned above. DRF allows me to choose the 
appropriate renderer. Though I should mention here, that the data is 
already serialised at that point: sometimes this creates issues for 
renderers that might desire more information to do their work. Just 
mentioning that render stages need to be accounted for too. This may not 
be a problem for core Django as it doesn't have stages.


Again, sorry, but still hoping this helped in some way.

LP,
Jure


On 19/11/2019 01:06, Matemática A3K wrote:


I agree with Adam that it should be deprecated with no replacement.

The content negotiation is something that should be in but not as a 
replacement of it, as a general improvement.


I think there shouldn't be a replacement because "is_ajax" asks 
whether it came from a ""regular"" browser instead of jQuery and with 
the content negotiation you ask if the requester accepts a type - 
which can also lead to errors because the client may also accept other 
types (no example coming to my mind), and if so, it will lead to 
undesired behavior.


The right approach would be making AJAX requests request JSON output 
explicitly, by using a dedicated endpoint or by appending something 
that manifests their intention - like in 
https://docs.djangoproject.com/en/2.2/topics/class-based-views/mixins/#more-than-just-html 
is done with a get parameter. Not decide the response type by where it 
came from as it is unreliable as stated before, it provides 
convenience in some use cases but can lead to errors.


Seems better to me to refactor the view code so you can write a 
different view for Ajax requests that returns a JSON without code 
duplication.


As a shortcut, something like "For simple AJAX endpoints wrap your 
view with (something like) a "jsonview" decorator which will check the 
accept header (with something like Claude's code), return the 
appropriate error code if not, set the response type accordingly, and 
you should return a dict of strings (you have to take care of the 
serialization, i.e with 
https://docs.djangoproject.com/en/2.2/topics/serialization/#serialization-formats-json). 



On Mon, Nov 18, 2019 at 3:28 PM Tom Forbes > wrote:


What I meant by that is it’s not an approach that scales well to
lots of views. It might be better to have separate endpoints to
return JSON (e.g adding a /json suffix), and in the past this has
made services I’ve worked on a lot more maintainable and easy to
understand. But it’s not as quick to do as `if request.is_ajax()`
and requires a bit more upfront work. If you find you need to do
this a lot then maybe something more structured like Django Rest
Framework will be a better choice, which also handles content
negotiation really well (it can produce XML, CSV, JSON, etc etc).



On 18 Nov 2019, at 15:18, Matthew Pava mailto:matthew.p...@iss.com>> wrote:

“In my opinion there are not many good reasons to have to change
behaviour if a request is made via XHR. I think the most common
usage is to have a single view that returns a JSON response or a
HTML response depending on if XHR is used
(https://github.com/search?l=Python&q=request.is_ajax&type=Code),
which isn’t great and isn’t reliable.”
I do this. What would the best way to handle this? Perhaps the
proper practice should be documented when it is deprecated?
*From:*django-developers@googlegroups.com

[mailto:django-developers@googlegroups.com]*On Behalf Of*Tom Forbes
*Sent:*Saturday, November 16, 2019 10:16 AM
*To:*django-developers@googlegroups.com

*Subject:*Re: Deprecate HttpRequest.is_ajax
I would agree. Flask has done the same:
|DeprecationWarning: Request.is_xhr is deprecated. Given that the
X-Requested-With header is not a part of any spec, it is not
reliable|
In my opinion there are not many good reasons to have to change
behaviour if a request is made via XHR. I think the most common
usage is to have a single view tha

Re: Deprecate HttpRequest.is_ajax

[Matemática A3K]

On Tue, Nov 19, 2019 at 1:29 PM Jure Erznožnik 
wrote:

> Sorry for barging in like this, but this is actually a problem I have been
> dealing with quite a bit lately, so:
>
> In my work I very often have to decide, depending on what's calling, what
> the rendered output might be. Ultimately I went with DRF and its content
> negotiation, though even that one - as implemented - sometimes isn't
> sufficient.
>
> See, the problem sometimes isn't that you request JSON and then get JSON,
> request HTML and get HTML.
>
I think content negotiation is about giving the option to request the
content in different formats, not rendering different content based on
which format is requested.

> You also have to cater for exceptions. Maybe a 4xx would return additional
> objects to insert into the DOM while a 200 would be fine with a JSON or
> even without data. What about 500?
>
This (and below) is about how to design a particular API for your needs, I
think it is out of the scope of the problem discussed. The problem
discussed is that is_ajax is not a reliable way to determine the origin of
a request (and then format the content of a response)

> I'm currently handling this with custom headers and the caller (the
> browser) tells the server what kind of outputs it can handle in different
> types of output.
>
> The server then performs the branching at certain code points,
> specifically the ones mentioned above. DRF allows me to choose the
> appropriate renderer. Though I should mention here, that the data is
> already serialised at that point: sometimes this creates issues for
> renderers that might desire more information to do their work. Just
> mentioning that render stages need to be accounted for too. This may not be
> a problem for core Django as it doesn't have stages.
>
> Again, sorry, but still hoping this helped in some way.
>
LP,
> Jure
>
>
> On 19/11/2019 01:06, Matemática A3K wrote:
>
>
> I agree with Adam that it should be deprecated with no replacement.
>
> The content negotiation is something that should be in but not as a
> replacement of it, as a general improvement.
>
> I think there shouldn't be a replacement because "is_ajax" asks whether it
> came from a ""regular"" browser instead of jQuery and with the content
> negotiation you ask if the requester accepts a type - which can also lead
> to errors because the client may also accept other types (no example coming
> to my mind), and if so, it will lead to undesired behavior.
>
> The right approach would be making AJAX requests request JSON output
> explicitly, by using a dedicated endpoint or by appending something that
> manifests their intention - like in
> https://docs.djangoproject.com/en/2.2/topics/class-based-views/mixins/#more-than-just-html
> is done with a get parameter. Not decide the response type by where it came
> from as it is unreliable as stated before, it provides convenience in some
> use cases but can lead to errors.
>
> Seems better to me to refactor the view code so you can write a different
> view for Ajax requests that returns a JSON without code duplication.
>
> As a shortcut, something like "For simple AJAX endpoints wrap your view
> with (something like) a "jsonview" decorator which will check the accept
> header (with something like Claude's code), return the appropriate error
> code if not, set the response type accordingly, and you should return a
> dict of strings (you have to take care of the serialization, i.e with
> https://docs.djangoproject.com/en/2.2/topics/serialization/#serialization-formats-json).
>
>
> On Mon, Nov 18, 2019 at 3:28 PM Tom Forbes  wrote:
>
>> What I meant by that is it’s not an approach that scales well to lots of
>> views. It might be better to have separate endpoints to return JSON (e.g
>> adding a /json suffix), and in the past this has made services I’ve worked
>> on a lot more maintainable and easy to understand. But it’s not as quick to
>> do as `if request.is_ajax()` and requires a bit more upfront work. If you
>> find you need to do this a lot then maybe something more structured like
>> Django Rest Framework will be a better choice, which also handles content
>> negotiation really well (it can produce XML, CSV, JSON, etc etc).
>>
>
>> On 18 Nov 2019, at 15:18, Matthew Pava  wrote:
>>
>> “In my opinion there are not many good reasons to have to change
>> behaviour if a request is made via XHR. I think the most common usage is to
>> have a single view that returns a JSON response or a HTML response
>> depending on if XHR is used (
>> https://github.com/search?l=Python&q=request.is_ajax&type=Code), which
>> isn’t great and isn’t reliable.”
>>
>> I do this. What would the best way to handle this? Perhaps the proper
>> practice should be documented when it is deprecated?
>>
>> *From:* django-developers@googlegroups.com [
>> mailto:django-developers@googlegroups.com
>> ] *On Behalf Of *Tom Forbes
>> *Sent:* Saturday, November 16, 2019 10:16 AM
>> *To:* django-developers@googlegroups.com
>> 

Re: Deprecate HttpRequest.is_ajax

[Matemática A3K]

On Tue, Nov 19, 2019 at 9:20 PM Matemática A3K 
wrote:

>
>
> On Tue, Nov 19, 2019 at 1:29 PM Jure Erznožnik 
> wrote:
>
>> Sorry for barging in like this, but this is actually a problem I have
>> been dealing with quite a bit lately, so:
>>
>> In my work I very often have to decide, depending on what's calling, what
>> the rendered output might be. Ultimately I went with DRF and its content
>> negotiation, though even that one - as implemented - sometimes isn't
>> sufficient.
>>
>> See, the problem sometimes isn't that you request JSON and then get JSON,
>> request HTML and get HTML.
>>
> I think content negotiation is about giving the option to request the
> content in different formats, not rendering different content based on
> which format is requested.
>
>> You also have to cater for exceptions. Maybe a 4xx would return
>> additional objects to insert into the DOM while a 200 would be fine with a
>> JSON or even without data. What about 500?
>>
> This (and below) is about how to design a particular API for your needs, I
> think it is out of the scope of the problem discussed. The problem
> discussed is that is_ajax is not a reliable way to determine the origin of
> a request (and then format the content of a response)
>

(let me try to expand myself a bit so we can refocus)

If it is deprecated, then the question that arises naturally is "What would
be the proper/new way of doing it?" because it is a pattern that have been
applied previously (i.e.
https://docs.djangoproject.com/en/2.2/topics/class-based-views/generic-editing/#ajax-example),
then decorators have been proposed and refactors.

It seems to me that if there is no reliable way of determining it from the
back-end side, then in the end it will be a convention between the front
and the back. This could be a GET parameter, a "ClientWants: JSONOrNothing"
header, or whatever convention you like to make, but not rely on a
convention which seems to be fading out.

You can ensure the actual convention by setting the header manually (as
stated in the is_ajax doc) - as you do with the CSRF token. Another
convention could be better (i.e. "accepts json")

So far, is what the discussion went through to my understanding :)

> I'm currently handling this with custom headers and the caller (the
>> browser) tells the server what kind of outputs it can handle in different
>> types of output.
>>
>> The server then performs the branching at certain code points,
>> specifically the ones mentioned above. DRF allows me to choose the
>> appropriate renderer. Though I should mention here, that the data is
>> already serialised at that point: sometimes this creates issues for
>> renderers that might desire more information to do their work. Just
>> mentioning that render stages need to be accounted for too. This may not be
>> a problem for core Django as it doesn't have stages.
>>
>> Again, sorry, but still hoping this helped in some way.
>>
> LP,
>> Jure
>>
>>
>> On 19/11/2019 01:06, Matemática A3K wrote:
>>
>>
>> I agree with Adam that it should be deprecated with no replacement.
>>
>> The content negotiation is something that should be in but not as a
>> replacement of it, as a general improvement.
>>
>> I think there shouldn't be a replacement because "is_ajax" asks whether
>> it came from a ""regular"" browser instead of jQuery and with the content
>> negotiation you ask if the requester accepts a type - which can also lead
>> to errors because the client may also accept other types (no example coming
>> to my mind), and if so, it will lead to undesired behavior.
>>
>> The right approach would be making AJAX requests request JSON output
>> explicitly, by using a dedicated endpoint or by appending something that
>> manifests their intention - like in
>> https://docs.djangoproject.com/en/2.2/topics/class-based-views/mixins/#more-than-just-html
>> is done with a get parameter. Not decide the response type by where it came
>> from as it is unreliable as stated before, it provides convenience in some
>> use cases but can lead to errors.
>>
>> Seems better to me to refactor the view code so you can write a different
>> view for Ajax requests that returns a JSON without code duplication.
>>
>> As a shortcut, something like "For simple AJAX endpoints wrap your view
>> with (something like) a "jsonview" decorator which will check the accept
>> header (with something like Claude's code), return the appropriate error
>> code if not, set the response type accordingly, and you should return a
>> dict of strings (you have to take care of the serialization, i.e with
>> https://docs.djangoproject.com/en/2.2/topics/serialization/#serialization-formats-json).
>>
>>
>> On Mon, Nov 18, 2019 at 3:28 PM Tom Forbes  wrote:
>>
>>> What I meant by that is it’s not an approach that scales well to lots of
>>> views. It might be better to have separate endpoints to return JSON (e.g
>>> adding a /json suffix), and in the past this has made services I’ve worked
>>> on a lot more maintainabl