Fellow Reports -- April 2021

[Carlton Gibson]

Hi all. 



Calendar Week 13 -- ending 04 April.


Triaged:

https://code.djangoproject.com/ticket/32610 -- 
django.utils.version.get_git_changeset() always returns None. (Accepted)
https://code.djangoproject.com/ticket/32604 -- File uploads larger than 
FILE_UPLOAD_MAX_MEMORY_SIZE get wrong unix group based on setgid (Accepted)
https://code.djangoproject.com/ticket/32603 -- Add transaction handling to 
Changelist list_editable processing. (Accepted)



Reviewed:

https://github.com/django/django/pull/14208 -- Fixed #32610 -- Fixed 
get_git_changeset().
https://github.com/django/django/pull/14204 -- Removed unnecessary 
left/right in admin sidebar CSS.
https://github.com/django/django/pull/11291 -- Fixed #30416 -- Restore 
terminal state when reloading.
https://github.com/django/django/pull/14187 -- Fixed #32594 -- 
Signal.disconnect() returning None
https://github.com/django/django/pull/13305 -- Fixed #31885 -- Updated SMTP 
Email Backend to use an SSLContext.
https://github.com/django/django/pull/13728 -- Add settings 
EMAIL_MESSAGEID_FQDN
https://github.com/django/django/pull/14203 -- Refs #32105 -- Changed 
ExceptionReporter.(html|text)_template_path attributes to properties.
https://github.com/django/django/pull/14180 -- Fixed #32532 -- Raised nicer 
error if a test label is a file path.
https://github.com/django/django/pull/14202 -- Changed Happen to Happens in 
troubleshooting.txt
https://github.com/django/django/pull/14196 -- Fixed #32595 -- Fixed 
SchemaEditor.quote_value() crash with bytes.
https://github.com/django/django/pull/13841 -- Fixed #32316 -- Deferred 
accessing __file__.
https://github.com/django/django/pull/14195 -- Fixed #29127 -- Prevented 
DiscoverRunner from hiding tagged test with syntax errors.



Authored:

https://github.com/django/djangoproject.com/pull/1083 -- Updated 
robots.docs.txt for Django 3.2.
https://github.com/django/djangoproject.com/pull/1079 -- Added Pycharm 
banners for 2021 fundraiser.





Calendar Week 14 -- ending 11 April.


Released Django 3.2.


Triaged:

https://code.djangoproject.com/ticket/32611 -- runtests.py's 
bisect_tests() and paired_tests() don't always need to do setup and 
teardown (Accepted)



Reviewed:

https://github.com/django/django/pull/14240 -- Refs #32074 -- Removed usage 
of Python's deprecated distutils.version package.
https://github.com/django/django/pull/14236 -- Refs #31578 -- Removed 
outdated notes about MyISAM in GIS docs.
https://github.com/django/django/pull/14231 -- Refs #32074 -- Made 
ExclusionConstraint.__repr__() use Deferrable.__repr__().
https://github.com/django/django/pull/14212 -- Fixed #32544 -- Adapted 
tests to confirm support for GDAL 3.2/GEOS 3.9.
https://github.com/django/django/pull/14228 -- Refs #32074 -- Backported 
Enum.__repr__() from Python 3.10.
https://github.com/django/daphne/pull/364 -- Use partial() to wrap 
Server.handle_reply() #364
https://github.com/django/daphne/pull/365 -- Lint with pre-commit #365
https://github.com/django/django/pull/13840 -- Add Github actions for 
MacOS, Windows and linting
https://github.com/django/django/pull/14224 -- Refs #30156 -- Corrected 
version in SpatiaLite install instructions.
https://github.com/django/django/pull/14221 -- Fixed #32614 -- Fixed 
MiddlewareSyncAsyncTests tests with asgiref 3.3.2+.



Authored:

https://github.com/django/code.djangoproject.com/pull/110 -- Updated 
default Django version to 3.2. #110
https://github.com/django/django/pull/14223 -- Updated asgiref dependency 
for 3.2 release series.





Calendar Week 15 -- ending 18 April.


Triaged:

https://code.djangoproject.com/ticket/32654 -- docstring in modules created 
by startapp (wontfix)
https://code.djangoproject.com/ticket/32655 -- Improve error if a string is 
passed as an extra_test to DiscoverRunner.build_suite() (Accepted)
https://code.djangoproject.com/ticket/32651 -- Combining Q with a Q 
containing Exists crashes (Accepted)
https://code.djangoproject.com/ticket/32652 -- Obsolete FAQ reference in 
triaging docs (Accepted)
https://code.djangoproject.com/ticket/32647 -- Select multiple action 
checkboxes with shift+mouseclick in django admin (Accepted)
https://code.djangoproject.com/ticket/32636 -- 
QuerySet.values()/values_list() crashes on a combined queryset ordered by 
"extra" select. (Accepted)
https://code.djangoproject.com/ticket/32606 -- Allow rich configuration of 
Selenium tests (wontfix)
https://code.djangoproject.com/ticket/32640 -- Non-manager instance 
assigned to model class' objects is silently ignored (Accepted)
https://code.djangoproject.com/ticket/32630 -- Autoreloader doesn't 
work on Windows 10. (worksforme)



Reviewed:

https://github.com/django/django/pull/14265 -- Fixed #32645 -- Fixed 
QuerySet.update() crash when ordered by joined fields on MySQL/MariaDB.
https://github.com/django/django/pull/14267 -- [3.2.x] Fixed #32548 -- 
Fixed crash when combining Q() objects with boolean expressions.
https://github.com/django/django/pull/14263 -- Recommend django<4

Paid Contributions for specific feature in Django

['Taylor' via Django developers (Contributions to Django itself)]

Hi All,
I am not sure if this is the place to ask this question, so sorry in 
advance, but I am looking to pay for a feature request. I would like for 
someone to build a Django DB adapter for Snowflake DB. 
https://www.snowflake.com/

It would be happy if it was a part of Django core or a separate package. 
Either way, it would be 100% open source after completion. Is there any 
process for this type of thing? If there isn't, is there anyone on here 
interested in this type of work?

Best,
Taylor

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/d2a082af-c014-47d9-909f-87298a2e117bn%40googlegroups.com.

Re: Paid Contributions for specific feature in Django

['Adam Johnson' via Django developers (Contributions to Django itself)]

Hi Taylor

FYI Disney have already made a snowflake backend - they were asking about
open sourcing it in this thread:
https://groups.google.com/g/django-developers/c/po9dS-2h4lg/m/Qa8H2h_6AwAJ
. You might want to get in touch with them.

Otherwise, if you wanted this developed, it would probably be as a third
party package. There are a few people on this mailing list working on
database backends who might be interested in the work. You can also post
gigs on the forum and twitter.

Thanks,

Adam

On Tue, 20 Apr 2021 at 22:39, 'Taylor' via Django developers (Contributions
to Django itself)  wrote:

> Hi All,
> I am not sure if this is the place to ask this question, so sorry in
> advance, but I am looking to pay for a feature request. I would like for
> someone to build a Django DB adapter for Snowflake DB.
> https://www.snowflake.com/
>
> It would be happy if it was a part of Django core or a separate package.
> Either way, it would be 100% open source after completion. Is there any
> process for this type of thing? If there isn't, is there anyone on here
> interested in this type of work?
>
> Best,
> Taylor
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/d2a082af-c014-47d9-909f-87298a2e117bn%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAMyDDM1jxb%3D5uDbrb%3D2RmLiEtjCZtA1zM-%3D%2BXujgW5uFFpZAUA%40mail.gmail.com.

Re: Paid Contributions for specific feature in Django

[Tim Graham]

Hi Taylor,

As Adam said, it would be nice not to duplicate work that Disney's already 
done, but if I can help, I'm open to consulting and have written backends 
for CockroachDB (https://github.com/cockroachdb/django-cockroachdb) and 
Google Cloud Spanner 
(https://github.com/googleapis/python-spanner-django/). I still maintain 
the former and the latter has been turned over to Google.

(Incidentally, I'd rather reply privately but that option seems to be 
disabled for this group? "Reply to author" is greyed out and when I hover 
it says, "You do not have permissions to reply to author in this group." 
I'm not sure it was intentionally configured like that as I believe that 
option was available before Google Groups rolled out their latest 
interface.)
On Tuesday, April 20, 2021 at 5:39:34 PM UTC-4 Taylor wrote:

> Hi All,
> I am not sure if this is the place to ask this question, so sorry in 
> advance, but I am looking to pay for a feature request. I would like for 
> someone to build a Django DB adapter for Snowflake DB. 
> https://www.snowflake.com/
>
> It would be happy if it was a part of Django core or a separate package. 
> Either way, it would be 100% open source after completion. Is there any 
> process for this type of thing? If there isn't, is there anyone on here 
> interested in this type of work?
>
> Best,
> Taylor
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/23067fa0-3497-4e4e-a12d-4b8efd159ca9n%40googlegroups.com.

Re: Paid Contributions for specific feature in Django

['Taylor Hakes' via Django developers (Contributions to Django itself)]

Thanks Adam for linking to that thread and thanks Tim for offering to write
the backend. It would be great if we can open source what Disney has
already done (to avoid duplicate work). I will continue the discussion over
on that thread and see where it goes.

On Tue, Apr 20, 2021 at 4:02 PM Tim Graham  wrote:

> Hi Taylor,
>
> As Adam said, it would be nice not to duplicate work that Disney's already
> done, but if I can help, I'm open to consulting and have written backends
> for CockroachDB (https://github.com/cockroachdb/django-cockroachdb) and
> Google Cloud Spanner (https://github.com/googleapis/python-spanner-django/).
> I still maintain the former and the latter has been turned over to Google.
>
> (Incidentally, I'd rather reply privately but that option seems to be
> disabled for this group? "Reply to author" is greyed out and when I hover
> it says, "You do not have permissions to reply to author in this group."
> I'm not sure it was intentionally configured like that as I believe that
> option was available before Google Groups rolled out their latest
> interface.)
> On Tuesday, April 20, 2021 at 5:39:34 PM UTC-4 Taylor wrote:
>
>> Hi All,
>> I am not sure if this is the place to ask this question, so sorry in
>> advance, but I am looking to pay for a feature request. I would like for
>> someone to build a Django DB adapter for Snowflake DB.
>> https://www.snowflake.com/
>>
>> It would be happy if it was a part of Django core or a separate package.
>> Either way, it would be 100% open source after completion. Is there any
>> process for this type of thing? If there isn't, is there anyone on here
>> interested in this type of work?
>>
>> Best,
>> Taylor
>>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/django-developers/q5Qpvun9Hco/unsubscribe
> .
> To unsubscribe from this group and all its topics, send an email to
> django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/23067fa0-3497-4e4e-a12d-4b8efd159ca9n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CADkiSVzRybneFoJ_e8iQQ4iOKFwsUa4rwFy7%2BtZBPfHQ0vYZrg%40mail.gmail.com.

Re: Snowflake db backend

['Taylor' via Django developers (Contributions to Django itself)]

Hey Everyone,

Sorry to open up an old thread. 

Tim - were you ever able to open source your Snowflake backend? We would 
love to use it and even devote resources (developers or funding for 
contractors) to improving it and getting the tests passing. At Cedar, we 
were planning on creating our own Snowflake backend, but it would be great 
to not duplicate work here. What are your thoughts?

Best,
Taylor

On Wednesday, January 27, 2021 at 1:08:13 AM UTC-8 f.apo...@gmail.com wrote:

> Hi Scott,
>
> Thank you for your response, this is very helpful.
>
> On Tuesday, January 26, 2021 at 11:38:18 PM UTC+1 foug...@apps.disney.com 
> wrote:
>
>> Snowflake does not support lastrowid.  So, we grab the last ID inserted 
>> with a 'SELECT MAX(pk_name) FROM table_name'.  This is obviously prone to 
>> failure.  Assigning an ID during the INSERT would provide similar results 
>> on all backends.
>>
>
> U, the 'SELECT MAX()' is not going to fly well, you are right. 
> Assigning an ID during INSERT has it's own problems. In postgresql it would 
> be possible because you could just select the next value from the created 
> sequence (more or less), but with IDENTITY columns this might get harder. I 
> do not think there is a sensible way to do this in MySQL at all. While 
> lastrowid support is a nice to have, Django should work (mostly?) without 
> it: 
> https://github.com/django/django/blob/464a4c0c59277056b5d3c1132ac1b4c6085aee08/django/db/models/sql/compiler.py#L1372-L1387
>  
> -- the requirement here is that your database is at least able to return 
> values after insert. From the looks of it, it does not I think? Or 
> differently put: Which ways does snowflake offer to get an ID? Solely by 
> providing it directly into insert?
>
> The feature flag `supports_transactions` really means 
>> `supports_nested_transactions`.  Snowflake supports a single level of 
>> transaction, BEGIN + (ROLLBACK|COMMIT).  Multiple BEGINS contribute to the 
>> current (only) transaction.  Since I have to set this flag to False, no 
>> transactions are used, even ones that are supported and testing grinds to a 
>> crawl with all of the table truncations and repopulation.  Since Django 
>> *normally* operates in auto-commit mode, this isn't a huge deal. 
>>  Snowflake also doesn't support save points, but the feature flag seems to 
>> do what is expected when disabled.
>>
>
> Hu, which database support nested BEGIN? As far as I am aware Django does 
> never nest BEGINs -- do you have an example for me? I'd very much like to 
> fix this. From a quick glance at the code we only start a transaction if we 
> are not already in one: 
> https://github.com/django/django/blob/master/django/db/transaction.py#L196-L208
>
> Snowflake does not support column references without a FROM or JOIN 
>> clause.  I've only encountered this used in the unit tests.
>>
>
> Ok, see below.
>
> I've been able to work around most of the function differences by adding 
>> as_snowflake methods to the appropriate classes.  There are a few cases 
>> like JSON_OBJECT that don't translate well, which work, but not entirely as 
>> expected.
>>
>
> Perfect, this sounds as if our extension system works as intended in this 
> area.
>
> A search for 'connection.vendor == ' in the core code shows one example of 
>> where core backends are given privileged access to core Django inner 
>> workings that 3rd party backends don't.
>>
>
> We have been working to get rid of those: 
> https://github.com/django/django/commit/275dd4ebbabbbe758c7219a3d666953d5a7b072f#diff-69f332030b6f25f8f4d95842548d847139ee2cc163aef18f1c8d83b90f3f9e5f
>  
> -- This is solely in 3.2, but Tim can suggest a workaround for earlier 
> versions (he was doing something similar in his cockroachdb tests before 
> 3.2).
>
> Please take these comments as observations, not complaints.  I understand 
>> project evolution and the resistance to change something that doesn't seem 
>> broken.  Maybe keep some of these thoughts in mind when a change is made to 
>> the core backends or the compiler and consider how other backends will need 
>> to implement that new feature.
>>
>
> No offense taken at all. If somehow possible I'd like to encourage you to 
> work with us on your pain points. I think at least some of those are 
> addressed or at least addressable. I am happy to offer code and fixes, but 
> I'll need a few more details especially wrt transaction handling.
>
> Cheers,
> Florian
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/03abf31b-75db-433e-9d35-ead6fefff5a4n%40googlegroups.com.

Re: Snowflake db backend

['Taylor' via Django developers (Contributions to Django itself)]

Sorry, I meant to write Scott, not Tim. I shouldn't write emails late at 
night.

- Taylor

On Tuesday, April 20, 2021 at 10:29:46 PM UTC-7 Taylor wrote:

> Hey Everyone,
>
> Sorry to open up an old thread. 
>
> Tim - were you ever able to open source your Snowflake backend? We would 
> love to use it and even devote resources (developers or funding for 
> contractors) to improving it and getting the tests passing. At Cedar, we 
> were planning on creating our own Snowflake backend, but it would be great 
> to not duplicate work here. What are your thoughts?
>
> Best,
> Taylor
>
> On Wednesday, January 27, 2021 at 1:08:13 AM UTC-8 f.apo...@gmail.com 
> wrote:
>
>> Hi Scott,
>>
>> Thank you for your response, this is very helpful.
>>
>> On Tuesday, January 26, 2021 at 11:38:18 PM UTC+1 foug...@apps.disney.com 
>> wrote:
>>
>>> Snowflake does not support lastrowid.  So, we grab the last ID inserted 
>>> with a 'SELECT MAX(pk_name) FROM table_name'.  This is obviously prone to 
>>> failure.  Assigning an ID during the INSERT would provide similar results 
>>> on all backends.
>>>
>>
>> U, the 'SELECT MAX()' is not going to fly well, you are right. 
>> Assigning an ID during INSERT has it's own problems. In postgresql it would 
>> be possible because you could just select the next value from the created 
>> sequence (more or less), but with IDENTITY columns this might get harder. I 
>> do not think there is a sensible way to do this in MySQL at all. While 
>> lastrowid support is a nice to have, Django should work (mostly?) without 
>> it: 
>> https://github.com/django/django/blob/464a4c0c59277056b5d3c1132ac1b4c6085aee08/django/db/models/sql/compiler.py#L1372-L1387
>>  
>> -- the requirement here is that your database is at least able to return 
>> values after insert. From the looks of it, it does not I think? Or 
>> differently put: Which ways does snowflake offer to get an ID? Solely by 
>> providing it directly into insert?
>>
>> The feature flag `supports_transactions` really means 
>>> `supports_nested_transactions`.  Snowflake supports a single level of 
>>> transaction, BEGIN + (ROLLBACK|COMMIT).  Multiple BEGINS contribute to the 
>>> current (only) transaction.  Since I have to set this flag to False, no 
>>> transactions are used, even ones that are supported and testing grinds to a 
>>> crawl with all of the table truncations and repopulation.  Since Django 
>>> *normally* operates in auto-commit mode, this isn't a huge deal. 
>>>  Snowflake also doesn't support save points, but the feature flag seems to 
>>> do what is expected when disabled.
>>>
>>
>> Hu, which database support nested BEGIN? As far as I am aware Django does 
>> never nest BEGINs -- do you have an example for me? I'd very much like to 
>> fix this. From a quick glance at the code we only start a transaction if we 
>> are not already in one: 
>> https://github.com/django/django/blob/master/django/db/transaction.py#L196-L208
>>
>> Snowflake does not support column references without a FROM or JOIN 
>>> clause.  I've only encountered this used in the unit tests.
>>>
>>
>> Ok, see below.
>>
>> I've been able to work around most of the function differences by adding 
>>> as_snowflake methods to the appropriate classes.  There are a few cases 
>>> like JSON_OBJECT that don't translate well, which work, but not entirely as 
>>> expected.
>>>
>>
>> Perfect, this sounds as if our extension system works as intended in this 
>> area.
>>
>> A search for 'connection.vendor == ' in the core code shows one example 
>>> of where core backends are given privileged access to core Django inner 
>>> workings that 3rd party backends don't.
>>>
>>
>> We have been working to get rid of those: 
>> https://github.com/django/django/commit/275dd4ebbabbbe758c7219a3d666953d5a7b072f#diff-69f332030b6f25f8f4d95842548d847139ee2cc163aef18f1c8d83b90f3f9e5f
>>  
>> -- This is solely in 3.2, but Tim can suggest a workaround for earlier 
>> versions (he was doing something similar in his cockroachdb tests before 
>> 3.2).
>>
>> Please take these comments as observations, not complaints.  I understand 
>>> project evolution and the resistance to change something that doesn't seem 
>>> broken.  Maybe keep some of these thoughts in mind when a change is made to 
>>> the core backends or the compiler and consider how other backends will need 
>>> to implement that new feature.
>>>
>>
>> No offense taken at all. If somehow possible I'd like to encourage you to 
>> work with us on your pain points. I think at least some of those are 
>> addressed or at least addressable. I am happy to offer code and fixes, but 
>> I'll need a few more details especially wrt transaction handling.
>>
>> Cheers,
>> Florian
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroup