Re: Fellow Reports - March 2024

[Mariusz Felisiak]

Week ending March 24

*Triaged:*
https://code.djangoproject.com/ticket/35312 - FileNotFoundError escapes 
from run_formatters() (invalid)
https://code.djangoproject.com/ticket/34059 - Validation of check 
constraints on JSONField key transforms with None produces invalid SQL on 
PostgreSQL. (fixed)
https://code.djangoproject.com/ticket/35316 - Support scalars as query 
parameters in admin changelist filters (for backward compatibility). 
(wontfix)
https://code.djangoproject.com/ticket/35318 - QuerySet.exclude() docs 
contain a potentially misleading statement regarding the restrictiveness of 
examples. (invalid)
https://code.djangoproject.com/ticket/35319 - Move the link to add a 
related object inside the .selector-available column for 
FilteredSelectMultiple. (accepted)
https://code.djangoproject.com/ticket/35317 - Add the possibility to do 
prefetches for only a subset of instances (wontfix)
https://code.djangoproject.com/ticket/35321 - Django Admin in Mobile 
Safari zooms in on text inputs (accepted)
https://code.djangoproject.com/ticket/35320 - Remove unnecessary 
django.core.files.move._samefile() hook. (accepted)
https://code.djangoproject.com/ticket/35325 - CharField max_length 
ignored (duplicate)
https://code.djangoproject.com/ticket/35327 - Add note to create custom 
user model in tutorial (wontfix)

*Reviewed/committed:*
https://github.com/django/django/pull/17963 - Refs #31014 -- Added srid 
argument to FromWKB/FromWKT() GIS functions.
https://github.com/django/django/pull/17984 - Fixed #35309 -- Made 
prefetch clear ordering for single-valued relationships.
https://github.com/django/django/pull/17916 - Fixed #35257 -- Corrected 
resolving output_field for IntegerField/DecimalField with NULL.
https://github.com/django/code.djangoproject.com/pull/188 - Fixed #187 
-- Restored support for restructuredtext on wiki pages.
https://github.com/django/django/pull/18000 - Fixed #35320 -- Removed 
unnecessary django.core.files.move._samefile() hook.
https://github.com/django/django/pull/17266 - Fixed #34841 -- Avoided 
rendering apps on state still requiring mutation.
https://github.com/django/django/pull/18001 - Fixed #35323 -- Prevented 
file_move_safe() from trying to overwrite existing file when 
allow_overwrite is False.

*Reviewed:*
https://github.com/django/django/pull/17545 - Updated docs for Django 
releases, including details about fetching and processing translations.

*Authored:*
https://github.com/django/django/pull/17988 - Stopped using 
byte-compiled 3rd-party packages in daily builds for byte-compiled Django.
https://github.com/django/django/pull/17989 - Refs #34059, Refs #34060 
-- Removed outdated warning about validation of JSONField constraints.
https://github.com/django/django/pull/17993 - Fixed ResourceWarning 
from unclosed SQLite connection on Python 3.13+.
https://github.com/django/django/pull/17997 - Fixed ResourceWarning 
from unclosed SQLite connection in test_utils on Python 3.13+.
https://github.com/django/django/pull/18004 - Added 
ModelState.from_model() test for abstract model with unnamed indexes.

Best,
Mariusz

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/deeba107-d777-48b6-8819-c92d08177cfan%40googlegroups.com.

Re: Intermittent IntegrityError on Model Save with auto_now and auto_now_add Fields

['Michael Lissner' via Django developers (Contributions to Django itself)]

Looks like this issue isn't affecting lots of folks, since nobody is piping 
up, so I'll just add that if anybody arrives here in the future, we'll be 
tracking this in a public issue here: 

https://github.com/freelawproject/courtlistener/issues/3359

My theory is it has something to do with race conditions/timings and async, 
since I *think* it started around the time we started switching to async, 
but others on the team tell me that's a red herring (it's not the cause). 

Anyhow, if anybody ever arrives here in the future, we fixed it 

 
by overriding the save method to catch IntegrityErrors. It's pretty lame to 
have code like this in our save method, but it seems to be working: 

try: 
# Without a transaction wrapper, a failure will invalidate outer 
transactions 
with transaction.atomic(): 
super(Docket, self).save( 
update_fields=update_fields, *args, **kwargs 
) 
except IntegrityError: 
# Temporary patch while we solve #3359 
# If the error is not related to `date_modified` it will raise again 
self.date_modified = timezone.now() 
super(Docket, self).save( 
update_fields=update_fields, *args, **kwargs 
)

Anyhow, thanks for reading, everybody.

On Monday, March 18, 2024 at 7:08:22 AM UTC-7 Matthew Pava wrote:

> Hi Bill,
>
> We ended up using a package called Django-audit-log: 
> https://pypi.org/project/django-audit-log/.
>
> It’s outdated now, but we used the source code for their 
> CreatingUserField, LastUserField, CreationDateTimeField, and 
> ModificationDateTimeField. More modern packages may have enhanced features 
> than these.
>
>  
>
> In your case, I may just add a save method to the abstract model and add 
> timezone.now() to the corresponding fields. If there’s not a pk, then 
> populate the created date field.
>
>  
>
> You are right, though. Perhaps the Django community can explain the issue 
> between auto_now and auto_now_add fields.
>
>  
>
> *From:* 'William Palin' via Django developers (Contributions to Django 
> itself)  
> *Sent:* Sunday, March 17, 2024 9:59 AM
> *To:* Django developers (Contributions to Django itself) <
> django-d...@googlegroups.com>
> *Subject:* Intermittent IntegrityError on Model Save with auto_now and 
> auto_now_add Fields
>
>  
>
> Hello Django community,
>
>  
>
> We are reaching out after encountering a persistent and elusive issue that 
> manifests as an IntegrityError during the save operation of a Django model. 
> This problem has been sporadically occurring since last year and has 
> successfully stumped four seasoned Django developers on our team. The error 
> seems to involve the non-update of auto_now fields upon model save, leading 
> us to suspect a deeper issue within Django, though we are cautious about 
> drawing premature conclusions. Given the complex and intermittent nature of 
> this bug, we are seeking insights, advice, or any form of guidance from the 
> community.
>
>  
>
> *Issue Overview:*
>
>  
>
> Our model inherits date_created and date_modified fields from an abstract 
> base class designed to standardize these timestamps across our models. Here 
> is how the abstract base class is defined:
>
>  
>
>  
>
> class AbstractDateTimeModel(models.Model):
>
> """An abstract base class for most models"""
>
> 
>
> date_created = models.DateTimeField(
>
> help_text="The moment when the item was created.",
>
> auto_now_add=True,
>
> db_index=True,
>
> )
>
> date_modified = models.DateTimeField(
>
> help_text="The last moment when the item was modified. A value in 
> year"
>
>   " 1750 indicates the value is unknown",
>
> auto_now=True,
>
> db_index=True,
>
> )
>
>  
>
> class Meta:
>
> abstract = True
>
>  
>
> *The Problem:*
>
>  
>
> Intermittently, the .save() method triggers an IntegrityError, apparently 
> because the auto_now field (date_modified) does not get created. A specific 
> instance of this error showed that while date_created is correctly 
> populated, date_modified remains null, which directly leads to the 
> IntegrityError upon attempting to insert the record:
>
>  
>
> [datetime.datetime(2023, 10, 2, 14, 33, 49, 99833, 
> tzinfo=datetime.timezone.utc), None, ...]
>
> 'INSERT INTO "search_docket" ("date_created", "date_modified",... ]
>
>  
>
> *What We've Tried:*
>
>  
>
>- Investigated the possibility of an issue with update_fields being 
>non-null and excluding date_modified during .save(), but confirmed through 
>Sentry logs that update_fields was indeed None in all instances of the 
>error.
>- Attempted to reproduce the issue in a controlled environment without 
>success, leaving us without a clear direction for a solution.
>
>  
>
> *Request for Help:*
>
>  
>
> We're wondering if this could point to an undocumented edge case in 
> Djang