QuerySet.contains() for .values() and .values_list()

[Johan Schiff]

https://github.com/django/django/pull/13038

Above pull request (Ticket #24141) has a discussion on how to handle 
qs.contains() for QuerySets with qs.values() or qs.values_list() calls.

Current patch checks self._result_cache if exists and iterable class is 
ModelIterable, otherwise returns .filter(pk=obj.pk).exists(), i.e. hitting 
the database.

These are the options I see:
1. As above.
2. Throw an exception if iterable class is not ModelIterable (when using 
.values or .values_list).
3. Accept dict lookup for .values and tuple lookup for .values_list 
querysets, so that the .contains() query matches what you get from the 
iterable.

It seems kind of neat to be able to check if an object is in a QuerySet, 
even if you group that QuerySet using 
.values('group').annotate(sum=Sum('value')).order_by(). But that was never 
the case I saw for QuerySet.contains(), so I have no real preference here.

Johan

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/5d77428e-a4a4-4bda-a3bb-2811a88e3797n%40googlegroups.com.

Fellow Reports - January 2021

[Mariusz Felisiak]

Week ending January 10, 2021

*Triaged:*
    https://code.djangoproject.com/ticket/32310 - Clarify 
ManyToManyField.through note about reverse accessors names for 
non-symmetrical recursive relationships. (accepted)
    https://code.djangoproject.com/ticket/32314 - Allow autoreloading 
of `python -m pkg_other_than_django runserver`. (accepted)
    https://code.djangoproject.com/ticket/32318 - Collect on annotated 
values causes error. (duplicate)
    https://code.djangoproject.com/ticket/32320 - DateField uses 
unlocalized function when auto_now is True (invalid)
    https://code.djangoproject.com/ticket/32321 - Add system checks for 
invalid model field names for functional indexes in Meta.indexes. (created)
    https://code.djangoproject.com/ticket/32323 - Lazy object id fetch 
on object initialization. (duplicate)
    https://code.djangoproject.com/ticket/32324 - Adding more blocks to 
the contrib.admin site. (needsinfo)
    https://code.djangoproject.com/ticket/32322 - Incorrect wrapping of 
autocomplete cleaner on Chrome. (accepted)
    https://code.djangoproject.com/ticket/32327 - 
get_random_secret_key() should return a valid secret key. (wontfix)
    https://code.djangoproject.com/ticket/32326 - Proposing a more 
concrete example of 'Streaming large CSV files' (wontfix)
    https://code.djangoproject.com/ticket/32328 - Add to admin a 
friendly widget for ArrayField. (needsinfo)
    https://code.djangoproject.com/ticket/32331 - Using non-ASCII chars 
in EMAIL_HOST_PASSWORD raises UnicodeEncodeError. (invalid)
    https://code.djangoproject.com/ticket/32330 - Inheritance for 
already existing models will throw an exception (worksforme)
    https://code.djangoproject.com/ticket/32332 - Saving parent object 
after setting on child leads to data loss for parents with non-numeric 
primary key. (accepted)
    https://code.djangoproject.com/ticket/32333 - _raw_delete won't 
work as expected on union'ed QuerySets (invalid)
    https://code.djangoproject.com/ticket/32334 - Backwards migration 
with RenameModel om MySQL failing with content_type.name does not exist 
(invalid)


*Reviewed/committed:*
    https://github.com/django/django/pull/13826 - Replaced 
DatabaseFeatures.supports_left_right_lookups with skipUnlessGISLookup.
    https://github.com/django/django/pull/13831 - Simplified 
django.db.utils.load_backend().
    https://github.com/django/django/pull/13830 - Refs #28741 -- Doc'd 
SESSION_COOKIE_DOMAIN requirement with CSRF_USE_SESSIONS.
    https://github.com/django/django/pull/13811 - Fixed #32298 -- Fixed 
URLValidator hostname length validation.
    https://github.com/django/django/pull/13838 - Fixed #32310 -- Fixed 
note about reverse accessors for intermediate table for self-referential 
ManyToManyField.
    https://github.com/django/django/pull/13731 - Fixed #32231 -- 
Allowed passing None params to QuerySet.raw().
    https://github.com/django/django/pull/13837 - Fixed #32314 -- Fixed 
detection when started non-django modules with "python -m" in autoreloader.
    https://github.com/django/django/pull/13846 - Fixed #32322 -- Fixed 
autocomplete widget wrapping.
    https://github.com/django/django/pull/13849 - Refs #32191 -- Added 
Signer.sign_object()/unsign_object().
    https://github.com/django/django/pull/13839 - Fixed #32315 -- Doc'd 
FieldFile.path.
    https://github.com/django/django/pull/13800 - Fixed #32191 -- Made 
CookieStorage use RFC 6265 compliant format.


*Reviewed:*
    https://github.com/django/django/pull/13134 - Fixed #31747 -- Fixed 
model enumeration via admin URLs.


Best,
Mariusz

--
You received this message because you are subscribed to the Google Groups "Django 
developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/dcf458ed-378a-2178-e82b-8c7ee87e5226%40gmail.com.

Re: Status of 3.2 release blockers.

[Paul Ganssle]

As we come closer to the deadline, I thought I'd like to bring attention to
the one aspect of the zoneinfo-support PR that I am least certain about but
which hasn't gotten much discussion, detailed here:
https://github.com/django/django/pull/13877#issuecomment-758769902

*Note*: I don't think anything I'm talking about in this e-mail needs to
block the existing PR!

The tl;dr is that `timezone.make_aware` and some date truncation logic both
take an `is_dst` parameter, which duplicates pytz's behavior. The current
behavior as of 3.1 is that `is_dst` is a no-op unless you are using pytz.
In the current version of the PR, `is_dst` is a no-op unless you are using
pytz, but particularly in the case of date truncation logic, that *may* defeat
some peoples' expectations. The options I see are:

1. Implement is_dst mostly like it's implemented in pytz-deprecation-shim,
where explicitly passing `is_dst=None` will raise an exception on ambiguous
and imaginary times, leaving it blank is a no-op, and specifying `True` or
`False` sets the `fold` parameter to match what `pytz` would give you. If
people are already able to use `make_aware` with non-pytz zones, this
*may* introduce
new exceptions into their code, which would not necessarily be backwards
compatible. Depending on how broken support for non-pytz zones is "on the
ground", this may not matter at all, since making theoretically
backwards-incompatible changes to an otherwise broken function is not
really backwards incompatible...

2. Implement is_dst, but require opt-in for the error-raising behavior:
`is_dst=None` would raise errors only in the `pytz` case, `is_dst='error'`
would always raise errors, and `is_dst=True` and `is_dst=False` would set
the `fold` parameter to match what `pytz` would give you. This should be
effectively backwards compatible, since leaving `is_dst` unspecified would
keep the behavior as-is and specifying True/False with a non-pytz zone
never worked anyway (and as such might be considered a bug anyway).

3. Leave as-is. External users who want to replicate `is_dst`-style
behavior can use wrapper functions or subclasses to accomplish this anyway.
I think the most annoying one to replicate would be date truncation logic
.
People who want truncation with `is_dst` support would either want to wrap
the output of their `.annotate` calls (e.g. `model =
DTModel.objects.annotate(truncated_start=TruncDay('start_datetime')).get();
model.truncated_start = apply_is_dst(model.truncated_start, is_dst=True)`),
or they would want to create a parallel truncation hierarchy subclassing
from TruncBase, where is_dst support is added in convert_value

(or
TruncBase is monkey-patched to this effect).

4. Leave `make_aware` as is, but add `is_dst` support into
TruncBase.convert_value, similar to what I'm doing in
`django.forms.utils.from_current_timezone` — in that case we're just
raising exceptions on error and not trying to re-implement `is_dst` in
terms of `fold`, which is trickier than you'd think
,
but the implementation in pytz_deprecation_shim is pretty well-tested, so I
am not too worried about it.

Let me know what y'all want to do. I can also do this as a separate PR, as
I think the "support zoneinfo" PR works as a standalone.

One thing to consider is that even if we don't do this for 3.2, if 4.0
comes with a "hard switch" to zoneinfo, we'll need to decide what the
behavior of `is_dst` is going to be, and what the future if `is_dst` looks
like. One advantage that pytz_deprecation_shim has over Django is that it's
replicating pytz's implementation, so it already needs a similar exception
hierarchy to pytz's. Django would need to introduce its own exception
hierarchy for "what happens when you try and create a datetime that is
ambiguous or doesn't exist." There's also the problem that not all folds or
gaps are created by DST transitions, and it's not entirely clear what
`is_dst` should do in that case. Probably the easiest thing is to leave it
in place and make it well-defined, or to deprecate `is_dst` in favor of a
new argument like `preferred_fold` that uses the semantics of folds instead
of the semantics of DST.

Best,
Paul

On Tue, Jan 12, 2021 at 12:48 PM Paolo Melchiorre 
wrote:

> I agree with Adam.
>
> I've also left a small comment on the PR.
>
> Paolo
>
> On Tue, Jan 12, 2021 at 5:59 PM Adam Johnson  wrote:
> >
> > I think it's worth merging into 3.2. The change is quite small, the
> potential benefits are quite large, and some users live LTS to LTS so could
> be left without an option for a long time.
> >
> > I've left some review comments on the PR.
> >
> > On 

Re: Status of 3.2 release blockers.

[Carlton Gibson]

Hi Paul,

Thanks for this follow up. To give context, I think there’s enough support
(here and on the PR) to say, OK let’s take it.

Mariusz and I have two others on the list, which we’ll get in tomorrow.
I’ll then branch stable/3.2.x (and we can bootstrap Django 4.0 and continue
main line development.)

I’ll finish review on your PR, merge and backport to stable/3.2.x, looking
to do 3.2a1 on Tuesday.

That means we have a few days to think about whether we can do anything for
is_dst, for 3.2.

On an initial read (but I’m getting tired now) I’d go for your Option 3.
Folks opting to use zoneinfo in Django 3.2 are (by definition) in the “I
know what I’m up to” club and can implement wrappers if they want.

We have a bit of time to discuss 4.0 but I’m inclined towards your
“Probably the easiest thing...” (We’ll offer the shim as an option but
folds are the new preferred way right? Let’s have them, I’d think.

If others have capacity to put their brains to this, that would be much
appreciated.

Thanks for your input here Paul!

Kind regards, Carlton

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAJwKpyRVDFG%3DUhA2uzmoCPN0BM%2BQS0oGQgmtJiZQdG1Oc1mjZw%40mail.gmail.com.

Re: Status of 3.2 release blockers.

[Adam Johnson]

I would also lean towards option 3. If someone makes wrappers they might
also release them in a third party package.

On Wed, 13 Jan 2021 at 17:46, Carlton Gibson 
wrote:

> Hi Paul,
>
> Thanks for this follow up. To give context, I think there’s enough support
> (here and on the PR) to say, OK let’s take it.
>
> Mariusz and I have two others on the list, which we’ll get in tomorrow.
> I’ll then branch stable/3.2.x (and we can bootstrap Django 4.0 and continue
> main line development.)
>
> I’ll finish review on your PR, merge and backport to stable/3.2.x, looking
> to do 3.2a1 on Tuesday.
>
> That means we have a few days to think about whether we can do anything
> for is_dst, for 3.2.
>
> On an initial read (but I’m getting tired now) I’d go for your Option 3.
> Folks opting to use zoneinfo in Django 3.2 are (by definition) in the “I
> know what I’m up to” club and can implement wrappers if they want.
>
> We have a bit of time to discuss 4.0 but I’m inclined towards your
> “Probably the easiest thing...” (We’ll offer the shim as an option but
> folds are the new preferred way right? Let’s have them, I’d think.
>
> If others have capacity to put their brains to this, that would be much
> appreciated.
>
> Thanks for your input here Paul!
>
> Kind regards, Carlton
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/CAJwKpyRVDFG%3DUhA2uzmoCPN0BM%2BQS0oGQgmtJiZQdG1Oc1mjZw%40mail.gmail.com
> 
> .
>


-- 
Adam

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAMyDDM3_qCjOiDUtJkeanDM-7ASDX0Qc%2BA3QtX9WCoABWLiRMQ%40mail.gmail.com.

Re: Admin accessibility

[Thibaud Colas]

Hi all,

I know setting up the team is still under way but in the meantime I wanted 
to bring your attention to a set of issues I recently reported about the 
accessibility of Django forms:

- The as_table, as_ul, as_p output styles being inappropriate, 
https://code.djangoproject.com/ticket/32338
- Markup issues with fields based on multiple radio or checkbox, 
https://code.djangoproject.com/ticket/32339
- Usability concerns with text fields that expect specific formatting, 
https://code.djangoproject.com/ticket/32340

Those issues are what I’ve come across the most while working on Django 
sites, and I think are by far what end users are most likely to run into. 
So I thought it would be worthwhile to report the problems sooner rather 
than later, and share it here for others’ consideration. I’d also 
appreciate on what makes for a good issue report for accessibility in 
particular if anyone has feedback on that area – I was quite keen to make 
more screen recordings but the ticket tracker only supports very small 
images and video seemed a bit overkill. But am happy to upload bug reports 
to YouTube if others would value this.

Best regards,

Thibaud

On Thursday, 12 November 2020 at 09:24:22 UTC carlton...@gmail.com wrote:

> Thanks all — I think we're there. 
>
> All positives expressed. Do I need to ensure those who've not 
> commented/voted do so? 
>
>
> One question while we're here: 
>
> * Do we have any specialist designers on board who could help serve on 
> this team? 
> * Related who can I ping ref the design on djangoproject.com? (Who did 
> the design work do we recall? — there are credits at the bottom of the 
> website, but I'm wondering if someone can point me straight to the 
> responsible people?)
>
> Why? Read on... 
>
> So we have this PR on to fix the color contrasts on the main website: 
> https://github.com/django/djangoproject.com/pull/998
>
> All good, but there's some question about the color palette in use. (There 
> are several shades of green, we need to adjust some, and it appears the 
> Official™ shade isn't actually in use, so...) 
> Having a specialist designer on board would enable that discussion to go 
> much more quickly and securely down the Happy Path there, without the risk 
> of death-by-a-thousand-cuts to the whole aesthetic. 
> This would be a big value-add to have available, without I think being a 
> big time commitment for the folks involved. 
>
> Kind Regards,
>
> Carlton
>
>
> On Monday, 2 November 2020 at 12:51:23 UTC+1 James Bennett wrote:
>
>> Taking this as a question posed under the DEP 10 voting process of 
>> whether or not to accept DEP 11:
>>
>> I vote +1.
>>
>> I think the basic idea is good; there are a couple things worth 
>> clarifying, mostly in terms of oversight, but that can be addressed in 
>> later edits.
>>
>> And by "oversight" I mean a clear statement of how members of the team 
>> are to be chosen, who will do the choosing, and whether the ultimate 
>> responsibility lies with the Django Technical Board or with the DSF Board 
>> of Directors. Since the scope of the DEP includes things that the Technical 
>> Board lacks authority over, it may need to be the DSF Board or a mix of 
>> both; also, the DSF Board would need to formally accept that responsibility.
>>
>>
>> On Thu, Oct 29, 2020 at 4:01 AM Carlton Gibson  
>> wrote:
>>
>>> Hello all. 
>>>
>>> Can I ask the Technical Board for Review and Resolution 
>>> 
>>>  
>>> on DEP 11 please? 
>>>
>>> https://github.com/django/deps/pull/69/
>>>
>>> (Please let me know if I've done that wrong.) 
>>>
>>> Kind regards, 
>>> Carlton
>>>
>>> On Thursday, 22 October 2020 at 10:43:03 UTC+2 Carlton Gibson wrote:
>>>
 Hi all. 

 The DEP for creating an accessibility team is looking good. 
 https://github.com/django/deps/pull/69

 I'm going to ask for formal consideration shortly. 
 If you have an interest/a moment, over the weekend say, to give it a 
 glance before that, then any comments will be gratefully received I'm 
 sure. 

 Thanks. 

 Kind Regards,

 Carlton


 On Wednesday, 16 September 2020 at 00:17:35 UTC+2 Thibaud Colas wrote:

> I’d love to! But worth bearing in mind that I haven’t done any Django 
> contributions before.
>
> I decided to end my talk with a call for feedback on your DEP – this 
> feels like the ideal call to action to end on.
>
> On Monday, 14 September 2020 at 18:00:03 UTC+1 t...@carrick.eu wrote:
>
>> Carlton, I think that would be useful, thanks.
>>
>> Thibaud, shall I add you to the implementation team? It seems like 
>> you're doing more work on this than I am lately. I think we could still 
>> use 
>> one or perhaps two more people, but I think it's a good start.
>>
>> On Mon, 14 Sep 2020 at 16:44, Carlton Gibson  
>> wrote:
>>
>