Admin: Allow FilterSpecs to return Q-likes

[Andy Baker]

In a few cases I've had to do filtering in Python because it wasn't possible 
purely at the db level. (mainly in cases where I'm expected small result sets 
obviously). I'd like to see this remain possible with any future changes.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/e2310ee2-1f8b-4581-8566-0e37390b392a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Admin: Allow FilterSpecs to return Q-likes

[Andy Baker]

Actually - my recollection was faulty. As the queryset method always has to 
return a queryset (dur) I am not sure that I'm actually doing anything that 
couldn't be expressed as a Q object. I'm just doing some funky stuff to get my 
queryset in shape.

So I suppose my question is this - are there any operations that return a 
queryset that couldn't be captured in a Q object?

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/5518cd9b-5765-44c1-bccf-36824f1bf552%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: undeclared variable appearing in django/db/models/__init__.py

[Peter Inglesby]

Hi Tim,

Disappointingly, I don't think Django's doing anything off the wall here!

When you import a package's submodule, the name of that submodule is added
to the package's namespace.  This is what allows you to put `import p.m`,
and to then to use `p.m` in your code -- module m is added to package p's
namespace.

In the Django code, we end up importing django.db.models.functions (via
django.db.models.manager and django.db.models.query)  in
django/db/models/__init__.py.  `functions` gets added to the namespace of
the package django.db.models, which Python does by adding it to the
 __init__ module's locals(), which makes `functions` available as a label
in the remainder of the code.

Note that I can't actually find the chapter and verse for how this works in
the Python docs, but I've been caught out by something similar in the past!

Hope this helps,

Peter.

On 7 October 2016 at 02:10, Tim Graham  wrote:

> I'm seeing an issue while running the Django test suite that I can't
> explain.
>
> If you add print(functions) before
> from django.db.models.manager import Manager
> in https://github.com/django/django/blob/master/django/db/
> models/__init__.py#L15, it gives name 'functions' is not defined  as I
> would expect.
>
> However, if you add the same print on the line after, it gives  'django.db.models.functions' from '/home/tim/code/django/django/
> db/models/functions/__init__.py'>. A "functions" variable is magically
> defined!
>
> Why does importing django.db.models.manager have this side effect?
> manager.py imports django.db.models.query which imports
> django.db.models.functions. If I move the functions import to an inner
> import in query.py as done in https://github.com/django/django/pull/7348,
> there's no more magic "functions" variable available in
> db/models/__init__.py.
>
> Can you explain this? I checked all the "from *" imports in
> models/__init__.py to rule that out.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/django-developers/d8247cc5-e9e8-4725-9714-
> 8a30b0ebc60c%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAENJrPkNusSN5kLABmLM%2BFEvUm20-HbDiC1HA5WKBLqazmmLaQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Admin: Allow FilterSpecs to return Q-likes

[charettes]

> So I suppose my question is this - are there any operations that return a 
queryset that couldn't be captured in a Q object?

Annotations are a good example of operations that cannot be expressed in a 
Q object.

On the ticket I suggested relying on the "sticky filter" feature of the ORM 
to implement a combinable filter spec.

Sticky filters allow chained filter() calls to reuse aliases, making 
filter(lookup=1).filter(lookup=2) equivalent to filter(lookup=1, lookup=2) 
while the queryset is marked a "sticky"

They are only used internally by the ORM but I remember a ticket where 
Anssii mentioned we wight want to expose the feature through a more 
adequate API.

e.g.

with queryset.reuse_aliases() as qs:
qs = qs.filter(lookup=1).filter(lookup=2)

assert qs == queryset.filter(lookup=1, lookup=2)

This would make it easy to implement a "combinable" filter spec and allow 
users to solve the issues that have surfaced recently about how the admin 
does search and filtering through multiple joins. For example, one could 
override get_search_results() to reuse aliases easily by relying on this 
method and implementing the "combinable" filter spec class would only be a 
matter of creating a composite filter spec that reuse aliases in queryset() 
before delegating to each of its nested filters.

Simon

Le vendredi 7 octobre 2016 05:52:02 UTC-4, Andy Baker a écrit :
>
> Actually - my recollection was faulty. As the queryset method always has 
> to return a queryset (dur) I am not sure that I'm actually doing anything 
> that couldn't be expressed as a Q object. I'm just doing some funky stuff 
> to get my queryset in shape.
>
> So I suppose my question is this - are there any operations that return a 
> queryset that couldn't be captured in a Q object?
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/87229a92-860d-408c-91a5-81182cb8fb81%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

disclosing security release dates on django-announce

[Tim Graham]

The Django team proposes [0] to add the following to the security policy:

Approximately one week before public disclosure, ...
we notify django-announce [1] of the date and approximate time of the
upcoming security release. No information about the issues is given. This 
is to 
aid organizations that need to ensure they have staff available to handle 
triaging our announcement and upgrade Django as needed.

[0] https://github.com/django/django/pull/7356
[1] 
https://docs.djangoproject.com/en/dev/internals/mailing-lists/#django-announce

Feedback is welcome.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/20c490af-14ea-4822-a153-daf663c48aa5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Django ORM query syntax enhancement

[Robert Roskam]

+1 from me. I really like this approach to help making my queries more DRY.

I also like how lightweight the library is altogether. Well done!


For those looking for a direct link, here you 
go: https://github.com/Nepherhotep/django-orm-sugar



On Thursday, October 6, 2016 at 1:04:56 PM UTC-4, Alexey Zankevich wrote:
>
> Hey all,
>
> Just want to announce recent changes in Django ORM Sugar library, which 
> might be useful in future Django query syntax enhancement (if ever happens).
> 1. Now it supports indexes and slices:
>
> >>> Q.data.owner.other_pets[0].name='Fishy'
> Q(data__owner__other_pets__0__name='Fishy')
> >>> Q.tags[0] == 'thoughts'
> Q(tags__0='thoughts')
> >>> Q.tags[0:2].contains(['thoughts']) 
> Q(tags__0_2__contains=['thoughts'])
>
>
> 2. Previously, it was only possible to call defined method (like it is 
> done for *is_not_null()* function) or registered with decorator. Now if 
> you try to call unknown function of sugar Q, it will internally pass 
> function name and append it as *__functionname *to lookup field:
>
> >>> Q.user.username.contains('Rodriguez')
> Q(user__username__contains='Rodriguez')
>
>
> There is no such function "contains" anymore in sugar Q, however, it just 
> adds *__contains* to lookup field and passes parameter to it.
>
> 3. It will pass a tuple to lookup if multiple arguments passed:
>
> >>> Q.user.create_datetime.range(d1, d2)
> Q(user__create_datetime__range=(d1, d2))
>
>
> I think the library is at the final state now and isn't going to get new 
> substantial features, but responses are highly appreciated.
>
> Regards,
> Alexey
>
>
> On Sunday, August 16, 2015 at 4:18:26 PM UTC+3, Alexey Zankevich wrote:
>>
>> Hi all,
>>
>> This topic is related to the current ORM query syntax with underscores.
>> There are lots of arguing related to it, anyway it has pros and cons.
>>
>> Let's take a concrete example of querying a model:
>>
>> >>> 
>> GameSession.objects.filter(user__profile__last_login_date__gte=yesterday)
>>
>>
>> Pros:
>>
>> 1. The syntax is easy to understand
>> 2. Can be extended with custom transforms and lookups
>>
>> However, there are several cons:
>>
>> 1. Long strings is hard to read, especially if we have fields with 
>> underscores.
>> It's really easy to make a mistake by missing one:
>>
>> >>> 
>> GameSession.objects.filter(user_profile__last_login_date__gte=yesterday)
>>
>> Not easy to catch missing underscore between user and profile, is it? Even
>> though, it's not easy to say whether it should be "user_profile" 
>> attribute or
>> user.profile foreign key.
>>
>> 2. Query strings can't be reused, thus the approach violates DRY 
>> principle.
>> For example, we need to order results by last_login_date:
>>
>> >>> 
>> GameSession.objects.filter(user__profile__last_login_date__gte=yesterday) \
>> .order_by('user__profile__last_login_date')
>>
>> We can't keep user__profile_login_date as a variable as in the first part 
>> of the
>> expression we use a keyword argument, meanwhile in the second part - just 
>> a 
>> string. And thus we just have to type query path twice.
>>
>> 3. Lookup names not natural to Python language and require to be 
>> remembered or
>> looked up in documentation. For example, "__gte" or "__lte" lookups tend 
>> to be
>> confused with "ge" and "le" due to similarity to methods "__ge__" and 
>> "__le__".
>>
>> 4. Lookup keywords limited to a single argument only, very inconvenient 
>> when
>> necessary to filter objects by range.
>>
>> I was thinking a lot trying to solve those issues, keeping in mind Django
>> approaches. Finally I came up with solution to extend Q objects with dot
>> expression syntax:
>>
>> >>> GameSession.objecs.filter(Q.user.profile.last_login_date >= yesterday)
>>
>> Q is a factory instance for old-style Q objects. Accessing attribute by 
>> dot
>> returns a child factory, calling factory will instantiate old-style Q 
>> object.
>>
>> >>> Q
>> 
>>
>> >>> Q.user.profile
>> 
>>
>> >>> Q(user__name='Bob')
>> 
>>
>> It overrides operators, so comparing factory with value returns a related 
>> Q
>> object:
>>
>> >>> Q.user.name == 'Bob'
>> 
>>
>> Factory has several helper functions for lookups which aren't related to 
>> any
>> Python operators directly:
>>
>> >>> Q.user.name.icontains('Bob')
>> 
>>
>> And helper to get query path as string, which requred by order_by or
>> select_related queryset methods: 
>>
>> >>> Q.user.profile.last_login_date.get_path()
>> 'user__profile__last_login_date'
>>
>> You can check implementation and more examples here
>> https://github.com/Nepherhotep/django-orm-sugar
>>
>> How it solves issues:
>>
>> #1. Dots hard to confuse with underscores
>> #2. Query paths can be reused:
>>
>> >>> factory = Q.user.profile.last_login_date
>> >>> query = GameSession.objects.filter(factory >= yesterday)
>> >>> query = query.order_by(factory.get_path())
>>
>> #3. Not neccessary to remember most of lookup names and use comparison 
>> operators
>> instead.
>> #4. Possible to use multiple 

Re: disclosing security release dates on django-announce

[Markus Holtermann]

While we haven't decided of any particular format, you can expect the 
announcements to look a bit 
like 
https://mta.openssl.org/pipermail/openssl-announce/2016-September/76.html

/Markus

On Friday, October 7, 2016 at 4:58:00 PM UTC+2, Tim Graham wrote:
>
> The Django team proposes [0] to add the following to the security policy:
>
> Approximately one week before public disclosure, ...
> we notify django-announce [1] of the date and approximate time of the
> upcoming security release. No information about the issues is given. This 
> is to 
> aid organizations that need to ensure they have staff available to handle 
> triaging our announcement and upgrade Django as needed.
>
> [0] https://github.com/django/django/pull/7356
> [1] 
> https://docs.djangoproject.com/en/dev/internals/mailing-lists/#django-announce
>
> Feedback is welcome.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/0e0ce8fc-3a99-448c-92c6-907052b30b97%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: undeclared variable appearing in django/db/models/__init__.py

[Tim Graham]

Thanks for the pointer! I think Django is misbehaving by importing "from 
django.db.models import *" in "django/contrib/gis/db/models/__init__.py" 
while django.db.models doesn't declare an __all__, so the "functions" 
submodule can be imported there which obscures the version at 
contrib/gis/db/models/functions.py.

I updated the PR to to add __all__ as needed and that seems to fix the 
issue I was having.

https://github.com/django/django/pull/7348

On Friday, October 7, 2016 at 7:31:42 AM UTC-4, Peter Inglesby wrote:
>
> Hi Tim,
>
> Disappointingly, I don't think Django's doing anything off the wall here!
>
> When you import a package's submodule, the name of that submodule is added 
> to the package's namespace.  This is what allows you to put `import p.m`, 
> and to then to use `p.m` in your code -- module m is added to package p's 
> namespace.
>
> In the Django code, we end up importing django.db.models.functions (via 
> django.db.models.manager and django.db.models.query)  in 
> django/db/models/__init__.py.  `functions` gets added to the namespace of 
> the package django.db.models, which Python does by adding it to the 
>  __init__ module's locals(), which makes `functions` available as a label 
> in the remainder of the code.
>
> Note that I can't actually find the chapter and verse for how this works 
> in the Python docs, but I've been caught out by something similar in the 
> past!
>
> Hope this helps,
>
> Peter.
>
> On 7 October 2016 at 02:10, Tim Graham > 
> wrote:
>
>> I'm seeing an issue while running the Django test suite that I can't 
>> explain.
>>
>> If you add print(functions) before
>> from django.db.models.manager import Manager
>> in 
>> https://github.com/django/django/blob/master/django/db/models/__init__.py#L15,
>>  
>> it gives name 'functions' is not defined  as I would expect.
>>
>> However, if you add the same print on the line after, it gives > 'django.db.models.functions' from 
>> '/home/tim/code/django/django/db/models/functions/__init__.py'>. A 
>> "functions" variable is magically defined!
>>
>> Why does importing django.db.models.manager have this side effect? 
>> manager.py imports django.db.models.query which imports 
>> django.db.models.functions. If I move the functions import to an inner 
>> import in query.py as done in https://github.com/django/django/pull/7348, 
>> there's no more magic "functions" variable available in 
>> db/models/__init__.py.
>>
>> Can you explain this? I checked all the "from *" imports in 
>> models/__init__.py to rule that out.
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-develop...@googlegroups.com .
>> To post to this group, send email to django-d...@googlegroups.com 
>> .
>> Visit this group at https://groups.google.com/group/django-developers.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/d8247cc5-e9e8-4725-9714-8a30b0ebc60c%40googlegroups.com
>>  
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/4ed3ce0d-7695-4d29-8971-caad31c21140%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Form.Media: render inline css/js

[James Pic]

Nice one Matthias !

We might have something similar, in dal, discussion is undergoing:
https://github.com/yourlabs/django-autocomplete-light/issues/756#issuecomment-252368089

Currently ddf also renders json in a script tag in the middle of the form,
i think I might take the same direction as you did there too:
https://github.com/yourlabs/django-dynamic-fields/blob/master/src/ddf/form.py#L28-L56

Ideally I'd have this provided by ddf.FormMixin, but I'm currently blocked
by a sort of bug-to-fix or feature-to-add issue:
https://code.djangoproject.com/ticket/27317 but I need to recenter the
discussion there, my initial description wasn't adequate as the suggestion
made in the comment isn't actually possible - or at least, is possible, but
doesn't do the job.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CALC3Kaewwcg%2B10OPx4aXJrymgYOYiDtW8gO8k%2BQ41Q4KnjSdBw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.