Re: QueryDict and ordering

2018-01-08 Thread Ole Laursen 
2018-01-05 19:24 GMT+01:00 Tim Graham :
> Preservation of dict ordering is guaranteed in Python 3.7+ so that
> officially fixes this, correct?
> https://mail.python.org/pipermail/python-dev/2017-December/151283.html

Sure, I can wait for that. Sorry for the noise, I should read that
list more often.


Ole

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CANb2Ov%2BRJHiDMjYXK64wXxmom0KCEtqRARwJkwGp3Xko60%3DcPg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Sealing or locking QuerySets -- a suggestion to prevent surprise N+1 queries.

2018-01-08 Thread Gordon Wrigley 
Regarding auto prefetch, after the discussion here I created a 
ticket https://code.djangoproject.com/ticket/28586
It didn't get much attention presumably because it was around Django 2 
release time.
I have a todo note to add tests and doco and generally get it to a mergable 
state in order to push the conversation along.
Also the pull as it stands has a little bug I have to upload the fix for.

On Friday, January 5, 2018 at 2:41:49 AM UTC, Josh Smeaton wrote:
>
> I wasn't aware of this new feature Shai, thanks for pointing it out!
>
> For this particular case I'd prefer locking to be bound to a particular 
> queryset rather than the database as a whole. I would also expect it to 
> fail loudly when accessing a non-fetched related object (book.author), 
> which can be a common cause of pain.
>
> I'm also still very interested in auto-prefetch Adam, is there any help I 
> can lend?
>
> On Thursday, 4 January 2018 17:32:19 UTC+11, Shai Berger wrote:
>>
>> Hi all, 
>>
>> Django 2.0 has a new feature[1] which allows you to say "I expect no 
>> actual 
>> database queries in this piece of code". It is not designed to stop a 
>> specific 
>> queryset from spawning requests, so getting it to do exactly what's asked 
>> for 
>> in this thread may be a little involved, but if your goal is to prevent 
>> surprise queries, I think it is actually better than sealing a single 
>> queryset. 
>>
>> HTH, 
>> Shai 
>>
>> [1] https://docs.djangoproject.com/en/2.0/topics/db/instrumentation/ 
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/fe4c44aa-bd01-4196-a04a-b8daefe49f50%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Change Local Memory Cache to Use LRU #28977

2018-01-08 Thread Grant Jenks 
Josh, it's nice to meet you here. I cited your django-lrucache-backend 
project in the original post of the Trac ticket. I'm also the author of 
DiskCache http://www.grantjenks.com/docs/diskcache/ which your project refs 
for benchmarks :) I added some benchmark data to the Trac ticket which may 
interest you.

Thank you, Josh and Adam, for the +1's. I've summarized the feedback and 
updated the ticket at https://code.djangoproject.com/ticket/28977 It's now 
marked as Accepted and Has patch. I've created a pull request 
at https://github.com/django/django/pull/9555 Please review when able.

Grant

Ps. As always the docs are excellent. Even the docs for contributing are 
extremely helpful!


On Friday, January 5, 2018 at 1:46:38 PM UTC-8, Adam Johnson wrote:
>
> I'm +1 for moving to LRU too, the eviction algorithm has always looked 
> weird to me. And Josh's library shows there are valid uses of local memory 
> caching in applications - perhaps moreso these days than when Django added 
> caching and memcached was the latest thing.
>  
>
>> You can also get a very nice bump in throughput if you eliminate the 
>> `validate_key` check
>
>
> +1 to adding an option to disable the check as well. If you're using a 
> LocMemCache in production, you probably don't care about compatibility with 
> memcached, because you'll be using it for different types of data.
>
> On 5 January 2018 at 02:53, Josh Smeaton  > wrote:
>
>> To lend some weight to this, I've implemented an LRU loc mem cache and 
>> have done some benchmarking. There are some graphs in the readme: 
>> https://github.com/kogan/django-lrucache-backend - which I've written a 
>> little about 
>> https://devblog.kogan.com/blog/a-smarter-local-memory-django-cache-backend 
>> (I don't think my particular implementation is useful to core, as it relies 
>> on a 3rd party C lib, but using OrderedDict is cool!).
>>
>> You can also get a very nice bump in throughput if you eliminate the 
>> `validate_key` check, which does a character by character check of the key 
>> to avoid issues with particular characters in memcache.
>>
>> We don't want to be promoting locmemcache too much, but that said, if we 
>> can provide a better default then we should in my opinion.
>>
>> On Friday, 5 January 2018 10:12:39 UTC+11, Grant Jenks wrote:
>>>
>>> Hi all--
>>>
>>> Long time user, first time poster, here. Thank you all for Django!
>>>
>>> The current local memory cache (locmem) in Django uses a pseudo-random 
>>> culling strategy. Rather than random, the OrderedDict data type can be used 
>>> to implement an LRU eviction policy. A prototype implementation is already 
>>> used by functools.lru_cache and Python 3 now supports 
>>> OrderedDict.move_to_end and OrderedDict.popitem to ease the implementation.
>>>
>>> I have created an example set of changes at 
>>> https://github.com/grantjenks/django/tree/ticket_28977 in commit 
>>> https://github.com/grantjenks/django/commit/b06574f6713d4b7d367d7a11e0268fb62f5fd1d1
>>>
>>> Is there a consensus as to the value of these changes?
>>>
>>> Sincerely,
>>> Grant Jenks
>>>
>>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-develop...@googlegroups.com .
>> To post to this group, send email to django-d...@googlegroups.com 
>> .
>> Visit this group at https://groups.google.com/group/django-developers.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/fdb91901-c378-4258-9201-d24a9f5f103e%40googlegroups.com
>>  
>> 
>> .
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> -- 
> Adam
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/460cee35-9eb6-426a-8847-27ee398fcaea%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Change Local Memory Cache to Use LRU #28977

2018-01-08 Thread Josh Smeaton 
Nice to meet you too. Your benchmarking code was extremely handy when I was 
profiling lru-cache-backend, so thank you!

Are you able to run the same benchmarks using this version of the cache to 
see how it performs in low/medium/high eviction scenarios? I think those 
benchmarks will be nicer than the simpler cache.get() ones you have on the 
ticket.

I'll get to reviewing that change some time this week regardless.



On Tuesday, 9 January 2018 08:58:55 UTC+11, Grant Jenks wrote:
>
> Josh, it's nice to meet you here. I cited your django-lrucache-backend 
> project in the original post of the Trac ticket. I'm also the author of 
> DiskCache http://www.grantjenks.com/docs/diskcache/ which your project 
> refs for benchmarks :) I added some benchmark data to the Trac ticket which 
> may interest you.
>
> Thank you, Josh and Adam, for the +1's. I've summarized the feedback and 
> updated the ticket at https://code.djangoproject.com/ticket/28977 It's 
> now marked as Accepted and Has patch. I've created a pull request at 
> https://github.com/django/django/pull/9555 Please review when able.
>
> Grant
>
> Ps. As always the docs are excellent. Even the docs for contributing are 
> extremely helpful!
>
>
> On Friday, January 5, 2018 at 1:46:38 PM UTC-8, Adam Johnson wrote:
>>
>> I'm +1 for moving to LRU too, the eviction algorithm has always looked 
>> weird to me. And Josh's library shows there are valid uses of local memory 
>> caching in applications - perhaps moreso these days than when Django added 
>> caching and memcached was the latest thing.
>>  
>>
>>> You can also get a very nice bump in throughput if you eliminate the 
>>> `validate_key` check
>>
>>
>> +1 to adding an option to disable the check as well. If you're using a 
>> LocMemCache in production, you probably don't care about compatibility with 
>> memcached, because you'll be using it for different types of data.
>>
>> On 5 January 2018 at 02:53, Josh Smeaton  wrote:
>>
>>> To lend some weight to this, I've implemented an LRU loc mem cache and 
>>> have done some benchmarking. There are some graphs in the readme: 
>>> https://github.com/kogan/django-lrucache-backend - which I've written a 
>>> little about 
>>> https://devblog.kogan.com/blog/a-smarter-local-memory-django-cache-backend 
>>> (I don't think my particular implementation is useful to core, as it relies 
>>> on a 3rd party C lib, but using OrderedDict is cool!).
>>>
>>> You can also get a very nice bump in throughput if you eliminate the 
>>> `validate_key` check, which does a character by character check of the key 
>>> to avoid issues with particular characters in memcache.
>>>
>>> We don't want to be promoting locmemcache too much, but that said, if we 
>>> can provide a better default then we should in my opinion.
>>>
>>> On Friday, 5 January 2018 10:12:39 UTC+11, Grant Jenks wrote:

 Hi all--

 Long time user, first time poster, here. Thank you all for Django!

 The current local memory cache (locmem) in Django uses a pseudo-random 
 culling strategy. Rather than random, the OrderedDict data type can be 
 used 
 to implement an LRU eviction policy. A prototype implementation is already 
 used by functools.lru_cache and Python 3 now supports 
 OrderedDict.move_to_end and OrderedDict.popitem to ease the implementation.

 I have created an example set of changes at 
 https://github.com/grantjenks/django/tree/ticket_28977 in commit 
 https://github.com/grantjenks/django/commit/b06574f6713d4b7d367d7a11e0268fb62f5fd1d1

 Is there a consensus as to the value of these changes?

 Sincerely,
 Grant Jenks

 -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "Django developers (Contributions to Django itself)" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to django-develop...@googlegroups.com.
>>> To post to this group, send email to django-d...@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/django-developers.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/django-developers/fdb91901-c378-4258-9201-d24a9f5f103e%40googlegroups.com
>>>  
>>> 
>>> .
>>>
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>
>>
>> -- 
>> Adam
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.goog