Re: Working towards a simpler GCBV implementation?

[Asif Saif Uddin]

IMHO, we can include vanilna views like simpler implementations for GCBV 
and may be ViewSets As well. extensively document the old iplementations to 
newer implementations, and not deprecating the current implementations that 
early. they way django regex_based url was converted to sipler path() based 
url routing, that could be a good path for this.

Best,
Asif

On Wednesday, January 11, 2017 at 6:10:22 PM UTC+6 James Bennett wrote:

> Please read Florian's message. 2.0 is not going to be "we bumped the major 
> version, now we can pile in lots of breaking changes".
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/096772bd-108d-44e6-b2a5-b3f88548f791n%40googlegroups.com.

Re: Working towards a simpler GCBV implementation?

[Asif Saif Uddin]

all good new things were disruptive in django no matter what. urls and 
paths existed but the newer approach were recommended for new projects. the 
current GCBV are not so simple.

On Tuesday, January 3, 2017 at 7:20:24 PM UTC+6 Adam Johnson wrote:

> I think this is probably too disruptive a change for Django core, 
> especially after so long with the current GCBV implementations - it would 
> require all users to rewrite their CBV's. Possibly the documentation could 
> recommend django-vanilla-views?
>
> On 3 January 2017 at 13:02, Asif Saifuddin  wrote:
>
>> Hi,
>>
>> I have started work on https://github.com/django/django/pull/7783 for 
>> converting django built in generic views to django-vanilla-views.
>>
>> I would like to hear what you think before I proceed for more.
>>
>> Thanks,
>>
>> Asif
>>
>> On Friday, September 16, 2016 at 1:37:36 AM UTC+6, Asif Saifuddin wrote:
>>>
>>> Hi Tom,
>>>
>>> I am basically +1 to see this change in the django core. The package is 
>>> 3 years old and should be tested enough. If you/other core team members 
>>> thinks that now is a good time to include it to core and deprecation of 
>>> older API, then I will be willing to work and send PR for this.
>>>
>>> Looking for others opinions.
>>>
>>> Thanks,
>>>
>>> Asif
>>>
>>> On Thursday, October 3, 2013 at 3:09:58 PM UTC+6, Tom Christie wrote:

 Hi folks,

 I recently released an alternative implementation of Django's existing 
 class based views.  The intention was to mirror the *exact* same set of 
 functionality that we currently provide, but simplify the implementation 
 and API.  It's nothing cleverer or grander than a clean re-write, but the 
 end result is *significantly* less complex.  The class hierarchy is 
 trivial, the API is much smaller, and the flow control is much more 
 obvious.

 I won't go into any more here, as there's plenty of detail in the 
 documentation:

 http://django-vanilla-views.org

 There's also useful context in the related blog post, here:

 http://dabapps.com/blog/fixing-djangos-generic-class-based-views/

 The difficult thing here really is that there's no obvious approach to 
 introducing something like this in a backwards compatible way.

 It might be that we could take an incremental approach using the 
 standard deprecation process, but given the nature of the GCBVs it'd 
 likely 
 be pretty awkward.
 There might be some bits of deprecation process we simply wouldn't be 
 able to deal with in any sensible way, and we'd certainly end up with a 
 seriously gnarly implementation during the deprecation period.

 I'd be interested in getting some opinions from folks on the following:

 * If a simpler GCBV implementation along the lines of 
 django-vanilla-views is something we think we should working towards.
 * What approaches we might be able to take to dealing with backwards 
 compatibility if we did want to do so.

 Thanks for your time.

   Tom

 -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-developers+unsubscr...@googlegroups.com.
>> To post to this group, send email to django-developers@googlegroups.com.
>> Visit this group at https://groups.google.com/group/django-developers.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/5792872e-157c-4ba6-87c1-bf0f5a07d981%40googlegroups.com
>>  
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> -- 
> Adam
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/261b0ef2-ca18-4547-b9c6-27ba96a702aen%40googlegroups.com.

Proposed change in ORM model save logic

[Barry Johnson]

Trac ticket #33191 was recently closed as a “wontfix”, but Carlton encouraged 
bringing the matter to this group.  The actual issue is a small one, but it 
seems that Django could do better than it is doing today with one small change, 
and avoid the need for a counter-intuitive "x = x" statement added to 
application code.
The claim is that Django is unnecessarily clearing the cached entry for an 
in-memory related object at a time that cache does not need to be cleared, and 
that this cache clearing can result in unwanted lazy reads down the line.  A 
one-line fix for the problem is suggested.
Many apologies in advance for the extreme length of this post.  The requested 
change is small and subtle (although quite beneficial in some cases); we must 
dive deep into code to understand WHY it is important.

Background
———
My team is developing a major retail point of sale application; it’s currently 
live in nearly 1,000 stores and we’re negotiating contracts that could bring it 
to another 10,000 stores across the US over the next three to five years before 
going world-wide.  The backend is Django and PostgreSQL.
One of the typical problems we face has to do with importing data from existing 
systems.  This data usually comes to us in large sets of flat ASCII files, 
perhaps a gigabyte or few at a time.  We have to parse this incoming data and 
fill our tables as quickly as possible.  It’s not usual for one such data 
import to create tens of millions of rows across a more than a hundred tables.  
Our processes to do this are working reasonably well.
In many cases, the incoming stream of data generates related records in several 
tables at a time.  For example, our product import populates related data 
across a dozen tables organized into parent/child hierarchies that are up to 
four levels deep.  The incoming data is grouped by product (not by functional 
data type); the rows and columns of the records for a single product will 
create ORM objects scattered across those dozen ORM models.  The top-level 
model, Product, has four child models with references to Product; and each of 
those child models may have other child tables referring back to THOSE models, 
etc.
If the database schema sounds surprisingly complex, it is.  Big box retail is 
more complex than most people realize.  Each store typically carries 30,000 to 
70,000 individual products; some are even larger.  Some of the retail chains to 
which we market our systems have more than $1 billion USD per year in sales.
To be efficient, we process this incoming data in chunks:  We may instantiate 
ORM objects representing, say, 5,000 products and all of their related child 
objects.  For example, we’ll create a new instance of a Product model, then 
instantiate the various children that reference that product.  So a very 
typical pattern is something like
 product = Product(values)
 pv = ProductVariant(product=product, **more_values)
 upc = UPC(product_variant=product_variant, **upc_values)
It’s not that simple, of course; we’re reading in sequential data that 
generates multiple instances of the children in loops, etc., but essentially 
building a list of products and the multi-level hierarchy of child objects that 
dependent upon each of those products.  We then use bulk_create() to create all 
of the top-level products in one operation, then bulk_create() the various 
lists of first-level children, then bulk_create the children of that second 
level, etc.  LOTS of bulk creates.

Prior to Django 3.2, we had to explicitly set the associated “_id” field for 
each child’s reference to a parent after the list of parents were bulk_created. 
 Using our examples above, we would see things like:
bulk_create(list_of_products)
for each variant in list_of_product_variants:
variant.product_id = variant.product.id
bulk_create(list_of_product_variants)
for each upc in list_of_upc_entries:
   upc.product_variant_id = upc.product_variant.id
bulk_create(list_of_upc_entries)
[…]
Again, this is somewhat simplifying the code, but the key takeaway is that 
older versions of Django required us to manually pick up the primary key value 
from recently created instances and set the associated “_id” value in each 
instance pointing at the recently created objects.
As expected, setting the “_id” value of a foreign key field clears the internal 
cache entry containing the reference to the parent object.  We would fully 
expect that if we said “variant.product_id = 10”, that “variant.product” would 
be considered “not yet loaded”, and a reference to variant.product would 
generate a lazy read.  We don’t disagree with that interpretation or behavior, 
and that is the existing behavior that Carlton implicitly referenced when he 
closed the ticket in question as “wontfix”.
But…

Relatively Recent Django Changes
——
In the Django 3.2 timeframe, Ticket #29497 (“Saving parent object after setting 
on child leads to an unexpected data l