Re: Adding generated common table expressions

[Anssi Kääriäinen]

+1 for the INSERT and UPDATE parts.

For the .attach() method, I wonder if it would be better to work on having 
generic subquery support in the ORM instead of targeting CTEs specifically. 
>From Django's perspective there isn't any big difference between:
WITH fooquery AS (SELECT * FROM foo WHERE ...)
SELECT * FROM baz JOIN fooquery ON ...
and
SELECT * FROM baz JOIN (SELECT * FROM foo WHERE ...) fooquery ON ...

The great thing about the subquery form is that it works on every database, 
and to my knowledge the subquery support is surprisingly standard on all 
database Django needs to care about.

If we want full support for either of the above cases, then we need some 
way to tell to the Django's ORM what the fooquery (either as subquery or 
with CTE) produces. For simple cases this won't be complex (say, for 
.values() it's just a list of expressions), but what to do when the 
subquery/CTE contains .select_related() for example? Then we'd have a 
situation where the subquery produces a list of expressions but those 
expressions might refer to different models in the CTE.

So, the starting point should be to allow only "simple" queries in 
.attach(). This means that the attached query must be either .values() 
queryset, or a queryset containing expressions from single model only (no 
.extra(), .annotate() or select_related() added). In addition I think 
.attach() should support subqueries instead of using only CTEs. We get 
support on every backend instead of PostgreSQL only for pretty much the 
same amount of effort.

 - Anssi

On Saturday, March 18, 2017 at 11:28:53 AM UTC+2, Josh Smeaton wrote:
>
> Thanks for bringing this up Ashley, and for all of the detail you 
> provided. I'd certainly like to see CTEs make their way into Django, 
> provided we could come up with a nice enough API. From the look of it, 
> you've already got something that works with an okay API so I'm hopeful.
>
> I'd be very interested in seeing your POC too if you're able to share.
>
> From looking very briefly at django-cte-trees it doesn't aim to support 
> user defined CTEs for anything other than recursive queries. I'd be 
> interested in seeing, as part of a DEP, how CTE inclusion in django core 
> could support the cte-trees project from an API perspective.
>
> On Friday, 17 March 2017 22:28:17 UTC+11, Ashley Waite wrote:
>>
>> Hey all,
>>
>>
>> I'd like to suggest adding Common Table Expression (CTE) query generation 
>> as a feature to Django.
>>
>> I've been working on a project that required manipulation of many records 
>> at once, and as with many ORMs found that this wasn't an ideal use-case in 
>> Django. As the rest of our code base and related projects are in Django, 
>> there was a strong preference to find a way to do it and keep to the same 
>> model-is-the-truth design.
>>
>> I first did this by writing some hackish functions using raw querysets 
>> and generating my own CTE based queries, but it lacked ideal flexibility 
>> and maintainability. So I've now written some modifications into my Django 
>> to do this in a more Django-esque way and think that this functionality 
>> would be beneficial within the project itself, but am unsure exactly where 
>> to start the conversation about that.
>>
>>
>> *Why generate CTE based queries from querysets?*
>>
>> By allowing querysets to be attached to each other, and setting 
>> appropriate WHERE clauses, arbitrary and nested SQL queries can be 
>> generated. Where the results of the queries are only necessary for the 
>> execution of following queries this saves a very substantial amount of time 
>> and database work. Once these features exist, other functionality can also 
>> transparently use these to generate more efficient queries (such as large 
>> IN clauses).
>>
>> This allows several powerful use cases I think Django would benefit from:
>>
>>
>> *Large 'IN' clauses*, can be implemented as CTEs reducing expensive 
>> lookups to a single CTE INNER JOIN. For sets of thousands to match from 
>> tables of millions of records this can be a very substantial gain.
>>
>>
>> *Composite 'IN' conditions,* where multiple fields must match and you're 
>> matching against a large set of condition rows. In my usage this was "where 
>> the md5/sha hashes match one of the million md5/sha tuples in my match 
>> set". This is simply a CTE JOIN with two clauses in the WHERE.
>>
>>
>> *Nested data creation*, where the parent doesn't yet exist. Django 
>> doesn't currently do this as the primary keys are needed, and this makes 
>> normalised data structures unappealing. Using INSERTs as CTEs that supply 
>> those keys to following statements means that entire nested data structures 
>> of new information can be recreated in the database at once, efficiently 
>> and atomically.
>>
>>
>> *Non-uniform UPDATE*s, such that a modified set of objects can all be 
>> updated with different data at the same time by utilising a CTE values 
>> statement JOINed to the UPDATE statem

Re: Composite key development

[Anssi Kääriäinen]

Just my late 2 cents to this...

First, I wouldn't put too much weight on the DEP proposals. Looking back to 
them now, I don't think they went to the right direction.

For Michal Petrucha's work, it was really close to being merged some years 
ago. The problem was that migrations happened at the same time, and that 
resulted in need of substantial changes to Michal's work, and unfortunately 
this never happened. So, I'd put a lot of weight on this work.

If I'd were to take on this task, I'd try to develop the feature with the 
idea of merging the work in multiple batches. For example this might be a 
possible plan:
  1. Introduce virtual/composite field, only CREATE TABLE migrations 
support at this point, not documented as public API
  2. Add support for primary key and unique composite fields, again 
non-public
  3. Work on foreign keys, again non-public
  4. Migrations support
  5. Make the work public API (that is, document the work)

This would give a chance to get the work done in incremental pieces. Doing 
this all in one go is such a large task that there is a big risk of not 
getting this done at all.

Whatever approach you take, I really hope you'll be able to work on this 
and get this long waited feature in to Django.

 - Anssi

On Wednesday, March 1, 2017 at 7:08:35 AM UTC+2, Asif Saifuddin wrote:
>
> Hi Julien,
>
> I will publish it very soon.
>
> Thanks
>
> On Wednesday, March 1, 2017 at 5:58:50 AM UTC+6, Julien Phalip wrote:
>>
>> Hi Asif,
>>
>> That sounds good. On first look I did have some reservations about some 
>> of the design details in the current DEP, especially around the observer 
>> pattern for data binding. But I’m going to have to dig deeper into the 
>> implementation to get a clearer idea.
>>
>> It’d be great if you could publish your work-in-progress at some point so 
>> we can discuss the approach in more detail.
>>
>> Thanks,
>>
>> Julien
>>
>> On Mon, Feb 27, 2017 at 9:03 PM, Asif Saifuddin  wrote:
>>
>>> Hi Julian,
>>>
>>> I have been also reviewing and studying the previous works, deps, 
>>> discussions, codes and tickets. I have also been working to prepare a new 
>>> dep based on the previous works.
>>>
>>> Like what Michal said, from my observation, I found the works and 
>>> approaches of Michal is quite better and it's possible to break the work 
>>> down into smaller parts to implement gradually.
>>>
>>> I am not quite sure how much work or which approaches of  Thomas 
>>> Stephenson in 2015 could be mixed or needed with Michal's approach. In my 
>>> humble opinion Michal spent more time in working on this and I also found 
>>> his insights and suggestions on this topic more sensible.
>>>
>>> I would also like to work on this. My Dep is not yet ready for a push.
>>>
>>> You could certainly review and give you input on the Dep. Some core 
>>> members suggested me to finalize a Dep before div into code.
>>>
>>> Let me know what your thoughts.
>>>
>>> Regards,
>>>
>>> Asif
>>>
>>> I have also contact with 
>>>
>>> On Tuesday, November 29, 2016 at 6:10:38 PM UTC+6, Craig Kerstiens wrote:

 Hi all,

 My company (Citus Data) is interested in sponsoring some Django work. 
 In particular work on support for composite primary keys. From what I 
 understand this wouldn't be the first time the work has been explored and 
 it sounds like it has a number of intricacies to it (
 https://github.com/django/deps/blob/master/draft/0191-composite-fields.rst 
 and 
 https://github.com/django/deps/blob/master/draft/0192-standalone-composite-fields.rst).
  
 Our hope here is that it would be done in line with something that could 
 eventually become part of an official Django release vs. a one-off work 
 around. 

 While we know there's no guarantee of it being accepted, we'd love to 
 find someone with some knowledge of all the existing areas that would have 
 to be touched and has some experience contributing to Django core to 
 improve that likelihood. As part of the work, we would want the two 
 existing DEPs to be completed and work towards getting those accepted by 
 the broader community. And hopefully it goes without saying, but we'd 
 fully 
 expect all the work to done in the open similar to other Django 
 development. 

 If you're interested in doing this work please reach out as we'd love 
 to discuss further. And if we have enough interest we'll be doing a full 
 CFP for the work to try to keep the process as fair as possible. 

 ~ Craig

 -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "Django developers (Contributions to Django itself)" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to django-develop...@googlegroups.com.
>>> To post to this group, send email to django-d...@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/django-de

Re: Permission classes for Class Based Views

[Paweł Święcki]

Good idea. I use permission classes in DRF and I would use them in Django. 
Using `|` for "or" is something to be discussed, though.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/718cf38f-0106-4c1e-8909-fab3f18eeb06%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Inconsistent Behavior in Auth with UserAttributeSimilarityValidator

[Andrew Pinkham]

Hi,
I've found some inconsistent behavior in how password validation via the 
`UserAttributeSimilarityValidator` is handled by `contrib/auth`.  I did not 
find any mention of this in Trac or on the mailing list, but apologies if this 
has already been discussed.

AFAICT: When creating a user, the password is only checked in similarity to the 
`username` field. When changing a password, the password is checked in 
similarity to `username`, `first_name`, `last_name`, and `email`.

This seems undesirable - it is possible to set a password at creation time that 
might be rejected if set during a password change.

In short:
When `SetPasswordForm` calls password validation, it has all of the fields on 
the `User` instance, but when `UserCreationForm` does so, it manually adds the 
one field being checked.

In depth:
In `django/contrib/auth/forms.py`, the `SetPasswordForm` and `UserCreationForm` 
both call `password_validation.validate_password()` in the `clean_password2()` 
method. In both instances, the forms pass `password2` and `self.instance`. In 
the `UserCreationForm`, the `ModelForm` hasn't yet created the `User` instance. 
`UserCreationForm` manually adds `username` to the empty `User` instance on 
line 105 to allow for user attribute validation.

Rather than check the validity of the passwords in the `password2` clean 
method, it seems like it would be better to wait until the `User` instance has 
been created. We can use the `ModelForm` `_post_clean()` hook to achieve this.

def _post_clean(self):
super()._post_clean()  # creates self.instance
password = self.cleaned_data.get("password1")
if password:
try:
password_validation.validate_password(password, self.instance)
except ValidationError as error:
self.add_error("password1", error)

This keeps the two forms' behaviors consistent. Note that I've opted to use 
`password1` instead of `password2` because it feels more logical to show the 
problem above both passwords, rather than between the two fields.

Is there a reason *not* to make this change? Are there specific reasons for why 
we are only checking the `username` field during creation? 

Feedback appreciated. If others approve of this, I will open a PR.

Andrew
http://jambonsw.com
http://django-unleashed.com

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/878428A0-4F59-403A-A611-DD9208A605C9%40andrewsforge.com.
For more options, visit https://groups.google.com/d/optout.

Provide forms field/widget type information for use in templates

[Paweł Adamczak]

Hiya,

I would like to 'resurrect' ticket #13009 
, or at least discuss it.

I think that the suggested feature would be beneficial, as it comes up 
every time I'm implementing HTML flats different types of fields require 
different rendering (the most common example being checkboxes, where the 
input field is before the label). Adding an easy, builtin method of 
checking the field type/widget would be much nicer in my opinion then using 
then using  - just for example - this approach 
.

That question also comes up in couple of StackOverflow questions:
- 
http://stackoverflow.com/questions/3927018/django-how-to-check-if-field-widget-is-checkbox-in-the-template
 
(viewed 7594 times)
- 
http://stackoverflow.com/questions/1809874/get-type-of-django-form-widget-from-within-template
 
(viewed 15015 times)
- 
http://stackoverflow.com/questions/20424375/django-templates-how-to-know-form-field-type-and-add-any-buttons-based-on-field
 
(viewed 3633 times)

>From the ticket:

> Broadly the idea has merit, but it needs a bit of finesse - specifically, 
> what determines the "type" of the field? Do we just use the type of the 
> input element on the widget? What do we do in the case of multi-widgets?

 
As far as I understand Django tries to structure its widget structure  
based
 
on the HTML spec , and I 
would follow that convention as close as possible, with widget 'types' (the 
nomenclature would probably need to be different) like 'input_text', 
'input_email', ..., 'input_radio', 'input_checkbox', and 'select'. In other 
cases we would just mimic the widget name ('multi_widget'). 

What do you think?
I would be happy to contribute to Django and implement the above in a 
feature pull request.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/02725db5-3873-4651-9f2e-01c7b2af9f85%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Provide forms field/widget type information for use in templates

[Collin Anderson]

Hi,

If it helps It looks like there's a widget.input_type attribute, so you can
at least do {% if theform.thefield.field.widget.input_type == "checkbox" %}
etc.

Collin


On Mon, Mar 20, 2017 at 9:46 AM, Paweł Adamczak  wrote:

> Hiya,
>
> I would like to 'resurrect' ticket #13009
> , or at least discuss it.
>
> I think that the suggested feature would be beneficial, as it comes up
> every time I'm implementing HTML flats different types of fields require
> different rendering (the most common example being checkboxes, where the
> input field is before the label). Adding an easy, builtin method of
> checking the field type/widget would be much nicer in my opinion then using
> then using  - just for example - this approach
> .
>
> That question also comes up in couple of StackOverflow questions:
> - http://stackoverflow.com/questions/3927018/django-how-
> to-check-if-field-widget-is-checkbox-in-the-template (viewed 7594 times)
> - http://stackoverflow.com/questions/1809874/get-type-of-
> django-form-widget-from-within-template (viewed 15015 times)
> - http://stackoverflow.com/questions/20424375/django-
> templates-how-to-know-form-field-type-and-add-any-buttons-based-on-field
> (viewed 3633 times)
>
> From the ticket:
>
>> Broadly the idea has merit, but it needs a bit of finesse - specifically,
>> what determines the "type" of the field? Do we just use the type of the
>> input element on the widget? What do we do in the case of multi-widgets?
>
>
> As far as I understand Django tries to structure its widget structure
> based
> on the HTML spec , and
> I would follow that convention as close as possible, with widget 'types'
> (the nomenclature would probably need to be different) like 'input_text',
> 'input_email', ..., 'input_radio', 'input_checkbox', and 'select'. In other
> cases we would just mimic the widget name ('multi_widget').
>
> What do you think?
> I would be happy to contribute to Django and implement the above in a
> feature pull request.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/django-developers/02725db5-3873-4651-9f2e-
> 01c7b2af9f85%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFO84S4wndx1sKYf9hnQsytCZeLgjGBeJZJM94xr-SJazZx-4Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Provide forms field/widget type information for use in templates

[Paweł Adamczak]

Huh, don't know how I missed that. Thanks!

In that case, I would add the information to the docs, to the ticket, and 
close it : -)

On Monday, March 20, 2017 at 4:36:17 PM UTC, Collin Anderson wrote:
>
> Hi,
>
> If it helps It looks like there's a widget.input_type attribute, so you 
> can at least do {% if theform.thefield.field.widget.input_type == 
> "checkbox" %} etc. 
>
> Collin
>
>
> On Mon, Mar 20, 2017 at 9:46 AM, Paweł Adamczak  > wrote:
>
>> Hiya,
>>
>> I would like to 'resurrect' ticket #13009 
>> , or at least discuss it.
>>
>> I think that the suggested feature would be beneficial, as it comes up 
>> every time I'm implementing HTML flats different types of fields require 
>> different rendering (the most common example being checkboxes, where the 
>> input field is before the label). Adding an easy, builtin method of 
>> checking the field type/widget would be much nicer in my opinion then using 
>> then using  - just for example - this approach 
>> .
>>
>> That question also comes up in couple of StackOverflow questions:
>> - 
>> http://stackoverflow.com/questions/3927018/django-how-to-check-if-field-widget-is-checkbox-in-the-template
>>  
>> (viewed 7594 times)
>> - 
>> http://stackoverflow.com/questions/1809874/get-type-of-django-form-widget-from-within-template
>>  
>> (viewed 15015 times)
>> - 
>> http://stackoverflow.com/questions/20424375/django-templates-how-to-know-form-field-type-and-add-any-buttons-based-on-field
>>  
>> (viewed 3633 times)
>>
>> From the ticket:
>>
>>> Broadly the idea has merit, but it needs a bit of finesse - 
>>> specifically, what determines the "type" of the field? Do we just use the 
>>> type of the input element on the widget? What do we do in the case of 
>>> multi-widgets?
>>
>>  
>> As far as I understand Django tries to structure its widget structure  
>> based
>>  
>> on the HTML spec , 
>> and I would follow that convention as close as possible, with widget 
>> 'types' (the nomenclature would probably need to be different) like 
>> 'input_text', 'input_email', ..., 'input_radio', 'input_checkbox', and 
>> 'select'. In other cases we would just mimic the widget name 
>> ('multi_widget'). 
>>
>> What do you think?
>> I would be happy to contribute to Django and implement the above in a 
>> feature pull request.
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-develop...@googlegroups.com .
>> To post to this group, send email to django-d...@googlegroups.com 
>> .
>> Visit this group at https://groups.google.com/group/django-developers.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/02725db5-3873-4651-9f2e-01c7b2af9f85%40googlegroups.com
>>  
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/1301a519-acba-4d6f-a619-5da7db176d47%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

I would like to discuss my proposal for a working way to call .add() on an m2m with through= model

[Luis Masuelli]

I was reading this link in the official history 
 and this other link 
in this group 
, 
but still do not understand why the issue against a way to call .add() to 
add a through model. I thought several approaches (they are all backward 
compatible for the end-user) that could work:

   1. Avoid the restriction to call add() if the model has only the two FK 
   fields.
   2. An additional way to call .add() specifying additional fields to fill 
   (this one is not mine; was discussed in the linked thread). You risk 
   getting a (wrapped?) exception if you do not populate other fields 
   appropriately.
   3. (This one was the first I thought and perhaps the easiest to 
   implement) Allow the ManyToManyField to specify a kwarg like 
   `through_factory=` which expects a callable which would populate more data. 
   The restriction to call .add() would remain if no `through_factory=` is 
   specified.
   4. Avoid the restriction to call delete() if the model has only the two 
   FK fields. 
   
I considered these cases:

   - It is quite trivial a model with only two fields, but perhaps the 
   intermediate models could have additional useful methods. I see no trouble 
   having such model and allowing .add() or .delete() calls there.
   - Having a special factory method would allow calls to .add() since we'd 
   be providing a way to make .add() actually know how to create an instance 
   of the intermediate model.
   - You can combine these points, implement one, none, or all of them.
   - (I did not consider extended use cases for delete() intentionally, 
   since perhaps a strange model (with different field pairs) could fit in 
   different relationships, although I cannot think in a case with multiple 
   relationships not incurrin in, perhaps, duplicate data when tried to be 
   used as through= model, but anyway I prefer to keep silence for other cases 
   involving delete(), but the case I stated).
   - It is up to the user to be careful regarding migrating an intermediate 
   model regarding adding, changing, or deleting fields. But anyway, this 
   applies to any modification in the database models right now.
   - Points 1, 2, 3 and/or 4 could be implemented with no clash. A combined 
   approach of 1, 2, 3 would look like this (this flow only applies for the 
   case when the through= is present - such scenario right now only consists 
   of raising an exception; the case with no through= model would not be 
   affected at all):
  - Instantiate `instance = ThroughModel(fka=a, fkb=b, 
  **kwargs_from_add)` with the respective model instances a and b, from 
  classes A and B which hold the desired M2M relationship. In this case, 
the 
  point 2 just adds the **kwargs_from_add. If point 2 is not implemented, 
no 
  **kwargs_from_add would be present.
  - (*If point 3 is implemented*) Call the callable in 
  `through_factory=` invoking it `like_this(instance)`, if the callable is 
  present. It is expected to save the instance.
  - (*If either point 1 is implemented and the model has only two 
  fields, or point 2 is implemented*) Manually save the instance (if 
  point 3 was not implemented or it was but the factory callable was not 
  specified). (*Otherwise - point 2 not implemented AND (point 1 not 
  implemented or model with more than two fields*)) Raise the currently 
  implemented exception for the .add() method with through= specified (with 
a 
  different string message) because the through_factory was not present, 
and 
  so the framework does not know how to populate additional fields.
  - Catch-and-reraise (or don't catch at all and let them be) the error 
  for missing data in the tried-to-save model.
   - An example of the callable in point 3 would be like this (just an 
   example for, say, a game!):
  - def on_added(through_model_instance):
  through_model_instance.balance = 1000.0 #although this one could 
  be a default value at db level.
  through_model_instance.save()
  through_model_instance.achievements.create(tag=
  'joined-this-relation')
  
I understand these approaches, combined or not, could not be perfect or 
bug-free. But I'd like to discuss them instead of plainly discard them. 
AFAIK right now the calls to .add() are disallowed when through is 
specified.

Could we work or at least discuss these use cases now? The history I quoted 
seems to be pretty historic, and we're in 2017. The first thing I'd 
like to know right not (with django 1.10 alive) is the feasability of this 
feature in a future django version.

I await your comments and/or criticism ^^.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop 

Any interest in integrating Django Admin Sortable into core?

[Brandon Taylor]

Hello Django Developers,

I've been a longtime user (since before the "newforms-admin" branch) and 
proponent of Django and am the creator and maintainer of Django Admin 
Sortable: https://github.com/iambrandontaylor/django-admin-sortable

I've been fortunate enough in my career to have been able to do Django 
full-time as the former Lead Developer of The Texas Tribune, a Principal 
Developer at USA Today and as a Technical Architect at Inmar in North 
Carolina. So thank you, for such a great framework and for making so many 
years of my life so enjoyable as a developer.

If you're not familiar with Django Admin Sortable, it's a mixin-based way 
to add drag-and-drop ordering to just about any kind of object in Django 
Admin - something I needed to do on basically every Django project I ever 
built. Other popular frameworks such as Keystone.js have drag-and-drop 
ordering baked into them, so why not Django? I'm hoping you could take what 
I've done and not only integrate it, but make it better in ways I couldn't. 
I personally think it would be awesome to simply be able to add a Meta 
property to a model to enable this functionality without having to inherit 
the Mixin.

It would be an honor to be able to contribute to Django's core 
functionality after so many years of using the framework. I'm hopeful 
you'll consider my proposal.

Kindest Regards,
Brandon Taylor

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/389733b0-cccf-4359-9cc7-0d6883f9d495%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: status of 1.11 release blockers

[Tim Graham]

We're not quite down with blockers -- one was reported a few hours ago, so 
I'm postponing the RC until tomorrow.

On Saturday, March 18, 2017 at 7:49:43 PM UTC-4, Tim Graham wrote:
>
> A handful of blockers (mostly in the final stages) remain as of right now 
> [0]. We're on track to issue the release candidate on Monday.
>
> [0] 
> https://code.djangoproject.com/query?status=!closed&severity=Release%20blocker&desc=1&order=changetime
>
> On Saturday, February 11, 2017 at 5:14:34 PM UTC-5, Tim Graham wrote:
>>
>> Thank you to everyone who's tested their apps with Django 1.11. The beta 
>> release is planned for Monday, February 20. After that, only release 
>> blocking bug fixes will be accepted on the stable/1.11.x branch. There are 
>> currently no reported, unfixed regressions or other release blockers for 
>> 1.11.
>>
>> On Tuesday, January 17, 2017 at 11:20:36 AM UTC-5, Tim Graham wrote:
>>>
>>> I've reviewed and merged as much as I can (thank you to everyone who 
>>> helped!) and plan to create the stable/1.11.x branch and issue the alpha 
>>> release in approximately 7 hours.
>>>
>>> On Monday, December 26, 2016 at 1:17:01 PM UTC-5, Tom Christie wrote:

 > 
 this feature https://github.com/django/django/pull/7482 will also be a 
 very good inclusion.

 Can't see simplified URL routing being complete before the feature 
 freeze. What might be feasible is getting the URL kwarg type conversion 
 basics in, so that it can fully exist as a third party package, with full 
 support slated for 2.0.

>>>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/31d4f1b8-5a09-4c8a-8f26-41914f0eda4a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: I would like to discuss my proposal for a working way to call .add() on an m2m with through= model

[Collin Anderson]

Hi,

Check out https://code.djangoproject.com/ticket/9475

Seems like it would be fine if Django allowed add() and let any errors
about missing data bubble-up. I personally think passing in a defaults dict
(just like get_or_create does) would also be fine, but a callback seems
like overkill.

Here's a proof of concept:
https://github.com/django/django/compare/master...collinanderson:ticket9475

Collin


On Mon, Mar 20, 2017 at 5:46 PM, Luis Masuelli 
wrote:

> I was reading this link in the official history
>  and this other
> link in this group
> ,
> but still do not understand why the issue against a way to call .add() to
> add a through model. I thought several approaches (they are all backward
> compatible for the end-user) that could work:
>
>1. Avoid the restriction to call add() if the model has only the two
>FK fields.
>2. An additional way to call .add() specifying additional fields to
>fill (this one is not mine; was discussed in the linked thread). You risk
>getting a (wrapped?) exception if you do not populate other fields
>appropriately.
>3. (This one was the first I thought and perhaps the easiest to
>implement) Allow the ManyToManyField to specify a kwarg like
>`through_factory=` which expects a callable which would populate more data.
>The restriction to call .add() would remain if no `through_factory=` is
>specified.
>4. Avoid the restriction to call delete() if the model has only the
>two FK fields.
>
> I considered these cases:
>
>- It is quite trivial a model with only two fields, but perhaps the
>intermediate models could have additional useful methods. I see no trouble
>having such model and allowing .add() or .delete() calls there.
>- Having a special factory method would allow calls to .add() since
>we'd be providing a way to make .add() actually know how to create an
>instance of the intermediate model.
>- You can combine these points, implement one, none, or all of them.
>- (I did not consider extended use cases for delete() intentionally,
>since perhaps a strange model (with different field pairs) could fit in
>different relationships, although I cannot think in a case with multiple
>relationships not incurrin in, perhaps, duplicate data when tried to be
>used as through= model, but anyway I prefer to keep silence for other cases
>involving delete(), but the case I stated).
>- It is up to the user to be careful regarding migrating an
>intermediate model regarding adding, changing, or deleting fields. But
>anyway, this applies to any modification in the database models right now.
>- Points 1, 2, 3 and/or 4 could be implemented with no clash. A
>combined approach of 1, 2, 3 would look like this (this flow only applies
>for the case when the through= is present - such scenario right now only
>consists of raising an exception; the case with no through= model would not
>be affected at all):
>   - Instantiate `instance = ThroughModel(fka=a, fkb=b,
>   **kwargs_from_add)` with the respective model instances a and b, from
>   classes A and B which hold the desired M2M relationship. In this case, 
> the
>   point 2 just adds the **kwargs_from_add. If point 2 is not implemented, 
> no
>   **kwargs_from_add would be present.
>   - (*If point 3 is implemented*) Call the callable in
>   `through_factory=` invoking it `like_this(instance)`, if the callable is
>   present. It is expected to save the instance.
>   - (*If either point 1 is implemented and the model has only two
>   fields, or point 2 is implemented*) Manually save the instance (if
>   point 3 was not implemented or it was but the factory callable was not
>   specified). (*Otherwise - point 2 not implemented AND (point 1 not
>   implemented or model with more than two fields*)) Raise the
>   currently implemented exception for the .add() method with through=
>   specified (with a different string message) because the through_factory 
> was
>   not present, and so the framework does not know how to populate 
> additional
>   fields.
>   - Catch-and-reraise (or don't catch at all and let them be) the
>   error for missing data in the tried-to-save model.
>- An example of the callable in point 3 would be like this (just an
>example for, say, a game!):
>   - def on_added(through_model_instance):
>   through_model_instance.balance = 1000.0 #although this one
>   could be a default value at db level.
>   through_model_instance.save()
>   through_model_instance.achievements.create(tag='joined-this-
>   relation')
>
> I understand these approaches, combined or not, could not be perfect or
> bug-free. But I'd like to discuss them instead of plainly discard them.
> 

Re: I would like to discuss my proposal for a working way to call .add() on an m2m with through= model

[Adam Johnson]

It does seem like a somewhat arbitrary historical restriction. Collin's PoC
change is surprisingly small and simple.

Seems like it would be fine if Django allowed add() and let any errors
> about missing data bubble-up.
>

Agree, this is also a precedent from get_or_create.


> I personally think passing in a defaults dict (just like get_or_create
> does) would also be fine, but a callback seems like overkill.
>

This is a more consistent approach to the callback. One could always use
custom logic in the through model's save method, or a signal, to achieve
things that can't be done with through_defaults.

On 21 March 2017 at 00:46, Collin Anderson  wrote:

> Hi,
>
> Check out https://code.djangoproject.com/ticket/9475
>
> Seems like it would be fine if Django allowed add() and let any errors
> about missing data bubble-up. I personally think passing in a defaults dict
> (just like get_or_create does) would also be fine, but a callback seems
> like overkill.
>
> Here's a proof of concept: https://github.com/
> django/django/compare/master...collinanderson:ticket9475
>
> Collin
>
>
> On Mon, Mar 20, 2017 at 5:46 PM, Luis Masuelli 
> wrote:
>
>> I was reading this link in the official history
>>  and this other
>> link in this group
>> ,
>> but still do not understand why the issue against a way to call .add() to
>> add a through model. I thought several approaches (they are all backward
>> compatible for the end-user) that could work:
>>
>>1. Avoid the restriction to call add() if the model has only the two
>>FK fields.
>>2. An additional way to call .add() specifying additional fields to
>>fill (this one is not mine; was discussed in the linked thread). You risk
>>getting a (wrapped?) exception if you do not populate other fields
>>appropriately.
>>3. (This one was the first I thought and perhaps the easiest to
>>implement) Allow the ManyToManyField to specify a kwarg like
>>`through_factory=` which expects a callable which would populate more 
>> data.
>>The restriction to call .add() would remain if no `through_factory=` is
>>specified.
>>4. Avoid the restriction to call delete() if the model has only the
>>two FK fields.
>>
>> I considered these cases:
>>
>>- It is quite trivial a model with only two fields, but perhaps the
>>intermediate models could have additional useful methods. I see no trouble
>>having such model and allowing .add() or .delete() calls there.
>>- Having a special factory method would allow calls to .add() since
>>we'd be providing a way to make .add() actually know how to create an
>>instance of the intermediate model.
>>- You can combine these points, implement one, none, or all of them.
>>- (I did not consider extended use cases for delete() intentionally,
>>since perhaps a strange model (with different field pairs) could fit in
>>different relationships, although I cannot think in a case with multiple
>>relationships not incurrin in, perhaps, duplicate data when tried to be
>>used as through= model, but anyway I prefer to keep silence for other 
>> cases
>>involving delete(), but the case I stated).
>>- It is up to the user to be careful regarding migrating an
>>intermediate model regarding adding, changing, or deleting fields. But
>>anyway, this applies to any modification in the database models right now.
>>- Points 1, 2, 3 and/or 4 could be implemented with no clash. A
>>combined approach of 1, 2, 3 would look like this (this flow only applies
>>for the case when the through= is present - such scenario right now only
>>consists of raising an exception; the case with no through= model would 
>> not
>>be affected at all):
>>   - Instantiate `instance = ThroughModel(fka=a, fkb=b,
>>   **kwargs_from_add)` with the respective model instances a and b, from
>>   classes A and B which hold the desired M2M relationship. In this case, 
>> the
>>   point 2 just adds the **kwargs_from_add. If point 2 is not 
>> implemented, no
>>   **kwargs_from_add would be present.
>>   - (*If point 3 is implemented*) Call the callable in
>>   `through_factory=` invoking it `like_this(instance)`, if the callable 
>> is
>>   present. It is expected to save the instance.
>>   - (*If either point 1 is implemented and the model has only two
>>   fields, or point 2 is implemented*) Manually save the instance (if
>>   point 3 was not implemented or it was but the factory callable was not
>>   specified). (*Otherwise - point 2 not implemented AND (point 1 not
>>   implemented or model with more than two fields*)) Raise the
>>   currently implemented exception for the .add() method with through=
>>   specified (with a different string message) because the 
>> through_factory was
>>   not pr