Re: Don't assume that missing fields from POST data are equal to an empty string value.

[Tai Lee]

>From my perspective, this issue is about silent data loss, which is about
one of the worst bugs you can have and one that Django typically tries very
hard to avoid or fix, even if it breaks compatibility.

It is extremely easy to cause silent data loss with the behaviour being
discussed. For whatever reason, if a field is not provided in the request
data, then Django is assuming it is an empty string and overwriting
existing data.

Sure, if you're in complete control over your backend and frontend and you
explicitly tell your model forms which fields to *include* (not exclude),
you can protect yourself against *one* scenario where additional fields are
added to a model but the developer forgets to update the corresponding form
class.

But that's not the only scenario. If you *exclude* fields, then new fields
added to the model will still be included automatically, and if the form
data doesn't include them, there could be silent data loss.

And the client submitting data could be completely outside the control of
the backend developer. Either it's managed by another developer or another
team, or it's entirely 3rd party code that the backend developer doesn't
even know exists. Explicitly defining which fields to include or exclude
won't help prevent silent data loss here.

Cheers.
Tai.


On Tue, Jan 23, 2018 at 10:13 AM, Curtis Maloney 
wrote:

> On 01/22/2018 06:03 PM, Anupam Jain wrote:
>
>> Wow - I just realised that we have been losing data for sometime on our
>> web platform since there were some fields in the ModelForm that were hidden
>> and not being sent. They were all being overwritten as blank values. Thank
>> God, we use django-reversion to track changes. Will take us sometime to
>> recover the data though.
>>
>
> Just something vaguely related that this post prompted in me ...
>
> My general guide is... if you're using hidden fields, then you're probably
> doing it wrong.
>
> There are very few valid cases for passing data in hidden fields. In most
> cases you really want to exclude the fields from the model. This has the
> added benefit of protecting from a malicious user who edits the values of
> the hidden fields.
>
> A ModelForm will _only_ update fields on the model that it has fields for.
> No matter what the user - or your code [such as clean functions] - manages
> to get into the cleaned_data dict.
>
> --
> Curtis
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Django developers  (Contributions to Django itself)" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/to
> pic/django-developers/w8UKCLjOMpg/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit https://groups.google.com/d/ms
> gid/django-developers/f06e134e-f596-3938-0bdf-daea0a56d505%40tinbrain.net.
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAEJB-EOZX8BuTAXKX63zUmP3XwtaPuwSgajXF7gO%2BVBd%2BHJ2hQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Why Django render DateField type as text instead of date ?

[Ruchit Bhatt]

Hi folks,
Here i am eager to find out why django render model dateField type as text 
??



instead of 


*Is there any alternative or inbuilt method to render datefield as date ?*


This is my code

*models.py*
class HumanUser(AbstractUser):
birth_date = models.DateField(null=True, blank=True, verbose_name=u
"DOB")


*forms.py*
class profile_form(forms.ModelForm):

def __init__(self, *args, **kwargs):
kwargs.setdefault('label_suffix', '')
super(profile_form, self).__init__(*args, **kwargs)
for field in self.fields:
self.fields[field].widget.attrs.update({'class': 'form-control 
form-control-line', })

class Meta:
model = User
fields = [
'full_name',
'birth_date',
'mobile',
'bio',
'gender',
]


*HTML output*





-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/caa35f1c-3a8e-4e45-954b-14876efdf8db%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Adding a bulk_save method to models

[Neal Todd]

Hi Tom,

That's great, should be a helpful addition to core. Will follow the ticket 
and PR.

Neal

(Apologies - I hadn't spotted that you'd already referenced 
django-bulk-update in your ticket when I left my drive-by comment!)

On Monday, January 22, 2018 at 7:41:11 PM UTC, Tom Forbes wrote:
>
> Hey Neal,
>
> Thank you very much for pointing that out, I actually found out about this 
> package as I was researching the ticket - I wish I had known about this a 
> couple of years ago as it would have saved me a fair bit of CPU and brain 
> time!
>
> I think that module is a good starting point and proves that it’s 
> possible, however I think the implementation can be improved upon if we 
> bring it inside core. I worked on a small PR to add this 
> 
>  
> and the implementation was refreshingly simple. It still needs docs, a 
> couple more tests and to fix a strange error with sqlite on Windows, but 
> overall it seems like a lot of gain for a small amount of code.
>
> Tom 
>
>
> On 22 January 2018 at 15:10:53, Neal Todd (ne...@torchbox.com 
> ) wrote:
>
> Hi Tom,
>
> A built-in bulk save that's more flexible than update would certainly be 
> nice. Just in case you haven't come across it though, there is a package 
> called django-bulk-update:
>
> https://github.com/aykut/django-bulk-update
>
> I've found it very useful on a number of occassions where update isn't 
> quite enough but the loop-edit-save pattern is too slow to be convenient.
>
> Probably some useful things in there when considering the API and approach.
>
> Cheers, Neal 
>
> On Friday, January 19, 2018 at 5:49:48 PM UTC, Tom Forbes wrote: 
>>
>> Hello all,
>>
>> I’d love for some feedback on an idea I’ve been mulling around lately, 
>> namely adding a bulk_save method to Dango.
>>
>> A somewhat common pattern for some applications is to loop over a list of 
>> models, set an attribute and call save on them. This unfortunately can 
>> issue a lot of database queries which can be a significant slowdown. You 
>> can work around this by using ‘.update()’ in some cases, but not all.
>>
>> It seems it would be possible to use a CASE statement in SQL to handle 
>> bulk-updating many rows with differing values. For example:
>>
>> SomeModel.object.filter(id__in=[1,2]).update(
>> some_field=Case(
>> When(id=1, then=Value('Field value for ID=1')),
>> When(id=2, then=Value('Field value for ID=2'))
>> )
>> )
>>
>> I’ve made a ticket for this here: 
>> https://code.djangoproject.com/ticket/29037
>>
>> I managed to get a 70x performance increase using this technique on a 
>> fairly large table, and it seems it could be applicable to many projects 
>> just like bulk_create.
>>
>> The downsides to this is that it can produce very large SQL statements 
>> when updating many rows (I had MySQL complain about a 10MB statement once), 
>> but this can be overcome with batching and other optimisations (i.e the 
>> same values can use WHEN id IN (x, y, z) rather than 3 individual WHEN 
>> statements).
>>
>> I’m imagining an API very similar to bulk_create, but spend any time on a 
>> patch I thought I would ask if anyone have any feedback on this suggestion. 
>> Would this be a good addition to Dango?
>>
>>
>> --
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-develop...@googlegroups.com .
> To post to this group, send email to django-d...@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/5988d579-7843-4c42-a6f9-1e389c58ece6%40googlegroups.com
>  
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/ccf32398-57d1-427e-89de-8581cd2a52c3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

RE: Default Authorization BackEnd Denying Permissions if ObjectProvided

[Mehmet Dogan]

Thanks for the response. Do you think what Florian or I sent is a good example 
to include in the docs for the way #1?

From: Carlton Gibson
Sent: Monday, January 22, 2018 2:13 AM
To: Django developers (Contributions to Django itself)
Subject: Re: Default Authorization BackEnd Denying Permissions if ObjectProvided

Hi Mehmet, 

Sorry for the slow reply. I didn't have capacity to return to this last week. 

> Hey Carlton ... What do you say?

Well, my first thought is that I don't see any great consensus for a change 
here.
I don't see it as **my** decision to make. 

This has been open a while, however, so...

I've used this API, with Django Guardian and without. It never occurred to me 
that there was a problem with ModelBackend's behaviour. 

For me the it (i.e. the docs) says:

> I don't do object permissions so I always return `False` if you ask. 

That seems clear, safe, and unambiguous. 

I see users read the API as hierarchical. Having thought about this issue,
ultimately, I think it's a misunderstanding (and so a docs issue).

I come back to Florian's point about what the right API should be? 

`user.has_perm("for.change_bar", obj=some_bar)`

What are we offering here? 
The ability to cycle through a number of backends checking permissions, **plus**
a (moderately) simple default permissions system. That's it. 
(We're not offering the most full-featured ACL-powered goodness. That's 
deliberate.)

Is this a good API for that? I think probably yes. 

Short of looking at ≈all the major frameworks out there and seeing what they 
offer instead, I don't see a ground for change. Not currently. 

I do see two ways forward: 

* A possible change to the docs: highlight what we're doing (again) 
  — provide the example of an alternate backend, with the other behaviour. 
  
* Possible backwards compatible refactoring of the authentication and 
authorization 
  roles of the authentication backends. Right now we have a class with two 
  responsibilities, so splitting that may make sense. (It may make future steps 
  clearer.) I think that would need to be done piecemeal and in PRs, with tests 
  and docs etc to be sensibly assessed. (It's impossible to assess code on the 
  mailing list, at least for me.)
  
Given the scope of the permissions system, I'm not convinced we need to make an 
actual code change here. (i.e. I'm not convinced about need for the second 
refactoring part.) The goal is to provide a simple but extensible system. We 
have that. I don't see any limitation that can't be worked around in user code 
if it doesn't suit. 

For me, I'd close as won't fix.
https://code.djangoproject.com/ticket/20218

Kind Regards,

Carlton




-- 
You received this message because you are subscribed to a topic in the Google 
Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this topic, visit 
https://groups.google.com/d/topic/django-developers/MLWfvPPVwDk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to 
django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/1fb3ac66-8d78-4393-b3ca-83cba330bccf%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/5a6740fd.f8429d0a.f1b4e.0c3c%40mx.google.com.
For more options, visit https://groups.google.com/d/optout.

Re: Don't assume that missing fields from POST data are equal to an empty string value.

[Collin Anderson]

Maybe it would be helpful to have an option where it errors if
(non-boolean) fields are non-existent in the POST data? (Though I have some
GET-forms where I would not want that behavior) Maybe I could somehow set
allow_missing_fields = True in that case?

On Tue, Jan 23, 2018 at 4:56 AM, Tai Lee  wrote:

> From my perspective, this issue is about silent data loss, which is about
> one of the worst bugs you can have and one that Django typically tries very
> hard to avoid or fix, even if it breaks compatibility.
>
> It is extremely easy to cause silent data loss with the behaviour being
> discussed. For whatever reason, if a field is not provided in the request
> data, then Django is assuming it is an empty string and overwriting
> existing data.
>
> Sure, if you're in complete control over your backend and frontend and you
> explicitly tell your model forms which fields to *include* (not exclude),
> you can protect yourself against *one* scenario where additional fields are
> added to a model but the developer forgets to update the corresponding form
> class.
>
> But that's not the only scenario. If you *exclude* fields, then new fields
> added to the model will still be included automatically, and if the form
> data doesn't include them, there could be silent data loss.
>
> And the client submitting data could be completely outside the control of
> the backend developer. Either it's managed by another developer or another
> team, or it's entirely 3rd party code that the backend developer doesn't
> even know exists. Explicitly defining which fields to include or exclude
> won't help prevent silent data loss here.
>
> Cheers.
> Tai.
>
>
> On Tue, Jan 23, 2018 at 10:13 AM, Curtis Maloney 
> wrote:
>
>> On 01/22/2018 06:03 PM, Anupam Jain wrote:
>>
>>> Wow - I just realised that we have been losing data for sometime on our
>>> web platform since there were some fields in the ModelForm that were hidden
>>> and not being sent. They were all being overwritten as blank values. Thank
>>> God, we use django-reversion to track changes. Will take us sometime to
>>> recover the data though.
>>>
>>
>> Just something vaguely related that this post prompted in me ...
>>
>> My general guide is... if you're using hidden fields, then you're
>> probably doing it wrong.
>>
>> There are very few valid cases for passing data in hidden fields. In most
>> cases you really want to exclude the fields from the model. This has the
>> added benefit of protecting from a malicious user who edits the values of
>> the hidden fields.
>>
>> A ModelForm will _only_ update fields on the model that it has fields
>> for. No matter what the user - or your code [such as clean functions] -
>> manages to get into the cleaned_data dict.
>>
>> --
>> Curtis
>>
>> --
>> You received this message because you are subscribed to a topic in the
>> Google Groups "Django developers  (Contributions to Django itself)" group.
>> To unsubscribe from this topic, visit https://groups.google.com/d/to
>> pic/django-developers/w8UKCLjOMpg/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to
>> django-developers+unsubscr...@googlegroups.com.
>> To post to this group, send email to django-developers@googlegroups.com.
>> Visit this group at https://groups.google.com/group/django-developers.
>> To view this discussion on the web visit https://groups.google.com/d/ms
>> gid/django-developers/f06e134e-f596-3938-0bdf-daea0a56d505%40tinbrain.net
>> .
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit https://groups.google.com/d/ms
> gid/django-developers/CAEJB-EOZX8BuTAXKX63zUmP3XwtaPuwSgajXF
> 7gO%2BVBd%2BHJ2hQ%40mail.gmail.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFO84S5WXub3qofQKh%3DzfCvJLLh9foJ%3DipS0Zgv-hgzatTX%2BUA%40mail.gmail.com.
For more options, visit https://g

Re: Why Django render DateField type as text instead of date ?

[Collin Anderson]

Hi,

The history of why is here:
https://code.djangoproject.com/ticket/16630#comment:11

You can get type="date" like so on your form: birth_date
= forms.DateField(attrs={'type': 'date'}, required=False)

Thanks,
Collin


On Tue, Jan 23, 2018 at 12:43 AM, Ruchit Bhatt  wrote:

> Hi folks,
> Here i am eager to find out why django render model dateField type as text
> ??
>
> 
>
> instead of
> 
>
> *Is there any alternative or inbuilt method to render datefield as date ?*
>
>
> This is my code
>
> *models.py*
> class HumanUser(AbstractUser):
> birth_date = models.DateField(null=True, blank=True, verbose_name=
> u"DOB")
>
>
> *forms.py*
> class profile_form(forms.ModelForm):
>
> def __init__(self, *args, **kwargs):
> kwargs.setdefault('label_suffix', '')
> super(profile_form, self).__init__(*args, **kwargs)
> for field in self.fields:
> self.fields[field].widget.attrs.update({'class': 'form-control
> form-control-line', })
>
> class Meta:
> model = User
> fields = [
> 'full_name',
> 'birth_date',
> 'mobile',
> 'bio',
> 'gender',
> ]
>
>
> *HTML output*
>
>  id="id_birth_date" />
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/django-developers/caa35f1c-3a8e-4e45-954b-
> 14876efdf8db%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFO84S7iFbgnV7FqE56xvM_EuY5sYA9jXAgErxmkie8ijgxbFg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.