Re: Password reset token safety

[Florian Apolloner]

Hi,

On Thursday, November 8, 2018 at 8:12:51 AM UTC+1, Alex Toussaint wrote:
>
> The attacker can have access to the password hash but no longer to the 
> last login. if that same attacker is exploiting a vulnerability that gets 
> patched just after (ex. Heartbleed) or has view on past data (ex. backups)
>

After patching a vulernability like Heartbleed one needs to rotate all 
leaked secrets (if in doubt, that means all); this includes the SECRET_KEY 
which immediately invalidates all tokens and logs out all users.

Cheers,
Florian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/61ac0c4b-aa39-48e5-b008-ad16dc0fcf0b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Proposal to extend support for object permission

[Carlton Gibson]

Hi Tobias, 

Thank you for your effort here. Sorry for the slowish response: it's just a 
question of getting round to it. 

The basic issue here is people what to avoid this: 

if user.has_perm('foo.change_bar', obj=bar) or 
user.has_perm('foo.change_bar'):
...

(Or they find it surprising or ...) 

Last time this came up (around #20218) there weren't very many or strong 
opinions, so I came down against a breaking change. 
(I don't mind the existing permissions: they've been the same ≈forever, so 
I see no reason to change.) 

What you've done in your two PRs is nice I think: 

* https://github.com/django/django/pull/10601
* https://github.com/django/django/pull/10600

* Added a `DefaultObjectBackend` on an opt-in basis which gives the 
alternate behaviour. 
* Added the `get_premission_object()` hook, so that can be overridden. 

Both of these have small footprints and no breaking changes. So I think +1 
yes. 

My only concern thus far is bringing out the change well enough in the 
release notes and docs. 
(Split between the two PRs I'm not sure it quite does that.) 

Kind Regards,

Carlton

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/a544987a-1a6a-4d5b-81e0-db8783f597aa%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Postgres PGCrypto in Django?

[Carlton Gibson]

Hi Peter, 

Thanks for the follow-up here. For me, optional field-level encryption 
would be a cool addition. 
 
Can I solicit some opinions: 

* Do we think this would merit a place in `contrib.postgres` (vs the 
third-party package)?
* Postgres buffs: any comment on the actual implementation?  

Kind Regards,

Carlton

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/7307627b-802e-4824-a488-d913a5890df8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Proposal to extend support for object permission

[Tobias Bengfort]

On 08/11/2018 11:43, Carlton Gibson wrote:
> My only concern thus far is bringing out the change well enough in the 
> release notes and docs. 
> (Split between the two PRs I'm not sure it quite does that.) 

My impression is that authentication backends and object permissions
have already been underdocumented before these changes. I would like to
help improving the documentation in general. However, the content and
structure of the documentation could be very different depending on
which of my changes get accepted. So my idea was to rework the
documentation afterwards.

tobias

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/0dd42320-0761-ef5d-4877-9aba8b814134%40posteo.de.
For more options, visit https://groups.google.com/d/optout.

Re: Proposal to extend support for object permission

[Carlton Gibson]

OK, makes sense. 

My current thinking is to Accept both. Perhaps you could put docs changes 
you'd make in a third PR (or ticket if you like, to discuss the outline)?
(Maybe one PR with three commits makes it easier to review as a whole.) 

Thanks for the input! 🏆

On Thursday, 8 November 2018 11:55:54 UTC+1, Tobias Bengfort wrote:
>
> On 08/11/2018 11:43, Carlton Gibson wrote: 
> > My only concern thus far is bringing out the change well enough in the 
> > release notes and docs. 
> > (Split between the two PRs I'm not sure it quite does that.) 
>
> My impression is that authentication backends and object permissions 
> have already been underdocumented before these changes. I would like to 
> help improving the documentation in general. However, the content and 
> structure of the documentation could be very different depending on 
> which of my changes get accepted. So my idea was to rework the 
> documentation afterwards. 
>
> tobias 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/747af93e-4798-47fe-a809-beec551b8162%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

CharField specified as not null on an oracle backend

[vafzal]

Dear Django Team,

The current behaviour of Django with an Oracle backend is to ignore the 
null parameter on CharFields. The implication of this is that all text 
fields are effectively optional, with the justification being that Oracle 
treats nulls and empty strings in a similar manner. I don't believe this is 
a good design approach, to re-iterate the argument from the Django tracker 
(issue 29904): 
I agree that an empty string and a null value are under most circumstances 
interchangeable, but this is being conflated with the concept of 
nullability.
Nullability has nothing to do with the representation of the null, just 
wether or not the field may be null.
i.e If a column has a not null constraint, it should not accept an empty 
string or 'Null' 
With Django's current behaviour, since declaring a field as not null has no 
effect, both 'Null' and blank strings may be inserted without issue.
I would argue that this is a broken design. How are raw queries enforced, 
or additional clients that have no knowledge of the Django's internal 
checks and conversions? 
If Django needs to do internal checks, it should use the value of 'blank' 
and not 'null' to be consistent with other backends. Oracle supports not 
null constraints, why not exploit that?

I would argue that this behaviour should be changed to match the behaviour 
on other backends, where an explicit not null constraint is added to the 
column. 
Is there any reason that this should not be done, other than the 
development effort required to add the constraint. If not would be happy to 
create a PR, provided the request is accepted.

Many Thanks,
Vackar

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/e5a0f360-2d50-46f5-bad4-52d55db5a114%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

CharField with Oracle Backend Null Behaviour

[vafzal]

Dear Django Team,

CharField on an Oracle backend has some strange behaviour. Because oracle 
treats empty strings and nulls in a similar fashion, Django simply ignore 
the null parameter (which should control nullability) on the field.

I agree that an empty string and a null value are under most circumstances 
interchangeable, but this is being conflated with the concept of 
nullability.
Nullability has nothing to do with the representation of the null, just 
wether or not the field may be null.
i.e If a column has a not null constraint, it should not accept an empty 
string or 'Null' 
With Django's current behaviour, since declaring a field as not null has no 
effect, both 'Null' and blank strings may be inserted without issue.
I would argue that this is a broken design. How are raw queries enforced, 
or additional clients that have no knowledge of the Django's internal 
checks and conversions? 
If Django needs to do internal checks, it should use the value of 'blank' 
and not 'null' to be consistent with other backends. Oracle supports not 
null constraints, why not exploit that?

If this would be accepted, would be happy to create a PR. What are your 
thoughts?

Many Thanks,
Vackar

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/3d4d214d-e1f5-4568-9382-d4f58dba4a86%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: CharField with Oracle Backend Null Behaviour

[Florian Apolloner]

Oracle treats NULL and empty varchar2 the same; so null=True/False is 
simply not possible on Oracle for CharField. I am not sure what you are 
proposing here…

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/a13fab3f-366a-4cb8-89c2-de4cc717d5b1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: CharField with Oracle Backend Null Behaviour

[Václav Řehák]

While it is probably not possible to change how Django treats this, many 
newcomers find it super confusing. To make developer experience in our 
Oracle-based project better, we started using the following workaround for 
fields which are required to be not null and not empty string:

class NonEmptyCharField(models.CharField):
"""
CharField preventing empty string and null

It workarounds two problems:

- CharField has a default value '' which allows silent save of model 
without an error even
  if we forget to set mandatory fields. This is solved by setting the 
default to None causing
 db error on not null constraint.

- The above is not sufficient for Oracle as the db backend has 
hardcoded null=True regardless
  of what we set. This is changed by empty_strings_allowed=False.
"""
empty_strings_allowed = False

def __init__(self, *args, **kwargs):
kwargs.setdefault('default', None)
super().__init__(*args, **kwargs)

Dne čtvrtek 8. listopadu 2018 13:43:25 UTC+1 Florian Apolloner napsal(a):
>
> Oracle treats NULL and empty varchar2 the same; so null=True/False is 
> simply not possible on Oracle for CharField. I am not sure what you are 
> proposing here…
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/831649d2-80d2-410d-ac4d-9304010eb6a0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: CharField with Oracle Backend Null Behaviour

[vafzal]

Hi Florian,

Thanks for getting back to me. Allow me elaborate
My main question is: what is the rationale for enforcing null value checks 
in middleware, and not delegating to DB?

My proposal would be:
- If null=False is specified, then add an explicit not null constraint at 
the db level
- When converting between python and sql, convert empty strings to nulls

This would prevent insertion of blank strings and nulls into required 
columns, regardless of the client accessing the DB - something that can 
currently only be mimicked at the middleware level, and requires all 
clients to use Django. 

Hope this makes things clearer.

Thanks,
Vackar


On Thursday, 8 November 2018 12:43:25 UTC, Florian Apolloner wrote:
>
> Oracle treats NULL and empty varchar2 the same; so null=True/False is 
> simply not possible on Oracle for CharField. I am not sure what you are 
> proposing here…
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/d10086be-7da1-4f44-bb0a-2093b4b2baeb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: CharField with Oracle Backend Null Behaviour

[vafzal]

Vaclav, this is an interesting approach. I know it's a very simple field, 
but have you though of creating an open source package for this, looks 
really useful.
Would probably call it RequiredCharField though, the double negative in 
NonEmptyCharField can make it a slightly harder to read.


On Thursday, 8 November 2018 13:10:30 UTC, Václav Řehák wrote:
>
>
> While it is probably not possible to change how Django treats this, many 
> newcomers find it super confusing. To make developer experience in our 
> Oracle-based project better, we started using the following workaround for 
> fields which are required to be not null and not empty string:
>
> class NonEmptyCharField(models.CharField):
> """
> CharField preventing empty string and null
>
> It workarounds two problems:
>
> - CharField has a default value '' which allows silent save of model 
> without an error even
>   if we forget to set mandatory fields. This is solved by setting the 
> default to None causing
>  db error on not null constraint.
>
> - The above is not sufficient for Oracle as the db backend has 
> hardcoded null=True regardless
>   of what we set. This is changed by empty_strings_allowed=False.
> """
> empty_strings_allowed = False
>
> def __init__(self, *args, **kwargs):
> kwargs.setdefault('default', None)
> super().__init__(*args, **kwargs)
>
> Dne čtvrtek 8. listopadu 2018 13:43:25 UTC+1 Florian Apolloner napsal(a):
>>
>> Oracle treats NULL and empty varchar2 the same; so null=True/False is 
>> simply not possible on Oracle for CharField. I am not sure what you are 
>> proposing here…
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/df93c3e1-6d5c-429e-a0a9-a699da571e52%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: CharField with Oracle Backend Null Behaviour

[Florian Apolloner]

On Thursday, November 8, 2018 at 3:52:01 PM UTC+1, vaf...@exscientia.co.uk 
wrote:
>
> - If null=False is specified, then add an explicit not null constraint at 
> the db level
>

As far as I understand Oracle makes no difference between null and an empty 
string. So if we were to add a not-null constraint, you could also no 
longer insert an empty string; which seems counterintuitive. Note that 
blank=True/False should be used there as validation on whether or not you 
want to allow empty values.

This would prevent insertion of blank strings and nulls into required 
> columns, regardless of the client accessing the DB
>

Insertion of blank string and null is prevent if you set blank=False and 
run model validation, isn't it?

I have to say my Oracle knowledge is somewhat limited, so please be patient 
and explain a bit more if I missunderstand you completly (code examples 
where you think the behavior is wrong would also help).

Cheers,
Florian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/cd760491-764a-41eb-b9d4-5ab8c9b6fa6d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: CharField with Oracle Backend Null Behaviour

[vafzal]

Hi Florian

No probs, most people don't understand Oracle, it's a bit of a black art. 
Some background: Partly for legacy and partly for justifiable reasons, 
oracle converts blank strings to null.
Back to this issue, as you say, if you implemented the above 
recommendations you would not be able to insert a blank string into 
required fields, but this would be 100% in line with how Oracle works 
anyway. Some DBs handle this differently, and I don't want to start a 
debate around which is better / correct, but my feeling is that one should 
go with the model as described by the DB. i.e. if using Postgres, use the 
Postgres model, when working with Oracle, use the oracle model etc. 

My main concern currently is that required fields are not enforced at the 
db level, which makes using it with other clients difficult. I would much 
prefer that constraints be added, and accept that empty strings cannot be 
inserted into required columns.

Let me know what you think,
Vackar


On Thursday, 8 November 2018 16:10:34 UTC, Florian Apolloner wrote:
>
> On Thursday, November 8, 2018 at 3:52:01 PM UTC+1, vaf...@exscientia.co.uk 
> wrote:
>>
>> - If null=False is specified, then add an explicit not null constraint at 
>> the db level
>>
>
> As far as I understand Oracle makes no difference between null and an 
> empty string. So if we were to add a not-null constraint, you could also no 
> longer insert an empty string; which seems counterintuitive. Note that 
> blank=True/False should be used there as validation on whether or not you 
> want to allow empty values.
>
> This would prevent insertion of blank strings and nulls into required 
>> columns, regardless of the client accessing the DB
>>
>
> Insertion of blank string and null is prevent if you set blank=False and 
> run model validation, isn't it?
>
> I have to say my Oracle knowledge is somewhat limited, so please be 
> patient and explain a bit more if I missunderstand you completly (code 
> examples where you think the behavior is wrong would also help).
>
> Cheers,
> Florian
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/39e60068-d08b-44a2-b37a-d855120dfaf1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: CharField with Oracle Backend Null Behaviour

[vafzal]

Vaclav,

With your approach, fields are correctly created with / without null 
constraints based on the value of the 'null' parameter.
However subsequent changes are not being picked up by the migrations 
framework. i.e. if I change null=True to null=False, constraint is not 
dropped. The same is true going from False to True, the constraint is not 
added. Did you have to monkey patch the migrations module to get it to 
work, or do you have some other approach?

Many Thanks,
Vackar

On Thursday, 8 November 2018 13:10:30 UTC, Václav Řehák wrote:
>
>
> While it is probably not possible to change how Django treats this, many 
> newcomers find it super confusing. To make developer experience in our 
> Oracle-based project better, we started using the following workaround for 
> fields which are required to be not null and not empty string:
>
> class NonEmptyCharField(models.CharField):
> """
> CharField preventing empty string and null
>
> It workarounds two problems:
>
> - CharField has a default value '' which allows silent save of model 
> without an error even
>   if we forget to set mandatory fields. This is solved by setting the 
> default to None causing
>  db error on not null constraint.
>
> - The above is not sufficient for Oracle as the db backend has 
> hardcoded null=True regardless
>   of what we set. This is changed by empty_strings_allowed=False.
> """
> empty_strings_allowed = False
>
> def __init__(self, *args, **kwargs):
> kwargs.setdefault('default', None)
> super().__init__(*args, **kwargs)
>
> Dne čtvrtek 8. listopadu 2018 13:43:25 UTC+1 Florian Apolloner napsal(a):
>>
>> Oracle treats NULL and empty varchar2 the same; so null=True/False is 
>> simply not possible on Oracle for CharField. I am not sure what you are 
>> proposing here…
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/fb28b18e-70c2-4fe8-b88d-98edd18e5f2b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: CharField with Oracle Backend Null Behaviour

[Florian Apolloner]

Hi Vackar,

Thank you, now we are getting somewhere!

On Thursday, November 8, 2018 at 5:36:53 PM UTC+1, vaf...@exscientia.co.uk 
wrote:
>
> My main concern currently is that required fields are not enforced at the 
> db level, which makes using it with other clients difficult. I would much 
> prefer that constraints be added, and accept that empty strings cannot be 
> inserted into required columns.
>

Okay, now I get you. I have no strong feelings how our oracle backend 
should behave here; other than your suggested change would be (highly?) 
backwards incompatible and that alone might prevent it from getting merged. 
Maybe Felix can chime in here with his oracle knowledge.

FWIW, Django has quite a few places where it does stuff in the framework 
itself instead of at the database level. Examples include the handling of 
database default values (we don't, defaults are in the application, even 
for simple integers and strings) as well as handling of cascades on 
deletes. If one cannot live with those issues, it is imo currently best to 
manage the database independent from Django. We are certainly open to 
improving in those areas though.

Cheers,
Florian


-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/94f68c2f-2639-4fe1-b552-a0f7a11f3509%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: CharField with Oracle Backend Null Behaviour

[vafzal]

Hi Florian,

Yes you are correct, backwards compatibility would break with this change. 
As Django already mimics the required checks in middleware for nulls, only 
difference I can see going forward would be that newly inserted blank 
strings would be read back back as None, instead of '' if nulls are 
allowed. And where nulls are not allowed, neither would make it into the db.

Look forward to hearing your thoughts Felix.

Many Thanks,
Vackar

On Thursday, 8 November 2018 17:06:05 UTC, Florian Apolloner wrote:
>
> Hi Vackar,
>
> Thank you, now we are getting somewhere!
>
> On Thursday, November 8, 2018 at 5:36:53 PM UTC+1, vaf...@exscientia.co.uk 
> wrote:
>>
>> My main concern currently is that required fields are not enforced at the 
>> db level, which makes using it with other clients difficult. I would much 
>> prefer that constraints be added, and accept that empty strings cannot be 
>> inserted into required columns.
>>
>
> Okay, now I get you. I have no strong feelings how our oracle backend 
> should behave here; other than your suggested change would be (highly?) 
> backwards incompatible and that alone might prevent it from getting merged. 
> Maybe Felix can chime in here with his oracle knowledge.
>
> FWIW, Django has quite a few places where it does stuff in the framework 
> itself instead of at the database level. Examples include the handling of 
> database default values (we don't, defaults are in the application, even 
> for simple integers and strings) as well as handling of cascades on 
> deletes. If one cannot live with those issues, it is imo currently best to 
> manage the database independent from Django. We are certainly open to 
> improving in those areas though.
>
> Cheers,
> Florian
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/b654a353-9e72-42bd-85e0-26e6bad2e89c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: CharField with Oracle Backend Null Behaviour

[Mariusz Felisiak]

Hi,

Oracle treats empty strings and NULLs in the same way, that's why we 
had to decide on some workaround that will cover both python's (other Dbs') 
cases i.e. NULLs and "" (see 
https://docs.djangoproject.com/en/2.1/ref/databases/#null-and-empty-strings). 
I don't see much value in changing current behavior from *"empty strings 
always"* to *"NULLs always"*, because it will be backward incompatible and 
IMO it doesn't change anything, still non-Oracle people will be surprised 
by this behavior.
I agree that creating a *"NOT NULL" *constraint in case when "null=False*" 
*(or even "blank=False") may be a helpful addition to the current behavior 
but I'm not sure how doable it is if we take into account migration of 
existing projects.

Best,
Mariusz

W dniu czwartek, 8 listopada 2018 18:33:52 UTC+1 użytkownik 
vaf...@exscientia.co.uk napisał:
>
> Hi Florian,
>
> Yes you are correct, backwards compatibility would break with this change. 
> As Django already mimics the required checks in middleware for nulls, only 
> difference I can see going forward would be that newly inserted blank 
> strings would be read back back as None, instead of '' if nulls are 
> allowed. And where nulls are not allowed, neither would make it into the db.
>
> Look forward to hearing your thoughts Felix.
>
> Many Thanks,
> Vackar
>
> On Thursday, 8 November 2018 17:06:05 UTC, Florian Apolloner wrote:
>>
>> Hi Vackar,
>>
>> Thank you, now we are getting somewhere!
>>
>> On Thursday, November 8, 2018 at 5:36:53 PM UTC+1, 
>> vaf...@exscientia.co.uk wrote:
>>>
>>> My main concern currently is that required fields are not enforced at 
>>> the db level, which makes using it with other clients difficult. I would 
>>> much prefer that constraints be added, and accept that empty strings cannot 
>>> be inserted into required columns.
>>>
>>
>> Okay, now I get you. I have no strong feelings how our oracle backend 
>> should behave here; other than your suggested change would be (highly?) 
>> backwards incompatible and that alone might prevent it from getting merged. 
>> Maybe Felix can chime in here with his oracle knowledge.
>>
>> FWIW, Django has quite a few places where it does stuff in the framework 
>> itself instead of at the database level. Examples include the handling of 
>> database default values (we don't, defaults are in the application, even 
>> for simple integers and strings) as well as handling of cascades on 
>> deletes. If one cannot live with those issues, it is imo currently best to 
>> manage the database independent from Django. We are certainly open to 
>> improving in those areas though.
>>
>> Cheers,
>> Florian
>>
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/403d5d50-660b-426c-a911-579d82273259%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Requiring sqlparse for sqlite introspection

[Tim Graham]

I created a ticket and a pull request to add sqlparse as a required 
dependency.
https://code.djangoproject.com/ticket/29934
https://github.com/django/django/pull/10622

On Saturday, November 3, 2018 at 4:47:30 PM UTC-4, charettes wrote:
>
> > So you want to add it to Django's install_requires even though it won't 
> be necessary if SQLite isn't used?
>
> That's my understanding and what I was advocating for.
>
> Simon
>
> Le samedi 3 novembre 2018 10:09:55 UTC-4, Tim Graham a écrit :
>>
>> So you want to add it to Django's install_requires even though it won't 
>> be necessary if SQLite isn't used? It seems okay to me. It's an extra 40k 
>> or so of disk space but that's not much compared to all the extra stuff 
>> Django comes with that every Django project doesn't necessarily use. We 
>> also have the requires_sqlparse_for_splitting feature flag which we could 
>> remove if we can assume sqlparse is installed.
>>
>> On Sunday, October 28, 2018 at 4:59:54 AM UTC-4, Adam Johnson wrote:
>>>
>>> I'm fine with adding it as a dependency, my experience has been that 
>>> it's a stable, well-maintained package over the past few years I've used it.
>>>
>>> On Sun, 28 Oct 2018 at 05:00, charettes  wrote:
>>>
 After a bit of work to minimize the cases where sqlparse would be a 
 required at runtime for SQLite to AddConstraint/RemoveConstraint 
 operations 
 [0] I came to the conclusion that it would make more sense to make 
 sqlparse 
 an hard dependency of Django 2.2.

 The two reasons backing this conclusions are

 1. Given we run the suite with sqlparse installed on CI it will be 
 really hard to prevent inadvertently breaking the promise of a soft 
 dependency on sqlparse for Meta.constraints only. I guess we could 
 have a daily CI job but that would quickly get out of hand once we have to 
 perform backport and such.

 2. There's a few instances of fragile regex parsing that could be made 
 more reliable if sqlparse was an hard dependency for SQLite. Two 
 examples are
  - 
 https://github.com/django/django/blob/f892781b957f674806a227a10c58768f66a48c07/django/db/backends/sqlite3/introspection.py#L90-L146
  - 
 https://github.com/django/django/blob/f892781b957f674806a227a10c58768f66a48c07/django/db/backends/sqlite3/schema.py#L91-L127

 Given the transition to require pytz in Django 1.11 went smoothly and 
 our needs for sqlparse are similar to provide a solid experience on SQLite 
 I'd be +1 on requiring it from Django 2.2 to a point where the lowest 
 version of SQLite we support has better introspection capabilities.

 Simon

 [0] https://github.com/django/django/pull/10572

 Le mercredi 10 octobre 2018 14:53:53 UTC-4, Tim Graham a écrit :
>
> sqlparse is already installed as part of Django's tests. The question 
> is whether sqlparse should be mandatory for SQLite users (i.e. when 
> getting 
> started with a new project an error message will say, "You must install 
> sqlparse to use SQLite" (I don't think there's a way to install it 
> automatically unless we install it for all users, regardless of database) 
> or if we should try to make it optional and only raise an error if a 
> project's migrations require it.
>
> The question is "what percentage of SQLite projects are going to end 
> up needing sqlparse based on their migrations? If it's high enough, it 
> might make for a better user experience to have users install it when 
> they're getting started instead of a lazy error when get_constraints() 
> is called."
>
> On Monday, October 8, 2018 at 3:22:32 AM UTC-4, Andrew Godwin wrote:
>>
>> Adding sqlparse into the introspection code for SQLite specifically 
>> means it's going to be a runtime dependency for anything to do with 
>> migrations.
>>
>> I'm alright having that be the case, but I do think we should make 
>> sure the tests run with SQLite as otherwise a large section of the most 
>> complicated code in migrations won't be tested properly.
>>
>> Andrew
>>
>>
>> On Mon, 8 Oct 2018, 00:59 Ian Foote,  wrote:
>>
>>> Hi all,
>>>
>>> On my pull request (https://github.com/django/django/pull/10406) 
>>> refactoring how Django creates database constraints I introduced a 
>>> dependency on sqlparse in the sqlite introspection code. This allows 
>>> Django 
>>> to correctly read information about constraints on sqlite, particularly 
>>> the 
>>> name.
>>>
>>> When reviewing (
>>> https://github.com/django/django/pull/10406#issuecomment-424542217) 
>>> Tim Graham raised the question of whether we should make sqlparse a 
>>> mandatory requirement for the sqlite tests or if we should skip those 
>>> tests 
>>> that require it. In later discussion (
>>> https://github.com/django/django/p

Re: CharField with Oracle Backend Null Behaviour

[Dan Davis]

Part of the reason to use Django is to assure some level of database
portability.  I use Oracle exclusively at work, although there are some
plans to move towards PostgreSQL.  I think blank=True should govern whether
the field may be null or not.

So, from my perspective, we should limit ourselves with tinkering with how
the backend represents such constraints at the database level.

On Thu, Nov 8, 2018 at 2:14 PM Mariusz Felisiak 
wrote:

> Hi,
>
> Oracle treats empty strings and NULLs in the same way, that's why we
> had to decide on some workaround that will cover both python's (other Dbs')
> cases i.e. NULLs and "" (see
> https://docs.djangoproject.com/en/2.1/ref/databases/#null-and-empty-strings).
> I don't see much value in changing current behavior from *"empty strings
> always"* to *"NULLs always"*, because it will be backward incompatible
> and IMO it doesn't change anything, still non-Oracle people will be
> surprised by this behavior.
> I agree that creating a *"NOT NULL" *constraint in case when
> "null=False*" *(or even "blank=False") may be a helpful addition to the
> current behavior but I'm not sure how doable it is if we take into account
> migration of existing projects.
>
> Best,
> Mariusz
>
> W dniu czwartek, 8 listopada 2018 18:33:52 UTC+1 użytkownik
> vaf...@exscientia.co.uk napisał:
>>
>> Hi Florian,
>>
>> Yes you are correct, backwards compatibility would break with this
>> change.
>> As Django already mimics the required checks in middleware for nulls,
>> only difference I can see going forward would be that newly inserted blank
>> strings would be read back back as None, instead of '' if nulls are
>> allowed. And where nulls are not allowed, neither would make it into the db.
>>
>> Look forward to hearing your thoughts Felix.
>>
>> Many Thanks,
>> Vackar
>>
>> On Thursday, 8 November 2018 17:06:05 UTC, Florian Apolloner wrote:
>>>
>>> Hi Vackar,
>>>
>>> Thank you, now we are getting somewhere!
>>>
>>> On Thursday, November 8, 2018 at 5:36:53 PM UTC+1,
>>> vaf...@exscientia.co.uk wrote:

 My main concern currently is that required fields are not enforced at
 the db level, which makes using it with other clients difficult. I would
 much prefer that constraints be added, and accept that empty strings cannot
 be inserted into required columns.

>>>
>>> Okay, now I get you. I have no strong feelings how our oracle backend
>>> should behave here; other than your suggested change would be (highly?)
>>> backwards incompatible and that alone might prevent it from getting merged.
>>> Maybe Felix can chime in here with his oracle knowledge.
>>>
>>> FWIW, Django has quite a few places where it does stuff in the framework
>>> itself instead of at the database level. Examples include the handling of
>>> database default values (we don't, defaults are in the application, even
>>> for simple integers and strings) as well as handling of cascades on
>>> deletes. If one cannot live with those issues, it is imo currently best to
>>> manage the database independent from Django. We are certainly open to
>>> improving in those areas though.
>>>
>>> Cheers,
>>> Florian
>>>
>>>
>>> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/403d5d50-660b-426c-a911-579d82273259%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFzonYaJUKr8NZBX_sPGVxMr%3DgJyk-AMhRgmVkEc%3DNjNzFvsKg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.