Re: Pendolumn

[Roger Gammans]

On Tue, 2020-09-08 at 18:01 -0700, Damiano Porta wrote:
> Hello everybody!Can I use Pendolumn (https://pendulum.eustace.io/) as
> default datetime library for my models?

You probably want to  write a custom DateTimeField subclass and
override to_python(). Pendolumn date times claim to be stdlib datetimes
(eg the doc says instance(pendulom_datetime-obj, datetime.datetime) is
True ) so conversion the other way may work fine with the existing
DateTimeField code.

See here: 
https://docs.djangoproject.com/en/3.1/howto/custom-model-fields/#converting-values-to-python-objects

Nb: This is probably a question for the django-users list not the
django-developers list.

-- 
Roger Gammans 
Gamma Science Ltd. (GB Nr. 07356014 )

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/0874de593ac3d4cdd79a8314dd2af302db31532d.camel%40gammascience.co.uk.

Re: Pendolumn

[Adam Johnson]

The Pendulum README has an example model field class:
https://github.com/sdispater/pendulum#limitations

On Thu, 10 Sep 2020 at 08:22, Roger Gammans 
wrote:

> On Tue, 2020-09-08 at 18:01 -0700, Damiano Porta wrote:
>
> Hello everybody!
> Can I use Pendolumn (https://pendulum.eustace.io/) as default datetime
> library for my models?
>
>
> You probably want to write a custom DateTimeField subclass and override
> to_python(). Pendolumn date times claim to be stdlib datetimes (eg the doc
> says instance(pendulom_datetime-obj, datetime.datetime) is True ) so
> conversion the other way may work fine with the existing DateTimeField code.
>
> See here:
> https://docs.djangoproject.com/en/3.1/howto/custom-model-fields/#converting-values-to-python-objects
>
> Nb: This is probably a question for the django-users list not the
> django-developers list.
>
> --
>
> Roger Gammans 
> Gamma Science Ltd. (GB Nr. 07356014 )
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/0874de593ac3d4cdd79a8314dd2af302db31532d.camel%40gammascience.co.uk
> 
> .
>


-- 
Adam

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAMyDDM2%3DZEaxdd%2BB%3Dkc8B-JHeQ7%3DKZAUgb9DF0Cg99rVPo6oJQ%40mail.gmail.com.

What the purpose of having function that is not working correctly?

[Alexander Lyabah]

What the purpose of having function that is not working correctly, when you 
may not have this function at all and thing is changed.

I'm talking here about function Query.__str__

Bellow I show you an example:

In [19]: str(TimelineEvent.objects.filter(id__gt=100).query)
Out[19]: 'SELECT "timeline_timelineevent"."id", 
"timeline_timelineevent"."created_on", 
"timeline_timelineevent"."modified_on", 
"timeline_timelineevent"."action_on", 
"timeline_timelineevent"."timeline_id", 
"timeline_timelineevent"."content_type_id", 
"timeline_timelineevent"."object_id", "timeline_timelineevent"."action", 
"timeline_timelineevent"."meta" FROM "timeline_timelineevent" WHERE 
"timeline_timelineevent"."id" > 100 ORDER BY 
"timeline_timelineevent"."action_on" DESC, "timeline_timelineevent"."id" 
DESC'
In [20]: Timeline.objects.raw(str(TimelineEvent.objects.filter(id__gt=100).
query))
Out[20]:  100 ORDER BY 
"timeline_timelineevent"."action_on" DESC, "timeline_timelineevent"."id" 
DESC>
In [21]: len(Timeline.objects.raw(str(TimelineEvent.objects.filter(id__gt=
100).query)))
Out[21]: 89827
In [23]: str(TimelineEvent.objects.filter(id__gt=100, created_on__gt=
datetime(2020, 1,1)).query)
Out[23]: 'SELECT "timeline_timelineevent"."id", 
"timeline_timelineevent"."created_on", 
"timeline_timelineevent"."modified_on", 
"timeline_timelineevent"."action_on", 
"timeline_timelineevent"."timeline_id", 
"timeline_timelineevent"."content_type_id", 
"timeline_timelineevent"."object_id", "timeline_timelineevent"."action", 
"timeline_timelineevent"."meta" FROM "timeline_timelineevent" WHERE 
("timeline_timelineevent"."created_on" > 2020-01-01 00:00:00+00:00 AND 
"timeline_timelineevent"."id" > 100) ORDER BY 
"timeline_timelineevent"."action_on" DESC, "timeline_timelineevent"."id" 
DESC'
In [24]: len(Timeline.objects.raw(str(TimelineEvent.objects.filter(id__gt=
100, created_on__gt=datetime(2020, 1, 1)).query)))
---
ProgrammingError Traceback (most recent call last)
/usr/local/lib/python3.5/dist-packages/django/db/backends/utils.py in 
_execute(self, sql, params, *ignored_wrapper_args)
83 else:
---> 84 return self.cursor.execute(sql, params)
85 
ProgrammingError: syntax error at or near "00"
LINE 1: ...timeline_timelineevent"."created_on" > 2020-01-01 00:00:00+0...
^

And this function is recommended on stackoverflow all other the place.

What would change if instead of 

def __str__(self):
sql, params = self.sql_with_params()
return sql % params

We will have

def __str__(self):
return '{} | {}'.format(sql, params)

In that case nobody will ever want to use it as argument for raw function.

Thank you. And please, let me know what you think about it

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/44595903-f679-4d08-abb8-80d2c8dec6e9n%40googlegroups.com.

Re: What the purpose of having function that is not working correctly?

[Adam Johnson]

>
> What the purpose of having function that is not working correctly?
>

You'll go further in life if you use more descriptive, less inflammatory
subjects.

Also it's polite to search the ticket tracker for related issues before
posting on the list. This is the relevant ticket:
https://code.djangoproject.com/ticket/25705

Query.__str__ has never been intended as more than a debugging aid. Part of
the reason it's hard to compute the *actual* SQL is because Django doesn't
do it - it's done by the database library on cursor.execute(). This is more
than string quoting - some objects have special adapters, for example JSON
values. On top of this, there's no standard method in the database
libraries for SQL interpolation - it was never specified in DB API (
https://www.python.org/dev/peps/pep-0249/ ).

psycopg2 has a mogrify() function, but the other libarries don't - it may
require calling some internal/undocumented methods. Solving this ticket
needs some research and work. Your contributions would be welcome.

On Thu, 10 Sep 2020 at 09:15, Alexander Lyabah  wrote:

>
> What the purpose of having function that is not working correctly, when
> you may not have this function at all and thing is changed.
>
> I'm talking here about function Query.__str__
>
> Bellow I show you an example:
>
> In [19]: str(TimelineEvent.objects.filter(id__gt=100).query)
> Out[19]: 'SELECT "timeline_timelineevent"."id",
> "timeline_timelineevent"."created_on",
> "timeline_timelineevent"."modified_on",
> "timeline_timelineevent"."action_on",
> "timeline_timelineevent"."timeline_id",
> "timeline_timelineevent"."content_type_id",
> "timeline_timelineevent"."object_id", "timeline_timelineevent"."action",
> "timeline_timelineevent"."meta" FROM "timeline_timelineevent" WHERE
> "timeline_timelineevent"."id" > 100 ORDER BY
> "timeline_timelineevent"."action_on" DESC, "timeline_timelineevent"."id"
> DESC'
> ​
> In [20]: Timeline.objects.raw(str(TimelineEvent.objects.filter(id__gt=100
> ).query))
> Out[20]:  "timeline_timelineevent"."created_on", "timeline_timelineevent".
> "modified_on", "timeline_timelineevent"."action_on",
> "timeline_timelineevent"."timeline_id", "timeline_timelineevent".
> "content_type_id", "timeline_timelineevent"."object_id",
> "timeline_timelineevent"."action", "timeline_timelineevent"."meta" FROM
> "timeline_timelineevent" WHERE "timeline_timelineevent"."id" > 100 ORDER
> BY "timeline_timelineevent"."action_on" DESC, "timeline_timelineevent".
> "id" DESC>
> ​
> In [21]: len(Timeline.objects.raw(str(TimelineEvent.objects.filter(id__gt=
> 100).query)))
> Out[21]: 89827
> ​
> In [23]: str(TimelineEvent.objects.filter(id__gt=100, created_on__gt=
> datetime(2020, 1,1)).query)
> Out[23]: 'SELECT "timeline_timelineevent"."id",
> "timeline_timelineevent"."created_on",
> "timeline_timelineevent"."modified_on",
> "timeline_timelineevent"."action_on",
> "timeline_timelineevent"."timeline_id",
> "timeline_timelineevent"."content_type_id",
> "timeline_timelineevent"."object_id", "timeline_timelineevent"."action",
> "timeline_timelineevent"."meta" FROM "timeline_timelineevent" WHERE
> ("timeline_timelineevent"."created_on" > 2020-01-01 00:00:00+00:00 AND
> "timeline_timelineevent"."id" > 100) ORDER BY
> "timeline_timelineevent"."action_on" DESC, "timeline_timelineevent"."id"
> DESC'
> ​
> In [24]: len(Timeline.objects.raw(str(TimelineEvent.objects.filter(id__gt=
> 100, created_on__gt=datetime(2020, 1, 1)).query)))
> --
> -
> ProgrammingError Traceback (most recent call last)
> /usr/local/lib/python3.5/dist-packages/django/db/backends/utils.py in
> _execute(self, sql, params, *ignored_wrapper_args)
> 83 else:
> ---> 84 return self.cursor.execute(sql, params)
> 85
> ​
> ProgrammingError: syntax error at or near "00"
> LINE 1: ...timeline_timelineevent"."created_on" > 2020-01-01 00:00:00+0...
> ^
>
> And this function is recommended on stackoverflow all other the place.
>
> What would change if instead of
>
> def __str__(self):
> sql, params = self.sql_with_params()
> return sql % params
>
> We will have
>
> def __str__(self):
> return '{} | {}'.format(sql, params)
>
> In that case nobody will ever want to use it as argument for raw function.
>
> Thank you. And please, let me know what you think about it
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/44595903-f679-4d08-abb8-80d2c8dec6e9n%40googlegroups.com
> 
> .
>


-- 
Adam

-- 
You received this message because you are s

Re: What the purpose of having function that is not working correctly?

[Moses Mugisha]

+1

On Thu, Sep 10, 2020, 11:35 AM Adam Johnson  wrote:

> What the purpose of having function that is not working correctly?
>>
>
> You'll go further in life if you use more descriptive, less inflammatory
> subjects.
>
> Also it's polite to search the ticket tracker for related issues before
> posting on the list. This is the relevant ticket:
> https://code.djangoproject.com/ticket/25705
>
> Query.__str__ has never been intended as more than a debugging aid. Part
> of the reason it's hard to compute the *actual* SQL is because Django
> doesn't do it - it's done by the database library on cursor.execute(). This
> is more than string quoting - some objects have special adapters, for
> example JSON values. On top of this, there's no standard method in the
> database libraries for SQL interpolation - it was never specified in DB API
> ( https://www.python.org/dev/peps/pep-0249/ ).
>
> psycopg2 has a mogrify() function, but the other libarries don't - it may
> require calling some internal/undocumented methods. Solving this ticket
> needs some research and work. Your contributions would be welcome.
>
> On Thu, 10 Sep 2020 at 09:15, Alexander Lyabah 
> wrote:
>
>>
>> What the purpose of having function that is not working correctly, when
>> you may not have this function at all and thing is changed.
>>
>> I'm talking here about function Query.__str__
>>
>> Bellow I show you an example:
>>
>> In [19]: str(TimelineEvent.objects.filter(id__gt=100).query)
>> Out[19]: 'SELECT "timeline_timelineevent"."id",
>> "timeline_timelineevent"."created_on",
>> "timeline_timelineevent"."modified_on",
>> "timeline_timelineevent"."action_on",
>> "timeline_timelineevent"."timeline_id",
>> "timeline_timelineevent"."content_type_id",
>> "timeline_timelineevent"."object_id", "timeline_timelineevent"."action",
>> "timeline_timelineevent"."meta" FROM "timeline_timelineevent" WHERE
>> "timeline_timelineevent"."id" > 100 ORDER BY
>> "timeline_timelineevent"."action_on" DESC, "timeline_timelineevent"."id"
>> DESC'
>> ​
>> In [20]: Timeline.objects.raw(str(TimelineEvent.objects.filter(id__gt=100
>> ).query))
>> Out[20]: > "timeline_timelineevent"."created_on", "timeline_timelineevent".
>> "modified_on", "timeline_timelineevent"."action_on",
>> "timeline_timelineevent"."timeline_id", "timeline_timelineevent".
>> "content_type_id", "timeline_timelineevent"."object_id",
>> "timeline_timelineevent"."action", "timeline_timelineevent"."meta" FROM
>> "timeline_timelineevent" WHERE "timeline_timelineevent"."id" > 100 ORDER
>> BY "timeline_timelineevent"."action_on" DESC, "timeline_timelineevent".
>> "id" DESC>
>> ​
>> In [21]: len(Timeline.objects.raw(str(TimelineEvent.objects.filter(id__gt
>> =100).query)))
>> Out[21]: 89827
>> ​
>> In [23]: str(TimelineEvent.objects.filter(id__gt=100, created_on__gt=
>> datetime(2020, 1,1)).query)
>> Out[23]: 'SELECT "timeline_timelineevent"."id",
>> "timeline_timelineevent"."created_on",
>> "timeline_timelineevent"."modified_on",
>> "timeline_timelineevent"."action_on",
>> "timeline_timelineevent"."timeline_id",
>> "timeline_timelineevent"."content_type_id",
>> "timeline_timelineevent"."object_id", "timeline_timelineevent"."action",
>> "timeline_timelineevent"."meta" FROM "timeline_timelineevent" WHERE
>> ("timeline_timelineevent"."created_on" > 2020-01-01 00:00:00+00:00 AND
>> "timeline_timelineevent"."id" > 100) ORDER BY
>> "timeline_timelineevent"."action_on" DESC, "timeline_timelineevent"."id"
>> DESC'
>> ​
>> In [24]: len(Timeline.objects.raw(str(TimelineEvent.objects.filter(id__gt
>> =100, created_on__gt=datetime(2020, 1, 1)).query)))
>> -
>> --
>> ProgrammingError Traceback (most recent call last)
>> /usr/local/lib/python3.5/dist-packages/django/db/backends/utils.py in
>> _execute(self, sql, params, *ignored_wrapper_args)
>> 83 else:
>> ---> 84 return self.cursor.execute(sql, params)
>> 85
>> ​
>> ProgrammingError: syntax error at or near "00"
>> LINE 1: ...timeline_timelineevent"."created_on" > 2020-01-01
>> 00:00:00+0...
>> ^
>>
>> And this function is recommended on stackoverflow all other the place.
>>
>> What would change if instead of
>>
>> def __str__(self):
>> sql, params = self.sql_with_params()
>> return sql % params
>>
>> We will have
>>
>> def __str__(self):
>> return '{} | {}'.format(sql, params)
>>
>> In that case nobody will ever want to use it as argument for raw function.
>>
>> Thank you. And please, let me know what you think about it
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to django-developers+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/django-developers/44595903-f679-4d08-abb8-80d2c8dec6e9n%40googlegroups.com
>> 

Re: What the purpose of having function that is not working correctly?

[Alexander Lyabah]

I'm sorry for not being polite. Believe my I love what you have built 
already and what to support in any way I can.

My question is exactly what is in the subject is, it is not some kind of 
clickbait. I've done some research before, and I understand the complexity 
of the task you want to accomplish in the linked ticket.

My question and concerns are more about - why we have it in the first 
place. If the intention is to debug rather than using in production - we 
can change the function in a way I suggested above, and no need to close 
the linked ticket at all.

The problem with the function is that it is actually working, but not 
always, and because of that, other people are suggesting it on 
StackOverflow, using it in prod, and may, eventually catch weird 
exceptions, which leads to a bad experience with Django in general.

Also, want to say it again, English is not my first language, and some 
words may sound not polite at all. It is not my intention, I respect all 
the work you have done with Django, and very thankful for continue working 
on it.



On Thursday, September 10, 2020 at 11:36:04 AM UTC+3 Adam Johnson wrote:

> What the purpose of having function that is not working correctly?
>>
>
> You'll go further in life if you use more descriptive, less inflammatory 
> subjects.
>
> Also it's polite to search the ticket tracker for related issues before 
> posting on the list. This is the relevant ticket: 
> https://code.djangoproject.com/ticket/25705
>
> Query.__str__ has never been intended as more than a debugging aid. Part 
> of the reason it's hard to compute the *actual* SQL is because Django 
> doesn't do it - it's done by the database library on cursor.execute(). This 
> is more than string quoting - some objects have special adapters, for 
> example JSON values. On top of this, there's no standard method in the 
> database libraries for SQL interpolation - it was never specified in DB API 
> ( https://www.python.org/dev/peps/pep-0249/ ).
>
> psycopg2 has a mogrify() function, but the other libarries don't - it may 
> require calling some internal/undocumented methods. Solving this ticket 
> needs some research and work. Your contributions would be welcome.
>
> On Thu, 10 Sep 2020 at 09:15, Alexander Lyabah  
> wrote:
>
>>
>> What the purpose of having function that is not working correctly, when 
>> you may not have this function at all and thing is changed.
>>
>> I'm talking here about function Query.__str__
>>
>> Bellow I show you an example:
>>
>> In [19]: str(TimelineEvent.objects.filter(id__gt=100).query)
>> Out[19]: 'SELECT "timeline_timelineevent"."id", 
>> "timeline_timelineevent"."created_on", 
>> "timeline_timelineevent"."modified_on", 
>> "timeline_timelineevent"."action_on", 
>> "timeline_timelineevent"."timeline_id", 
>> "timeline_timelineevent"."content_type_id", 
>> "timeline_timelineevent"."object_id", "timeline_timelineevent"."action", 
>> "timeline_timelineevent"."meta" FROM "timeline_timelineevent" WHERE 
>> "timeline_timelineevent"."id" > 100 ORDER BY 
>> "timeline_timelineevent"."action_on" DESC, "timeline_timelineevent"."id" 
>> DESC'
>> In [20]: Timeline.objects.raw(str(TimelineEvent.objects.filter(id__gt=100
>> ).query))
>> Out[20]: > "timeline_timelineevent"."created_on", "timeline_timelineevent".
>> "modified_on", "timeline_timelineevent"."action_on", 
>> "timeline_timelineevent"."timeline_id", "timeline_timelineevent".
>> "content_type_id", "timeline_timelineevent"."object_id", 
>> "timeline_timelineevent"."action", "timeline_timelineevent"."meta" FROM 
>> "timeline_timelineevent" WHERE "timeline_timelineevent"."id" > 100 ORDER 
>> BY "timeline_timelineevent"."action_on" DESC, "timeline_timelineevent".
>> "id" DESC>
>> In [21]: len(Timeline.objects.raw(str(TimelineEvent.objects.filter(id__gt
>> =100).query)))
>> Out[21]: 89827
>> In [23]: str(TimelineEvent.objects.filter(id__gt=100, created_on__gt=
>> datetime(2020, 1,1)).query)
>> Out[23]: 'SELECT "timeline_timelineevent"."id", 
>> "timeline_timelineevent"."created_on", 
>> "timeline_timelineevent"."modified_on", 
>> "timeline_timelineevent"."action_on", 
>> "timeline_timelineevent"."timeline_id", 
>> "timeline_timelineevent"."content_type_id", 
>> "timeline_timelineevent"."object_id", "timeline_timelineevent"."action", 
>> "timeline_timelineevent"."meta" FROM "timeline_timelineevent" WHERE 
>> ("timeline_timelineevent"."created_on" > 2020-01-01 00:00:00+00:00 AND 
>> "timeline_timelineevent"."id" > 100) ORDER BY 
>> "timeline_timelineevent"."action_on" DESC, "timeline_timelineevent"."id" 
>> DESC'
>> In [24]: len(Timeline.objects.raw(str(TimelineEvent.objects.filter(id__gt
>> =100, created_on__gt=datetime(2020, 1, 1)).query)))
>> -
>> --
>> ProgrammingError Traceback (most recent call last)
>> /usr/local/lib/python3.5/dist-packages/django/db/backends/utils.py in 
>> _execute(self, sql, params, *ignored_wrapper_args)
>> 83 else:
>> ---> 84

Re: What the purpose of having function that is not working correctly?

[Mariusz Felisiak]

We also have other related tickets #24803 
 and #24991. 
 Please take a look at the 
last PR  and feel-free to 
continue it, if you really want to fix these issues.

Best,
Mariusz

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/23f3ee92-19df-4279-ae59-f8ab6914e092n%40googlegroups.com.

Re: What the purpose of having function that is not working correctly?

[אורי]

On Thu, Sep 10, 2020 at 12:17 PM Alexander Lyabah 
wrote:

>
> Also, want to say it again, English is not my first language, and some
> words may sound not polite at all. It is not my intention, I respect all
> the work you have done with Django, and very thankful for continue working
> on it.
>

+1

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CABD5YeFT5emMQ%2B9h%2B9U8t%2B_cHVDcLWwyL84QNEw11p75tZV66Q%40mail.gmail.com.

Re: What the purpose of having function that is not working correctly?

[Florian Apolloner]

On Thursday, September 10, 2020 at 11:16:56 AM UTC+2 Alexander Lyabah wrote:

> The problem with the function is that it is actually working, but not 
> always, and because of that, other people are suggesting it on 
> StackOverflow, using it in prod, and may, eventually catch weird 
> exceptions, which leads to a bad experience with Django in general.
>

No, this function is never working in a useful way. It does client side 
interpolation of query params which should be done by the drivers instead, 
even when it works it is potentially dangerous. The actual problem is that 
people on StackOverflow recommend to use it. FWIW Since it is solely a 
debugging aid I'd actually break any usage of it by adding "--" to the 
start of it (or similar)
 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/75c7da85-1e69-40d6-a9e2-d901fb867e41n%40googlegroups.com.

ConditionalGetMiddleware MD5

[Francisco Couzo]

I think it would be a good idea to make ConditionalGetMiddleware use a hash 
function that's not as easy to find a collision as MD5, most probably 
SHA-256 or BLAKE2.
I don't see a problem with just changing it, it will just invalidate the 
old cache.
If there's an agreement on changing the hash function, I can make the PR.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/ff591d46-97fc-43d6-9d1c-d0ba24d7b1a8n%40googlegroups.com.

Re: ConditionalGetMiddleware MD5

[Adam Johnson]

What would this protect against?

On Thu, 10 Sep 2020 at 03:56, Francisco Couzo 
wrote:

> I think it would be a good idea to make ConditionalGetMiddleware use a
> hash function that's not as easy to find a collision as MD5, most probably
> SHA-256 or BLAKE2.
> I don't see a problem with just changing it, it will just invalidate the
> old cache.
> If there's an agreement on changing the hash function, I can make the PR.
>
>
>
>
>
>
>
>
>
> --
>
>
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
>
>
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
>
>
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/ff591d46-97fc-43d6-9d1c-d0ba24d7b1a8n%40googlegroups.com
> 
> .
>
>
> --
Adam

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAMyDDM1Db5tRDNXQEMLuX6UdOmjhUq0_-Dr_9kK%3DB5AUqsXG%2Bg%40mail.gmail.com.

Re: ConditionalGetMiddleware MD5

[Francisco Couzo]

User 1 uploads a file
User 2 downloads it, and caches it
User 1 uploads a new file to the same URL, with the same MD5 hash
User 2 will keep using the old file indefinitely

Sure, user 1 has to upload two files with the same hash on purpose

On Thu, Sep 10, 2020 at 11:07 AM Adam Johnson  wrote:

> What would this protect against?
>
> On Thu, 10 Sep 2020 at 03:56, Francisco Couzo 
> wrote:
>
>> I think it would be a good idea to make ConditionalGetMiddleware use a
>> hash function that's not as easy to find a collision as MD5, most probably
>> SHA-256 or BLAKE2.
>> I don't see a problem with just changing it, it will just invalidate the
>> old cache.
>> If there's an agreement on changing the hash function, I can make the PR.
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> --
>>
>>
>> You received this message because you are subscribed to the Google Groups
>> "Django developers (Contributions to Django itself)" group.
>>
>>
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to django-developers+unsubscr...@googlegroups.com.
>>
>>
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/django-developers/ff591d46-97fc-43d6-9d1c-d0ba24d7b1a8n%40googlegroups.com
>> 
>> .
>>
>>
>> --
> Adam
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/CAMyDDM1Db5tRDNXQEMLuX6UdOmjhUq0_-Dr_9kK%3DB5AUqsXG%2Bg%40mail.gmail.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAHx8S1vpoP9txnZrV4fTfwRJBCF2q-QtjnYpw%2BwAgGKbg4V5yQ%40mail.gmail.com.

Re: ConditionalGetMiddleware MD5

[Taymon A. Beal]

That attack doesn't work with the recommended production setup because
Django doesn't serve uploaded files in that setup.

That being said, some users might be doing that anyway since setting
up production-worthy upload hosting is such a pain, and even if they
don't, they might have other views that somehow allow users to control
the response body. So I think this should be treated as a
not-super-severe-but-still-worth-fixing security issue.

What backwards-compatibility considerations exist? Do we consider it
normal for upgrading to a different Django version to bust users'
caches? I can't immediately think of any bad consequences of changing
the hash function, apart from that one. Busting users' caches doesn't
sound that terrible, given that, even if the hash function were
changed on every release (which of course it wouldn't be; SHA-2 has
been the most generally-recommended hash function for 15 years and
there are no signs that this will change), it would still only happen
once every eight months, and it's fairly rare for anything to be
cached that long in the first place, I think.

Taymon


On Thu, Sep 10, 2020 at 1:16 PM Francisco Couzo  wrote:
>
> User 1 uploads a file
> User 2 downloads it, and caches it
> User 1 uploads a new file to the same URL, with the same MD5 hash
> User 2 will keep using the old file indefinitely
>
> Sure, user 1 has to upload two files with the same hash on purpose
>
> On Thu, Sep 10, 2020 at 11:07 AM Adam Johnson  wrote:
>>
>> What would this protect against?
>>
>> On Thu, 10 Sep 2020 at 03:56, Francisco Couzo  wrote:
>>>
>>> I think it would be a good idea to make ConditionalGetMiddleware use a hash 
>>> function that's not as easy to find a collision as MD5, most probably 
>>> SHA-256 or BLAKE2.
>>> I don't see a problem with just changing it, it will just invalidate the 
>>> old cache.
>>> If there's an agreement on changing the hash function, I can make the PR.
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> --
>>>
>>>
>>> You received this message because you are subscribed to the Google Groups 
>>> "Django developers (Contributions to Django itself)" group.
>>>
>>>
>>> To unsubscribe from this group and stop receiving emails from it, send an 
>>> email to django-developers+unsubscr...@googlegroups.com.
>>>
>>>
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/django-developers/ff591d46-97fc-43d6-9d1c-d0ba24d7b1a8n%40googlegroups.com.
>>>
>>>
>> --
>> Adam
>>
>> --
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-developers+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/CAMyDDM1Db5tRDNXQEMLuX6UdOmjhUq0_-Dr_9kK%3DB5AUqsXG%2Bg%40mail.gmail.com.
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/django-developers/CAHx8S1vpoP9txnZrV4fTfwRJBCF2q-QtjnYpw%2BwAgGKbg4V5yQ%40mail.gmail.com.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAHRQ%3D85KUrrsv%3DeY7YW%3DXb%3DeD%2BdY9-eeeQQW0YuB5ckDmUgfEg%40mail.gmail.com.

Re: ConditionalGetMiddleware MD5

[Francisco Couzo]

If changing ConditionalGetMiddleware to use SHA-256
It also might be a good to change it on FileBasedCache,
_generate_cache_key, and generate_cache_header_key too
Also, _generate_cache_key is just blindly concatenating hashes, so ['foo',
'bar'] is equal to ['fo', 'obar'], I don't know it might be a problem, but
it just doesn't looks right


On Thu, Sep 10, 2020 at 10:14 PM Taymon A. Beal 
wrote:

> That attack doesn't work with the recommended production setup because
> Django doesn't serve uploaded files in that setup.
>
> That being said, some users might be doing that anyway since setting
> up production-worthy upload hosting is such a pain, and even if they
> don't, they might have other views that somehow allow users to control
> the response body. So I think this should be treated as a
> not-super-severe-but-still-worth-fixing security issue.
>
> What backwards-compatibility considerations exist? Do we consider it
> normal for upgrading to a different Django version to bust users'
> caches? I can't immediately think of any bad consequences of changing
> the hash function, apart from that one. Busting users' caches doesn't
> sound that terrible, given that, even if the hash function were
> changed on every release (which of course it wouldn't be; SHA-2 has
> been the most generally-recommended hash function for 15 years and
> there are no signs that this will change), it would still only happen
> once every eight months, and it's fairly rare for anything to be
> cached that long in the first place, I think.
>
> Taymon
>
>
> On Thu, Sep 10, 2020 at 1:16 PM Francisco Couzo 
> wrote:
> >
> > User 1 uploads a file
> > User 2 downloads it, and caches it
> > User 1 uploads a new file to the same URL, with the same MD5 hash
> > User 2 will keep using the old file indefinitely
> >
> > Sure, user 1 has to upload two files with the same hash on purpose
> >
> > On Thu, Sep 10, 2020 at 11:07 AM Adam Johnson  wrote:
> >>
> >> What would this protect against?
> >>
> >> On Thu, 10 Sep 2020 at 03:56, Francisco Couzo 
> wrote:
> >>>
> >>> I think it would be a good idea to make ConditionalGetMiddleware use a
> hash function that's not as easy to find a collision as MD5, most probably
> SHA-256 or BLAKE2.
> >>> I don't see a problem with just changing it, it will just invalidate
> the old cache.
> >>> If there's an agreement on changing the hash function, I can make the
> PR.
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> --
> >>>
> >>>
> >>> You received this message because you are subscribed to the Google
> Groups "Django developers (Contributions to Django itself)" group.
> >>>
> >>>
> >>> To unsubscribe from this group and stop receiving emails from it, send
> an email to django-developers+unsubscr...@googlegroups.com.
> >>>
> >>>
> >>> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/ff591d46-97fc-43d6-9d1c-d0ba24d7b1a8n%40googlegroups.com
> .
> >>>
> >>>
> >> --
> >> Adam
> >>
> >> --
> >> You received this message because you are subscribed to the Google
> Groups "Django developers (Contributions to Django itself)" group.
> >> To unsubscribe from this group and stop receiving emails from it, send
> an email to django-developers+unsubscr...@googlegroups.com.
> >> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/CAMyDDM1Db5tRDNXQEMLuX6UdOmjhUq0_-Dr_9kK%3DB5AUqsXG%2Bg%40mail.gmail.com
> .
> >
> > --
> > You received this message because you are subscribed to the Google
> Groups "Django developers (Contributions to Django itself)" group.
> > To unsubscribe from this group and stop receiving emails from it, send
> an email to django-developers+unsubscr...@googlegroups.com.
> > To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/CAHx8S1vpoP9txnZrV4fTfwRJBCF2q-QtjnYpw%2BwAgGKbg4V5yQ%40mail.gmail.com
> .
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers  (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/CAHRQ%3D85KUrrsv%3DeY7YW%3DXb%3DeD%2BdY9-eeeQQW0YuB5ckDmUgfEg%40mail.gmail.com
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAHx8S1tk%2BjjKgpkOdaKyMti8DeLiLLcO6GxjHJTguc47Q-P80g%40mail.gmail.com.