The cotent types framework unreasonably limits model name length.

[Richard Campen]

Hello

I have hit what I feel is an arbitrary limit on the length of a django 
model class name. I am using the PostgreSQL backend, which smartly 
truncates table names above a certain size (normally 63 characters) which 
means in theory a table name can be of indeterminate length, as PostgreSQL 
will truncate it appropriately. However, if the model class name is greater 
than 100 characters than an error is thrown when saving the model name to 
the `django_content_type` model as the `ContentType.model` field uses a 
CharField with a limit of 100. This arbitrarily restricts the size of the 
model name when the db backend can handle it fine.

I tried to go back in time to figure out if there was any context in 
setting the `ContentType.model` field max_length at 100 chars, but it was 
made before the Django project was migrated to git.

I feel it would be best to switch this field to a TextField, as even 255 
characters seems an unreasonable limit to impose when the db backend can 
support longer names, though perhaps having a smaller (but configurable) 
max_length on the TextField would still be desirable.

What are peoples feelings on this?

Cheers,
Richard

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/26478a0a-5809-449f-b17d-d7223e2cfb3do%40googlegroups.com.

Re: The cotent types framework unreasonably limits model name length.

[Richard Campen]

I guess I have two answers to that:

1) Specifically, in this instance I am scripting the creation of a whole 
bunch of Django models as there are too many to do manually. A couple out 
of the hundred or so models have names that are just over 100 characters 
(102, 103, etc) which is where I found this limit.

2) More generally, why should Django impose any kind of limit at all on the 
model length? If python doesn't impose a limit on class names, and the db 
backend doesn't limit it, (and even the Django ORM doesn't limit it) why 
should a contrib package like content types limit it? Also to preempt the 
question, yes I need content types as it is required by other things 
running in the application.

Side note: If Django truly wishes to limit the Model Name length, doing so 
directly would be more appropriate in the model metaclass, rather than as a 
side effect of a content types.

Cheers
R


On Tuesday, 11 August 2020 15:19:00 UTC+12, Uri wrote:
>
> How can a class name be more than 100 characters? Can you give an example?
>
> A limit of 100 characters seems reasonable to me, maybe even 60 would be 
> enough.
>
> אורי
> u...@speedy.net 
>
>
> On Tue, Aug 11, 2020 at 6:06 AM Richard Campen  > wrote:
>
>> Hello
>>
>> I have hit what I feel is an arbitrary limit on the length of a django 
>> model class name. I am using the PostgreSQL backend, which smartly 
>> truncates table names above a certain size (normally 63 characters) which 
>> means in theory a table name can be of indeterminate length, as PostgreSQL 
>> will truncate it appropriately. However, if the model class name is greater 
>> than 100 characters than an error is thrown when saving the model name to 
>> the `django_content_type` model as the `ContentType.model` field uses a 
>> CharField with a limit of 100. This arbitrarily restricts the size of the 
>> model name when the db backend can handle it fine.
>>
>> I tried to go back in time to figure out if there was any context in 
>> setting the `ContentType.model` field max_length at 100 chars, but it was 
>> made before the Django project was migrated to git.
>>
>> I feel it would be best to switch this field to a TextField, as even 255 
>> characters seems an unreasonable limit to impose when the db backend can 
>> support longer names, though perhaps having a smaller (but configurable) 
>> max_length on the TextField would still be desirable.
>>
>> What are peoples feelings on this?
>>
>> Cheers,
>> Richard
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-d...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/26478a0a-5809-449f-b17d-d7223e2cfb3do%40googlegroups.com
>>  
>> <https://groups.google.com/d/msgid/django-developers/26478a0a-5809-449f-b17d-d7223e2cfb3do%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/396d07c8-060d-4d35-98eb-e5bbea502a24o%40googlegroups.com.

Re: The cotent types framework unreasonably limits model name length.

[Richard Campen]

> You should be able to work around this for now with a migration in your 
own app that uses RunSQL to alter the column for ContentType.model.

I tried this approach however the ORM level field validation still kicks as 
it still expects VARCHAR 100 even if the db column is now a different type.

On Tuesday, 11 August 2020 20:52:17 UTC+12, Adam Johnson wrote:
>
> It does seem unreasonable. I think a migration to TextField would be 
> warranted.
>
> You should be able to work around this for now with a migration in your 
> own app that uses RunSQL to alter the column for ContentType.model.
>
> On Tue, 11 Aug 2020 at 04:26, Richard Campen  > wrote:
>
>> I guess I have two answers to that:
>>
>> 1) Specifically, in this instance I am scripting the creation of a whole 
>> bunch of Django models as there are too many to do manually. A couple out 
>> of the hundred or so models have names that are just over 100 characters 
>> (102, 103, etc) which is where I found this limit.
>>
>> 2) More generally, why should Django impose any kind of limit at all on 
>> the model length? If python doesn't impose a limit on class names, and the 
>> db backend doesn't limit it, (and even the Django ORM doesn't limit it) why 
>> should a contrib package like content types limit it? Also to preempt the 
>> question, yes I need content types as it is required by other things 
>> running in the application.
>>
>> Side note: If Django truly wishes to limit the Model Name length, doing 
>> so directly would be more appropriate in the model metaclass, rather than 
>> as a side effect of a content types.
>>
>> Cheers
>> R
>>
>>
>> On Tuesday, 11 August 2020 15:19:00 UTC+12, Uri wrote:
>>>
>>> How can a class name be more than 100 characters? Can you give an 
>>> example?
>>>
>>> A limit of 100 characters seems reasonable to me, maybe even 60 would be 
>>> enough.
>>>
>>> אורי
>>> u...@speedy.net
>>>
>>>
>>> On Tue, Aug 11, 2020 at 6:06 AM Richard Campen  
>>> wrote:
>>>
>>>> Hello
>>>>
>>>> I have hit what I feel is an arbitrary limit on the length of a django 
>>>> model class name. I am using the PostgreSQL backend, which smartly 
>>>> truncates table names above a certain size (normally 63 characters) which 
>>>> means in theory a table name can be of indeterminate length, as PostgreSQL 
>>>> will truncate it appropriately. However, if the model class name is 
>>>> greater 
>>>> than 100 characters than an error is thrown when saving the model name to 
>>>> the `django_content_type` model as the `ContentType.model` field uses a 
>>>> CharField with a limit of 100. This arbitrarily restricts the size of the 
>>>> model name when the db backend can handle it fine.
>>>>
>>>> I tried to go back in time to figure out if there was any context in 
>>>> setting the `ContentType.model` field max_length at 100 chars, but it was 
>>>> made before the Django project was migrated to git.
>>>>
>>>> I feel it would be best to switch this field to a TextField, as even 
>>>> 255 characters seems an unreasonable limit to impose when the db backend 
>>>> can support longer names, though perhaps having a smaller (but 
>>>> configurable) max_length on the TextField would still be desirable.
>>>>
>>>> What are peoples feelings on this?
>>>>
>>>> Cheers,
>>>> Richard
>>>>
>>>> -- 
>>>> You received this message because you are subscribed to the Google 
>>>> Groups "Django developers (Contributions to Django itself)" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send 
>>>> an email to django-d...@googlegroups.com.
>>>> To view this discussion on the web visit 
>>>> https://groups.google.com/d/msgid/django-developers/26478a0a-5809-449f-b17d-d7223e2cfb3do%40googlegroups.com
>>>>  
>>>> <https://groups.google.com/d/msgid/django-developers/26478a0a-5809-449f-b17d-d7223e2cfb3do%40googlegroups.com?utm_medium=email&utm_source=footer>
>>>> .
>>>>
>>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-d...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/396d07c8-060d-4d35-98eb-e5bbea502a24o%40googlegroups.com
>>  
>> <https://groups.google.com/d/msgid/django-developers/396d07c8-060d-4d35-98eb-e5bbea502a24o%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>>
>
>
> -- 
> Adam
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/b3f9e0bc-85d1-407a-9a4a-2e3f338dbaa5o%40googlegroups.com.

Re: The cotent types framework unreasonably limits model name length.

[Richard Campen]

I think a problem with hashing names here is that it would break the whole 
`apps.get_model(, )` functionality that content 
types uses, as the model name stored in the db is not the actual name 
stored in the apps registry (and modifying the registry functionality feels 
like the wrong approach here).

> *contenttypes* should support all backends,  You need to remember about 
DB-restrictions, e.g. we cannot change `ContentType.model` to a `TextField` 
because TEXT columns cannot be used in unique constraints on MySQL. See similar 
discussion  
about `User.last_name`.

I think a key difference with the `User.last_name` example is that Django 
provides a way to avoid this all together by setting a custom User model. 
The ContentType model is much more tightly coupled, so unless we want to 
implement a similar way to override the ContentType model, it feels like we 
aught to be more accommodating with the max_length here.

With this in mind (and my awareness that me hitting this issue was an edge 
case already) perhaps the most straight forward solution would be to leave 
the `model` field as CharField but just set its max_length to 255. This 
would be consistent with the approach taken with `Permission.name` 
. Sort of feels like kicking 
the bucket down the road, but perhaps it is far enough down said road to no 
longer be an issue.

On Wednesday, 12 August 2020 05:51:58 UTC+12, charettes wrote:
>
> > Suffix-hashing long names like Simon suggests may not be 
> backwards-compatible. 
>
> Could you elaborate on that?
>
> Assuming model names > 100 characters never worked wouldn't only 
> suffix-hashing model names > 100 characters be backward compatible?
>
> Simon
>
> Le mardi 11 août 2020 à 12:05:46 UTC-4, Shai Berger a écrit :
>
>> AFAIK Postgres, in these cases, simply truncates the name. This means: 
>>
>> 1) Generating models with names longer than 63 characters on postgres 
>> is fragile. You may find yourself with more than one model trying to 
>> use the same table name. 
>>
>> 2) Suffix-hashing long names like Simon suggests may not be 
>> backwards-compatible. 
>>
>> My 2 cents, 
>> Shai. 
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/57e57ee9-cc45-46bc-b205-fb87173109c5o%40googlegroups.com.

Intended effect of returning None from ManifestFilesMixin.file_hash

[Richard Campen]

Hello

I am currently looking at implementing a custom file hashing function when 
using the ManifestFilesMixin class to provide white list based file hashing 
of static files.

In the process I have found what seems to be an unintended result of 
returning `None` from a custom `file_hash()` function when using the mixin. 
Looking at the original ticket 

 
that resulted in the refactoring of the file hashing into it's own function 
(for the explicit intention of enabling custom algorithms when using the 
mixin) there is no indication of intent for the current behaviour.

Essentially, when returning a string from a custom` file_hash()` 
implementation, the resulting file name is 
`..` as expected, whereas returning `None` 
results in `None.`

The fact that the string `None` appears in the file name seems undesirable, 
and the fact it occurs without a preceding `.` (inconsistent with the 
filename structure when a string is returned) suggests to me it is 
unintentional.

I have my own fix to this locally that leaves the file name untouched if 
`file_hash()` returns `None`. Before going through the process of starting 
a ticket etc. I was curious if anyone had any thoughts on the intent of 
injecting `None` into the file name. Bug or feature.

Cheers
Richard

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/050868a3-b941-4cd1-b4e4-7b1b6329d8b4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.