Re: Fellow Reports -- April 2021

[Carlton Gibson]

Hi all. 


Calendar Week 16 -- ending 25 April.


Triaged:

https://code.djangoproject.com/ticket/32671 -- CSS var() causes relative 
URLs to break in Safari (wontfix)
https://code.djangoproject.com/ticket/32668 -- Separate test-collection 
setup from runtests.py's setup() for use in get_app_test_labels() 
(Accepted)



Reviewed:

https://github.com/django/django/pull/14307 -- Fixed isolation of 
test_rename_table_renames_deferred_sql_references().
https://github.com/django/django/pull/14304 -- Refs #32673 -- Fixed lookups 
crash when comparing against lookups on Oracle.
https://github.com/django/django/pull/14303 -- Fixed isolation of 
test_showmigrations_unmigrated_app().
https://github.com/django/django/pull/14290 -- Refs #28034 -- Corrected 
docs example in contributing tutorial.
https://github.com/django/django/pull/14300 -- Fixed isolation of 
test_showmigrations_unmigrated_app().
https://github.com/django/django/pull/14296 -- Used assertCountEqual() in 
ExcludeTests.test_exclude_subquery().
https://github.com/django/django/pull/14271 -- Fixed #32650 -- Fixed 
handling subquery aliasing on queryset combination.
https://github.com/django/django/pull/14291 -- Fixed #32665 -- Fixed caches 
system check crash when STATICFILES_DIRS is a list of 2-tuples.



Authored:

https://github.com/django/django/pull/14288 -- Fixed #32647 -- Restored 
multi-row select with shift-modifier in admin changelist.


Kind Regards,

Carlton

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/7ef3438d-5a9a-4166-9ae6-eabd2af7c67en%40googlegroups.com.

Loading CSV Data in Model

['Muhammad Asim Khaskheli' via Django developers (Contributions to Django itself)]

Hi,

I wanted to ask how to load csv data into the model. What steps do we have 
to follow .

I have made this function but how exactly to run this code because, python 
manage.py loaddata wont work. 

def import_data():
with open('sample-dataset.csv') as f:
reader = csv.reader(f)
for row in reader:
if row[0] != 'user_id':
_, created = Funnel.objects.get_or_create(
user_id=row[0],
event=row[1],
timestamp=row[2],
session_id=row[3]
)


-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/ab4f4d9d-87b6-45e4-b490-1d665b1795can%40googlegroups.com.

APPEND_SLASH behavior

[Tidiane Dia]

Hello, I posted this  on #django-users but I think here is the right place to 
post it. 
To give more context, this  
is the related issue on Wagtail which lead me here.

In general, if a user defines a "catch all non-recognized patterns" 
URLpattern (let's call it catch-all-404) as Wagtail admin does, something 
like:

# Default view (will show 404 page)
urlpatterns = [
... (some patterns here) ...

re_path(r'^', home.default),
]
the APPEND_SLASH setting has no effect.

For example, if */account/* is a valid path, when a user requests 
*/account,* he will be routed to the catch-all-404 view, regardless of the 
APPEND_SLASH setting. That is due to the check being done here 

 
which considers that the current path is a valid one since it matches the 
catch-all-404 pattern. 

This is not the desired behavior, for Wagtail admin at least. 

Instead of calling the should_redirect_with_slash in the process_response 
method 
,
 
a trick I found to solve this issue is to call a 
should_force_redirect_with_slash 
method that directly checks if appending a slash to the current path turns 
it into a valid one. 
Its only difference with the should_redirect_with_slash method is skipping 
the line highlighted above.

This seems very specific and I don't know if Django wants to handle this 
itself (and if so, I don't know if it's going to be this way too) or rather 
let the user adapt.

In both cases however, the current check being done in the process_response 
method for the CommonMiddleware(here 
)
 
seems irrelevant to me unless I am missing *something*. 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/6d39c5c0-bd79-48ec-8ae4-bad4ae023237n%40googlegroups.com.

Re: Loading CSV Data in Model

['Adam Johnson' via Django developers (Contributions to Django itself)]

Hi!

I think you've found the wrong mailing list for this post. This mailing
list is for discussing the development of Django itself, not for support
using Django. This means the discussions of bugs and features in Django
itself, rather than in your code using it. People on this list are unlikely
to answer your support query with their limited time and energy.

For support, please follow the "Getting Help" page:
https://docs.djangoproject.com/en/3.1/faq/help/ . This will help you find
people who are willing to support you, and to ask your question in a way
that makes it easy for them to answer.

Thanks for your understanding and all the best,

Adam

On Thu, 29 Apr 2021 at 10:22, 'Muhammad Asim Khaskheli' via Django
developers (Contributions to Django itself) <
django-developers@googlegroups.com> wrote:

> Hi,
>
> I wanted to ask how to load csv data into the model. What steps do we have
> to follow .
>
> I have made this function but how exactly to run this code because, python
> manage.py loaddata wont work.
>
> def import_data():
> with open('sample-dataset.csv') as f:
> reader = csv.reader(f)
> for row in reader:
> if row[0] != 'user_id':
> _, created = Funnel.objects.get_or_create(
> user_id=row[0],
> event=row[1],
> timestamp=row[2],
> session_id=row[3]
> )
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/ab4f4d9d-87b6-45e4-b490-1d665b1795can%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAMyDDM3%2Bj%2Bz0giz2hGfNjg%3Dsi0MJUcyfEJZ%3DbUj8S4%2Bs0Muskw%40mail.gmail.com.

Re: Loading CSV Data in Model

[Franck Tchouanga]

To load a csv file in python you use * pd.read_csv('your file',
index_col=0) . *

*So you need to use the library pandas.*

*Hope it helps.*

On Thu, Apr 29, 2021 at 3:23 PM 'Muhammad Asim Khaskheli' via Django
developers (Contributions to Django itself) <
django-developers@googlegroups.com> wrote:

> Hi,
>
> I wanted to ask how to load csv data into the model. What steps do we have
> to follow .
>
> I have made this function but how exactly to run this code because, python
> manage.py loaddata wont work.
>
> def import_data():
> with open('sample-dataset.csv') as f:
> reader = csv.reader(f)
> for row in reader:
> if row[0] != 'user_id':
> _, created = Funnel.objects.get_or_create(
> user_id=row[0],
> event=row[1],
> timestamp=row[2],
> session_id=row[3]
> )
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/ab4f4d9d-87b6-45e4-b490-1d665b1795can%40googlegroups.com
> 
> .
>


-- 
Best Wishes

*Mr Tchouanga*

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CANRJ%3D3n5ejYQ0FV888dBsySS8xMLZo4B1-PwEDAis0BDhXdYBw%40mail.gmail.com.

Re: Loading CSV Data in Model

[Franck Tchouanga]

*import pandas as pd*

*data = pd.read_csv('your file', index_col=0)*

On Thu, Apr 29, 2021 at 3:51 PM Franck Tchouanga 
wrote:

> To load a csv file in python you use * pd.read_csv('your file',
> index_col=0) . *
>
> *So you need to use the library pandas.*
>
> *Hope it helps.*
>
> On Thu, Apr 29, 2021 at 3:23 PM 'Muhammad Asim Khaskheli' via Django
> developers (Contributions to Django itself) <
> django-developers@googlegroups.com> wrote:
>
>> Hi,
>>
>> I wanted to ask how to load csv data into the model. What steps do we
>> have to follow .
>>
>> I have made this function but how exactly to run this code because,
>> python manage.py loaddata wont work.
>>
>> def import_data():
>> with open('sample-dataset.csv') as f:
>> reader = csv.reader(f)
>> for row in reader:
>> if row[0] != 'user_id':
>> _, created = Funnel.objects.get_or_create(
>> user_id=row[0],
>> event=row[1],
>> timestamp=row[2],
>> session_id=row[3]
>> )
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to django-developers+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/django-developers/ab4f4d9d-87b6-45e4-b490-1d665b1795can%40googlegroups.com
>> 
>> .
>>
>
>
> --
> Best Wishes
>
> *Mr Tchouanga*
>


-- 
Best Wishes

*Mr Tchouanga*

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CANRJ%3D3mZSjo8_r88c3zH09zJjWMT6HE40nARLrx1LvqmZJ%3D3rQ%40mail.gmail.com.

Re: APPEND_SLASH behavior

['Adam Johnson' via Django developers (Contributions to Django itself)]

I don't think Django should change here. The current APPEND_SLASH behaviour
is conservative and expected. Django can't tell the difference between a
catch-all view that "shouldn't really catch the URL", and any other view.

Notably your suggestion would undo the work in django 3.2 to add a
catch-all view to the admin to prevent potentially sensitive URL's being
enumerated:
https://docs.djangoproject.com/en/dev/releases/3.2/#django-contrib-admin

You could instead change *your* catch-all view to do similar processing to
CommonMiddleware - check if appending a slash to the URL and resolving it
leads to a view (other than your catch-all view), and redirect.

On Thu, 29 Apr 2021 at 08:13, Tidiane Dia  wrote:

> Hello, I posted this on #django-users but I think here is the right place
> to post it.
> To give more context, this
>  is the related issue on
> Wagtail which lead me here.
>
> In general, if a user defines a "catch all non-recognized patterns"
> URLpattern (let's call it catch-all-404) as Wagtail admin does, something
> like:
>
> # Default view (will show 404 page)
> urlpatterns = [
> ... (some patterns here) ...
>
> re_path(r'^', home.default),
> ]
> the APPEND_SLASH setting has no effect.
>
> For example, if */account/* is a valid path, when a user requests
> */account,* he will be routed to the catch-all-404 view, regardless of
> the APPEND_SLASH setting. That is due to the check being done here
> 
> which considers that the current path is a valid one since it matches the
> catch-all-404 pattern.
>
> This is not the desired behavior, for Wagtail admin at least.
>
> Instead of calling the should_redirect_with_slash in the process_response
> method
> ,
> a trick I found to solve this issue is to call a 
> should_force_redirect_with_slash
> method that directly checks if appending a slash to the current path turns
> it into a valid one.
> Its only difference with the should_redirect_with_slash method is skipping
> the line highlighted above.
>
> This seems very specific and I don't know if Django wants to handle this
> itself (and if so, I don't know if it's going to be this way too) or rather
> let the user adapt.
>
> In both cases however, the current check being done in the
> process_response method for the CommonMiddleware(here
> )
> seems irrelevant to me unless I am missing *something*.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/6d39c5c0-bd79-48ec-8ae4-bad4ae023237n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAMyDDM1%3DvMpkAU38Tn7YkonkWvsSneCWxBN_TFb1F6t%2BrzvQVw%40mail.gmail.com.

Re: APPEND_SLASH behavior

[Tidiane Dia]

Yes I suggested doing that  work at the middleware level, but it's not the 
preferred solution due to maintanability  concerns.

However, you haven't mentionned the unneccesary check (I think) being done 
in the CommonMiddleware's process_response method ? 

Le jeudi 29 avril 2021 à 21:54:02 UTC+2, Adam Johnson a écrit :

> I don't think Django should change here. The current APPEND_SLASH 
> behaviour is conservative and expected. Django can't tell the difference 
> between a catch-all view that "shouldn't really catch the URL", and any 
> other view.
>
> Notably your suggestion would undo the work in django 3.2 to add a 
> catch-all view to the admin to prevent potentially sensitive URL's being 
> enumerated: 
> https://docs.djangoproject.com/en/dev/releases/3.2/#django-contrib-admin
>
> You could instead change *your* catch-all view to do similar processing to 
> CommonMiddleware - check if appending a slash to the URL and resolving it 
> leads to a view (other than your catch-all view), and redirect.
>
> On Thu, 29 Apr 2021 at 08:13, Tidiane Dia  wrote:
>
>> Hello, I posted this on #django-users but I think here is the right 
>> place to post it. 
>> To give more context, this 
>>  is the related issue on 
>> Wagtail which lead me here.
>>
>> In general, if a user defines a "catch all non-recognized patterns" 
>> URLpattern (let's call it catch-all-404) as Wagtail admin does, something 
>> like:
>>
>> # Default view (will show 404 page)
>> urlpatterns = [
>> ... (some patterns here) ...
>>
>> re_path(r'^', home.default),
>> ]
>> the APPEND_SLASH setting has no effect.
>>
>> For example, if */account/* is a valid path, when a user requests 
>> */account,* he will be routed to the catch-all-404 view, regardless of 
>> the APPEND_SLASH setting. That is due to the check being done here 
>> 
>>  
>> which considers that the current path is a valid one since it matches the 
>> catch-all-404 pattern. 
>>
>> This is not the desired behavior, for Wagtail admin at least. 
>>
>> Instead of calling the should_redirect_with_slash in the 
>> process_response method 
>> ,
>>  
>> a trick I found to solve this issue is to call a 
>> should_force_redirect_with_slash 
>> method that directly checks if appending a slash to the current path turns 
>> it into a valid one. 
>> Its only difference with the should_redirect_with_slash method is 
>> skipping the line highlighted above.
>>
>> This seems very specific and I don't know if Django wants to handle this 
>> itself (and if so, I don't know if it's going to be this way too) or rather 
>> let the user adapt.
>>
>> In both cases however, the current check being done in the 
>> process_response method for the CommonMiddleware(here 
>> )
>>  
>> seems irrelevant to me unless I am missing *something*. 
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-develop...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/6d39c5c0-bd79-48ec-8ae4-bad4ae023237n%40googlegroups.com
>>  
>> 
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/5bf53263-4eae-4b62-b74c-a06e00d963f9n%40googlegroups.com.

Re: APPEND_SLASH behavior

[Tidiane Dia]

It seems that my first link doesn't work.

I was saying that I ran the coverage of the project and found that this 
line(here 
)
 
is never hit because the two conditions can never be met at the same time.
Indeed the process_request method already handles if a slash should be 
appended. 
When APPEND_SLASH is set to True, if we have a 404 error after 
process_request has been called, the condition should_redirect_with_slash 
can never be met.



Le jeudi 29 avril 2021 à 22:07:18 UTC+2, Tidiane Dia a écrit :

> Yes I suggested doing that  work at the middleware level, but it's not the 
> preferred solution due to maintanability  concerns.
>
> However, you haven't mentionned the unneccesary check (I think) being done 
> in the CommonMiddleware's process_response method ? 
>
> Le jeudi 29 avril 2021 à 21:54:02 UTC+2, Adam Johnson a écrit :
>
>> I don't think Django should change here. The current APPEND_SLASH 
>> behaviour is conservative and expected. Django can't tell the difference 
>> between a catch-all view that "shouldn't really catch the URL", and any 
>> other view.
>>
>> Notably your suggestion would undo the work in django 3.2 to add a 
>> catch-all view to the admin to prevent potentially sensitive URL's being 
>> enumerated: 
>> https://docs.djangoproject.com/en/dev/releases/3.2/#django-contrib-admin
>>
>> You could instead change *your* catch-all view to do similar processing 
>> to CommonMiddleware - check if appending a slash to the URL and resolving 
>> it leads to a view (other than your catch-all view), and redirect.
>>
>> On Thu, 29 Apr 2021 at 08:13, Tidiane Dia  wrote:
>>
>>> Hello, I posted this on #django-users but I think here is the right 
>>> place to post it. 
>>> To give more context, this 
>>>  is the related issue 
>>> on Wagtail which lead me here.
>>>
>>> In general, if a user defines a "catch all non-recognized patterns" 
>>> URLpattern (let's call it catch-all-404) as Wagtail admin does, something 
>>> like:
>>>
>>> # Default view (will show 404 page)
>>> urlpatterns = [
>>> ... (some patterns here) ...
>>>
>>> re_path(r'^', home.default),
>>> ]
>>> the APPEND_SLASH setting has no effect.
>>>
>>> For example, if */account/* is a valid path, when a user requests 
>>> */account,* he will be routed to the catch-all-404 view, regardless of 
>>> the APPEND_SLASH setting. That is due to the check being done here 
>>> 
>>>  
>>> which considers that the current path is a valid one since it matches the 
>>> catch-all-404 pattern. 
>>>
>>> This is not the desired behavior, for Wagtail admin at least. 
>>>
>>> Instead of calling the should_redirect_with_slash in the 
>>> process_response method 
>>> ,
>>>  
>>> a trick I found to solve this issue is to call a 
>>> should_force_redirect_with_slash 
>>> method that directly checks if appending a slash to the current path turns 
>>> it into a valid one. 
>>> Its only difference with the should_redirect_with_slash method is 
>>> skipping the line highlighted above.
>>>
>>> This seems very specific and I don't know if Django wants to handle this 
>>> itself (and if so, I don't know if it's going to be this way too) or rather 
>>> let the user adapt.
>>>
>>> In both cases however, the current check being done in the 
>>> process_response method for the CommonMiddleware(here 
>>> )
>>>  
>>> seems irrelevant to me unless I am missing *something*. 
>>>
>>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "Django developers (Contributions to Django itself)" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to django-develop...@googlegroups.com.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/django-developers/6d39c5c0-bd79-48ec-8ae4-bad4ae023237n%40googlegroups.com
>>>  
>>> 
>>> .
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/d1f72d4e-2048-4883-928c-c34d111aefe5n%40googlegroups.com.