Using forms for bulk records

[Elias Coutinho]

Good morning people.

As always my problem is conceptual. I have improved a lot, but still in the 
fight.
1 - I have a list of data and I know how to add it to another model at one 
time. (Easy)
2 - It turns out that in this list some data will be filled by the user 
before saving and will be repeated for each record as a Research and 
Interviewed Data field.
3 - I repeat! It already works with the data entered manually! putting the 
person instance and the date on the arm.

QUESTION *

I have a list of people and wanted to click on one, there it would be 
redirected to a form (where it is done) where I would already show the 
person I chose (I have not done yet) and there would be two more fields and 
Interviewer and logically the interviewer would type in the name of the (in 
a text field) and the interview date (search_key).

Then when recording he would have to use this view below:

https://github.com/ElectronicElectronics/supportalterdata/blob/master/search_core/core/views.py#L66-L87


What better way to open a screen for the user to choose this data before 
saving?

I will post this doubt on other lists with the same text. (Do not be 
surprised)

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/1ddbf3b2-1334-4dd6-b798-6549db8147e8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Requiring sqlparse for sqlite introspection

[Tim Graham]

So you want to add it to Django's install_requires even though it won't be 
necessary if SQLite isn't used? It seems okay to me. It's an extra 40k or 
so of disk space but that's not much compared to all the extra stuff Django 
comes with that every Django project doesn't necessarily use. We also have 
the requires_sqlparse_for_splitting feature flag which we could remove if 
we can assume sqlparse is installed.

On Sunday, October 28, 2018 at 4:59:54 AM UTC-4, Adam Johnson wrote:
>
> I'm fine with adding it as a dependency, my experience has been that it's 
> a stable, well-maintained package over the past few years I've used it.
>
> On Sun, 28 Oct 2018 at 05:00, charettes > 
> wrote:
>
>> After a bit of work to minimize the cases where sqlparse would be a 
>> required at runtime for SQLite to AddConstraint/RemoveConstraint operations 
>> [0] I came to the conclusion that it would make more sense to make sqlparse 
>> an hard dependency of Django 2.2.
>>
>> The two reasons backing this conclusions are
>>
>> 1. Given we run the suite with sqlparse installed on CI it will be 
>> really hard to prevent inadvertently breaking the promise of a soft 
>> dependency on sqlparse for Meta.constraints only. I guess we could have 
>> a daily CI job but that would quickly get out of hand once we have to 
>> perform backport and such.
>>
>> 2. There's a few instances of fragile regex parsing that could be made 
>> more reliable if sqlparse was an hard dependency for SQLite. Two 
>> examples are
>>  - 
>> https://github.com/django/django/blob/f892781b957f674806a227a10c58768f66a48c07/django/db/backends/sqlite3/introspection.py#L90-L146
>>  - 
>> https://github.com/django/django/blob/f892781b957f674806a227a10c58768f66a48c07/django/db/backends/sqlite3/schema.py#L91-L127
>>
>> Given the transition to require pytz in Django 1.11 went smoothly and our 
>> needs for sqlparse are similar to provide a solid experience on SQLite I'd 
>> be +1 on requiring it from Django 2.2 to a point where the lowest version 
>> of SQLite we support has better introspection capabilities.
>>
>> Simon
>>
>> [0] https://github.com/django/django/pull/10572
>>
>> Le mercredi 10 octobre 2018 14:53:53 UTC-4, Tim Graham a écrit :
>>>
>>> sqlparse is already installed as part of Django's tests. The question is 
>>> whether sqlparse should be mandatory for SQLite users (i.e. when getting 
>>> started with a new project an error message will say, "You must install 
>>> sqlparse to use SQLite" (I don't think there's a way to install it 
>>> automatically unless we install it for all users, regardless of database) 
>>> or if we should try to make it optional and only raise an error if a 
>>> project's migrations require it.
>>>
>>> The question is "what percentage of SQLite projects are going to end up 
>>> needing sqlparse based on their migrations? If it's high enough, it might 
>>> make for a better user experience to have users install it when they're 
>>> getting started instead of a lazy error when get_constraints() is 
>>> called."
>>>
>>> On Monday, October 8, 2018 at 3:22:32 AM UTC-4, Andrew Godwin wrote:

 Adding sqlparse into the introspection code for SQLite specifically 
 means it's going to be a runtime dependency for anything to do with 
 migrations.

 I'm alright having that be the case, but I do think we should make sure 
 the tests run with SQLite as otherwise a large section of the most 
 complicated code in migrations won't be tested properly.

 Andrew


 On Mon, 8 Oct 2018, 00:59 Ian Foote,  wrote:

> Hi all,
>
> On my pull request (https://github.com/django/django/pull/10406) 
> refactoring how Django creates database constraints I introduced a 
> dependency on sqlparse in the sqlite introspection code. This allows 
> Django 
> to correctly read information about constraints on sqlite, particularly 
> the 
> name.
>
> When reviewing (
> https://github.com/django/django/pull/10406#issuecomment-424542217) 
> Tim Graham raised the question of whether we should make sqlparse a 
> mandatory requirement for the sqlite tests or if we should skip those 
> tests 
> that require it. In later discussion (
> https://github.com/django/django/pull/10406#issuecomment-427362983), 
> Tim raised the point that the introspection code is used by migrations.
>
> I'm not clear myself what the best route forwards here is, so I'm 
> bringing it to Django Developers for discussion (as Tim suggested). 
>
> Thanks,
> Ian
>
> -- 
> You received this message because you are subscribed to the Google 
> Groups "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send 
> an email to django-develop...@googlegroups.com.
> To post to this group, send email to django-d...@googlegroups.com.
> Visit this group at https:/

Re: Using forms for bulk records

[Tim Graham]

This mailing list is for the development of Django itself, not how to solve 
problems with Django. Please don't cross-post from django-users.

On Saturday, November 3, 2018 at 9:57:41 AM UTC-4, Elias Coutinho wrote:
>
> Good morning people.
>
> As always my problem is conceptual. I have improved a lot, but still in 
> the fight.
> 1 - I have a list of data and I know how to add it to another model at one 
> time. (Easy)
> 2 - It turns out that in this list some data will be filled by the user 
> before saving and will be repeated for each record as a Research and 
> Interviewed Data field.
> 3 - I repeat! It already works with the data entered manually! putting the 
> person instance and the date on the arm.
>
> QUESTION *
>
> I have a list of people and wanted to click on one, there it would be 
> redirected to a form (where it is done) where I would already show the 
> person I chose (I have not done yet) and there would be two more fields and 
> Interviewer and logically the interviewer would type in the name of the (in 
> a text field) and the interview date (search_key).
>
> Then when recording he would have to use this view below:
>
>
> https://github.com/ElectronicElectronics/supportalterdata/blob/master/search_core/core/views.py#L66-L87
>
>
> What better way to open a screen for the user to choose this data before 
> saving?
>
> I will post this doubt on other lists with the same text. (Do not be 
> surprised)
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/918ca0f4-b887-417b-a857-55d88e1eeec1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Using forms for bulk records

[Adam Johnson]

This mailing list is for the development of Django itself, not for support.
Use the django-usersmailing list for that, or IRC #djangoon freenode, or a
site like StackOverflow.

On Sat, 3 Nov 2018 at 13:48, Elias Coutinho 
wrote:

> Good morning people.
>
> As always my problem is conceptual. I have improved a lot, but still in
> the fight.
> 1 - I have a list of data and I know how to add it to another model at one
> time. (Easy)
> 2 - It turns out that in this list some data will be filled by the user
> before saving and will be repeated for each record as a Research and
> Interviewed Data field.
> 3 - I repeat! It already works with the data entered manually! putting the
> person instance and the date on the arm.
>
> QUESTION *
>
> I have a list of people and wanted to click on one, there it would be
> redirected to a form (where it is done) where I would already show the
> person I chose (I have not done yet) and there would be two more fields and
> Interviewer and logically the interviewer would type in the name of the (in
> a text field) and the interview date (search_key).
>
> Then when recording he would have to use this view below:
>
>
> https://github.com/ElectronicElectronics/supportalterdata/blob/master/search_core/core/views.py#L66-L87
>
>
> What better way to open a screen for the user to choose this data before
> saving?
>
> I will post this doubt on other lists with the same text. (Do not be
> surprised)
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/1ddbf3b2-1334-4dd6-b798-6549db8147e8%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>
-- 
Adam

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAMyDDM07Wq0tRH3saHBw4GYTnbyJByfs3DJnq2y-94XWMrKT6w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Requiring sqlparse for sqlite introspection

[charettes]

> So you want to add it to Django's install_requires even though it won't 
be necessary if SQLite isn't used?

That's my understanding and what I was advocating for.

Simon

Le samedi 3 novembre 2018 10:09:55 UTC-4, Tim Graham a écrit :
>
> So you want to add it to Django's install_requires even though it won't be 
> necessary if SQLite isn't used? It seems okay to me. It's an extra 40k or 
> so of disk space but that's not much compared to all the extra stuff Django 
> comes with that every Django project doesn't necessarily use. We also have 
> the requires_sqlparse_for_splitting feature flag which we could remove if 
> we can assume sqlparse is installed.
>
> On Sunday, October 28, 2018 at 4:59:54 AM UTC-4, Adam Johnson wrote:
>>
>> I'm fine with adding it as a dependency, my experience has been that it's 
>> a stable, well-maintained package over the past few years I've used it.
>>
>> On Sun, 28 Oct 2018 at 05:00, charettes  wrote:
>>
>>> After a bit of work to minimize the cases where sqlparse would be a 
>>> required at runtime for SQLite to AddConstraint/RemoveConstraint operations 
>>> [0] I came to the conclusion that it would make more sense to make sqlparse 
>>> an hard dependency of Django 2.2.
>>>
>>> The two reasons backing this conclusions are
>>>
>>> 1. Given we run the suite with sqlparse installed on CI it will be 
>>> really hard to prevent inadvertently breaking the promise of a soft 
>>> dependency on sqlparse for Meta.constraints only. I guess we could have 
>>> a daily CI job but that would quickly get out of hand once we have to 
>>> perform backport and such.
>>>
>>> 2. There's a few instances of fragile regex parsing that could be made 
>>> more reliable if sqlparse was an hard dependency for SQLite. Two 
>>> examples are
>>>  - 
>>> https://github.com/django/django/blob/f892781b957f674806a227a10c58768f66a48c07/django/db/backends/sqlite3/introspection.py#L90-L146
>>>  - 
>>> https://github.com/django/django/blob/f892781b957f674806a227a10c58768f66a48c07/django/db/backends/sqlite3/schema.py#L91-L127
>>>
>>> Given the transition to require pytz in Django 1.11 went smoothly and 
>>> our needs for sqlparse are similar to provide a solid experience on SQLite 
>>> I'd be +1 on requiring it from Django 2.2 to a point where the lowest 
>>> version of SQLite we support has better introspection capabilities.
>>>
>>> Simon
>>>
>>> [0] https://github.com/django/django/pull/10572
>>>
>>> Le mercredi 10 octobre 2018 14:53:53 UTC-4, Tim Graham a écrit :

 sqlparse is already installed as part of Django's tests. The question 
 is whether sqlparse should be mandatory for SQLite users (i.e. when 
 getting 
 started with a new project an error message will say, "You must install 
 sqlparse to use SQLite" (I don't think there's a way to install it 
 automatically unless we install it for all users, regardless of database) 
 or if we should try to make it optional and only raise an error if a 
 project's migrations require it.

 The question is "what percentage of SQLite projects are going to end up 
 needing sqlparse based on their migrations? If it's high enough, it might 
 make for a better user experience to have users install it when they're 
 getting started instead of a lazy error when get_constraints() is 
 called."

 On Monday, October 8, 2018 at 3:22:32 AM UTC-4, Andrew Godwin wrote:
>
> Adding sqlparse into the introspection code for SQLite specifically 
> means it's going to be a runtime dependency for anything to do with 
> migrations.
>
> I'm alright having that be the case, but I do think we should make 
> sure the tests run with SQLite as otherwise a large section of the most 
> complicated code in migrations won't be tested properly.
>
> Andrew
>
>
> On Mon, 8 Oct 2018, 00:59 Ian Foote,  wrote:
>
>> Hi all,
>>
>> On my pull request (https://github.com/django/django/pull/10406) 
>> refactoring how Django creates database constraints I introduced a 
>> dependency on sqlparse in the sqlite introspection code. This allows 
>> Django 
>> to correctly read information about constraints on sqlite, particularly 
>> the 
>> name.
>>
>> When reviewing (
>> https://github.com/django/django/pull/10406#issuecomment-424542217) 
>> Tim Graham raised the question of whether we should make sqlparse a 
>> mandatory requirement for the sqlite tests or if we should skip those 
>> tests 
>> that require it. In later discussion (
>> https://github.com/django/django/pull/10406#issuecomment-427362983), 
>> Tim raised the point that the introspection code is used by migrations.
>>
>> I'm not clear myself what the best route forwards here is, so I'm 
>> bringing it to Django Developers for discussion (as Tim suggested). 
>>
>> Thanks,
>> Ian
>>
>> -- 
>> You received this message bec

Postgres PGCrypto in Django?

[Peter Farrell]

Hi All,

I saw Carlton's short keynote at DjangoCon last month and he suggested that 
I message this group.

I'm one of the maintainers of Django-Pgcrypto-Fields which offers field 
level encryption using the PostgeSQL pgcryto library. We support Django 
1.11, and 2+ including filtering, aggregates and annotations. We have 100% 
unit test coverage (although the test suite could use a little refactoring 
for clarity).

https://github.com/incuna/django-pgcrypto-fields

I wanted to ping the group to see if anyone else would be interested in 
code reviewing the project for anything that could be done better or 
differently? I'm asking because what I know about the ORM is all by 
splunking though the Django source over time.

Also, there is potential for inclusion as part of the Django contrib for 
postgres. If there is interest,  I'd happily volunteer to help maintain. So 
I'm curious what people's opinions are regarding this.

Best,
Peter J. Farrell

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/b20663a5-962c-4e9a-8e02-7fc4de1aa7ac%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Introspection of querysets

[Dan Davis]

I was thinking of providing a package containing a couple of DRF renderers 
that work directly from querysets.   DRF Serializers are great when you 
need to render a hierarchical structure, and/or when you expect to parse a 
POST body containing the structure.  For returning non-hierarchical 
results, querysets ought to be fine.   when I attempted to code this up, I 
realized that I needed for the renderer to know which fields are in the 
body to write a CSV or XLSX.  I eventually decided that the renderers would 
assume that their input queryset was a values or values list queryset.

There is also the issue of simply being more DRY within Django itself - 
multiple Iterables combine list(values_select) + list(annotation_select) + 
list(extra_select), so a general public function could make that could 
slightly better.

It isn't too bad to look at _iterable_class - it just is clearly not a 
public interface.

On Friday, November 2, 2018 at 1:34:07 PM UTC-4, Tim Graham wrote:.

> Could you explain the use case for the code that needs to handle the 
> different types of iterators? list
>
 

> On Friday, November 2, 2018 at 11:18:34 AM UTC-4, Dan Davis wrote:
>>
>> I'm wondering two things:
>>
>>- Is there any non-internals way to know what sort of iterable a 
>>queryset is set to do?   Should there be?   Background - users should not 
>>write code to look at isinstance(queryset._iterable_class, 
>>FubarIterator) because that is a private interface (and it may be 
>>code no longer in 2.x, since I'm currently looking at 1.11.x).
>>- Is looking at query.values_select, query.annotation_select, and 
>>extra_select as is done by the ValuesIterable, ValuesListIterable, 
>>and FlatValuesListItereable a supported interface?   Is there 
>>documentation for this?   Should there be?
>>
>> Thanks,
>>
>> -Dan
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/7f162ffb-b970-4064-87ef-af67fc0da917%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Fellow Reports - October 2018

[Tim Graham]

Week ending November 3, 2018

Triaged

---

https://code.djangoproject.com/ticket/29904 - Not Null CharField Doesn't 
generate DB constraint on Oracle (invalid)

https://code.djangoproject.com/ticket/29908 - Foreign key isn't set on 
object after related set access if ForeignKey uses to_field (accepted)

https://code.djangoproject.com/ticket/29912 - Make CharField/TextField 
remove newlines from input (wontfix)

https://code.djangoproject.com/ticket/29913 - prefetch_related() crash with 
foreign key to model with non-abstract base class and UUID pk (duplicate)

https://code.djangoproject.com/ticket/29915 - Allow icontains lookup to 
accept uuids with or without dashes (accepted)

Authored

--

https://github.com/django/django/pull/10593 - Fixed #29890 -- Fixed 
FileSystemStorage crash if concurrent saves try to create the same 
directory.

Reviewed/committed

--

https://github.com/django/django/pull/10583 - Fixed #29903 -- Added error 
message for invalid WeekArchiveView week_format.

https://github.com/django/django/pull/10140 - Fixed #29547 -- Added support 
for partial indexes.

https://github.com/django/django/pull/10426 -  Fixed #29783 -- Added app 
label validation to showmigrations command.

https://github.com/django/django/pull/10578 - Fixed #29901 -- Allowed 
overriding an autocomplete/raw_id_fields/radio_fields widget with 
ModelAdmin.get_formset().

https://github.com/django/django/pull/10566 - Fixed #29886 -- Fixed 
unaccent lookup when PostgreSQL's standard_conforming_strings option is off
https://github.com/django/django/pull/10561 - Fixed #29883 -- Added 
selenium hub support to runtests.py.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/745cdc0f-1561-45be-934a-e2ba3d1c36c5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Introspection of querysets

[Adam Johnson]

I have code in my package Django-MySQL that uses the internals of
querysets. It’s not so bad to maintain with enough test coverage and
ensuring it keeps up to date with the latest Django. The fact the interface
is private means that it could change at any time, but practically that
doesn’t mean often, and rarely or never in a patch release.

See code in/around
https://github.com/adamchainz/django-mysql/blob/master/django_mysql/models/query.py

On Sat, 3 Nov 2018 at 22:22, Dan Davis  wrote:

> I was thinking of providing a package containing a couple of DRF renderers
> that work directly from querysets.   DRF Serializers are great when you
> need to render a hierarchical structure, and/or when you expect to parse a
> POST body containing the structure.  For returning non-hierarchical
> results, querysets ought to be fine.   when I attempted to code this up, I
> realized that I needed for the renderer to know which fields are in the
> body to write a CSV or XLSX.  I eventually decided that the renderers would
> assume that their input queryset was a values or values list queryset.
>
> There is also the issue of simply being more DRY within Django itself -
> multiple Iterables combine list(values_select) + list(annotation_select) +
> list(extra_select), so a general public function could make that could
> slightly better.
>
> It isn't too bad to look at _iterable_class - it just is clearly not a
> public interface.
>
> On Friday, November 2, 2018 at 1:34:07 PM UTC-4, Tim Graham wrote:.
>
>> Could you explain the use case for the code that needs to handle the
>> different types of iterators? list
>>
>
>
>> On Friday, November 2, 2018 at 11:18:34 AM UTC-4, Dan Davis wrote:
>>>
>>> I'm wondering two things:
>>>
>>>- Is there any non-internals way to know what sort of iterable a
>>>queryset is set to do?   Should there be?   Background - users should not
>>>write code to look at isinstance(queryset._iterable_class,
>>>FubarIterator) because that is a private interface (and it may be
>>>code no longer in 2.x, since I'm currently looking at 1.11.x).
>>>- Is looking at query.values_select, query.annotation_select, and
>>>extra_select as is done by the ValuesIterable, ValuesListIterable,
>>>and FlatValuesListItereable a supported interface?   Is there
>>>documentation for this?   Should there be?
>>>
>>> Thanks,
>>>
>>> -Dan
>>>
>>> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/7f162ffb-b970-4064-87ef-af67fc0da917%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>
-- 
Adam

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAMyDDM0fH2xq83Dgzd0pg%2B_%3D%2B%2B8a%2BvUW5EgkYXo0HkYvebWnaw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Introspection of querysets

[Dan Davis]

OK - why borrow trouble. There is plenty to do just maintaining current 
interfaces, and adding major features - like the new db functions in 2+.

On Saturday, November 3, 2018 at 7:36:00 PM UTC-4, Adam Johnson wrote:
>
> I have code in my package Django-MySQL that uses the internals of 
> querysets. It’s not so bad to maintain with enough test coverage and 
> ensuring it keeps up to date with the latest Django. The fact the interface 
> is private means that it could change at any time, but practically that 
> doesn’t mean often, and rarely or never in a patch release.
>
> See code in/around 
>
> https://github.com/adamchainz/django-mysql/blob/master/django_mysql/models/query.py
>
> On Sat, 3 Nov 2018 at 22:22, Dan Davis > 
> wrote:
>
>> I was thinking of providing a package containing a couple of DRF 
>> renderers that work directly from querysets.   DRF Serializers are great 
>> when you need to render a hierarchical structure, and/or when you expect to 
>> parse a POST body containing the structure.  For returning non-hierarchical 
>> results, querysets ought to be fine.   when I attempted to code this up, I 
>> realized that I needed for the renderer to know which fields are in the 
>> body to write a CSV or XLSX.  I eventually decided that the renderers would 
>> assume that their input queryset was a values or values list queryset.
>>
>> There is also the issue of simply being more DRY within Django itself - 
>> multiple Iterables combine list(values_select) + list(annotation_select) + 
>> list(extra_select), so a general public function could make that could 
>> slightly better.
>>
>> It isn't too bad to look at _iterable_class - it just is clearly not a 
>> public interface.
>>
>> On Friday, November 2, 2018 at 1:34:07 PM UTC-4, Tim Graham wrote:.
>>
>>> Could you explain the use case for the code that needs to handle the 
>>> different types of iterators? list
>>>
>>  
>>
>>> On Friday, November 2, 2018 at 11:18:34 AM UTC-4, Dan Davis wrote:

 I'm wondering two things:

- Is there any non-internals way to know what sort of iterable a 
queryset is set to do?   Should there be?   Background - users should 
 not 
write code to look at isinstance(queryset._iterable_class, 
FubarIterator) because that is a private interface (and it may be 
code no longer in 2.x, since I'm currently looking at 1.11.x).
- Is looking at query.values_select, query.annotation_select, and 
extra_select as is done by the ValuesIterable, ValuesListIterable, 
and FlatValuesListItereable a supported interface?   Is there 
documentation for this?   Should there be?

 Thanks,

 -Dan

 -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-develop...@googlegroups.com .
>> To post to this group, send email to django-d...@googlegroups.com 
>> .
>> Visit this group at https://groups.google.com/group/django-developers.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/7f162ffb-b970-4064-87ef-af67fc0da917%40googlegroups.com
>>  
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
> -- 
> Adam
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/3759d99d-1279-4b2c-b0cb-b219b06b6ff9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Requiring sqlparse for sqlite introspection

[Dan Davis]

I just joined as a contributor, but I've shipped an appliance install 
running using rpms, anaconda (the other one), and pungi.   Depending on 
sqlparse doesn't seem to me a big deal.  It already gets invoked for me 
during migrations.  I cannot recall what caused it to be installed. One 
thing we should definitely do however, is more clearly document that RunSQL 
migrations will split your SQL on statement boundaries to the extent it can 
unless you run manual SQL migrations like this:

RunSQL([forward_compound_statement], [backward_compound_statement])


Creating functions with Oracle doesn't work without this.

On Saturday, November 3, 2018 at 4:47:30 PM UTC-4, charettes wrote:
>
> > So you want to add it to Django's install_requires even though it won't 
> be necessary if SQLite isn't used?
>
> That's my understanding and what I was advocating for.
>
> Simonon 
>
> Le samedi 3 novembre 2018 10:09:55 UTC-4, Tim Graham a écrit :
>>
>> So you want to add it to Django's install_requires even though it won't 
>> be necessary if SQLite isn't used? It seems okay to me. It's an extra 40k 
>> or so of disk space but that's not much compared to all the extra stuff 
>> Django comes with that every Django project doesn't necessarily use. We 
>> also have the requires_sqlparse_for_splitting feature flag which we could 
>> remove if we can assume sqlparse is installed.
>>
>> On Sunday, October 28, 2018 at 4:59:54 AM UTC-4, Adam Johnson wrote:
>>>
>>> I'm fine with adding it as a dependency, my experience has been that 
>>> it's a stable, well-maintained package over the past few years I've used it.
>>>
>>> On Sun, 28 Oct 2018 at 05:00, charettes  wrote:
>>>
 After a bit of work to minimize the cases where sqlparse would be a 
 required at runtime for SQLite to AddConstraint/RemoveConstraint 
 operations 
 [0] I came to the conclusion that it would make more sense to make 
 sqlparse 
 an hard dependency of Django 2.2.

 The two reasons backing this conclusions are

 1. Given we run the suite with sqlparse installed on CI it will be 
 really hard to prevent inadvertently breaking the promise of a soft 
 dependency on sqlparse for Meta.constraints only. I guess we could 
 have a daily CI job but that would quickly get out of hand once we have to 
 perform backport and such.

 2. There's a few instances of fragile regex parsing that could be made 
 more reliable if sqlparse was an hard dependency for SQLite. Two 
 examples are
  - 
 https://github.com/django/django/blob/f892781b957f674806a227a10c58768f66a48c07/django/db/backends/sqlite3/introspection.py#L90-L146
  - 
 https://github.com/django/django/blob/f892781b957f674806a227a10c58768f66a48c07/django/db/backends/sqlite3/schema.py#L91-L127

 Given the transition to require pytz in Django 1.11 went smoothly and 
 our needs for sqlparse are similar to provide a solid experience on SQLite 
 I'd be +1 on requiring it from Django 2.2 to a point where the lowest 
 version of SQLite we support has better introspection capabilities.

 Simon

 [0] https://github.com/django/django/pull/10572

 Le mercredi 10 octobre 2018 14:53:53 UTC-4, Tim Graham a écrit :
>
> sqlparse is already installed as part of Django's tests. The question 
> is whether sqlparse should be mandatory for SQLite users (i.e. when 
> getting 
> started with a new project an error message will say, "You must install 
> sqlparse to use SQLite" (I don't think there's a way to install it 
> automatically unless we install it for all users, regardless of database) 
> or if we should try to make it optional and only raise an error if a 
> project's migrations require it.
>
> The question is "what percentage of SQLite projects are going to end 
> up needing sqlparse based on their migrations? If it's high enough, it 
> might make for a better user experience to have users install it when 
> they're getting started instead of a lazy error when get_constraints() 
> is called."
>
> On Monday, October 8, 2018 at 3:22:32 AM UTC-4, Andrew Godwin wrote:
>>
>> Adding sqlparse into the introspection code for SQLite specifically 
>> means it's going to be a runtime dependency for anything to do with 
>> migrations.
>>
>> I'm alright having that be the case, but I do think we should make 
>> sure the tests run with SQLite as otherwise a large section of the most 
>> complicated code in migrations won't be tested properly.
>>
>> Andrew
>>
>>
>> On Mon, 8 Oct 2018, 00:59 Ian Foote,  wrote:
>>
>>> Hi all,
>>>
>>> On my pull request (https://github.com/django/django/pull/10406) 
>>> refactoring how Django creates database constraints I introduced a 
>>> dependency on sqlparse in the sqlite introspection code. This allows 
>>> Django 
>>> to correctly re

backend specific tests

[Dan Davis]

So, the contributor guidelines page about unit tests mentions running 
database specific tests:

https://docs.djangoproject.com/en/2.1/internals/contributing/writing-code/unit-tests/#testing-other-python-versions-and-database-backends

I am working on ticket 29984, and it seems to me that since the TruncDay is 
attempting to cast to the timezone on the database level, it is working, 
and its job is done.  So, the fix should be at the backend level, and the 
ticket provides one for Postgres.

Following the unit test advice, I created a settings file.   But not using 
my Postgresql admin user.   I get the following errors:

psycopg2.ProgrammingError: permission denied to create extension "btree_gin"
HINT:  Must be superuser to create this extension.

I'm not going to do that - do you have any list of required extensions?

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/dfae7d19-a356-4b39-aab5-3148e0468082%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.