Re: Digest for django-developers@googlegroups.com - 1 update in 1 topic

2017-10-20 Thread Dmitriy Sintsov 
If you want to populate the data after migration is complete, several times
or each time, you may try to use post_migrate signal:
https://docs.djangoproject.com/en/1.11/ref/signals/#post-migrate

On Fri, Oct 20, 2017 at 6:44 AM,  wrote:

> django-developers@googlegroups.com
> 
>  Google
> Groups
> 
> 
> Topic digest
> View all topics
> 
>
>- Feature request: Allow to process data-migrations after flush
><#m_-4246296427101809839_group_thread_0> - 1 Update
>
> Feature request: Allow to process data-migrations after flush
> 
> Alan Arellano : Oct 16 01:45PM -0300
>
> Precisely, because data migrations are indeed supported, it should be an
> option.
>
> Today I'm in other situation, I have a data migration which requires
> being run several times (because updates needed data from remote
> sources), today I can't do this with an straightforward command.
>
>
> On 08/26/2017 04:58 AM, Adam Johnson wrote:
> Back to top <#m_-4246296427101809839_digest_top>
> You received this digest because you're subscribed to updates for this
> group. You can change your settings on the group membership page
> 
> .
> To unsubscribe from this group and stop receiving emails from it send an
> email to django-developers+unsubscr...@googlegroups.com.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAC5UD1x8h_YhEaoLR36qx38D51zhvEv3Rx0n_qTcCqMeJyR3WA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: PostgreSQL Partial Indexes package

2017-10-20 Thread Ashley Waite 
I did a similar thing (but only tested in postgres) a while back, using Q's 
/ filter clause to generate the where clause.

Perhaps with our powers combined we're close to a generalised 
implementation?

https://github.com/ashleywaite/django-more/blob/master/django_more/indexes.py



On Saturday, October 7, 2017 at 7:56:00 PM UTC+11, Mattias Linnap wrote:
>
> Hi django-developers,
>
> I have written a package that implements PostgreSQL and SQLite partial 
> indexes on top of the new class-based indexes: 
> https://github.com/mattiaslinnap/django-partial-index
> The most common use case is partial unique constraints, but I have a few 
> projects where non-unique partial indexes have turned out useful as well.
>
> I have a few questions on how to continue with this:
>
> 1. Right now the "where condition" expression is provided as a string, and 
> has to be different for PostgreSQL and SQLite in some common cases (for 
> example boolean literals). Is there a good abstraction for SQL expressions 
> somewhere in Django internals that I could use instead, something similar 
> to Q-expressions perhaps? In particular, to add validate_unique() support 
> to ModelForms, I would need to be able to extract all fields that are 
> mentioned in the where condition.
> 2. I've seen mentions of a "check constraints" support being in 
> development (https://github.com/django/django/pull/7615). Will that 
> include partial unique constraints, or is just for the per-column checks?
> 3. If separate, then it would be nice to one day get partial indexes 
> merged into the contrib.postgres package. Do you have any suggestions on 
> what needs to happen before that - more test coverage, more contributors, 
> more users, or similar?
>
> Best,
>
> Mattias
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/84f4431a-cdab-43c7-b96b-de4d2eb9805b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Database agnostic EnumField

2017-10-20 Thread Ashley Waite 
I've been working a bit on an EnumField implementation because it'll save 
me a lot of future time in a project, and existing implementations seem to 
be fragile, non-reversible, or one-database-only. Wondering why there isn't 
a PEP435 based EnumField in Django itself, I didn't find many answers with 
a search on the mailing list.

Is this a feature that would be considered, and if so, what would the 
expectations on it be?

I was a bit reluctant on all the implementations I could find because they 
seem to reduce to these issues:
* Using an int/char instead of database enum
* Being database vendor specific
* Requiring a non-standard enum or sub-class of it
* Not allowing enum reuse
* Not migrating changes statefully (ie, injecting type declaration on 
connection in postgres)
* Not allowing enum changes (add/remove/rename)
* Not parametising enum values (mysql)
* Not handling data consistency on enum changes

And realised, maybe that's why it's not a standard field.

I've done a POC implementation that works for mysql and postgres, and 
should be able to easily generalise to work on any database via two flags 
(has_enum, and requires_enum_declaration) that determine how to deal with 
changes to it.

It addresses all of these issues, migrates, and with a little more work can 
handle data effects as well.

So where should I go with this from here?
https://github.com/ashleywaite/django-more/tree/master/django_enum

- Ashley

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/317b5aea-b68f-467b-886d-68a49c7194c7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Database agnostic EnumField

2017-10-20 Thread Jani Tiainen 
Hi.

In general features that can live outside Django (doesn't require changes
in Django core to be implemented) needs quite well established de facto
standard in community to be included in the Django itself.

So best thing to do is to make sure that your implementation can work with
all current backends and preferably with all current supported Django
versions.

And get users to your enum field.


20.10.2017 14.55 "Ashley Waite"  kirjoitti:

I've been working a bit on an EnumField implementation because it'll save
me a lot of future time in a project, and existing implementations seem to
be fragile, non-reversible, or one-database-only. Wondering why there isn't
a PEP435 based EnumField in Django itself, I didn't find many answers with
a search on the mailing list.

Is this a feature that would be considered, and if so, what would the
expectations on it be?

I was a bit reluctant on all the implementations I could find because they
seem to reduce to these issues:
* Using an int/char instead of database enum
* Being database vendor specific
* Requiring a non-standard enum or sub-class of it
* Not allowing enum reuse
* Not migrating changes statefully (ie, injecting type declaration on
connection in postgres)
* Not allowing enum changes (add/remove/rename)
* Not parametising enum values (mysql)
* Not handling data consistency on enum changes

And realised, maybe that's why it's not a standard field.

I've done a POC implementation that works for mysql and postgres, and
should be able to easily generalise to work on any database via two flags
(has_enum, and requires_enum_declaration) that determine how to deal with
changes to it.

It addresses all of these issues, migrates, and with a little more work can
handle data effects as well.

So where should I go with this from here?
https://github.com/ashleywaite/django-more/tree/master/django_enum

- Ashley

-- 
You received this message because you are subscribed to the Google Groups
"Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/
msgid/django-developers/317b5aea-b68f-467b-886d-68a49c7194c7%40googlegroups.
com

.
For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAHn91ofgo8WKq8N2LA6v-mKp5249Xtcg09B04Poqe_NMAGisAA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Database agnostic EnumField

2017-10-20 Thread Tom Forbes 
There is a maybe/someday ticket for this here: https://code.djangoproject.
com/ticket/24342

If your interested you could work on adding this into Django through that
ticket, it seems like an interesting idea.

When removing an enum value, how do you handle existing records? Are all
supported databases consistent in throwing errors if existing rows are
still present with a removed enumeration value?

On Fri, Oct 20, 2017 at 12:55 PM, Ashley Waite 
wrote:

> I've been working a bit on an EnumField implementation because it'll save
> me a lot of future time in a project, and existing implementations seem to
> be fragile, non-reversible, or one-database-only. Wondering why there isn't
> a PEP435 based EnumField in Django itself, I didn't find many answers with
> a search on the mailing list.
>
> Is this a feature that would be considered, and if so, what would the
> expectations on it be?
>
> I was a bit reluctant on all the implementations I could find because they
> seem to reduce to these issues:
> * Using an int/char instead of database enum
> * Being database vendor specific
> * Requiring a non-standard enum or sub-class of it
> * Not allowing enum reuse
> * Not migrating changes statefully (ie, injecting type declaration on
> connection in postgres)
> * Not allowing enum changes (add/remove/rename)
> * Not parametising enum values (mysql)
> * Not handling data consistency on enum changes
>
> And realised, maybe that's why it's not a standard field.
>
> I've done a POC implementation that works for mysql and postgres, and
> should be able to easily generalise to work on any database via two flags
> (has_enum, and requires_enum_declaration) that determine how to deal with
> changes to it.
>
> It addresses all of these issues, migrates, and with a little more work
> can handle data effects as well.
>
> So where should I go with this from here?
> https://github.com/ashleywaite/django-more/tree/master/django_enum
>
> - Ashley
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at https://groups.google.com/group/django-developers.
> To view this discussion on the web visit https://groups.google.com/d/ms
> gid/django-developers/317b5aea-b68f-467b-886d-68a49c7194c7%
> 40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAFNZOJOqGuKL_eP0TxbL2TaH__j%2BYLKaOJAMQPhFdFWXya7zOg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Database agnostic EnumField

2017-10-20 Thread Ashley Waite 
Thanks for that link Tom, I thought I'd come across that before but forgot 
where I saw the discussion!
That definitely seems like the right place.

Jani, I patch in several changes to core classes in order to do this 
without any of those caveats, but have tried to use as few as was possible.

Specifically:
* Changes to the migration state classes to allow for state tracking of 
types
* Changes to schema editor to allow for parametised db_types
* Changes to migration autodetector to have operations generated and in the 
right places (enums before models)

Tom, to handle existing records I apply on_delete functionality. If 
declared on a field (optional) it provides a default behaviour for that 
field, but it also asks during makemigrations (see here 
https://github.com/ashleywaite/django-more/blob/master/django_enum/patches.py#L28
 
) and then gives the option to apply any of those equivalent behaviours to 
existing records, including setting the value to a newly added value so you 
can effectively 'rename' a value within the enum.

Then during migrate these are applied, which can update records 
accordingly, or block migration if there's data that has protect.

Though I do need to update this to do that part via the execute() method so 
that it plays properly with sqlmigrate.

- Ashley

On Saturday, October 21, 2017 at 5:39:30 AM UTC+11, Tom Forbes wrote:
>
> There is a maybe/someday ticket for this here: 
> https://code.djangoproject.com/ticket/24342
>
> If your interested you could work on adding this into Django through that 
> ticket, it seems like an interesting idea.
>
> When removing an enum value, how do you handle existing records? Are all 
> supported databases consistent in throwing errors if existing rows are 
> still present with a removed enumeration value?
>
> On Fri, Oct 20, 2017 at 12:55 PM, Ashley Waite  > wrote:
>
>> I've been working a bit on an EnumField implementation because it'll save 
>> me a lot of future time in a project, and existing implementations seem to 
>> be fragile, non-reversible, or one-database-only. Wondering why there isn't 
>> a PEP435 based EnumField in Django itself, I didn't find many answers with 
>> a search on the mailing list.
>>
>> Is this a feature that would be considered, and if so, what would the 
>> expectations on it be?
>>
>> I was a bit reluctant on all the implementations I could find because 
>> they seem to reduce to these issues:
>> * Using an int/char instead of database enum
>> * Being database vendor specific
>> * Requiring a non-standard enum or sub-class of it
>> * Not allowing enum reuse
>> * Not migrating changes statefully (ie, injecting type declaration on 
>> connection in postgres)
>> * Not allowing enum changes (add/remove/rename)
>> * Not parametising enum values (mysql)
>> * Not handling data consistency on enum changes
>>
>> And realised, maybe that's why it's not a standard field.
>>
>> I've done a POC implementation that works for mysql and postgres, and 
>> should be able to easily generalise to work on any database via two flags 
>> (has_enum, and requires_enum_declaration) that determine how to deal with 
>> changes to it.
>>
>> It addresses all of these issues, migrates, and with a little more work 
>> can handle data effects as well.
>>
>> So where should I go with this from here?
>> https://github.com/ashleywaite/django-more/tree/master/django_enum
>>
>> - Ashley
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers (Contributions to Django itself)" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-develop...@googlegroups.com .
>> To post to this group, send email to django-d...@googlegroups.com 
>> .
>> Visit this group at https://groups.google.com/group/django-developers.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/django-developers/317b5aea-b68f-467b-886d-68a49c7194c7%40googlegroups.com
>>  
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/efc9bde8-294c-4127-b2c1-718c1cbd6c63%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.