Re: Feature Proposal: per-settings fake migration override
Hi Flavio,
This sounds like a very risky and scary proposal, to be honest. The recommended
way would indeed be to fake the initial migration(s) on production.
Since it seems to me that this would only ever be needed once (you said devs
should just run the migration locally and the table is too big for any changes)
I'd probably go with the one time fake.
Alternatively, as of Django 1.9 you could move the transaction log model into a
dedicated app and disable migrations for that app.
As a last resort, why not make the model unmanaged (Meta.manage=False) and
apply the respective operations manually with RunSQL operations?
That are the ideas that just came to mind, there are probably others as well.
/Markus
On September 19, 2015 6:38:59 AM GMT+10:00, Flavio Curella
wrote:
>I'm not sure if this warrants a DEP, so I've decided to post it to the
>list
>first. If this needs to be a DEP, just let me know and I'll follow the
>DEP
>process.
>
>Rationale
>-
>
>Assume the following scenario:
>
>1. At some point in history, project A was deployed in production and
>was
>very successful.
>2. The project created a table called `transactions_log`, which is now
>considerably big and full of very valuable data, that we want to
>retain.
>3. Because of $reasons$, project B gets started as a future replacement
>of
>project B
>4. Project B wants to reuse `transactions_log` and the
>`transaction.models.Log` model unaltered
>
>What we have is a 'split' between what the initial migration for the
>`transaction.models.Log` model should do, between dev and production:
>
>* On local dev, it simply creates the table
>* On production, the migration must be faked.
>
>Traditionally, we would go to the person in charge of migration and
>tell
>them 'Hey, migration `transactions.0001_initial` must be faked'.
>But this seems error-prone. That information ('fake this one specific
>migration') could be lost in the communication workflow, and imposes
>the
>cognitive burden of having to remember another special case.
>
>Proposed solution
>
>
>Assume the following settings files:
>
>```
># settings/local.py
># ..usual list of settings...
>```
>
>```
># settings/production.py
>MIGRATION_FAKE = (
>('transactions', '0001_initial),
>)
>```
>
>* On `django-admin.py migrate --settings=settings.local`, the migration
>
>`transactions.0001` will be faked only if the `--fake` flag is passed
>(previous behavior).
>* On `django-admin.py migrate --settings=settings.production`, the
>migration `transactions.0001` will *always* be faked. The new setting
>will
>act as an ad-hoc, per-migration override of the global `--fake` option.
>
>I have a branch up as a Proof of Concept at
>https://github.com/fcurella/django/tree/feature/adhoc-fake (no test or
>doc
>changes yet). The code change necessary seems to be minimal, let me
>know if
>you think it's something worth having.
>
>Thank you,
>–Flavio.
--
You received this message because you are subscribed to the Google Groups
"Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/12F5779F-E1C9-4473-AF92-B52851465694%40markusholtermann.eu.
For more options, visit https://groups.google.com/d/optout.
Re: Ticket #23242 -- Approximate date_hierarchy
Another idea: instead of an enum, how about using the datetime.date.year,
month, day attributes?
If you prefer the enum, I don't really see an obvious place for it, so use
your best judgment and make it importable from contrib.admin I suppose.
On Saturday, September 19, 2015 at 12:38:10 AM UTC-4, Gavin Wahl wrote:
>
> Is it going to be possible to get this in before the feature freeze?
>
> On Thursday, September 10, 2015 at 10:21:29 AM UTC-6, Gavin Wahl wrote:
>>
>> The enum seems fine. Where should it be defined? Should the docs mention
>> the integer values at all, or are you required to use the enum?
>>
>> I don't think setting date_hierarchy to a tuple is any cleaner.
>>
>> On Monday, August 31, 2015 at 12:54:21 PM UTC-6, Tim Graham wrote:
>>>
>>> This seems okay at first glance, though I wonder if an enum-like object
>>> might be better than magic int/boolean values. Something like:
>>>
>>> class ApproximateWith(object):
>>> NONE = 0
>>> YEARS = 1
>>> MONTHS = 2
>>> DAYS = 3
>>>
>>> Do you think a separate ModelAdmin attribute better than allowing
>>> something like:
>>>
>>> date_hierarchy = ('pub_date', ApproximateWith.YEARS)
>>>
>>> Values that can be string or tuple have contributed to bugs in the past,
>>> but I thought it might be worth proposing.
>>>
>>> On Saturday, August 29, 2015 at 4:50:20 PM UTC-4, Gavin Wahl wrote:
I'm going to fix ticket #23242 by adding an option to approximate the
date_hierarchy options instead of calculating them exactly. The
implementation is straightforward but I'm not sure what the interface
should be.
The basic idea is instead of doing `SELECT DISTINCT date_trunc('year',
created_at) FROM table`, then a date_trunc('month', created_at) to get the
list of months, and so on, you can find the range with `SELECT
date_trunc('year', max(created_at)), date_trunc('year', min(creeated_at))`
and just assume that every year/month/day in between is filled in.
How should this feature be enabled? Should it be possible to
approximate the year list, but once you've selected the year switch to
exact? I'm thinking something like:
- approximate_date_hierarchy = False # default
- approximate_date_hierarchy = 1 # approximate years only
- approximate_date_hierarchy = 2 # approximate months and years
- approximate_date_hierarchy = 3 or True # approximate days, months,
and years
--
You received this message because you are subscribed to the Google Groups
"Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/8361534a-4cc9-48c8-a991-f84bd77a0b98%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: Feature Proposal: per-settings fake migration override
Hi,
On Saturday 19 September 2015 14:02:38 Markus Holtermann wrote:
> On September 19, 2015 6:38:59 AM GMT+10:00,
> Flavio Curella wrote:
> >
> ># settings/production.py
> >MIGRATION_FAKE = (
> >
> >('transactions', '0001_initial),
> >
> >)
>
> This sounds like a very risky and scary proposal, to be honest. The
> recommended way would indeed be to fake the initial migration(s) on
> production.
>
> Since it seems to me that this would only ever be needed once (you said
> devs should just run the migration locally and the table is too big for
> any changes) I'd probably go with the one time fake.
>
> Alternatively, as of Django 1.9 you could move the transaction log model
> into a dedicated app and disable migrations for that app.
>
> As a last resort, why not make the model unmanaged (Meta.manage=False) and
> apply the respective operations manually with RunSQL operations?
>
> That are the ideas that just came to mind, there are probably others as
> well.
>
My instinctive reaction was that
A) This is a highly specialized case, which doesn't warrant framework support;
B) I'd solve it (at the project level) by putting some identifying setting in
settings/production.py, something along the lines of
# settings/production.py
FAKE_MIGRATIONS_FOR_PROD = True
and add to the initial migration a RunPython operation which looks for this
setting and raises an exception if it's there. Just to make the failure
explicit -- if the migration tried to create an existing table, it would fail
anyway.
Note that, if the model is in its own app, migrations already have a mechanism
in them to auto-fake the migrations of an app where all model tables exist.
(and note that this thread is already drifting into django-users' domain)
Shai.
Re: Feature Proposal: per-settings fake migration override
On Saturday, September 19, 2015 at 6:03:06 AM UTC-5, Markus Holtermann
wrote:
>
> Hi Flavio,
>
> This sounds like a very risky and scary proposal, to be honest. The
> recommended way would indeed be to fake the initial migration(s) on
> production.
>
>
Could you elaborate on this point? I have the feeling my proposal might
have problematic consequences I have not thought of, and you did.
> Since it seems to me that this would only ever be needed once (you said
> devs should just run the migration locally and the table is too big for any
> changes) I'd probably go with the one time fake.
>
> Alternatively, as of Django 1.9 you could move the transaction log model
> into a dedicated app and disable migrations for that app.
>
> As a last resort, why not make the model unmanaged (Meta.manage=False) and
> apply the respective operations manually with RunSQL operations?
>
> That are the ideas that just came to mind, there are probably others as
> well.
>
I think my mistake here was calling this proposal a 'feature', when it's
more of an enhancement.
There are currently many ways to handle this and similar scenario. The goal
of my proposal was to enable a more convenient way.
>
> /Markus
>
> On September 19, 2015 6:38:59 AM GMT+10:00, Flavio Curella <
> flavio@gmail.com > wrote:
> >I'm not sure if this warrants a DEP, so I've decided to post it to the
> >list
> >first. If this needs to be a DEP, just let me know and I'll follow the
> >DEP
> >process.
> >
> >Rationale
> >-
> >
> >Assume the following scenario:
> >
> >1. At some point in history, project A was deployed in production and
> >was
> >very successful.
> >2. The project created a table called `transactions_log`, which is now
> >considerably big and full of very valuable data, that we want to
> >retain.
> >3. Because of $reasons$, project B gets started as a future replacement
> >of
> >project B
> >4. Project B wants to reuse `transactions_log` and the
> >`transaction.models.Log` model unaltered
> >
> >What we have is a 'split' between what the initial migration for the
> >`transaction.models.Log` model should do, between dev and production:
> >
> >* On local dev, it simply creates the table
> >* On production, the migration must be faked.
> >
> >Traditionally, we would go to the person in charge of migration and
> >tell
> >them 'Hey, migration `transactions.0001_initial` must be faked'.
> >But this seems error-prone. That information ('fake this one specific
> >migration') could be lost in the communication workflow, and imposes
> >the
> >cognitive burden of having to remember another special case.
> >
> >Proposed solution
> >
> >
> >Assume the following settings files:
> >
> >```
> ># settings/local.py
> ># ..usual list of settings...
> >```
> >
> >```
> ># settings/production.py
> >MIGRATION_FAKE = (
> >('transactions', '0001_initial),
> >)
> >```
> >
> >* On `django-admin.py migrate --settings=settings.local`, the migration
> >
> >`transactions.0001` will be faked only if the `--fake` flag is passed
> >(previous behavior).
> >* On `django-admin.py migrate --settings=settings.production`, the
> >migration `transactions.0001` will *always* be faked. The new setting
> >will
> >act as an ad-hoc, per-migration override of the global `--fake` option.
> >
> >I have a branch up as a Proof of Concept at
> >https://github.com/fcurella/django/tree/feature/adhoc-fake (no test or
> >doc
> >changes yet). The code change necessary seems to be minimal, let me
> >know if
> >you think it's something worth having.
> >
> >Thank you,
> >–Flavio.
>
>
--
You received this message because you are subscribed to the Google Groups
"Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/91388a3f-b62b-4c4c-b05f-de8cca3e4459%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: Feature Proposal: per-settings fake migration override
On Saturday, September 19, 2015 at 8:39:54 AM UTC-5, Shai Berger wrote:
>
> Hi,
>
> On Saturday 19 September 2015 14:02:38 Markus Holtermann wrote:
> > On September 19, 2015 6:38:59 AM GMT+10:00,
> > Flavio Curella > wrote:
> > >
> > ># settings/production.py
> > >MIGRATION_FAKE = (
> > >
> > >('transactions', '0001_initial),
> > >
> > >)
> >
> > This sounds like a very risky and scary proposal, to be honest. The
> > recommended way would indeed be to fake the initial migration(s) on
> > production.
> >
> > Since it seems to me that this would only ever be needed once (you said
> > devs should just run the migration locally and the table is too big for
> > any changes) I'd probably go with the one time fake.
> >
> > Alternatively, as of Django 1.9 you could move the transaction log model
> > into a dedicated app and disable migrations for that app.
> >
> > As a last resort, why not make the model unmanaged (Meta.manage=False)
> and
> > apply the respective operations manually with RunSQL operations?
> >
> > That are the ideas that just came to mind, there are probably others as
> > well.
> >
>
> My instinctive reaction was that
>
> A) This is a highly specialized case, which doesn't warrant framework
> support;
>
>
Other scenarios came to my mind, but I didn't mentioned them to keep the
proposal simple.
One other common scenario is for when you have to run slight different SQL
in production versus development, for example having to `CREATE INDEX ...
CONCURRENTLY`, which is a quite common necessity.
That said, if you think this proposal will encourage bad practices, I'm
with you and I'd rather drop it than provide an easy way for users to shoot
themselves in the foot.
> B) I'd solve it (at the project level) by putting some identifying setting
> in
> settings/production.py, something along the lines of
>
> # settings/production.py
> FAKE_MIGRATIONS_FOR_PROD = True
>
> and add to the initial migration a RunPython operation which looks for
> this
> setting and raises an exception if it's there. Just to make the failure
> explicit -- if the migration tried to create an existing table, it would
> fail
> anyway.
>
> Note that, if the model is in its own app, migrations already have a
> mechanism
> in them to auto-fake the migrations of an app where all model tables
> exist.
>
> (and note that this thread is already drifting into django-users' domain)
>
> Shai.
>
--
You received this message because you are subscribed to the Google Groups
"Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/c92af8e8-338c-4813-a5a0-fe28b978a58f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Fellow Report - September 19, 2015
On the infrastructure side, I added Python 3.5 support to Jenkins (Django
1.9 will support it) and removed our own “wheelhouse” of cached wheels
since Pip 7+ now provides that functionality. I also completed the “new
stable branch tasks” for bootstrapping Django 1.10 which can be merged
after we create the stable/1.9.x branch (see “Authored” section for the
pull request).
Triaged
---
https://code.djangoproject.com/ticket/25402 - logging.dict.dictConfig
should not be called when LOGGING_CONFIG is defined (won’t fix)
https://code.djangoproject.com/ticket/25405 - FlatpageFallbackMiddleware
doesn't find pages with leading slashes (worksforme)
https://code.djangoproject.com/ticket/25409 - Allow url and groups of urls
to be easily tagged and selected (someday/maybe)
https://code.djangoproject.com/ticket/25428 - Allow configuring the
database's identifier max length in the DATABASES setting (won’t fix)
Authored
https://github.com/django/django/pull/5294 - Refs #24215 -- Fixed Python
3.5 compatibility for unhandled lazy ops error.
https://github.com/django/django/pull/5143 - Bootstrapped Django 1.10.
Reviewed/committed
--
https://github.com/django/django/pull/5281 - Fixed #25393 -- Fixed MySQL
crash when adding text/blob field with unhashable default.
https://github.com/django/django/pull/5260 - Fixed #25377 -- Changed Count
queries to execute COUNT(*) instead of COUNT('*').
https://github.com/django/django/pull/5268 - Fixed #25404 -- Added line
numbers to TemplateSyntaxError strings.
https://github.com/django/django/pull/5164 - Fixed #25294 -- Allowed custom
BoundFields on forms.
https://github.com/django/django/pull/5288 - Refs #25149 -- Fixed
regression in admin date-time widget for timezones on the negative side of
UTC.
https://github.com/django/django/pull/4337 - Fixed #24496 -- Added CSRF
Referer checking against CSRF_COOKIE_DOMAIN.
https://github.com/django/django/pull/5249 - Refs #25294 -- Moved
BoundField to django.forms.boundfield.
https://github.com/django/django/pull/5302 - Fixed #25400 -- Fixed
regression in nonexistent features on gis backends.
https://github.com/django/django/pull/5219 - Refs #20625 -- Fixed custom
queryset chaining with values() and values_list().
https://github.com/django/django/pull/5221 - Fixed #25034 -- Converted
caches ImproperlyConfigured error to a system check.
https://github.com/django/django/pull/5307 - Refs #25422 -- Added a test
for a template tag with type annotations.
https://github.com/django/django/pull/5300 - Made assorted improvements to
the Oracle documentation.
https://github.com/django/django/pull/5136 - Fixed #25269 -- Allowed
method_decorator() to accept a list/tuple of decorators.
https://github.com/django/django/pull/5200 - Fixed #13110 -- Added support
for multiple enclosures in Atom feeds.
https://github.com/django/django/pull/5190 - Fixed #24944 -- Added
extra_email_context parameter to password_reset() view.
https://github.com/django/django/pull/5299 - Fixed #25296 -- Prevented
related object cache pollution when create() fails due to an unsaved object.
Reviews of core dev work
https://github.com/django/django/pull/5282 - Refs #14091 -- Substituted
parameters in connection.queries on SQLite.
https://github.com/django/django/pull/5277 - Fixed #25390 -- Allowed
specifying a start migration in squashmigrations.
https://github.com/django/django/pull/5303 - Fixed #25417 -- Added a field
check for invalid default values.
https://github.com/django/django/pull/5090 - Fixed #24629 -- Unified the
expressions and transform API.
--
You received this message because you are subscribed to the Google Groups
"Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-developers/03c5e4f3-b736-4fdb-90fc-a5693c08e631%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
