(Fixed Ryan's link: https://github.com/aspiredu/django-safemigrate )
On Sat, 14 Sep 2019 at 17:12, Adam Johnson <m...@adamj.eu> wrote: > I'm interested in your use case: why do you need the model changes without >> the migrations? I.e. why would you want to be in a state that's >> inconsistent between the ORM and the DB? Is it just so you can recover >> more easily if something goes wrong? >> > > It's not that one *wants* to be in that state, it's just inevitable with > the goal of zero downtime. With reasonable traffic on even one webserver, > it's inevitable there's a gap between the running of migrations and the > updating of the webserver to the new code. In this case it's best to make > sure both old and new code versions are compatible with the new version of > the database. > > There's plenty out there on the topic. A post I found useful recently was > this one from Braintree covering the different ways to help zero downtime > deployments PostgreSQL: > https://medium.com/braintree-product-technology/postgresql-at-scale-database-schema-changes-without-downtime-20d3749ed680 > > Thanks, > > Adam > > On Sat, 14 Sep 2019 at 16:09, Dylan Young <dylanyoungmei...@gmail.com> > wrote: > >> Thanks for the tips; I'll take a look at those code paths. >> >> I'm interested in your use case: why do you need the model changes >> without the migrations? I.e. why would you want to be in a state that's >> inconsistent between the ORM and the DB? Is it just so you can recover >> more easily if something goes wrong? >> >> Recent versions of Django make it easier to modify the test command, so >> I may take a look at spinning up an external package for this. >> >> Le sam. 14 sept. 2019 5:36 a.m., Adam Johnson <m...@adamj.eu> a écrit : >> >>> Hi Dylan, >>> >>> In my experience, high availability deployments often need to commit >>> model changes without migration changes. For example, when removing a >>> (nulable) field, the best approach is to: >>> >>> 1. Remove the field from Django >>> 2. Test >>> 3. Deploy >>> 4. Run makemigrations to generate the migration file that removes it >>> from the database >>> 5. Test >>> 6. Deploy >>> >>> The test run in step 2 should *not* have any automated migrations built. >>> The tests should confirm that removing the field from Django without >>> removing it from the DB safe. >>> >>> If you want to build your suggested behaviour (makemigrations before >>> every test run) you can do it by: >>> >>> 1. Creating a custom test runner that subclasses DiscoverRunner >>> 2. Overriding its setup_databases() method to do >>> call_command('makemigrations') before super() >>> 3. Catching the names of those made migrations, and >>> overriding teardown_databases() to delete them (or maybe not?) >>> >>> I'd be interested to know how that works! I suspect it will be fine for >>> smaller projects, but for larger projects that care about preventing >>> downtime during deployment, it will backfire. >>> >>> As an extra benefit, it makes testing between branches *so much* easier >>>> because you don't have to worry about rolling back migrations on the >>>> test_db (a PITA as far as I can see). >>>> >>> >>> If you're using the default behaviour, the test DB is generated on every >>> run. The `--keepdb` flag is optional and can be used to speed up test runs >>> on the same branch but you should run tests without it after switching. I >>> agree it can be a pain. >>> >>> Maybe we could have a guard on that flag: detect if git is used, and the >>> branch has changed since last run, and then warn or just hard recreate? If >>> you'd be interested in exploring this, it can also be tested by subclassing >>> the migrate command. Maybe make a ticket? >>> >>> Regarding the last, is there a flag to make migrate target the >>>> test_db(s)? >>>> >>> >>> Not that I know of but I did something similar on one project where I >>> had a management command to create a test runner and call its >>> setup_databases() method. That should be sufficient! >>> >>> Thanks, >>> >>> Adam >>> >>> On Fri, 13 Sep 2019 at 19:04, Dylan Young <dylanyoungmei...@gmail.com> >>> wrote: >>> >>>> This is an old issue that (as far as I can see) was never resolved. >>>> >>>> My proposed solution would be a flag to the test command to >>>> automatically `makemigrations && migrate`. When this behaviour is enabled, >>>> it would also roll back any generated migrations at the end of the test >>>> run. The flag might not be necessary (could be default behaviour of >>>> `test`) >>>> >>>> This way you get the benefits of all worlds: >>>> 1) uncommitted migration files will be clearly visible after the test >>>> run (for those inclined to forget to makemigrations). >>>> 2) The extraneous non-migration creation code can be removed from >>>> Django (probably already has been at this point). >>>> 3) Tests are always run against the current state of models >>>> 4) Tests are always run at current state of migrations (because it >>>> generates the migrations). >>>> >>>> As an extra benefit, it makes testing between branches *so much* easier >>>> because you don't have to worry about rolling back migrations on the >>>> test_db (a PITA as far as I can see). >>>> >>>> Regarding the last, is there a flag to make migrate target the >>>> test_db(s)? >>>> >>>> Best, >>>> >>>> Casey >>>> >>>> On Tuesday, 25 March 2014 14:21:51 UTC-3, Bernie Sumption wrote: >>>>> >>>>> Hi Django devs, >>>>> >>>>> I've just started a new project in 1.7b, and the development >>>>> experience working with unit tests on models is more complicated than it >>>>> was in 1.6. It's all down to how the throwaway test databases are created. >>>>> In 1.6, the create_test_db function "Creates a new test database and runs >>>>> syncdb against it." In 1.7, it runs "migrate". >>>>> >>>>> While generally speaking, migrate is the new syncdb, this behaviour is >>>>> not ideal for tests. In 1.6 "syncdb" created a database reflecting the >>>>> current state of the models in models.py. "migrate" creates a database >>>>> reflecting the state of the models at the last time makemigrations was >>>>> run. >>>>> If you're doing TDD and constantly making small changes to your models >>>>> then >>>>> runnning unit tests, you have to run makemigrations before each test run >>>>> to >>>>> get your tests to work. You therefore end up with many tiny migration >>>>> files >>>>> representing the minute-by-minute history of development. >>>>> >>>>> I came up with a pretty effective workaround that is working for me, >>>>> but I thought I'd post this here as others are sure to encounter this >>>>> issue, and I think that it makes more sense for the behaviour produced by >>>>> this workaround to be the default for running tests. >>>>> >>>>> If makemigrations has not yet been run, the "migrate" command treats >>>>> an app as unmigrated, and creates tables directly from the models just >>>>> like >>>>> syncdb did in 1.6. I defined a new settings module just for unit tests >>>>> called "settings_test.py", which imports * from the main settings module >>>>> and adds this line: >>>>> >>>>> MIGRATION_MODULES = {"myapp": "myapp.migrations_not_used_in_tests"} >>>>> >>>>> Then I run tests like this: >>>>> >>>>> DJANGO_SETTINGS_MODULE="myapp.settings_test" python manage.py test >>>>> >>>>> This fools migrate into thinking that the app is unmigrated, and so >>>>> every time a test database is created it reflects the current structure of >>>>> models.py. >>>>> >>>>> So my feature request is as follows: >>>>> >>>>> If the new behaviour is by design and considered desirable, then it is >>>>> a big change from the previous version and should be prominently >>>>> documented >>>>> in the migrations and testing pages, along with the workaround. I'm happy >>>>> to write this documentation if that's the way you want to go. >>>>> >>>>> However, if the new behaviour is not by design but just a by-product >>>>> of the new migrations feature, I suggest making the workaround the default >>>>> behaviour. I don't (yet!) know enough about Django internals to volunteer >>>>> for this however. >>>>> >>>>> Thanks for your time, >>>>> >>>>> Bernie :o) >>>>> >>>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Django developers (Contributions to Django itself)" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to django-developers+unsubscr...@googlegroups.com. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/django-developers/2aee60d5-e96b-4c96-8f87-150d9de46215%40googlegroups.com >>>> <https://groups.google.com/d/msgid/django-developers/2aee60d5-e96b-4c96-8f87-150d9de46215%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> >>> >>> -- >>> Adam >>> >>> -- >>> You received this message because you are subscribed to a topic in the >>> Google Groups "Django developers (Contributions to Django itself)" group. >>> To unsubscribe from this topic, visit >>> https://groups.google.com/d/topic/django-developers/PWPj3etj3-U/unsubscribe >>> . >>> To unsubscribe from this group and all its topics, send an email to >>> django-developers+unsubscr...@googlegroups.com. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/django-developers/CAMyDDM3SKV07PPY3fBxyii4MmQHHAyqB6LJERugNr8nupr-FnQ%40mail.gmail.com >>> <https://groups.google.com/d/msgid/django-developers/CAMyDDM3SKV07PPY3fBxyii4MmQHHAyqB6LJERugNr8nupr-FnQ%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "Django developers (Contributions to Django itself)" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to django-developers+unsubscr...@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/django-developers/CAPGJNu54auVLyKALu4JsHnyyRDFs1mDa%3DPcBV96Dz8w-0n_CQQ%40mail.gmail.com >> <https://groups.google.com/d/msgid/django-developers/CAPGJNu54auVLyKALu4JsHnyyRDFs1mDa%3DPcBV96Dz8w-0n_CQQ%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> > > > -- > Adam > -- Adam -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAMyDDM0rbbPD%2BNxfbj_QkwxgbGMb01tV_%3DWfozBbYVjwKGR4YQ%40mail.gmail.com.
