Want to share lib I work previously for migration safety with postgres some time ago: https://github.com/tbicr/django-pg-zero-downtime-migrations there are also checker and replacement of unsafe mirations with safe replacement.
ср, 23 дек. 2020 г. в 12:18, Adam Johnson <m...@adamj.eu>: > Hi Tom, > > I love this idea and would like to help developers write safe migrations > too. > > You didn't mention this pre-existing project: > https://github.com/3YOURMIND/django-migration-linter . It might be worth > checking how that works. > > I'm not sure what any public API could look like here. In > django-linear-migrations ( > https://github.com/adamchainz/django-linear-migrations ) I tapped a bit > into the MigrationLoader class, which is undocumented but fairly stable. > Perhaps we could just document a few more of the migration internals? > > On Tue, 22 Dec 2020 at 22:58, Tom Forbes <t...@tomforb.es> wrote: > >> Hey, >> A fairly common problem with large and/or highly trafficked web >> applications is migration safety: you want your migrations to be >> non-locking and instantaneous. The specifics are very database dependent >> but in Postgres simply altering the nullability of a column can incur >> serious downtime depending on the size of the table in question. The same >> thing goes for adding a new non-nullable column to a table or even just >> creating an index without the `CONCURRENTLY` option. >> >> Code review is one way of catching these issues but an automated solution >> that runs as part of the test suite would be the preferred solution. I >> don’t think using some kind of linter for this is a good idea as it doesn’t >> have the context to decide if it _is_ a safe migration (i.e removing `NOT >> NULL` from a column in the same migration that creates the column is safe). >> A linter also doesn’t know which migrations are pending and which have been >> applied. >> >> Rails has the pretty nice strong_migrations Gem ( >> https://github.com/ankane/strong_migrations) which attempts to detect >> dangerous migrations. Unless I’m missing something right now it seems quite >> hard to add automated checks like this to Django migrations that can >> inspect the full set of “pending migration operations". We have the >> `pre_migrate` signal that takes a “plan” argument with the scary sounding >> caveat that the passed object has no public API, but using a signal feels >> kind of wrong. It should be part of a system check - i.e a warning that >> says “your app may die if this migration is applied”. >> >> I’d like to make this simpler to implement on a per-project basis (or in >> a third party app) but I’m not sure how. It should be a system check but >> the only suitable hook we have is the generic `register()` callback that >> takes a set of `app_configs`, and I’d expect going from that to a pending >> set of operations would be non trivial. A user-facing interface for this >> might be a function that is called for every specific type of migration >> operation with the operation itself and a list of all operations that would >> be applied of “migrate” was run. >> >> ``` >> @some_migration_callback_decorator(AlterField) >> def check_null(operation, pending_operations): >> If operation.is_going_to_not_null: >> if not table_being_created(operation.table_name, >> pending_operations): >> return Warning(f’Setting {operation.field_name} to NOT NULL >> may cause downtime’) >> ``` >> >> I’m not sure if Django should ship migration warnings itself but having >> the ability for projects to enforce arbitrary constraints on their >> migrations might be an interesting feature? >> >> Slightly related to https://code.djangoproject.com/ticket/31700, where >> we want to add more contextual/coloured outputs for dangerous operations. >> >> Tom >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Django developers (Contributions to Django itself)" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to django-developers+unsubscr...@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/django-developers/CAFNZOJNoi583wUoQp%2BRBB_%2B3i_vqSPRbVNwhiU_OG7V2eL8x3g%40mail.gmail.com >> <https://groups.google.com/d/msgid/django-developers/CAFNZOJNoi583wUoQp%2BRBB_%2B3i_vqSPRbVNwhiU_OG7V2eL8x3g%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> > > > -- > Adam > > -- > You received this message because you are subscribed to the Google Groups > "Django developers (Contributions to Django itself)" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to django-developers+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/django-developers/CAMyDDM1vtHgDU2VO-kyV9CM%2BxXRPbPmHKgUVJqYGxd2POdou%2BQ%40mail.gmail.com > <https://groups.google.com/d/msgid/django-developers/CAMyDDM1vtHgDU2VO-kyV9CM%2BxXRPbPmHKgUVJqYGxd2POdou%2BQ%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAL6gfQY-hCG3pH7asZ1GCqBuOOpYnOHgzMDMfyaYVpEkme4s8A%40mail.gmail.com.
