> As an example of why "backup the db" is harder than it sounds, you would
> need to backup the entire storage subsystem. 

Mossop suggested I chime in on this thread, but I think Ben has covered much of 
what I'd say!

Some set of schema changes is safe or possible to downgrade: rearranging 
deckchairs, reversible optimizations, etc.

Some additional set can be downgraded thanks to semantic restrictions in place: 
adding a purely optional field, for example. (That doesn't mean that it's 
trivial to make SQLite or another disk format behave correctly, of course, and 
one can't always tell when an optional field isn't really optional in every 
circumstance….)

A broader set is very difficult to downgrade. Code to get the downgrade right 
is hard to fix in the wild and hard to test. Systems that spread data across 
multiple data sources (multiple files, prefs, caches…) get really hairy.

Leaving old files on disk is almost never the correct solution: think of the 
effect that this has on Sync, for example. If you're thinking about leaving 
files on disk for anything other than disaster recovery, you should reconsider.

In short: it's hard to make non-trivial changes to a schema, particularly in 
SQLite, and expect to be able to hop between versions. The tradeoff in effort 
and bugs versus benefit is slanted. If we're communicating to our users that 
they can safely do so — "just try this out on Nightly with your main profile!" 
— we should stop doing so.

Two alternatives to consider:

- Support two storage versions in parallel for a while: ship the code and use 
the old one until the switch is flipped. This is acceptable if you're not in a 
rush to ship. Hands up if you're not in a rush to ship.
- Don't allow downgrades.

The difficulty of evolving storage was one of the motivations for Project 
Mentat. Schema fragments in Mentat can evolve additively without a version bump 
if the additions are safe. Clients can introspect the schema and operate in a 
degraded mode if they can get what they need. Additive fragments don't affect 
older fragments at all. And a higher-level schema makes some of the DB churn we 
see in our Firefoxes less likely.

Storage is hard.
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Reply via email to