Ian Jackson <ijack...@chiark.greenend.org.uk> writes: > I'm not sure why that's a problem for emacs addons. In this scheme > you only need to run *compilation* of some addon during that addon's > postinst, and the postinst of the emacs flavour. In each case you can > do processing only of packages that are `configured`, plus this one. > > The cleanup tasks don't seem like they'd need much in the way of > working dependencies.
A general issue is that the current approach was designed with incorrect assumptions, so any of the corner cases (and not so corner cases) that we'd thought through carefully when it was established might now (and are likely now to) be broken. As an example related to the failures that precipitated all this, the current emacs policy requires all add-ons to depend on emacsen-common, and for them to call its commands at various points to handle installs/rebuilds and removals. i.e. emacsen-common is the library/subsystem that handles all the dependency ordering, etc. But that's just wrong if you can't assume much about the state (or even availability) of emacsen-common from those maintainer scripts, i.e. when we designed all this, I would never have guessed that even if you depend on emacsen-common, it might not actually be ready (or even unpacked) from say your prerm. That was a surprise, and means we have to come up with an approach that's sound given what actually *is* promised, and at least as far as dependencies go, it sounds like the only place you can (nearly always(?)) count on your dependencies is "postinst configure" (and from the triggers docs, I'd thought now maybe also "postinst triggered"). > However, note that dpkg does *not* bring packages out of `installed` > just because their dependencies regress to less good states, provided > the dependency isn't actually removed. Hmm, what kind of "less good state"? > ISTM that you ought to aim to promise that once an add-on is in state > `installed` it is ready (subject to the caveat above). That would > mean any things that invoke the addon, and Depend on it, will work > properly. Oh, of course -- that's how the entire current policy is designed. It was set up to make as few assumptions as possible about what an emacs flavor or add-on might need to do to configure or clean itself up, while also providing an inter-add-on dependency respecting install/removal process. ...but now that we have a lot more experience with add-ons, it could be that we don't actually need such a general mechanism, and if so, that's a possibility worth keeping in mind, in case it allows simpler solutions. It's also perhaps worth noting that we don't actually care about triggers per se. I'd just started looking at them when among other things, I wondered if they might be able to eliminate the need for our own (currently a bit sketchy) dependency ordering. > Let's consider three packages: emacsen package E (of some flavour); > addon packages A and B where B uses features from A. > > When we reinstall A we need to rebuild both it and B. > > We could achive this by doing it in A's postinst but in a large run > it'll probably have to done again later, eg if emacsen are being > updated too. The bigger issue I'd have imagined is that the state of B is not known at that point? > Realistically, E must be interested in A, so that A gets rebuilt. (We > don't want to do this the other way around, because that would cause > an emacs to be considered not-`installed` simply because some addon > hadn't been compiled.) Without having thought about it carefully -- that might be OK... i.e. it might be fine if the whole system (add-ons and flavors) comes out of "up to date" for a bit, as long as it re-converges eventually. Particularly if (as may well be the case now for most flavors and add-ons), the packages still "work fine" in the interim. Say if the install/remove process really was just recompilation, and the preinsts could adequately clean everything up, then in that arrangement, the worst you might get is some extraneous per-user compilation artifacts if a user were to run an emacs flavor before all the recompilations were finished. > If we update E, E's postinst will need to rebuild both A and B. So > E's postinst needs to be able to do a topological sort of the addons, > so that it can compile them in the right order. > > Given that E's postinst can do the sort, it can reliably recompile > everything. But from E's postinst, we have no idea what state the add-ons are in? And if so, and we had to be "completely conservative", the most we could do is rebuild any dependency graph sub-trees we could find where the add-ons were past their "postinst configure"s ... and then we'd have to ensure we get to the rest later somehow. One correct solution might involve some command we know will always "run last" in any given apt/dpkg run, and I'd wondered if triggers might be able to support that. > dpkg tries to defer trigger processing. And you mustn't "defer work" > and return success from a trigger processing postinst invocation, > since you might never be called again. (I guess you could manually > retrigger but this seems fraught and also unnecessary.) I think retriggering might be necessary for the "goes last" approach(es). Overall, of course, I'd like to understand the facilities we currently have available, and exactly what they promise while reasoning about solutions. I think I have a better handle on what dpkg promises with respect to dependency states from maintscript invocations, i.e. generally, not much, in the limiting case, outside "postinst configure" (and now maybe "postinst triggered"). There it's likely, and as likely as it'll ever be, that your deps will be configured -- elsewhere, not so much. I'd also like to understand what we might be able to count on with respect to triggers. For example, is it plausible to put them in roughly the same class as "postinst configure" with respect to the dependencies will be available/configured there? More concretely, and for any who haven't seen it, included below is one tentative idea we had been considering as a possible way to use triggers to fix the current issues. Don't take the specific details too seriously yet, and I would love to find out that we/I'm just over-complicating things somehow. (If it actually is a sensible approach, I do have some of the code sketched out, and it also has some bits in common with the current policy/implementation.) # Overview: The general approach is to maintain relevant state files in /var/lib/emacsen-common/..., to manipulate those to reflect changes, and to then trigger all the add-ons to re-evaluate the state to accommodate changes (in some cases, an add-on will decide there's nothing to do). We currently assume that "postinst triggered" invocations are as likely to have the package dependencies configured as "postinst configure". Some testing has suggested that's the case (e.g. invocations were ordered with respect to dependencies), but if that's not generally true, then this approach may not be suitable. This arrangement has the advantage of letting the packaging infrastructure handle the dependency ordering as compared to the current implementation where we try to parse dpkg deps output, tsort any add-ons found, etc. # Approach: Have an "installed" state file for each emacs flavor that's touched whenever that flavor is changed and only exists if the last postinst configure/triggered for that flavor was successful, i.e. it indicates whether add-ons should consider that flavor when building. Have a "configured" state file for each add-on that exists whenever the package has been successfully (postinst) configured. It may or may not be installed for every flavor yet. It may also not exist if the add-on is out of date with respect to its dependencies. Have a per-flavor "installed" state file for for each add-on that's touched whenever that add-on is successfully installed for the given flavor. i.e. if it doesn't exist or is stale relative to the flavor's corresponding state file, then we need to reinstall for that flavor. Whenever a flavor changes, trigger emacsen-common-add-on-assess-rebuild All add-ons will be sensitive to that trigger, and will use the installed state files to reinstall for the relevant flavors. Whenever an add-on changes, remove the configured files for all reverse dependency add-ons, and trigger emacsen-common-add-on-assess-rebuild. Each add on and flavor package must include the relevant (empty) emacsen-common state directories. # Background: https://people.debian.org/~srivasta/MaintainerScripts.html#sec-6 https://www.debian.org/doc/debian-policy/ch-maintainerscripts.html#summary-of-ways-maintainer-scripts-are-called Thanks -- Rob Browning rlb @defaultvalue.org and @debian.org GPG as of 2011-07-10 E6A9 DA3C C9FD 1FF8 C676 D2C4 C0F0 39E9 ED1B 597A GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4
