[Dropping the bug tracker. I suspect that the first part of this message may be of more general interest, and the latter (excepting perhaps the "how we want to handle epochs" bit that mentions tag encoding) will be primarily of interest to Arnaud, where it goes into detail regarding the current packaging process.]
Arnaud Fontaine <ar...@debian.org> writes: > I have almost finished preparing emacs-snapshot[0] and as I said to Rob, > my plan is to keep emacs-snapshot package as close as possible to > emacsXY package. I know that you've been working on unversioning emacs > package (BTW Where is that repository?). I haven't published any of the changes yet, but I'll push them to the current salsa repository once they're ready (likely soon): https://salsa.debian.org/rlb/deb-emacs > * bin_priority (for update-alternatives): I think it would make sense > to have an higher one for emacs-snapshot, I don't have a strong opinion about this at the moment, but I wondered if it might also be possible to argue the opposite, i.e. that emacs-snapshot's priority should be low, so that if you had both emacs and emacs-snapshot installed, you'd have to explicitly request emacs-snapshot be the default, either globally by update-alternatives, or locally with a suitable ~/bin symlink. That way you'd only run snapshot in cases where you knew you wanted it, and other things (particularly anything that's automated) would still use the "stable" emacs. On multi-user systems, such an arrangement might be even more desirable -- not as sure about single-user systems. > what about a number big enough so it doesn't clash with future stable > release (such as 999)? Let's set this aside until we know what what we want to do with respect to the ordering question above, but if we end up needing it, I was pondering some kind of encoding like: 1 four-major-digits 0-or-1-indicating-snapshot-status i.e. 100250 or something else that lets us sort the way we want (in this case assuming the upstream doesn't produce emacs 1000.0 in a timeframe we care about). > * I have made several changes to emacs25 branch, feel free to merge > them if they look fine to you (mostly debian/copyright work and > fixing lintian warnings): > https://salsa.debian.org/arnau/deb-emacs/commits/deb/emacs25/d/sid/master Let's plan to work on this once I get the unversioning finished -- it's imminent (most of it was finished just after debconf last year, but then I got side-tracked). I'll be happy to help figure out how to handle any conflicts, etc. > And some packaging questions I will document in debian/README.source: > > * Do you plan to keep using git-dpm? (I'm asking because Dima, the > current packager of emacs-snapshot, is using gbp-pq. I have no > preference/opinion on this subject, just asking) Yes, I do for now -- I've been using it for a good while, and I use it in my other packages, so while I'm not opposed to switching if/when something else is available that handles the main things I find helpful about git-dpm, I'd like for it to appear worth the cost, and it'll take some time. > I ran the following command after importing patches from emacs25 and > merging them with the ones from emacs-snapshot from Dima: > > $ git-dpm init --record-patch-name > ../emacs-snapshot_20180414-1+git836dce6.orig.tar.xz > deb/emacs-snapshot/d/sid/upstream > > However, I'm not so familiar with git-dpm, so would you mind > explaining how you use it for emacs25/unversioned emacs? Happy to, and note that I'm just describing what I currently do, not arguing that any particular bit has to be done this way... When beginning work on a new package revision, I make sure I'm on deb/PACKAGE/d/sid/master and: * gbp dch --git-author --git-log=--first-parent -aS --meta --full (yes, I use gbp-dch because it handles --meta, etc.) * git commit -am 'Adjust debian/changelog for UNRELEASED 25.2+1-6 development' then if I only need to change debian/* I just use git normally, making commits as appropriate, and (generally) writing the commit messages with an eye toward the summary line and some or all of the body becoming an entry in debian/changelog. I also include Closes: #NNNN lines at the end of any relevant commit messages, and I always try to remember to include appropriate thanks to those who have helped with the changes. Then when I'm ready to release: * gbp dch --git-author --git-log=--first-parent -aR --meta --full * Edit debian/changelog to clean up the entries to add punctuation to the summary line if dch didn't, avoid splitting Closes entries across lines, fill lines (M-q), and add blank lines (C-j) between entries (because it's been that way for a long time, and because I'm still inclined to prefer reading it that way -- though I don't feel all that strongly about it). * git commit -am 'Update debian/changelog for 25.2+1-6 release' Then I prepare the packages. I often use fakeroot debian/rules binary, etc. for non-release builds, but for the releases: * sbuild-update -udcar unstable-amd64-sbuild # If it's stale * sbuild -d unstable -sA --build-failed-commands='%SBUILD_SHELL' * Poke at the packages: install them locally, see if notmuch still works, etc. * If all is well, sign them, "git tag -s deb/PACKAGE/v/VERSION -s deb/PACKAGE/v/VERSION", and upload. Whenever I need to edit existing upstream patches, I do the following (most often separately, once for each patch that needs changing): * git-dpm checkout-patched * git rebase -i HASH_PRECEDING_PATCH_THAT_NEEDS_EDITING # Set 'e' for the patch I want to change * Make the changes. * git rebase --continue * git-dpm update-patches * git commit --amend I do that last amend to change the merge commit message to something suitable for the debian/changelog (via gbp-dch), which is why I handle each patch separately in most cases. Of course it's possible there might be a coordinated change that affects multiple patches that should be handled with a single merge (all withing the same checkout-patched/update-patches sequence). Whenever I need to create a new upstream patch: * git-dpm checkout-patched * git cherry-pick UPSTREAM # or git-am, or ... * Edit commit message(s) to look like those in the other patches, i.e. write them in NEWS style given that we automatically generate NEWS entries from the patch headers. I also try to remember to add appropriate DEP-? pseudo-headers. * git-dpm update-patches * git commit --amend During the amend, as for non-debian/ changes above, I change the merge commit message to something suitable for incorporation into debian/changelog via gbp-dch (not NEWS style). For example: Block remote code execution via enriched text Add 0012-A-remote-execution-exploit-via-enriched-text-has-bee.patch to fix the problem. Thanks to David Bremner for the alert and Salvatore Bonaccorso for reporting the problem to Debian. Closes: 875447 All of the above approximately captures "steady state" work, but as you might imagine, given the DFSG split, incorporating new upstream changes is more difficult. I'll discuss that process in answer to your other question below. > * I followed deb-emacs25[1] naming scheme for branches and tags, thus: > + Branches: > deb/emacs-snapshot/d/sid/master > deb/emacs-snapshot/d/sid/upstream > + Tags: > deb/emacs-snapshot/v/upstream/20180414-1+git836dce6 Looks good -- though we'll have to ponder how we want to handle epochs (now that the unversioned package needs to add one). Sean said that there was a DEP suggesting they be included in versions via "%", e.g. 1%20180414-1+git836dce6. I'm not sure what I think about that yet -- locally I've just created two tags VERSION and 1%VERSION. If there's no harm in that, then maybe it'd be the most flexible... > I guess unversioned emacs Git repository follows this scheme too, > right? Yep - the unversioned work very closely matches the existing work, just with a flavor of emacs instead of emacsXY. > * Could you please explain how you remove non-DFSG documentation (such > as emacs.texi) from the Git repository? Yep. Unsurprisingly I imagine, this is one of the less entertaining, and more complicated bits. Though the incremental work is notably less than the original split was. Below I'll describe my current git-dpm based process. Some of this is work that would have to be done regardless, and some of it is git-dpm specific. And note that when I new release comes out, we completely ignore the upstream archive since we're going to have to repackage it anyway. The packages are built from the upstream tags. Also, I've created all of the instructions in this mail from notes and/or memory, so be somewhat cautious, e.g. double-check that the commands seem sensible if you try them. To handle a new major release Y, given previous major X (minor releases are simpler, but handled similarly): * Validate upstream tag if there's a sig. Since there doesn't currently appear to be a solid trust chain between my keyring and theirs, I rely on the key I've seen in the previous release(s): GNUPGHOME=some/where/release-gnupg git tag -v emacs-Y.1 * Update .git/config to add git-dpm entries for new branches, i.e.: [branch "deb/emacsY/d/sid/master"] dpmUpstreamBranch = deb/emacsY/d/sid/upstream dpmPatchedBranch = deb/emacsY/d/sid/patched * git checkout -b deb/emacsY/d/sid/master deb/emacsX/d/sid/master * gbp dch --git-author --git-log=--first-parent -aS --meta --full -N Y.1+1-1 * git commit -am 'Adjust debian/changelog for UNRELEASED Y.1+1-1 development' At this point, in general, we just merge the new upstream into the existing upstream, but there could be cases where we'd want to "start over", i.e. branch .../d/sid/upstream directly from the upstream tag. For the typical merging case: * git checkout -b deb/emacsY/d/sid/upstream deb/emacsX/d/sid/upstream * git merge emacs-Y * Use various commands (git, etc.) to figure out how to handle the dfsg split. Try to determine what new or existing files have to be removed, and whether any previously removed files can be re-added. I imagine we could automate some of this, but especially in the past, and unless we were planning to generalize it for "other packages", this work was fairly rare and so I've just handled it in an ad-hoc manner. During this process, keep track of any corresponding changes that will need to be made to the non-dfsg package. Examples from my rough notes (untested): - Check additions/deletions/renames and add/remove them from each repo as needed (probably a way to have git handle this directly): diff -u <(git ls-files --with-tree emacs-X) \ <(git ls-files --with-tree emacs-Y) | less ...rm new files non-dfsg files, i.e. anything with gfdl invariants (front/back/sections), etc. - Check path changes (i.e. new/deleted paths, etc.). Check deleted-by-us to make sure license hasn't changed, etc. and git-rm any that should still be gone, etc. git diff --diff-filter=ACRB(maybe D) --name-status emacs-X emacs-Y - Of course, we can also compare the previous sid/upstream upstream against the new tag's tree. * Commit the final tree with a suitable commit message - perhaps even just starting with an existing one (cf. git commit -c REF), e.g: commit 53dad1e8f5cd98546a215217b1f6dcd74fe03981 Author: Rob Browning <r...@defaultvalue.org> Date: Mon Apr 4 22:46:24 2011 -0500 Remove files that appear to be incompatible with the DFSG Files that appear to be incompatible with the DFSG have been removed. A number of files have been removed from this package because their licenses are not compatible with the Debian Free Software Guidelines (DFSG), or because it wasn't completely clear that their licenses are compatible. ... Once we have the upstream tree we want: * git archive --prefix=emacsY-Y.1+1.orig/ deb/emacsY/v/Y.1+1 | xz -9v > \ ../emacsY_Y.1+1.orig.tar.xz * pristine-tar commit ../emacsY_Y.1+1.orig.tar.xz deb/emacsY/v/Y.1+1 * chmod a-w ../emacsY_Y.1+1.orig.tar.xz * git tag -sm deb/emacsY/v/Y.1+1 deb/emacsY/v/Y.1+1 * git-dpm record-new-upstream ../emacsY_Y.1+1.orig.tar.xz deb/emacsY/d/sid/upstream * git-dpm rebase-patched * git-dpm update-patches * git commit --amend --no-edit -m 'Merge upstream version Y.1' * Fix debian/rules, etc. for major version change (bin_priority, etc.). * debian/rules debian-sync * git grep -F OLDVERSION debian # (and fixup and oversights) * git commit -am 'Update debian/ for Y.1' * Read NEWS; maybe add commits to adjust dependencies, ./configure, etc. * gbp dch --ignore-branch --git-author --git-log=--first-parent -aR --meta --full * git commit -am 'Update debian/changelog for Y.1+1-1 release' * Examine/test packages, etc. * If all is well, sign them, "git tag -s deb/PACKAGE/v/VERSION -s deb/PACKAGE/v/VERSION", and upload. For what it's worth, I often make a copy of my whole git tree before beginning work I'm not certain about, so that it's trivial to "revert" if I make a mistake, or if some command doesn't do what I expected, (particularly for major updates). Though of course it'd usually be possible to just note the relevant tags and refs before you start so that you can roll them back manually if you need to -- but "cp -a" and "mv" are simpler. Hope this helps. -- Rob Browning rlb @defaultvalue.org and @debian.org GPG as of 2011-07-10 E6A9 DA3C C9FD 1FF8 C676 D2C4 C0F0 39E9 ED1B 597A GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4
