Re: Git Packaging: Native source formats
Hi, On Thu, Aug 29, 2019 at 09:42:50PM +0200, Philipp Kern wrote: > Obviously I'm not bound to that format being "3.0 (native)" but some > "3.0 (dumb)" that just tars up the whole tree without caring about the > version scheme would then be nice to have as a replacement. ;-) Are you planning to upload these? Simon
Re: Git Packaging: Native source formats
On Thu, Aug 29, 2019 at 01:26:08PM -0700, Russ Allbery wrote: > While this is not an argument against *ever* using 3.0 (native) or some > equivalent when packaging upstream software, I have found it to help > relationships with upstream considerably in the past to represent the > package as their software as released plus a set of Debian patches. [...] > There's also a psychological value for some upstreams to representing all > changes to upstream as patches. If they're worried Debian is doing some > weird nonsense (from their perspective) to their package, they can look at > debian/rules for build flags and then at debian/patches and see exactly > what we've done, and feel assured that the rest of the package is exactly > as they released it. That's easy for them to do; downloading the full > package and then diffing against their tree is more awkward and can't be > done with some casual web browsing. Yes. In fact, not only upstreams, but in some cases downstreams and other people with a stake in how the package is put together. When I converted Debian's OpenSSH packaging to 3.0 (quilt), it was partly because a company using my packages was concerned about the difficulty about picking apart the changes in the Debian diff so that they could audit the logical changes that I was making; even though they were a large company with lots of highly-competent staff, just providing a public VCS that they could look through and pick apart the monolithic delta for themselves still turned out to be a prohibitively difficult barrier. Doing the work to provide cleanly-separated patches (which in many cases evolve over time) was a useful exercise for me and made things instantly much more comprehensible for them. This was a while ago, of course, but it was that experience that converted me to being convinced that providing broken-out patches in an easy-to-consume format is an invaluable service to our users. quilt itself is very much a lowest common denominator and I certainly think it's best treated as an export format from richer git history; but just exporting stuff from git without that annotation of logically-divided-up differences from upstream isn't an adequate substitute. > This is one of those cases where knowing your upstream is invaluable. A > lot of upstreams won't care in the slightest, some upstreams are Git > experts and know exactly how to get the data they care about, and others > are still using Subversion or even CVS and will find anything related to > Git opaque and frustrating. Some upstreams actively encourage downstream > modifications and will seek them out and incorporate them, some upstreams > are happily oblivious and only look at patches or PRs submitted to them, > and some upstreams will get actively upset at what they view as > unmotivated or incompatible changes and may require delicate handling > (part of which, in my experience, is effortless transparency so that they > don't feel like anything is being hidden from them). And also bear in mind the case where doing a git clone maybe takes quite a while over a slow network link, but going to salsa or sources.debian.org and having a quick look through debian/patches/ takes a fraction of the time and is all you need. -- Colin Watson [cjwat...@debian.org]
Re: Git Packaging: Native source formats
On Fri, Aug 30, 2019 at 12:29:45AM +0200, Thomas Goirand wrote:
> Now, you're talking about upstream using bzr or hg. These are the very
> few minority (and counting...). We may as well get rid of hg and bzr in
> Debian if it doesn't get fixed so it uses Python 3 only... (well, I
> guess someone will wake up and do the work, so this argument doesn't
> count...).
I appreciate that this is a side note from the point you're making, but
bzr was forked as brz ("breezy"); that fork supports Python 3 and is
compatible with existing bzr branches. I expect we will indeed need to
remove bzr at some point, but that shouldn't be a major problem for most
upstreams still using it.
--
Colin Watson [cjwat...@debian.org]
Re: Git Packaging: Native source formats
Theodore Y. Ts'o writes ("Re: Git Packaging: Native source formats"):
> On Thu, Aug 29, 2019 at 11:23:01AM +0100, Ian Jackson wrote:
> > I think dgit ought to be compatible with the idea of shipping
> > upstream's .asc's about, but maybe there are bugs. I don't ever do
> > this so I don't know if it works and I doubt there are tests for it.
> >
> > So, if you have a package where you want to use dgit push and you find
> > the upstream .asc is not being included, please file bug(s).
>
> The problem I have is that "dgit gbp" doesn't extract the upstream
> .asc. Not a big deal, I use /tmp/gbp for my build directory, and I
> manually checkout and populate it with the .asc file. But building
> from "dgit clone" won't generate same package as I do (which includes
> the .asc file for the orig.tar.gz file.)
When you say
building from "dgit clone" won't generate same package as I do
(which includes the .asc file for the orig.tar.gz file.
I'm not an expert on .asc handling but I think what you mean is
Steps to reproduce
0. mkdir bpd
1. dgit --build-products-dir=../bpd clone FOO
2. cd FOO
3. dch -i wombat
4. git commit -a -m changelog
3. dgit --build-products-dir=../bpd gbp-build
Expected behaviour
The generated .changes, and the generated .dsc contain
FOO_VSN.orig.tar.gz.asc, just as they are in uploads not
made with dgit.
Observed behaviour
The .asc is not mentioned in either the generated .dsc or the
generated .changes.
? That would be a bug. Can you file it and tell me the value of
`FOO' please :-) ?
(I'm not sure exactly what you mean by "doesn't extract the upstream
.asc" since I don't know what it would be "extracted" from. I tried
to see if I could reproduce for myself, but a quick check through this
subthread didn't turn up the package name.)
Ian.
PS hardcoding directories in /tmp in one's finger macros usually
amounts to embedding tmpfile race vulnerabilities in one's brain.
Even if one's own machine has per-user /tmp, one's wetware becomes
vulnerable when combined with a differently configured computer. IMO
one should make and use a ~/tmp or something.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: Git Packaging: Native source formats [and 1 more messages]
Philipp Kern writes ("Re: Git Packaging: Native source formats"):
> While this may be true on some level, it is also important to be able to
> build packages from checked-out source trees (say, git repositories)
> without an original source present.
Quite.
For example, if one wants to build binaries with sbuild, it is (right
now, anyway) necessary to build a source package because that's how
sbuild transports the source into the build environment. Right now I,
horrifyingly, have to advise [1] users that in some circumstances they
should run this command:
sbuild -c stretch -A --no-clean-source \
--dpkg-source-opts='-Zgzip -z1 --format=1.0 -sn'
[1] https://manpages.debian.org/testing/dgit/dgit-user.7.en.html#Using_sbuild
Simon Richter writes ("Re: Git Packaging: Native source formats"):
> On Thu, Aug 29, 2019 at 09:42:50PM +0200, Philipp Kern wrote:
> > Obviously I'm not bound to that format being "3.0 (native)" but some
> > "3.0 (dumb)" that just tars up the whole tree without caring about the
> > version scheme would then be nice to have as a replacement. ;-)
>
> Are you planning to upload these?
Obviously not to Debian. I don't think that invalidates the point.
Users are supposed to be able to modify and build software on their
own systems without any expectation that the result will go to Debian.
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Re: Git Packaging: Native source formats
Russ Allbery writes ("Re: Git Packaging: Native source formats"):
> [ discussion of benefits of maintaining the Debian delta as
> a linear series of broken-down changes ]
>
> There are obviously ways to represent this with a pure Git repository, but
> apart from using a patch system on top of 3.0 (native), at which point I
> don't understand why one wouldn't just use 3.0 (quilt), they require
> multiple branches and thus aren't available directly in the archive.
This is not true. There are at least two ways of doing this without
using a patch system: git-debrebase and git-dpm.
Both of these use only a single primary git branch which contains both
upstream history, and Debian changes to upstream files represented as
git commits.
I can't speak for git-dpm, but with git-debrebase and 1.0 native
source format, there would not any patch files.
> Extracting specific changes by comparing only two Git branches with a
> complex merge history is certainly possible to do with native Git tools,
> but I would classify it as an advanced Git skill. I think there are a lot
> of upstreams using Git for whom that operation would still be quite
> challenging.
This is also not that hard, in simple cases. There is a tool
git-debcherry which can do it automatically. I haven't used it but
AIUI if your Debian delta queue has few commits, and doesn't have
commits which involve merge conflicts with upstream merges (basically,
if each change is carried Debian only for a short time), it will
always produce the nice output you would hope for.
> This is one of those cases where knowing your upstream is invaluable.
I certainly agree with this. I don't think anyone is saying that
using (say) a merging git workflow with a native source package format
should be universal, or even the default.
Ian.
--
Ian JacksonThese opinions are my own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
Zdobądź nowych klientów. Generowanie leadów B2B.
Dzień dobry, zajmujemy się generowaniem wartościowych* leadów sprzedażowych* B2B. Naszym celem jest zwiększenie sprzedaży Twojej firmy. Jeśli jesteś zainteresowany otrzymaniem szczegółowych informacji, zwracamy się z uprzejmą prośbą o odpowiedź o treści * Tak *oraz pozostawienie* numeru kontaktowego*. Z wyrazami szacunku, Zespół Lead Generation.
Bug#938935: O: freedink-data -- adventure and role-playing game (assets)
Package: wnpp Severity: normal I orphaned the freedink-data package. See https://lists.debian.org/debian-devel-games/2019/08/msg00014.html The package description is: Dink Smallwood is an adventure/role-playing game, similar to Zelda, made by RTsoft. Besides twisted humour, it includes the actual game editor, allowing players to create hundreds of new adventures called Dink Modules or D-Mods for short. . This package contains the original game story, along with free sound and music replacements.
Bug#938934: O: freedink -- humorous top-down adventure and role-playing game
Package: wnpp Severity: normal I orphaned the freedink package. See https://lists.debian.org/debian-devel-games/2019/08/msg00014.html The package description is: Dink Smallwood is an adventure/role-playing game, similar to classic Zelda, made by RTsoft. Besides twisted humor, it includes the actual game editor, allowing players to create hundreds of new adventures called Dink Modules or D-Mods for short. . GNU FreeDink is a new and portable version of the game engine, which runs the original game as well as its D-Mods, with close compatibility, under multiple platforms. . This package is a metapackage to install the game, its data and a front-end to manage game options and D-Mods.
Bug#938937: O: freedink-dfarc -- frontend and .dmod installer for GNU FreeDink
Package: wnpp Severity: normal I orphaned the freedink-dfarc package. See https://lists.debian.org/debian-devel-games/2019/08/msg00014.html The package description is: Dink Smallwood is an adventure/role-playing game, similar to Zelda, made by RTsoft. Besides twisted humor, it includes the actual game editor, allowing players to create hundreds of new adventures called Dink Modules or D-Mods for short. . DFArc2 makes it easy to play and manage the Dink Smallwood game and its numerous D-Mods. Dink Smallwood is an adventure/role-playing game, similar to Zelda, made by RTsoft. Besides twisted humor, it includes the actual game editor, allowing players to create hundreds of new adventures called Dink Modules or D-Mods for short. . DFArc2 makes it easy to play and manage the Dink Smallwood game and its numerous D-Mods.
Bug#938950: ITP: python-pgbouncer -- Fixture to bring up temporary pgbouncer instances
Package: wnpp Severity: wishlist Owner: Colin Watson * Package name: python-pgbouncer Version : 0.0.9 Upstream Author : Canonical Ltd. * URL : https://launchpad.net/python-pgbouncer * License : AGPL-3+ Programming Lang: Python Description : Fixture to bring up temporary pgbouncer instances A fixture (https://launchpad.net/python-fixtures) for creating a temporary instance of pgbouncer, a lightweight connection pooler for PostgreSQL. It is intended for use during development and testing. I'm packaging this because it's a build-dependency of newer versions of storm; that package was recently removed, but I plan to reintroduce it once I finish getting Python 3 support landed upstream. I plan to maintain this within the Debian Python Modules Team. -- Colin Watson [cjwat...@debian.org]
Re: Git Packaging: Native source formats
Ian Jackson writes:
> Russ Allbery writes ("Re: Git Packaging: Native source formats"):
>> [ discussion of benefits of maintaining the Debian delta as
>> a linear series of broken-down changes ]
>>
>> There are obviously ways to represent this with a pure Git repository, but
>> apart from using a patch system on top of 3.0 (native), at which point I
>> don't understand why one wouldn't just use 3.0 (quilt), they require
>> multiple branches and thus aren't available directly in the archive.
> This is not true. There are at least two ways of doing this without
> using a patch system: git-debrebase and git-dpm.
> Both of these use only a single primary git branch which contains both
> upstream history, and Debian changes to upstream files represented as
> git commits.
It's a fair point that I didn't account for a rebase workflow in my
analysis, and that's definitely an alternative.
However, while analyzing a rebased branch isn't as hard for other people
as a branch with a complex merge history, it does mean that upstream has
to find a way to extract patches to their code from a branch that also has
packaging-only changes and their upstream changes, and this is non-trivial
for a lot of people. It's certainly way harder than just pointing them at
a directory full of patches.
> This is also not that hard, in simple cases. There is a tool
> git-debcherry which can do it automatically. I haven't used it but AIUI
> if your Debian delta queue has few commits, and doesn't have commits
> which involve merge conflicts with upstream merges (basically, if each
> change is carried Debian only for a short time), it will always produce
> the nice output you would hope for.
This lets you generate the patches for people on demand, but they aren't
just sitting out there for anyone to look at whenever they want.
I suppose it could be provided as an automated service that publishes the
results, but that would be a piece of infrastructure someone has to run.
Anyway, this is probably only applicable to a minority of upstreams and
packages, and more upstreams these days would just like all patches as
PRs.
--
Russ Allbery (r...@debian.org)
Re: Git Packaging: Native source formats
On Friday, August 30, 2019 12:05:45 PM EDT Russ Allbery wrote:
> Ian Jackson writes:
> > Russ Allbery writes ("Re: Git Packaging: Native source formats"):
> >> [ discussion of benefits of maintaining the Debian delta as
> >>
> >> a linear series of broken-down changes ]
> >>
> >> There are obviously ways to represent this with a pure Git repository,
> >> but
> >> apart from using a patch system on top of 3.0 (native), at which point I
> >> don't understand why one wouldn't just use 3.0 (quilt), they require
> >> multiple branches and thus aren't available directly in the archive.
> >
> > This is not true. There are at least two ways of doing this without
> > using a patch system: git-debrebase and git-dpm.
> >
> > Both of these use only a single primary git branch which contains both
> > upstream history, and Debian changes to upstream files represented as
> > git commits.
>
> It's a fair point that I didn't account for a rebase workflow in my
> analysis, and that's definitely an alternative.
>
> However, while analyzing a rebased branch isn't as hard for other people
> as a branch with a complex merge history, it does mean that upstream has
> to find a way to extract patches to their code from a branch that also has
> packaging-only changes and their upstream changes, and this is non-trivial
> for a lot of people. It's certainly way harder than just pointing them at
> a directory full of patches.
>
> > This is also not that hard, in simple cases. There is a tool
> > git-debcherry which can do it automatically. I haven't used it but AIUI
> > if your Debian delta queue has few commits, and doesn't have commits
> > which involve merge conflicts with upstream merges (basically, if each
> > change is carried Debian only for a short time), it will always produce
> > the nice output you would hope for.
>
> This lets you generate the patches for people on demand, but they aren't
> just sitting out there for anyone to look at whenever they want.
>
> I suppose it could be provided as an automated service that publishes the
> results, but that would be a piece of infrastructure someone has to run.
>
> Anyway, this is probably only applicable to a minority of upstreams and
> packages, and more upstreams these days would just like all patches as
> PRs.
It's not particularly rare for me to poke through other distros package
patches when I'm trying to figure out how to solve a problem. I suspect I'm
not the only one. I think having explicit patches available is a really good
thing for cross-distro collaboration (in addition to upstream collaboration).
Scott K
Re: Git Packaging: Native source formats
On Fri, 30 Aug 2019 at 09:05:45 -0700, Russ Allbery wrote: > [git-debcherry] lets you generate the patches for people on demand, but > they aren't just sitting out there for anyone to look at whenever they want. I think that's really important from the transparency point of view that you touched on in a previous mail. If the upstream has an opinion about a change (either wanting to apply it to their codebase, or thinking it's wrong and wanting us to stop applying it) and needs the context of why we made that change, then I think it's important that we make it as straightforward as possible for them to see what we've done with their code. IMO part of a downstream maintainer's job is to present the changes in a way that an upstream can, if not necessarily agree with, then at least reason about: * which of their later changes we backported * which other changes we made, and why * how we're compiling it (build-time options, etc.) This is why I try to encourage my colleagues and co-maintainers to use DEP-3 on their patches, and reorder debian/patches/series with backports from upstream before upstreamable patches, which are before Debian-specific changes. The changes we *used to* make are somewhat secondary: it's important for downstream maintainers to be able to dig into the history, and it's occasionally interesting for upstreams to be able to dig into that history too, but the version(s) we're currently recommending to our users are the most important. > Anyway, this is probably only applicable to a minority of upstreams and > packages, and more upstreams these days would just like all patches as > PRs. For all the upstreamable changes yes, but sometimes we have to make Debian-specific changes that aren't upstreamable, and we should do that carefully, avoiding the perception that we're randomly forking software that the upstream maintainer feels strongly about. I'm quite aware of this as an upstream whose downstreams sometimes make changes that I wouldn't have accepted, or even changes that I have already rejected. (It was a particularly surreal experience when I was maintaining dbus in Debian with a patch included that I had rejected upstream, although that's fixed in buster.) smcv
Re: Summary: Git Packaging: Native source formats
> "Ian" == Ian Jackson writes: Ian> I certainly agree with this. I don't think anyone is saying Ian> that using (say) a merging git workflow with a native source Ian> package format should be universal, or even the default. Correct. My take away from this discussion though is that using a native package format even when there are upstream sources is an appropriate tool for maintainers to use sometimes. I think that is a significant change over our feelings years ago. Back in the day I think there was approach a project consensus that using native format packages with upstreams that were native to Debian was basically always wrong. My reading of this discussion is that it's a tool that maintainers should have and it's sometimes reasonable to use that tool. People have done a great job of bringing up athings to think about for maintainers considering using that tool. I've certainly found the discussion educational. It's changed my thinking about when I would choose to use native format packages in the future. So, my high level summary is that it's an option maintainers have, but there are many things maintainers should consider when electing to use that option. I think it would be great to capture the items people brought up here in a wiki page for people to look at when they consider using native format packages. I'd be really greatful if someone else made a first cut at turning this discussion into such a page, but if it doesn't happen I'll work to find time. Thanks for all your great things to consider! --Sam signature.asc Description: PGP signature
Bug#938978: ITP: ap51-flash -- firmware flasher for ethernet connected routers and access points
Package: wnpp Severity: wishlist Owner: Sven Eckelmann X-Debbugs-CC: debian-devel@lists.debian.org * Package name: ap51-flash Version : 2019.0 Upstream Author : Marek Lindner * URL : https://github.com/ap51-flash/ap51-flash * License : GPL-3+ Programming Lang: C Description : firmware flasher for ethernet connected routers and access points ap51-flash is a tool to simplify the automatic firmware deployment for a multitude of home routers and wireless access points. . ap51-flash can identify target device(s), select the correct firmware image and perform the required communication to carry out the installation procedure. It works without the need for a local TFTP server or manual, target device specific network configuration. Note: The implementation currently only supports Architecture linux-any. But it should in theory be possible to port it to kfreebsd-any. signature.asc Description: This is a digitally signed message part.
Re: Git Packaging: Native source formats
On Fri, 2019-08-30 at 12:20 -0400, Scott Kitterman wrote: > On Friday, August 30, 2019 12:05:45 PM EDT Russ Allbery wrote: > > This lets you generate the patches for people on demand, but they aren't > > just sitting out there for anyone to look at whenever they want. > > > > I suppose it could be provided as an automated service that publishes the > > results, but that would be a piece of infrastructure someone has to run. > > > > Anyway, this is probably only applicable to a minority of upstreams and > > packages, and more upstreams these days would just like all patches as > > PRs. > > It's not particularly rare for me to poke through other distros package > patches when I'm trying to figure out how to solve a problem. I suspect I'm > not the only one. I think having explicit patches available is a really good > thing for cross-distro collaboration (in addition to upstream collaboration). > > Scott K I'll second that looking through patches a distro has supplied can be very useful. It's very convenient to have explicit patches you can just check quickly to see if it could solve an issue or not.
Re: Git Packaging: Native source formats
On Thu, 29 Aug 2019 22:03:18 -0400, Theodore Y. Ts'o wrote: > The problem I have is that "dgit gbp" doesn't extract the upstream > .asc. This sounds like #872864 in git-buildpackage. Cheers, gregor -- .''`. https://info.comodo.priv.at -- Debian Developer https://www.debian.org : :' : OpenPGP fingerprint D1E1 316E 93A7 60A8 104D 85FA BB3A 6801 8649 AA06 `. `' Member VIBE!AT & SPI Inc. -- Supporter Free Software Foundation Europe `- NP: Rebekka Bakken & Wolfgang Muthspiel: Angela signature.asc Description: Digital Signature
Re: Git Packaging: Native source formats
On Fri, Aug 30, 2019 at 09:05:45AM -0700, Russ Allbery wrote: > Ian Jackson writes: > > This is also not that hard, in simple cases. There is a tool > > git-debcherry which can do it automatically. I haven't used it but AIUI > > if your Debian delta queue has few commits, and doesn't have commits > > which involve merge conflicts with upstream merges (basically, if each > > change is carried Debian only for a short time), it will always produce > > the nice output you would hope for. > > This lets you generate the patches for people on demand, but they aren't > just sitting out there for anyone to look at whenever they want. > > I suppose it could be provided as an automated service that publishes the > results, but that would be a piece of infrastructure someone has to run. Since git-debcherry is used to export the patches when creating the source package, such a service already exists -- https://sources.debian.org/ Now, git-debcherry doesn't always make the most human consumable patch series. For example, the neovim 0.3.4-3[0] upload in which I cherry-picked a large number of patches for a security fix. Rather than having distinct patches for most of those commits, they were mostly munged into a single "debcherry fixup" patch. [0]: https://sources.debian.org/src/neovim/0.3.4-3/debian/patches/ Cheers, -- James GPG Key: 4096R/91BF BF4D 6956 BD5D F7B7 2D23 DFE6 91AE 331B A3DB
Bug#939011: ITP: ocaml-stdcompat -- compatibility module for OCaml standard library
Package: wnpp Severity: wishlist Owner: Stéphane Glondu * Package name: ocaml-stdcompat Version : 10 Upstream Author : Thierry Martinez * URL : https://github.com/thierry-martinez/stdcompat * License : BSD Programming Lang: OCaml Description : compatibility module for OCaml standard library Stdcompat is a compatibility layer allowing programs to use some recent additions to the OCaml standard library while preserving the ability to be compiled on former versions of OCaml. . The module Stdcompat provides some definitions for values and types introduced in recent versions of the standard library. These definitions are just aliases to the matching definition of the standard library if the latter is recent enough. Otherwise, the module Stdcompat provides an alternative implementation. This package is a dependency of pyml, which is a replacement for pycaml and needed by coccinelle. It will be maintained in ocaml-team.
