FTP-Master Review Process
Hello fellow masochists, ... err, I mean Debian Developers, There has been a lot of talk lately about the FTP-Masters team and how some issues should be fixed. Back in December, I alluded to a proposal that I was working on. I hoped to get a little further into writing source prior to this point, but life happens and now seems to be a much more appropriate time than later. In my "proposal" [1], I outline the concerns that have been noted (from mailing lists, IRC, and private discussions) and then provide a few potential solutions. Although some source code exists, this is a discussion for later. I want to take a very systematic and structured approach to this. To outline... 1. Identify a problem 2. Evaluate the cause 3. Consider solutions 4. Evaluate the solution - Does it solve the problem? - Does it introduce new problems? - What will it take to implement? - Would any developer be willing to work on it? - Will it be maintainable in the long-term? - etc. 5. Build some prototypes 6. Re-evaluate the solution (copy #4) 7. Finish design requirements 8. Start building 9. Lotsa testing 10. Start discussing implementation Note... I am using the term proposal because very little of what I put forth is in any way a design document. It's just many thoughts tossed at a page that now require critical review. (critical -- be picky, not rude) Once the discussion is over (or degrades), I will begin writing a design document. After complete, I will ask for additional comment and interested developers. Side note- An important road block to be aware of is that implementing almost any of the proposed solutions will require significant (time & effort) changes to the main archive server. It will also likely require some legal counsel. I'll omit the gritty details since this is mostly just FYI, but it's also worth considering team size and availability (see: $current_issue) when it comes to implementation. [1] https://salsa.debian.org/mtecknology/ftpmaster-proposal -- Michael Lustfield
Re: FTP-Master Review Process
Hi fellow devs,
Just appending some of my side notes:
Michael's ftpmaster-proposal shares some common motivations with my
previous mail ("Intermediate representation ... license review"), and we
basically reached in an agreement on the problems existing in our
FTP-master working (NEW) process. The difference between our proposals
is that mine focuses on a small, specific, and dedicated problem, while
Michael's concerns problems in a larger-scale perspective -- a superset
of mine. Or say bottom-up v.s. top-down.
I'm still asynchronously working on my tool for helping NEW queue reviewing
process, and it may solve a portion of the issues pointed out by the proposal.
But note that my action on this will be slow due to my $college stuff.
When have enough time, I will document the core thoughts and
specifications of my work-in-progress project and post them here.
Simple demo code will come after that.
On Sun, Mar 29, 2020 at 05:37:01AM -0500, Michael Lustfield wrote:
> Hello fellow masochists,
> ... err, I mean Debian Developers,
>
> There has been a lot of talk lately about the FTP-Masters team and how some
> issues should be fixed. Back in December, I alluded to a proposal that I was
> working on. I hoped to get a little further into writing source prior to this
> point, but life happens and now seems to be a much more appropriate time than
> later.
>
>
> In my "proposal" [1], I outline the concerns that have been noted (from
> mailing
> lists, IRC, and private discussions) and then provide a few potential
> solutions.
>
> Although some source code exists, this is a discussion for later. I want to
> take
> a very systematic and structured approach to this.
>
> To outline...
> 1. Identify a problem
> 2. Evaluate the cause
> 3. Consider solutions
> 4. Evaluate the solution
> - Does it solve the problem?
> - Does it introduce new problems?
> - What will it take to implement?
> - Would any developer be willing to work on it?
> - Will it be maintainable in the long-term?
> - etc.
> 5. Build some prototypes
> 6. Re-evaluate the solution (copy #4)
> 7. Finish design requirements
> 8. Start building
> 9. Lotsa testing
> 10. Start discussing implementation
>
> Note... I am using the term proposal because very little of what I put forth
> is
> in any way a design document. It's just many thoughts tossed at a page that
> now
> require critical review. (critical -- be picky, not rude)
>
> Once the discussion is over (or degrades), I will begin writing a design
> document. After complete, I will ask for additional comment and interested
> developers.
>
>
> Side note-
> An important road block to be aware of is that implementing almost any of the
> proposed solutions will require significant (time & effort) changes to the
> main
> archive server. It will also likely require some legal counsel. I'll omit the
> gritty details since this is mostly just FYI, but it's also worth considering
> team size and availability (see: $current_issue) when it comes to
> implementation.
>
>
> [1] https://salsa.debian.org/mtecknology/ftpmaster-proposal
>
> --
> Michael Lustfield
Bug#955297: ITP: python3-clickhouse-driver -- Python driver with native interface for ClickHouse
Package: wnpp Severity: wishlist Owner: Federico Ceratto * Package name: python3-clickhouse-driver Version : 0.1.3-1 Upstream Author : Marilyn System, Konstantin Lebedev * URL : https://github.com/mymarilyn/clickhouse-driver * License : MIT Programming Lang: Python, C Description : Python driver with native interface for ClickHouse Python driver for ClickHouse with native interface support. It supports external data for query processing, query settings, compression, TLS, native Python types, query progress information, result streaming and accessing server logs. Maintained at https://salsa.debian.org/debian/python-clickhouse-driver
Re: RFC: Standardizing source package artifacts build paths
On Mon, 2020-03-09 at 09:23:46 -0400, Sam Hartman wrote: > I'm concerned about a leading . at least for: > > * the debian/tmp replacement > * the replacement for the package install directories under debian. > > I think that maintaining those directories such that ls shows them will > be more friendly for new maintainers. > So I'd prefer something like debian/build rather than debian/.build for > at least those directories. > > I don't care as much about substvars, debhelper intermediates, debhelper > logs and the like. As mentioned on the proposal the use of a hidden directory is to reduce clutter (getting the packaged bits alongside the generated pathnames when listing the debian/ directory contents has always seemed extremely distracting, which also makes it too easy for them to end up on a VCS, f.ex.) and to avoid collisions with existing packaging (due to temporary directory usage or due to staged package directories). There's also precedent with hidden directories with debhelper, even for the staged directories for dbgsyms packages. Making internal stuff non-hidden would make it more distracting, and if this would mean having at least two hierarchies (one hidden and one non-hidden), that can be even more confusing and is more work to cleanup and ignore, which goes in the opposite direction of simplifying things IMO. While it's true that we might need to use such pathnames in debian/rules or debhelper fragment files (which some might consider ugly), IMO that has always felt like a sign that there's something missing in our packaging helpers/tools. To me this calls f.ex. for making dpkg-dev and debhelper support build flavors (for multiple builds) or for debhelper to automatically look in the relevant build directories when looking up for files, so in addition to look for them in the installation stage directory and the source package root directory, it would look there too, removing the major needs for having to list the full pathnames explicitly. Thanks, Guillem
Re: RFC: Standardizing a new Protected field
Hi! On Mon, 2020-03-09 at 10:23:36 +0100, Guillem Jover wrote: > Summary > --- > > The goal of the following proposal is to standardize a field to split > part of the Essential packages, and add support for it in the package > management stack. There is currently an Important field, that has the > correct semantics but has a very confusing name and is only supported > by apt anyway, so this new field would phase out that one. Given there's been no concerns brought up, we'll be going ahead with these changes in dpkg (for 1.20.1) and apt. I'll add a spec to dpkg and file a bug to debian-policy to add it there too. But if there are still unvoiced concerns, please let us know! There's always time to put this on hold or revert, etc. :) Thanks, Guillem
Bug#955306: ITP: libperl-minimumversion-fast-perl -- Find a minimum required version of perl for Perl code
Package: wnpp Owner: Dominique Dumont Severity: wishlist X-Debbugs-CC: debian-devel@lists.debian.org, debian-p...@lists.debian.org * Package name: libperl-minimumversion-fast-perl Version : 0.18 Upstream Author : tokuhirom * URL : https://metacpan.org/release/Perl-MinimumVersion-Fast * License : Artistic or GPL-1+ Programming Lang: Perl Description : Find a minimum required version of perl for Perl code Perl::MinimumVersion::Fast module takes Perl source code and calculates the minimum version of perl required to be able to run it. Because it is based on goccy's Compiler::Lexer, it can do this without having to actually load the code. Perl::MinimumVersion::Fast is an alternative fast & lightweight implementation of Perl::MinimumVersion. The package will be maintained under the umbrella of the Debian Perl Group. This module is a dependency of libdist-zilla-plugin-minimumperlfast-perl -- Generated with the help of dpt-gen-itp(1) from pkg-perl-tools.
Re: trimming changelogs
Hi! On Fri, 2020-03-20 at 00:50:29 +0100, Adam Borowski wrote: > In the rush for cutting away small bits of minbase, it looks like we forgot > a big pile of junk: /usr/share/doc/ As has been mentioned on the thread, this is IMO a non-issue, as these pathnames can be trimmed automatically with dpkg path-excludes. > On the other hand, changelogs are valuable. Unlike some folks on IRC > I wouldn't want to tightly trim all packages. Unlike minbase or > prio:important, your average 5GB install doesn't care about a few megs > here and there. Thus: do we want to trim manually or globally? > > A global trim would require a lot less work. A manual trim would allow > managing packages: dpkg is everywhere, dpkg-dev is not. libsystemd0 is > essential, systemd doesn't belong in containers. gcc-9-base is included > on tiny installs, gcc-9 on dev boxes and buildds with plenty of space. > Plus, manual trimming would also allow axing old upstream cruft. TBH I find trimmed changelogs rather annoying. We already got problems with changelogs that omit information (say a salad of Closes for bugs fixed in a new upstream version), or non-descriptive changes, or outright omitted changes, etc. To me trimming them and making it a habit of having to download them from the network, triggers the question of why we'd even ship them? (Just to be clear I'd find that to be a worse decision. :) Doing that trimming globally would also mean that this applies even to packages that are for local or derived use where something like «apt changelog» will in most cases not work. A proposal I've been floating around from time to time has been to instead make the changelog and copyright files deb/dpkg metadata (which they really are anyway IMO). This would allow the following properties: - The interfaces would become something like dpkg --show-copyright or dpkg --show-changelog, with a pager f.ex., which is more obvious for new people to discover than having to look for a file in the filesystem. - Extracting them for apt-listchanges and similar would be way way quicker (needing to extract the .deb control member instead of the data control member). - The contents could get compressed in whatever format is most appropriate (xz f.ex.). - The contents could get automatically de-duplicated on demand. - The contents could be configured to be dropped as well. - The changelog file would stop being a problem for multiarch binNMUs. - This would get rid of most of the issues with symlinked /usr/share/doc dirs(?), and the contents would always match the package version instead of now when sometimes we end up with mismatched content and version if the dependencies are not strict enough. Thanks, Guillem
Re: RFC: Standardizing source package artifacts build paths
On Mar 29, Guillem Jover wrote: > While it's true that we might need to use such pathnames in debian/rules > or debhelper fragment files (which some might consider ugly), IMO that > has always felt like a sign that there's something missing in our > packaging helpers/tools. If you believe this then it means that you have been maintaining too modern or too much simple packages. https://salsa.debian.org/md/inn/-/blob/master/debian/rules https://salsa.debian.org/md/inn2/-/blob/master/debian/rules https://salsa.debian.org/md/kmod/-/blob/master/debian/rules https://salsa.debian.org/md/binkd/-/blob/master/debian/rules https://salsa.debian.org/md/libxcrypt/-/blob/master/debian/rules -- ciao, Marco signature.asc Description: PGP signature
Re: RFC: Standardizing source package artifacts build paths
On Sun, 2020-03-29 at 22:48:04 +0200, Marco d'Itri wrote: > On Mar 29, Guillem Jover wrote: > > While it's true that we might need to use such pathnames in debian/rules > > or debhelper fragment files (which some might consider ugly), IMO that > > has always felt like a sign that there's something missing in our > > packaging helpers/tools. > If you believe this then it means that you have been maintaining too > modern or too much simple packages. Aww thanks, you are, as always, way too kind, and you probably even listed too many reasons there. > https://salsa.debian.org/md/inn/-/blob/master/debian/rules > https://salsa.debian.org/md/inn2/-/blob/master/debian/rules > https://salsa.debian.org/md/kmod/-/blob/master/debian/rules > https://salsa.debian.org/md/binkd/-/blob/master/debian/rules > https://salsa.debian.org/md/libxcrypt/-/blob/master/debian/rules Most of these show some of the things I had in mind. Which have not been supported (easily) in the past or yet by our packaging toolchain, like: - declarative file metadata, - renaming files, - flavor builds (like udeb vs no-udeb, or w/ features disabled in configure, etc.), Some other usages there though, look like things that can really be simplified in the packaging, by say, just: - using dh_install, dh_link and friends, - being explicit in things to install in the .install fragments instead of just adding whole hierarchies, - using a temporary destdir (like debhelper does by default) and only installing the desired pathnames to the final installation directory. Other things there can be simplified by improving the upstream build system, also by parametrizing things, and submitting these upstream or just carrying as local patches. So yes, thanks, to me these examples seem to still confirm that the explicit usage of the staged directories is an indication of things that need fixing or improving in either the packaging or the toolchain. And, of course there are always going to be remaining sticking points not covered by features I or others have in mind, but IMO their presence will still mean there's something to improve somewhere. Regards, Guillem
Overinterpretation of DFSG? QR code for receiving donation is non-free???
Hi fellow devs, I think sometimes the DFSG has been over-interpreted. Here I'm talking about the recent REJECTion of src:smartdns from our NEW queue, where QR code pictures used for donation have been deemed DFSG non-free [1]. I'm not satisfied with the explanation, and I think there is over-interpretation on DFSG. I poked ftp-master about this problem: spwhitton: I'm quite confused about REJECTION of src:smartdns. Why can the QR code pictures for software author to receive donations be DFSG-nonfree? And I got the following explanations: lumin: IIRC that was not the only reason for REJECT. Otherwise I would have PRODded. lumin: An image of a QR code wouldn't be the preferred form of modification. They are usually generated from something. If the file it was generated from isn't present and the tool to generate it isn't in Debian, then it can't be shipped. Requiring preferred form of modification is one area where Debian is often stricter than licenses due to DFSG. The pictures we're talking about are: * https://salsa.debian.org/debian/smartdns/-/blob/master/doc/alipay_donate.jpg * https://salsa.debian.org/debian/smartdns/-/blob/master/doc/wechat_donate.jpg "alipay" and "wechat" are the top-2 domination payment platforms in Chinese market. And the two QR code pictures are generated from the corresponding APPs by the upstream author. The whole software project is licensed under GPL-3 and the QR codes are used for receiving donations. Why are they non-free? Treating this files as non-free could lead to further problems. 1. If I stripped the donation codes from the source. I believe such behaviour is **unethical**. 2. If I decoded the QR code and replaced them with the underlying URLs. There is no Chinese user who pay through URL instead of QR code. 3. If I stripped the donation codes but re-generated them during the package build process. "Oh damn, this QR code does not look like the original one and the hashsum mismatches. Has the Debian developer forged the QR code to be evil?" I mean there will be doubt if the distributed QR code is not byte-to-byte equivalent to the one distributed by upstream author. Is a QR code for donation really DFSG non-free? Is DFSG over-interpreted in this case? How should package maintainers deal with QR codes ethically? [1] The package has been REJECT'ed for two reasons: 1. "doc/*_donate.jpg are probably not DFSG-free" 2. Missing copyright information for "package/luci-compat/tool/po2lmo/src/*" There is no problem with the second point. This mail only talks about the first point. signature.asc Description: PGP signature
Re: Overinterpretation of DFSG? QR code for receiving donation is non-free???
Hello Mo, Le lundi, 30 mars 2020, 07.54:23 h CEST Mo Zhou a écrit : > I think sometimes the DFSG has been over-interpreted. Here I'm talking about > the recent REJECTion of src:smartdns from our NEW queue, where QR code > pictures used for donation have been deemed DFSG non-free [1]. I'm not > satisfied with the explanation, and I think there is over-interpretation on > DFSG. > > I poked ftp-master about this problem: > >spwhitton: I'm quite confused about REJECTION of src:smartdns. Why > can the QR code pictures for software author to receive donations be > DFSG-nonfree? > > And I got the following explanations: > >lumin: IIRC that was not the only reason for REJECT. > Otherwise I would have PRODded. > >lumin: An image of a QR code wouldn't be the preferred form of > modification. They are usually generated from something. If the file it > was generated from isn't present and the tool to generate it isn't in > Debian, then it can't be shipped. Requiring preferred form of modification > is one area where Debian is often stricter than licenses due to DFSG. > > The pictures we're talking about are (…) > > Why are they non-free? They are non-free, because they cannot be rebuilt from their preferred form of modification. > Treating this files as non-free could lead to further problems. > > 1. If I stripped the donation codes from the source. > I believe such behaviour is **unethical**. It's allowed by GPL-3 licensing. Actually, we (Debian) require that this must be possible. But you can't be coerced into doing this (you can always opt to "not package for Debian"). > 2. If I decoded the QR code and replaced them with the underlying URLs. > There is no Chinese user who pay through URL instead of QR code. > > 3. If I stripped the donation codes but re-generated them during the > package build process. > "Oh damn, this QR code does not look like the original one and the > hashsum mismatches. Has the Debian developer forged the QR code to be > evil?" I mean there will be doubt if the distributed QR code is not > byte-to-byte equivalent to the one distributed by upstream author. We're _building_ source code towards binary artifacts all the time. Doing this (and being trusted to be doing it correctly) is one of the defining characteristics of being a "shipping-binaries" Linux distribution. The whole point of this exercise is that our build processes are auditable, and that eventual forgeries can be found, reported, and fixed. If you don't consider the result of your builds trustworthy, "we have a problem". > Is a QR code for donation really DFSG non-free? QR codes are artifacts in binary form, not in their preferred form of modification. By function, QR codes are vehicles of binary information, and can be easily reconstructed from said binary information without loss (of information). Frankly, simple lines like the following in debian/rules would do it: echo "http://donation-url.example.com/?vendor-id"; | qrencode -o qr.png > Is DFSG over-interpreted in this case? IANAFM, but I don't think so. > How should package maintainers deal with QR codes ethically? Asking package maintainers to rebuild functionally-equivalent QR-codes during the build-process seems entirely reasonable to me. -- OdyX signature.asc Description: This is a digitally signed message part.
Re: Overinterpretation of DFSG? QR code for receiving donation is non-free???
On Mon, Mar 30, 2020 at 5:54 AM Mo Zhou wrote: > * > https://salsa.debian.org/debian/smartdns/-/blob/master/doc/alipay_donate.jpg > * > https://salsa.debian.org/debian/smartdns/-/blob/master/doc/wechat_donate.jpg In addition to the QR codes, these both contain images of what look like television or game characters, which I assume are non-free in their own right. If it weren't for the TV/game characters it would be pretty trivial to build these images from scratch, just pass this data to your favourite QR code generator: $ zbarimg alipay_donate.jpg QR-Code:HTTPS://QR.ALIPAY.COM/FKX04578CLNPKSW9W28R97 scanned 1 barcode symbols from 1 images in 0.02 seconds $ zbarimg wechat_donate.jpg QR-Code:wxp://f2f0MsJfRxpVxhbSnb7gkVkiPw7EhfUQ8P_I scanned 1 barcode symbols from 1 images in 0.02 seconds -- bye, pabs https://wiki.debian.org/PaulWise
