Bug#1088206: ITP: glow -- Render Markdown on the CLI
Package: wnpp Severity: wishlist Owner: Otto Kekäläinen * Package name: glow Version : 1.2.0-1 Upstream Author : Charm * URL : https://github.com/charmbracelet/glow * License : Expat Programming Lang: Go Description : Render Markdown on the CLI Glow is a terminal based Markdown reader designed from the ground up to bring out the beauty—and power—of the CLI. Use it to discover markdown files, read documentation directly on the command line. Glow will find local Markdown files in subdirectories or a local git repository. See also ITP for golang-github-muesli-mango-cobra and golang-github-muesli-mango-pflag.
Bug#1088208: ITP: golang-github-muesli-mango-pflag -- pflag adapter for mango
Package: wnpp Severity: wishlist Owner: Otto Kekäläinen * Package name: golang-github-muesli-mango-pflag Version : 0.1.0-1 Upstream Author : Christian Muehlhaeuser * URL : https://github.com/muesli/mango-pflag * License : Expat Programming Lang: Go Description : pflag adapter for mango mango-pflag is a pflag adapter for mango (https://github.com/muesli/mango) I intend to package Glow (https://github.com/charmbracelet/glow) for Debian, and this is an indirect dependency of it.
Bug#1088207: ITP: golang-github-muesli-mango-cobra -- cobra adapter for mango
Package: wnpp Severity: wishlist Owner: Otto Kekäläinen * Package name: golang-github-muesli-mango-cobra Version : 1.2.0-1 Upstream Author : Christian Muehlhaeuser * URL : https://github.com/muesli/mango-cobra * License : Expat Programming Lang: Go Description : cobra adapter for mango mango-cobra is a cobra adapter for mango (https://github.com/muesli/mango) I intend to package Glow (https://github.com/charmbracelet/glow) for Debian, and this is an indirect dependency of it.
Builds that pass locally but fail on sbuild? (Re: Reviving schroot as used by sbuild)
Hi Simon! > There are lots of options for doing this, some of which are listed in > <https://wiki.debian.org/SystemBuildTools#Package_build_tools>. > > All of these have the same problem as cowbuilder, pbuilder, and any > other solution that is not sbuild + schroot: it isn't (currently) what > the production Debian buildds use, therefore it is entirely possible > (perhaps even likely, depending on what packages you maintain) that your > package will build successfully and pass tests in your own local builder, > but then fail to build or fail tests on the buildds as a result of some > quirk of how schroot sets up its chroots, which is a worse-than-RC bug > making the package unreleasable. Could you point me to some Debian Bug # or otherwise share examples of cases when a build succeeded locally but failed on official Debian builders due to something that is specific for sbuild/schroot? I have never run in such a situation despite doing Debian packaging for 10 years with fairly complex C++ software targeting all archs Debian supports. Also as a member of the Salsa-CI team I don't recall ever seeing a bug report about something built on Salsa in a container successfully but failed to build on actual buildd. I am not dismissive of your claim - as a very senior DD you surely have those experiences - I am just curious to learn what those cases might have been. I could imagine that buildd builds fail if they the source was prepared in a non-hermetic environment that ran as root, or had network access, or if build environment was unclean and debian/control was missing some dependencies, but that is elementary hermetic build environment properties and not inherently something that *only* sbuild/schroot does. Related, you might want to take a peek at the source code of https://salsa.debian.org/otto/debcraft how it supports both Podman and Docker, and how it generates the 'root.tar.gz' equivalent container automatically based on debian/control and debian/changelog contents, and then runs the actual build as a regular non-root user in a container that has no network access. If I learn about other requirements for a hermetic build environment I would be happy to incorporate it. - Otto
Re: Vendoring an unmaintained library?
Enriching original email: This is about whether packages transmission-gtk should use embedded copy of libb64 or depend on the outdated Debian package libb64. Upstream for libb64 seems dead and transmission devs have improved their embedded/vendored copy of libb64. Direct link: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1073005 First question: Are the libb64 improvements by Transmission devs generic, could they be copied as patches to the Debian libb64 package for everyone's benefit?
Re: DD's, Debian Mentors needs you!
Hi! > https://mentors.debian.net/package/uriparser/ > https://mentors.debian.net/package/mailgraph/ This is the same maintainer, I can take this on. > https://mentors.debian.net/package/selint/ I have worked with this maintainer before, I can help him now too. After this Hexwalk still needs a mentor.
Re: GR: tag2upload
Hi! On Thu, 4 Jul 2024 at 13:32, Debian Project Secretary - Kurt Roeckx wrote: > A general resolution about tag2upload has been started. Details are at: > https://www.debian.org/vote/2024/vote_002 This GR was published without much context. I at least was much more enlightened after reading the Linux Weekly News article on the topic: https://lwn.net/Articles/978324/, others might find it useful as well.
Request for feedback on draft: DEP-18: Enable true open collaboration on all Debian packages
Hi all, I have drafted a new DEP at https://salsa.debian.org/dep-team/deps/-/merge_requests/8 titled "DEP-18: Enable true open collaboration on all Debian packages". Direct link to raw text: https://salsa.debian.org/dep-team/deps/-/raw/798bfa5a1e1609afd4e48ee20aff80a82bcd4a2f/web/deps/dep18.mdwn This would have been a great topic to discuss in person at DebConf, but unfortunately I can't attend this year, so I'll just kick this off on the mailing list. This is continuation to the 'single maintainership' discussions earlier this year. I also think that more unified and open collaboration processes could help decrease maintainer burnout, lower barrier for existing and new maintainers to help with multiple packages, and overall perhaps also improve quality of uploads by having more attention on the source code prior to upload. If you think this proposal makes sense, please press the thumbs up button. If you have comments, please share them here or on the draft itself. Thanks, Otto
Re: Request for feedback on draft: DEP-18: Enable true open collaboration on all Debian packages
Hi! > > I have drafted a new DEP at > > https://salsa.debian.org/dep-team/deps/-/merge_requests/8 titled "DEP-18: > > Enable true open collaboration on all Debian packages". > > > > Direct link to raw text: > > https://salsa.debian.org/dep-team/deps/-/raw/798bfa5a1e1609afd4e48ee20aff80a82bcd4a2f/web/deps/dep18.mdwn .. > Sorry, but I disagree that the only true collaboration is Salsa-rich > collaboration. Where are my options to mirror the data at Salsa, as I > can do with mailinglists and Debbugs, to work with it also offline? First of all, thanks for maintaining 650+ packages in Debian. You have for sure developed an optimal workflow for yourself. I also do a lot of email based stuff myself and often work offline. Note that GitLab does allow responding by email to notifications and to carry reviews by email. Depending on configuration, one could even submit patches by email[1]. There are also command-line tools that allow various actions without having to use the browser [2,3]. I do however understand that people who already have an optimal workflow are probably not keen to change it unless the new workflow is vastly superior, and GitLab/Salsa isn't always perfect in all regards. I do however believe that in the grand scheme of things promoting the five key principles outlined in DEP-18 would benefit Debian as a whole. Also, I can see that you are already following principles 1 and 2 of the proposal by having almost all of your packages in git and on salsa.debian.org. As the DEP-18 draft text says, strict enforcement is not a wise tactic in the context of fostering collaboration, and I would not expect you to move away from what you are doing now, as it works for you and benefits Debian with 650+ packages. In your case following the principles 3, 4 and 5 would probably not be appropriate in cost vs benefit. For example running Salsa CI or asking for code reviews prior to upload would probably just slow things down too much without the gain in your case. However, I would still argue that DEP-18 would be useful as a general guideline in Debian and in particular for team maintained packages and important packages (as discussed in the "end single maintainership thread"). Having such principles published would help maintainers (at least new maintainers) to align on a set of basic principles that help drive more collaborative development. [1] https://docs.gitlab.com/ee/user/project/merge_requests/creating_merge_requests.html [2] https://manpages.debian.org/unstable/devscripts/salsa.1.en.html [3] https://manpages.debian.org/unstable/glab/glab.1.en.html
Re: Request for feedback on draft: DEP-18: Enable true open collaboration on all Debian packages
Hi! Thanks for the comments. Responding inline to the questions you raised: > Then it makes no effort to define communication channels. Before making some > time consuming change, it's usually better to just ask if that's a good idea .. > Then the document should enforce a particular commit style. I've seen several > people who do a bunch of unrelated changes in a single commit. The idea in DEP-18 is to define a few key principles that enable collaboration. There is no need to publish recommendations on communicating channels or git commit message styles. We have maintainer guides, team-specific guides and a lot of other documentation for that kind of stuff. Also, for a DEP to recommend something it should already be popular and widely supported by maintainers, hence the five principles it now has. > > Run Salsa CI at least once before every upload to ensure minimal level of > > quality > > Can you tell me how to do a CI for python3-motor, given that it's a client for > a proprietary database? > > Testing GUI packages, or fonts packages, or clients is generally very > difficult > if not impossible. What's the plan in those cases? Salsa CI mimics what the official Debian QA systems do. You don't need to write tests specific for Salsa CI. In the case of python3-motor, just improve on the autopkgtests it has, and both Salsa CI and Debian CI will run them. Salsa CI just brings the benefit that you can more easily catch regressions in packaging or new upstream versions _before_ uploading to Debian, thus protecting the quality of the packages in official repositories. > > While collaboration can happen based purely on developers submitting patch > > files as email attachments directly to each other, to mailing lists or in > > bug > > reports, it does not scale well. > > It scales better than having to open salsa, find out the git:// url, add a new > remote in the local git, diff the patch locally, then of course I can't just > merge the patch locally because salsa doesn't know about that, so I have to go > again on salsa and click to merge. You can find the git url in the package debian/control Vcs-* fields. No matter what way you do a patch, you need to download the sources in some way. Doing a git clone isn't really extra work. I am not sure I followed your description, but if you merge something locally and push to Salsa, it will automatically detect that a commit in a Merge Request landed on the target branch and mark the Merge Request merged on git push. > All of this with salsa being not fast nor responsive by any definition of > those > terms. I agree that Salsa is sometimes a bit sluggish (https://salsa.debian.org/salsa/support/-/issues/395), but I hope we can do something to improve it and it is now a permanent reason to not use Salsa. > > Allow changes to be reviewed before uploads to Debian > > I guess this means that we should have some mandatory waiting time from the > last commit to the upload. How long would this time be? Would it apply even if > we're talking about fixing a libc upload that will break any system where it > gets installed? I will try to reword "5. Allow changes to be reviewed before uploads to Debian" to be explicit about not prescribing anything too specific. It is up to each team and maintainer to decide what and when to ask for reviews. The main point in DEP-18 is that when you use salsa.debian.org, the code is published as source first, tested with Salsa CI and potentially reviewed instead of just being uploaded directly without any room for collaboration. - Otto
Re: Request for feedback on draft: DEP-18: Enable true open collaboration on all Debian packages
Hi! On Fri, 2 Aug 2024 at 02:27, Andrea Pappacoda wrote: .. > Before a certain way of doing things can be mandated or "warmly > recommended", the technology has to be as flawless as possible - and > today I wouldn't call Salsa "flawless", would you? Issues with > Salsa/GitLab: > > 1. It is so slow that it makes me want to close by browser and do > something else instead > 2. It doesn't support email-based workflows > 3. It has a lot of features we don't need. Or rather, it has more > features we *don't* need than we do. > 4. Its user interface is confusing, it's often hard to find what I'm > looking for > 5. Did I mention how slow it is? > 6. It is developed by a company who's philosophy and interests are > misaligned with Debian's > 7. The product it tries to mimic, GitHub, is also misaligned with > Debian's philosophy and interests > > While performance is something we could reasonably do something about, > all the other points are part of GitLab by design, and we're stuck with > them. GitLab of course has more features than what we need; it does not mean it has to be used. The basic feature set in GitLab to browse repositories, show and compare history, show CI status, list MRs, separate drafts and ready ones, show approvals, show assignees etc work well. GitHub has done many things well and trying to compete with them is a good thing. Both GitLab and GitHub also support email workflows to some degree (but of course most people use them via the UI as the UI offers many things email does not and never will). I agree that Salsa is sometimes a bit sluggish (https://salsa.debian.org/salsa/support/-/issues/395), but I hope we can do something to improve it and it is now a permanent reason to not use Salsa. ... > But the point isn't that much about being able to use emails > specifically, it is about having the possibility of having a software > forge which is interoperable with a wide range of tools and can stay > out of the way if wanted. With GitLab, you either use the full package > (which includes browsing, reviewing, and commenting) or you get a > sub-optimal experience. But it doesn't have to be this way. > > I'll now talk a bit about SourceHut. Not to suggest that we should use > that instead, but just to point out how things *can* be done. I have a paid SourceHut account and have been testing it for some time. I can see the appeal of simplicity, but for actual team work it is *too* simple. It will probably take a couple of years for it to mature. Note that a DEP is not a vehicle to drive out new version control systems or source forges. It is a vehicle to formalize something that is already popular as an official best practice. The salsa.debian.org is what we have today, and Debian has been using for a 5+ years. If some day in the future something vastly superior to git or GitLab surfaces and a lot of people start using it, the DEP could be updated. But I would not today recommend new Debian maintainers to host on SourceHut nor GitHub nor self-host. They can mirror there if they want, but for collaboration prior to uploading to work well the source code should be on salsa.debian.org for now. When I today reviewed recent submissions on mentors.debian.net, most of them were hosted on Salsa, but 4 on GitHub and 1 in nowhere in git at all. As it currently stands, there is no official document I could lean on to ask the submitters to please use salsa.debian.org instead. This is one of the motivations for this DEP.
Re: Request for feedback on draft: DEP-18: Enable true open collaboration on all Debian packages
Hi, On Fri, 2 Aug 2024 at 16:27, Johannes Schauer Marin Rodrigues wrote: > Quoting Otto Kekäläinen (2024-08-02 17:23:51) > > I agree that Salsa is sometimes a bit sluggish > > (https://salsa.debian.org/salsa/support/-/issues/395), > > what kind of hardware do you have? For people like me who are on slower I have a 4-year old Dell XPS 13 (that came with Ubuntu preinstalled and has perfect Linux support, great laptop by the way). > hardware, the web experience is absolutely not funny and "a bit sluggish" > doesn't begin to describe it. It is hard to find any other website I'm > visiting > that is slower than gitlab. For example, when I run this on my Firefox I get a > score of 1.53: https://browserbench.org/Speedometer3.0/#summary I got 9.25. Some of the GitLab slowness is due to the server side, and some due to JavaScript. I prefer to use mild terms "a bit sluggish" out of empathy to the people who maintain Salsa, as it is probably not fun for them to read too vocal complaints after the incredible amount of work they have put in to get us this far. I am very grateful for their work and with the language in the bug report I try to be constructive on what things to prioritize next. ... > You now said "I hope we can do something to improve it and it is now a > permanent reason to not use Salsa." twice: > > Did you typo it twice or do you actually mean that it's now a permanent reason > to not use salsa? Thanks for pointing out the typo. I meant 'not a permanent reason'. I typo now/not frequently for some odd reason and since both words are in the dictionary, my spell checker does not flag it. > I am not suggesting salsa use because I think it's the best thing since the > invention of sliced bread. But personally, I rather use something suboptimal > if > it means that we can more or less agree on a "default" and I think that the > current situation (submission of patches by mail and the debian archive is the > bts) is deterring to newcomers. I know that most people probably have faster > machines than I. As it was pointed out elsewhere in this thread, Debian work > already implies a lot more waiting than "a few seconds" for salsa to finish > loading a page or finish yet another animation, so meh. With the glab tool I > think I can be happy enough. This I think summarizes well the opinion of most people I have spoken with. GitLab is not perfect, but it was chosen and we have been using it for 5+ years mostly successfully. Most packages are there and we should state that it is the recommended option. Currently the second most popular option is to use GitHub, or not use any vcs at all. A lot of this thread has gone into people expressing their dislike for Salsa. Most people still use Salsa - I guess the silent mass isn't chiming in in this thread that much now. Some people probably have very optimized email workflows, and nobody can argue against the statement that a pure email workflow for sure is simple, and has its elegance. But it also has shortcomings such as no lack of metadata/status, no built-in way to run CI and share the code+logs+CI results etc. Following the principles 1 (use git) and 2 (use Salsa) allows for the next principles 3, 4 and 5 to take place. I would be curious to hear more views about them. DEP-18 summary (https://salsa.debian.org/dep-team/deps/-/merge_requests/8): 1. Have package source code in version control, using git 2. Host package source on Debian's code forge Salsa 3. Run Salsa CI at least once before every upload to ensure minimal level of quality 4. Allow Merge Requests to be submitted 5. Allow changes to be reviewed before uploads to Debian My plan is to summarize the discussion and update the proposal in the coming days, incorporating as many views as possible. I will also in the next update include additional relevant context info such as tag2upload, glab and examples of how to easily work with both Debian bug tracker and MRs in parallel. Please share your views, and also +1 or -1 the proposal on Salsa so we can incorporate that too in measuring the support of this. Remember that DEPs are soft rules - unlike General Resolutions (GRs) - and maintainers who have specific reasons to not want to use git or Salsa will not be forced to do so.
Re: Request for feedback on draft: DEP-18: Enable true open collaboration on all Debian packages
Hi! > I have three feelings. > > > 1. Debian workflows are too fractured. The project would be better if we > asked people to standardize around a single (or a small number) of workflows. > To do so, the workflow would need to be flexible enough to handle the wide > range of technical needs of all the packages and upstream configurations. > > > 2. Standardizing around a single (or small number of) workflows will make > some people unhappy. But that is an acceptable price to pay because of the > general benefit to the project as long as the correct solution is adopted. > Unity is more important than minority opinions on this particular issue. > > > 3. I do not yet have the wisdom to ascertain what the correct solution is. > Until I do, I applaud those who attempt to push this discussion forward, and > I follow it closely, but I haven’t commented. I think adopting an incorrect > mandated (or maybe even recommended) workflow is worse than the fractured > status quo. > > > Number 3 is why I haven’t previously commented. In regards to DEP-18, I > don’t know if it is the correct way to go for many of the criticisms that > have already been expressed. But, if it isn’t DEP-18, I think it will > eventually be something. And, although this might not be a popular opinion > among some, I think Debian should get to the point that there is one workflow > (or a very small number of workflows) that all packages are expected to > follow, both for packaging and for collaboration. Thanks for voicing this! I think a lot of people have the same feeling that if there were a bit more common principles and practices across all packages and maintainers, contributing to multiple different packages would be easier for old maintainers, and learning how to contribute efficiently in the first place would be easier for new maintainers. Also I fully agree that suddenly forcing everybody to do something unpractical would be detrimental to Debian. Therefore the DEP-18 proposal is based on the principles most packages and teams already follow based on trends.debian.net and what I know about e.g. Python, Golang and kernel team policies. There is also no way a volunteer project such as Debian could mandate a two-person-rule as it would instantly grind all single-maintainer packages to a halt. It is and will be OK to have single maintainer packages now and going forward if there is just one person interested in the package. DEP-18 hopefully helps to unify some things and allow for more people to step up and help with packages they have not worked on before, but I intentionally want it to be a fairly soft mechanism, and not overly prescriptive in the details. I also object to having any votes or mandatory rules on this. This is just a DEP on purpose and not a GR. Who knows if a superior alternative to for example git surfaces in the next 15 years. If at that time people *really* want to move away from git they should be allowed to do so and not be prohibited by too strict rules. For the reasons 2 and 3 in Soren's list, let's continue to discuss what are the best practices and principles. I am sure we can eventually get a consensus that suits most people and creates the best environment for easier collaboration.
Re: lintian.debian.org off ?
Hi! On Fri, 9 Aug 2024 at 04:12, Nicolas Peugnet wrote: > > If there is interest in providing a page that only list the tag > > description (without the affected packages), it would be easier to add > > it to the existing UDD page (with an additional parameter for example) > > than to create a separate service. > > As I said, The reason I made this is because it was very frustrating to > click on links to lintian.debian.org as a newbie. Each time, I expected > to see more information about the tag to help me understand what it > exactly meant and how to fix it. > Currently these links lead to nowhere. I think this should be fixed as > it adds a lot of friction for newcomers. +1 to this and the rest of Nicolas' message. Additionally I wanted to raise that a lot of newcomers use search engines to learn what the issues are. Also when discussing code quality with upstream developers, they might be interested to learn more, and when they enter in a search machine e.g. 'executable-in-usr-share-docbase' they will get zero relevant results. When lintian.debian.org was running, it used to always be the top result and an easy place to refer upstreams and newcomers to read up on the issue. Nicolas' implementation (https://lintian.club1.fr/) to list all tags on one page and link to UDD seems like a reasonable compromise in functionality and maintenance effort.
Salsa connection errors or slowness has improved (Re: Request for feedback on draft: DEP-18)
Hi! In the DEP-18 thread surprisingly many (e.g. Salvo, Johannes, Andrea, Gioele) complained about Salsa being slow to load, or having connectivity issues. I am thus happy to note that the Salsa admins posted in https://salsa.debian.org/salsa/support/-/issues/395 a comment stating that salsa.debian.org is about to get a hardware update which should make it a tad snappier. Salsa recently gained a shared riscv64 runner so you can now test a new architecture in Salsa-CI, and based on https://salsa.debian.org/salsa/support/-/issues/266 and https://salsa.debian.org/salsa/support/-/issues/301 more shared runners on various architectures might be coming. Salvo and others also pointed out that "git push" occasionally failed. According to https://salsa.debian.org/salsa/support/-/issues/381 that issue should be fixed by now. I also remember seeing this earlier, but I have not experienced it a single time in at least two weeks. One remaining thing that affects negatively salsa.debian.org are occasional HTTP/2 errors. If you also experience these, you might want to +1 the issue https://salsa.debian.org/salsa/support/-/issues/404 or help the Salsa admins debug it. As you probably guessed, I am a heavy user of Salsa, and very grateful to Salsa admin for maintaining it! - Otto
Re: Accepting DEP14?
Yes to finalizing DEP-14 soon, but first I think we need to complete the technical work to have git-buildpackage use DEP-14 branch names by default. I tried implementing it but turned out a bit too involved.. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=829444 Use DEP14 branch layout by default = master -> debian/latest = upstream -> upstream/latest Any hands available to help with this?
debian/gbp.cnf analytics? (Re: Re: Accepting DEP14?)
Hi! I am happily using debian/gbp.conf and debian-branch=debian/latest in all of my packages but based on the DEP14 discussion seems some people prefer debian/sid or debian/unstable (and some of them upload to experimental from the branch despite the name, and some maintain a separate debian/experimental branch for experimental uploads). However this the responses are just a sample based on who happens to have time to read debian-devel@ discussions. I tried to use codesearch.debian.net to find out how many packages have a debian/gbp.conf but it seems it can't be used to simply list packages that have a specific file, it always also needs a search terms to look up inside the file. With https://codesearch.debian.net/search?q=path%3Adebian%2Fgbp.conf+debian-branch+%3F%3D+%3Fdebian%2Flatest&literal=0 I was able to find that 1655 packages have either "debian-branch = debian/latest" or "debian-branch=debian/latest". Is there some easy way to iterate every single Debian package and extract just one single file from them without having to download all packages? I'd like to see how many % of all Debian packages have a gbp.conf file, and then download all of them to do stats on what they contain. - Otto
Re: debian/gbp.cnf analytics? (Re: Re: Accepting DEP14?)
Hi! ## How many source packages are in Debian unstable as of today? ± zgrep "^Package: " Sources.gz | wc -l 38199 ## How many source packages have a gbp.conf? ± ls -1 *_gbp.conf | wc -l 13570 (24629 do not have it) ## What is the most popular 'debian-branch'? Note! The Sources.gz used to analyze this was from Debian unstable so one would not expect to see any debian/bookworm or debian/12-bookworm kind of lines here. ± grep --no-filename --only-matching --max-count=1 --perl-regex "^debian-branch( )?=( )?\K([^ ]*)" *gbp.conf | sort | uniq -c | sort -nr 2284 debian/master 1898 debian/sid 1655 debian/latest 990 master 520 debian 304 debian/unstable 179 debian/main 156 main 133 debian-sid 28 debian/experimental 24 unstable 20 debian-unstable 12 experimental 5 debian-experimental 4 latest 3 sid 2 llvm18/main 2 llvm17/main 2 llvm16/main 2 llvm15/main 2 llvm14/main 2 debian-pkg 2 deb ## What is the most popular 'upstream-branch'? ± grep --no-filename --only-matching --max-count=1 --perl-regex "^upstream-branch( )?=( )?\K([^ ]*)" *gbp.conf | sort | uniq -c | sort -nr 1846 upstream/latest 1488 upstream 220 master 130 upstream-sid 47 upstream/master 30 main 15 upstream-unstable 14 upstream/sid 10 master-dfsg 9 there-is-no-upstream-branch 6 dfsg 5 release 4 upstream-experimental 4 dfsg-orig 3 upstream-tarball 3 upstream-release 3 upstream-dfsg 3 invalid 3 dfsg_clean 3 debian-upstream ## What is the most popular 'upstream-tag' format? ± grep --no-filename --only-matching --max-count=1 --perl-regex "^upstream-tag( )?=( )?\K([^ ]*)" *gbp.conf | sort | uniq -c | sort -nr 943 upstream/%(version)s 350 v%(version)s 267 %(version)s 23 'v%(version)s' 11 '%(version)s' 9 upstream/v%(version)s 4 version/%(version)s 4 'upstream/%(version)s' 3 upstream-tarball/v%(version)s 3 snapshot-%(version)s 3 release-%(version)s 2 v%(version%~%-)s 2 version_%(version)s 2 upstream/%(version)s+dfsg 2 release/v%(version)s ## How many packages have a 'upstream-vcs-tag' and what is it typically? ± grep --no-filename --only-matching --max-count=1 --perl-regex "^upstream-vcs-tag( )?=( )?\K([^ ]*)" *gbp.conf | sort | uniq -c | sort -nr 214 %(version)s 187 v%(version)s 156 %(version%~%.)s 126 %(version%~%-)s 52 v%(version%~%-)s 19 v%(version%~%.)s 8 release/%(version)s 5 release/%(version)s/final 3 release-%(version)s 2 v-%(version)s 2 rel-%(version)s 2 gnupg-%(version)s ## How many packages have 'pristine-tar'? ± grep --no-filename --only-matching --max-count=1 --perl-regex "^pristine-tar( )?=( )?\K([^ ]*)" *gbp.conf | sort | uniq -c | sort -nr 9098 True 508 False 169 true 46 false 3 1 ## How many packages have 'upstream-signatures'? ± grep --no-filename --only-matching --max-count=1 --perl-regex "^upstream-signatures( )?=( )?\K([^ ]*)" *gbp.conf | sort | uniq -c | sort -nr 7 on 6 True 2 auto ## How many packages have 'sign-tags'? ± grep --no-filename --only-matching --max-count=1 --perl-regex "^sign-tags( )?=( )?\K([^ ]*)" *gbp.conf | sort | uniq -c | sort -nr 2587 True 55 true 9 False ## Which lines in gbp.conf in general are most common? ± cat *_gbp.conf | sort | uniq -c | sort -nr 13032 [DEFAULT] 10116 8450 pristine-tar = True 2746 [import-orig] 2553 sign-tags = True 1771 debian-branch = debian/sid 1734 upstream-branch = upstream/latest 1731 [buildpackage] 1547 dist = DEP14 1527 debian-branch = debian/latest 1446 debian-branch = debian/master 1307 upstream-branch = upstream 1117 [dch] 1059 patch-numbers = False 987 filter = [ '.gitignore', '.travis.yml', '.git*' ] 967 [pq] 873 debian-branch = master 870 upstream-tag = upstream/%(version)s 771 debian-branch=debian/master 729 # Configuration file for git-buildpackage and friends 691 pristine-tar=True 678 multimaint-merge = True 604 debian-tag = debian/%(version)s 526 filter = */.git* 501 pristine-tar = False 489 filter=[ '.gitignore', '.travis.yml', '.git*' ] 466 debian-branch = debian 436 filter-pristine-tar = True 369 compression = xz 360 # Always use pristine-tar. In the light of these stats I am fine with current version of the DEP-14 text, and I am happy with what I settled on in my packages, in particular the most complex one (https://salsa.debian.org/mariadb-team/mariadb-server/-/blob/debian/latest/debian/gbp.conf) that uses basically all features of git-buildpackage and DEP-14). Also, it would be cool if trends.debian.net included some kindof gbp.conf stats to track how things evolve over time. For that wishlist request I filed https://salsa.debian.org
DEP18 follow-up: What would be the best path to have all top-150 packages use Salsa CI?
Hi! In short: I would very much like to see all top-150 packages run Salsa CI at least once before being uploaded to unstable. What people think is a reasonable way to proceed to reach this goal? Background: We have had several cases recently where an upload to Debian unstable causes widespread failure in unstable, and it could have been easily prevented if Salsa CI pipeline had run for the package and revealed the problem before upload to archive for everyone to suffer. This message was prompted by the fact that right now we have a massive large of Python packages affected by the "No module named 'packaging'" bug [1] which would have been caught by Salsa CI running the autopkgtest job before upload [2]. I don't want to blame maintainers for missing on some details while doing new releases - it happens. But systematically not using available and easy testing facilities does annoy me. We can't have Salsa CI for all of Debian overnight, but at least widespread issues could be mitigated significantly by having Salsa CI at least on the top-150 or top-500 packages. We do not have stats on how many packages use Salsa CI, but we have stats on how many are not even hosted on Salsa: ``` curl -LO https://trends.debian.net/vcs-hosting_unstable-packages.csv curl -LO https://popcon.debian.org/sourcemax/by_inst for x in $(tail -n +13 by_inst | head -n 150 | cut -c 7-25) do grep -E "^$x," vcs-hosting_unstable-packages.csv done | grep -v salsa dpkg,1.22.10,other coreutils,9.4-3.1,no vcs acl,2.3.2-2,other zlib,1:1.3.dfsg+really1.3.1-1,no vcs attr,1:2.5.2-1,other hostname,3.23+nmu2,no vcs readline,8.2-4,no vcs e2fsprogs,1.47.1-1,other base-files,13.3,no vcs bash,5.2.21-2.1,not git expat,2.6.2-1,no vcs gettext,0.22.5-2,no vcs diffutils,1:3.10-1,no vcs libbsd,0.12.2-1,other sqlite3,3.46.0-1,no vcs dmidecode,3.6-1,other pciutils,1:3.13.0-1,other libxdmcp,1:1.1.2-3,git on alioth wget,1.24.5-2,no vcs file,1:5.45-3,other laptop-detect,0.16,other fuse,2.9.9-8.1,no vcs lsof,4.95.0-1.1,no vcs scowl,2020.12.07-2,other emacsen-common,3.0.5,no vcs libusb-1.0,2:1.0.27-1,no vcs setuptools,70.3.0-2,no vcs traceroute,1:2.1.5-1,no vcs liblockfile,1.17-1,github ``` If you are a maintainer of a top-150 package and want help in getting it on Salsa and have Salsa CI activated, just email debian-salsa-ci@, and we will guide you through how to best run `gbp import-dscs --debian-branch=debian/latest --upstream-branch=upstream/latest --pristine-tar`, what to put in a README.source how to activate Salsa CI with potential customizations you feel are make sense. We have already done this to many projects, but as surprisingly many maintainers prefer not to receive contributions, we encourage those who want to have help to reach out themselves. When I proposed DEP18[1] my main motivation was to get more packages on git and on salsa.debian.org so that it is easier to collaborate on the next upload with the maintainer and all interested contributors before the upload is done. Collaborating on packages by NMUs is just not a modern and efficient way to collaborate. On top of this, I also wished that packages would use Salsa CI, at least once before the upload. This helps the maintainer get assurance of the package health before upload. Running Salsa CI on Merge Requests automatically helps contributors get immediate feedback, and it also gives the maintainer assurance that contributions don't cause easily testable regressions. Many people raised concerns on DEP18, and some of them are valid, and I will continue to ponder about it and how to restructure the proposal to foster collaboration without alienating any of our current maintainers who have a well optimized existing workflow. However, pushing for wider Salsa CI adoption I think makes sense as a goal by itself, and I would be keen to hear what people think is a reasonable way to proceed with that? [1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1079175, https://salsa.debian.org/salsa-ci-team/pipeline/-/issues/376 [2] https://salsa.debian.org/python-team/packages/setuptools/-/jobs/6156348 [3] https://salsa.debian.org/dep-team/deps/-/merge_requests/8
Re: DEP18 follow-up: What would be the best path to have all top-150 packages use Salsa CI?
Hi! > > I would very much like to see all top-150 packages run Salsa CI at > > least once before being uploaded to unstable. What people think is a > > reasonable way to proceed to reach this goal? > Advertise widely and frequently that there is a pool of people which is > happy to help investigating the failed CI jobs. > Then start personally advocating the benefits of CI to the maintainers > of these packages: I expect that in most cases you will find out that > they are not using CI just because they are not well informed about it. So maybe just send a mass email to the maintainers of these 150 packages? > Recently I enabled CI for most of my packages, and the experience has > been generally positive. Glad to hear! ... > Of course, this works best if a package *HAS* autopkgtests, so it would > be great if more people contributed non-trivial tests to the packages > they care about. Autopkgtests are of course nice, but Salsa CI will also help detect if the build is broken, or if e.g. debian/control relationships are broken and package is uninstallable etc. The coverage is not complete, but if the basic things that Salsa CI tests for break, then the package is most likely going to cause havoc once it lands in unstable and require an urgent re-upload. As examples, the failing build of https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1071245 broke installation of gnupg on Debian unstable and thus everything that uses gnupg and could have been prevented with simple Salsa CI run (the package has now Salsa CI, so this won't repeat), or https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1021336 where pam files tried to overwrite files in other key packages, breaking Debian unstable installations and chroot creation for everyone would have been caught by basic Salsa CI tests (MR to include Salsa CI suggested in https://salsa.debian.org/vorlon/pam/-/merge_requests/20). Using Salsa CI will save both the maintainer some headaches, and protect everyone using unstable and doing packaging work from being affected by a temporarily broken dependency. > Something that many developers are probably not aware of is that they > can enable CI for a repository with no code changes at all and with > a single command: > > salsa update_projects $NAMESPACE/$PROJECT \ > --jobs yes --ci-config-path recipes/debian.yml@salsa-ci-team/pipeline > > (salsa(1) is part of devscripts) > > And then they can immediately schedule a new pipeline without having to > push a new commit: > > https://salsa.debian.org/$NAMESPACE/$PROJECT/-/pipelines/new > > This allows to see how Salsa CI works with very low friction and no > committment at all: worst case it can be disabled again and nobody will > notice. :-) Thanks, these are great to point out! i will add them to the Salsa CI README in next docs update (https://salsa.debian.org/salsa-ci-team/pipeline/-/merge_requests/519).
Re: DEP18 follow-up: What would be the best path to have all top-150 packages use Salsa CI?
Hi! > I'd like to rephrase your quest. What you really want is unstable to be > less unstable. Whilst a number of people disagree with that notion, I I don't think anybody disagrees that breaking a core packages and stopping (nearly) all other DDs from doing development for a day or two is acceptable. I do agree that there are some developers who view unstable as a CI system by itself, and we occasionally see certain DDs do multiple uploads in a few days for the same package after they read the QA system results. This is better than nothing, and maybe OK for small/unimportant packages. For important packages I think it would be vital to have enough CI should run before upload unstable, and use unstable only to detect stuff that is hard to gate keep with automatic testing (as you wrote in slightly different words). > One complaint I've seen about this workflow and one that I agree with is > the waiting time. Checking salsa-ci before uploading incurs an extra > context switch. What we'd really like to do is click the equivalent to The CI waiting time is always less than human review waiting time. For example in https://salsa.debian.org/debian/entr/-/merge_requests/10 the CI took 10 minutes, but it will take several days for a reviewer to respond on the MR itself. Also, I think a 10-minute break is a good thing for the author themselves. The maintainer should get some fresh air and do the final review of their changes after a small break to ensure nothing was done too hastily. > "Auto-merge" (if the pipeline passes). As I write this, Sean or Ian will > likely come along and say tag2upload. Working in this direction and > enabling a "upload if ci passes" would bring the salsa-ci experience to > another level. This would be super cool, in particular if it was extensible to get triggered automatically when a new upload has had a minimum number of approvers in the MR, and it would got merged and tagged and uploaded automatically. > Let me suggest that there are more ways to do this. Freexian is putting > a ton of effort into https://debusine.debian.net. It can do much of the > same tasks as salsa-ci already (with less flexibility). Extending it to > act as an upload-proxy that forwards your upload to the archive if > builds pass could be another option of improving unstable quality. In > earlier times, debomatic.debian.net was used as a pre-upload QA tool if > I remember correctly. I used to use debomatic.debian.net, and I have been reading about Debusine and the funding it gets from the Sovereign Fund. I tried to use Debusine back in March, but the client requires Python 3.11+ which my system does not have. I will need to make a new attempt soon. Thus, I don't have any good opinion about it yet. However, as you probably guess, I have a strong preference for workflows that integrate version control and code reviews and CI. This is what Salsa does (and GitLab, GitHub and alike). > Then the top-150 packages tend to be packages with unusual aspects. For > instance, the git repositories for gcc-VER, glibc and linux all lack > upstream sources. For linux, there is a pipeline, but in order to > complete in a timely manner, it enables the pkg.linux.quick build > profile and the pipeline is elaborate with a complex extract-source > stage. It's not a matter of just enabling the pipeline for our core > packages but spending a lot of time fiddling with the settings until it I have been reading the Kernel team Salsa CI before, and I am impressed how the pipeline has been customized, and glad to see it was green for the latest upload: https://salsa.debian.org/kernel-team/linux/-/pipelines/720187 I suspect you are right that some of the top-150 packages are special case that can't use Salsa CI easily, but for example the setuptools that had #1079175 yesterday would have been able to run Salsa CI without anything custom (just a debian/gbp.conf file to define how it needs to be built from git, see https://salsa.debian.org/python-team/packages/setuptools/-/commits/debian/latest/). > works. I guess that sending a working pipeline configuration for these > could improve the situation. Would it also make sense to source I have done that - details visible in my MR history at https://salsa.debian.org/dashboard/merge_requests?scope=all&state=all&author_username=otto&search=%22enable+salsa%22 - and will continue to. For example, after #1071245 happened, I sent to GnuPG an MR that enabled Salsa CI and included fixes to make it green. Inspired by #1021336 I just sent one to PAM this week. Some maintainers happily accept it, but surprisingly many reject, and I have no backbone from any DEP or GR or policy to ask them to seriously consider using Salsa CI. Some even say that they don't want to look at MRs at all, no matter how many fixes might be pending there on a silver plat
Re: [Debian-salsa-ci] DEP18 follow-up: What would be the best path to have all top-150 packages use Salsa CI?
Hi! > And is this web page authoratative? Or just a false search positive? > > https://salsa.debian.org/salsa-ci-team/pipeline#basic-use > > It doesn't mention the "salsa" command at all, but maybe that isn't > the right web page. This goes back to my observation that it would be > helpful if there was better documentation to make life easier for > package maintainers. Yes, it is the authoritative page. I will update the page based on discussions on debian-devel. Draft pending at https://salsa.debian.org/salsa-ci-team/pipeline/-/merge_requests/519 Thanks for sharing your experiences!
Re: DEP18 follow-up: Salsa CI vs Debusine
Hi Helmut! > > Let me suggest that there are more ways to do this. Freexian is putting > > a ton of effort into https://debusine.debian.net. It can do much of the > > same tasks as salsa-ci already (with less flexibility). Extending it to > > act as an upload-proxy that forwards your upload to the archive if > > builds pass could be another option of improving unstable quality. In > > earlier times, debomatic.debian.net was used as a pre-upload QA tool if > > I remember correctly. > > I used to use debomatic.debian.net, and I have been reading about > Debusine and the funding it gets from the Sovereign Fund. I tried to > use Debusine back in March, but the client requires Python 3.11+ which > my system does not have. I will need to make a new attempt soon. Thus, Ok, I tested Debusine again and managed to build a package following the tutorial with the build definition: ``` build_components: - any - all environment: debian/match:codename=trixie:variant=sbuild host_architecture: amd64 input: source_artifact: 507533 ``` However, I struggle to figure out how to do something similar to what Salsa CI. Do you happen to have at hand a complete build+test definition I could copy-paste and test? Also, I see the build reports 'success' despite Lintian failing on an error. Additionally, seems this system does not send out email notifications? At https://freexian-team.pages.debian.net/debusine/reference/faq.html it says in two ways that the core design philosophy is to invent something new instead of using something existing, as the existing thing isn't controlled by Debian. That makes sense in some situations, but in the case of a code forge with review, build and test features, it seems that trying to build a new custom thing from scratch is perhaps a bit too ambitious? GitLab isn't perfect, but I'd say that the setup Debian has now with salsa.debian.org is pretty good, and I'd rather polish it than fund building a new system from scratch. I am however willing to test Debusine a bit more to see if it has hidden powers that could be amazing.
Re: Validating tarballs against git repositories
On Tue, 2 Apr 2024 at 17:19, Jeremy Stanley wrote: > On 2024-04-02 16:44:54 -0700 (-0700), Russ Allbery wrote: > [...] > > I think a shallow clone of depth 1 is sufficient, although that's not > > sufficient to get the correct version number from Git in all cases. > [...] > > Some tools (python3-reno, for example) want to inspect the commits > and historical tags on branches, in order to do things like > assembling release notes documents. I don't know if any reno-using > projects packaged in Debian get release notes included, but if they > do then shallow clones would break that process. The python3-pbr You could use --depth=99 perhaps? Usually the difference of having depth=1 or 99 isn't that big unless there was a recent large refactoring. Git repositories that are very big (e.g. LibreOffice, MariaDB) have hundreds of thousands of commits, and by doing a depth=99 clone you avoid 99.995% of the history, and in projects where changelog/release notes is based on git commits, then 99 commits is probably enough. I wanted to also highlight that git-buildpackage supports '--depth=', so this is a command and well supported usage pattern (https://manpages.debian.org/testing/git-buildpackage/gbp-clone.1.en.html#depth).
DEP-18: Git and GitLab usage in other Linux distros (Re: Representing Debian Metadata in Git)
/cgit.freebsd.org/ports/tree/databases/mariadb114-server Custom ports directory, package source builds based on Makefiles. Authoritative hosting on cgit, but they have also Codeberg, GitHub and GitLab mirrors. Most contributions seem to be patch files attached to bug reports (e.g. https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=274494) but there are also some GitHub PRs (https://github.com/freebsd/freebsd-ports/pulls). The GitLab mirror had zero Merge Requests. ## OpenBSD ports Example: https://openports.pl/path/databases/mariadb Custom ports directory. Source code hosted on both csvweb and GitHub mirror. Monorepo with only packaging files (Makefile + patch files and more). Instead of upstream sources there are files named 'distinfo' that contain the upstream tar.gz name and SHA256. ## MacPorts Example: https://ports.macports.org/port/mariadb-10.11/details/ -> https://github.com/macports/macports-ports/blob/master/databases/mariadb-10.11/Portfile Custom ports directory software. Packages in custom Portfile in monorepo with SHA256 references to upstream tarballs. ## Homebrew Example: https://formulae.brew.sh/formula/mariadb -> https://github.com/Homebrew/homebrew-core/blob/master/Formula/m/mariadb.rb Again, custom ports directory, custom packaging format, files on GitHub in a monorepo and upstream tarballs referenced by sha256. ## Chocolatey Example: https://community.chocolatey.org/packages/mariadb -> https://github.com/mkevenaar/chocolatey-packages/tree/master/automatic/mariadb This is the Windows equivalent of Homebrew. Packaging in .nuspec files, fully custom thing. A centralized directory and seems a lot of the packages are in a single monorepo on GitHub, not sure if that is all, though. Conclusions & personal comments: - Everyone uses git. Sentiments that the Debian repository is enough of version control, or that git would somehow not be suitable for Debian packaging, seem detached from practical reality. - Everyone uses normal git. The fact that Debian has two or three branches (debian/latest + upstream/latest + pristine-tar) is a bit special. I would still consider it warranted, as we have higher standards regarding hosting the source code and having a full audit trail of changes. If this is ever simplified, the somewhat obvious solution would be to introduce a new debian/latest branch structure that is just the current 'debian/*' contents plus a file, perhaps called 'upstream-source', with url and sha256 sum of upstream tarball. - GitHub and GitLab are quite common, people know how to use them. The fact that we have salsa.debian.org is an asset for us and helps attract new people. Thanks to various cli-tools people preferring to do all work from the command-line (or from Emacs) should not inherently be worse off if we have more contributors joining the project who use Salsa. - Pull Requests and Merge Requests are also very common. I think the best course forward would be to have an open mind in accepting contributions both as git pull requests as well as patch files in email. - The number of contributors/maintainers is low everywhere. Ending single-person maintainership is not going to happen any soon, but hopefully, we can work towards first increasing the pool of contributors who participate, and then expand on practices around Merge Requests and reviews and maybe have some kind of formal sign-offs from at least two people before upload. Initially, perhaps only for the top-150 packages. But before we can institute review workflows, we need to have more unification around the version control and basic packaging workflows. - Otto
DEP-18 discussion summary (Re: Request for feedback on draft: DEP-18: Enable true open collaboration on all Debian packages)
Hi! While I intend to continue on iterating DEP-18, here is a summary to those who did not wade through the 140+ messages on the topic. Unfortunately, the summary itself is also a bit long :) Summary of mailing list discussion in https://lists.debian.org/debian-devel/2024/07/msg00429.html ## Overall Sentiment There was a broad consensus that Debian workflows are too fractured and would benefit from more standardization and unification. However, there were differing opinions on the right approach to achieve this. Soren Stoutner expressed this sentiment clearly: > 1. Debian workflows are too fractured. The project would be better if we > asked people to standardize around a single (or a small number) of workflows. > To do so, the workflow would need to be flexible enough to handle the wide > range of technical needs of all the packages and upstream configurations. > 2. Standardizing around a single (or small number of) workflows wil make some > people unhappy. But that is an acceptable price to pay because of the general > benefit to the project as long as the correct solution is adopted. Unity is > more important than minority opinions on this particular issue. Similarly, Andrey Rakhmatullin argued that while some may resign, "the '1000 DDs status quo' problem also means that more people leave than join _anyway_. Not the unity per se, but having significantly lower barriers to start contributing" is important. ## Git and Gitlab Usage Multiple participants noted that most other Linux distributions use Git as the primary version control system, often with GitLab or GitHub for collaboration. Debian's multi-branch approach with pristine-tar was seen as somewhat unique. There were differing views on whether Debian should move closer to the more common Git-based workflows used elsewhere, or maintain its own custom approach, which of git-buildpackage and dgit are the most common ones (both with multiple ways to use them). ## Mandatory vs Optional Policies Some participants, like Salvo Tomaselli, felt DEP-18 was too prescriptive in mandating specific tools and workflows, and that a more flexible, optional approach would be better: > Keep in mind that unhappy people quit. I don't think that unity is so > important that we're willing to sacrifice project members. Others, including Soren Stoutner, argued that standardization was important, even if it made some people initially unhappy, as long as the right solution was adopted: "Unity is more important than minority opinions on this particular issue." ## Maintainer Workflows There were concerns that requiring specific Git and Gitlab practices could create burdens for existing maintainers, especially single-person maintainers. Sean Whitton described his own preferences as a maintainer: > I am happy to use salsa for git hosting and access management. I love that I > can easily grant push access to my non-DD team members. But, I turn off salsa > MRs for the repos of all packages I regularly upload. I would hope that this > DEP can be written such as to account for these sorts of choices. Fabio > Fantoni suggested allowing maintainers to specify their preferred > collaboration methods in a machine-readable way, for example through a > "Collaboration-Policy" field in debian/control. ## Performance and Reliability Multiple participants, including Salvo Tomaselli, Johannes Schauer Marin Rodrigues, Andrea Pappacoda, and Gioele Barabucci, complained about Salsa/GitLab being slow or unreliable at times, which deterred contribution. Improvements to performance and uptime were seen as important. In response, Otto Kekäläinen noted that the Salsa admins had posted about upcoming hardware upgrades and other improvements to address these issues at https://salsa.debian.org/salsa/support/-/issues. ## Machine-Readable Metadata Fabio Fantoni and Niels Thykier proposed including more machine-readable metadata about packaging workflows (e.g. in debian/control) to help automate contributor onboarding. Niels Thykier outlined some specific examples of information that could be captured: > Does this package use `gbp dch` (or some other mechanisms) to generate the > changelog OR should I include a changelog entry with my patch. Does this > package use some form of automatic formatting that I should apply when I do > my changes (if `wrap-and-sort`, then which options)? Does the maintainer > prefer MR via salsa or BTS with patches for when I want to submit my changes > for review. ## Overall There seemed to be general agreement that improving collaboration was important, but the right approach was still being debated. ## Mailing list participants - Jonas Smedegaard - Salvo Tomaselli - Luca Boccassi - Charles Plessy - Marco d'Itri - Sean W
Re: Request for feedback on draft: DEP-18: Enable true open collaboration on all Debian packages
Hi! > NOTE: The following idea might be out-of-scope in DEP-18, but specific > use-case to improve > collaboration in Debian, IMHO. > > Here is just an idea: put collaboration policy metadata in debian/control. > (The following idea assumes that non-maintainer DD tend to hesitate to > commit/merge it) > > * Collaboration-Policy: debian/CONTRIBUTION.md > Yes, we have README.source already, but it might be better to note > in a separate file about collaboration. > * Collaboration-Policy-Commit: yes > It declares that the package owner encourages non-maintainer DD to > commit directly without merge request. > * Collaboration-Policy-Merge: yes > It declares that the package owner encourages non-maintainer DD to > allow merge requests. > (DD has maintainer right in salsa.d.o by default as you know, but > you can merge without worry if there is it.) > * Collaboration-Policy-LowThresholdNmu: yes > It declares that LowThresholdNmu rule [1] is applied. > * Collabollation-Policy-NMU-Delay: 15 > It declares that NMU delay the package owner wants. I agree that the CONTRIBUTING.md pattern is common on GitHub/GitLab, but we have already thousands of packages with debian/README.source (a couple also with README.source.md) as this file is documented in https://www.debian.org/doc/debian-policy/ch-source.html#source-package-handling-debian-readme-source. This should be to go-to location for any generic info on how to maintain the package or contribute to it. We also have some debian/source/* files as documented in https://www.debian.org/doc/manuals/maint-guide/dother.en.html#sourcel (and https://manpages.debian.org/unstable/dpkg-dev/dpkg-source.1.en.html#FILES) that instruct how patches should be managed in the sources. Perhaps there could be more metadata to define how patch submissions should be managed Nearly ten thousand packages also have a debian/gbp.conf file, which in a way is also a way to communicate (automatically) how to build the package and how to contribute to it correctly. I am tempted to put a gbp.conf and README.source template in DEP-18, but that would probably not be received well by those who prefer dgit, although dgit can be used together with git-buildpackage as well.
Re: Request for feedback on draft: DEP-18: Enable true open collaboration on all Debian packages
Hi Jonathan! The discussion was summarized in a separate "Summay" email by me on this list, and a comment in the MR (which merges the two discussions) and it just happened that the next day it was also covered in https://lwn.net/Articles/986480/ I am currently writing revision 2 of the proposal. I will notify when published. - Otto
Re: lintian.debian.org off ?
Hi! Thank you, Nicolas and Luis-Philippe, stepping up here is highly appreciated and will benefit a large amount of Debian packagers and their collaborators. > > Please be honest :) I don't mind it at all if you tell me: "yeah, that > > was only a proof of concep", or "I'm motivated now, but I don't know if > > I'll still be in 3 years". > > I definitely built it with maintenance in mind. I am willing to maintain > this code for as long as possible, but I cannot guarantee that I will be > able to do it forever! > I will be responsive if contacted by email or if an issue is created on > the GitHub repository. > > I also took the time to setup a CI with most of the code properly > tested. This should allow to minimize future breakages when modifying > the code. I am happy to help with writing a GitLab CI pipeline and do occasional code reviews, if you want to consider hosting the code at https://salsa.debian.org/lintian, Debian's official code hosting and collaboration platform. It can be instead of or in parallel with your GitHub repository. - Otto
Re: DEP-18 discussion summary (Re: Request for feedback on draft: DEP-18: Enable true open collaboration on all Debian packages)
Hi! > Hi Otto, > Until recently I generally found Salsa response to be adequate, > but for the last couple of days it has been so > excruciatingly slow as to be almost unusable. > > > In response, Otto Kekäläinen noted that the Salsa admins > > had posted about upcoming hardware upgrades and other improvements to > > address these issues at > > https://salsa.debian.org/salsa/support/-/issues. > > Which particular issue here relates to the planned hardware upgrade? My information was based on what Salsa admin posted at https://salsa.debian.org/salsa/support/-/issues/395 I have also witnessed Salsa being very slow in the past couple of days, but I am not aware of what is going on. Hopefully it gets fixed soon.
Re: lintian.debian.org off ?
Hi! Thanks both Lucas and Nicolas for your contributions to Debian. Can we agree to have both UDD and lintian.debian.org, as the work to develop the required systems already happened? I think both websites have their benefits. Having a lintian.debian.org site with links to man pages and additional information caters well to the newbie packager or occasional package collaborator. The lintian.debian.org page can link to the equivalent UDD page, and the UDD page and continue to list all affected packages. Debian should be a place open for collaboration, and we should welcome contributions, in particular in this case as it is a legit and working implementation to produce renewed contents for lintian.debian.org. - Otto
DpartialMirror (was Re: [ANNOUNCE] debian-multimirror)
> On Wed, 29 Sep 2004, Pedro Larroy wrote: > > As a long standing debmirror user and with the knowledge that there is a > debpartial-mirror project which is very actively developed I just wonder > if people have to much spare time to invent one wheel after an other. > Sorry for the late reply but in case you mean with deppartial-mirror project my "DpartialMirror" project, I've to say that I set its state to mature. While I still use it daily I don't actively develop it further since as long as Debian doesn't use gzip --rsyncable, it doesn't make much sense. Without it the gain just a few percents. See "http://dpartialmirror.sourceforge.net/";. O. Wyss -- See a huge pile of work at "http://wyodesktop.sourceforge.net/";
Synching mirrors and clients (was: Re: apt-proxy v2 and rsync)
> > Can anyone explain why rsync is no longer considered an appropriate
> > method for fetching Packages files?
>
> IIRC the problem is that rsync is quite CPU-heavy on the servers, so while
> the mirrors have the (network) resources to feed downloads to 100s of
> users, they don't have the (CPU) resources for a few dozen rsyncs.
>
Why do you keep on saying this without providing _any_ figures!
Why always synching the full mirror when only about 1% of the files
changes daily? Just change to single file synching and most of your CPU
load is gone. Single file rsync doesn't need any CPU power to discover
the changed files. Single file rsync touches only the changed files,
only about 1%, so at least disk access is much less while probably also
lowers the CPU load.
If gzip --rsyncable would be used the CPU load would dramatically be
lowered, much lower than with _any_ other synching. As a side effect the
use bandwidth would be equally well be lowered. IMO rsync is very useful
if don't right.
Prove of concept
To finally produce some figures and prove this concept two servers are
needed, the first one an ordinary source mirror, the second a secondary
mirror with different mirror directories for each of the test cases. On
the first server the CPU load is measured, on the second the different
sync scripts are run:
- Ordniary full mirroring rsynch as today in use
- DpartialMirror sync script ("http://dpartialmirror.sourceforge.net/";)
- Deb-mirror sync script
- ???
- Sync with wget, etc.
IMO this will show which is the best solution for full mirrors.
Now limit the secondary mirror to support only one architecture and do
the test above again. This will show the best solution for the commonly
used mirrors.
In a third step limit the packages to what an ordinary user has, just
use popularity-contest or I could provide my dpkg --getselections. This
will show the best solution for servers from clients impact.
Now if you feel advantous, repack as many package on the source mirror
with gzip --rsyncable and notice the difference.
O. Wyss
--
See a huge pile of work at "http://wyodesktop.sourceforge.net/";
Re: Synching mirrors and clients (was: Re: apt-proxy v2 and rsync)
> > > IIRC the problem is that rsync is quite CPU-heavy on the servers, so while > > > the mirrors have the (network) resources to feed downloads to 100s of > > > users, they don't have the (CPU) resources for a few dozen rsyncs. > > > > Why do you keep on saying this without providing _any_ figures! > > Who is "you" here? Please pay attention to attribution on mailing list > postings - especially if you're starting a new thread with your mail. I > posted this statement about cpu load of rsync, and did that exactly once, > so I don't "keep saying it". Also, I put in an IIRC, so you have clear > indication that I'm not too sure - somebody asked about the reason, I > answered with that I heard was the reason when the last person asked. > I don't meant you personally but this statement about the CPU load, mostly accompanied with some vage numbers is repeated all the time and whenever I ask for exact figures only excuses or even not an aswer is provided. Your statement about "feed downloads to 100s ...(CPU) resources for a few dozen rsyncs" implied you have actually seen such CPU loads. Now you say you just repeated from hear say. How much value did your message have to the OP? Does it have any value? Well the main problem is not your message but that nobody is able or willing to set up a reasonable test case and provide accurate figures. Maybe this is a non issue because Debian has more than enough systems and don't has to care about. O. Wyss -- See a huge pile of work at "http://wyodesktop.sourceforge.net/";
Re: New stable version after Sarge
Tim Cutts <[EMAIL PROTECTED]> wrote: > > Seriously. There's just no way you're going to change the way Debian > > makes releases, or rather, doesn't. It's too big, and there are just > > too damn many people involved, many of whom simply don't care about > > releases. As long as we maintain our current release criteria (which I > > don't necessarily think we should change) we will get slower and slower > > as we get bigger and bigger. > Which is a rather clear sign that the way Debian makes releases has outgrown its usefulness. > Which could be seen as a problem by some; but in some ways it's > probably the way to go. As far as my own use of Debian goes, almost > every machine I install runs testing, and has done for years. There's > a level of protection in there thanks to the rules that are in place, > and I rather like the incremental improvement approach as opposed to > release-based. > Me too, but it might be completely different if you do it for business critical systems. > With the trend as it is at the moment, the endpoint is that Debian will > eventually stop releasing altogether (some end users probably think > this has already happened!) and will essentially become an upstream, > developer-oriented, steadily evolving distribution from which the likes > of Ubuntu take regular snapshots for the masses to use. > Stopping releasing might be a good idea but there should be a better way. IMO the problem is the stable release isn't updated on a regulare basis. It might be a better idea to divide Debian into subsystems which could be released much faster when needed. O. Wyss -- Development of frame buffer drivers: http://linux-fbdev.sf.net Sample code snippets for wxWidgets: http://wxcode.sf.net How to build well-designed applications: http://wxguide.sf.net Desktop with a consistent look and feel: http://wyodesktop.sf.net
Re: Debian mirror scripts
Goswin von Brederlow <[EMAIL PROTECTED]> wrote: > > Sure? Anyway DpartialMirror "http://dpartialmirror.sourceforge.net/"; > > can. > > > A note of caution: > > | 2004-04-03 (wyo) Since Debian does not change its policy to add > | adequate support for rsync'ing package mirrors, I don't actively > | develop DpartialMirror further. > :-( Sad, isn't it? > Any user of dpartialmirror should check out mirrorer from alioth. I > only glanced at the webpage and haven't used dpartialmirror but now > that mirrorer has filtering support it looks like mirrorer could > replace dpartialmirror completly and I'm also thinking about retireing > debmirror in favour of a wrapper to mirrorer. > I guess mirrorer doesn't care for bandwith saving as DpartialMirror, correct me if I'm wrong. O. Wyss -- Development of frame buffer drivers: http://linux-fbdev.sf.net Sample code snippets for wxWidgets: http://wxcode.sf.net How to build well-designed applications: http://wxguide.sf.net Desktop with a consistent look and feel: http://wyodesktop.sf.net -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Debian mirror scripts
Goswin von Brederlow <[EMAIL PROTECTED]> wrote: > [EMAIL PROTECTED] (Otto Wyss) writes: > > > Goswin von Brederlow <[EMAIL PROTECTED]> wrote: > > > >> > Sure? Anyway DpartialMirror "http://dpartialmirror.sourceforge.net/"; > >> > can. > >> > > > I guess mirrorer doesn't care for bandwith saving as DpartialMirror, > > correct me if I'm wrong. > > Currently it will always redownload the Packages/Sources files as gzip > on every update to fix a bug in the apt methods. But I already > suggested only updating those that don't match the Release file. And, > unless you have an rsync method for apt, it won't rsync files. > Why there isn't there already a rsync method for apt is probably a mystery nobody ever will solve. > While rsyncing the Packages files sounds like a good idea to save > traffic it actualy is a bit insignificant compared to the daily > traffic of new sources and debs. > Do you mean there are up to 100 new packages each day? I get between 50 - 150 packages updated each day for just i386. Or do you mean there are 100 new versions? DpartialMirror handles new versions of packages (sources and deps) in a way it save about 1% even when the packages are normal gzip'ed. It would save around 10% - 50% with rsyncable. Is there any plan to add this feature to mirrored? O. Wyss -- Development of frame buffer drivers: http://linux-fbdev.sf.net Sample code snippets for wxWidgets: http://wxcode.sf.net How to build well-designed applications: http://wxguide.sf.net Desktop with a consistent look and feel: http://wyodesktop.sf.net -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Debian mirror scripts
Goswin von Brederlow wrote: > > > Why there isn't there already a rsync method for apt is probably a > > mystery nobody ever will solve. > > It is not wanted due to rsync causing excessive server load. > That is simply not true. This statement is repeated all the time but nobody ever was able to show hard figures. Where rsync produces much load is during the phase when it collects all the files for synchronisation and not during MD5 computation but this is is only due to not well designed scripts. DpartialMirror doesn't impose this phase since it only require single-file transfers and does the file collecting phase on the client. > New versions. The size of the Packages files is comparatively tiny > compared to all the debs. Even the 1% saving for rsyncing debs is > hardly worth it due to the extra traffic for the checksums and the > server load it causes. > Sorry rsync reports the overall use, incl. checksums etc. Of course 1% saving doesn't make much sense so that's the main reason I don't develop DpartialMirror further. Anyway the next time a distribution concept is designed it will be based on a p2p solution. > zsync has the option of looking into gziped files and rsync them as if > they would be ungziped (while still just downloading chunks of the > gziped file). Its a bit more complex algorithm but works even better > than rsyncable files and rsync. > As long as zsync allows multi-file transfers it won't be better that rsync. O. Wyss -- Development of frame buffer drivers: http://linux-fbdev.sf.net Sample code snippets for wxWidgets: http://wxcode.sf.net How to build well-designed applications: http://wxguide.sf.net Desktop with a consistent look and feel: http://wyodesktop.sf.net -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Debian mirror scripts
Goswin von Brederlow <[EMAIL PROTECTED]> wrote: > Otto Wyss <[EMAIL PROTECTED]> writes: > > reply (that is what I get roughly) to the server would waste 75 hours > on waiting for the initial three-way handshake for a connect. And > another 50 hours for the round-robin sending the name of a file and > getting the data. > Did you messure this figures on a real Debian mirror or is it just what you think? > can use simple http/1.1. It's a win-win situation. > Perfect go ahead. Even if DpartialMirror isn't developed further I use it almost daily and are quite happy with it. And I guess this won't change in the future. O. Wyss -- Development of frame buffer drivers: http://linux-fbdev.sf.net Sample code snippets for wxWidgets: http://wxcode.sf.net How to build well-designed applications: http://wxguide.sf.net Desktop with a consistent look and feel: http://wyodesktop.sf.net -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Restoring lost keymap
Sorry if I ask here but nobody in debia-users seems to know an answer. During installation of console-tools a few weeks ago I lost the keymap of my swiss-german keyboard in the console. As typical console-tools doesn't have a man page and it's documentation is useless. Does anybody know how to restore (load) the correct keymap? The locales seems to be correct "DE_ch". O. Wyss -- Development of frame buffer drivers: http://linux-fbdev.sf.net Sample code snippets for wxWidgets: http://wxcode.sf.net How to build well-designed applications: http://wxguide.sf.net Desktop with a consistent look and feel: http://wyodesktop.sf.net -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Debian mirror scripts
> Tried to work around this with a simple script that merges my > packages into the local mirror and regenerates everything as > needed. But sadly this doesn't seem to be perfect :-( The installer > just doesn't want to get some of these packages, even if the md5's > are correct. Switching from http to ftp gets some more of them and > stucks some packages later. Grrr. > Look at DpartialMirror "http://dpartialmirror.sourceforge.net/";. I use it regularly to build a second mirror. O. Wyss -- Development of frame buffer drivers: http://linux-fbdev.sf.net Sample code snippets for wxWidgets: http://wxcode.sf.net How to build well-designed applications: http://wxguide.sf.net Desktop with a consistent look and feel: http://wyodesktop.sf.net -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Debian mirror scripts
Goswin von Brederlow <[EMAIL PROTECTED]> wrote: > Debmirror is purely a mirror tool. It will download the Meta files > just like any other file. > > You can easily switch between mirror of equal contents but not create > Packages files reflecting what is locally available. > Sure? Anyway DpartialMirror "http://dpartialmirror.sourceforge.net/"; can. O. Wyss -- Development of frame buffer drivers: http://linux-fbdev.sf.net Sample code snippets for wxWidgets: http://wxcode.sf.net How to build well-designed applications: http://wxguide.sf.net Desktop with a consistent look and feel: http://wyodesktop.sf.net -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Package distribution, a concept for a modern package distribution
Since around last October, I've considered to make my concept for a modern package distribution public but I wanted to wait until Debian/sarge was released which is now the case. And since the Debconf5 in Helsinki is just around the corner it's about the right time. The concept is based on an LDAP server (or simiar) as a replacement for the Packages file and on a P2P network for package distribution (see http://wyodesktop.sourceforge.net/index.php?page=pkgdist.html). IMO it would make a lot sense if this concept is discussed at the Debconf5. I'm not actively work on this concept and its implementation since I've _no_ time, sorry. If someone else is interested just say so. O. Wyss -- Development of frame buffer drivers: http://linux-fbdev.sf.net Sample code snippets for wxWidgets: http://wxcode.sf.net How to build well-designed applications: http://wyoguide.sf.net Desktop with a consistent look and feel: http://wyodesktop.sf.net -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Package distribution, a concept for a modern package distribution
> On Fri, Jun 24, 2005 at 06:14:46PM +0200, Otto Wyss wrote: > > The concept is based on an LDAP server (or simiar) as a replacement for > > the Packages file and on a P2P network for package distribution (see > > http://wyodesktop.sourceforge.net/index.php?page=pkgdist.html). IMO it > > would make a lot sense if this concept is discussed at the Debconf5. > > i actually wrote something that exported a local mirror/server's Packages.gz > file into an LDAP directory[1], as well as wrote the beginnings of an > add-on method to apt to query this server. as far as speed/transfer > efficiency goes, we're an order or two of magnitude at the very least. > however, there hasn't seemed to be much interest from others in my > continuing it, so i've been focusing on other things lately. > I've seen some messages about an LDAP implementation around last october but I couldn't find them. I'm quite sure an LDAP solution is much better than the current solution. But before implementing it, it has to be evaluated against other way so it's truely the best. > also, there's a limitation in apt that it expects the list of > packages to be retrieved through the same method as the packages > themselves, which would get a little hairy with LDAP (you don't want to > be holding the packages themselves, in LDAP of course). there could > be a quick-hack workaround for this by having ldap-ftp/ldap-http methods > that wrap around the ftp/http for the actual fetching, but a real fix > would be to patch apt to allow for this. such a patch would also make > it easier to distribute the packages list via other methodst too. > I wouldn't base any work on apt but start a complete new way of package distribution. The advantage is the stabe apt will keep on working while the new solution won't be hampered by any current limitation. IMO a P2P network would be a much better solution but yet again it has to be checked if it this is really true or if there are better alternatives. > anyway if there are more people interested in working on this, i'd be > willing to put my code in cvs/svn and start up an alioth project. > The best way to start a new package distribution is to name a place where it can be discussed off Debian-devel. Since this concept probably has many more implications, like what about Debian installer etc, I think it would make a lot of sense to first collect any pros and cons and discuss them at Debconf5 and possibly on a list. Since I can't attend Debconf5 myself I'd appreciate if someone could keep track of the discussion there and forward it to the list. So the first action should be to set up or name this package distribution list and start collecting arguments there. Besides, until a better place is found I'll keep "http://wyodesktop.sourceforge.net/index.php?page=pkgdist.html"; updated, so you may send any input to the wyodesktop-users list or directly to me (list would be better because less spam prone). O. Wyss -- Development of frame buffer drivers: http://linux-fbdev.sf.net Sample code snippets for wxWidgets: http://wxcode.sf.net How to build well-designed applications: http://wyoguide.sf.net Desktop with a consistent look and feel: http://wyodesktop.sf.net -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
/sbin/halt always changes its access rights
I've set the s attrtibute of halt since on my desktop any user may stop the system. But about each second month or so it's set back to it's original rights probably by a package upgrade. Is there a way to keep the access rights or any better way to handle these kind of problems. O. Wyss -- Development of frame buffer drivers: http://linux-fbdev.sf.net Sample code snippets for wxWidgets: http://wxcode.sf.net How to build well-designed applications: http://wxguide.sf.net Desktop with a consistent look and feel: http://wyodesktop.sf.net -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Identical packages have different size/md5sum in stable/unstable (i.e. perspic-texts)
Sorry, I don't know if this is the right group for this kind of message. I've discovered that serveral identical packages in stable and unstable does have different size/md5sum listed in the corresponding "Packages.gz" files. I.e. "perspic-texts_1.4.6.deb" in "binary-all/misc" on a debian mirror are separate files, have the same size (14475Kb) but different timestamps (stable Apr 3/unstable Feb 5). In the corresponding "Packages.gz" only "stable" matches (see below). Package.gz version "stable": - Package: perspic-texts Version: 1.4-6 [...] Filename: dists/stable/main/binary-powerpc/misc/perspic-texts_1.4-6.deb Size: 14823392 MD5sum: f623ab81724bc88f3614c2c9cef769e3 [...] Package.gz version "unstable" --- Package: perspic-texts Version: 1.4-6 [...] Filename: dists/unstable/main/binary-powerpc/misc/perspic-texts_1.4-6.deb Size: 14823398 MD5sum: 2c4aca0088904122a964477012259f30 [...] The following packages have the same problem: main/binary-all/devel/scalapack-test-common_1.6-13.deb size mismatch main/binary-all/misc/perspic-texts_1.4-6.deb size mismatch main/binary-i386/graphics/terraform_0.5.2-1.deb size mismatch main/binary-i386/misc/perspic_1.4-6.deb size mismatch main/binary-powerpc/admin/psmisc_19-2.deb size mismatch O. Wyss
RFC: Changing the Packages files
With the introduction of the packages pool, I'm going to propose the following change to the Packages files: 1. The filename tells what the Packages files contains: Packages files should be independent of the their location, therefor the name has to reflect their contents, i.e. "Packages-$DIST-$ARCH" or simply "$TYPE-$DIST-$ARCH" where $TYPE="Main"|"Contrib"|"NonUS"|"Nonfree" 2. The location of the Packages file does define the base of the packages mirror structur: This means the "Filename:" attribut starts from the location of the Packages file, allowing for a much flexibler naming of the structur. These two changes means the Packages files for all distributions could be moved inside the pool as well. Also it is possible to have different kind of mirror structurs (i.e alphabetical versus functional) since the Packages files could be anywhere. And symlinks from binary-$ARCH to binary-all weren't nessecary. O. Wyss
Solving the compression dilema when rsync-ing Debian versions
It's commonly agreed that compression does prevent rsync from profit of older versions of packages when synchronizing Debian mirrors. All the discussion about fixing rsync to solve this, even trough a deb-plugin is IMHO not the right way. Rsync's task is to synchronize files without knowing what's inside. So why not solve the compression problem at the root? Why not try to change the compression in a way so it does produce a compressed result with the same (or similar) difference rate as the source? As my understanding of compression goes, all have a kind of lookup table at the beginning where all compression codes where declared. Each time this table is created new, each time slightly different than the previous one depending on the source. So to get similar results when compressing means using the same or at least an aquivalent lookup table. If it would be possible to feed the lookup table of the previous compressed file to the new compression process, an equal or at least similar compression could be achieved. Of course using allways the same lookup table means a deceasing of the compression rate. If there is an algorithmus which compares the old rate with an optimal rate, even this could be solved. This means a completly different compression from time to time. All depends how easy an aquivalent lookup table could be created without loosing to much of the compression rate. O. Wyss
Re: Solving the compression dilema when rsync-ing Debian versions
>> So why not solve the compression problem at the root? Why not try to >> change the compression in a way so it does produce a compressed result >> with the same (or similar) difference rate as the source? > >Are you going to hack at *every* different kind of file format that you >might ever want to rsync, to make it rsync friendly? > No, I want rsync not even to be mentioned. All I want is something similar to gzip --compress-like=old-foo foo where foo will be compressed as old-foo was or as aquivalent as possible. Gzip does not need to know anything about foo except how it was compressed. The switch "--compress-like" could be added to any compression algorithmus (bzip?) as long as it's easy to retrieve the compression scheme. Besides the following is completly legal but probably not very sensible gzip --compress-like=foo bar where bar will be compressed as foo even if they might be totally unrelated. Rsync-ing Debian packages will certainly take advantage of this solution but the solution itself is 100% pure compression specific. Anything which needs identical compression could profit from this switch. It's up to profiting application to provide the necessary wrapper around. >gzip --rsyncable, aloready implemented, ask Rusty Russell. The --rsyncable switch might yield the same result (I haven't checked it sofar) but will need some internal knowledge how to determine the old compression. As I read my mail again the syntax for "compressing like" could be gzip --compress=foo bar where bar is compressed as foo was. Foo is of course a compressed file (how else could the compression be retrieved) while bar is not. O. Wyss
Re: Solving the compression dilema when rsync-ing Debian versions
> > gzip --compress-like=old-foo foo > > > > where foo will be compressed as old-foo was or as aquivalent as > > possible. Gzip does not need to know anything about foo except how it > > was compressed. The switch "--compress-like" could be added to any > > compression algorithmus (bzip?) as long as it's easy to retrieve the > > No, this won't work with very many compression algorithms. Most > algorithms update their dictionaries/probability tables dynamically based > on input. There isn't just one static table that could be used for > another file, since the table is automatically updated after every (or > near every) transmitted or decoded symbol. Further, the algorithms start > with blank tables on both ends (compression and decompression), the > algorithm doesn't transmit the tables (which can be quite large for higher > order statistical models). > Well the table is perfectly static when the compression ends. Even if the table isn't transmitted itself, its information is contained in the compressed file, otherwise the file couldn't be decompressed either. O. Wyss
Re: Solving the compression dilema when rsync-ing Debian versions
> > gzip --compress-like=old-foo foo > > AFAIK thats NOT possible with gzip. Same with bzip2. > Why not. > I wish it where that simple. > I'm not saying it's simple, I'm saying it's possible. I'm not a compression speciallist but from the theory there is nothing which prevents this except from the actual implementation. Maybe it's time to design a compression alogrithmus which has this functionality (same difference rate as the source) from the ground up. O. Wyss
Re: Solving the compression dilema when rsync-ing Debian versions
>>> > gzip --compress-like=old-foo foo > >gzip creates a dictionary (that gets realy large) of strings that are >used and encodes references to them. At the start the dictionary is >empty, so the first char is pretty much unencoded and inserted into >the dictionary. The next char is encoded using the first one and so >on. That way longer and longer strings enter the dictionary. > ... > >So, as you see, that method is not possible. > Okay lets asume gzip knows anyhow the table of the previous compression. It starts compressing as usual, getting the first value to encode. Now instead of just putting it at the first position it looks in the old table and finds it at position 3. Gzip puts it at position 3 leaving the first 2 unused. Now everthing goes fine until a value isn't found. This time gzip appends it to the end of the table. Of course at a certain point these to table diverge to much so gzip starts using a new table. I don't know the outcome of such a compression but I think it will much better correspond to the sources. Besides this algorithmus could be used as gzip --compress=i386.gz where i386 does contain a optimal table for i386 binaries. It will give a better start compression rate while retaining an adaptable compression and it allows to specify th optimal compression for any case. I don't think mine is the best solution and I don't know if its working, it just gives an idea how the problem could be solved. The principle is to use the old compression scheme and adapt it as less as possible but as much as necessary. >But there is a little optimisation that can be used (and is used by >the --rsyncable patch): > This is of course a very good solution, I only wouldn't call it --rsyncable. I wouldn't make it an option at all. Anyhow it's not the NonPlusUltra solution, there are cases where it will fail. > > Maybe it's time to design a compression alogrithmus which has > > this functionality (same difference rate as the source) from > > the ground up. > >There are such algorithms, but they eigther allys use the same >dictionary or table (like some i386.exe runtime compressors that are >specialiesed to the patterns used in opcodes) or they waste space by >adding the dictionary/table to the compressed file. Thats a huge waste >with all the small diff files we have. > These all have fixed compression, as far as I know there isn't any which combines a fixed with an adaptable compression. O. Wyss
Re: A success story with apt and rsync
> From time to time the question arises on different forums whether it is > possible to efficiently use rsync with apt-get. Recently there has been a > thread here on debian-devel and it was also mentioned in Debian Weekly News > June 24th, 2003. However, I only saw different small parts of a huge and > complex problem set discussed at different places, I haven't find an > overview of the whole situation anywhere. > Sorry that I write so late but I'm not reading debian-devel regularly. I've started a solution to distribute Debian mirrors by rsync about 2 years ago. The only "impact" (if impact is the right word) of my soultion on Debian is the use of the rsync patch for gzip. Everything else is solve by my perl script so you might find ideas for your apt solution there. See "http://dpartialmirror.sourceforge.net/";. O. Wyss -- See "http://wxguide.sourceforge.net/"; for ideas how to design your app.
GCC for kernel compilation
I just upgraded to the current Sarge and also got GCC 3.3. It seems this version can't compile all the drivers in kernel 2.4.21. Which version should I use? And how do I set this version (Environment variable?) without deinstalling GCC 3.3? O. Wyss -- See "http://wxguide.sourceforge.net/"; for ideas how to design your app.
Building kernel with framebuffer support (was: Re: GCC for kernel compilation)
I know I'm getting off topic but I don't know a better place to ask and this subject might be interesting of other developers as well. > On Fri, 8 Aug 2003 04:14, Otto Wyss wrote: > > I just upgraded to the current Sarge and also got GCC 3.3. It seems this > > version can't compile all the drivers in kernel 2.4.21. Which version > > Which drivers and what errors do you get? If you tell us the errors then we > can get them fixed. I get lots of warning (i.e. variable fb isn't used in aty182 driver). I don't get these warnings when I use CC=gcc-2.95. It's a little bit difficult to trace warnings and errors during a kernel compilation. I'd be good if warnings and error where duplicated into a log file. I'm trying to build a kernel with framebuffer support for the Matrox G400 card but I always get "open /dev/fb0: No such device" when I use fbset. A week ago I added a new 123GB harddisk to my system, installed a new Debian 3.0r1 and upgraded to the current Sarge. But since then I can't use framebuffer anymore while with the old system I used it for about 2 years. For building I use the same settings (make menuconfig) as before, so it should work unless there are any hidden settings in 2.4.21. The command: ls -la /dev/fb0 crw--w 1 root video 29,0 Mar 14 2002 /dev/fb0 seems to be correct as much as I know. I've tried to config each setting new, I also installed the kernel-image-2.4.21-3-k6 but nothing helps. What can I do to fix this situation? O. Wyss -- See "http://wxguide.sourceforge.net/"; for ideas how to design your app.
Bug#54138: Immense, gains, in germany.
kicked -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Bug#25837: It's Natalia. Hi It's Natalia. We can be friend.
Hello My name is Natalia. how are you? I find your profile and e-mail on a site of acquaintances. I want to find the more friend and my love. If you is real are interested, answer to me and we can begin our acquaintance. A little about me. I was born 30 DEC 1979. I work as the manager in the insurance company. I want to find best friend and the man who can love me whom I will also ready to love and care. And i believe, i can have all part of what you want in soulmate, out of thousands of people that is on here, i find you to be my true choice and i hope that you should feel the same way too. It's m really a wonderful moment as am writing this letter to you and i pray that i should hear a good and sweat reply from you. You may be in long distance from me, but i believe that love can do everything. I believe love can move mountain and love can turn people's life to wonderful life and sweet one. Ok, i wish that you should write me in e-mail and lets talk and get to know more about each other. My new friend I ask you to write to me on this e-mail: [EMAIL PROTECTED] because the Internet here is very bad, but on e-mail I can check my mail easily. I will be great to read a nice letter from you. Hoping in God of love and in power of love I hope to hear from you. Thanks for the reading my letter. Natalia. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Package libc6-dev depends on linux-kernel-headers
Sorry this message go to the poster instead of the list. > > > There have always been some kernel headers in libc6-dev, they've just > > > been split out into a separate package now. Several of these headers > > > are referenced by headers provided by glibc which would break those > > > headers if linux-kernel-headers is not installed. > > > > I'd prefer the old way. > > And can you give a substantive reason? Without one your message makes > no sense. I didn't give a reason because it wouldn't change anything. I always download the kernel sources myself and build my kernel from scratch. I therefore don't want do download and install the header packages as well. Besides which version of headers does libc6 use/need? You may ask why I don't use any Debian kernel. Mostly because I need the framebuffer early on and also use an USB keyboard on my desktop. Most of the rest are just modules. O. Wyss -- See "http://wxguide.sourceforge.net/"; for ideas how to design your app.
Re: rename linux-kernel-headers to system-headers
> On Fri, Nov 07, 2003 at 10:45:32AM +, Jonathan Dowland wrote: > > On Thu, Nov 06, 2003 at 07:55:03PM +0100, Eduard Bloch wrote: > > > > > What not rename linux-kernel-headers to simple system-headers-linux? > > > This will prevent confused users (or: lazy to read the description users) > > > from asking this again and again. > > > > system-headers-linux is a bit vague and without knowing could be > > associated with the kernel just as strongly as with libc. > > > > How about libc-linux-headers? > > I second that, or perhaps libc6-linux-headers. If the package would have been named "libc6-linux-headers" to show its strong relationship with libc6 I had never started this thread. I'm not a fan of renaming but in this case IMO it seems to be appropriate. O. Wyss -- See "http://wxguide.sourceforge.net/"; for ideas how to design your app.
Library packages and their use
The discussion about the libc6-dev package and its headers let me to the impression that the Debian package structure isn't optimal for libraries. If anyone wants to build his own version of a package (i.e. libwxgtk2.4) he has to get all the dependent underlying dev packages as well. This is a long line of dependencies which mostly aren't wished and shouldn't be necessary. The problem arises because the 2 usual package lines (normal and dev packages) don't fit well with the needs of the users. There are 3 kinds of dissimilar user groups of a package: 1. Users just using a library linked to other packages 2. Developers building packages which depends on a library package 3. Developers building his own version of this library package Currently group 1 just uses the "normal" packages while group 2 + 3 have both to use the "dev" packages. Especially for group 2 this isn't a good solution leading to a long line of unnecessary dependencies. Solution 1: Splitting the 2 packages into 3. Not very attractive, it will more confuse than improve the situation. Maybe the dbg packages could take over the role of the 3. group. Solution 2: Packages are changed that group 1 + 2 can use the normal packages and only group 3 uses the dev. That means the normal library packages contain enough so that other packages depending on this can be build. Solution 3: Normal packages are for group 1, dev packages are for group 2 and group 3 has to get anything needed by other means (i.e. CVS). Usually group 3 is rather small and they tend to get anything by CVS anyway. I'm not sure if any of the above solutions will improve the situation but we should all try to remove dependencies wherever possible. And I'm not sure if any library package can be split this way but it should be tried. O. Wyss -- See "http://wxguide.sourceforge.net/"; for ideas how to design your app.
Sarge-i386-1.iso via jigdo-lite
Currently there seems to be a problem with the Sarge-i386-1.jigdo file. I tried to build/download a new CD but it complains 57 files where missing. Even getting the .jigdo/.template files again or choosing another mirror didn't help. The script can only be stopped so I don't know how to proceed further. O. Wyss -- See "http://wxguide.sourceforge.net/"; for ideas how to design your app.
Re: window manager recomendation
> Hello, > > Hoping this won't turn into a flame war, I am looking for > recommendations for a window manager. I tried quiet a few but none seem > to fit the bill yet. > I don't know if XFCE fits all your requirements but since it's not only a window manager but a light weight desktop I like it. The application menu is a little bit chaotic but usable. O. Wyss -- See "http://wxguide.sourceforge.net/"; for ideas how to design your app.
Installing kernel-image-2.4.22
I tried to upgrade the 2.2 kernel from Woody to 2.4.22 and installed kernel-image-2.4.22. During installation a large text but barely interpretable text about initrd.img is shown. Why can't the install make a fully correct lilo.conf by itself? Besides the text is wrong instead of "initrd=initrd.img" it should be "initrd=/initrd.img". So far so good but after installation I don't have network access anymore, probably because 2.4.22 has the network driver as a module. Instead of this sermone about initrd, it'd be better if this fact were mentioned in the installation. A hint to use modconf after installation would be very helpful. The question remains why kernel-image packages don't require the modconf packages. It seems to me obvious that modconf has to be run after a kernel upgrade. How do I install modconf without a network connect? Okay just boot into the old kernel with network support. Well the kernel-image package adds the second entry to lilo.conf but forgets to uncomment the prompt and timeout parameteres. I don't know what would happened if the new kernel wouldn't run. O. Wyss -- See "http://wxguide.sourceforge.net/"; for ideas how to design your app
Howto reconfigure alsa-modules-2.4.22-1-k6
I've installed alsa-modules-2.4.22-1-k6 but made a mistake when selecting the driver. So I tried dpkg-reconfigure alsa-modules-2.4.22-1-k6 but this doesn't show the driver list again! Okay getting dselect out, purge the package and install it again. But now the list isn't shown either. How do I get the driver list from this package? O. Wyss -- See "http://wxguide.sourceforge.net/"; for ideas how to design your app.
Re: Howto reconfigure alsa-modules-2.4.22-1-k6
> Otto Wyss dijo: > > dpkg-reconfigure alsa-modules-2.4.22-1-k6 > > > > but this doesn't show the driver list again! Okay getting dselect out, > > purge the package and install it again. But now the list isn't shown > > either. How do I get the driver list from this package? > > > dpkg-reconfigure alsa-base > > Anyway, this message would have fitted better in debian-user > First it was an oversight, sorry. But now I think the alsa packages are more broken. After successfully using dpkg-reconfigure alsa-base I got the following error messages: depmod: *** Unresolved symbols in /lib/modules/2.4.22-1-k6/alsa/snd-pdaudiocf.o depmod: *** Unresolved symbols in /lib/modules/2.4.22-1-k6/alsa/snd-vx-cs.o depmod: *** Unresolved symbols in /lib/modules/2.4.22-1-k6/alsa/snd-vxp440.o depmod: *** Unresolved symbols in /lib/modules/2.4.22-1-k6/alsa/snd-vxpocket.o I get the same messages when using update-modules. My alsa configuration looks good: ### DEBCONF MAGIC # This file was automatically generated by alsa-base's debconf stuff alias char-major-116 snd alias char-major-14 soundcore options snd major=116 cards_limit=4 alias sound-service-0-0 snd-mixer-oss alias sound-service-0-1 snd-seq-oss alias sound-service-0-3 snd-pcm-oss alias sound-service-0-8 snd-seq-oss alias sound-service-0-12 snd-pcm-oss alias snd-card-0 snd-ice1724 alias snd-slot-0 snd-card-0 alias sound-slot-0 snd-slot-0 and lspci shows 00:0b.0 Multimedia audio controller: IC Ensemble Inc ICE1724 [Envy24HT] (rev 01) So what's wrong now? O. Wyss -- See "http://wxguide.sourceforge.net/"; for ideas how to design your app.
Re: Howto reconfigure alsa-modules-2.4.22-1-k6
> > depmod: *** Unresolved symbols in > > /lib/modules/2.4.22-1-k6/alsa/snd-pdaudiocf.o > > depmod: *** Unresolved symbols in > > /lib/modules/2.4.22-1-k6/alsa/snd-vx-cs.o > > depmod: *** Unresolved symbols in > > /lib/modules/2.4.22-1-k6/alsa/snd-vxp440.o > > depmod: *** Unresolved symbols in > > /lib/modules/2.4.22-1-k6/alsa/snd-vxpocket.o > > At this point I would just remove those files. They are not needed by > your machine. Unless those are the devices you are running. But > seriously just remove them and re-run update-modules. There are some > funky things I have always (nearly always) seen with those modules. Even > in the OSS setup they are difficult to get properly compiled. > I've found out, it can be fixed by installing kernel-pcmcia package, see "http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=213887";. Installation is now okay but I get the following error ../../alsa-kernel/pci/ice1712/ice1724.c:1614: invalid EEPROM (size=120) I guess I have to ask about this in another newsgroup but where? O. Wyss -- See "http://wxguide.sourceforge.net/"; for ideas how to design your app.
Re: Porting Xconfigurator to Debian!
> Install discover, read-edid and mdetect before you install X, and you're > set. > mdetect hopefully doesn't choke on an USB-mouse anymore! O. Wyss
Re: Referring what kernel-images to build to the technical committee?
>In fact, most of the options could be auto-detected from >/proc/cpuinfo. > >It could also be useful as a hardware tester at install time: >"Would you like to test your hardware (and get a kernel custom >build for your hardware at the same time)? This process will >potentially take a long time." (Yes, I realize this idea is >a bit crazier than average.) > Even if your idea sounds a little bit crazy, this is probably the best way at the moment to get a well suited kernel. But I'd rather see a kernel where all options were compiled into separate modules so simply choosing the right modules constructs the optimal kernel. This could be done during an install or setup process. Sound crazy? Maybe now but how about in the future? O. Wyss -- Please CC.
Debian non-free mirror? Where are they?
I considered to include Debian non-free into my synchronisation scripts (see "http://dpartialmirror.sourceforge.net/";) but I couldn't find any mirror nor any information about. Where is non-free located? O. Wyss
Description to man pages
Since I hated to start dselect again and again just to read a package
description I wrote a script "dsc2man" which creates appropriate man
pages for each package.
To minimize possible conflicts with other names it creates man pages in
section 6 (games!). Of course this can be configured in the config file.
I'd rather like to know which is a better place for it.
The script does only create pages if none exists. But for upgrading the
force switch has to be used, which means any existing page will be
overwritten.
The script can be down loaded from
"http://dpartialmirror.sourceforge.net/dsc2man.html";
O. Wyss
--
Author of "Debian partial mirror synch script"
("http://dpartialmirror.sourceforge.net/";)
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Description to man pages
> dpkg -s
>
This doesn't show the package description!
O. Wyss
--
Author of "Debian partial mirror synch script"
("http://dpartialmirror.sourceforge.net/";)
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Description to man pages
> > To minimize possible conflicts with other names it creates man pages in
> > section 6 (games!). Of course this can be configured in the config file.
> > I'd rather like to know which is a better place for it.
>
> Use a subsection. For instance, somepackage(1dsc) goes in
> $(mandir)/man1/somepackage.1dsc.gz. This should avoid clashes, and you
> can pass man the '-e dsc' option to look at those pages exclusively. It
> might also be a good idea to write to /usr/local/man by default rather
> than /usr/share/man.
>
I didn't know that man has subsection, the man howto which I found on
the web didn't tell it.
I've now choosen "7dsc" since packages aren't commands.
> Would it be possible to make it easier to use for those who don't use
> debiansynch? I couldn't figure out how to get it to work at all -
> whatever I tried just ended up with "0 processed of 0". I don't have a
> local mirror, so I'd like it just to use the available file.
Actually dsc2man search for Packages files inside the basedir if no
distribution list is specified (empty parameter distsfile" in
"dsc2man.conf" or if the file isn't found). I've changed the behavior so
that searching is the default. Of course the Packages files have to be
located anywhere locally inside the searched basedir (regardless of
structure).
There was also a bug which prevented the search under certain
circumstances, but now it should work.
O. Wyss
--
Author of "Debian partial mirror synch script"
("http://dpartialmirror.sourceforge.net/";)
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Description to man pages
> Is it better than `apt-cache show foo` ?
>
No if you are a power user, otherwise yes. Beside not everbody has
apt-cache installed.
O. Wyss
--
Author of "Debian partial mirror synch script"
("http://dpartialmirror.sourceforge.net/";)
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Description to man pages
> On Thu, Apr 04, 2002 at 11:03:53PM +0200, Otto Wyss wrote:
> > I've now choosen "7dsc" since packages aren't commands.
>
> How about something more descriptive than "dsc"? Say, "package",
> "pkg", or "deb" (in my order of preference)?
>
I'm also not very happy with "dsc" but I neither are with the others.
What do anybody else think?
If there isn't another man page "man " will show the package
description. You could get a list of all man pages for a keyword with
"man -f ".
The best would be if "man would bring up a list of man pages
with a choose facility when more than one page exists. Maybe this change
in behavior could be set through an environment variable.
O. Wyss
--
Author of "Debian partial mirror synch script"
("http://dpartialmirror.sourceforge.net/";)
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Rsyncable GZIP (was Re: Package metadata server)
> > Some questions that need to be asked:
> > Howmany of our mirrors are rsyncable?
> How much load can the servers handle?
> How much more load does rsync do than a fast http server like tux?
>
Please show use any figures first before you assert this.
I know rsync imposes some load for the computing of the md5sum but
sendind only the difference outweighs it repeatedly.
O. Wyss
--
Author of "Debian partial mirror synch script"
("http://dpartialmirror.sourceforge.net/";)
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Rsyncable GZIP (was Re: Package metadata server)
> A large mirror in Australia does provide an rsync server to access debian
> packages. When redhat 7.0 came out so many people tried to rsync it at the
> same time, the machine promptly fell over.
>
What amazes me is that nobody is able or willing to provide any figures.
So I guess no provider of an rsync server is interested in this subject
and therefore it can't be a big problem.
I'm asking any provider of an ftp/rsync Debian server if any comparable
figures could be extracted from the server log. Or if anyone could
measure how much CPU load the download of the Packages/Packages.gz files
really reads.
O. Wyss
--
Author of "Debian partial mirror synch script"
("http://dpartialmirror.sourceforge.net/";)
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Description to man pages
> On Sat, Apr 06, 2002 at 10:01:16AM +0200, Otto Wyss wrote:
> > The best would be if "man would bring up a list of man pages
> > with a choose facility when more than one page exists. Maybe this change
> > in behavior could be set through an environment variable.
>
> No need. Try 'man -a '.
>
> Also, when more than one page exists man will ask you if you want to
> display the next one it's found after displaying the first one. Try it.
I'd rather like it if the menu is shown before not after the first man
page. If I knew another page is following I might jump directly there.
Also I'd rather like this to be the default if multiple pages where
available.
O. Wyss
--
Author of "Debian partial mirror synch script"
("http://dpartialmirror.sourceforge.net/";)
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Debian's problems, Debian's future
> http://lists.debian.org/debian-devel/2001/debian-devel-200111/msg00757.html
Thanks for this pointer.
My debiansynch script never runs into problem "1. rsync -r" since it
always does single file transfers. And for problem "2. rsync of near
identical files" it's not astonishing using a high cpu load for a short
period, an ftp transfer simply distributes its cpu load over a longer
period.
O. Wyss
--
Author of "Debian partial mirror synch script"
("http://dpartialmirror.sourceforge.net/";)
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: rsync and debian -- summary of issues
> > 3.1 Compressed files cannot be differenced
>
> I recall seeing some work done to determine how much savings you could
> expect if you used xdeltas of the uncompressed data. This would be the
> best result you could expect from gzip --rsyncable. I recall the numbers
> were disapointing, it was << 50% on average or somesuch. It would be nice
> if someone could find that email or repeat the experiments.
>
With the help of an admin from an rsync server I tested the download of
a with "gzip --rsyncable" compressed Packages.gz versus an uncompressed
Packages. The results are under
"http://lists.debian.org/debian-devel/2001/debian-devel-200106/msg01859.
html"
It just shows 2 days but the test went on for about a week with similar
results. While uncompressed is still the best for rsync download, it
shows a significant reduction (more than 50%) against a normal
compressed Packages.gz. Similar savings are possible if this is applied
to Debian packages.
O. Wyss
--
Author of "Debian partial mirror synch script"
("http://dpartialmirror.sourceforge.net/";)
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: rsync and debian -- summary of issues
> > http://rsync.samba.org/rsync-and-debian/
> >
> > I'd appreciate comments.
>
This is certainly a very informative page. I'd appreciate if the CPU
load problem could be solved somehow.
IMO the versioning patch from Paul Russell is not the right approach
since this is Debian specific and has nothing to do with rsync. This
should be done by a wrapper script. Maybe someone writes a debianrproxy
but then what's the difference to my script?
O. Wyss
--
Author of "Debian partial mirror synch script"
("http://dpartialmirror.sourceforge.net/";)
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Rsync and single file transfer
Most of the scripts/methods I've seen which downloads Debian packages
with rsync do only single file transfer. IMO this must be much more
server friendly than a multi file transfer (no filelist).
Is it possible run a rsync server with anonymous login but restricted to
single file transfer next to an rsync server with restricted login and
multi file transfer? This would allow access for secondary Debian
mirrors besides endusers. It also would allow to separate and measure
the impact endusers have on rsync servers.
Note: IMO a script like mine is also useful for secondary mirrors at
least if not all architectures are mirrored. Could any mirror
administrator make a test an publish any results?
O. Wyss
--
Author of "Debian partial mirror synch script"
("http://dpartialmirror.sourceforge.net/";)
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Packages still in Potato
IMO each package should at least once per release upload a status report. Also there was ample time for the transition of each package to the pool. These are the reason behind the mailing to each maintainer of packages still in Potato. While most of the answers I got were positive, there were some few I simply don't want to get a second time. Therefore I've lost interest into this matter. I'll append the list of maintainers/packages I haven't gotten any answers so others may finish what I started. Just keep in mind there might be wrong or missing entries since I have no desire to do any checks. For me this matter is now solved, with my script I'll simply apply option --transform=pool. O. Wyss --- [EMAIL PROTECTED] tkxanim [EMAIL PROTECTED] rtlinux-doc [EMAIL PROTECTED] theme-converters [EMAIL PROTECTED] termcap-compat [EMAIL PROTECTED] mdate [EMAIL PROTECTED] festvox-don festvox-rablpc16k festvox-rablpc8k festival-doc festlex-cmu festlex-poslex festvox-kallpc16k festvox-kallpc8k festvox-kdlpc16k [EMAIL PROTECTED] ibm-jdk1.1-installer [EMAIL PROTECTED] hbf-cns40-1 hbf-cns40-2 hbf-cns40-3 hbf-cns40-4 hbf-cns40-5 hbf-cns40-6 hbf-cns40-7 hbf-cns40-b5 xcin2.3 [EMAIL PROTECTED] libstdc++2.9-glibc2.1 [EMAIL PROTECTED] libgd-gif-tools libgd-gif1 [EMAIL PROTECTED] perlmenu [EMAIL PROTECTED] fda hpscanpbm iroffer makepasswd [EMAIL PROTECTED] gilt [EMAIL PROTECTED] adbbs [EMAIL PROTECTED] gwm gwm-doc [EMAIL PROTECTED] enscript xdigger [EMAIL PROTECTED] flexml [EMAIL PROTECTED] gnuhtml2latex [EMAIL PROTECTED] workbone [EMAIL PROTECTED] floppybackup [EMAIL PROTECTED] rcs ??? [EMAIL PROTECTED] gnosamba linleech [EMAIL PROTECTED] lclint-doc [EMAIL PROTECTED] scansort [EMAIL PROTECTED] nsmon [EMAIL PROTECTED] ttysnoop [EMAIL PROTECTED] debian-test [EMAIL PROTECTED] fortunes-it [EMAIL PROTECTED] knl [EMAIL PROTECTED] cern-httpd jargon [EMAIL PROTECTED] xmake [EMAIL PROTECTED] cvs-pcl elib [EMAIL PROTECTED] sendmail-wide [EMAIL PROTECTED] chos nstreams [EMAIL PROTECTED] cedicttools ttfprint lpkg -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Bug#745135: RFS: mariadb-10.0/10.0.10-1 [ITP] -- Latest version of worlds most popular non-Oracle database
Thanks for your interest in the package. I am the submitter and I haven't gotten any feedback so far on the package, so I assume it is simply waiting for somebody to have time to review it. It is almost identical to the current mariadb-5.5 package in Debian, so there shouldn't be any actual issues - we just need to wait. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/CAHj_TLDe8NPHw57DV+TFmTF_QBv_P=bq6odubg6ilfkudeb...@mail.gmail.com
Re: [debian-mysql] Will we see MariaDB in Jessie?
Hello, 2013/5/6 Thomas Goirand : > I wonder what the plans of the MySQL maintainers are concerning MySQL vs > MariaDB. Famously, Fedora made the switch. What will happen in Debian? > What kind of transition would this mean? Would it be a drop-in > replacement like Monty is pretending, or would it be harder? Others already replied, but let me make it clear that we are not going to make any "switch". We are merely adding MariaDB to Debian so that users can choose to either apt-get install mariadb or apt-get install mysql. Both will install smoothly. They both will have conflict-rules, so you cannot install and run them at the same time, but users will be able to easily switch between them, at least for now as they are still binary/configuration/database file format compatible. It is the up to the users to choose which flavor they like the most. Although personally I believe most Debian users will prefer MariaDB over the Oracle version. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CAHj_TLAgj8gae4FP1atrvdVTssY+CHZv782nD=kamydv0uy...@mail.gmail.com
Re: [debian-mysql] Will we see MariaDB in Jessie?
2013/5/6 Steven Ayre : > Having a policy on how such packages can coexist could then allow > other options to also be added cleanly (MySQL Cluster, Percona, Galera > etc). I've done my best to package MariaDB following best practices on Debian control files and conflict/replace rules. So I am confident to say we actually already have a way how to get MySQL flavors to coexist. What we seem to lack is _resources_. I guess we could package all of Percona, Galera etc is we had 15 team members to take care of all testing, security patching etc. Would you like to join? Volunteer by attending our next online meeting! Next online meeting is Thu 2013-05-09 at 20:00 GMT. Anybody can attend, just join the Google Hangout at https://plus.google.com/hangouts/_/calendar/b3R0b0BzZXJhdm8uZmk.29m1jpsppitqqvv76s609dbuf8 (fallback to #debian-mysql if Hangout does not work for somebody). -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CAHj_TLD2h7c_qrcOn7UMaVyRNT1Vo=y=rok+ersek2wj3tq...@mail.gmail.com
Re: Bug#732878: Add MariaDB as an alternative dependency
2013/12/25 Thomas Goirand : > Don't you think it would be more reasonable if the mariadb-client > contained a Provides: mysql-client, rather than changing each and every > software dependency in Debian? Currently the package contains "Provides: virtual-mysql-server" but I guess this needs to be re-evaluated in the packaging team and the rationale documented better, as I have already forgot why we ended up with what we have now.. > P.S: Do you know if the MariaDB package in Sid has the capability to run > in cluster, like for Galera? No, but I might package MariaDB Galera Cluster later if I have time and energy for additional packages. However before that I'll do MariaDB 10.0 (Debian now only got 5.5). -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CAHj_TLDtLiFvQzVOQ2p_VF1DKed4MAz076C2FCZV=rm1f8z...@mail.gmail.com
Re: Bug#732878: Add MariaDB as an alternative dependency
Hello, 2013/12/25 Thomas Goirand : > Don't you think it would be more reasonable if the mariadb-client > contained a Provides: mysql-client, rather than changing each and every > software dependency in Debian? > > Adding debian-devel@, as I think it should be discussed more broadly. We discussed this on the pkg-maint-mysql list and the recommended policy is now: All packages that at the moment depend directly on mysql-client should instead have something like: Depends: the-one-they-tested-with | virtual-mysql-client (or Suggests or Recommends) At the moment in unstable the packages mysql-server-5.5 and mariadb-server-5.5 have Provides: mysql-virtual-server and mysql-client-5-5 and mariadb-client-5.5 have Provides: mysql-virtual-client Later when other versions are uploaded to Debian (e.g. MySQL 5.6, MariaDB 10, Percona etc) they will include the same provides as long as they are compatible enough with MySQL 5.5 to be drop-in-replacements. Does this sound OK? -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CAHj_TLC7TFCpsiQXyakEb+=grgap8thethqsqqszxl4rygb...@mail.gmail.com
Re: Bug#732878: Add MariaDB as an alternative dependency
2013/12/30 Otto Kekäläinen : > We discussed this on the pkg-maint-mysql list and the recommended policy is > now: > > All packages that at the moment depend directly on mysql-client should > instead have something like: > > Depends: the-one-they-tested-with | virtual-mysql-client > (or Suggests or Recommends) > > At the moment in unstable the packages mysql-server-5.5 and > mariadb-server-5.5 have > Provides: mysql-virtual-server > > and mysql-client-5-5 and mariadb-client-5.5 have > Provides: mysql-virtual-client > > Later when other versions are uploaded to Debian (e.g. MySQL 5.6, > MariaDB 10, Percona etc) they will include the same provides as long > as they are compatible enough with MySQL 5.5 to be > drop-in-replacements. > > Does this sound OK? Just for the record, I've written down now this policy at https://wiki.debian.org/Teams/MySQL/virtual-mysql-server and it is now in effect as there was no other contersuggestions, and both the MySQL packages and MariaDB packages in Debian already abide to this policy. -- Otto Kekäläinen +358 44 566 2204 http://seravo.fi/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CAHj_TLCsbrW=rta7o0utnq8ouysbg6xyknox8z1q+sxaa3-...@mail.gmail.com
Re: The future of MariaDB
Hello David, Sorry for the long delay in the reply, this list has so much traffic that I cannot follow it normally. 2015-02-11 0:35 GMT+02:00 David McMackins : > The latest version of libmariadb in Debian no longer works as a drop-in > replacement for MySQL. The library's name and include path has changed > from mysql to mariadb. While I don't have a problem with someone trying > to use their own name, it means that build scripts relying on > mysql_config and code looking for mysql/mysql.h will break with the new > version. Because of this, I'm considering dropping support in my > software for MariaDB, since they have moved away from their original > purpose. The MariaDB version of the libmysqlclient18 package is not in Debian because I've been told that having two packages that provide the same library (soname) is strictly against the policy. The libmysqlclient18 from MariaDB *is* a drop-in replacement for the Oracle one, but it isn't available in Debian. You can download it pre-compiled for Debian from MariaDB.org. The problem you have saw is most likely is because you tried to use the MariaDB Connector for C. It is an LGPL licensed library that does share any code with the GPL licensed libmysqlclient18 library. The ABI they provide is somewhat similar because the use case dictates so, but the library in question is in not supposed to be a drop-in replacement for libmysqlclient18. I you still feel you used libmariadb2 to something it should do, please file a bug and explain the issue in detail. For details see https://packages.debian.org/jessie/libmysqlclient18 https://packages.debian.org/jessie/libmariadb2 I have helped with the packaging of both but I don't maintain either. Everything that comes from the mariadb-10.0 source package is a drop-in replacement, and I am glad to help with any issues you have regarding those packages. Regards, Otto -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/cahj_tlbhd0tw_ehzvffhxfl7pfc3h8pmqz-nrdups64qjo_...@mail.gmail.com
Re: Debian CI pipeline for Developers
la 10. marrask. 2018 klo 22.12 Agustin Henze (t...@debian.org) kirjoitti: .. > Please follow the README[0] to enable CI in your packages. > > [0] https://salsa.debian.org/salsa-ci-team/pipeline/blob/master/README.md I have been using the Salsa-CI (and modified versions of it) in all of my MariaDB repos at https://salsa.debian.org/mariadb-team since August, and it has been great. I have also got Merge Requests on Salsa where the contributor has used the same CI test as well. I warmly recommend everybody to adopt this in their workflow! - Otto
Help request: Contribute to MariaDB 10.3 in Debian efforts
Hello! I've spent the last months preparing MariaDB 10.3 for Debian. I am almost done, but there are a bunch of smaller issues I need help with. I would be glad for anybody who can look into the issues and give solution ideas - or even file a merge request on Debian's Gitlab instance where the Debian development happens: https://wiki.debian.org/Teams/MySQL/patches Git repository for the packaging: https://salsa.debian.org/mariadb-team/mariadb-10.3 Bugs filed against MariaDB in Debian: https://bugs.debian.org/cgi-bin/pkgreport.cgi?repeatmerged=no&src=mariadb-10.3 There are a few ones tagged newcomer friendly as well! Build status in Debian (for porters!): https://buildd.debian.org/status/package.php?p=mariadb-10.3 Please pick something that is close to your expertise and help out if you can! :) - Otto -- News about my work at https://twitter.com/ottokekalainen
Re: Help request: Contribute to MariaDB 10.3 in Debian efforts
Hello fellow Debian Developers and those who want to be one! MariaDB is a big package and it needs help. I've updated the list of newcomer friendly bugs in mariadb-10.3:https://bugs.debian.org/cgi-bin/pkgreport.cgi?repeatmerged=no&src=mariadb-10.3 Perfect for anybody who wants to learn Debian packaging and take their first steps in contributing to an existing package. I promise to review everything you send quickly! The CI has also been updated to extensively test all contributions for obvious regressions. See an overview of all 10 test steps at https://salsa.debian.org/mariadb-team/mariadb-10.3/pipelines/37641 > I've spent the last months preparing MariaDB 10.3 for Debian. I am > almost done, but there are a bunch of smaller issues I need help with. > I would be glad for anybody who can look into the issues and give > solution ideas - or even file a merge request on Debian's Gitlab > instance where the Debian development happens: > https://wiki.debian.org/Teams/MySQL/patches > > Git repository for the packaging: > https://salsa.debian.org/mariadb-team/mariadb-10.3 > > Bugs filed against MariaDB in Debian: > https://bugs.debian.org/cgi-bin/pkgreport.cgi?repeatmerged=no&src=mariadb-10.3 > > There are a few ones tagged newcomer friendly as well! > > Build status in Debian (for porters!): > https://buildd.debian.org/status/package.php?p=mariadb-10.3
Seeking hardening flag / blhc expoert
Hello!
Is there any hardening flag / cmake expert around who could help me
get the hardening flags perfect in MariaDB 10.3?
Current state of build logs issues:
https://qa.debian.org/bls/packages/m/mariadb-10.3.html
The blhc tool currently outputs this:
$ blhc --debian --line-numbers --color ${WORKING_DIR}/*.build || [ $? -eq 1 ]
9962:CPPFLAGS missing (-D_FORTIFY_SOURCE=2):
/usr/lib/ccache/x86_64-linux-gnu-g++
-I/tmp/building/package/builddir/storage/tokudb/PerconaFT/snappy/src/build_snappy
-std=c++11 -g -O2 -fdebug-prefix-map=/tmp/building/package=.
-fstack-protector-strong -Wformat -Werror=format-security -pie -fPIC
-Wl,-z,relro,-z,now -fstack-protector --param=ssp-buffer-size=4
-fno-rtti -Wno-shadow -Wno-implicit-fallthrough -std=c++11
-Wno-missing-field-initializers -Wno-missing-field-initializers
-Wstrict-null-sentinel -Winit-self -Wswitch -Wtrampolines -Wlogical-op
-Wno-error=missing-format-attribute -Wno-error=maybe-uninitialized
-fno-rtti -fno-exceptions -Wno-error=nonnull-compare -Wpacked
-fno-omit-frame-pointer -Wno-error=strict-overflow -fexceptions
-Wextra -Wno-missing-noreturn -Wmissing-declarations -Wpointer-arith
-Wcast-align -O2 -g -DNDEBUG -fPIC -Wno-sign-compare
-Wno-unused-function -Wno-unused-parameter -fvisibility=hidden -fPIC
-o CMakeFiles/snappy.dir/snappy.cc.o -c
/tmp/building/package/builddir/storage/tokudb/PerconaFT/snappy/src/build_snappy/snappy.cc
9964:CPPFLAGS missing (-D_FORTIFY_SOURCE=2):
/usr/lib/ccache/x86_64-linux-gnu-g++
-I/tmp/building/package/builddir/storage/tokudb/PerconaFT/snappy/src/build_snappy
-std=c++11 -g -O2 -fdebug-prefix-map=/tmp/building/package=.
-fstack-protector-strong -Wformat -Werror=format-security -pie -fPIC
-Wl,-z,relro,-z,now -fstack-protector --param=ssp-buffer-size=4
-fno-rtti -Wno-shadow -Wno-implicit-fallthrough -std=c++11
-Wno-missing-field-initializers -Wno-missing-field-initializers
-Wstrict-null-sentinel -Winit-self -Wswitch -Wtrampolines -Wlogical-op
-Wno-error=missing-format-attribute -Wno-error=maybe-uninitialized
-fno-rtti -fno-exceptions -Wno-error=nonnull-compare -Wpacked
-fno-omit-frame-pointer -Wno-error=strict-overflow -fexceptions
-Wextra -Wno-missing-noreturn -Wmissing-declarations -Wpointer-arith
-Wcast-align -O2 -g -DNDEBUG -fPIC -Wno-sign-compare
-Wno-unused-function -Wno-unused-parameter -fvisibility=hidden -fPIC
-o CMakeFiles/snappy.dir/snappy-c.cc.o -c
/tmp/building/package/builddir/storage/tokudb/PerconaFT/snappy/src/build_snappy/snappy-c.cc
9966:CPPFLAGS missing (-D_FORTIFY_SOURCE=2):
/usr/lib/ccache/x86_64-linux-gnu-g++
-I/tmp/building/package/builddir/storage/tokudb/PerconaFT/snappy/src/build_snappy
-std=c++11 -g -O2 -fdebug-prefix-map=/tmp/building/package=.
-fstack-protector-strong -Wformat -Werror=format-security -pie -fPIC
-Wl,-z,relro,-z,now -fstack-protector --param=ssp-buffer-size=4
-fno-rtti -Wno-shadow -Wno-implicit-fallthrough -std=c++11
-Wno-missing-field-initializers -Wno-missing-field-initializers
-Wstrict-null-sentinel -Winit-self -Wswitch -Wtrampolines -Wlogical-op
-Wno-error=missing-format-attribute -Wno-error=maybe-uninitialized
-fno-rtti -fno-exceptions -Wno-error=nonnull-compare -Wpacked
-fno-omit-frame-pointer -Wno-error=strict-overflow -fexceptions
-Wextra -Wno-missing-noreturn -Wmissing-declarations -Wpointer-arith
-Wcast-align -O2 -g -DNDEBUG -fPIC -Wno-sign-compare
-Wno-unused-function -Wno-unused-parameter -fvisibility=hidden -fPIC
-o CMakeFiles/snappy.dir/snappy-sinksource.cc.o -c
/tmp/building/package/builddir/storage/tokudb/PerconaFT/snappy/src/build_snappy/snappy-sinksource.cc
9968:CPPFLAGS missing (-D_FORTIFY_SOURCE=2):
/usr/lib/ccache/x86_64-linux-gnu-g++
-I/tmp/building/package/builddir/storage/tokudb/PerconaFT/snappy/src/build_snappy
-std=c++11 -g -O2 -fdebug-prefix-map=/tmp/building/package=.
-fstack-protector-strong -Wformat -Werror=format-security -pie -fPIC
-Wl,-z,relro,-z,now -fstack-protector --param=ssp-buffer-size=4
-fno-rtti -Wno-shadow -Wno-implicit-fallthrough -std=c++11
-Wno-missing-field-initializers -Wno-missing-field-initializers
-Wstrict-null-sentinel -Winit-self -Wswitch -Wtrampolines -Wlogical-op
-Wno-error=missing-format-attribute -Wno-error=maybe-uninitialized
-fno-rtti -fno-exceptions -Wno-error=nonnull-compare -Wpacked
-fno-omit-frame-pointer -Wno-error=strict-overflow -fexceptions
-Wextra -Wno-missing-noreturn -Wmissing-declarations -Wpointer-arith
-Wcast-align -O2 -g -DNDEBUG -fPIC -Wno-sign-compare
-Wno-unused-function -Wno-unused-parameter -fvisibility=hidden -fPIC
-o CMakeFiles/snappy.dir/snappy-stubs-internal.cc.o -c
/tmp/building/package/builddir/storage/tokudb/PerconaFT/snappy/src/build_snappy/snappy-stubs-internal.cc
Full log at:
https://salsa.debian.org/mariadb-team/mariadb-10.3/-/jobs/153422
d/rules:
https://salsa.debian.org/mariadb-team/mariadb-10.3/blob/master/debian/rules
Re: Seeking hardening flag / blhc expoert
Hello! > > Is there any hardening flag / cmake expert around who could help me > > get the hardening flags perfect in MariaDB 10.3? > Start with https://wiki.debian.org/Hardening#Notes_for_packages_using_CMake I've read this section many times over but I don't get it. A workaround is presented but since we are on a new debhelper it is advised not to be used. It suggests using /usr/share/dpkg/buildflags.mk but since we already call default.mk the buildflags.mk should be included. There are some variables set, but since the cmake command does not include them, changes in them does not have an effect. There is no explanation about that flags do what and which are the relevant ones, so blindly just defining everything does not seem like a savvy solution. I would appreciate if you can pinpoint what is the missing flag exactly and what is now not passed to cmake correctly.. > > d/rules: > > https://salsa.debian.org/mariadb-team/mariadb-10.3/blob/master/debian/rules > One of the problems is using $(MAKE) instead of dh_auto_build and so on. > There are other problems in this file. Since the build command is constructed in the override_dh_auto_configure stanza this is the only way I am aware that I can pass it on to dh_auto_build. I am happy to try out alternative ways if you have concrete suggestions on how to refactor the d/rules file Thanks for pointers and help!
Re: Seeking hardening flag / blhc expoert
So apparently the 'D_FORTIFY_SOURCE=2' is in CPPFLAGS (not read by cmake) but not in CXXFLAGS (read by cmake)[1]. So maybe I should define? CXXFLAGS=$(CXXFLAGS) $(CPPFLAGS) This is the current state of mysqld, should I be happy with this or is it relevant that all functions are protected? hardening-check --verbose --color mysqld mysqld: Position Independent Executable: yes Stack protected: yes Fortify Source functions: yes (some protected functions found) unprotected: strcpy unprotected: strcat unprotected: recv unprotected: snprintf unprotected: getcwd unprotected: readlink unprotected: memset unprotected: poll unprotected: fread unprotected: fgets unprotected: strncpy unprotected: sprintf unprotected: stpcpy unprotected: strncat unprotected: memcpy unprotected: read unprotected: confstr unprotected: pread64 unprotected: memmove unprotected: gethostname protected: strcpy protected: snprintf protected: vfprintf protected: memset protected: poll protected: vasprintf protected: fread protected: strncpy protected: sprintf protected: vsprintf protected: memcpy protected: fdelt protected: realpath protected: pread64 protected: vsnprintf protected: fprintf protected: memmove protected: printf Read-only relocations: yes Immediate binding: yes [1] https://cmake.org/Bug/view.php?id=12928
Re: Seeking hardening flag / blhc expoert
Hello! Thanks everybody for the pointers. I fixed it now with: Subject: [PATCH] Ensure cmake builds also apply CPPFLAGS flags for hardening to fully work --- debian/rules | 5 + 1 file changed, 5 insertions(+) diff --git a/debian/rules b/debian/rules index 3a16f8bfa..2e7536b9c 100755 --- a/debian/rules +++ b/debian/rules @@ -7,6 +7,11 @@ export DH_VERBOSE=1 export DEB_BUILD_MAINT_OPTIONS = hardening=+all DPKG_EXPORT_BUILDFLAGS = 1 include /usr/share/dpkg/default.mk +# CPPFLAGS are nor read by CMake, so copy them to CXXFLAGS +# See why at https://cmake.org/Bug/view.php?id=12928 +# This is needed for e.g. all automatic Debian hardening flags to apply on all cmake builds. +CFLAGS+=$(CPPFLAGS) +CXXFLAGS+=$(CPPFLAGS) # Only do a strict symbol checking on Linux ifneq (,$(filter linux,$(DEB_HOST_ARCH_OS))) https://salsa.debian.org/mariadb-team/mariadb-10.3/commit/fc4f33cf40d0a10ef5d1992accd2af734ba96356 Results at: https://salsa.debian.org/mariadb-team/mariadb-10.3/-/jobs/154355
Re: lintian.debian.org off ?
> > > Host lintian.debian.org not found: 3(NXDOMAIN) > > > > > > is this expected ? > > > > Yes, it is replaced by the UDD interface: > > > > https://wiki.debian.org/Services/lintian.debian.org > > https://udd.debian.org/lintian/ Could we please have a HTTP 301 redirect from lintian.debian.org to https://udd.debian.org/lintian/ so people will automatically find the new location? And perhaps a separate redirect from URL https://lintian.debian.org/manual to the new location, wherever the guide now is to be read. I used the old website frequently, and also shared links to the Lintian error explanations there frequently. Currently all links from all HTML reports from Lintian are broken, see https://mariadb-team.pages.debian.net/-/galera-4/-/jobs/4709137/artifacts/debian/output/lintian.html for example. Maybe they should now be updated to be links to UDD instead? Thanks for everyone maintaining our infra and working towards resolving this. > > There is no web based location for the description of each tag yet. > > I don't know if it is just me, but even udd gives me a 500 > when I try to check lintian output for any (existing) package. > > For example: https://udd.debian.org/lintian/?packages=nim I also just get error 500 when trying to look up LIntian errors for my own packages..
Salsa-CI featured on GitLab.com blog
Hi all! I just wanted to share that the story about Salsa-CI was featured a couple days ago at https://about.gitlab.com/blog/2023/09/19/debian-customizes-ci-tooling-with-gitlab/ Personally I think Salsa-CI is extremely useful and pleasant to use and understand, and it surely helped make Debian 12 a very solid and high quality release. Nice to see this getting more visibility! If you want to help boost the visibility, please vote on HackerNews: https://news.ycombinator.com/item?id=37635937 Thanks to Alexander and Joerg for maintaining Salsa itself, and to Santiago, Iñaki and Agustin for being the main drivers of https://salsa.debian.org/salsa-ci-team/pipeline since 2018. - Otto
Re: lintian.debian.org off ?
> > > > Host lintian.debian.org not found: 3(NXDOMAIN) > > > > > > > > is this expected ? > > > > > > Yes, it is replaced by the UDD interface: > > > > > > https://wiki.debian.org/Services/lintian.debian.org The page above links to two bug reports but I can't find any actual information about *why* the site https://lintian.debian.org/ was shut down? The site https://udd.debian.org/lintian-tag.cgi?tag=wrong-name-for-upstream-changelog only lists packages with the issue, while the old site had full explanations of the issue and felt quite friendly and easy to consume (non-CSS styled version still visible in Google cache, eg. https://webcache.googleusercontent.com/search?q=cache:AKmDGNJEIaAJ:https://lintian.debian.org/tags/wrong-name-for-upstream-changelog).
Re: lintian.debian.org off ?
Hi! Thanks for the context - so there is no need technical incompatibility at play, but mostly a matter of having resources and time to do it. .. > Regarding the 301 redirection I'll see with the interested parties (DSA > and Lintian maintainers) if this option is fine with everyone. I could easily write Ansible code to maintain a simple Nginx server, with 302 redirects https://lintian.debian.org/tags/(.*)/?$ -> https://udd.debian.org/lintian-tag.cgi?tag=$1, use same Ansible style as salsa.debian.org is maintained on (https://salsa.debian.org/salsa/salsa-ansible), and also donate a tiny virtual machine for Debian project if needed. Is there some special bureaucracy on top of that work to do to be able to contribute with this? I know Lintian tag info is available via command line, but I frequently need to educate upstreams about Lintian rules, and thus really also need a URL to share to them. Perhaps I could implement that later in the year. - Otto
