Re: [gentoo-dev] The uncertain future of repository mirrors
On 30/03/2025 10.11, Michael Mair-Keimberger wrote: On 2025-03-21 14:32, Michał Górny wrote: Hello, everyone. TL;DR: I'm thinking of shutting down all gentoo-mirror repositories, except for gentoo and guru. Hi, I guess i'm a bit late in this discussion but i wanted to let you know this would also affect my gentoo qa scripts. (https:// gentooqa.levelnine.at). Right now i'm checking the gentoo, guru, kde, science and pentoo repositories, syncing the repos from gentoo-mirror for the pre-created metadata. While the gentoo repository is probably the most valuable, kde, science and pentoo checks would be non functional and i probably would have to remove them. (at least until i update my script to create the metadata myself..) Updating the script would ideally just consist of adding the line pkg repo metadata regen --use-local $REPO to the script. As radhermit wrote, pkgcraft is pretty fast these days. I do this for multiple repos on every sync and barely notice it. - Flow OpenPGP_0x8CAC2A9678548E35.asc Description: OpenPGP public key OpenPGP_signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] Re: The meaning of attributes in repositories.xml?
Am Freitag, 28. März 2025, 09:23:42 Mitteleuropäische Sommerzeit schrieb Duncan: > Michał Górny posted on Fri, 28 Mar 2025 05:27:40 +0100 as excerpted: > > > Hello, > > > > I've looked at our repositories.xml and the quality/status attributes > > don't seem to be used very meaningfully. > > > > That is, by quality: > > > > core: gentoo [official] > > stable: opentransactions (?) [official (?!)] > > testing: hyprland-overlay, moexiami [both unofficial] > > experimental: everything else graveyard: unused > > > > By status: > > > > official: ago, alexxy, anarchy, andrey_utkin, cj-overlay, dilfridge, > > emacs, EmilienMottet, fordfrog, gentoo, gnome, gnustep, graaff, guru, > > haskell, java, jmbsvicetto, kde, libressl, maekke, masterlay, mschiff, > > multilib-portage, musl, mysql, opentransactions, pentoo, pinkbyte, > > qemu-init, qt, R_Overlay, rich0, riscv, rnp, ruby, science, sping, > > swegener, tex-overlay, toolchain, ukui, ulm, vGist, voyageur, x11 > > > > unofficial: everything else > > > > > > Which brings the significant question: are these attributes in any way > > meaningful? Is there a point in keeping them at all? Should we set > > some ground rules and make them used consistently? > > > > Of them all, only "core" makes sense right now. "stable" and "testing" > > are used only by random user overlays, with no apparent features. > > Similarly, "official" is used by a mix of developer and ex-developer > > repositories, developer and user project repositories, and a bunch of > > user repositories with no clearly distinct features. > > So what you didn't mention but I assume knew, thus making your question > more one of: "This seems to have changed, do we get stricter again or lose > the attributes which don't seem to mean anything any more"... > > My (user) understanding from "back in the day" when overlays were fairly > new and I first merged and configured layman (reading its config docs > where IIRC this came from to do so), keeping in mind that back then > overlays were a new concept and a major point from the detractors was fear > that actually providing official overlays management and documentation > would somehow implicate Gentoo if a user took advantage to distribute > overt malware: > > Status: > > * "Official" status meant managed by an official Gentoo project or > developer (who had gone thru the usual vetting process), thereby implying > the same security-trust level as the main Gentoo tree. That is, > regardless of quality (experimental, testing, etc), the contents should be > relatively trustworthy at minimum not to include deliberate ebuild/eclass > level malware. > > The implication of "official" was that any deliberate or "they went > through the vetting process and should have known better" security > violation (as opposed to quality/QA violation) in any "official" overlay > would be treated as if it had occurred in the main overlay, and would not > only trigger ejection of the dev in question but a reexamination of what > could be done to improve vetting to avoid it happening again in the > future, as well as possible prosecution as appropriate. > > * "Unofficial" status had rather less security-trust and was intended for > "ordinary users". Unvetted, "caveat emptor", "here be dragons" and "if it > breaks you get to keep the pieces". Security violations would of course > result in removal of the overlay from the list... after the fact. > > The implication was "If it's from an unofficial overlay, be sure you > either trust the author with effective root on your system or explicitly > examine the code before running it, because effective root on your system > is what you're giving them." > > ... > > I thus find it ... "unsettling"... to read that various user overlays have > apparently been marked "official" with no regard to that original policy. > While the original distinction may have arguably had alarmist motivations, > I definitely still find it useful, within a somewhat more limited context, > and consider "official" status among other factors when I consider adding > an overlay. > > Guru specifically, given its purpose and that I personally have it active > (but ATM unused), I wonder about having official status. I only "sort of" > use one ebuild from there, net-nntp/pan -- "sort of" because I used it as > a basis for my personal overlay's pan- live-git ebuild, when upstream > switched autotools -> cmake. (FWIW I've been "going to" contact and > coordinate with the primary author and perhaps add the - version to > guru as well once we do, but that's yet to happen...) Obviously I did the > appropriate "unofficial status level" security evaluation in the process > of converting it to live-git -. > > Quality: > > I /think/ the quality attribute /may/ have been introduced later as IDR > reading about it in the original layman docs, as I think back then the > /assumption/ was that "if it's only in
[gentoo-dev] Package up for grabs: net-misc/apt-cacher-ng
I'm going to drop myself from this package. I'm not using it anymore, and it keeps bumping into build issues with newer/alternative toolchains. It's one of those annoying Debian-as-de-facto-upstream packages, but it's ok with garden-path compilers/libc's. signature.asc Description: PGP signature
Re: [gentoo-dev] The uncertain future of repository mirrors
On 2025-03-21 14:32, Michał Górny wrote: Hello, everyone. TL;DR: I'm thinking of shutting down all gentoo-mirror repositories, except for gentoo and guru. Over 10 years ago, I've started the repository mirror & CI project. What started as a bunch of shell scripts on a user-donated server, has organically grown into a bigger bunch of shell scripts managed by Infra. Nevertheless, it's still a bunch of hacks glued together. Things don't work well all the time. Sometimes stuff randomly crashes, and I have to SSH and remove local checkouts to make it work. Sometimes the git repositories used to transfer logs grow so big they kill infra. Often some repository starts crashing this or another part and needs to be disabled. To be honest, I have no energy to keep maintaining this. I'm really tired of having to deal with stuff crashing and spamming my mailbox with failure mails. I'm tired of having to go through all the infra hoops just to disable another repository that can't work for one reason or another. In fact, I'm even tired that whenever people add new repositories to api.gentoo.org, I have to go through that idiotic GitHub clickety-click UI to stop receiving notifications for everything that happens in these repositories. So what I'm thinking about is winding most of the project down. We'd stop mirroring third-party repositories, and remove most of gentoo- mirror organization. What I'd like to leave is mirroring of gentoo and guru repositories, since these two we have control of, and are very important to Gentoo users. So, well, unless someone convinces me otherwise, I'm going to disable all other repositories over the next weekend, and remove their mirrors. Gentoo and GURU will still be mirrored, and CI will keep running as usual. Hi, I guess i'm a bit late in this discussion but i wanted to let you know this would also affect my gentoo qa scripts. (https://gentooqa.levelnine.at). Right now i'm checking the gentoo, guru, kde, science and pentoo repositories, syncing the repos from gentoo-mirror for the pre-created metadata. While the gentoo repository is probably the most valuable, kde, science and pentoo checks would be non functional and i probably would have to remove them. (at least until i update my script to create the metadata myself..) I guess it's not a big loss, just wanted to let you know. Regards Michael
Re: [gentoo-dev] The uncertain future of repository mirrors
On 2025-03-30 Sun 02:11, Michael Mair-Keimberger wrote: I guess i'm a bit late in this discussion but i wanted to let you know this would also affect my gentoo qa scripts. (https://gentooqa.levelnine.at). Right now i'm checking the gentoo, guru, kde, science and pentoo repositories, syncing the repos from gentoo-mirror for the pre-created metadata. While the gentoo repository is probably the most valuable, kde, science and pentoo checks would be non functional and i probably would have to remove them. (at least until i update my script to create the metadata myself..) Just to point out, generating metadata can be relatively quick these days for large repos using non-portage tools. Anyone using egencache or `emerge --regen` from portage (especially for new repos without any metadata) is wasting a lot of CPU cycles and memory mainly due to portage's inefficient design executing a new instance of bash per ebuild. Alternatives would be `pmaint regen` from pkgcore (roughly 4x faster than portage) and `pk repo metadata regen` from pkgcraft-tools (roughly 10x faster than portage while using 10x less memory than pkgcore). On semi-decent desktop hardware a full gentoo repo metadata run using pkgcraft takes ~10-20 seconds and less than a second for full verification only. For those interested in more design discussion and benchmarks see [1]. Tim [1]: https://pkgcraft.github.io/posts/metadata-cache-generation/
