Re: [arch-dev-public] openssl 3.0
On Thu, 2022-01-27 at 16:47 +0100, Christian Hesse via arch-dev-public wrote: > Pierre Schmitz via arch-dev-public > on > Sun, 2022/01/23 12:50: > > Next steps: > > 1) Let's agree on a time window where no other rebuild can take > > place > > within our staging repos. How about at least the first two weeks in > > February? > > I guess the ffmpeg 5.0 will be blocking for some time... Not necessarily. There are too many packages that don't build, I will maintain a temporary ffmpeg4.4 package to get this todo out quickly. Should have time for this over the weekend. Cheers, -- Maxime signature.asc Description: This is a digitally signed message part
[arch-dev-public] Starting x86_64_v3 port
Hi all, You may remember long in the past we discussed adding an x86_64_v3 port. From memory, pkgstats shows this will benefit ~2/3 (or 3/4?) of our users! So lets get this underway! Apart from tooling (devtools/dbscripts), we need to make some decisions. My plan is to seed the x86_64_v3 repos with x86_64 packages and then they get replaced as updates/rebuilds happen (potentially supplemented with a rebuild party). Pacman 6.0 can handle this. This is not a perfectly clean new port, but is substantially less of a burden than releasing a pure x86_64_v3 port. Our .BUILDINFO files do record the package architecture, so this mixture should not affect (e.g.) our reproducible builds etc. The decision to be made is who will package for this repo? I think these are the options: A) packagers upload both x86_64 and x86_64_v3 to the repos. Our build server will help those without x86_64_v3 machines. B) we recruit some packagers to build the x86_64_v3 packages. C) Some combination of A+B. My understanding is our x86_64 port started with B, then C, then A. I think with our build infrastructure now, we can start with A, but that is more of a burden for packagers. I doubt it will be much of a burden as x86_64_v3 specific build issues are unlikely. Is there any particular objection to requiring packagers upload both architectures? Allan
Re: [arch-dev-public] Starting x86_64_v3 port
On Sat, Jan 29, 2022 at 10:12:30AM +1000, Allan McRae via arch-dev-public wrote: > Is there any particular objection to requiring packagers upload both > architectures? I'm personally not really motivated doing the required builds. We have an underdeveloped infrastructure which hasn't changed since we abandoned i686 5 years ago. I'd personally like to see more work on our build tooling before we commit to new architectures. -- Morten Linderud PGP: 9C02FF419FECBE16 signature.asc Description: PGP signature
Re: [arch-dev-public] Starting x86_64_v3 port
On 29/1/22 11:13, Morten Linderud via arch-dev-public wrote: On Sat, Jan 29, 2022 at 10:12:30AM +1000, Allan McRae via arch-dev-public wrote: Is there any particular objection to requiring packagers upload both architectures? I'm personally not really motivated doing the required builds. We have an underdeveloped infrastructure which hasn't changed since we abandoned i686 5 years ago. I'd personally like to see more work on our build tooling before we commit to new architectures. FYI, it is a single extra command. Either of these will work... offload-build --arch x86_64_v3 extra-x86_64_v3-build Nothing else changes for the packager. Allan
Re: [arch-dev-public] Starting x86_64_v3 port
On Sat, Jan 29, 2022 at 11:22:32AM +1000, Allan McRae via arch-dev-public wrote: > On 29/1/22 11:13, Morten Linderud via arch-dev-public wrote: > > On Sat, Jan 29, 2022 at 10:12:30AM +1000, Allan McRae via arch-dev-public > > wrote: > > > Is there any particular objection to requiring packagers upload both > > > architectures? > > > > I'm personally not really motivated doing the required builds. We have an > > underdeveloped infrastructure which hasn't changed since we abandoned i686 5 > > years ago. > > > > I'd personally like to see more work on our build tooling before we commit > > to > > new architectures. > > FYI, it is a single extra command. Either of these will work... > > offload-build --arch x86_64_v3 > extra-x86_64_v3-build > > Nothing else changes for the packager. > > Allan I'll spend twice as long waiting for a package to build which increases the time spent packaging. Which again requires me to spend more time watching stuff fly by. This also assumes people are capable of using the buildserver which is not always the case either. This wasn't great with i686, and I'm not sure why we'd find this acceptable today? -- Morten Linderud PGP: 9C02FF419FECBE16 signature.asc Description: PGP signature
Re: [arch-dev-public] Starting x86_64_v3 port
On 29/1/22 10:12, Allan McRae via arch-dev-public wrote: Apart from tooling (devtools/dbscripts), we need to make some decisions. FYI - I just submitted merge requests for both devtools and dbscripts. These should be committed in tandem with work at devops end preparing the initial state of the x86_64_v3 repos. There may be more work needed on dbscripts to cleanup x86_64 packages from the x86_64_v3 repos, assuming we seed the repos that way. Allan
Re: [arch-dev-public] Starting x86_64_v3 port
On 29/1/22 11:28, Morten Linderud via arch-dev-public wrote: On Sat, Jan 29, 2022 at 11:22:32AM +1000, Allan McRae via arch-dev-public wrote: On 29/1/22 11:13, Morten Linderud via arch-dev-public wrote: On Sat, Jan 29, 2022 at 10:12:30AM +1000, Allan McRae via arch-dev-public wrote: Is there any particular objection to requiring packagers upload both architectures? I'm personally not really motivated doing the required builds. We have an underdeveloped infrastructure which hasn't changed since we abandoned i686 5 years ago. I'd personally like to see more work on our build tooling before we commit to new architectures. FYI, it is a single extra command. Either of these will work... offload-build --arch x86_64_v3 extra-x86_64_v3-build Nothing else changes for the packager. Allan I'll spend twice as long waiting for a package to build which increases the time spent packaging. Which again requires me to spend more time watching stuff fly by. I have concerns if that is how you spend your time... :) This also assumes people are capable of using the buildserver which is not always the case either. This wasn't great with i686, and I'm not sure why we'd find this acceptable today? What was not great with i686? We managed two architectures for many, many years. The reason for removing i686 was to do with outdated technology, not to do with build times or infrastructure. Do you have objections beyond not wanting to package for both repos? i.e. do you object to option C in my original email, where we have a team to keep the repos in sync when package maintainers do not build both? Allan
Re: [arch-dev-public] Starting x86_64_v3 port
On 1/29/22 02:12, Allan McRae via arch-dev-public wrote: The decision to be made is who will package for this repo? I think these are the options: A) packagers upload both x86_64 and x86_64_v3 to the repos. Our build server will help those without x86_64_v3 machines. B) we recruit some packagers to build the x86_64_v3 packages. C) Some combination of A+B. My understanding is our x86_64 port started with B, then C, then A. I am fine with either and could happily help with B in long term. For me the issue with either B or C is that our packages are often FTBFS and we are slow to fix them, generally. To make the port really usable for a B/C workflow, we need a way to fill in the time gap (because the old package could be unusable for the time, like missing a so-name bump etc). Do you find it acceptable if the x86_64_v3 rebuilders put back in the new x86_64 package until the build was fixed and probably a point pkgrel was added for the real x86_64_v3 rebuild, as long as we use B/C to build for x86_64_v3, in the long term? -- Regards, Felix Yan OpenPGP_signature Description: OpenPGP digital signature
Re: [arch-dev-public] Starting x86_64_v3 port
On 29/1/22 13:12, Felix Yan via arch-dev-public wrote: On 1/29/22 02:12, Allan McRae via arch-dev-public wrote: The decision to be made is who will package for this repo? I think these are the options: A) packagers upload both x86_64 and x86_64_v3 to the repos. Our build server will help those without x86_64_v3 machines. B) we recruit some packagers to build the x86_64_v3 packages. C) Some combination of A+B. My understanding is our x86_64 port started with B, then C, then A. I am fine with either and could happily help with B in long term. For me the issue with either B or C is that our packages are often FTBFS and we are slow to fix them, generally. To make the port really usable for a B/C workflow, we need a way to fill in the time gap (because the old package could be unusable for the time, like missing a so-name bump etc). Do you find it acceptable if the x86_64_v3 rebuilders put back in the new x86_64 package until the build was fixed and probably a point pkgrel was added for the real x86_64_v3 rebuild, as long as we use B/C to build for x86_64_v3, in the long term? I envisioned the x86_64_v3 repos were initially seeded with x86_64 packages. Then x86_64_v3 packages gradually replaced them as updates/rebuilds happen in x86_64. FTBFS for new packages should be less of an issue as long as the two repos do not get out of sync for long - I was considering the i686/x86_64 days where the sync with i686 was within days if not hours. But yes, a workaround could be adding x86_64 packages into x86_64_v3 if really needed. I'm not sure the workflow for doing that in terms of devtools/dbscripts. Allan
Re: [arch-dev-public] Starting x86_64_v3 port
On Sat, 2022-01-29 at 11:45 +1000, Allan McRae via arch-dev-public wrote: > On 29/1/22 11:28, Morten Linderud via arch-dev-public wrote: > > On Sat, Jan 29, 2022 at 11:22:32AM +1000, Allan McRae via arch-dev-public > > wrote: > > > On 29/1/22 11:13, Morten Linderud via arch-dev-public wrote: > > > > On Sat, Jan 29, 2022 at 10:12:30AM +1000, Allan McRae via arch-dev- > > > > public wrote: > > > > > Is there any particular objection to requiring packagers upload both > > > > > architectures? > > > > > > > > I'm personally not really motivated doing the required builds. We have > > > > an > > > > underdeveloped infrastructure which hasn't changed since we abandoned > > > > i686 5 > > > > years ago. > > > > > > > > I'd personally like to see more work on our build tooling before we > > > > commit to > > > > new architectures. > > > > > > FYI, it is a single extra command. Either of these will work... > > > > > > offload-build --arch x86_64_v3 > > > extra-x86_64_v3-build > > > > > > Nothing else changes for the packager. > > > > > > Allan > > > > I'll spend twice as long waiting for a package to build which increases the > > time > > spent packaging. Which again requires me to spend more time watching stuff > > fly > > by. > > I have concerns if that is how you spend your time... :) > > > This also assumes people are capable of using the buildserver which is not > > always the case either. > > > > This wasn't great with i686, and I'm not sure why we'd find this acceptable > > today? > > What was not great with i686? We managed two architectures for many, > many years. The reason for removing i686 was to do with outdated > technology, not to do with build times or infrastructure. > > Do you have objections beyond not wanting to package for both repos? > i.e. do you object to option C in my original email, where we have a > team to keep the repos in sync when package maintainers do not build both? > > Allan I think the issue is not that we don't want to package for both repos, but the increased workload that places on us. For example, the most painful packages to update for me personally are the cross compilers (and other elements of the cross toolchains, but mainly the compilers). This is something I am already struggling with, and adding a new architecture will make it twice as painful. https://archlinux.org/packages/?sort=&q=gcc&maintainer=FFY00&flagged=Flagged This could have been implemented by making x86_64_v3 kind of an overlay over the normal x86_64, but instead it is its hole new architecture requiring duplicated builds for all packages, and not just the ones where it actually makes a difference and/or the maintainer is willing to do it. I don't think this proposal considers the impact it will have on our already struggling staff properly, but alas, there's nothing I can do as the RFC was accepted. It will certainly deter me from adding a couple new packages to the repos, and perhaps drop some, FWIW. The best option seems B, or C, but I don't really understand how B is supposed to work in practice. Isn't the new architecture is supposed to be in sync with the current one? Because not doing so seems a little bit problematic, and I don't see a way for this to work otherwise. Anyway, I thank you for your time working on this, it is certainly valuable, I am just unsure, and, admittedly, perhaps a little bit pessimistic, about how this implementation impacts my commitment as an Arch packager. My two cents are that adding a x86_64_v3 would be amazing if had already reached the, still far, goal of having automated package builders. But we are not there yet, so this introduces a lot of manual labor :/ Cheers, Filipe Laíns signature.asc Description: This is a digitally signed message part
Re: [arch-dev-public] Starting x86_64_v3 port
Excerpts from Morten Linderud via arch-dev-public's message of January 29, 2022 2:28: On Sat, Jan 29, 2022 at 11:22:32AM +1000, Allan McRae via arch-dev-public wrote: On 29/1/22 11:13, Morten Linderud via arch-dev-public wrote: > On Sat, Jan 29, 2022 at 10:12:30AM +1000, Allan McRae via arch-dev-public wrote: > > Is there any particular objection to requiring packagers upload both > > architectures? > > I'm personally not really motivated doing the required builds. We have an > underdeveloped infrastructure which hasn't changed since we abandoned i686 5 > years ago. > > I'd personally like to see more work on our build tooling before we commit to > new architectures. FYI, it is a single extra command. Either of these will work... offload-build --arch x86_64_v3 extra-x86_64_v3-build Nothing else changes for the packager. Allan I'll spend twice as long waiting for a package to build which increases the time spent packaging. Which again requires me to spend more time watching stuff fly by. You *can* run the builds concurrently though since they're different architectures. So at least if you're using the build server it shouldn't be significantly slower unless you're building something that is heavy enough to build where you use ~all of the resources it has. -- Sincerely, Johannes Löthberg :: SA0DEM pgpNsIkbyvj8B.pgp Description: PGP signature
