LLVM build strategy (was: Re: [gentoo-dev] [RFC] New categories for LLVM)

2024-12-07 Thread Sam James
Michał Górny  writes:

> Hello,
>
> Given that the number of LLVM packages is growing, and probably will
> grow again (I'm introducing "offload" right now, expect at least MLIR
> soon, there are open requests for flang, polly...), I'd like to propose
> creating dedicated categories for these packages and moving them there.
>
> [...]
>
> WDYT?

I fear this sort of assumes we won't switch to monobuild any time soon.

I keep thinking [0] about how sustainable our current setup is:
* Fedora moved away from it for >=18 [1].
* As we saw with offload, it broke a few times in just a week. So it's
  only Gentoo who cares about this configuration AFAIK.
* It's not working great if we're already not able to easily package
  mlir, flang, or polly.

Violet attempted working on a merge before [2].

The trade-offs I see are:
* Increased disk space requirement for building LLVM

  I think that's a legitimate concern although perhaps not that big of a
  deal given it is, after all, a whole toolchain. GCC takes quite a bit
  to build too.

* Build time

  Build time for mono LLVM would increase as we're building more
  components at least for some users.

  But the time added by
  building more components (see below) should be balanced out by ccache if
  doing development and also, importantly, not needing to force on all
  targets anymore (they keep growing).

  The cumulative time should be the same (although maybe a bit less
  given the targets change) given that most people need the
  same set of components because of mesa, firefox, or other things which
  need libclang.

* Moving to an upstream supported-configuration

  We may have a bit of pain in moving to it and getting used to it, but
  we're no longer swimming upstream and being the only people caring
  about our choice of build configuration (and regularly having to
  explain and justify it to upstream).

  Upstream also recommend doing a bootstrap build. We don't and can't do
  that right now.

  Folks upstream state at every opportunity that they don't care about
  standalone builds,
  e.g. 
https://discourse.llvm.org/t/rfc-do-something-with-the-subproject-tarballs-in-the-release-page/75024/2.

* Better support for LLVM as a system toolchain

  The current setup doesn't work well for people using LLVM as a system
  toolchain (because some of the components *must* be upgraded together),
  it doesn't work well for people who want to use mlir/flang/polly, and it
  doesn't work well for users on constrained hardware because we have to
  force on all targets. It also prohibits more optimisation, PGO, and
  bootstrapping it to test reliability.

  (This is why I'm not too sympathetic to claims that the monobuild is
  mostly for binary distributions, because we're actually *more*
  vulnerable to issues as a result of it being split when building from
  source if using the LLVM toolchain.)

* Maintaining older LLVMs

  It's easier for older LLVMs to be either maintained by somebody else
  (for e.g. Rust's purposes) or in an overlay if it's a single package
  rather than many.

  This is also true for e.g. testing old snapshots or keeping binary
  packages to downgrade to once they got cleaned up.

At the moment, I fear us choosing the non-recommended path gives us very
little, and causes a bunch of problems in return.

I don't particularly care much personally about using LLVM as the system
toolchain and if that's the only reason to move to monobuild, I'd be far
less worried about it (even if I think using it as a system toolchain is
reasonable for someone to work on).

But if it's getting in the way of packaging things like flang and it's
100% on us to make standalone work - and keep it working - is it really
worth it?

I will note that it seems like the breakage has reduced upstream in the
last few cycles, although the offload case was pretty bad just in the
last week. On the other hand, if you think the problems with mlir and
friends are easily-solvable, then we may be fine as we are?

IIRC, Violet ended up running into issues with getting the CMake to
properly handle both shared and static libraries to install with the
monobuild, so maybe LLVM's CMake is just broken in both directions and
we just have to pick whichever one is least-worst for us (which might be
the standalone builds as we do now)?

[0] https://github.com/gentoo/gentoo/pull/32716#issuecomment-1713689410
[1] 
https://lists.fedoraproject.org/archives/list/de...@lists.fedoraproject.org/thread/23QT5ISGGOQUOMTOOLMUBV6MSBB2C2C6/#PDAXH7PZTA7SFSSKF65KIXYLPYHAI3SS
[2] https://github.com/gentoo/gentoo/pull/32716

sam



Re: [gentoo-dev] [RFC] New categories for LLVM

2024-12-07 Thread Sam James
Michał Górny  writes:

> Hello,
>
> Given that the number of LLVM packages is growing, and probably will
> grow again (I'm introducing "offload" right now, expect at least MLIR
> soon, there are open requests for flang, polly...), I'd like to propose
> creating dedicated categories for these packages and moving them there.
>
> If not anything else, this will help consistently applying flags
> and keywords to these packages (`/etc/portage/package.*` accept
> wildcards).

I quite like the idea of having a category to ease keywording and testing.

>
> My initial idea would be to use two categories: one for the toolchain
> packages, another for runtimes, e.g.:
>
> llvm-core/
>   clang
>   clang-common
>   clang-runtime
>   clang-toolchain-symlinks
>   lld
>   lld-toolchain-symlinks
>   lldb
>   llvm
>   llvm-common
>   llvm-toolchain-symlinks
>   llvmgold
>
> llvm-runtimes/
>   compiler-rt
>   compiler-rt-sanitizers
>   libclc
>   libcxx
>   libcxxabi
>   libomp (-> openmp?)
>   llvm-offload (-> offload)
>   llvm-unwind (-> unwind?)
>

I'm not sure if I'm sold on *two*. What happens for stuff like mlir
where it's not a runtime but it's arguably more of one than core?

It just doesn't feel like the division works great. Or maybe it's just
because I feel like llvm-core will keep growing and llvm-runtimes won't.

> clang-python, lit and llvm-ocaml would remain in their language
> categories.
>
> WDYT?

I'm going to start another reply for a subthread.

thanks,
sam



Re: [gentoo-dev] Bugs in pitivi ebuild

2024-12-07 Thread Grand Duet
I emerged gsteamer with introspection use flag enabled and it did not help.

I have just looked into
https://ports.freebsd.org/cgi/ports.cgi?query=pitivi&stype=all&sektion=all
and found out that media-libs/gstreamer-editing-services are also a
dependency of pitivi
that was not added as such into the Gentoo pitivi ebuild.

> сб, 7 дек. 2024 г. в 19:17, Filip Kobierski :
> >
> > > Missing soft dependency:
> > > - cvtracker GStreamer element not found on the system
> > >-> enables object tracking
> >
> > I think could besatisfied by media-libs/gstreamer[introspection].
> > Also from what I can tell librosa is not packaged
> > in any of the overlays tracked by https://gpo.zugaina.org.
> > If you want it check out
> > https://forums.gentoo.org/viewtopic-t-1169251.html
> >
> > regards
> > fkobi
> >
> > On Saturday, December 7th, 2024 at 15:15, Grand Duet  
> > wrote:
> >
> > > In short: at least gsound dependency has not been mentioned in the
> > > pitivi ebuild.
> > >
> >
> > > The full story has been explained below.
> > >
> >
> > > I have recently installed pitivi and tried to run it.
> > > It failed with the following error message:
> > > # pitivi
> > > Missing soft dependency:
> > > - GSound not found on the system
> > > -> enables sound notifications when rendering is complete
> > >
> >
> > > Missing soft dependency:
> > > - cvtracker GStreamer element not found on the system
> > > -> enables object tracking
> > >
> >
> > > Missing soft dependency:
> > > - librosa not found on the system
> > > -> enables beat detection functionality
> > >
> >
> > > Traceback (most recent call last):
> > > File "/usr/lib64/pitivi/python/pitivi/application.py", line 205, in
> > > do_activate
> > > self.create_main_window()
> > > File "/usr/lib64/pitivi/python/pitivi/application.py", line 212, in
> > > create_main_window
> > > self.gui.setup_ui()
> > > File "/usr/lib64/pitivi/python/pitivi/mainwindow.py", line 135, in 
> > > setup_ui
> > > self.editor.setup_ui()
> > > File "/usr/lib64/pitivi/python/pitivi/editorperspective.py", line
> > > 103, in setup_ui
> > > self._create_ui()
> > > File "/usr/lib64/pitivi/python/pitivi/editorperspective.py", line
> > > 260, in _create_ui
> > > self.mainhpaned.set_position(self.settings.mainWindowMainHPanePosition)
> > > TypeError: Argument 1 does not allow None as a value
> > >
> >
> > > After installing gsound and gstreamer, the error message have been
> > > reduced to the following:
> > > Missing soft dependency:
> > > - cvtracker GStreamer element not found on the system
> > > -> enables object tracking
> > >
> >
> > > Missing soft dependency:
> > > - librosa not found on the system
> > > -> enables beat detection functionality
> > >
> >
> > > Traceback (most recent call last):
> > > File "/usr/lib64/pitivi/python/pitivi/application.py", line 205, in
> > > do_activate
> > > self.create_main_window()
> > > File "/usr/lib64/pitivi/python/pitivi/application.py", line 212, in
> > > create_main_window
> > > self.gui.setup_ui()
> > > File "/usr/lib64/pitivi/python/pitivi/mainwindow.py", line 135, in 
> > > setup_ui
> > > self.editor.setup_ui()
> > > File "/usr/lib64/pitivi/python/pitivi/editorperspective.py", line
> > > 103, in setup_ui
> > > self._create_ui()
> > > File "/usr/lib64/pitivi/python/pitivi/editorperspective.py", line
> > > 260, in _create_ui
> > > self.mainhpaned.set_position(self.settings.mainWindowMainHPanePosition)
> > > TypeError: Argument 1 does not allow None as a value
> > >
> >
> > > So, at least dependency of the pitivi on gsound has not been mentioned
> > > in the pitivi ebuild which is definitely a bug.
> > > Unfortunately, I do not know how to install librosa and get rid of the
> > > other error messages.
> > >
> >
> > > I am currently use default/linux/amd64/23.0/desktop (stable) profile with 
> > > dwm
> > > without any desktop environment and think that this bug has appeared 
> > > because
> > > the pitivi ebuild has been tested only for
> > > default/linux/amd64/23.0/desktop/gnome profile.



Re: [gentoo-dev] Bugs in pitivi ebuild

2024-12-07 Thread Sam James
Grand Duet  writes:

> I emerged gsteamer with introspection use flag enabled and it did not help.
>
> I have just looked into
> https://ports.freebsd.org/cgi/ports.cgi?query=pitivi&stype=all&sektion=all
> and found out that media-libs/gstreamer-editing-services are also a
> dependency of pitivi
> that was not added as such into the Gentoo pitivi ebuild.

But media-libs/gstreamer-editing-services is in the ebuild?

>
>> сб, 7 дек. 2024 г. в 19:17, Filip Kobierski :
>> >
>> > > Missing soft dependency:
>> > > - cvtracker GStreamer element not found on the system
>> > >-> enables object tracking
>> >
>> > I think could besatisfied by media-libs/gstreamer[introspection].
>> > Also from what I can tell librosa is not packaged
>> > in any of the overlays tracked by https://gpo.zugaina.org.
>> > If you want it check out
>> > https://forums.gentoo.org/viewtopic-t-1169251.html
>> >
>> > regards
>> > fkobi
>> >
>> > On Saturday, December 7th, 2024 at 15:15, Grand Duet 
>> >  wrote:
>> >
>> > > In short: at least gsound dependency has not been mentioned in the
>> > > pitivi ebuild.
>> > >
>> >
>> > > The full story has been explained below.
>> > >
>> >
>> > > I have recently installed pitivi and tried to run it.
>> > > It failed with the following error message:
>> > > # pitivi
>> > > Missing soft dependency:
>> > > - GSound not found on the system
>> > > -> enables sound notifications when rendering is complete
>> > >
>> >
>> > > Missing soft dependency:
>> > > - cvtracker GStreamer element not found on the system
>> > > -> enables object tracking
>> > >
>> >
>> > > Missing soft dependency:
>> > > - librosa not found on the system
>> > > -> enables beat detection functionality
>> > >
>> >
>> > > Traceback (most recent call last):
>> > > File "/usr/lib64/pitivi/python/pitivi/application.py", line 205, in
>> > > do_activate
>> > > self.create_main_window()
>> > > File "/usr/lib64/pitivi/python/pitivi/application.py", line 212, in
>> > > create_main_window
>> > > self.gui.setup_ui()
>> > > File "/usr/lib64/pitivi/python/pitivi/mainwindow.py", line 135, in 
>> > > setup_ui
>> > > self.editor.setup_ui()
>> > > File "/usr/lib64/pitivi/python/pitivi/editorperspective.py", line
>> > > 103, in setup_ui
>> > > self._create_ui()
>> > > File "/usr/lib64/pitivi/python/pitivi/editorperspective.py", line
>> > > 260, in _create_ui
>> > > self.mainhpaned.set_position(self.settings.mainWindowMainHPanePosition)
>> > > TypeError: Argument 1 does not allow None as a value
>> > >
>> >
>> > > After installing gsound and gstreamer, the error message have been
>> > > reduced to the following:
>> > > Missing soft dependency:
>> > > - cvtracker GStreamer element not found on the system
>> > > -> enables object tracking
>> > >
>> >
>> > > Missing soft dependency:
>> > > - librosa not found on the system
>> > > -> enables beat detection functionality
>> > >
>> >
>> > > Traceback (most recent call last):
>> > > File "/usr/lib64/pitivi/python/pitivi/application.py", line 205, in
>> > > do_activate
>> > > self.create_main_window()
>> > > File "/usr/lib64/pitivi/python/pitivi/application.py", line 212, in
>> > > create_main_window
>> > > self.gui.setup_ui()
>> > > File "/usr/lib64/pitivi/python/pitivi/mainwindow.py", line 135, in 
>> > > setup_ui
>> > > self.editor.setup_ui()
>> > > File "/usr/lib64/pitivi/python/pitivi/editorperspective.py", line
>> > > 103, in setup_ui
>> > > self._create_ui()
>> > > File "/usr/lib64/pitivi/python/pitivi/editorperspective.py", line
>> > > 260, in _create_ui
>> > > self.mainhpaned.set_position(self.settings.mainWindowMainHPanePosition)
>> > > TypeError: Argument 1 does not allow None as a value
>> > >
>> >
>> > > So, at least dependency of the pitivi on gsound has not been mentioned
>> > > in the pitivi ebuild which is definitely a bug.
>> > > Unfortunately, I do not know how to install librosa and get rid of the
>> > > other error messages.
>> > >
>> >
>> > > I am currently use default/linux/amd64/23.0/desktop (stable) profile 
>> > > with dwm
>> > > without any desktop environment and think that this bug has appeared 
>> > > because
>> > > the pitivi ebuild has been tested only for
>> > > default/linux/amd64/23.0/desktop/gnome profile.



[gentoo-dev] Re: Bugs in pitivi ebuild

2024-12-07 Thread Grand Duet
In FreeBSD both gsound and gstreamer are dependecies of the pitivi package.
See: https://ports.freebsd.org/cgi/ports.cgi?query=pitivi&stype=all&sektion=all

Why Gentoo pitivi ebuild is so special that these two packages are not
considered to be dependencies of pitivi?

сб, 7 дек. 2024 г. в 16:15, Grand Duet :
>
> In short: at least gsound dependency has not been mentioned in the
> pitivi ebuild.
>
> The full story has been explained below.
>
> I have recently installed pitivi and tried to run it.
> It failed with the following error message:
> # pitivi
> Missing soft dependency:
> - GSound not found on the system
> -> enables sound notifications when rendering is complete
> Missing soft dependency:
> - cvtracker GStreamer element not found on the system
> -> enables object tracking
> Missing soft dependency:
> - librosa not found on the system
> -> enables beat detection functionality
> Traceback (most recent call last):
>   File "/usr/lib64/pitivi/python/pitivi/application.py", line 205, in
> do_activate
> self.create_main_window()
>   File "/usr/lib64/pitivi/python/pitivi/application.py", line 212, in
> create_main_window
> self.gui.setup_ui()
>   File "/usr/lib64/pitivi/python/pitivi/mainwindow.py", line 135, in setup_ui
> self.editor.setup_ui()
>   File "/usr/lib64/pitivi/python/pitivi/editorperspective.py", line
> 103, in setup_ui
> self._create_ui()
>   File "/usr/lib64/pitivi/python/pitivi/editorperspective.py", line
> 260, in _create_ui
> self.mainhpaned.set_position(self.settings.mainWindowMainHPanePosition)
> TypeError: Argument 1 does not allow None as a value
>
> After installing gsound and gstreamer, the error message have been
> reduced to the following:
> Missing soft dependency:
> - cvtracker GStreamer element not found on the system
> -> enables object tracking
> Missing soft dependency:
> - librosa not found on the system
> -> enables beat detection functionality
> Traceback (most recent call last):
>   File "/usr/lib64/pitivi/python/pitivi/application.py", line 205, in
> do_activate
> self.create_main_window()
>   File "/usr/lib64/pitivi/python/pitivi/application.py", line 212, in
> create_main_window
> self.gui.setup_ui()
>   File "/usr/lib64/pitivi/python/pitivi/mainwindow.py", line 135, in setup_ui
> self.editor.setup_ui()
>   File "/usr/lib64/pitivi/python/pitivi/editorperspective.py", line
> 103, in setup_ui
> self._create_ui()
>   File "/usr/lib64/pitivi/python/pitivi/editorperspective.py", line
> 260, in _create_ui
> self.mainhpaned.set_position(self.settings.mainWindowMainHPanePosition)
> TypeError: Argument 1 does not allow None as a value
>
> So, at least dependency of the pitivi on gsound has not been mentioned
> in the pitivi ebuild which is definitely a bug.
> Unfortunately, I do not know how to install librosa and get rid of the
> other error messages.
>
> I am currently use default/linux/amd64/23.0/desktop (stable) profile with dwm
> without any desktop environment and think that this bug has appeared because
> the pitivi ebuild has been tested only for
> default/linux/amd64/23.0/desktop/gnome profile.



Re: [gentoo-dev] Bugs in pitivi ebuild

2024-12-07 Thread Sam James
Grand Duet  writes:

> I emerged gsteamer with introspection use flag enabled and it did not help.

and this is already a dependency too.

>
> I have just looked into
> https://ports.freebsd.org/cgi/ports.cgi?query=pitivi&stype=all&sektion=all
> and found out that media-libs/gstreamer-editing-services are also a
> dependency of pitivi
> that was not added as such into the Gentoo pitivi ebuild.
>
>> сб, 7 дек. 2024 г. в 19:17, Filip Kobierski :
>> >
>> > > Missing soft dependency:
>> > > - cvtracker GStreamer element not found on the system
>> > >-> enables object tracking
>> >
>> > I think could besatisfied by media-libs/gstreamer[introspection].
>> > Also from what I can tell librosa is not packaged
>> > in any of the overlays tracked by https://gpo.zugaina.org.
>> > If you want it check out
>> > https://forums.gentoo.org/viewtopic-t-1169251.html
>> >
>> > regards
>> > fkobi
>> >
>> > On Saturday, December 7th, 2024 at 15:15, Grand Duet 
>> >  wrote:
>> >
>> > > In short: at least gsound dependency has not been mentioned in the
>> > > pitivi ebuild.
>> > >
>> >
>> > > The full story has been explained below.
>> > >
>> >
>> > > I have recently installed pitivi and tried to run it.
>> > > It failed with the following error message:
>> > > # pitivi
>> > > Missing soft dependency:
>> > > - GSound not found on the system
>> > > -> enables sound notifications when rendering is complete
>> > >
>> >
>> > > Missing soft dependency:
>> > > - cvtracker GStreamer element not found on the system
>> > > -> enables object tracking
>> > >
>> >
>> > > Missing soft dependency:
>> > > - librosa not found on the system
>> > > -> enables beat detection functionality
>> > >
>> >
>> > > Traceback (most recent call last):
>> > > File "/usr/lib64/pitivi/python/pitivi/application.py", line 205, in
>> > > do_activate
>> > > self.create_main_window()
>> > > File "/usr/lib64/pitivi/python/pitivi/application.py", line 212, in
>> > > create_main_window
>> > > self.gui.setup_ui()
>> > > File "/usr/lib64/pitivi/python/pitivi/mainwindow.py", line 135, in 
>> > > setup_ui
>> > > self.editor.setup_ui()
>> > > File "/usr/lib64/pitivi/python/pitivi/editorperspective.py", line
>> > > 103, in setup_ui
>> > > self._create_ui()
>> > > File "/usr/lib64/pitivi/python/pitivi/editorperspective.py", line
>> > > 260, in _create_ui
>> > > self.mainhpaned.set_position(self.settings.mainWindowMainHPanePosition)
>> > > TypeError: Argument 1 does not allow None as a value
>> > >
>> >
>> > > After installing gsound and gstreamer, the error message have been
>> > > reduced to the following:
>> > > Missing soft dependency:
>> > > - cvtracker GStreamer element not found on the system
>> > > -> enables object tracking
>> > >
>> >
>> > > Missing soft dependency:
>> > > - librosa not found on the system
>> > > -> enables beat detection functionality
>> > >
>> >
>> > > Traceback (most recent call last):
>> > > File "/usr/lib64/pitivi/python/pitivi/application.py", line 205, in
>> > > do_activate
>> > > self.create_main_window()
>> > > File "/usr/lib64/pitivi/python/pitivi/application.py", line 212, in
>> > > create_main_window
>> > > self.gui.setup_ui()
>> > > File "/usr/lib64/pitivi/python/pitivi/mainwindow.py", line 135, in 
>> > > setup_ui
>> > > self.editor.setup_ui()
>> > > File "/usr/lib64/pitivi/python/pitivi/editorperspective.py", line
>> > > 103, in setup_ui
>> > > self._create_ui()
>> > > File "/usr/lib64/pitivi/python/pitivi/editorperspective.py", line
>> > > 260, in _create_ui
>> > > self.mainhpaned.set_position(self.settings.mainWindowMainHPanePosition)
>> > > TypeError: Argument 1 does not allow None as a value
>> > >
>> >
>> > > So, at least dependency of the pitivi on gsound has not been mentioned
>> > > in the pitivi ebuild which is definitely a bug.
>> > > Unfortunately, I do not know how to install librosa and get rid of the
>> > > other error messages.
>> > >
>> >
>> > > I am currently use default/linux/amd64/23.0/desktop (stable) profile 
>> > > with dwm
>> > > without any desktop environment and think that this bug has appeared 
>> > > because
>> > > the pitivi ebuild has been tested only for
>> > > default/linux/amd64/23.0/desktop/gnome profile.



[gentoo-dev] Re: LLVM build strategy

2024-12-07 Thread Sam James
Sam James  writes:

> Michał Górny  writes:
>
>> Hello,
>>
>> Given that the number of LLVM packages is growing, and probably will
>> grow again (I'm introducing "offload" right now, expect at least MLIR
>> soon, there are open requests for flang, polly...), I'd like to propose
>> creating dedicated categories for these packages and moving them there.
>>
>> [...]
>>
>> WDYT?
>
> I fear this sort of assumes we won't switch to monobuild any time soon.

(I'm not even sure I'm advocating a move, it's more that I'd like to
flesh out the arguments for it so I have it clear in my mind.)

>
> I keep thinking [0] about how sustainable our current setup is:
> * Fedora moved away from it for >=18 [1].
> * As we saw with offload, it broke a few times in just a week. So it's
>   only Gentoo who cares about this configuration AFAIK.
> * It's not working great if we're already not able to easily package
>   mlir, flang, or polly.
>
> Violet attempted working on a merge before [2].
>
> The trade-offs I see are:
> * Increased disk space requirement for building LLVM
>
>   I think that's a legitimate concern although perhaps not that big of a
>   deal given it is, after all, a whole toolchain. GCC takes quite a bit
>   to build too.
>
> * Build time
>
>   Build time for mono LLVM would increase as we're building more
>   components at least for some users.
>
>   But the time added by
>   building more components (see below) should be balanced out by ccache if
>   doing development and also, importantly, not needing to force on all
>   targets anymore (they keep growing).
>
>   The cumulative time should be the same (although maybe a bit less
>   given the targets change) given that most people need the
>   same set of components because of mesa, firefox, or other things which
>   need libclang.
>
> * Moving to an upstream supported-configuration
>
>   We may have a bit of pain in moving to it and getting used to it, but
>   we're no longer swimming upstream and being the only people caring
>   about our choice of build configuration (and regularly having to
>   explain and justify it to upstream).
>
>   Upstream also recommend doing a bootstrap build. We don't and can't do
>   that right now.
>
>   Folks upstream state at every opportunity that they don't care about
>   standalone builds,
>   e.g. 
> https://discourse.llvm.org/t/rfc-do-something-with-the-subproject-tarballs-in-the-release-page/75024/2.
>
> * Better support for LLVM as a system toolchain
>
>   The current setup doesn't work well for people using LLVM as a system
>   toolchain (because some of the components *must* be upgraded together),
>   it doesn't work well for people who want to use mlir/flang/polly, and it
>   doesn't work well for users on constrained hardware because we have to
>   force on all targets. It also prohibits more optimisation, PGO, and
>   bootstrapping it to test reliability.
>
>   (This is why I'm not too sympathetic to claims that the monobuild is
>   mostly for binary distributions, because we're actually *more*
>   vulnerable to issues as a result of it being split when building from
>   source if using the LLVM toolchain.)
>
> * Maintaining older LLVMs
>
>   It's easier for older LLVMs to be either maintained by somebody else
>   (for e.g. Rust's purposes) or in an overlay if it's a single package
>   rather than many.
>
>   This is also true for e.g. testing old snapshots or keeping binary
>   packages to downgrade to once they got cleaned up.

Another issue with monobuild would be handling the case where
USE-depends-on-other-USE and you either have a lot of REQUIRED_USE, or
USE that do nothing. This is more of an issue for the runtimes, I think.

We would also have an issue where we either have to default-enable
Clang, or require many, many users to enable it manually/via autounmask
(as opposed to now where it gets dragged in as a dep).

> [...]



Re: LLVM build strategy (was: Re: [gentoo-dev] [RFC] New categories for LLVM)

2024-12-07 Thread Violet Purcell
On Sun, Dec 08, 2024 at 04:53:58AM +, Sam James wrote:
> I fear this sort of assumes we won't switch to monobuild any time soon.
> 
> I keep thinking [0] about how sustainable our current setup is:
> * Fedora moved away from it for >=18 [1].
> * As we saw with offload, it broke a few times in just a week. So it's
>   only Gentoo who cares about this configuration AFAIK.
> * It's not working great if we're already not able to easily package
>   mlir, flang, or polly.
> 
> Violet attempted working on a merge before [2].

Hey! Thanks for bringing up my work on this. I've also been continuing
this work in an overlay which I use myself on some of my machines:
https://codeberg.org/vimproved/llvm-overlay.

> IIRC, Violet ended up running into issues with getting the CMake to
> properly handle both shared and static libraries to install with the
> monobuild, so maybe LLVM's CMake is just broken in both directions and
> we just have to pick whichever one is least-worst for us (which might be
> the standalone builds as we do now)?

LLVM's CMake is definitely just rather broken in many ways, yeah. The
biggest issue I experienced when working on this was with LLD libraries.
Currently, sys-devel/lld is built with -DBUILD_SHARED_LIBS=ON, which
builds some LLD libraries used by e.g. zig as shared libraries. However,
there's no way to do this in monobuild since passing
-DBUILD_SHARED_LIBS=ON is not desirable. The solution I settled on was
splitting a sys-libs/lld-libs package which builds LLD with
-DBUILD_SHARED_LIBS=ON and then only installs the shared libraries.

As for sys-libs/llvm-libs, this package is a result of me trying to
figure out a way to have a statically linked system clang (for
performance purposes) while still having LLVM functional as a library.
After a lot of sifting through CMake spaghetti I settled on splitting
the LLVM monobuild into two packages: sys-devel/llvm and
sys-libs/llvm-libs. sys-devel/llvm installs the toolchain parts of LLVM
(statically linked), and sys-libs/llvm-libs installs the library and
CMake parts of LLVM.

This overlay is just for my personal use, and I don't think the above
package split would be viable for ::gentoo however.

One more thing that wasn't mentioned here, LLVM monobuild would also
potentially unblock PGO/BOLT optimizations for clang, so overall it
could have some pretty good speed improvements for system LLVM toolchain
users.

- Violet



Re: [gentoo-dev] Bugs in pitivi ebuild

2024-12-07 Thread Sam James
Grand Duet  writes:

> In short: at least gsound dependency has not been mentioned in the
> pitivi ebuild.

Please file bugs at bugs.gentoo.org so they can be tracked
properly. Thanks.



[gentoo-dev] Bugs in pitivi ebuild

2024-12-07 Thread Grand Duet
In short: at least gsound dependency has not been mentioned in the
pitivi ebuild.

The full story has been explained below.

I have recently installed pitivi and tried to run it.
It failed with the following error message:
# pitivi
Missing soft dependency:
- GSound not found on the system
-> enables sound notifications when rendering is complete
Missing soft dependency:
- cvtracker GStreamer element not found on the system
-> enables object tracking
Missing soft dependency:
- librosa not found on the system
-> enables beat detection functionality
Traceback (most recent call last):
  File "/usr/lib64/pitivi/python/pitivi/application.py", line 205, in
do_activate
self.create_main_window()
  File "/usr/lib64/pitivi/python/pitivi/application.py", line 212, in
create_main_window
self.gui.setup_ui()
  File "/usr/lib64/pitivi/python/pitivi/mainwindow.py", line 135, in setup_ui
self.editor.setup_ui()
  File "/usr/lib64/pitivi/python/pitivi/editorperspective.py", line
103, in setup_ui
self._create_ui()
  File "/usr/lib64/pitivi/python/pitivi/editorperspective.py", line
260, in _create_ui
self.mainhpaned.set_position(self.settings.mainWindowMainHPanePosition)
TypeError: Argument 1 does not allow None as a value

After installing gsound and gstreamer, the error message have been
reduced to the following:
Missing soft dependency:
- cvtracker GStreamer element not found on the system
-> enables object tracking
Missing soft dependency:
- librosa not found on the system
-> enables beat detection functionality
Traceback (most recent call last):
  File "/usr/lib64/pitivi/python/pitivi/application.py", line 205, in
do_activate
self.create_main_window()
  File "/usr/lib64/pitivi/python/pitivi/application.py", line 212, in
create_main_window
self.gui.setup_ui()
  File "/usr/lib64/pitivi/python/pitivi/mainwindow.py", line 135, in setup_ui
self.editor.setup_ui()
  File "/usr/lib64/pitivi/python/pitivi/editorperspective.py", line
103, in setup_ui
self._create_ui()
  File "/usr/lib64/pitivi/python/pitivi/editorperspective.py", line
260, in _create_ui
self.mainhpaned.set_position(self.settings.mainWindowMainHPanePosition)
TypeError: Argument 1 does not allow None as a value

So, at least dependency of the pitivi on gsound has not been mentioned
in the pitivi ebuild which is definitely a bug.
Unfortunately, I do not know how to install librosa and get rid of the
other error messages.

I am currently use default/linux/amd64/23.0/desktop (stable) profile with dwm
without any desktop environment and think that this bug has appeared because
the pitivi ebuild has been tested only for
default/linux/amd64/23.0/desktop/gnome profile.



Re: [gentoo-dev] Bugs in pitivi ebuild

2024-12-07 Thread Filip Kobierski
> Missing soft dependency:
> - cvtracker GStreamer element not found on the system
>-> enables object tracking

I think could besatisfied by media-libs/gstreamer[introspection].
Also from what I can tell librosa is not packaged
in any of the overlays tracked by https://gpo.zugaina.org.
If you want it check out
https://forums.gentoo.org/viewtopic-t-1169251.html

regards
fkobi

On Saturday, December 7th, 2024 at 15:15, Grand Duet  
wrote:

> In short: at least gsound dependency has not been mentioned in the
> pitivi ebuild.
> 

> The full story has been explained below.
> 

> I have recently installed pitivi and tried to run it.
> It failed with the following error message:
> # pitivi
> Missing soft dependency:
> - GSound not found on the system
> -> enables sound notifications when rendering is complete
> 

> Missing soft dependency:
> - cvtracker GStreamer element not found on the system
> -> enables object tracking
> 

> Missing soft dependency:
> - librosa not found on the system
> -> enables beat detection functionality
> 

> Traceback (most recent call last):
> File "/usr/lib64/pitivi/python/pitivi/application.py", line 205, in
> do_activate
> self.create_main_window()
> File "/usr/lib64/pitivi/python/pitivi/application.py", line 212, in
> create_main_window
> self.gui.setup_ui()
> File "/usr/lib64/pitivi/python/pitivi/mainwindow.py", line 135, in setup_ui
> self.editor.setup_ui()
> File "/usr/lib64/pitivi/python/pitivi/editorperspective.py", line
> 103, in setup_ui
> self._create_ui()
> File "/usr/lib64/pitivi/python/pitivi/editorperspective.py", line
> 260, in _create_ui
> self.mainhpaned.set_position(self.settings.mainWindowMainHPanePosition)
> TypeError: Argument 1 does not allow None as a value
> 

> After installing gsound and gstreamer, the error message have been
> reduced to the following:
> Missing soft dependency:
> - cvtracker GStreamer element not found on the system
> -> enables object tracking
> 

> Missing soft dependency:
> - librosa not found on the system
> -> enables beat detection functionality
> 

> Traceback (most recent call last):
> File "/usr/lib64/pitivi/python/pitivi/application.py", line 205, in
> do_activate
> self.create_main_window()
> File "/usr/lib64/pitivi/python/pitivi/application.py", line 212, in
> create_main_window
> self.gui.setup_ui()
> File "/usr/lib64/pitivi/python/pitivi/mainwindow.py", line 135, in setup_ui
> self.editor.setup_ui()
> File "/usr/lib64/pitivi/python/pitivi/editorperspective.py", line
> 103, in setup_ui
> self._create_ui()
> File "/usr/lib64/pitivi/python/pitivi/editorperspective.py", line
> 260, in _create_ui
> self.mainhpaned.set_position(self.settings.mainWindowMainHPanePosition)
> TypeError: Argument 1 does not allow None as a value
> 

> So, at least dependency of the pitivi on gsound has not been mentioned
> in the pitivi ebuild which is definitely a bug.
> Unfortunately, I do not know how to install librosa and get rid of the
> other error messages.
> 

> I am currently use default/linux/amd64/23.0/desktop (stable) profile with dwm
> without any desktop environment and think that this bug has appeared because
> the pitivi ebuild has been tested only for
> default/linux/amd64/23.0/desktop/gnome profile.

signature.asc
Description: OpenPGP digital signature


[gentoo-dev] [RFC] New categories for LLVM

2024-12-07 Thread Michał Górny
Hello,

Given that the number of LLVM packages is growing, and probably will
grow again (I'm introducing "offload" right now, expect at least MLIR
soon, there are open requests for flang, polly...), I'd like to propose
creating dedicated categories for these packages and moving them there.

If not anything else, this will help consistently applying flags
and keywords to these packages (`/etc/portage/package.*` accept
wildcards).

My initial idea would be to use two categories: one for the toolchain
packages, another for runtimes, e.g.:

llvm-core/
  clang
  clang-common
  clang-runtime
  clang-toolchain-symlinks
  lld
  lld-toolchain-symlinks
  lldb
  llvm
  llvm-common
  llvm-toolchain-symlinks
  llvmgold

llvm-runtimes/
  compiler-rt
  compiler-rt-sanitizers
  libclc
  libcxx
  libcxxabi
  libomp (-> openmp?)
  llvm-offload (-> offload)
  llvm-unwind (-> unwind?)

clang-python, lit and llvm-ocaml would remain in their language
categories.

WDYT?

-- 
Best regards,
Michał Górny



signature.asc
Description: This is a digitally signed message part


Re: [gentoo-dev] Bugs in pitivi ebuild

2024-12-07 Thread Grand Duet
Thank you for information about librosa.

I emerged gsteamer with introspection use flag enabled.

сб, 7 дек. 2024 г. в 19:17, Filip Kobierski :
>
> > Missing soft dependency:
> > - cvtracker GStreamer element not found on the system
> >-> enables object tracking
>
> I think could besatisfied by media-libs/gstreamer[introspection].
> Also from what I can tell librosa is not packaged
> in any of the overlays tracked by https://gpo.zugaina.org.
> If you want it check out
> https://forums.gentoo.org/viewtopic-t-1169251.html
>
> regards
> fkobi
>
> On Saturday, December 7th, 2024 at 15:15, Grand Duet  
> wrote:
>
> > In short: at least gsound dependency has not been mentioned in the
> > pitivi ebuild.
> >
>
> > The full story has been explained below.
> >
>
> > I have recently installed pitivi and tried to run it.
> > It failed with the following error message:
> > # pitivi
> > Missing soft dependency:
> > - GSound not found on the system
> > -> enables sound notifications when rendering is complete
> >
>
> > Missing soft dependency:
> > - cvtracker GStreamer element not found on the system
> > -> enables object tracking
> >
>
> > Missing soft dependency:
> > - librosa not found on the system
> > -> enables beat detection functionality
> >
>
> > Traceback (most recent call last):
> > File "/usr/lib64/pitivi/python/pitivi/application.py", line 205, in
> > do_activate
> > self.create_main_window()
> > File "/usr/lib64/pitivi/python/pitivi/application.py", line 212, in
> > create_main_window
> > self.gui.setup_ui()
> > File "/usr/lib64/pitivi/python/pitivi/mainwindow.py", line 135, in setup_ui
> > self.editor.setup_ui()
> > File "/usr/lib64/pitivi/python/pitivi/editorperspective.py", line
> > 103, in setup_ui
> > self._create_ui()
> > File "/usr/lib64/pitivi/python/pitivi/editorperspective.py", line
> > 260, in _create_ui
> > self.mainhpaned.set_position(self.settings.mainWindowMainHPanePosition)
> > TypeError: Argument 1 does not allow None as a value
> >
>
> > After installing gsound and gstreamer, the error message have been
> > reduced to the following:
> > Missing soft dependency:
> > - cvtracker GStreamer element not found on the system
> > -> enables object tracking
> >
>
> > Missing soft dependency:
> > - librosa not found on the system
> > -> enables beat detection functionality
> >
>
> > Traceback (most recent call last):
> > File "/usr/lib64/pitivi/python/pitivi/application.py", line 205, in
> > do_activate
> > self.create_main_window()
> > File "/usr/lib64/pitivi/python/pitivi/application.py", line 212, in
> > create_main_window
> > self.gui.setup_ui()
> > File "/usr/lib64/pitivi/python/pitivi/mainwindow.py", line 135, in setup_ui
> > self.editor.setup_ui()
> > File "/usr/lib64/pitivi/python/pitivi/editorperspective.py", line
> > 103, in setup_ui
> > self._create_ui()
> > File "/usr/lib64/pitivi/python/pitivi/editorperspective.py", line
> > 260, in _create_ui
> > self.mainhpaned.set_position(self.settings.mainWindowMainHPanePosition)
> > TypeError: Argument 1 does not allow None as a value
> >
>
> > So, at least dependency of the pitivi on gsound has not been mentioned
> > in the pitivi ebuild which is definitely a bug.
> > Unfortunately, I do not know how to install librosa and get rid of the
> > other error messages.
> >
>
> > I am currently use default/linux/amd64/23.0/desktop (stable) profile with 
> > dwm
> > without any desktop environment and think that this bug has appeared because
> > the pitivi ebuild has been tested only for
> > default/linux/amd64/23.0/desktop/gnome profile.