Bug#1003267: ITP: rocm-cmake -- AMD ROCm cmake modules
Package: wnpp Severity: wishlist Owner: ROCm Team X-Debbugs-Cc: debian-devel@lists.debian.org * Package name: rocm-cmake Version : 4.5.2 Upstream Author : AMD * URL : https://github.com/radeonopencompute/rocm-cmake * License : MIT/X Programming Lang: CMake Description : AMD ROCm cmake modules ROCM cmake modules provides cmake modules for common build tasks needed for the ROCM software stack. This will be maintained by the ROCm packaging team, under the debian-ai mailing list, and whose forge is at: https://salsa.debian.org/rocm-team .
Bug#1003270: ITP: rocm-smi-lib -- ROCm System Management Interface (ROCm SMI) Library
Package: wnpp Severity: wishlist Owner: ROCm Team X-Debbugs-Cc: debian-devel@lists.debian.org * Package name: rocm-smi-lib Version : 4.5.2 Upstream Author : AMD * URL : https://github.com/RadeonOpenCompute/rocm_smi_lib * License : NCSA Programming Lang: C++, Python, Cmake, C Description : ROCm System Management Interface (ROCm SMI) Library The ROCm System Management Interface Library, or ROCm SMI library, is part of the Radeon Open Compute ROCm software stack. It is a C library for Linux that provides a user space interface for applications to monitor and control GPU applications. This will be maintained by the ROCm packaging team, under the debian-ai mailing list, and whose forge is at: https://salsa.debian.org/rocm-team .
Fwd: ROCm installation [layout]
P.S. There was an issue on deb-devel and deb-mentors whose attachment limit
is 100kB, sending again to avoid confusion... you can find the image at:
https://lists.debian.org/debian-ai/2022/01/png1edBgW0YQi.png
ROCm is AMD compute software stack, a competitor to nvidia CUDA.
I have asked several times on debian-mentors IRC already, and on the
debian-ai
mailing list [1], but I hope that treating this topic systematically, rather
than incrementally, and on wide audience mailing lists rather than by chat
could benefit the packaging team, and maybe the stack aswell.
ROCm as a stack is still maturing and unifying, and it still feels
work-in-progress with regards to the build system. The glue of cmake is not
yet frozen, AMD is still improving on it.
"Native" Debian packages are starting to cover a significant portion of the
stack [2], and it would be great to figure out the installation topic
once and
for all.
The goal of this mail is to get feedback from the Debian community on
how such
a stack should be installed today.
The short-term objective is then to give this feedback to AMD with a
somewhat
"official recommendation for ROCm installation on Debian". Probably
through AMD
github? AMD asked for such feedback not too long ago to the packaging
team [3].
In case a "spec" is approved, it will not diffuse immediately to the current
4.5.2 packages from the team [4], as I doubt anyone in the packaging
team has
the energy to fight and patch the whole build system as it is now.
I see it more, as a short-term effort, of a way to align internally at
Debian
on how to install ROCm today. And second, as a mid-term benefit, we will
maybe
see the recommandations adopted and reflected in future AMD cmake code,
which
would then make packaging even easier.
The installation options and paths generally looked for by CMake
Lists/configs
are currently:
- various cmake project-specific flags for the install paths of the
components
HIP_CLANG_PATH, HIP_DEVICE_LIB_PATH, HIP_PATH, ROCM_PATH, ... see [5]
which
derives from Cordell Bloor's all-in-one install script [6]. The
exhaustive
list of whose I have a hard time finding [7], and AMD seems to be still
iterating on it.
- /opt/rocm as a default backup
I see at least three choices, and sub-decisions to be made:
- Multi-arch or not
nvidia toolkit supports aarch64 and a few others.
Cross-compiling ROCm from Debian could be interesting in a near-future.
- Nested or not
Other stacks and relatively important projects, such as postgresql or
llvm go
nested (there is a central /usr/lib/{llvm-13, postgresql} directory,
often with a sub ./bin, ...)
- Where to install machine-readable GPU code
There is at least 3 types of device-side (aka GPU) binary files -
.bc for bitcode,
.hsaco for HSA code object and
.co for code object.
Bitcode files are the machine readable form of the LLVM intermediate
representation. HSA (Heterogeneous System Architecture) and other
code object
files are AMD containers for GPU machine code. PostgreSQL does use llvm
bitcode files: since the install path is nested, they are in
/usr/lib/postgresql/14/lib/bitcode.
Since it is arch-independent in the sense of the CPU architecture, I have
been proposed that such code should reside in /usr/share.
What I tried to keep in mind is that:
- shared libraries should be easily discoverable in paths looked by
/etc/ld.so.conf
- there are only so much paths that cmake find_package in config mode
looks for [8].
I attached as an image a direct comparison between some arbitrary
combinations
of these decisions. The directories are bundled in the attached archive too.
- install_layout_proposal_v1 goes
multi-arch, flattened, and with GPU code in /usr/share
- install_layout_proposal_v2 goes
"ante-multi-arch", nested, and with GPU code in /usr/lib
You are welcome to correct the layout in these demo directories, and
ship them
back to the mailing list!
Taken together, the Filesystem Hierarchy Standard, the Debian policy and the
Multi-arch policy leave room, to my knowledge, for both of the above
proposals
to exist. If not formally, at least in the facts both install ways do exist,
for stacks of different scales.
Please forgive and correct any mistake that I made as I am still learning.
Best regards, Maxime
P.S. Thank you Andrey Rakhmatullin for the patience on IRC :)
[1] https://lists.debian.org/debian-ai/2021/12/msg00043.html
[2]
https://salsa.debian.org/rocm-team/community/team-project#packaging-progress
[3] https://lists.debian.org/debian-ai/2021/11/msg00053.html
[4] https://salsa.debian.org/rocm-team
[5] https://salsa.debian.org/rocm-team/rocrand/-/blob/master/debian/rules
[6] https://gist.github.com/cgmb/7cd9a481c42ce132b5d6420380becef3
[7] https://github.com/RadeonOpenCompute/ROCm/issues/1655
[8]
https://cmake.org/cmake/help/latest/command/find_package.html#config-mode-search-procedure
install_layout_proposals.tar
Description: Unix tar arc
Re: ROCm installation
On 1/12/22 20:17, M. Zhou wrote:
Hi,
Thanks for the updates.
On Wed, 2022-01-12 at 18:14 +0100, Maxime Chambonnet wrote:
"Native" Debian packages are starting to cover a significant portion of
the
stack [2], and it would be great to figure out the installation topic
The word "native" is ambiguous to a portion of developers as it may
also refer a native (debian/source/format) package.
For other readers: it's "offician debian package" in contrast to
"third-party debian packages by upstream.
on how to install ROCm today.
After skimming through the mail I realize what you actually meant
is the "ROCm file installation layout" right?
Yes, totally, I forgot a bit to extract myself from my point of view!
The installation options and paths generally looked for by CMake
Lists/configs
are currently:
- various cmake project-specific flags for the install paths of the
components
HIP_CLANG_PATH, HIP_DEVICE_LIB_PATH, HIP_PATH, ROCM_PATH, ... see
[5]
Headers and libraries should installed under the standard path,
so that the compiler and linker should be able to find them without
additional flags. Just install all stuff to /usr should be enough.
Currently for example rocm-hipamd installs to /usr/hip, and
lintian yells a lot. All to /usr is quite not clear enough.
- /opt/rocm as a default backup
There is no way for `/opt` as official debian package. If any component
breaks without any specific file under /opt, then it is a bug to fix.
Right!
I see at least three choices, and sub-decisions to be made:
- Multi-arch or not
nvidia toolkit supports aarch64 and a few others.
Cross-compiling ROCm from Debian could be interesting in a near-
future.
The rocm libraries and binary executables are architecture dependent.
Most of them should have Architecture: any in d/control.
Cross-compiling ROCm is not something worth being looked at IMHO.
ROCm targets on high performance computing. A hardware architecture
really capable of "high performance computing" can't be too weak
to compile ROCm itself.
That said, making the installation layout Multi-Arch aware is a
good practice. Most of the packages may have Multi-Arch: same
as long as they contain architecture-dependent files.
- Nested or not
Other stacks and relatively important projects, such as postgresql
or
llvm go
nested (there is a central /usr/lib/{llvm-13, postgresql} directory,
often with a sub ./bin, ...)
I did not understand this question. Do you mean something like
/usr/lib/rocm-{4.5.2,5.0.0},
or
/usr/lib/rocm-4.5.2/llvm ?
Rather the first, not sure I see a difference, in all cases, it looks
nested under "rocm-something" to me. And we further down agree
that nested is probably not the way.
- Where to install machine-readable GPU code
There is at least 3 types of device-side (aka GPU) binary files -
.bc for bitcode,
.hsaco for HSA code object and
.co for code object.
How are these files read by ROCm? Is there anything like
"PYTHONPATH" for the gpu code files? We should choose a
supported path compatible to debian policy.
There is a cmake flag / environment variable for now,
HIP_DEVICE_LIB_PATH :<
The current preferred layout is /usr/amdgcn/*.bc
BTW, are these files architecture-independent? Namely,
can arm64 and amd64 produce the exactly the same (e.g.
md5sum-identical) output?
I don't know, we discussed it last jitsi meeting and
I believe that no one tried yet :)
Bitcode files are the machine readable form of the LLVM intermediate
representation. HSA (Heterogeneous System Architecture) and other
code object
files are AMD containers for GPU machine code. PostgreSQL does use
llvm
bitcode files: since the install path is nested, they are in
/usr/lib/postgresql/14/lib/bitcode.
Since it is arch-independent in the sense of the CPU architecture, I
have
been proposed that such code should reside in /usr/share.
Nested layout for llvm and postgresql intends to allow multiple
versions of the software co-exist on the same system. For example,
llvm-{11,12,13} may be installed simultaneously on Debian.
We debian rocm team does not have so many contributors to support
multiple versions. Just do it the simplest way as we can.
The official repacked nvidia-cuda-toolkit is not relevant
to such nested layout.
Agreed
What I tried to keep in mind is that:
- shared libraries should be easily discoverable in paths looked by
/etc/ld.so.conf
- there are only so much paths that cmake find_package in config mode
looks for [8].
Shared objects from Multi-arch aware library packages should be
put at /usr/lib// as long as they are indended
for public usage.
Don't be misled by complicated setups such as llvm, postgresql or
the upstream non-standard installation path. In the standard setup
everything is likely becoming simpler. When you started to think
about ld.so.conf for a regular official debia
Re: Do we need to hide packages in NEW queue
On 1/31/22 10:35, Pirate Praveen wrote: On തി, ജനു 31 2022 at 10:07:32 രാവിലെ +0100 +0100, Stephan Lachnit wrote: On Sun, Jan 30, 2022 at 8:35 PM Russ Allbery wrote: I do think that the amount of effort that the project puts into this pre-screening is of sufficiently high magnitude that it would be worth paying a lawyer for a legal opinion about whether or not we need to do it. The savings to the project if we found out that we didn't, or that we could do something simpler and more easily automated, would be more than the cost of the legal opinion. +1 Looking at the last financial numbers I found [1], we have at least ~750k USD we could use for this purpose. I don't really know how expensive lawyers are, but I doubt that this would cost more than 10k. Heck, we could even hire two lawyers without any significant financial impact (maybe in the US and EU as I think these are probably the most prominent areas for potential copyright lawsuits), even if it costs 50k. IMHO that would be totally worth it. And instead of investing scarce man-hours into pre-screening, we could create a money pool for financial support in case there is a copyright lawsuit. The pre-screening in NEW doesn't prevent someone from claiming copyright infringement anyway, there is just a smaller chance that the lawsuit is justified. But sadly even winning a lawsuit can still cost a significant amount of money. I agree. We should get real lawyers involved, pay and settle this issue >once and for all. As a maintainer who maintains a large number of >packages, NEW queue is big friction point for me personally and I'd be >very happy to see a solution for it, other than the status quo. Even >if the status quo is correct, I'd like this to be backed by a legal >opinion that we can rely on. Is there any precedent of a lawsuit against Debian due to copyrighted content in its archives? The gross intellectual property theft, Oracle sources found somewhere, Oodle compression applying for sid... will likely not even pass NEW in any case, extensive pre-screening or not. While I am sure that helping one of the big four consulting firms, or Mazars, make a living, will not encounter particular difficulties from them; there surely can be found resources in the association and political landscapes, which will at least widen the discussion as to where to take advise from? On different scales, I see at least: ## French scope -CNIL - state entity [1] -APRIL - notable association [2] - ap...@april.org -Quadrature du net - notable association [3] ## EU scope -There was a man whom helped pass GDPR with Margrethe Vestager, was it Mathias Vermeulen? [4] -CCBE - The voice of European Lawyers -Reach to the Commission or Parliament directly? ## Global scope -GNU foundation -Linux foundation Ultimately, Debian is not bound to a particular territory? United Nations and its satellites [5]could be a relevant scope for inquiries. Thank you to everyone involved for trying to strike the right balance, between archives being a haven of quality and free software, and following the crazy pace of software complexity. Best regards, Maxime [1]https://www.cnil.fr/ [2]https://listes.april.org/wws?pk_vid=ead171ca7a6f2a4a16436549595cd1f6 [3]https://www.laquadrature.net/about/ [4]https://www.awo.agency/about/mathias-vermeulen/ [5]https://unctad.org/system/files/official-document/ecosoc_res_2021d30_Note_OpenSource_en.pdf
Re: Embedded buildpath via rpath using cmake
I don't know really where to add this bit of information in the discussion, so I am taking the latest leaf. The rpath trend, with or without cmake, is likely to continue as it is being reborn by spack, nix and guix. The points made by smcv about arbitrary code execution and reproducibility are interesting: could these be added to the "rpath issue" wiki entry? The arguments currently in there hold in my opinion little value. I am reaching out on spack's slack to try and discuss the former arguments. BR, Maxime [1] https://wiki.debian.org/RpathIssue
