On Fri, May 23, 2025 at 08:56:11PM +0200, Christian Kastner wrote: > I maintain llama.cpp and its underlying tensor library ggml. ggml has > various CPU and GPU backends that are heavily optimized, and include > various forms of dynamic dispatching. My goal for configuring the build > is to have something that maximally exploits CPU features on recent > hardware, whilst supporting everything down to the baseline. > > I started with amd64 and found a simple and effective solution: I have a > very recent CPU supporting up to AVX512, and using QEMU+kvm, I can > efficiently emulate older CPUs in a VM, and run tests there. > > Now I'd like to do the same with ARM, but I wonder how to best approach > this. > > The analogous solution would be to either get access to, or purchase a > system with an ARMv9 CPU, and use that to emulate older CPUs. However I > don't know how to best go about that. > > I see that Graviton4 is ARMv9, so that would be interesting, but I don't > know if they'll support nested kvm. I also looked into SBCs but the only > ARMv9 I found was the Radxa Orion O6, which was announced only months > ago, so I'm not sure that it would be well supported by Debian yet. > Though they seem to want to support EFI which in my eyes is a huge plus. > > I would appreciate any ideas or suggestions that might help me achieve > my goal. Most importantly, am I perhaps being too aggressive in looking > for ARMv9? Should I settle for something less recent?
Which older ARM chips are you hoping to emulate? If you mean doing what qemu+kvm can do on x64 where you can hide CPU feature flags to pretend to have an older CPU, I am not so sure you can do that with ARM. It does appear when searching that most 64bit arm chips do support 32 bit still, so perhaps it would work. I know at least a few of the high end server chips dropped 32 bit support though, but apparently it is not a common design choice yet. That Orion O6 does look pretty neat. It seems they even claim it comes with Debian although with a custom kernel installed and probably a few other extra packages added. -- Len Sorensen
