[AMD Official Use Only - AMD Internal Distribution Only] > -----Original Message----- > From: Igor Mammedov <imamm...@redhat.com> > Sent: Monday, December 16, 2024 9:40 AM > To: Luo, Zhigang <zhigang....@amd.com> > Cc: David Hildenbrand <da...@redhat.com>; qemu-devel@nongnu.org; > kra...@redhat.com > Subject: Re: [PATCH] hostmem-file: add the 'hmem' option > > On Tue, 10 Dec 2024 21:51:40 +0000 > "Luo, Zhigang" <zhigang....@amd.com> wrote: > > > [AMD Official Use Only - AMD Internal Distribution Only] > > > > > -----Original Message----- > > > From: David Hildenbrand <da...@redhat.com> > > > Sent: Tuesday, December 10, 2024 2:55 PM > > > To: Luo, Zhigang <zhigang....@amd.com>; qemu-devel@nongnu.org > > > Cc: kra...@redhat.com; Igor Mammedov <imamm...@redhat.com> > > > Subject: Re: [PATCH] hostmem-file: add the 'hmem' option > > > > > > On 10.12.24 20:32, Luo, Zhigang wrote: > > > > [AMD Official Use Only - AMD Internal Distribution Only] > > > > > > > > Hi David, > > > > > > > > > > Hi, > > > > > > >>> > > > >>> Thanks for your comments. > > > >>> Let me give you some background for this patch. > > > >>> I am currently engaged in a project that requires to pass the > > > >>> EFI_MEMORY_SP > > > >> (Special Purpose Memory) type memory from host to a virtual > > > >> machine within QEMU. This memory needs to be EFI_MEMORY_SP type > > > >> in the virtual machine as well. > > > >>> This particular memory type is essential for the functionality of my > > > >>> project. > > > >> > > > >> Which exact guest memory will be backed by this memory? All guest- > memory? > > > > [Luo, Zhigang] not all guest-memory. Only the memory reserved for > > > > specific > > > device. > > > > > > Can you show me an example QEMU cmdline, and how you would pass that > > > hostmem-file object to the device? > > > > > [Luo, Zhigang] the following is an example. m1 is the reserved memory for > > pci > device "0000:03:00.0". both the memory and pci device are set to same numa > node. > > > > -object memory-backend-ram,size=8G,id=m0 \ -object > > memory-backend-file,size=16G,id=m1,mem-path=/dev/dax0.0,prealloc=on,al > > ign=1G,hmem=on \ -numa node,nodeid=0,memdev=m0 -numa > > node,nodeid=1,memdev=m1 \ -device > > pxb-pcie,id=pcie.1,numa_node=1,bus_nr=2,bus=pcie.0 \ -device > > ioh3420,id=pcie_port1,bus=pcie.1,chassis=1 \ -device > > vfio-pci,host=0000:03:00.0,id=hostdev0,bus=pcie_port1 > > Is /dev/dax0.0 a part of host device 0000:03:00.0 that you pass-through to > guest > using vfio? > [Luo, Zhigang] from ACPI view, they are in same proximity domain and host device 0000:03:00.0 has closest distance to access /dev/dax0.0.
> > > > > > > > > > > >> > > > >> And, what is the guest OS going to do with this memory? > > > > [Luo, Zhigang] the device driver in guest will use this reserved memory. > > > > > > Okay, so just like CXL memory. > > > > > > > > > > >> > > > >> Usually, this SP memory (dax, cxl, ...) is not used as boot memory. > > > >> Like on a bare metal system, one would expect that only CXL > > > >> memory will be marked as special and put aside to the cxl driver, > > > >> such that the OS can boot on ordinary DIMMs, such that cxl can online > > > >> it etc. > > > >> > > > >> So maybe you would want to expose this memory using CXL-mem > > > >> device to the VM? Or a DIMM? > > > >> > > > >> I assume the alternative is to tell the VM on the Linux kernel > > > >> cmdline to set EFI_MEMORY_SP on this memory. I recall that there > > > >> is a way to > > > achieve that. > > > >> > > > > [Luo, Zhigang] I know this option. but it requires the end user to > > > > know where is the > > > memory location in guest side(start address, size). > > > > > > Right. > > > > > > > > > > > > > > >>> In Linux, the SPM memory will be claimed by hmem-dax driver by > > > >>> default. With > > > >> this patch I can use the following config to pass the SPM memory to > > > >> guest > VM. > > > >>> -object > > > >>> memory-backend-file,size=30G,id=m1,mem-path=/dev/dax0.0,prealloc > > > >>> =on, > > > >>> al > > > >>> ign=1G,hmem=on > > > >>> > > > >>> I was thinking to change the option name from "hmem" to "spm" to > > > >>> avoid > > > >> confusion. > > > >> > > > >> Likely it should be specified elsewhere, that you want specific > > > >> guest RAM ranges to be EFI_MEMORY_SP. For a DIMM, it could be a > > > >> property, similarly maybe for CXL- mem devices (no expert on that). > > > >> > > > >> For boot memory / machine memory it could be a machine property. > > > >> But I'll first have to learn which ranges you actually want to > > > >> expose that way, and what the VM will do with that information. > > > > [Luo, Zhigang] we want to expose the SPM memory reserved for specific > device. > > > And we will pass the SPM memory and the device to guest. Then the > > > device driver can use the SPM memory in guest side. > > > > > > Then the device driver should likely have a way to configure that, > > > not the memory backend. > > > > > > After all, the device driver will map it somehow into guest physical > > > address space (how?). > > > > > [Luo, Zhigang] from guest view, it's still system memory, but marked as > > SPM. So, > qemu will map the memory to guest physical address space. > > The device driver just claims to use the SPM memory in guest side. > > > > > > > > > >> > > > >>> > > > >>> Do you have any suggestions to achieve this more reasonable? > > > >> > > > >> The problem with qemu_ram_foreach_block() is that you would > > > >> indicate also DIMMs, virtio-mem, ... and even RAMBlocks that are > > > >> not even used for backing anything to the VM as EFI_MEMORY_SP, which is > wrong. > > > > [Luo, Zhigang] qemu_ram_foreach_block() will list all memory > > > > block, but in > > > pc_update_hmem_memory(), only the memory block with "hmem" flag will > > > be updated to SPM memory. > > > > > > Yes, but imagine a user passing such a memory backend to a > > > DIMM/virtio-mem/boot memory etc. It will have very undesired side effects. > > > > > [Luo, Zhigang] the user should know what he/she is doing when he/she set > > the flag > for the memory region. > > > > > > > -- > > > Cheers, > > > > > > David / dhildenb > >
