On 10/18/24 15:49, Zhi Wang wrote:
On 17/10/2024 19.57, Cédric Le Goater wrote:
External email: Use caution opening links or attachments
Hello,
On 5/18/23 04:45, Ira Weiny wrote:
Type 2 devices are not yet a reality. Developing core kernel support
is difficult without some test device to model against.
Define a type 2 device 'cxl-accel'. This device is derived from the
type 3 device and retains all that functionality for now.
Mock up a couple of accelerator features (Back Invalidate [BI] and
Unordered IO [UIO]) as examples for the RFC. These have no
functionality other than to report the features as present for software
to key off of.
Defining these devices in qemu can be done with the following example:
...
-device
cxl-accel,bus=sw0p0,volatile-memdev=cxl-ac-mem5,id=cxl-dev5,sn=0xCAFE0005
...
NOTE: I'm leaving off Michael Tsirkin for now because this is really
rough and I'm mainly sending it out because it was talked about in the
CXL community call on 5/16.
Not-Yet-Signed-off-by: Ira Weiny <ira.we...@intel.com>
A recent proposal to add support in VFIO for CXL passthrough uses
this emulated device and a sample driver for a proof of concept.
For this effort, it would be very useful to have a QEMU model for
a CXL type2 device, even partially implemented.
I haven't found any updates of this series. What are the plans for
upstream today ?
I was discussing this with Ira and Jonathan in the LPC and CXL discord
groups. (Irq and Jonathan, please feel free to correct me) I think we
all thought that having the emulated device support in QEMU and a sample
CXL type-2 driver in the upstream kernel could be valuable for
1) people who wish to contribute and propose changes, like refactor, and
code tweaks related to CXL type-2 support in the kernel. They can
validate their patches to check if they break something without a CXL
type-2 device.
2) people who wish to contribute on solving problems of CXL type-2
generic code, e.g. loading sequences of modules... They can get involved
without a real HW.
My gut feeling is I can start to work with folks to get it into the
mainline together with the sample driver when CXL type-2 support is
merged. So people can play with it.
I did use an emulated Type2 device for my initial development and I'm
still using it for wider testing. It is basically same than the Type3
with small changes. I think a proper solution should include command
line options for configuring the device with flexibility or maybe
referring to a file with that specific configuration. Note there exist
optional functionalities like HDM decoder, and CXL.cache will need such
a flexibility as well.
The RFC contains the driver for such emulated device implemented inside
the tools/testing/cxl directory, although it has changed since then, but
happy to share it. It is almost equal to the code inside efx_cxl.c along
with the functions/macros for defining the driver.
FWIW, although I'm working on Type2 support, I really think qemu could
help for development and testing complex CXL infrastructures, more for
memory expanders aka Type3 than Type2. I know this requires a good
effort but IMO, it will pay off.
3) people who would like to try on vfio-cxl.
What would be nice to include in the patchset V1 in my mind:
(Ira and Jonathan and folks please feel free to comment)
Must have:
- HDM decoder support (>1 HDM decoder). (which I have done it in my
tree, so the CXL core can create a CXL region)
Nice to have:
- CXLType2 device system reset handler. As what mentioned in my cover
letter, a system reset handler is required to reset the device state.
Linux kernel CXL core assumes the HW is pre-configured by FW if it sees
CXL.mem is enabled when enumerating the device. So I have to kill QEMU
instead of resetting the virtual machine when debugging.
- CXLType2 device in the patch should be derived from PCIDevice
(current it is derived from the CXLType3 device, which carries quite
some unnecessary stuff to present to the guest)
- Refactor of the current QEMU FWMS code to deliver the guest access
to the virtual memory backend to the correct emulated CXL device (which
is currently hardcoded to connect to cxl-type3 device in QEMU. Without
this, the access to the CXL region from the guest seems problematic but
creating CXL region is still fine.)
Thanks,
Zhi.
For vfio-cxl, see :
* [RFC] vfio: introduce vfio-cxl to support CXL type-2 accelerator
passthrough
https://lore.kernel.org/kvm/20240920223446.1908673-1-z...@nvidia.com
* [RFC] Introduce vfio-cxl to support CXL type-2 device passthrough
https://lore.kernel.org/all/20240921071440.1915876-1-z...@nvidia.com/
Thanks,
C.
---
Ira Weiny (5):
hw/cxl: Use define for build bug detection
hw/cxl: Refactor component register initialization
hw/cxl: Derive a CXL accelerator device from Type-3
hw/cxl/accel: Add Back-Invalidate decoder capbility structure
hw/cxl: Add UIO HDM decoder register fields
docs/system/devices/cxl.rst | 11 ++++++
hw/cxl/cxl-component-utils.c | 80
+++++++++++++++++++-----------------------
hw/mem/cxl_type3.c | 39 ++++++++++++++++++++
include/hw/cxl/cxl_component.h | 51 +++++++++++++++++++--------
include/hw/cxl/cxl_device.h | 16 +++++++++
include/hw/pci/pci_ids.h | 1 +
6 files changed, 141 insertions(+), 57 deletions(-)
---
base-commit: 8eb2a03258313f404ca0c8609a8f9009b9b4318c
change-id: 20230517-rfc-type2-dev-c2d661a29d96
Best regards,
