Hello,
This series adds virtio-gpu userptr support for ROCm native compute
contexts. The guest kernel pins an existing userspace mapping with
FOLL_LONGTERM and exposes it to the host as ordinary CREATE_BLOB
backing entries, avoiding a second shmem allocation and memcpy.
A non-zero userptr field selects that path. Probe
VIRTGPU_PARAM_USERPTR before using the field so an older guest kernel
rejects the request instead of silently creating a shmem blob.
GPU-readonly backing is requested with the virtio CREATE_BLOB wire
flag VIRTIO_GPU_BLOB_FLAG_USE_READONLY (also
VIRTGPU_BLOB_FLAG_USE_READONLY). The device MUST NOT write such a
resource. The guest accepts that flag and pins without FOLL_WRITE
only when the device advertised VIRTIO_GPU_F_BLOB_READONLY
(VIRTGPU_PARAM_BLOB_READONLY). Otherwise CREATE_BLOB fails, matching
the CROSS_DEVICE host-capability check.
Patches overview:
1. Add VIRTIO_GPU_CAPSET_ROCM capability for compute workloads
2. Extend DRM/virtio UAPI with userptr, PARAM_USERPTR,
USE_READONLY, and F_BLOB_READONLY
3. Implement core userptr functionality with page management
4. Wire blob ioctl creation to userptr objects and probe the
readonly feature
Tests:
- Full OPENCL CTS tests passed on ROCm 5.7.0 in V2000 platform.
- Near 70% percentage of OPENCL CTS tests passed on ROCm 7.0 W7900 platform.
- most HIP catch tests passed on ROCm 7.0 W7900 platform.
- Some AI applications enabled on ROCm 7.0 W7900 platform.
- latest ROCm 7.14 and ROCm 10 testing is ongoing.
V8 changes:
- Drop guest-only USERPTR ioctl flags; a non-zero userptr field
selects the path
- Add VIRTGPU_PARAM_USERPTR so older kernels do not silently
ignore the new field
- Request GPU-readonly backing with virtio wire flag USE_READONLY
- Accept USE_READONLY and omit FOLL_WRITE only if the host
advertised VIRTIO_GPU_F_BLOB_READONLY
- Restore userptr blobs after hibernation without the shmem path
V7 changes:
- Mask guest-only DRM flags out of CREATE_BLOB wire blob_flags
- Clear userptr->pages after pin failure to avoid double-free
- DMA-map userptr SG only when virtio_gpu_use_dma_api() is required
- Use DMA_TO_DEVICE for readonly blobs
- Sync userptr SG for the device on TRANSFER_TO_HOST
- Mark writable pages dirty when unpinning
- Reject USERPTR unless blob_mem is VIRTGPU_BLOB_MEM_GUEST
- Disallow PRIME export of userptr objects
- Note that CAPSET_ROCM uses ID 8 because ID 7 is taken by VIRCL
V6 changes:
- Rebase onto drm-misc-next
- Keep USE_USERPTR / USERPTR_RDONLY as guest-only DRM flags; drop the
virtio wire-header patch (5 patches down to 4)
- Fix userptr lifetime, DMA mapping, memlock accounting, alignment
checks, and PRIME SG export
- Updated corresponding cover letter and commit messages
V5 changes:
- Add VIRTIO_GPU_BLOB_FLAG_USERPTR_RDONLY definition to patch 2
- Dropped unused VIRTIO_GPU_F_RESOURCE_USERPTR feature bit in patch 2
- Included VIRTIO_GPU_BLOB_FLAG_USERPTR_RDONLY in
VIRTGPU_BLOB_FLAG_USE_MASK in patch 5
- Add check for userptr feature in patch 5 before creating userptr blob
resource
- Updated corresponding cover letter and commit messages
V4 changes:
- Renamed VIRTIO_GPU_CAPSET_HSAKMT to VIRTIO_GPU_CAPSET_ROCM
- Remove userptr feature probing cause it can reuse the guest
blob resource code path, reduce patch count from 6 to 5
- Updated corresponding commit messages
- Consolidated userptr feature detection in final patch
- Update corresponding cover letter content
V3 changes:
- Split into focused patches for easier review
- Removed complex interval tree userptr management
- Simplified resource creation without deduplication
- Added VIRTGPU_PARAM_RESOURCE_USERPTR for feature detection
- Improved UAPI documentation and error handling
- Enhanced code quality with proper cleanup paths
- Removed MMU notifier dependencies for simplicity
- Fixed resource lifecycle management issues
V2: - Split add HSAKMT context and blob userptr resource to two patches.
- Remove MMU notifier related patches, cause use not moveable user space
memory with MMU notifier is not a good idea.
- Remove HSAKMT context check when create context, let all the context
support the userptr feature.
- Remove MMU notifier related content in cover letter.
- Add more comments for patch 6 in cover letter.
Previous version:
https://lore.kernel.org/dri-devel/[email protected]/
Honglei Huang (4):
drm/virtio-gpu: Add VIRTIO_GPU_CAPSET_ROCM capability
drm/virtgpu api: add blob userptr resource
drm/virtio: implement userptr support for zero-copy memory access
drm/virtio: wire blob ioctl creation to userptr objects
drivers/gpu/drm/virtio/Makefile | 3 +-
drivers/gpu/drm/virtio/virtgpu_debugfs.c | 1 +
drivers/gpu/drm/virtio/virtgpu_drv.h | 43 +++
drivers/gpu/drm/virtio/virtgpu_ioctl.c | 35 ++-
drivers/gpu/drm/virtio/virtgpu_kms.c | 8 +-
drivers/gpu/drm/virtio/virtgpu_object.c | 22 ++
drivers/gpu/drm/virtio/virtgpu_userptr.c | 365 +++++++++++++++++++++++
drivers/gpu/drm/virtio/virtgpu_vq.c | 33 +-
include/uapi/drm/virtgpu_drm.h | 13 +
include/uapi/linux/virtio_gpu.h | 7 +
10 files changed, 513 insertions(+), 17 deletions(-)
create mode 100644 drivers/gpu/drm/virtio/virtgpu_userptr.c
base-commit: 766bfba0f3bae329f99b42dcabc3ef11fa368f0b
--
2.34.1
