Re: [PATCH 2/2] drm/amdgpu: Synchronize after mapping into a compute VM

[Chen, Xiaogang]

On 2/24/2023 5:36 PM, Felix Kuehling wrote:
Caution: This message originated from an External Source. Use proper caution 
when opening attachments, clicking links, or responding.


Compute VMs use user mode queues for command submission. They cannot use
a CS ioctl to synchronize with pending PTE updates and flush TLBs. Do
this synchronization in amdgpu_gem_va_ioctl for compute VMs.

Signed-off-by: Felix Kuehling <felix.kuehl...@amd.com>
---
  drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c | 16 ++++++++++++++--
  1 file changed, 14 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
index 6936cd63df42..7de5057c40ec 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
@@ -601,7 +601,7 @@ int amdgpu_gem_metadata_ioctl(struct drm_device *dev, void 
*data,
  static void amdgpu_gem_va_update_vm(struct amdgpu_device *adev,
                                     struct amdgpu_vm *vm,
                                     struct amdgpu_bo_va *bo_va,
-                                   uint32_t operation)
+                                   uint32_t operation, uint32_t flags)
  {
         int r;

@@ -620,6 +620,18 @@ static void amdgpu_gem_va_update_vm(struct amdgpu_device 
*adev,
         }

         r = amdgpu_vm_update_pdes(adev, vm, false);
+       if (r)
+               goto error;
+
+       if (vm->is_compute_context) {
+               if (bo_va->last_pt_update)
+                       r = dma_fence_wait(bo_va->last_pt_update, true);
+               if (!r && vm->last_update)
+                       r = dma_fence_wait(vm->last_update, true);
+               if (!r)
+                       r = amdgpu_amdkfd_flush_tlb(adev, vm,
+                           TLB_FLUSH_LEGACY);

When kfd flush its vm tlb it uses TLB_FLUSH_LEGACY after map and 
TLB_FLUSH_HEAVYWEIGHT after unmap. Here use amdgpu_gen_va_ioctl to 
map/unmap buffer in kfd vm. Should we keep consistent with kfd:

use TLB_FLUSH_LEGACY if args->operation is AMDGPU_VA_OP_MAP, uses 
TLB_FLUSH_HEAVYWEIGHT if args->operation is AMDGPU_VA_OP_UNMAP?

+       }

  error:
         if (r && r != -ERESTARTSYS)
@@ -789,7 +801,7 @@ int amdgpu_gem_va_ioctl(struct drm_device *dev, void *data,
         }
         if (!r && !(args->flags & AMDGPU_VM_DELAY_UPDATE) && !amdgpu_vm_debug)
                 amdgpu_gem_va_update_vm(adev, &fpriv->vm, bo_va,
-                                       args->operation);
+                                       args->operation, args->flags);

  error_backoff:
         ttm_eu_backoff_reservation(&ticket, &list);
--
2.34.1