On Tue, May 20, 2025 at 3:31 PM Dave Airlie <[email protected]> wrote: > > On Wed, 21 May 2025 at 07:53, Rob Clark <[email protected]> wrote: > > > > On Tue, May 20, 2025 at 2:25 PM Dave Airlie <[email protected]> wrote: > > > > > > On Sat, 17 May 2025 at 02:20, Rob Clark <[email protected]> wrote: > > > > > > > > On Fri, May 16, 2025 at 2:01 AM Danilo Krummrich <[email protected]> > > > > wrote: > > > > > > > > > > On Thu, May 15, 2025 at 02:57:46PM -0700, Rob Clark wrote: > > > > > > On Thu, May 15, 2025 at 10:55 AM Danilo Krummrich <[email protected]> > > > > > > wrote: > > > > > > > Anyways, I don't agree with that. Even if you can tweak your > > > > > > > driver to not run > > > > > > > into trouble with this, we can't introduce a mode that violates > > > > > > > GOUVM's internal > > > > > > > lifetimes and subsequently fix it up with WARN_ON() or BUG_ON(). > > > > > > > > > > > > > > I still don't see a real technical reason why msm can't be > > > > > > > reworked to follow > > > > > > > those lifetime rules. > > > > > > > > > > > > The basic issue is that (a) it would be really awkward to have two > > > > > > side-by-side VM/VMA management/tracking systems. But in legacy > > > > > > mode, > > > > > > we have the opposite direction of reference holding. (But at the > > > > > > same > > > > > > time, don't need/use most of the features of gpuvm.) > > > > > > > > > > Ok, let's try to move this forward; I see three options (in order of > > > > > descending > > > > > preference): > > > > > > > > > > 1) Rework the legacy code to properly work with GPUVM. > > > > > 2) Don't use GPUVM for the legacy mode. > > > > > . > > > > > . > > > > > . > > > > > 3) Get an ACK from Dave / Sima to implement those workarounds for > > > > > MSM in > > > > > GPUVM. > > > > > > > > > > If you go for 3), the code introduced by those two patches should be > > > > > guarded > > > > > with a flag that makes it very clear that this is a workaround > > > > > specifically > > > > > for MSM legacy mode and does not give any guarantees in terms of > > > > > correctness > > > > > regarding lifetimes etc., e.g. DRM_GPUVM_MSM_LEGACY_QUIRK. > > > > > > > > I'm not even sure how #2 would work, other than just copy/pasta all of > > > > drm_gpuvm into msm, which doesn't really seem great. > > > > > > > > As for #1, even if I could get it to work, it would still be a lot > > > > more mmu map/unmap (like on every pageflip, vs the current state that > > > > the vma is kept around until the object is freed). For the > > > > non-VM_BIND world, there are advantages to the BO holding the ref to > > > > the VMA, rather than the other way around. Even at just a modest > > > > single layer 1080p the map takes ~.2ms and unmap ~.3ms (plus the unmap > > > > costs a tlbinv). So from that standpoint, #3 is the superior option. > > > > > > > > > > Before we get to #3, I'll need a bit more info here on why you have to > > > map/unmap the VMA on every pageflip. > > > > Previously we'd keep the VMA hanging around until the GEM obj is > > freed. But that can't work if the VMA (via the VM_BO) is holding a > > reference to the GEM obj. > > > > I was kinda thinking about keeping the VMA around until the handle is > > closed.. but that doesn't cover the dma-buf case (ie. when you > > re-import the dma-buf fd each frame.. I know android does this, unsure > > about other wsi's). > > > > > But actually I think 2 is the best option, I think in nouveau this is > > > where we ended up, we didn't modify the old submission paths at all > > > and kept the old bo/vm lifetimes. > > > > > > We just added completely new bind/exec ioctls and you can only use one > > > method once you've opened an fd. > > > > hmm, but that means tracking VMAs against a single BO differently.. > > which.. at least seems ugly.. > > I don't think it is if you already have the code to do that, and just > add gpuvm support in parallel. > > You also have to figure out that the world is moving towards Vulkan > for everything so any optimisations you've made for particular legacy > paths will need to be dealt with in the future picture anyways. > > But I'd rather not hack gpuvm into being something it isn't, if there > is a meaningful commonality in legacy bo/vm bindings across drivers, > we could create something new, but the ref counting and handling is > pretty fundamental to gpuvm architecture. > > There should only be two paths, legacy and gpuvm, and you shouldn't > ever be mixing them on a particular exec path, since you should only > have a vm per userspace fd, and can pick which way to use it the first > time someone calls it.
It's not as much about the exec path, as it is about making all the non-exec paths (like shrinker/residency) have to deal with two completely different things.. But I think I have figured out something workable. I add an extra refcnt per BO for the vma, incremented by userspace holding a gem handle, userspace holding a dma-buf fd, or (ofc) actual pin for scanout. When the refcount is above zero I defer teardown in the kms->vm until it drops to zero. It isn't _exactly_ the same as lazy VMA teardown when the BO is freed, but it is effectively the same thing. And whenever the vma_ref is greater than zero, the BO has something else holding a ref so the ref loop doesn't matter. If there is no userspace process holding a reference to the BO via handle or dma-buf fd, then it isn't going to be used again in a swapchain, so the difference btwn tearing down the VMA when the vma_ref drops to zero vs when the BO is freed doesn't amount to anything. It's a bit weird adding some extra mechanism specifically for the scanout vm, and maybe a bit uglier (depending on eye-of-beholder) than making gpuvm work in either way (since the latter was a pretty straightforward patch), but less ugly than having to parallel mechanisms. So if you _really_ don't like the WEAK_REF flag, I have a workable alternative that addresses the performance problems. BR, -R
