Re: [Mesa-dev] Plumbing explicit synchronization through the Linux ecosystem
Could you elaborate. If there's something missing from my mental model of how implicit sync works, I'd like to have it corrected. People continue claiming that AMD is somehow special but I have yet to grasp what makes it so. (Not that anyone has bothered to try all that hard to explain it.) --Jason On March 13, 2020 21:03:21 Marek Olšák wrote: There is no synchronization between processes (e.g. 3D app and compositor) within X on AMD hw. It works because of some hacks in Mesa. Marek On Wed, Mar 11, 2020 at 1:31 PM Jason Ekstrand wrote: All, Sorry for casting such a broad net with this one. I'm sure most people who reply will get at least one mailing list rejection. However, this is an issue that affects a LOT of components and that's why it's thorny to begin with. Please pardon the length of this e-mail as well; I promise there's a concrete point/proposal at the end. Explicit synchronization is the future of graphics and media. At least, that seems to be the consensus among all the graphics people I've talked to. I had a chat with one of the lead Android graphics engineers recently who told me that doing explicit sync from the start was one of the best engineering decisions Android ever made. It's also the direction being taken by more modern APIs such as Vulkan. ## What are implicit and explicit synchronization? For those that aren't familiar with this space, GPUs, media encoders, etc. are massively parallel and synchronization of some form is required to ensure that everything happens in the right order and avoid data races. Implicit synchronization is when bits of work (3D, compute, video encode, etc.) are implicitly based on the absolute CPU-time order in which API calls occur. Explicit synchronization is when the client (whatever that means in any given context) provides the dependency graph explicitly via some sort of synchronization primitives. If you're still confused, consider the following examples: With OpenGL and EGL, almost everything is implicit sync. Say you have two OpenGL contexts sharing an image where one writes to it and the other textures from it. The way the OpenGL spec works, the client has to make the API calls to render to the image before (in CPU time) it makes the API calls which texture from the image. As long as it does this (and maybe inserts a glFlush?), the driver will ensure that the rendering completes before the texturing happens and you get correct contents. Implicit synchronization can also happen across processes. Wayland, for instance, is currently built on implicit sync where the client does their rendering and then does a hand-off (via wl_surface::commit) to tell the compositor it's done at which point the compositor can now texture from the surface. The hand-off ensures that the client's OpenGL API calls happen before the server's OpenGL API calls. A good example of explicit synchronization is the Vulkan API. There, a client (or multiple clients) can simultaneously build command buffers in different threads where one of those command buffers renders to an image and the other textures from it and then submit both of them at the same time with instructions to the driver for which order to execute them in. The execution order is described via the VkSemaphore primitive. With the new VK_KHR_timeline_semaphore extension, you can even submit the work which does the texturing BEFORE the work which does the rendering and the driver will sort it out. The #1 problem with implicit synchronization (which explicit solves) is that it leads to a lot of over-synchronization both in client space and in driver/device space. The client has to synchronize a lot more because it has to ensure that the API calls happen in a particular order. The driver/device have to synchronize a lot more because they never know what is going to end up being a synchronization point as an API call on another thread/process may occur at any time. As we move to more and more multi-threaded programming this synchronization (on the client-side especially) becomes more and more painful. ## Current status in Linux Implicit synchronization in Linux works via a the kernel's internal dma_buf and dma_fence data structures. A dma_fence is a tiny object which represents the "done" status for some bit of work. Typically, dma_fences are created as a by-product of someone submitting some bit of work (say, 3D rendering) to the kernel. The dma_buf object has a set of dma_fences on it representing shared (read) and exclusive (write) access to the object. When work is submitted which, for instance renders to the dma_buf, it's queued waiting on all the fences on the dma_buf and and a dma_fence is created representing the end of said rendering work and it's installed as the dma_buf's exclusive fence. This way, the kernel can manage all its internal queues (3D rendering, display, video encode, etc.) and know which things to submit in what order. For the last few years, we've h
Re: [Mesa-dev] Plumbing explicit synchronization through the Linux ecosystem
The synchronization works because the Mesa driver waits for idle (drains the GFX pipeline) at the end of command buffers and there is only 1 graphics queue, so everything is ordered. The GFX pipeline runs asynchronously to the command buffer, meaning the command buffer only starts draws and doesn't wait for completion. If the Mesa driver didn't wait at the end of the command buffer, the command buffer would finish and a different process could start execution of its own command buffer while shaders of the previous process are still running. If the Mesa driver submits a command buffer internally (because it's full), it doesn't wait, so the GFX pipeline doesn't notice that a command buffer ended and a new one started. The waiting at the end of command buffers happens only when the flush is external (Swap buffers, glFlush). It's a performance problem, because the GFX queue is blocked until the GFX pipeline is drained at the end of every frame at least. So explicit fences for SwapBuffers would help. Marek On Sun., Mar. 15, 2020, 22:49 Jason Ekstrand, wrote: > Could you elaborate. If there's something missing from my mental model of > how implicit sync works, I'd like to have it corrected. People continue > claiming that AMD is somehow special but I have yet to grasp what makes it > so. (Not that anyone has bothered to try all that hard to explain it.) > > > --Jason > > On March 13, 2020 21:03:21 Marek Olšák wrote: > >> There is no synchronization between processes (e.g. 3D app and >> compositor) within X on AMD hw. It works because of some hacks in Mesa. >> >> Marek >> >> On Wed, Mar 11, 2020 at 1:31 PM Jason Ekstrand >> wrote: >> >>> All, >>> >>> Sorry for casting such a broad net with this one. I'm sure most people >>> who reply will get at least one mailing list rejection. However, this >>> is an issue that affects a LOT of components and that's why it's >>> thorny to begin with. Please pardon the length of this e-mail as >>> well; I promise there's a concrete point/proposal at the end. >>> >>> >>> Explicit synchronization is the future of graphics and media. At >>> least, that seems to be the consensus among all the graphics people >>> I've talked to. I had a chat with one of the lead Android graphics >>> engineers recently who told me that doing explicit sync from the start >>> was one of the best engineering decisions Android ever made. It's >>> also the direction being taken by more modern APIs such as Vulkan. >>> >>> >>> ## What are implicit and explicit synchronization? >>> >>> For those that aren't familiar with this space, GPUs, media encoders, >>> etc. are massively parallel and synchronization of some form is >>> required to ensure that everything happens in the right order and >>> avoid data races. Implicit synchronization is when bits of work (3D, >>> compute, video encode, etc.) are implicitly based on the absolute >>> CPU-time order in which API calls occur. Explicit synchronization is >>> when the client (whatever that means in any given context) provides >>> the dependency graph explicitly via some sort of synchronization >>> primitives. If you're still confused, consider the following >>> examples: >>> >>> With OpenGL and EGL, almost everything is implicit sync. Say you have >>> two OpenGL contexts sharing an image where one writes to it and the >>> other textures from it. The way the OpenGL spec works, the client has >>> to make the API calls to render to the image before (in CPU time) it >>> makes the API calls which texture from the image. As long as it does >>> this (and maybe inserts a glFlush?), the driver will ensure that the >>> rendering completes before the texturing happens and you get correct >>> contents. >>> >>> Implicit synchronization can also happen across processes. Wayland, >>> for instance, is currently built on implicit sync where the client >>> does their rendering and then does a hand-off (via wl_surface::commit) >>> to tell the compositor it's done at which point the compositor can now >>> texture from the surface. The hand-off ensures that the client's >>> OpenGL API calls happen before the server's OpenGL API calls. >>> >>> A good example of explicit synchronization is the Vulkan API. There, >>> a client (or multiple clients) can simultaneously build command >>> buffers in different threads where one of those command buffers >>> renders to an image and the other textures from it and then submit >>> both of them at the same time with instructions to the driver for >>> which order to execute them in. The execution order is described via >>> the VkSemaphore primitive. With the new VK_KHR_timeline_semaphore >>> extension, you can even submit the work which does the texturing >>> BEFORE the work which does the rendering and the driver will sort it >>> out. >>> >>> The #1 problem with implicit synchronization (which explicit solves) >>> is that it leads to a lot of over-synchronization both in client space >>> and in driver/device space. The
