Re: [Mesa-dev] Plumbing explicit synchronization through the Linux ecosystem
On Thu, Mar 12, 2020 at 6:36 PM Jason Ekstrand wrote: > From the perspective of a Wayland compositor (I used to play in this > space), they'd love to implement the new explicit sync extension but > can't. Sure, they could wire up the extension, but the moment they go > to flip a client buffer to the screen directly, they discover that KMS > doesn't support any explicit sync APIs. So, yes, they can technically > implement the extension assuming the EGL stack they're running on has > the sync_file extensions but any client buffers which come in using > the explicit sync Wayland extension have to be composited and can't be > scanned out directly. As a 3D driver developer, I absolutely don't > want compositors doing that because my users will complain about > performance issues due to the extra blit. Maybe this is something for the Marketing Department to solve? Sell the extra processing that can be done during such extra blit as a feature? As a former user of a wide-gamut monitor that has no sRGB mode, and a gamer, I would definitely accept the extra step (color conversion, not "just a blit"!) between the application and the actual output. In fact, I have set up compicc just for this purpose. Games with poisonous oversaturated colors (because none of the game authors care about wide-gamut monitors) are worse than the same games affected by the very small performance penalty due to the conversion. We just need a Marketing Person to come up with a huge list of other cases where such compositing step is required for correctness, and declare that direct scanout is something that makes no sense in the present day, except possibly on embedded devices. Of course the above trolling does not solve the problem related to inability to be sure about the correct API usage. -- Alexander E. Patrakov CV: http://pc.cd/PLz7 ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Meson links with duplicate symbols
https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4186 From: Jose Fonseca Sent: Tuesday, March 10, 2020 17:20 To: ML mesa-dev Cc: Neha Bhende ; Dylan Baker Subject: Re: Meson links with duplicate symbols I figured out enough of Meson to workout a fix. I'll post it for review tomorrow (after I figure out what's the exact process for a MR.) Jose From: Jose Fonseca Sent: Friday, March 6, 2020 15:08 To: ML mesa-dev ; Dylan Baker Cc: Neha Bhende Subject: Meson links with duplicate symbols While reviewing my colleagues efforts to consume Meson for Windows I stumbled across two issues. First is that even though we added the "WARNING: Scons support is in the process of being deprecated on on windows platforms (including mingw). If you haven't already please try using meson for windows builds. Be sure to report any issues you run into" to scons, we didn't update mesa/docs/install.html with instructions on how to build Meson w/ Mingw for windows, so it's not obvious to a newbie how to do it. It took me quite a while to reverse engineer the stuff in .gitlab-ci/... till I figured out the way to do it. I think that adding example command lines would go a long way. Second and most important is that Meson is actually linking with duplicate symbols. This was being hidden because meson doesn't use whole linking for everything, whereas SCons does. It's easy to repro: $ git diff diff --git a/src/gallium/targets/libgl-gdi/meson.build b/src/gallium/targets/libgl-gdi/meson.build index cd4e02d1fa9..fb8a42439c8 100644 --- a/src/gallium/targets/libgl-gdi/meson.build +++ b/src/gallium/targets/libgl-gdi/meson.build @@ -33,10 +33,8 @@ libopengl32 = shared_library( include_directories : [ inc_common, inc_wgl, inc_gallium_winsys_sw, inc_gallium_drivers, ], - link_whole : [libwgl], - link_with : [ -libgallium, libglsl, libmesa_gallium, libwsgdi, libglapi_static, libglapi - ], + link_whole : [libwgl, libgallium, libglsl, libmesa_gallium, libwsgdi, libglapi_static, libglapi], + link_with : [], dependencies : [ dep_ws2_32, idep_nir, idep_mesautil, driver_swrast, driver_swr, ], $ meson --cross .gitlab-ci/x86_64-w64-mingw32 . build/meson-windows-x86_64-debug $ ninja -C build/meson-windows-x86_64-debug/ ninja: Entering directory `build/meson-windows-x86_64-debug/' [2/2] Linking target src/gallium/targets/libgl-gdi/opengl32.dll. FAILED: src/gallium/targets/libgl-gdi/opengl32.dll x86_64-w64-mingw32-g++ -o src/gallium/targets/libgl-gdi/opengl32.dll 'src/gallium/targets/libgl-gdi/c96d1e6@@opengl32@sha/libgl_gdi.c.obj' -Wl,--allow-shlib-undefined -shared ../../src/gallium/targets/libgl-gdi/../../state_trackers/wgl/opengl32.def -Wl,--start-group -Wl,--out-implib=src/gallium/targets/libgl-gdi/opengl32.dll.a -Wl,--whole-archive src/gallium/state_trackers/wgl/libwgl.a src/gallium/auxiliary/libgallium.a src/compiler/glsl/libglsl.a src/mesa/libmesa_gallium.a src/gallium/winsys/sw/gdi/libwsgdi.a src/mapi/glapi/libglapi_static.a -Wl,--no-whole-archive -Wl,--nxcompat -Wl,--dynamicbase -static-libgcc -static-libstdc++ src/compiler/nir/libnir.a src/compiler/libcompiler.a src/util/libmesa_util.a src/util/format/libmesa_format.a subprojects/zlib-1.2.11/libz.dll.a src/gallium/drivers/softpipe/libsoftpipe.a src/compiler/glsl/glcpp/libglcpp.a src/mesa/libmesa_common.a src/mesa/libmesa_sse41.a -lws2_32 -pthread -lm -lkernel32 -luser32 -lgdi32 -lwinspool -lshell32 -lole32 -loleaut32 -luuid -lcomdlg32 -ladvapi32 -Wl,--end-group src/mesa/libmesa_common.a(main_shaderapi.c.obj): In function `_mesa_get_shader_include_cursor': /home/jfonseca/work/vmware/opengl/mesa/build/meson-windows-x86_64-debug/../../src/mesa/main/shaderapi.c:3185: multiple definition of `_mesa_get_shader_include_cursor' src/compiler/glsl/glcpp/libglcpp.a(pp_standalone_scaffolding.c.obj):/home/jfonseca/work/vmware/opengl/mesa/build/meson-windows-x86_64-debug/../../src/compiler/glsl/glcpp/pp_standalone_scaffolding.c:49: first defined here src/mesa/libmesa_common.a(main_shaderapi.c.obj): In function `_mesa_set_shader_include_cursor': /home/jfonseca/work/vmware/opengl/mesa/build/meson-windows-x86_64-debug/../../src/mesa/main/shaderapi.c:3191: multiple definition of `_mesa_set_shader_include_cursor' src/compiler/glsl/glcpp/libglcpp.a(pp_standalone_scaffolding.c.obj):/home/jfonseca/work/vmware/opengl/mesa/build/meson-windows-x86_64-debug/../../src/compiler/glsl/glcpp/pp_standalone_scaffolding.c:57: first defined here src/mesa/libmesa_common.a(main_shaderapi.c.obj): In function `_mesa_lookup_shader_include': /home/jfonseca/work/vmware/opengl/mesa/build/meson-windows-x86_64-debug/../../src/mesa/main/shaderapi.c:3404: multiple definition of `_mesa_lookup_shader_include' src/compiler/glsl/glcpp/libglcpp.a(pp_standalone_scaffolding.c.obj):/home/jfonseca/work/vmware/opengl/mesa/build/meson-windows-x86_64-debug/../../src/compiler/glsl/glcpp/pp_standalone_
Re: [Mesa-dev] Plumbing explicit synchronization through the Linux ecosystem
There is no synchronization between processes (e.g. 3D app and compositor) within X on AMD hw. It works because of some hacks in Mesa. Marek On Wed, Mar 11, 2020 at 1:31 PM Jason Ekstrand wrote: > All, > > Sorry for casting such a broad net with this one. I'm sure most people > who reply will get at least one mailing list rejection. However, this > is an issue that affects a LOT of components and that's why it's > thorny to begin with. Please pardon the length of this e-mail as > well; I promise there's a concrete point/proposal at the end. > > > Explicit synchronization is the future of graphics and media. At > least, that seems to be the consensus among all the graphics people > I've talked to. I had a chat with one of the lead Android graphics > engineers recently who told me that doing explicit sync from the start > was one of the best engineering decisions Android ever made. It's > also the direction being taken by more modern APIs such as Vulkan. > > > ## What are implicit and explicit synchronization? > > For those that aren't familiar with this space, GPUs, media encoders, > etc. are massively parallel and synchronization of some form is > required to ensure that everything happens in the right order and > avoid data races. Implicit synchronization is when bits of work (3D, > compute, video encode, etc.) are implicitly based on the absolute > CPU-time order in which API calls occur. Explicit synchronization is > when the client (whatever that means in any given context) provides > the dependency graph explicitly via some sort of synchronization > primitives. If you're still confused, consider the following > examples: > > With OpenGL and EGL, almost everything is implicit sync. Say you have > two OpenGL contexts sharing an image where one writes to it and the > other textures from it. The way the OpenGL spec works, the client has > to make the API calls to render to the image before (in CPU time) it > makes the API calls which texture from the image. As long as it does > this (and maybe inserts a glFlush?), the driver will ensure that the > rendering completes before the texturing happens and you get correct > contents. > > Implicit synchronization can also happen across processes. Wayland, > for instance, is currently built on implicit sync where the client > does their rendering and then does a hand-off (via wl_surface::commit) > to tell the compositor it's done at which point the compositor can now > texture from the surface. The hand-off ensures that the client's > OpenGL API calls happen before the server's OpenGL API calls. > > A good example of explicit synchronization is the Vulkan API. There, > a client (or multiple clients) can simultaneously build command > buffers in different threads where one of those command buffers > renders to an image and the other textures from it and then submit > both of them at the same time with instructions to the driver for > which order to execute them in. The execution order is described via > the VkSemaphore primitive. With the new VK_KHR_timeline_semaphore > extension, you can even submit the work which does the texturing > BEFORE the work which does the rendering and the driver will sort it > out. > > The #1 problem with implicit synchronization (which explicit solves) > is that it leads to a lot of over-synchronization both in client space > and in driver/device space. The client has to synchronize a lot more > because it has to ensure that the API calls happen in a particular > order. The driver/device have to synchronize a lot more because they > never know what is going to end up being a synchronization point as an > API call on another thread/process may occur at any time. As we move > to more and more multi-threaded programming this synchronization (on > the client-side especially) becomes more and more painful. > > > ## Current status in Linux > > Implicit synchronization in Linux works via a the kernel's internal > dma_buf and dma_fence data structures. A dma_fence is a tiny object > which represents the "done" status for some bit of work. Typically, > dma_fences are created as a by-product of someone submitting some bit > of work (say, 3D rendering) to the kernel. The dma_buf object has a > set of dma_fences on it representing shared (read) and exclusive > (write) access to the object. When work is submitted which, for > instance renders to the dma_buf, it's queued waiting on all the fences > on the dma_buf and and a dma_fence is created representing the end of > said rendering work and it's installed as the dma_buf's exclusive > fence. This way, the kernel can manage all its internal queues (3D > rendering, display, video encode, etc.) and know which things to > submit in what order. > > For the last few years, we've had sync_file in the kernel and it's > plumbed into some drivers. A sync_file is just a wrapper around a > single dma_fence. A sync_file is typically created as a by-product of > submitting work (3D,
