Re: Split render/display SoCs, Mesa's renderonly, and Wayland dmabuf hints
Thanks for the feedback! Replying to both Daniels below. On Tuesday, April 20th, 2021 at 1:14 PM, Daniel Vetter wrote: > On Tue, Apr 20, 2021 at 12:18 PM Daniel Stone wrote: > > >> There are a few solutions: > >> > >> 1. Require compositors to discover the render device by trying to import > >>a buffer. For each available render device, the compositor would > >>allocate a buffer, export it as a DMA-BUF, import it to the > >>display-only device, then try to drmModeAddFB. > > > > > > I don't think this is actually tractable? Assuming that 'allocate a buffer' > > means 'obtain a gbm_device for the render node directly and allocate a > > gbm_bo from it', even with compatible formats and modifiers this will fail > > for more restrictive display hardware. imx-drm and pl111 (combined with vc4 > > on some Raspberry Pis) will fail this, since they'll take different > > allocation paths when they're bound through kmsro vs. directly, accounting > > for things like contiguous allocation. So we'd get false negatives on at > > least some platforms. Right. It would work for drivers using renderonly_create_gpu_import_for_resource, but wouldn't work for drivers using renderonly_create_kms_dumb_buffer_for_resource. Doing the reverse (creating a buffer on the display-only device, import it on the render-only device) wouldn't work either, since some drivers will perform DMA transfers under-the-hood without telling user-space. So the check would succeed even if the pipeline is inefficient. > >> 2. Allow compositors to query the render device magically opened by > >>kmsro. This could be done either via EGL_EXT_device_drm, or via a > >>new EGL extension. > > > > This would be my strong preference, and I don't entirely understand > > anholt's pushback here. The way I see it, GBM is about allocation for > > scanout, and EGL is about rendering. If, on a split GPU/display system, we > > create a gbm_device from a KMS display-only device node, then creating an > > EGLDisplay from that magically binds us to a completely different DRM GPU > > node, and anything using that EGLDisplay will use that GPU device to > > render. > > > > Being able to discover the GPU device node through the device query is > > really useful, because it tells us exactly what implicit magic EGL did > > under the hood, and about the device that EGL will use. Being able to > > discover the display node is much less useful; it does tell us how GBM will > > allocate buffers, but the user already knows which device is in use because > > they supplied it to GBM. I see the display node as a property of GBM, and > > the GPU node as a property of EGL, even if EGL does do (*waves hands*) > > stuff under the hood to ensure the two are compatible. GBM can be used for non-scanout allocation too. This is useful for clients that want to have more control over the allocation parameters, the swapchain and want to manage DMA-BUFs directly. > > If we had EGL_EXT_explicit_device, things get even more weird, I think; > > would the device query on an EGLDisplay created with a combination of a > > gbm_device native display handle and an explicit EGLDevice handle return > > the scanout device from GBM or the GPU device from EGL? On my reading, I'd > > expect it to be the latter; if the queries returned very different things > > based on whether GPU device selection was implicit (returning the KMS node) > > or explicit (GPU node), that would definitely violate the principle of > > least surprise. Right now as implemented by Mesa, things are pretty weird indeed. EGL_EXT_device_enumeration will only expose a list of one EGLDevice per DRM render node, plus one EGLDevice for software rendering. If you create an EGLDisplay from a gbm_device from a display-only device, then a new EGLDevice for this display-only device gets created and is returned by EGL_EXT_device_query. It would make more sense to me to return the EGLDevice for the render-only device in EGL_EXT_device_query. However that means we need to pass back this information through Mesa's DRI interface and throught the Gallium interface. That sounds pretty annoying, but was what I tried to suggest earlier here: https://gitlab.freedesktop.org/mesa/mesa/-/issues/4178#note_815040 > >> 3. Allow compositors to query the kernel drivers to know which devices > >>are compatible with each other. Some uAPI to query a compatible > >>display device from a render-only device, or vice-versa, has been > >>suggested in the past. > > > > > > What does 'compatible' mean? Would an Intel iGPU and and AMD dGPU be > > compatible with each other? Would a Mali GPU bound to system memory through > > AMBA be as compatible with the display controller as it would with an AMD > > GPU on PCIE? I think a query which only exposed whether or not devices > > could share dmabufs with each other is far too generic to be helpful for > > the actual usecase we have, as well as not being useful enough for other > > usecases (
How to make infinite-scrolling GUI widgets under Wayland ?
Hello, I am working on the multimedia sequencer ossia score (https://ossia.io). I am trying to make sure that it works fine for Linux users under wayland. For audio (and in general multimedia) apps, there is a very common family of GUI widget which allows to scroll a value "infinitely" (or at least much more than the height of the screen) - the usual user interaction is : * User clicks on the widget : the mouse cursor disappears (some apps will give a kind of highlight to the widget to indicate that it is being acted upon) * User moves their mouse upwards or downwards to increase the value * At some point they reach the top or the bottom of the screen : to continue scrolling transparently, the app translates the mouse cursor to the top of the screen * When the user releases the mouse the cursor reappears at the position where it was clicked. This is needed in two case : - To make sliders / knobs with easily adjustable values - To implement infinite- or at least very long scrolling in minimaps. I made a few videos: - Video 1: example of slider with the feature in Ableton Live (one of the most used music making app) : https://streamable.com/ecepvc - Video 2 : example of minimap in Ableton Live : https://streamable.com/epc7r1 - Video 3 : Example of the pain induced by software that do not support the feature: here I want to change the tempo but cannot because the mouse hits the top of the screen. Thus I have to release and go click on it again : https://streamable.com/bniht5 Thus, my question is : what must I do as the developer of such an app to make sure that my users will have widgets that will work "as expected" for people in the media creation field, whatever the wayland compositor my users are using (KDE Plasma, GNOME, sway...). I don't want them to have an inferior user experience on Linux when compared to other operating systems. The googling I've done so far seems to say that there is no way to position the cursor absolutely directly through wayland, but the only other way seems to be through /dev/uinput which needs root permissions to write to (and my userbase are artists who generally don't have the technical skillset to know that they must mark things as root; I could always check and show a popup that requests the user to do chmod u+rwx on /dev/uinput on startup but that would be my last resort...) Thanks ! Jean-Michaël ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: How to make infinite-scrolling GUI widgets under Wayland ?
On Mon, Apr 26, 2021 at 04:57:01PM +0200, Jean-Michaël Celerier wrote: > Hello, > I am working on the multimedia sequencer ossia score (https://ossia.io). > > I am trying to make sure that it works fine for Linux users under wayland. > > For audio (and in general multimedia) apps, there is a very common family > of GUI widget which allows to scroll a value "infinitely" (or at least much > more than the height of the screen) - the usual user interaction is : > * User clicks on the widget : the mouse cursor disappears (some apps will > give a kind of highlight to the widget to indicate that it is being acted > upon) > * User moves their mouse upwards or downwards to increase the value > * At some point they reach the top or the bottom of the screen : to > continue scrolling transparently, the app translates the mouse cursor to > the top of the screen > * When the user releases the mouse the cursor reappears at the position > where it was clicked. > > This is needed in two case : > - To make sliders / knobs with easily adjustable values > - To implement infinite- or at least very long scrolling in minimaps. > > I made a few videos: > - Video 1: example of slider with the feature in Ableton Live (one of the > most used music making app) : https://streamable.com/ecepvc > > - Video 2 : example of minimap in Ableton Live : > https://streamable.com/epc7r1 > > - Video 3 : Example of the pain induced by software that do not support the > feature: here I want to change the tempo but cannot because the mouse hits > the top of the screen. Thus I have to release and go click on it again : > https://streamable.com/bniht5 > > Thus, my question is : what must I do as the developer of such an app to > make sure that my users will have widgets that will work "as expected" for > people in the media creation field, whatever the wayland compositor my > users are using (KDE Plasma, GNOME, sway...). I don't want them to have an > inferior user experience on Linux when compared to other operating systems. > > The googling I've done so far seems to say that there is no way to position > the cursor absolutely directly through wayland, but the only other way > seems to be through /dev/uinput which needs root permissions to write to > (and my userbase are artists who generally don't have the technical > skillset to know that they must mark things as root; I could always check > and show a popup that requests the user to do chmod u+rwx on /dev/uinput on > startup but that would be my last resort...) To implement the behavior shown in video 1 and 2, the "Wayland" way is depends on two protocols extension: * Pointer locking extension[0] * Relative pointer motions extension[1] With the core wl_pointer protocol object, you can control when the pointer cursor sprite is showing or not; with the pointer locking extension, you can effectively lock the pointer position to its position, meaning it will stay put and won't "escape" your application window; and lastly, with the relative pointer motions extension, you can receive pointer motions that are not affected by actual movement of the pointer cursor sprite, i.e. even if the pointer hit an edge or doesn't move, you'll still receive the motions as sent from the device. I think most compositors support these two protocol extensions as they are a corner stone in how e.g. 3D FPS games work. Jonas [0] https://gitlab.freedesktop.org/wayland/wayland-protocols/-/blob/master/unstable/pointer-constraints/pointer-constraints-unstable-v1.xml [1] https://gitlab.freedesktop.org/wayland/wayland-protocols/-/blob/master/unstable/relative-pointer/relative-pointer-unstable-v1.xml > > Thanks ! > Jean-Michaël > ___ > wayland-devel mailing list > wayland-devel@lists.freedesktop.org > https://lists.freedesktop.org/mailman/listinfo/wayland-devel ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: How to make infinite-scrolling GUI widgets under Wayland ?
Thanks, it looks exactly like it. Now just have to see how to make it work with Qt but that's another question :) Kind regards, Jean-Michaël On Mon, Apr 26, 2021 at 5:08 PM Jonas Ådahl wrote: > On Mon, Apr 26, 2021 at 04:57:01PM +0200, Jean-Michaël Celerier wrote: > > Hello, > > I am working on the multimedia sequencer ossia score (https://ossia.io). > > > > I am trying to make sure that it works fine for Linux users under > wayland. > > > > For audio (and in general multimedia) apps, there is a very common family > > of GUI widget which allows to scroll a value "infinitely" (or at least > much > > more than the height of the screen) - the usual user interaction is : > > * User clicks on the widget : the mouse cursor disappears (some apps will > > give a kind of highlight to the widget to indicate that it is being acted > > upon) > > * User moves their mouse upwards or downwards to increase the value > > * At some point they reach the top or the bottom of the screen : to > > continue scrolling transparently, the app translates the mouse cursor to > > the top of the screen > > * When the user releases the mouse the cursor reappears at the position > > where it was clicked. > > > > This is needed in two case : > > - To make sliders / knobs with easily adjustable values > > - To implement infinite- or at least very long scrolling in minimaps. > > > > I made a few videos: > > - Video 1: example of slider with the feature in Ableton Live (one of the > > most used music making app) : https://streamable.com/ecepvc > > > > - Video 2 : example of minimap in Ableton Live : > > https://streamable.com/epc7r1 > > > > - Video 3 : Example of the pain induced by software that do not support > the > > feature: here I want to change the tempo but cannot because the mouse > hits > > the top of the screen. Thus I have to release and go click on it again : > > https://streamable.com/bniht5 > > > > Thus, my question is : what must I do as the developer of such an app to > > make sure that my users will have widgets that will work "as expected" > for > > people in the media creation field, whatever the wayland compositor my > > users are using (KDE Plasma, GNOME, sway...). I don't want them to have > an > > inferior user experience on Linux when compared to other operating > systems. > > > > The googling I've done so far seems to say that there is no way to > position > > the cursor absolutely directly through wayland, but the only other way > > seems to be through /dev/uinput which needs root permissions to write to > > (and my userbase are artists who generally don't have the technical > > skillset to know that they must mark things as root; I could always check > > and show a popup that requests the user to do chmod u+rwx on /dev/uinput > on > > startup but that would be my last resort...) > > To implement the behavior shown in video 1 and 2, the "Wayland" way is > depends on two protocols extension: > > * Pointer locking extension[0] > * Relative pointer motions extension[1] > > With the core wl_pointer protocol object, you can control when the > pointer cursor sprite is showing or not; with the pointer locking > extension, you can effectively lock the pointer position to its > position, meaning it will stay put and won't "escape" your application > window; and lastly, with the relative pointer motions extension, you can > receive pointer motions that are not affected by actual movement of the > pointer cursor sprite, i.e. even if the pointer hit an edge or doesn't > move, you'll still receive the motions as sent from the device. > > I think most compositors support these two protocol extensions as they > are a corner stone in how e.g. 3D FPS games work. > > > Jonas > > [0] > https://gitlab.freedesktop.org/wayland/wayland-protocols/-/blob/master/unstable/pointer-constraints/pointer-constraints-unstable-v1.xml > [1] > https://gitlab.freedesktop.org/wayland/wayland-protocols/-/blob/master/unstable/relative-pointer/relative-pointer-unstable-v1.xml > > > > > Thanks ! > > Jean-Michaël > > > ___ > > wayland-devel mailing list > > wayland-devel@lists.freedesktop.org > > https://lists.freedesktop.org/mailman/listinfo/wayland-devel > > ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/wayland-devel
