Re: Wayland debugging with Qtwayland, gstreamer waylandsink, wayland-lib and Weston
On 05/03/2024 12:26, Pekka Paalanen wrote: On Mon, 4 Mar 2024 17:59:25 + Terry Barnaby wrote: On 04/03/2024 15:50, Pekka Paalanen wrote: On Mon, 4 Mar 2024 14:51:52 + Terry Barnaby wrote: On 04/03/2024 14:14, Pekka Paalanen wrote: On Mon, 4 Mar 2024 13:24:56 + Terry Barnaby wrote: On 04/03/2024 09:41, Pekka Paalanen wrote: On Mon, 4 Mar 2024 08:12:10 + Terry Barnaby wrote: While I am trying to investigate my issue in the QtWayland arena via the Qt Jira Bug system, I thought I would try taking Qt out of the equation to simplify the application a bit more to try and gain some understanding of what is going on and how this should all work. So I have created a pure GStreamer/Wayland/Weston application to test out how this should work. This is at: https://portal.beam.ltd.uk/public//test022-wayland-video-example.tar.gz This tries to implement a C++ Widget style application using native Wayland. It is rough and could easily be doing things wrong wrt Wayland. However it does work to a reasonable degree. However, I appear to see the same sort of issue I see with my Qt based system in that when a subsurface of a subsurface is used, the Gstreamer video is not seen. This example normally (UseWidgetTop=0) has a top level xdg_toplevel desktop surface (Gui), a subsurface to that (Video) and then waylandsink creates a subsurface to that which it sets to de-sync mode. When this example is run with UseWidgetTop=0 the video frames from gstreamer are only shown shown when the top subsurface is manually committed with gvideo->update() every second, otherwise the video pipeline is stalled. This is intentional. From wl_subsurface specification: Even if a sub-surface is in desynchronized mode, it will behave as in synchronized mode, if its parent surface behaves as in synchronized mode. This rule is applied recursively throughout the tree of surfaces. This means, that one can set a sub-surface into synchronized mode, and then assume that all its child and grand-child sub-surfaces are synchronized, too, without explicitly setting them. This is derived from the design decision that a wl_surface and its immediate sub-surfaces form a seamlessly integrated unit that works like a single wl_surface without sub-surfaces would. wl_subsurface state is state in the sub-surface's parent, so that the parent controls everything as if there was just a single wl_surface. If the parent sets its sub-surface as desynchronized, it explicitly gives the sub-surface the permission to update on screen regardless of the parent's updates. When the sub-surface is in synchronized mode, the parent surface wants to be updated in sync with the sub-surface in an atomic fashion. When your surface stack looks like: - main surface A, top-level, root surface (implicitly desynchronized) - sub-surface B, synchronized - sub-surface C, desynchronized Updates to surface C are immediately made part of surface B, because surface C is in desynchronized mode. If B was the root surface, all C updates would simply go through. However, surface B is a part of surface A, and surface B is in synchronized mode. This means that the client wants surface A updates to be explicit and atomic. Nothing must change on screen until A is explicitly committed itself. So any update to surface B requires a commit on surface A to become visible. Surface C does not get to override the atomicity requirement of surface A updates. This has been designed so that software component A can control surface A, and delegate a part of surface A to component B which happens to the using a sub-surface: surface B. If surface B parts are further delegated to another component C, then component A can still be sure that nothing updates on surface A until it says so. Component A sets surface B to synchronized to ensure that. That's the rationale behind the Wayland design. Thanks, pq Ah, thanks for the info, that may be why this is not working even in Qt then. This seems a dropoff in Wayland to me. If a software module wants to display Video into an area on the screen at its own rate, setting that surface to de-synced mode is no use in the general case with this policy. It is of use, if you don't have unnecessary sub-surfaces in synchronized mode in between, or you set all those extra sub-surfaces to desynchronized as well. Well they may not be necessary from the Wayland perspective, but from the higher level software they are useful to modularise/separate/provide a join for the software modules especially when software modules are separate like Qt and GStreamer. Sorry to hear that. I would have thought that if a subsurface was explicitly set to de-synced mode then that would be honoured. I can't see a usage case for it to be ignored and its commits synchronised up the tree ? Resizing the window is the main use case. In order to resize surface A, you also need to resize and paint surface B, and for surface B you also need t
Re: Wayland debugging with Qtwayland, gstreamer waylandsink, wayland-lib and Weston
On Fri, 8 Mar 2024 14:50:30 + Terry Barnaby wrote: > On 05/03/2024 12:26, Pekka Paalanen wrote: > > On Mon, 4 Mar 2024 17:59:25 + > > Terry Barnaby wrote: > > ... > >> I would have thought it better/more useful to have a Wayland API call > >> like "stopCommiting" so that an application can sort things out for this > >> and other things, providing more application control. But I really have > >> only very limited knowledge of the Wayland system. I just keep hitting > >> its restrictions. > >> > > Right, Wayland does not work that way. Wayland sees any client as a > > single entity, regardless of its internal composition of libraries and > > others. > > > > When Wayland delivers any event, whether it is an explicit resize event > > or an input event (or maybe the client just spontaneously decides to), > > that causes the client to want to resize a window, it is then up to the > > client itself to make sure it resizes everything it needs to, and keeps > > everything atomic so that the end user does not see glitches on screen. > > > > Sub-surfaces' synchronous mode was needed to let clients batch the > > updates of multiple surfaces into a single atomic commit. It is the > > desync mode that was a non-mandatory add-on. The synchronous mode was > > needed, because there was no other way to batch multiple > > wl_surface.commit requests to apply simultaneously guaranteed. Without > > it, if you commit surface A and then surface B, nothing will guarantee > > that the compositor would not show A updated and B not on screen for a > > moment. > > > > Wayland intentionally did not include any mechanism in its design > > intended for communication between a single client's internal > > components. Why use a display server as an IPC middle-man for something > > that should be process-internal communication. After all, Wayland is > > primarily a protocol - inter-process communication. > > Well as you say it is up to the client to perform all of the surface > resize work. So it seems to me, if the client had an issue with pixel > perfect resizing it could always set any of its desynced surfaces to > sync mode, or just stop the update to them, while it resizes. I don't > see why Wayland needs to ignore the clients request to set a subsurface > desynced down the tree. You're right, but it's in the spec now. I've gained a bit more experience in the decade after writing the sub-surface spec. You can still work around it by setting all sub-surfaces always desync. > In fact does it return an error to the client > when the Wayland server ignores this command ? There is no "return error" in Wayland. Either a request succeeds, or the client is disconnected with an error. It's all asynchronous, too. Any possibility for graceful failure must be designed into protocol extensions at one step higher level. If there is room for a graceful failure, it will be mentioned in the XML spec with explicit messages to communicate it. Which command do you mean? There is no "ignore" with wl_surface nor wl_subsurface. wl_surface.commit is always acted upon, but the sub-surface sync mode determines whether the state update goes to the screen or to a cache. No state update is ignored unless you destroy your objects. The frame callbacks that seem to go unanswered are not ignored, they are just sitting in the cache waiting to apply when the parent surface actually updates on screen. Thanks, pq pgpdoXGYQYgtl.pgp Description: OpenPGP digital signature
Re: Wayland debugging with Qtwayland, gstreamer waylandsink, wayland-lib and Weston
On 08/03/2024 15:23, Pekka Paalanen wrote: On Fri, 8 Mar 2024 14:50:30 + Terry Barnaby wrote: On 05/03/2024 12:26, Pekka Paalanen wrote: On Mon, 4 Mar 2024 17:59:25 + Terry Barnaby wrote: ... I would have thought it better/more useful to have a Wayland API call like "stopCommiting" so that an application can sort things out for this and other things, providing more application control. But I really have only very limited knowledge of the Wayland system. I just keep hitting its restrictions. Right, Wayland does not work that way. Wayland sees any client as a single entity, regardless of its internal composition of libraries and others. When Wayland delivers any event, whether it is an explicit resize event or an input event (or maybe the client just spontaneously decides to), that causes the client to want to resize a window, it is then up to the client itself to make sure it resizes everything it needs to, and keeps everything atomic so that the end user does not see glitches on screen. Sub-surfaces' synchronous mode was needed to let clients batch the updates of multiple surfaces into a single atomic commit. It is the desync mode that was a non-mandatory add-on. The synchronous mode was needed, because there was no other way to batch multiple wl_surface.commit requests to apply simultaneously guaranteed. Without it, if you commit surface A and then surface B, nothing will guarantee that the compositor would not show A updated and B not on screen for a moment. Wayland intentionally did not include any mechanism in its design intended for communication between a single client's internal components. Why use a display server as an IPC middle-man for something that should be process-internal communication. After all, Wayland is primarily a protocol - inter-process communication. Well as you say it is up to the client to perform all of the surface resize work. So it seems to me, if the client had an issue with pixel perfect resizing it could always set any of its desynced surfaces to sync mode, or just stop the update to them, while it resizes. I don't see why Wayland needs to ignore the clients request to set a subsurface desynced down the tree. You're right, but it's in the spec now. I've gained a bit more experience in the decade after writing the sub-surface spec. You can still work around it by setting all sub-surfaces always desync. Oh you wrote it, thanks for the work! So maybe time for version n+1 then :) Actually allowing sub/subsurfaces to work in desync should not break any existing clients as they cannot use it yet. Obviously new clients written for it would not work on older Wayland servers though. Its difficult to desync all the higher surfaces in a Qt or probably other Widget set application, they are controlled by Qt and Qt does not give you access to the subsurfaces it has created. It would be better to have had a wl_surface_set_desync(wl_surface*) rather than a wl_subsurface_set_desync(wl_subsurface*). With clients using lots of libraries/subsystems it is better to not use their internal workings unless you have to. Normally you try and work at the least common denominator, in this case the Wayland display system as that is the shared module they both use (at least when driving a Wayland display server). This is why it is nice to have a surface that is almost totally independent of others and just is shown/not shown, is over/below etc. other surfaces like an XWindow. The Wayland surfaces are mainly this as far as I can see, apart from this desync mode although maybe there are others. I have asked in the Qt forums if they could provide some sort of API to allow the setting of desync up the tree, but this may not happen and it might be difficult for them as it could mess up their applications rendering. It also does not match other display system API's that they support. The higher level QWidgets ideally need synced surfaces, its just the Video surfaces that need desync. Really I think this is the Wayland servers job. In fact does it return an error to the client when the Wayland server ignores this command ? There is no "return error" in Wayland. Either a request succeeds, or the client is disconnected with an error. It's all asynchronous, too. Any possibility for graceful failure must be designed into protocol extensions at one step higher level. If there is room for a graceful failure, it will be mentioned in the XML spec with explicit messages to communicate it. Which command do you mean? I meant the wl_subsurface_set_desync() API call on a sub/subsurface that doesn't work. As no errors were returned it took a long time to find out why things weren't working, just some lower level threads locked up. Personally I think these sort of occasional, performance irrelevant, types of methods/requests/commands should be synchronous (maybe under an asynchronous comms system) and return an error. Makes developing clients muc
