> -----Original Message----- > From: Pekka Paalanen <pekka.paala...@collabora.com> > Sent: Thursday, June 13, 2024 3:32 AM > To: Hoosier, Matt <matt.hoos...@garmin.com> > Cc: 's...@cmpwn.com' <s...@cmpwn.com>; 'cont...@emersion.fr' > <cont...@emersion.fr>; 'Marius Vlad' <marius.v...@collabora.com>; > 'wayland-devel@lists.freedesktop.org' <wayland- > de...@lists.freedesktop.org>; 'Daniel Stone' <dan...@fooishbar.org> > Subject: Re: Full-motion zero-copy screen capture in Weston > > On Wed, 12 Jun 2024 21:35:48 +0000 > "Hoosier, Matt" <matt.hoos...@garmin.com> wrote: > > > > -----Original Message----- > > > From: Hoosier, Matt > > > Sent: Wednesday, June 12, 2024 8:37 AM > > > To: Pekka Paalanen <pekka.paala...@collabora.com> > > > Cc: s...@cmpwn.com; cont...@emersion.fr; Marius Vlad > > > <marius.v...@collabora.com>; wayland-devel@lists.freedesktop.org; > > > Daniel Stone <dan...@fooishbar.org> > > > Subject: RE: Full-motion zero-copy screen capture in Weston > > > > > > > -----Original Message----- > > > > From: Pekka Paalanen <pekka.paala...@collabora.com> > > > > Sent: Wednesday, June 12, 2024 4:03 AM > > > > To: Hoosier, Matt <matt.hoos...@garmin.com> > > > > Cc: s...@cmpwn.com; cont...@emersion.fr; Marius Vlad > > > > <marius.v...@collabora.com>; wayland-devel@lists.freedesktop.org; > > > > Daniel Stone <dan...@fooishbar.org> > > > > Subject: Re: Full-motion zero-copy screen capture in Weston > > > > > > > > On Mon, 10 Jun 2024 14:18:39 +0000 "Hoosier, Matt" > > > > <matt.hoos...@garmin.com> wrote: > > > > > > > > > Yes, the native Linux driver for the hardware still does end up > > > > > being responsible for one or more planes. > > > > > > > > > > Other than trying to arrange the pieces so that there's some API > > > > > that offers the client an option to guarantee that the source of > > > > > the screen capture involves the writeback connector (similar to > > > > > what you've done with weston_output_capture), I don't really > > > > > think it would be a good idea to make Weston explicitly aware of > > > > > any of this funny hypervisor business going on. > > > > > > > > > > > > > I very much agree. > > > > > > Ack > > > > > > > > > > > > > > > The title on this email conversation was probably poorly chosen, > > > > > in retrospect. I'm not so much concerned with keeping absolutely > > > > > to a zero-copy mechanism as I am to using hardware mechanisms > > > > > (GL rendering, DRI writeback, hardware-accelerated blits as > > > > > necessary) all the way through. > > > > > > > > > > After seeing the way that the Pipewire backend handles streaming > > > > > (especially with > > > > > https://gitlab.freedesktop.org/wayland/weston/- > > > /merge_requests/1366), > > > > > I'm almost thinking that it would be preferable to maybe > > > > > structure the design of this feature something like this: > > > > > > > > > > * Add some sort of API on weston_output by which it can > > > > > advertise the ability to lay hands on the explicit renderbuffer (e.g. > > > > > gbm_surface_get_front_buffer()). This roughly equates to whether > > > > > it's a primary, non-nested backend. That is, DRM. > > > > > > > > > > * When processing the weston.ini mirror-of relationships at > > > > > startup, check whether the source output of the mirror-of > > > > > declaration supports that ability above. If so: > > > > > - Wire up a signal so that the source output announces each > > > > > newly rendered frame to any/all mirror-of outputs. > > > > > - The virtual output's backend allocates its own buffer pool > > > > > in exactly the same way that MR 1366 already does. > > > > > - Upon receipt of each signal announcing a new frame's > > > > > renderbuffer from the source output, the virtual output does a > > > > > very cheap > > > > > glBlitFramebuffer() to get the contents into its own buffer pool. > > > > > This avoids the possibility of an unresponsive client tying up > > > > > the source output's buffer. > > > > > > > > > > * If the source output isn't one that supports exposing the > > > > > underlying renderbuffer, then the mirror-of relationship > > > > > continues as with MR > > > > > 1476 just to invoke an explicit weston_renderer pass to draw the > > > > > correct logical contents. > > > > > > > > > > How does this strike you? > > > > > > > > Sorry, I don't understand how that idea is relevant to the KMS > > > > writeback > > > case. > > > > Did you imply that DRM-backend could deliver a KMS-writeback FB > > > > instead > > > of > > > > the rendered FB? > > > > > > Just for the same of argument, yes. But I take your point below that > > > this entire idea to drive the mirroring output directly from the > > > source output's rendering doesn't fit the plan for the mirror-of > > > semantics. > > > > > > > > > > > This is not the current plan for mirror-of. It does not allow the > > > > mirror output > > > to > > > > run on its own pace independently of the source output. Re-using > > > > the source output's rendered FB would also be a problem for color > > > > management. The FB is rendered for that output, and the color > > > > properties of the mirror may be different, needing an independent > rendering anyway. > > > > > > > > The fundamental difference is who defines the properties of the stream. > > > > KMS writeback steam properties are necessarily defined by the KMS > output. > > > > Mirror-of is for the opposite, for full flexibility. E.g. a remote > > > > mirror may not want to run at the full framerate, the physical > > > > monitor resolution, or with > > > the > > > > color capabilities of the source output in order to save bandwidth > > > > and > > > improve > > > > latency. > > > > > > Okay, understood. Although I'm a bit curious how you can say that > > > one output "mirrors" another whose resolution doesn't even match. > > > Maybe you have scaling in mind? > > It's hard to pick the right words. See e.g. the thread at > https://gitlab.freedesktop.org/wayland/weston/- > /merge_requests/1476#note_2435244 > > Current upstream has "same-as" for shared-CRTC KMS cloning, which would > probably be renamed to "clone-of". Do we then re-purpose "same-as" for > something else? We've also talked about "mirror-of" and "share-position- > with". > > What the exact words and keys will be, we're not committed to yet. There are > several different kinds of mirroring. All the kinds that expose a new > wl_output > and have a separate [output] section in weston.ini, except for old "same-as" / > new "clone-of", will create an independent weston_output and be mirroring > based on exposing the same desktop area as some source output. > > (If one thinks of mirrors in real world, they almost never reproduce the > original > exactly, and they can be made to distort the image. They tend to at least > "mirror" things, that is, swap left and right for > instance.) > > > > > > > > > It seems to me that we will need two different mechanisms for the > > > > two > > > cases. I > > > > believe KMS writeback streaming is worthwhile to support, but not > > > > via > > > mirror- > > > > of key. Maybe the writeback streaming should be built into the > > > > DRM- > > > backend > > > > as a special pipewire source? Then it would also be guaranteed to > > > > be KMS writeback. It is some amount of code and testing > > > > duplication, but I think it would be the simplest. > > > > > > Fair enough. I had the beginning of a complicated scheme of signals > > > fallback rendering in mind. If you think it's justified to just add > > > a special case directly for this, that's better to me. > > > > > > > > > > > I don't see KMS writeback streaming as specific to your curious > > > > virtualization case. I can imagine it being useful in general, to > > > > ensure that display controller output is correct for example, or > > > > when overall rendering efficiency is more important than optimal stream > format. > > > > > > > > > > > > Thanks, > > > > pq > > > > > > The backends mechanics to do that seem fairly obvious. Once > > > https://gitlab.freedesktop.org/wayland/weston/-/merge_requests/1366/ > > > merges, there will be a ready-made way to allocate PipeWire buffers > > > that are backed by dmabuf memory with whatever GBM flags seem > > > appropriate to make the dmabuf eligible to register as a KMS > > > framebuffer. > > > > One thing that's not clear to me: what are the rules about which bits > > of backend code are allowed to link to various libraries? Weston has > > two different endpoints that separately integrate Pipewire: the > > pipewire-plugin and the pipewire-backend. The plugin seems to be > > mainly concerned to implement the API for Weston's virtual outputs, > > and I don't see any obvious way that the pipewire-backend module > > interacts with pipewire-plugin at all. > > Pipewire-backend and pipewire-plugin are completely independent entities, > they are not intended to interact. Pipewire-plugin is also on its way out: > https://gitlab.freedesktop.org/wayland/weston/-/issues/914 > > Backends are somewhat more free to pull in new dependencies than libweston > core, because backends can be not built. Of course, DRM-backend being such > a central component, we'd probably want a separate build option to enable > "DRM KMS writeback Pipewire support" or such, defaulting to enabled. > > > If we added some support for streaming out the contents of a DRM > > output over PipeWire, would it be expected to somehow route all access > > to libpipewire through one or the other of those module through one of > > the API structs? Or should it be done by just directly using > > libpipewire in backend-drm? > > Just use libpipewire directly. We would be having two independent pipewire > users then: pipewire-backend and DRM-backend, with the pipewire-plugin > going away. I'm not familiar with libpipewire though, so if it makes it > difficult or > somehow sub-optimal to keep the two users completely separate, then we'll > have to figure out a way to share the pipewire context. I wouldn't go wrapping > pipewire API per se. > > Maybe an internal static helper library, and whoever initializes it first, the > helper will register itself with the "plugin registry" and further > initializers will > automatically get the existing instance. > > > > > > > What do you think about the front-end logic to configure and > > > activate this special KMS writeback PipeWire source? Using dedicated > > > keys in the weston.ini file's [output] section for the KMS connector > > > seems like the straightforward way to me. I suppose we could talk > > > about a protocol extension, but that's (a) a lot more work and (b) > > > obscures the KMS connector IDs to the client, who sees only the > > > wl_output. > > Yes, I think the way to go is to do exactly that with weston.ini. If that's > enough, > then no need to consider any IPC or protocol. > > > Thanks, > pq
Hmm. As I read through the history of the original support for writeback screenshots (https://gitlab.freedesktop.org/wayland/weston/-/merge_requests/458) and get some initial results just trying to drive a steady stream of writeback composition, I'm not sure this can work effectively. MR 458 has a fairly long discussion about the restriction that no further commits must be made to the source CRTC or any of the other KMS objects reachable on its graph, while a writeback composition referencing that CRTC is still in flight. Daniel Vetter confirmed that interpretation. I think this means that unless you are extremely lucky that: (a) your hardware's writeback mechanism is synchronous with scanout, and (b) the fence fd manages to fire and get dispatched immediately upon scanout/writeback passing through the final scanline, and (c) there's a really large vertical back porch , there will be no time left to enqueue a page flip for the immediately succeeding vblank. Seems like this will almost always cut the frame achievable update rate on the main connector in half. Did I misinterpret something in there? Early results look like the kernel state machine gets confused (with VKMS anyway) if I queue up two writeback operations in flight at the same time. -Matt
