Yes, the native Linux driver for the hardware still does end up being
responsible for one or more planes.
Other than trying to arrange the pieces so that there's some API that offers
the client an option to guarantee that the source of the screen capture
involves the writeback connector (similar to what you've done with
weston_output_capture), I don't really think it would be a good idea to make
Weston explicitly aware of any of this funny hypervisor business going on.
The title on this email conversation was probably poorly chosen, in retrospect.
I'm not so much concerned with keeping absolutely to a zero-copy mechanism as I
am to using hardware mechanisms (GL rendering, DRI writeback,
hardware-accelerated blits as necessary) all the way through.
After seeing the way that the Pipewire backend handles streaming (especially
with https://gitlab.freedesktop.org/wayland/weston/-/merge_requests/1366), I'm
almost thinking that it would be preferable to maybe structure the design of
this feature something like this:
* Add some sort of API on weston_output by which it can advertise the ability
to lay hands on the explicit renderbuffer (e.g.
gbm_surface_get_front_buffer()). This roughly equates to whether it's a
primary, non-nested backend. That is, DRM.
* When processing the weston.ini mirror-of relationships at startup, check
whether the source output of the mirror-of declaration supports that ability
above. If so:
- Wire up a signal so that the source output announces each newly rendered
frame to any/all mirror-of outputs.
- The virtual output's backend allocates its own buffer pool in exactly the
same way that MR 1366 already does.
- Upon receipt of each signal announcing a new frame's renderbuffer from the
source output, the virtual output does a very cheap glBlitFramebuffer() to get
the contents into its own buffer pool. This avoids the possibility of an
unresponsive client tying up the source output's buffer.
* If the source output isn't one that supports exposing the underlying
renderbuffer, then the mirror-of relationship continues as with MR 1476 just to
invoke an explicit weston_renderer pass to draw the correct logical contents.
How does this strike you?
-Matt
> -Original Message-
> From: Pekka Paalanen
> Sent: Monday, June 10, 2024 3:03 AM
> To: Hoosier, Matt
> Cc: s...@cmpwn.com; cont...@emersion.fr; Marius Vlad
> ; wayland-devel@lists.freedesktop.org
> Subject: Re: Full-motion zero-copy screen capture in Weston
>
> On Fri, 7 Jun 2024 14:16:32 +
> "Hoosier, Matt" wrote:
>
> > > What do you mean you can capture all virtual machines with KMS
> > > writeback?
> > >
> > > What's the architecture there? How do virtual machines access KMS
> > > hardware? Why would Weston be able to capture their outputs at all?
> >
> > The world of virtualization on commercially supported embedded SOCs
> > for big-scale production use is wild. Every vendor typically has only
> > a narrow range of supported hypervisors. Usually, one -- and an
> > in-house model at that. There will typically be at least one bizarre
> > twist on the virtualization method for display controllers.
> >
> > One fad is for one of the virtual machines -- typically, the one
> > running a normal-style GNU/Linux Yocto system -- to own privileged I/O
> > maps of most of the hardware, and it more or less runs the same
> > drivers inside this VM that the SOC maker has already written for
> > their bare-metal Linux deployment. Most hardware peripherals are just
> > exposed to the other guest VMs by having the privileged Linux VM host
> > export some sort of hypervisor-integrated VirtIO-style backend for the
> > hardware. The guests see VirtIO devices. This approach goes by the
> > name of "paravirtualization".
> >
> > But for graphics and display, there is almost always some additional
> > mechanism to sidestep this pure paravirtualization because it's
> > perceived as too expensive. So the vendor may do something like
> > designate some subset of planes on each connector to be "directly"
> > manipulated by the non-GNU/Linux guest VMs. The hypervisor executive
> > runs a tiny little server that receives the stream of plane updates,
> > and during the vsync it programs the appropriate display controller
> > hardware registers to refer to the new frame's contents. It's very
> > limited -- the guests VMs whose scene updates are couched using this
> > mechanism are not able to do modesets or reconfigure the overall DRI
> > scene topology.
> >
> > The key point here is that because Linux is running a full physical
> > driver, the writeback connector gets the results of blending all the
> > layers -- even those whose contents are programmed using the awkward
> > side channel.
> >
> > I'm not a big fan on this approach. But it is there, and I'd like to
> > cope with it. I have a use-case that requires Linux to get a complete
> > picture of the physical contents getting scanned out by the connector.
> >
>
> I
