Re: Full-motion zero-copy screen capture in Weston

[Hoosier, Matt]

> What do you mean you can capture all virtual machines with KMS
writeback?
>
> What's the architecture there? How do virtual machines access KMS
hardware? Why would Weston be able to capture their outputs at all?

The world of virtualization on commercially supported embedded SOCs for 
big-scale production use is wild. Every vendor typically has only a narrow 
range of supported hypervisors. Usually, one -- and an in-house model at that. 
There will typically be at least one bizarre twist on the virtualization method 
for display controllers.

One fad is for one of the virtual machines -- typically, the one running a 
normal-style GNU/Linux Yocto system -- to own privileged I/O maps of most of 
the hardware, and it more or less runs the same drivers inside this VM that the 
SOC maker has already written for their bare-metal Linux deployment. Most 
hardware peripherals are just exposed to the other guest VMs by having the 
privileged Linux VM host export some sort of hypervisor-integrated VirtIO-style 
backend for the hardware. The guests see VirtIO devices. This approach goes by 
the name of "paravirtualization". 

But for graphics and display, there is almost always some additional mechanism 
to sidestep this pure paravirtualization because it's perceived as too 
expensive. So the vendor may do something like designate some subset of planes 
on each connector to be "directly" manipulated by the non-GNU/Linux guest VMs. 
The hypervisor executive runs a tiny little server that receives the stream of 
plane updates, and during the vsync it programs the appropriate display 
controller hardware registers to refer to the new frame's contents. It's very 
limited -- the guests VMs whose scene updates are couched using this mechanism 
are not able to do modesets or reconfigure the overall DRI scene topology.

The key point here is that because Linux is running a full physical driver, the 
writeback connector gets the results of blending all the layers -- even those 
whose contents are programmed using the awkward side channel.

I'm not a big fan on this approach. But it is there, and I'd like to cope with 
it. I have a use-case that requires Linux to get a complete picture of the 
physical contents getting scanned out by the connector.

-Matt

On 6/3/24, 3:54 AM, "Pekka Paalanen" mailto:pekka.paala...@collabora.com>> wrote:


On Fri, 31 May 2024 13:26:12 +
"Hoosier, Matt" mailto:matt.hoos...@garmin.com>> 
wrote:


> 
> My goal is to implement this screen capture with a guarantee that the
> copy comes from a KMS writeback connector. I know this sounds like an
> odd thing to insist on. Let's say that in my industry, the system is
> rarely only running Linux directly on the bare metal. Using the
> writeback hardware on the display controller allows to get a copy of
> content from all the virtual machines in the system.
> 


What do you mean you can capture all virtual machines with KMS
writeback?


What's the architecture there? How do virtual machines access KMS
hardware? Why would Weston be able to capture their outputs at all?


> Frankly, weston_output_capture_v1's model that clients allocate the
> buffers would make it very difficult to support efficient screen
> capture for more than one simultaneous client. You can only target
> one writeback framebuffer per page flip. Having the compositor manage
> the buffers' lifetimes and just publishing out handles (in the style
> of those two wlr extensions) to those probably fits better.


That's certainly true.


The disadvantage is that buffer allocations are accounted to the
compositor (which can manage its own memory consumption, sure), and
either the compositor allocates a new buffer for every frame (possibly
serious overhead) or it needs to wait for all consumers to tell that
they are done with the buffer before it can be used again. Clients
might hold on to the buffer indefinitely or just a little too long,
which is the risk.




Thanks,
pq

RE: Ways to test Weston during development (Re: Full-motion zero-copy screen capture in Weston)

[Hoosier, Matt]

Would Meson’s dependency wrapping capabilities be a viable solution here? I 
think that most of Weston’s dependencies that have aggressive version 
requirements are themselves also Meson projects.

The Weston CI configuration builds a bunch of its dependencies (Mesa, libdrm, 
libwayland …) manually. I wonder why Meson wrapping was not used for this?

-Matt

From: Hoosier, Matt 
Sent: Wednesday, June 5, 2024 7:44 AM
To: Daniel Stone ; Pekka Paalanen 

Cc: s...@cmpwn.com; cont...@emersion.fr; Marius Vlad 
; wayland-devel@lists.freedesktop.org
Subject: Re: Ways to test Weston during development (Re: Full-motion zero-copy 
screen capture in Weston)

Thanks, everybody.

After some trial and error, I find that if I install seatd in the host and the 
seatd dev package in the Toolbox container and I then symlink the host seatd 
socket into /tmp on the container, Weston seems to start up okay on my physical 
KMS connectors:

user@host:~$ toolbox enter
…

user@toolbox:~$ sudo ln -s /run/host/run/seatd.socket /run/
user@toolbox:~$ weston --backend=drm

-Matt

From: Daniel Stone mailto:dan...@fooishbar.org>>
Date: Wednesday, June 5, 2024 at 5:07 AM
To: Pekka Paalanen 
mailto:pekka.paala...@collabora.com>>
Cc: "Hoosier, Matt" mailto:matt.hoos...@garmin.com>>, 
"s...@cmpwn.com" 
mailto:s...@cmpwn.com>>, 
"cont...@emersion.fr" 
mailto:cont...@emersion.fr>>, Marius Vlad 
mailto:marius.v...@collabora.com>>, 
"wayland-devel@lists.freedesktop.org"
 
mailto:wayland-devel@lists.freedesktop.org>>
Subject: Re: Ways to test Weston during development (Re: Full-motion zero-copy 
screen capture in Weston)

Hi, On Wed, 5 Jun 2024 at 09: 09, Pekka Paalanen  wrote: > On Tue, 4 Jun 2024 20: 33: 48 + > "Hoosier, Matt"  wrote: > > Tactical question: I somehow missed until


Hi,



On Wed, 5 Jun 2024 at 09:09, Pekka Paalanen

mailto:pekka.paala...@collabora.com>> wrote:

> On Tue, 4 Jun 2024 20:33:48 +

> "Hoosier, Matt" mailto:matt.hoos...@garmin.com>> 
> wrote:

> > Tactical question: I somehow missed until this point that the remote

> > and pipewire plugins will only run if the DRM backend is being used.

> >

> > But the DRM backend *really* doesn't want to start nowadays unless

> > you're running on a system with seatd and/or logind available.

> > Toolbox [1] is the de facto way to develop on bleeding edge copies of

> > components these days. But it logind and seatd aren't exposed into it.

> >

> > How do Weston people interactively develop on the Weston DRM backend

> > nowadays?

> >

> > [1] 
> > https://urldefense.com/v3/__https://docs.fedoraproject.org/en-US/fedora-silverblue/toolbox/__;!!EJc4YC3iFmQ!X9PIq-FMYhremNjhMhD6uGHUdYcYt4QKcl5CIjQAi0gYX5IyFx63gK1QZDWepwQBxB3mu1WTvjK2SJoQzfs$

>

> I'm doing it old-school on my workstation, without any containers. What

> dependencies my distribution does not provide, I build and install

> manually into a prefix under $HOME:

>

> https://urldefense.com/v3/__https://www.collabora.com/news-and-blog/blog/2020/04/10/clean-reliable-setup-for-dependency-installation/__;!!EJc4YC3iFmQ!X9PIq-FMYhremNjhMhD6uGHUdYcYt4QKcl5CIjQAi0gYX5IyFx63gK1QZDWepwQBxB3mu1WTvjK2Y5E0gB4$

>

> The "clean and reliable" is probably outdated in this era of

> containers...



Yes, doing it in containers is a little bit tricky since it's not

exactly the design case. Honestly, on my Silverblue systems, I just

install a bunch of relevant dependencies into the system image with

rpm-ostree, and have a pile of self-built dependencies in a local

prefix.



This might give you some insight however:

https://urldefense.com/v3/__https://github.com/containers/toolbox/issues/992__;!!EJc4YC3iFmQ!X9PIq-FMYhremNjhMhD6uGHUdYcYt4QKcl5CIjQAi0gYX5IyFx63gK1QZDWepwQBxB3mu1WTvjK21Tr-34M$



It probably needs some minor changes in Weston but does at least seem doable ...



Cheers,

Daniel

Re: Ways to test Weston during development (Re: Full-motion zero-copy screen capture in Weston)

[Daniel Stone]

Hi Matt,

On Fri, 7 Jun 2024 at 15:30, Hoosier, Matt  wrote:
> Would Meson’s dependency wrapping capabilities be a viable solution here? I 
> think that most of Weston’s dependencies that have aggressive version 
> requirements are themselves also Meson projects.
>
> The Weston CI configuration builds a bunch of its dependencies (Mesa, libdrm, 
> libwayland …) manually. I wonder why Meson wrapping was not used for this?

We don't want to rebuild Mesa every time. We could've built it as a
subproject and cached it, but it didn't seem to offer much any
advantage over just installing it into the system.

We could probably add some subprojects, but you'd probably end up
pulling in more components as well - e.g. if you want to run Mesa with
its software renderer or the AMD drivers, you'll also need to use LLVM
- and at what point does your easy subproject build turn into, well, a
full distribution?

I guess one thing we could do is to jazz the CI build up a little so
it's easier to pull the OCI and run it inside a toolbox, as well as
reuse those scripts locally.

Cheers,
Daniel

RE: Ways to test Weston during development (Re: Full-motion zero-copy screen capture in Weston)

[Hoosier, Matt]

Hi Daniel,

Okay, makes sense that you don’t want to have to repeat the dependencies’ 
builds for every CI test. I’m not arguing that you should – it was just more a 
thought experiment to see whether riding Meson subprojects is a reasonable idea 
for establishing a development environment.

I get your point that that can become a deep rabbit hole. But it seems that you 
didn’t have any need to build LLVM and similar just to support the hand-built 
copy of Mesa that’s in the CI. Is there some reason why a deeper set of 
transitive dependencies would be needed using Meson subprojects than when 
building by hand? Seems like I could probably just mimic what you’ve done. 
Maybe your point is that the CI is a very constrained environment that’s known 
not to need ATI or llvmpipe, but a general developer situation with physical 
machines would?

-Matt

From: Daniel Stone 
Sent: Friday, June 7, 2024 10:17 AM
To: Hoosier, Matt 
Cc: Pekka Paalanen ; Marius Vlad 
; wayland-devel@lists.freedesktop.org
Subject: Re: Ways to test Weston during development (Re: Full-motion zero-copy 
screen capture in Weston)

Hi Matt, On Fri, 7 Jun 2024 at 15: 30, Hoosier, Matt  wrote: > Would Meson’s dependency wrapping capabilities be a viable 
solution here? I think that most of Weston’s dependencies that have aggressive 
version
jQcmQRYFpfptBannerStart
Hi Matt,



On Fri, 7 Jun 2024 at 15:30, Hoosier, Matt 
mailto:matt.hoos...@garmin.com>> wrote:

> Would Meson’s dependency wrapping capabilities be a viable solution here? I 
> think that most of Weston’s dependencies that have aggressive version 
> requirements are themselves also Meson projects.

>

> The Weston CI configuration builds a bunch of its dependencies (Mesa, libdrm, 
> libwayland …) manually. I wonder why Meson wrapping was not used for this?



We don't want to rebuild Mesa every time. We could've built it as a

subproject and cached it, but it didn't seem to offer much any

advantage over just installing it into the system.



We could probably add some subprojects, but you'd probably end up

pulling in more components as well - e.g. if you want to run Mesa with

its software renderer or the AMD drivers, you'll also need to use LLVM

- and at what point does your easy subproject build turn into, well, a

full distribution?



I guess one thing we could do is to jazz the CI build up a little so

it's easier to pull the OCI and run it inside a toolbox, as well as

reuse those scripts locally.



Cheers,

Daniel