Re: Full-motion zero-copy screen capture in Weston

2024-05-29 Thread Pekka Paalanen
On Tue, 28 May 2024 20:04:45 +0300
Marius Vlad  wrote:

> On Tue, May 28, 2024 at 03:53:23PM +, Hoosier, Matt wrote:
> > Hi Marius,  
> Hi,
> > 
> > Okay, I guess that answers the bit about needing to screen-scrape to get 
> > content into Pipewire now.
> > 
> > But I'm still a little unclear about a couple things, if I were to try to 
> > build on this PW backend as a starting point:
> > 
> > First, it looks to me like when you use the PW backend to Weston, that
> > becomes your display. That is, rendering directly targets it. I was
> > hoping for a way to get it to broadcast the very same framebuffer(s)
> > that are getting scanned out for the current frame by the DRM
> > backend.  
> With https://gitlab.freedesktop.org/wayland/weston/-/merge_requests/341
> the PipeWire output can mirror out the native DRM one. It aims at having
> a way to configure VNC, RDP and PipeWire to screen-share the output. 
> > 
> > Second, I'm don't see the path to getting this to leverage the
> > DRM_MODE_CONNECTOR_WRITEBACK hardware (like the weston_screen_recorder
> > does).  I think that any layering would be forced to
> > be offloaded to the GPU ahead of time. Maybe I missed some implication
> > of what you were pointing out here?  
> No sorry, I haven't really implied that, just pointed that out there's
> some work for PipeWire gaining dmabuf. 
> 
> Screen-sharing an output is done
> with multiple backends, and configuring Weston front-end is it with 
> https://gitlab.freedesktop.org/wayland/weston/-/merge_requests/341.

Hi Matt, Marius,

multiple backends necessarily means that each backend renders its own
weston_output independently. While it's fine for most screen-sharing
use cases, I don't think it can ever be as efficient as KMS writeback
could be.

Getting a dmabuf-based video stream from DRM-backend using KMS
writeback needs integration in the DRM-backend. I don't know how that
would look like, but I suspect these ideas would be good:

- builds on and extends the existing writeback support instead of
  something new on the side

- does not use the DRM-backend "virtual output" API; this was a
  workaround for the lack of multi-backend support in the past and has
  the same disadvantages as multi-backend. (I don't think the DRM
  virtual output API has any reason to exist anymore, multi-backend
  should supersede it after converting the users.)

I also think that implementing the usual desktop portals in Weston
would be a good thing, and not something to avoid, if those can provide
a nice external interface for the use case.


Thanks,
pq


pgp2frA_CAvcb.pgp
Description: OpenPGP digital signature


RE: Full-motion zero-copy screen capture in Weston

2024-05-29 Thread Hoosier, Matt
Thanks, Pekka.

So what would be an upstream-friendly way to put hooks deep down into the DRM 
backend to exploit the KMS writeback connectors, but still allow an XDG Desktop 
portal implementation to be usable on all the backends? Or maybe that's an 
unnecessary goal to begin with; a desktop session somewhat implies the 
compositor is driving the physical hardware?

> -Original Message-
> From: Pekka Paalanen 
> Sent: Wednesday, May 29, 2024 2:43 AM
> To: Marius Vlad ; Hoosier, Matt
> 
> Cc: wayland-devel@lists.freedesktop.org
> Subject: Re: Full-motion zero-copy screen capture in Weston
> 
> On Tue, 28 May 2024 20:04:45 +0300
> Marius Vlad  wrote:
> 
> > On Tue, May 28, 2024 at 03:53:23PM +, Hoosier, Matt wrote:
> > > Hi Marius,
> > Hi,
> > >
> > > Okay, I guess that answers the bit about needing to screen-scrape to get
> content into Pipewire now.
> > >
> > > But I'm still a little unclear about a couple things, if I were to try to 
> > > build on
> this PW backend as a starting point:
> > >
> > > First, it looks to me like when you use the PW backend to Weston,
> > > that becomes your display. That is, rendering directly targets it. I
> > > was hoping for a way to get it to broadcast the very same
> > > framebuffer(s) that are getting scanned out for the current frame by
> > > the DRM backend.
> > With
> > https://gitlab.freedesktop.org/wayland/weston/-/merge_requests/341
> > the PipeWire output can mirror out the native DRM one. It aims at
> > having a way to configure VNC, RDP and PipeWire to screen-share the
> output.
> > >
> > > Second, I'm don't see the path to getting this to leverage the
> > > DRM_MODE_CONNECTOR_WRITEBACK hardware (like the
> > > weston_screen_recorder does).  I think that any layering would be
> > > forced to be offloaded to the GPU ahead of time. Maybe I missed some
> > > implication of what you were pointing out here?
> > No sorry, I haven't really implied that, just pointed that out there's
> > some work for PipeWire gaining dmabuf.
> >
> > Screen-sharing an output is done
> > with multiple backends, and configuring Weston front-end is it with
> > https://gitlab.freedesktop.org/wayland/weston/-/merge_requests/341.
> 
> Hi Matt, Marius,
> 
> multiple backends necessarily means that each backend renders its own
> weston_output independently. While it's fine for most screen-sharing use
> cases, I don't think it can ever be as efficient as KMS writeback could be.
> 
> Getting a dmabuf-based video stream from DRM-backend using KMS
> writeback needs integration in the DRM-backend. I don't know how that
> would look like, but I suspect these ideas would be good:
> 
> - builds on and extends the existing writeback support instead of
>   something new on the side
> 
> - does not use the DRM-backend "virtual output" API; this was a
>   workaround for the lack of multi-backend support in the past and has
>   the same disadvantages as multi-backend. (I don't think the DRM
>   virtual output API has any reason to exist anymore, multi-backend
>   should supersede it after converting the users.)
> 
> I also think that implementing the usual desktop portals in Weston would be a
> good thing, and not something to avoid, if those can provide a nice external
> interface for the use case.
> 
> 
> Thanks,
> pq