I'm adding features to VKMS! What would you like to see?
Hi Wayland folks! TL;DR: I'm working on extending VKMS and wanted feedback from other compositor/wayland devs. // Background I work on the ChromeOS compositor, and recently I've been doing a bunch of stuff to improve our testing setup. At the moment, my main focus is improving our ability to write integration tests against DRM/KMS. It's pretty tricky to get right. Working with mocks of DRM loses all the useful helpers that live within the kernel, which would need to be rewritten (and kept up-to-date) in userspace. Stuff like writeback support would be even harder. Earlier this year, VKMS came up as a potential solution. I was happy to see that Weston is already using it. I've started thinking about what features from the wild we'd need, and started digging into the code. // Current Status I recently sent out my first patchset, which will let userspace build their own DRM drivers with ConfigFS. This implicitly adds support for multi-display setups which were impossible to test before. This also allows for multiple virtual DRM drivers to be created and used at the same time, which may increase test parallelism? Haven't tried it yet. v1 patchset: https://patchwork.kernel.org/project/dri-devel/list/?series=662676 cover letter: https://lists.freedesktop.org/archives/dri-devel/2022-July/365647.html // Rough Plans The big features I want to target with this work are: - Multi-display and movable planes. This is mostly covered by the ConfigFS changes. - Hot plugging. - Color, color management and HDR. Loads of new formats, support for color properties not currently implemented. Making sure writeback buffers are useful for this. - Improve IGT testing for VKMS (for new features and existing skipped tests) // Questions - What VKMS features could help your testing the most? - How much do you care about writeback buffer support vs CRC checks in tests atm? - What kinds of bugs do you get around DRM/KMS? - Any thoughts in general? Thanks! -- Jim
Re: I'm adding features to VKMS! What would you like to see?
Hi! On Thursday, July 28th, 2022 at 20:54, Jim Shargo wrote: > TL;DR: I'm working on extending VKMS and wanted feedback from other > compositor/wayland devs. Very nice, thanks for reaching out! > - What VKMS features could help your testing the most? I think the most valuable features are the ones which typically require special hw to test (such as DP-MST hub, HDR monitor, bad DP cable…). Here are a few ideas: - Connector hotplug (switch status from disconnected to connected and back) - Connector registering as it happens on DP-MST setups (new connectors appearing, old connectors disappearing) - Basic format modifier support (to make sure that code-path is tested in CI) - link-status change emulation - FB_DAMAGE_CLIPS handling - Multiple planes with customizable constraints (not sure how this would look like yet) - A handful of simpler props like panel_orientation, non-desktop, TILE, etc > - How much do you care about writeback buffer support vs CRC checks > in tests atm? Maybe a question for Pekka? > - What kinds of bugs do you get around DRM/KMS? I think most bugs are around modesets (black screens). This may sound pretty basic, but it sounds like hw differences (ie. large test matrix and different limitations), hubs and different sequences of user actions (plug/unplug/config changes) make turning on a display a complicated operation. Or maybe it's the most noticeable bug, so users are more likely to open a bug report for it. (Note, sometimes the bug ends up being a driver issue.) Hope this helps, Simon
Re: I'm adding features to VKMS! What would you like to see?
Hi Am 28.07.22 um 20:54 schrieb Jim Shargo: Hi Wayland folks! TL;DR: I'm working on extending VKMS and wanted feedback from other compositor/wayland devs. Thanks for helping! // Background I work on the ChromeOS compositor, and recently I've been doing a bunch of stuff to improve our testing setup. At the moment, my main focus is improving our ability to write integration tests against DRM/KMS. It's pretty tricky to get right. Working with mocks of DRM loses all the useful helpers that live within the kernel, which would need to be rewritten (and kept up-to-date) in userspace. Stuff like writeback support would be even harder. Earlier this year, VKMS came up as a potential solution. I was happy to see that Weston is already using it. I've started thinking about what features from the wild we'd need, and started digging into the code. // Current Status I recently sent out my first patchset, which will let userspace build their own DRM drivers with ConfigFS. This implicitly adds support for multi-display setups which were impossible to test before. This also allows for multiple virtual DRM drivers to be created and used at the same time, which may increase test parallelism? Haven't tried it yet. v1 patchset: https://patchwork.kernel.org/project/dri-devel/list/?series=662676 cover letter: https://lists.freedesktop.org/archives/dri-devel/2022-July/365647.html // Rough Plans The big features I want to target with this work are: - Multi-display and movable planes. This is mostly covered by the ConfigFS changes. - Hot plugging. - Color, color management and HDR. Loads of new formats, support for color properties not currently implemented. Making sure writeback buffers are useful for this. - Improve IGT testing for VKMS (for new features and existing skipped tests) // Questions - What VKMS features could help your testing the most? - How much do you care about writeback buffer support vs CRC checks in tests atm? - What kinds of bugs do you get around DRM/KMS? - Any thoughts in general? I think damage handling might be useful. Userspace tests can set damage areas and readback the the results or CRC checksum. See the kernel's drm_plane_enable_fb_damage_clips() and struct drm_atomic_helper_damage_iter. Best regards Thomas Thanks! -- Jim -- Thomas Zimmermann Graphics Driver Developer SUSE Software Solutions Germany GmbH Maxfeldstr. 5, 90409 Nürnberg, Germany (HRB 36809, AG Nürnberg) Geschäftsführer: Ivo Totev OpenPGP_signature Description: OpenPGP digital signature
Re: I'm adding features to VKMS! What would you like to see?
On Thu, 28 Jul 2022 14:54:31 -0400 Jim Shargo wrote: > Hi Wayland folks! > > TL;DR: I'm working on extending VKMS and wanted feedback from other > compositor/wayland devs. Hi Jim! > // Background > > I work on the ChromeOS compositor, and recently I've been doing a > bunch of stuff to improve our testing setup. At the moment, my main > focus is improving our ability to write integration tests against > DRM/KMS. > > It's pretty tricky to get right. Working with mocks of DRM loses all > the useful helpers that live within the kernel, which would need to be > rewritten (and kept up-to-date) in userspace. Stuff like writeback > support would be even harder. > > Earlier this year, VKMS came up as a potential solution. I was happy > to see that Weston is already using it. I've started thinking about > what features from the wild we'd need, and started digging into the > code. > > // Current Status > > I recently sent out my first patchset, which will let userspace build > their own DRM drivers with ConfigFS. This implicitly adds support for > multi-display setups which were impossible to test before. This also > allows for multiple virtual DRM drivers to be created and used at the > same time, which may increase test parallelism? Haven't tried it yet. > > v1 patchset: > https://patchwork.kernel.org/project/dri-devel/list/?series=662676 > cover letter: > https://lists.freedesktop.org/archives/dri-devel/2022-July/365647.html Personally I've disabled mail deliver from dri-devel list in order to focus my time better, but please do CC me on VKMS things. The ConfigFS UAPI looked nice to me, not having any contact with ConfigFS before. > // Rough Plans > > The big features I want to target with this work are: > - Multi-display and movable planes. This is mostly covered by the > ConfigFS changes. Yes! \o/ > - Hot plugging. Yes! \o/ > - Color, color management and HDR. Loads of new formats, support for > color properties not currently implemented. Making sure writeback > buffers are useful for this. Yes! \o/ Please check VKMS patches from Igor Torrente: https://lore.kernel.org/dri-devel/20220614000226.93297-1-igormtorre...@gmail.com/ They add some new format support with a new compositing algorithm. > - Improve IGT testing for VKMS (for new features and existing skipped tests) Awesome, though not directly benefiting me/Weston. Making sure all drivers implement the KMS properties the same way is crucial. > > // Questions > > - What VKMS features could help your testing the most? For Weston, hotplug might be the immediate low-hanging fruit. Multiple planes with writeback. Multiple outputs with different "hardware" timing cycles. Different plane z-order and format restrictions. Anything really. Also, getting feedback on HDMI/DP infoframe contents. That's going to become especially important with HDR monitors, and while there is not much to test (the driver and hardware are supposed to be just pass-through for metadata), it would be nice to check userspace is using the KMS UAPI right. > - How much do you care about writeback buffer support vs CRC checks > in tests atm? It is much cleaner to use writeback for Weston KMS tests than it would be to use CRC checks. In fact, once we get to high precision pixel formats or testing things like LUT KMS elements, I think CRC checks will become useless. We cannot store CRC values because they are driver and hardware dependent, so every test would need to realize the same scene twice: once with the bells and whistles, and once without to generate the reference CRC. That gets us the CRC computation itself, but it does not get us the KMS color pipeline. The KMS pipeline might be rounding values differently than what we do for the reference image. Even simple alpha-blending might be untestable with CRC because of that. CRC cannot allow any tolerance in the pixel values. Being able to use and control the per-pixel error tolerance is going to be absolutely crucial, so writeback is the only way. > - What kinds of bugs do you get around DRM/KMS? I guess things we did not expect more than mistakes in Weston, or situations we don't handle: - link-status - temporary EBUSY https://gitlab.freedesktop.org/wayland/weston/-/issues/435 - needing to downgrade the framebuffer format modifier on an already used CRTC in order to be able to light up another CRTC (whole-device or global bandwidth limitations) - other KMS clients leaving the KMS in unexpected state > - Any thoughts in general? Awesome! Woo! \o/ I do hope you find reviewers. Thanks, pq pgpC7EH2PcUse.pgp Description: OpenPGP digital signature
Re: I'm adding features to VKMS! What would you like to see?
Hi Jim, On Fri, 29 Jul 2022 at 08:30, Jim Shargo wrote: > TL;DR: I'm working on extending VKMS and wanted feedback from other > compositor/wayland devs. Awesome! :) Glad to see it, and yeah, I second everything Pekka said. Having hotplug in particular would be really great. > // Questions > > - What VKMS features could help your testing the most? > - How much do you care about writeback buffer support vs CRC checks > in tests atm? > - What kinds of bugs do you get around DRM/KMS? > - Any thoughts in general? One thing I've really wanted is corner-case handling which just can't be done in generic code. Weston is really aggressive at trying to get things into planes, but we can only test those on actual systems with particular semantics. I'd love to be able to programmatically fake those to be able to check our fallbacks in an automatic way. About the best idea I've come up with for that is being able to attach an eBPF hook to atomic_check. The absolute MVP would be no arguments and an errno return; if you completely control the environment, then you can store a counter in a map and return a particular error for the n'th attempt. But a better one would allow you to inspect the properties on each object in the atomic state, and also things like framebuffer attributes (dimensions, format, modifier, etc) so you could take action accordingly. Thanks for taking this on! Really looking forward to it. Cheers, Daniel
Re: I'm adding features to VKMS! What would you like to see?
On Fri, Jul 29, 2022 at 12:09:27PM +0100, Daniel Stone wrote: > Hi Jim, > > On Fri, 29 Jul 2022 at 08:30, Jim Shargo wrote: > > TL;DR: I'm working on extending VKMS and wanted feedback from other > > compositor/wayland devs. > > Awesome! :) Glad to see it, and yeah, I second everything Pekka said. > Having hotplug in particular would be really great. Enabling multi monitor and hotplug testing would be on my top wish list too. > > > // Questions > > > > - What VKMS features could help your testing the most? > > - How much do you care about writeback buffer support vs CRC checks > > in tests atm? > > - What kinds of bugs do you get around DRM/KMS? > > - Any thoughts in general? > > One thing I've really wanted is corner-case handling which just can't > be done in generic code. Weston is really aggressive at trying to get > things into planes, but we can only test those on actual systems with > particular semantics. > > I'd love to be able to programmatically fake those to be able to check > our fallbacks in an automatic way. About the best idea I've come up > with for that is being able to attach an eBPF hook to atomic_check. > The absolute MVP would be no arguments and an errno return; if you > completely control the environment, then you can store a counter in a > map and return a particular error for the n'th attempt. But a better > one would allow you to inspect the properties on each object in the > atomic state, and also things like framebuffer attributes (dimensions, > format, modifier, etc) so you could take action accordingly. Being able to queue errors would be great indeed. So far I've done this by mocking a subset of the libdrm API, which I could use to test e.g. failed commits handling: https://gitlab.gnome.org/jadahl/mutter/-/commit/93995754484536c61c7ddeffa2e50e4aef092a78 Jonas > > Thanks for taking this on! Really looking forward to it. > > Cheers, > Daniel
Re: I'm adding features to VKMS! What would you like to see?
On Fri, Jul 29, 2022 at 3:30 AM Jim Shargo wrote: > > Hi Wayland folks! > > TL;DR: I'm working on extending VKMS and wanted feedback from other > compositor/wayland devs. >> // Background > > I work on the ChromeOS compositor, and recently I've been doing a > bunch of stuff to improve our testing setup. At the moment, my main > focus is improving our ability to write integration tests against > DRM/KMS. > > It's pretty tricky to get right. Working with mocks of DRM loses all > the useful helpers that live within the kernel, which would need to be > rewritten (and kept up-to-date) in userspace. Stuff like writeback > support would be even harder. > > Earlier this year, VKMS came up as a potential solution. I was happy > to see that Weston is already using it. I've started thinking about > what features from the wild we'd need, and started digging into the > code. > > // Current Status > > I recently sent out my first patchset, which will let userspace build > their own DRM drivers with ConfigFS. This implicitly adds support for > multi-display setups which were impossible to test before. This also > allows for multiple virtual DRM drivers to be created and used at the > same time, which may increase test parallelism? Haven't tried it yet. > > v1 patchset: > https://patchwork.kernel.org/project/dri-devel/list/?series=662676 > cover letter: > https://lists.freedesktop.org/archives/dri-devel/2022-July/365647.html > > // Rough Plans > > The big features I want to target with this work are: > - Multi-display and movable planes. This is mostly covered by the > ConfigFS changes. > - Hot plugging. > - Color, color management and HDR. Loads of new formats, support for > color properties not currently implemented. Making sure writeback > buffers are useful for this. > - Improve IGT testing for VKMS (for new features and existing skipped tests) > > // Questions > > - What VKMS features could help your testing the most? > - How much do you care about writeback buffer support vs CRC checks > in tests atm? > - What kinds of bugs do you get around DRM/KMS? > - Any thoughts in general? Hey, nice work! So, not really related to wayland, but it would be cool if we could add a generic vkms helper library for drivers to use for virtual display functionality. E.g., for headless GPUs in data centers or for virtualization. We ended up pulling a version of dkms into amdgpu to replace our old non-atomic virtual display support. We tried at the time to make a generic helper, but ended up running into a lot of problems around memory access because on real GPUs the vkms is backed by vram rather than system memory. I know we could use vkms as a standalone driver, but that runs into the same issue in that the vkms is backed by system memory rather than vram. The latter is desirable because we want to be able to use vram directly so we can use the framebuffer as a source for hardware video encoding for example. Alex > > Thanks! > -- Jim
Re: I'm adding features to VKMS! What would you like to see?
On Fri, Jul 29, 2022 at 12:07:20PM -0400, Alex Deucher wrote: > On Fri, Jul 29, 2022 at 3:30 AM Jim Shargo wrote: > > > > Hi Wayland folks! > > > > TL;DR: I'm working on extending VKMS and wanted feedback from other > > compositor/wayland devs. > >> // Background > > > > I work on the ChromeOS compositor, and recently I've been doing a > > bunch of stuff to improve our testing setup. At the moment, my main > > focus is improving our ability to write integration tests against > > DRM/KMS. > > > > It's pretty tricky to get right. Working with mocks of DRM loses all > > the useful helpers that live within the kernel, which would need to be > > rewritten (and kept up-to-date) in userspace. Stuff like writeback > > support would be even harder. > > > > Earlier this year, VKMS came up as a potential solution. I was happy > > to see that Weston is already using it. I've started thinking about > > what features from the wild we'd need, and started digging into the > > code. > > > > // Current Status > > > > I recently sent out my first patchset, which will let userspace build > > their own DRM drivers with ConfigFS. This implicitly adds support for > > multi-display setups which were impossible to test before. This also > > allows for multiple virtual DRM drivers to be created and used at the > > same time, which may increase test parallelism? Haven't tried it yet. > > > > v1 patchset: > > https://patchwork.kernel.org/project/dri-devel/list/?series=662676 > > cover letter: > > https://lists.freedesktop.org/archives/dri-devel/2022-July/365647.html > > > > // Rough Plans > > > > The big features I want to target with this work are: > > - Multi-display and movable planes. This is mostly covered by the > > ConfigFS changes. > > - Hot plugging. > > - Color, color management and HDR. Loads of new formats, support for > > color properties not currently implemented. Making sure writeback > > buffers are useful for this. > > - Improve IGT testing for VKMS (for new features and existing skipped > > tests) > > > > // Questions > > > > - What VKMS features could help your testing the most? > > - How much do you care about writeback buffer support vs CRC checks > > in tests atm? > > - What kinds of bugs do you get around DRM/KMS? > > - Any thoughts in general? > > Hey, nice work! > > So, not really related to wayland, but it would be cool if we could > add a generic vkms helper library for drivers to use for virtual > display functionality. E.g., for headless GPUs in data centers or for > virtualization. With a risk of continuing being a bit off topic... What use cases is there a need to pass this functionality via virtual mode setting to implement virtual monitors in display servers, for headless and virtualization? I realize that there is a use case for virtual mode setting outputs for virtual machines meant to e.g. test run distributions while imitating real hardware as close as possible, but for actual intended remote access to headless machines or virtual machines, compositors can just render to offscreen framebuffers and communicate in whatever way, e.g. using DMA buffers via some IPC, with the software solution intended to provide the actual access. Jonas
Re: I'm adding features to VKMS! What would you like to see?
On Fri, Jul 29, 2022 at 1:28 PM Jonas Ådahl wrote: > > On Fri, Jul 29, 2022 at 12:07:20PM -0400, Alex Deucher wrote: > > On Fri, Jul 29, 2022 at 3:30 AM Jim Shargo wrote: > > > > > > Hi Wayland folks! > > > > > > TL;DR: I'm working on extending VKMS and wanted feedback from other > > > compositor/wayland devs. > > >> // Background > > > > > > I work on the ChromeOS compositor, and recently I've been doing a > > > bunch of stuff to improve our testing setup. At the moment, my main > > > focus is improving our ability to write integration tests against > > > DRM/KMS. > > > > > > It's pretty tricky to get right. Working with mocks of DRM loses all > > > the useful helpers that live within the kernel, which would need to be > > > rewritten (and kept up-to-date) in userspace. Stuff like writeback > > > support would be even harder. > > > > > > Earlier this year, VKMS came up as a potential solution. I was happy > > > to see that Weston is already using it. I've started thinking about > > > what features from the wild we'd need, and started digging into the > > > code. > > > > > > // Current Status > > > > > > I recently sent out my first patchset, which will let userspace build > > > their own DRM drivers with ConfigFS. This implicitly adds support for > > > multi-display setups which were impossible to test before. This also > > > allows for multiple virtual DRM drivers to be created and used at the > > > same time, which may increase test parallelism? Haven't tried it yet. > > > > > > v1 patchset: > > > https://patchwork.kernel.org/project/dri-devel/list/?series=662676 > > > cover letter: > > > https://lists.freedesktop.org/archives/dri-devel/2022-July/365647.html > > > > > > // Rough Plans > > > > > > The big features I want to target with this work are: > > > - Multi-display and movable planes. This is mostly covered by the > > > ConfigFS changes. > > > - Hot plugging. > > > - Color, color management and HDR. Loads of new formats, support for > > > color properties not currently implemented. Making sure writeback > > > buffers are useful for this. > > > - Improve IGT testing for VKMS (for new features and existing skipped > > > tests) > > > > > > // Questions > > > > > > - What VKMS features could help your testing the most? > > > - How much do you care about writeback buffer support vs CRC checks > > > in tests atm? > > > - What kinds of bugs do you get around DRM/KMS? > > > - Any thoughts in general? > > > > Hey, nice work! > > > > So, not really related to wayland, but it would be cool if we could > > add a generic vkms helper library for drivers to use for virtual > > display functionality. E.g., for headless GPUs in data centers or for > > virtualization. > > With a risk of continuing being a bit off topic... > > What use cases is there a need to pass this functionality via virtual > mode setting to implement virtual monitors in display servers, for > headless and virtualization? > > I realize that there is a use case for virtual mode setting outputs for > virtual machines meant to e.g. test run distributions while imitating real > hardware as close as possible, but for actual intended remote access to > headless machines or virtual machines, compositors can just render to > offscreen framebuffers and communicate in whatever way, e.g. using DMA > buffers via some IPC, with the software solution intended to provide the > actual access. Being able to run any distro (or compositor) unmodified on headless GPUs on servers is the big one. But you can do other things like fetch EDIDs from displays on your local system and then inject them into the virtual connectors on the server and start a virtual desktop with the same "displays" as the local one which you can then use to stream to your local system which is useful for thin clients. We also use it for pre-silicon emulation and early silicon bring-up internally when we still want a standard "desktop" on the GPU. Alex
