On Tue, 15 Nov 2022 00:11:56 +0100
Sebastian Wick wrote:
> There are still regular bug reports about monitors (sinks) and sources
> disagreeing about the quantization range of the pixel data. In
> particular sources sending full range data when the sink expects
> limited range. From a user space perspective, this is all hidden in
> the kernel. We send full range data to the kernel and then hope it
> does the right thing but as the bug reports show: some combinations of
> displays and drivers result in problems.
>
> In general the whole handling of the quantization range on linux is
> not defined or documented at all. User space sends full range data
> because that's what seems to work most of the time but technically
> this is all undefined and user space can not fix those issues. Some
> compositors have resorted to giving users the option to choose the
> quantization range but this really should only be necessary for
> straight up broken hardware.
>
> Quantization Range can be explicitly controlled by AVI InfoFrame or
> HDMI General Control Packets. This is the ideal case and when the
> source uses them there is not a lot that can go wrong. Not all
> displays support those explicit controls in which case the chosen
> video format (IT, CE, SD; details in CTA-861-H 5.1) influences which
> quantization range the sink expects.
>
> This means that we have to expect that sometimes we have to send
> limited and sometimes full range content. The big question however
> that is not answered in the docs: who is responsible for making sure
> the data is in the correct range? Is it the kernel or user space?
>
> If it's the kernel: does user space supply full range or limited range
> content? Each of those has a disadvantage. If we send full range
> content and the driver scales it down to limited range, we can't use
> the out-of-range bits to transfer information. If we send limited
> range content and the driver scales it up we lose information.
>
> Either way, this must be documented. My suggestion is to say that the
> kernel always expects full range data as input and is responsible for
> scaling it to limited range data if the sink expects limited range
> data.
Hi Sebastian,
you are proposing the that driver/hardware will do either no range
conversion, or full-to-limited range conversion. Limited-to-full range
conversion would never be supported.
I still wonder if limited-to-full range conversion could be useful with
video content.
> Another problem is that some displays do not behave correctly. It must
> be possible to override the kernel when the user detects such a
> situation. This override then controls if the driver converts the full
> range data coming from the client or not (Default, Force Limited,
> Force Full). It does not try to control what range the sink expects.
> Let's call this the Quantization Range Override property which should
> be implemented by all drivers.
In other words, a CRTC "quantization range conversion" property with
values:
- auto, with the assumption that color pipeline always produces full-range
- identity
- full-to-limited
(- limited-to-full)
If this property was truly independent of the metadata being sent to
the sink, and of the framebuffer format, it would allow us to do four
ways: both full/limited framebuffer on both full/limited sink. It would
allow us to send sub-blacks and super-whites as well.
More precisely, framebuffers would always have *undefined* quantization
range. The configuration of the color pipeline then determines how that
data is manipulated into a video signal.
So I am advocating the same design as with color spaces: do not tell
KMS what your colorspaces are. Instead tell KMS what operations it
needs to do with the pixel data, and what metadata to send to the sink.
> All drivers should make sure their behavior is correct:
>
> * check that drivers choose the correct default quantization range for
> the selected mode
Mode implying a quantization range is awkward, but maybe the kernel
established modes should just have a flag for it. Then userspace would
know. Unless the video mode system is extended to communicate
IT/CE/SD/VIC and whatnot to userspace, making the modes better defined.
Then userspace would know too.
> * whenever explicit control is available, use it and set the
> quantization range to full
> * make sure that the hardware converts from full range to limited
> range whenever the sink expects limited range
> * implement the Quantization Range Override property
>
> I'm volunteering for the documentation, UAPI and maybe even the drm
> core parts if there is willingness to tackle the issue.
Is it a good idea to put even more automation/magic into configuring
the color pipeline and metadata for a sink, making them even more
intertwined?
I would prefer the opposite direction, making thing more explicit and
orthogonal.
Thanks,
pq
> Appendix A: Broadcast RGB property
>
> A few drivers already implement the Broadcast RGB property
