RE: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline

2023-09-07 Thread Shankar, Uma


> -Original Message-
> From: Pekka Paalanen 
> Sent: Tuesday, September 5, 2023 5:03 PM
> To: Shankar, Uma 
> Cc: intel-...@lists.freedesktop.org; Borah, Chaitanya Kumar
> ; dri-de...@lists.freedesktop.org; wayland-
> de...@lists.freedesktop.org; Harry Wentland ;
> Sebastian Wick ; ville.syrj...@linux.intel.com;
> Jonas Adahl 
> Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane
> Color Pipeline
> 
> On Mon, 4 Sep 2023 13:44:49 +
> "Shankar, Uma"  wrote:
> 
> > > -Original Message-
> > > From: dri-devel  On Behalf
> > > Of Pekka Paalanen
> > > Sent: Wednesday, August 30, 2023 5:59 PM
> > > To: Shankar, Uma 
> > > Cc: intel-...@lists.freedesktop.org; Borah, Chaitanya Kumar
> > > ; dri-de...@lists.freedesktop.org;
> > > wayland- de...@lists.freedesktop.org
> > > Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed
> > > Plane Color Pipeline
> > >
> > > On Wed, 30 Aug 2023 08:59:36 +
> > > "Shankar, Uma"  wrote:
> > >
> > > > > -Original Message-
> > > > > From: Harry Wentland 
> > > > > Sent: Wednesday, August 30, 2023 1:10 AM
> > > > > To: Shankar, Uma ;
> > > > > intel-...@lists.freedesktop.org; dri-
> > > > > de...@lists.freedesktop.org
> > > > > Cc: Borah, Chaitanya Kumar ;
> > > > > wayland- de...@lists.freedesktop.org
> > > > > Subject: Re: [RFC 01/33] drm/doc/rfc: Add RFC document for
> > > > > proposed Plane Color Pipeline
> > > > >
> > > > >
> > > > >
> > > > > On 2023-08-29 12:03, Uma Shankar wrote:
> > > > > > Add the documentation for the new proposed Plane Color Pipeline.
> > > > > >
> > > > > > Co-developed-by: Chaitanya Kumar Borah
> > > > > > 
> > > > > > Signed-off-by: Chaitanya Kumar Borah
> > > > > > 
> > > > > > Signed-off-by: Uma Shankar 
> > > > > > ---
> > > > > >   .../gpu/rfc/plane_color_pipeline.rst  | 394 
> > > > > > ++
> > > > > >   1 file changed, 394 insertions(+)
> > > > > >   create mode 100644
> > > > > > Documentation/gpu/rfc/plane_color_pipeline.rst
> > > > > >
> > > > > > diff --git a/Documentation/gpu/rfc/plane_color_pipeline.rst
> > > > > > b/Documentation/gpu/rfc/plane_color_pipeline.rst
> > > > > > new file mode 100644
> > > > > > index ..60ce515b6ea7
> > > > > > --- /dev/null
> > > > > > +++ b/Documentation/gpu/rfc/plane_color_pipeline.rst
> > >
> > > ...
> > >
> > > Hi Uma!
> >
> > Thanks Pekka for the feedback and useful inputs.
> 
> Hi Uma,
> 
> sorry to say, but the overall feeling I get from this proposal is that it is 
> just the
> current color related KMS properties wrapped in a pipeline blob. That is not 
> the
> re-design I believe we are looking for, and the feeling is based on several 
> details
> that are just copied from the current property design. Also the "private" 
> stuff has
> to go.

Hi Pekka,
Ok, got the concerns in general.  We will try to evaluate in deeper detail the
property based design and come back if there are some issues or inputs.
 
At Intel we don't need private as of now, but we thought of having an option to
enable any custom hardware or vendor. But we can drop the same for now if
community doesn't feel the need for it.

> All the varying LUT entries, varying LUT precision, 1D/3D LUTs, varying LUT 
> tap
> distribution, and parametrized curves are good development, but right now we
> are looking at things one step higher level: the overall color pipeline 
> design and
> how to represent any operation. Most of this series is considering details 
> below
> the current attention level, hence I'm paying attention only to the first few
> patches.

We will need to precisely describe the hardware in userspace. Number of luts, 
precision,
segments etc.., we can't just pass EOTF's as enum from userspace and let driver 
put
hardcoded values to LUT. This will be nothing but an extension of descriptive 
behaviour.
This will be needed as there are multiple colorspaces possible and even LUTS 
can be
used to perform tone mapping. So, we need userspace to be able to program luts 
directly.

This is something we must expose to userspace. We will check if this can be 
fitted in
property based approach.

Thanks for all the feedback and comments on the design. We will work on it and 
get back.

Regards,
Uma Shankar

> More below.
> 
> > > > > > +This color pipeline is then packaged within a blob for the
> > > > > > +user space to retrieve it. Details can be found in the next
> > > > > > +section
> > > > > > +
> > > > >
> > > > > Not sure I like blobs that contain other blob ids.
> > > >
> > > > It provides flexibility and helps with just one interface to
> > > > userspace. Its easy to handle and manage once we get the hang of it 😊.
> > > >
> > > > We can clearly define the steps of parsing and data structures to
> > > > be used while interpreting and parsing the blobs.
> > >
> > > Don't forget extendability. Possibly every single struct will need
> > > some kind of versioning, and then it's not simple to parse anymore.
> > > Add to that new/old k

Re: [RFC 01/33] drm/doc/rfc: Add RFC document for proposed Plane Color Pipeline

2023-09-07 Thread Christopher Braga




On 8/29/2023 12:03 PM, Uma Shankar wrote:

Add the documentation for the new proposed Plane Color Pipeline.

Co-developed-by: Chaitanya Kumar Borah 
Signed-off-by: Chaitanya Kumar Borah 
Signed-off-by: Uma Shankar 
---
  .../gpu/rfc/plane_color_pipeline.rst  | 394 ++
  1 file changed, 394 insertions(+)
  create mode 100644 Documentation/gpu/rfc/plane_color_pipeline.rst

diff --git a/Documentation/gpu/rfc/plane_color_pipeline.rst 
b/Documentation/gpu/rfc/plane_color_pipeline.rst
new file mode 100644
index ..60ce515b6ea7
--- /dev/null
+++ b/Documentation/gpu/rfc/plane_color_pipeline.rst
@@ -0,0 +1,394 @@
+===
+ Plane Color Pipeline: A UAPI proposal
+===
+
+To build the proposal on, lets take the premise of a color pipeline as shown
+below.
+

Hi Uma,
Thanks for posting this. A few comments, with some echoing the 
sentiments of other commenters on the patch set.



+ +---+
+ |RAM|
+ |  +--++-++-+   |
+ |  | FB 1 ||  FB 2   || FB N|   |
+ |  +--++-++-+   |
+ +---+
+   |  Plane Color Hardware Block |
+ ++
+ | +---v-+   +---v---+   +---v--+ |
+ | | Plane A |   | Plane B   |   | Plane N  | |
+ | | Pre-CSC |   | Pre-CSC   |   | Pre-CSC  | |
+ | +---+-+   +---+---+   +---+--+ |
+ | | |   ||
+ | +---v-+   +---v---+   +---v--+ |
+ | |Plane A  |   | Plane B   |   | Plane N  | |
+ | |CSC/CTM  |   | CSC/CTM   |   | CSC/CTM  | |
+ | +---+-+   ++--+   ++-+ |
+ | |  |   |   |
+ | +---v-+   +v--+   +v-+ |
+ | | Plane A |   | Plane B   |   | Plane N  | |
+ | |Post-CSC |   | Post-CSC  |   | Post-CSC | |
+ | +---+-+   ++--+   ++-+ |
+ | |  |   |   |
+ ++
++--v--v---v---|
+||   ||
+||   Pipe Blender||
++++
+|||
+|+---v--+ |
+||  Pipe Pre-CSC| |
+||  | |
+|+---+--+ |
+||Pipe Color  |
+|+---v--+ Hardware|
+||  Pipe CSC/CTM| |
+||  | |
+|+---+--+ |
+|||
+|+---v--+ |
+||  Pipe Post-CSC   | |
+||  | |
+|+---+--+ |
+|||
++-+
+ |
+ v
+Pipe Output
+
+Each plane consists of the following color blocks
+ * Pre-CSC : This block can used to linearize the input frame buffer data.
+ The linear data then can be further acted on by the following
+ color hardware blocks in the display hardware pipeline
+
+ * CSC/CTM: Used to program color transformation matrix, this block is used
+to perform color space conversions like BT2020 to BT709 or BT601
+etc. This block acts on the linearized data coming from the
+Pre-CSC HW block.
+
+ * Post-CSC: This HW block can be used to non-linearize frame buffer data to
+ match the sink. Another use case of it could be to perform Tone
+ mapping for HDR use-cases.
+
+Data from multiple planes will then be fed to pipe/crtc where it will get 
blended.
+There is a similar set of HW blocks available at pipe/crtc level which acts on
+this blended data.
+
+Below is a sample usecase fo video playback with sub-titles and playback
+controls
+
+β”Œβ”β”Œβ”€β” β”Œβ”€β”β”Œβ”€β”
+β”‚FB1 β”‚β”‚PRE-CSC  β”‚ β”‚ CTM Matrix  β”‚β”‚ POST-CSCβ”‚
+β”‚β”œβ”€β”€β”€β–Ίβ”‚Linearizeβ”œβ–Ίβ”‚ BT709 toβ”œβ”€β”€β”€β–Ίβ”‚ SDR to HDR  β”‚
+β”‚BT709 SDR   β”‚β”‚ β”‚ β”‚ BT2020  β”‚β”‚ Tone Mappingβ”œβ”€β”
+β””β”˜β””β”€β”˜ β””β”€β”˜β””β”€β”˜ β”‚
+(subtitles)  β”‚
+ β”‚
+β”Œβ”β”Œβ”€β” β”Œβ”€β”β”Œβ”€β” β”‚
+β”‚FB2 β”‚β”‚PRE-CSC  β”‚ β”‚ CTM Matrix  β”‚β”‚ POST-CSCβ”‚ β”‚
+β”‚β”œβ”€β”€β”€β–Ίβ”‚Linearizeβ”œ