On 4/25/19 11:36 AM, Boris Brezillon wrote:
> On Thu, 25 Apr 2019 10:21:16 +0200
> Hans Verkuil <hverk...@xs4all.nl> wrote:
>
>> On 4/25/19 10:06 AM, Boris Brezillon wrote:
>>> On Thu, 25 Apr 2019 09:12:21 +0200
>>> Boris Brezillon <boris.brezil...@collabora.com> wrote:
>>>
>>>> This series introduces the decoding infrastructure that will be
>>>> used to add support for other codecs such as VP8, VP9 and H.264.
>>>>
>>>> As explained in the cover letter for the v1 patchset,
>>>> the driver is now exposing two video device nodes.
>>>> The VPU encoder is exposed on /dev/video0, and the VPU decoder
>>>> is exposed on /dev/video1. Both devices are tied to the same
>>>> memory-to-memory queue, and same media device /dev/media0.
>>>>
>>>> Therefore there are two media graphs:
>>>>
>>>> ┌────────────────────────────────┐
>>>> │ rockchip,rk3399-vpu-enc-source │
>>>> │ /dev/video0 │
>>>> └────────────────────────────────┘
>>>> ┃
>>>> ┃
>>>> ▼
>>>> ┌────────────────────────────────┐
>>>> │ rockchip,rk3399-vpu-enc-proc │
>>>> └────────────────────────────────┘
>>>> ┃
>>>> ┃
>>>> ▼
>>>> ┌────────────────────────────────┐
>>>> │ rockchip,rk3399-vpu-enc-sink │
>>>> │ /dev/video0 │
>>>> └────────────────────────────────┘
>>>>
>>>> ┌────────────────────────────────┐
>>>> │ rockchip,rk3399-vpu-dec-source │
>>>> │ /dev/video1 │
>>>> └────────────────────────────────┘
>>>> ┃
>>>> ┃
>>>> ▼
>>>> ┌────────────────────────────────┐
>>>> │ rockchip,rk3399-vpu-dec-proc │
>>>> └────────────────────────────────┘
>>>> ┃
>>>> ┃
>>>> ▼
>>>> ┌────────────────────────────────┐
>>>> │ rockchip,rk3399-vpu-dec-sink │
>>>> │ /dev/video1 │
>>>> └────────────────────────────────┘
>>>>
>>>> Of course, this work has been possible thanks to Jonas Karlman, who did
>>>> the initial MPEG-2 decoding work and also got mpv+ffmpeg working using
>>>> the Request API.
>>>
>>> Forgot to provide the v4l2-compliance results, so here it is:
>>>
>>> "
>>> v4l2-compliance SHA: 95f39aae48c6e8e5cc81e6ae6293d999f70a909b, 64 bits
>>>
>>> Compliance test for rockchip-vpu device /dev/media0:
>>>
>>> Media Driver Info:
>>> Driver name : rockchip-vpu
>>> Model : rockchip-vpu
>>> Serial :
>>> Bus info : platform: rockchip-vpu
>>> Media version : 5.1.0
>>> Hardware revision: 0x00000000 (0)
>>> Driver version : 5.1.0
>>>
>>> Required ioctls:
>>> test MEDIA_IOC_DEVICE_INFO: OK
>>>
>>> Allow for multiple opens:
>>> test second /dev/media0 open: OK
>>> test MEDIA_IOC_DEVICE_INFO: OK
>>> test for unlimited opens: OK
>>>
>>> Media Controller ioctls:
>>> test MEDIA_IOC_G_TOPOLOGY: OK
>>> Entities: 6 Interfaces: 2 Pads: 8 Links: 8
>>> test MEDIA_IOC_ENUM_ENTITIES/LINKS: OK
>>> test MEDIA_IOC_SETUP_LINK: OK
>>>
>>> Total for rockchip-vpu device /dev/media0: 7, Succeeded: 7, Failed: 0,
>>> Warnings: 0
>>> --------------------------------------------------------------------------------
>>> Compliance test for rockchip-vpu device /dev/video0:
>>>
>>> Driver Info:
>>> Driver name : rockchip-vpu
>>> Card type : rockchip,rk3399-vpu-enc
>>> Bus info : platform: rockchip-vpu
>>> Driver version : 5.1.0
>>> Capabilities : 0x84204000
>>> Video Memory-to-Memory Multiplanar
>>> Streaming
>>> Extended Pix Format
>>> Device Capabilities
>>> Device Caps : 0x04204000
>>> Video Memory-to-Memory Multiplanar
>>> Streaming
>>> Extended Pix Format
>>> Detected JPEG Encoder
>>> Media Driver Info:
>>> Driver name : rockchip-vpu
>>> Model : rockchip-vpu
>>> Serial :
>>> Bus info : platform: rockchip-vpu
>>> Media version : 5.1.0
>>> Hardware revision: 0x00000000 (0)
>>> Driver version : 5.1.0
>>> Interface Info:
>>> ID : 0x0300000c
>>> Type : V4L Video
>>> Entity Info:
>>> ID : 0x00000001 (1)
>>> Name : rockchip,rk3399-vpu-enc-source
>>> Function : V4L2 I/O
>>> Pad 0x01000002 : 0: Source
>>> Link 0x02000008: to remote pad 0x1000005 of entity
>>> 'rockchip,rk3399-vpu-enc-proc': Data, Enabled, Immutable
>>>
>>> Required ioctls:
>>> test MC information (see 'Media Driver Info' above): OK
>>> test VIDIOC_QUERYCAP: OK
>>>
>>> Allow for multiple opens:
>>> test second /dev/video0 open: OK
>>> test VIDIOC_QUERYCAP: OK
>>> test VIDIOC_G/S_PRIORITY: OK
>>> test for unlimited opens: OK
>>>
>>> Debug ioctls:
>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>
>>> Input ioctls:
>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>
>>> Output ioctls:
>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>
>>> Input/Output configuration ioctls:
>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>> test VIDIOC_G/S_EDID: OK (Not Supported)
>>>
>>> Control ioctls:
>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
>>> test VIDIOC_QUERYCTRL: OK
>>> test VIDIOC_G/S_CTRL: OK
>>> test VIDIOC_G/S/TRY_EXT_CTRLS: OK
>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>> Standard Controls: 2 Private Controls: 0
>>>
>>> Format ioctls:
>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>> test VIDIOC_G/S_PARM: OK (Not Supported)
>>> test VIDIOC_G_FBUF: OK (Not Supported)
>>> test VIDIOC_G_FMT: OK
>>> test VIDIOC_TRY_FMT: OK
>>> test VIDIOC_S_FMT: OK
>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>> test Cropping: OK (Not Supported)
>>> test Composing: OK (Not Supported)
>>> test Scaling: OK
>>>
>>> Codec ioctls:
>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>
>>> Buffer ioctls:
>>> fail: v4l2-test-buffers.cpp(709): q.create_bufs(node, 1, &fmt)
>>> != EINVAL
>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
>>> test VIDIOC_EXPBUF: OK
>>> testRequests:1760 ret 0
>>> fail: v4l2-test-buffers.cpp(1820): !supports_requests
>>> test Requests: FAIL
>>>
>>> Total for rockchip-vpu device /dev/video0: 45, Succeeded: 43, Failed: 2,
>>> Warnings: 0
>>> --------------------------------------------------------------------------------
>>> Compliance test for rockchip-vpu device /dev/video1:
>>>
>>> Driver Info:
>>> Driver name : rockchip-vpu
>>> Card type : rockchip,rk3399-vpu-dec
>>> Bus info : platform: rockchip-vpu
>>> Driver version : 5.1.0
>>> Capabilities : 0x84204000
>>> Video Memory-to-Memory Multiplanar
>>> Streaming
>>> Extended Pix Format
>>> Device Capabilities
>>> Device Caps : 0x04204000
>>> Video Memory-to-Memory Multiplanar
>>> Streaming
>>> Extended Pix Format
>>> Detected Stateless Decoder
>>> Media Driver Info:
>>> Driver name : rockchip-vpu
>>> Model : rockchip-vpu
>>> Serial :
>>> Bus info : platform: rockchip-vpu
>>> Media version : 5.1.0
>>> Hardware revision: 0x00000000 (0)
>>> Driver version : 5.1.0
>>> Interface Info:
>>> ID : 0x0300001a
>>> Type : V4L Video
>>> Entity Info:
>>> ID : 0x0000000f (15)
>>> Name : rockchip,rk3399-vpu-dec-source
>>> Function : V4L2 I/O
>>> Pad 0x01000010 : 0: Source
>>> Link 0x02000016: to remote pad 0x1000013 of entity
>>> 'rockchip,rk3399-vpu-dec-proc': Data, Enabled, Immutable
>>>
>>> Required ioctls:
>>> test MC information (see 'Media Driver Info' above): OK
>>> test VIDIOC_QUERYCAP: OK
>>>
>>> Allow for multiple opens:
>>> test second /dev/video1 open: OK
>>> test VIDIOC_QUERYCAP: OK
>>> test VIDIOC_G/S_PRIORITY: OK
>>> test for unlimited opens: OK
>>>
>>> Debug ioctls:
>>> test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
>>> test VIDIOC_LOG_STATUS: OK (Not Supported)
>>>
>>> Input ioctls:
>>> test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
>>> test VIDIOC_ENUMAUDIO: OK (Not Supported)
>>> test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
>>> test VIDIOC_G/S_AUDIO: OK (Not Supported)
>>> Inputs: 0 Audio Inputs: 0 Tuners: 0
>>>
>>> Output ioctls:
>>> test VIDIOC_G/S_MODULATOR: OK (Not Supported)
>>> test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
>>> test VIDIOC_ENUMAUDOUT: OK (Not Supported)
>>> test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
>>> test VIDIOC_G/S_AUDOUT: OK (Not Supported)
>>> Outputs: 0 Audio Outputs: 0 Modulators: 0
>>>
>>> Input/Output configuration ioctls:
>>> test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
>>> test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
>>> test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
>>> test VIDIOC_G/S_EDID: OK (Not Supported)
>>>
>>> Control ioctls:
>>> test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
>>> test VIDIOC_QUERYCTRL: OK
>>> test VIDIOC_G/S_CTRL: OK
>>> fail: v4l2-test-controls.cpp(673): try_ext_ctrls returned an
>>> error (22)
>>> test VIDIOC_G/S/TRY_EXT_CTRLS: FAIL
>>> test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
>>> test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
>>> Standard Controls: 2 Private Controls: 0
>>>
>>> Format ioctls:
>>> test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
>>> test VIDIOC_G/S_PARM: OK (Not Supported)
>>> test VIDIOC_G_FBUF: OK (Not Supported)
>>> test VIDIOC_G_FMT: OK
>>> test VIDIOC_TRY_FMT: OK
>>> fail: v4l2-test-formats.cpp(806): fmt_cap.g_colorspace() != col
>>> test VIDIOC_S_FMT: FAIL
>>> test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
>>> test Cropping: OK (Not Supported)
>>> test Composing: OK (Not Supported)
>>> test Scaling: OK
>>>
>>> Codec ioctls:
>>> test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
>>> test VIDIOC_G_ENC_INDEX: OK (Not Supported)
>>> test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)
>>>
>>> Buffer ioctls:
>>> fail: v4l2-test-buffers.cpp(709): q.create_bufs(node, 1, &fmt)
>>> != EINVAL
>>> test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
>>> test VIDIOC_EXPBUF: OK
>>> test Requests: OK (Not Supported)
>>>
>>> Total for rockchip-vpu device /dev/video1: 45, Succeeded: 42, Failed: 3,
>>> Warnings: 0
>>>
>>> Grand Total for rockchip-vpu device /dev/media0: 97, Succeeded: 92, Failed:
>>> 5, Warnings: 0
>>>
>>> "
>>>
>>> 2 of those failures are expected:
>>>
>>> * "fail: v4l2-test-formats.cpp(806): fmt_cap.g_colorspace() != col": is
>>> caused
>>> by the reset done by the driver on the non-coded end when the coded end of
>>> the pipeline is modified (which follows the behavior described in
>>> stateless
>>> decoder spec)
>>
>> So is this a driver FIXME, or an issue in the compliance test itself?
>
> Looks like the spec mandates the opposite, so it's a driver bug. I'll
> fix that. The main issue I see is that the RK encoder code has this
> comment:
>
> /*
> * Current raw format might have become invalid with newly
> * selected codec, so reset it to default just to be safe and
> * keep internal driver state sane. User is mandated to set
> * the raw format again after we return, so we don't need
> * anything smarter.
> */
>
> Does that mean we have a different behavior between encoders and
> decoders?
Yes. This is true for the stateful encoder as well: setting the capture
format can change the output format.
For a decoder it is the other way around: setting the output format
can change the capture format.
Note that we don't have a stateless encoder spec yet, but it makes sense
that it behaves in the same way in this respect as the stateful encoder.
>
>>
>>>
>>> * "fail: v4l2-test-controls.cpp(673): try_ext_ctrls returned an error (22)":
>>> default values for mpeg2 ctrls are not suitable, so doing
>>> get_ext_ctrl()+try_ext_ctrl() does not work.
>>
>> Hmm, this might require some changes to this test. Where does it fail in the
>> driver? Is that the std_validate() check in v4l2-ctrls.c?
>
> Don't remember where it complained exactly, but it was in some generic
> function, yes.
>
>>
>> If so, then I think it would be wise to add an initializer for
>> V4L2_CTRL_TYPE_MPEG2_SLICE_PARAMS
>> to std_init() so that it creates a proper default.
>
> Okay.
>
>>
>> I also noticed that support for these new compound types is missing in
>> the std_log() function in v4l2-ctrls.c. It should probably log the
>> type. Check with v4l2-ctl --log-status to see how it looks in the
>> kernel log.
>
> Will look at it too.
>
>> Obviously, this can be done in a separate patch since it's
>> independent of the rockchip driver.
>
> Sure.
>
Regards,
Hans
