Problem solved; it turns out that VAAPI has a postprocessing module which can do (among other things) format conversions. The FFMPEG interface to this is the scale_vaapi filter in libavfilter. As integrating the scale_vaapi filter would have been tricky for my specific use case, I instead used the VAAPI interface directly to set up a pipeline without any filters that copies the video output surface onto a provided target DMABUF.
In case anyone is interested in the code, see [0]. [0] https://gitlab.freedesktop.org/mstoeckl/waypipe/blob/77c92bb1a40b4ac779414d9ec2ace9841b283993/video.c#L207-363 On Tue, 9 Jul 2019 19:14:34 -0400 "M. Stoeckl" <[email protected]> wrote: > Hello, > > I have the following problem: there is a program outside of my control > which uses and produces DRM PRIME fds only with a specific format (for > example, DRM_FORMAT_XRGB8888/AV_PIX_FMT_BGR0), and I'd like to decode > an input video format (such as H264, or VP9) in YUV colorspace and > have its frames be written to the DRM PRIME fds, without copying the > entire frame back to the CPU to do a color format conversion. > > With the h264_vaapi encoder, I can use av_hwframe_map with > AV_PIX_FMT_DRM_PRIME frames (with format DRM_FORMAT_XRGB8888) to > produce AV_PIX_FMT_VAAPI frames with the software format > AV_PIX_FMT_NV12, that the encoder then uses successfully. [Unless I > am mistaken, this calls vaCreateSurfaces using the DRM frame data, > and the VAAPI driver does the format conversion work.] > > However, I'm not sure how best to decode with VAAPI; a similar > approach trying av_hwframe_map before the decoder started produced > AVERROR(ENOSYS). When I configure AVVAAPIFramesContext to have the > frames be produced from an existing DRM PRIME file descriptor, the > decoder runs, but the original file descriptor contents aren't > updated. Alternatively, converting the frames produced by the > decoder, with av_hwframe_map, to DRM PRIME frames gives me data in > NV12 format, which isn't what I want. > > Has anyone managed to get something similar working, or even any > GPU-only decoding pipeline that can decode YUV video into RGB images, > handling any conversions on-GPU? > > Any additional advice (or pointers to relevant example code not > already in FFMPEG) would be appreciated. _______________________________________________ Libav-user mailing list [email protected] https://ffmpeg.org/mailman/listinfo/libav-user To unsubscribe, visit link above, or email [email protected] with subject "unsubscribe".
