On 14/02/2018 23:45, Mark Thompson wrote:
On 11/02/18 18:14, Mark Thompson wrote:
From: Jun Zhao <[email protected]>
This removes the arbitrary limit on the allowed number of slices and
parameter buffers.
From ffmpeg commit e4a6eb70f471eda36592078e8fa1bad87fc9df73.
Signed-off-by: Mark Thompson <[email protected]>
---
libavcodec/vaapi_encode.c | 42 ++++++++++++++++++++++++++++++------------
libavcodec/vaapi_encode.h | 6 ++----
2 files changed, 32 insertions(+), 16 deletions(-)
diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c
index 47795ba73..8cba847f7 100644
--- a/libavcodec/vaapi_encode.c
+++ b/libavcodec/vaapi_encode.c
...
@@ -313,15 +321,14 @@ static int vaapi_encode_issue(AVCodecContext *avctx,
}
}
- av_assert0(pic->nb_slices <= MAX_PICTURE_SLICES);
+ pic->slices = av_mallocz_array(pic->nb_slices, sizeof(*pic->slices));
+ if (!pic->slices) {
+ err = AVERROR(ENOMEM);
+ goto fail;
+ }
This fails for non-slice codecs (VP8, VP9) because av_mallocz_array() returns NULL when
asked for zero array elements. (ffmpeg has a hack marked "OS X on SDK 10.6 has a
broken posix_memalign implementation" which makes this case call malloc(1) and
return a real pointer instead.)
Changed locally to only do the allocation if nb_slices > 0.
Ok.
_______________________________________________
libav-devel mailing list
[email protected]
https://lists.libav.org/mailman/listinfo/libav-devel
