PR #21173 opened by cgutman URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/21173 Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/21173.patch
I ran into this double-free when `ff_vk_alloc_mem()` failed in `ff_vk_video_common_init()` because the GPU was out of memory. In this case, `ff_vk_video_common_init()` calls `ff_vk_video_common_uninit()` in the `fail:` path which leaves dangling object handles in `FFVkVideoCommon`. Those get freed again when the destructor of `FFVulkanDecodeShared` calls `ff_vk_video_common_uninit()` a second time. I also included a simple leak fix that I found while investigating. This should be cherry-picked into `release/8.0` and `release/7.1`, but I can also create new PRs for that if necessary. >From d72a59dbd3df829e06b5229cab93e3989cdba909 Mon Sep 17 00:00:00 2001 From: Cameron Gutman <[email protected]> Date: Thu, 11 Dec 2025 17:39:16 -0600 Subject: [PATCH 1/2] lavc/vulkan_video: fix double-free if ff_vk_decode_init() fails ff_vk_video_common_init() calls ff_vk_video_common_uninit() on failure which leaves dangling object handles. Those get freed again when the destructor of FFVulkanDecodeShared calls ff_vk_video_common_uninit() again. Signed-off-by: Cameron Gutman <[email protected]> --- libavcodec/vulkan_video.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/libavcodec/vulkan_video.c b/libavcodec/vulkan_video.c index 819940460f..66fc493ff1 100644 --- a/libavcodec/vulkan_video.c +++ b/libavcodec/vulkan_video.c @@ -349,17 +349,21 @@ av_cold void ff_vk_video_common_uninit(FFVulkanContext *s, av_freep(&common->mem); - if (common->layered_view) + if (common->layered_view) { vk->DestroyImageView(s->hwctx->act_dev, common->layered_view, s->hwctx->alloc); + common->layered_view = VK_NULL_HANDLE; + } av_frame_free(&common->layered_frame); av_buffer_unref(&common->dpb_hwfc_ref); - if (common->yuv_sampler) + if (common->yuv_sampler) { vk->DestroySamplerYcbcrConversion(s->hwctx->act_dev, common->yuv_sampler, s->hwctx->alloc); + common->yuv_sampler = VK_NULL_HANDLE; + } } av_cold int ff_vk_video_common_init(AVCodecContext *avctx, FFVulkanContext *s, -- 2.49.1 >From cb01782b90bef5af569e3ff9940b9803f7033a8c Mon Sep 17 00:00:00 2001 From: Cameron Gutman <[email protected]> Date: Thu, 11 Dec 2025 21:52:37 -0600 Subject: [PATCH 2/2] lavc/vulkan_video: fix leak on CreateVideoSessionKHR failure Signed-off-by: Cameron Gutman <[email protected]> --- libavcodec/vulkan_video.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/libavcodec/vulkan_video.c b/libavcodec/vulkan_video.c index 66fc493ff1..d73479d14d 100644 --- a/libavcodec/vulkan_video.c +++ b/libavcodec/vulkan_video.c @@ -398,8 +398,10 @@ av_cold int ff_vk_video_common_init(AVCodecContext *avctx, FFVulkanContext *s, /* Create session */ ret = vk->CreateVideoSessionKHR(s->hwctx->act_dev, session_create, s->hwctx->alloc, &common->session); - if (ret != VK_SUCCESS) - return AVERROR_EXTERNAL; + if (ret != VK_SUCCESS) { + err = AVERROR_EXTERNAL; + goto fail; + } /* Get memory requirements */ ret = vk->GetVideoSessionMemoryRequirementsKHR(s->hwctx->act_dev, -- 2.49.1 _______________________________________________ ffmpeg-devel mailing list -- [email protected] To unsubscribe send an email to [email protected]
