PR #20994 opened by Steven Xiao (younengxiao)
URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/20994
Patch URL: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/20994.patch
# FFmpeg Patch Submission: D3D12 Video Encoder Intra Refresh Support
## Patch Title
avcodec/d3d12va_encode: Add intra refresh support for H.264 and HEVC encoders
## 1. Description
This patch adds intra refresh support to FFmpeg's D3D12 hardware video encoders
for H.264 and HEVC codecs. Intra refresh is a technique that gradually
refreshes the video by encoding rows or regions as intra macroblocks/CTUs
spread over multiple frames, rather than using periodic I-frames. This provides
better error resilience for video streaming while maintaining more consistent
bitrate.
The D3D12 Video Encode API supports row-based intra refresh mode, and this
patch exposes that capability through FFmpeg's command-line interface for both
h264_d3d12va and hevc_d3d12va encoders.
## 2. How to Test
### Command Line Syntax
**H.264 Encoder:**
```bash
ffmpeg -init_hw_device d3d12va -hwaccel d3d12va -hwaccel_output_format d3d12 \
-i input.mp4 \
-c:v h264_d3d12va \
-intra_refresh_mode ROW_BASED \
-intra_refresh_duration 30 \
-g 60 \
output.h264
```
**HEVC Encoder:**
```bash
ffmpeg -init_hw_device d3d12va -hwaccel d3d12va -hwaccel_output_format d3d12 \
-i input.mp4 \
-c:v hevc_d3d12va \
-intra_refresh_mode ROW_BASED \
-intra_refresh_duration 30 \
-g 60 \
output.hevc
```
### Parameters
- `-intra_refresh_mode`: Set to `ROW_BASED` to enable row-based intra refresh,
or `NONE` to disable
- `-intra_refresh_duration`: Number of frames over which to spread the intra
refresh (default: 0 = use GOP size)
- `-g`: GOP size (should typically be larger than intra refresh duration)
### Verification
To verify intra refresh is working:
1. Check encoder logs for: `"Intra refresh: mode=1, duration=X frames"`
2. Analyze the output bitstream to confirm gradual intra coding pattern
## 3. Test Summary for Different Hardware
### AMD Radeon RX 9070 XT ✅ FULLY WORKING
**Test Configuration:**
- Input: 1920x1080 @ 60fps H.264 video
- Settings: ROW_BASED mode, 30 frame duration, GOP size 60
- Encoded: 100 frames
**H.264 Results:**
- Status: ✅ PASSED
- Intra refresh confirmed in bitstream
**HEVC Results:**
- Status: ✅ PASSED
- Intra refresh confirmed in bitstream
**Conclusion:** Full intra refresh support on AMD hardware. The feature works
reliably for both H.264 and HEVC encoding.
### NVIDIA GPUs ⚠️ DRIVER LIMITATION
**Test Configuration:** Same as AMD
**H.264 and HEVC Results:**
- Driver reports `IsSupported=true` for intra refresh mode
- However, `MaxIntraRefreshFrameDuration=0` (revealed via D3D12 debug layer)
- Encoding fails with: `D3D12 ERROR: ENCODE_FRAME_UNSUPPORTED_PARAMETERS -
Intra refresh duration specified (30) exceeds the maximum supported number (0)`
**Root Cause:**
- D3D12 API limitation: `CheckFeatureSupport` only validates MODE support, not
duration constraints
- NVIDIA drivers currently don't support non-zero intra refresh durations
- This is a driver limitation, not an FFmpeg implementation issue
**Workaround:** Users on NVIDIA GPUs should disable intra refresh
(`-intra_refresh_mode NONE`) until driver support improves.
### INTEL GPUs ⚠️ DRIVER LIMITATION
**Test Configuration:** Same as AMD
**H.264 Results:**
- Driver reports `IsSupported=true` for intra refresh mode
- However, `MaxIntraRefreshFrameDuration=1` (revealed via D3D12 debug layer)
- Encoding fails with: `D3D12 ERROR: ENCODE_FRAME_UNSUPPORTED_PARAMETERS -
Intra refresh duration specified (30) exceeds the maximum supported number (1)`
**Root Cause:**
- D3D12 API limitation: `CheckFeatureSupport` only validates MODE support, not
duration constraints
- INTEL drivers currently ONLY support value "1" intra refresh durations
- When set Intra refresh duration to 1, in fact all frames are I-frame in
bitstream
- This is a driver limitation, not an FFmpeg implementation issue
**HEVC Results:**
- Driver doesn't support this feature. Report
`D3D12_VIDEO_ENCODER_VALIDATION_FLAG_INTRA_REFRESH_MODE_NOT_SUPPORTED`
**Workaround:** Users on NVIDIA GPUs should disable intra refresh
(`-intra_refresh_mode NONE`) until driver support improves.
### Summary Table
| Hardware | H.264 | HEVC | Notes |
|----------|-------|------|-------|
| AMD Radeon RX 9070 XT | ✅ Working | ✅ Working | Fully functional,
production-ready |
| NVIDIA GPUs | ❌ Not supported | ❌ Not supported | Driver limitation: max
duration = 0 |
| Intel GPUs | ❌ Support, but all I-frame | ❌ Not supported | Driver limitation
|
## 4. Known Limitations
1. **D3D12 API Constraint**: The feature check API doesn't expose maximum
duration limits, so validation happens during encoding rather than
initialization
2. **Driver-Dependent**: Support varies by GPU vendor and driver version
4. **Warning Message**: The implementation includes a warning when intra
refresh is enabled, advising users that support may vary
From 0c66b0433f1dcea6cb25bb49d0a5d9d91cf0dcad Mon Sep 17 00:00:00 2001
From: stevxiao <[email protected]>
Date: Fri, 21 Nov 2025 20:39:22 -0500
Subject: [PATCH] D3D12 Video Encoder Intra Refresh Support for H.264 and HEVC
---
libavcodec/d3d12va_encode.c | 75 ++++++++++++++++++++++++++++++--
libavcodec/d3d12va_encode.h | 27 ++++++++++++
libavcodec/d3d12va_encode_h264.c | 3 +-
libavcodec/d3d12va_encode_hevc.c | 3 +-
4 files changed, 103 insertions(+), 5 deletions(-)
diff --git a/libavcodec/d3d12va_encode.c b/libavcodec/d3d12va_encode.c
index aa8a5982be..8a382c3f2f 100644
--- a/libavcodec/d3d12va_encode.c
+++ b/libavcodec/d3d12va_encode.c
@@ -204,10 +204,17 @@ static int d3d12va_encode_issue(AVCodecContext *avctx,
int barriers_ref_index = 0;
D3D12_RESOURCE_BARRIER *barriers_ref = NULL;
+ D3D12_VIDEO_ENCODER_SEQUENCE_CONTROL_FLAGS seq_flags =
D3D12_VIDEO_ENCODER_SEQUENCE_CONTROL_FLAG_NONE;
+
+ // Request intra refresh if enabled
+ if (ctx->intra_refresh.Mode !=
D3D12_VIDEO_ENCODER_INTRA_REFRESH_MODE_NONE) {
+ seq_flags |=
D3D12_VIDEO_ENCODER_SEQUENCE_CONTROL_FLAG_REQUEST_INTRA_REFRESH;
+ }
+
D3D12_VIDEO_ENCODER_ENCODEFRAME_INPUT_ARGUMENTS input_args = {
.SequenceControlDesc = {
- .Flags = D3D12_VIDEO_ENCODER_SEQUENCE_CONTROL_FLAG_NONE,
- .IntraRefreshConfig = { 0 },
+ .Flags = seq_flags,
+ .IntraRefreshConfig = ctx->intra_refresh,
.RateControl = ctx->rc,
.PictureTargetResolution = ctx->resolution,
.SelectedLayoutMode =
D3D12_VIDEO_ENCODER_FRAME_SUBREGION_LAYOUT_MODE_FULL_FRAME,
@@ -352,7 +359,7 @@ static int d3d12va_encode_issue(AVCodecContext *avctx,
}
}
- input_args.PictureControlDesc.IntraRefreshFrameIndex = 0;
+ input_args.PictureControlDesc.IntraRefreshFrameIndex =
ctx->intra_refresh_frame_index;
if (base_pic->is_reference)
input_args.PictureControlDesc.Flags |=
D3D12_VIDEO_ENCODER_PICTURE_CONTROL_FLAG_USED_AS_REFERENCE_PICTURE;
@@ -540,6 +547,12 @@ static int d3d12va_encode_issue(AVCodecContext *avctx,
pic->fence_value = ctx->sync_ctx.fence_value;
+ // Update intra refresh frame index for next frame
+ if (ctx->intra_refresh.Mode !=
D3D12_VIDEO_ENCODER_INTRA_REFRESH_MODE_NONE) {
+ ctx->intra_refresh_frame_index =
+ (ctx->intra_refresh_frame_index + 1) %
ctx->intra_refresh.IntraRefreshDuration;
+ }
+
if (d3d12_refs.ppTexture2Ds)
av_freep(&d3d12_refs.ppTexture2Ds);
@@ -1191,6 +1204,58 @@ static int
d3d12va_encode_init_gop_structure(AVCodecContext *avctx)
return 0;
}
+static int d3d12va_encode_init_intra_refresh(AVCodecContext *avctx)
+{
+ FFHWBaseEncodeContext *base_ctx = avctx->priv_data;
+ D3D12VAEncodeContext *ctx = avctx->priv_data;
+ HRESULT hr;
+ D3D12_FEATURE_DATA_VIDEO_ENCODER_INTRA_REFRESH_MODE intra_refresh_support
= {
+ .NodeIndex = 0,
+ .Codec = ctx->codec->d3d12_codec,
+ .Profile = ctx->profile->d3d12_profile,
+ .Level = ctx->level,
+ .IntraRefreshMode = ctx->intra_refresh.Mode,
+ };
+
+ if (ctx->intra_refresh.Mode == D3D12_VIDEO_ENCODER_INTRA_REFRESH_MODE_NONE)
+ return 0;
+
+ // Check driver support for the intra refresh mode
+ hr = ID3D12VideoDevice3_CheckFeatureSupport(ctx->video_device3,
+ D3D12_FEATURE_VIDEO_ENCODER_INTRA_REFRESH_MODE,
+ &intra_refresh_support, sizeof(intra_refresh_support));
+
+ if (FAILED(hr) || !intra_refresh_support.IsSupported) {
+ av_log(avctx, AV_LOG_WARNING,
+ "Requested intra refresh mode not supported by driver,
disabling.\n");
+ ctx->intra_refresh.Mode = D3D12_VIDEO_ENCODER_INTRA_REFRESH_MODE_NONE;
+ ctx->intra_refresh.IntraRefreshDuration = 0;
+ return 0;
+ }
+
+ // Set duration: use GOP size if not specified
+ if (ctx->intra_refresh.IntraRefreshDuration == 0) {
+ ctx->intra_refresh.IntraRefreshDuration = base_ctx->gop_size;
+ av_log(avctx, AV_LOG_VERBOSE, "Intra refresh duration set to GOP size:
%d\n",
+ ctx->intra_refresh.IntraRefreshDuration);
+ }
+
+ // Initialize frame index
+ ctx->intra_refresh_frame_index = 0;
+
+ av_log(avctx, AV_LOG_VERBOSE, "Intra refresh: mode=%d, duration=%d
frames\n",
+ ctx->intra_refresh.Mode, ctx->intra_refresh.IntraRefreshDuration);
+
+ // Some drivers report IsSupported=true but have
MaxIntraRefreshFrameDuration=0
+ // We need to check this AFTER initializing other components that query
encoder caps
+ // For now, just warn and disable if we suspect issues
+ av_log(avctx, AV_LOG_WARNING,
+ "Intra refresh requested but may not be fully supported by this
GPU. "
+ "If encoding fails, disable intra refresh.\n");
+
+ return 0;
+}
+
static int d3d12va_create_encoder(AVCodecContext *avctx)
{
FFHWBaseEncodeContext *base_ctx = avctx->priv_data;
@@ -1569,6 +1634,10 @@ int ff_d3d12va_encode_init(AVCodecContext *avctx)
goto fail;
}
+ err = d3d12va_encode_init_intra_refresh(avctx);
+ if (err < 0)
+ goto fail;
+
err = d3d12va_encode_create_recon_frames(avctx);
if (err < 0)
goto fail;
diff --git a/libavcodec/d3d12va_encode.h b/libavcodec/d3d12va_encode.h
index 5bd1eedb7f..04d0ddee6d 100644
--- a/libavcodec/d3d12va_encode.h
+++ b/libavcodec/d3d12va_encode.h
@@ -264,6 +264,16 @@ typedef struct D3D12VAEncodeContext {
D3D12_VIDEO_ENCODER_SEQUENCE_GOP_STRUCTURE gop;
D3D12_VIDEO_ENCODER_LEVEL_SETTING level;
+
+ /**
+ * Intra refresh configuration
+ */
+ D3D12_VIDEO_ENCODER_INTRA_REFRESH intra_refresh;
+
+ /**
+ * Current frame index within intra refresh cycle
+ */
+ UINT intra_refresh_frame_index;
} D3D12VAEncodeContext;
typedef struct D3D12VAEncodeType {
@@ -355,4 +365,21 @@ int ff_d3d12va_encode_close(AVCodecContext *avctx);
D3D12VA_ENCODE_RC_MODE(VBR, "Variable-bitrate"), \
D3D12VA_ENCODE_RC_MODE(QVBR, "Quality-defined variable-bitrate")
+#define D3D12VA_ENCODE_INTRA_REFRESH_MODE(name, desc) \
+ { #name, desc, 0, AV_OPT_TYPE_CONST, { .i64 =
D3D12_VIDEO_ENCODER_INTRA_REFRESH_MODE_ ## name }, \
+ 0, 0, FLAGS, .unit = "intra_refresh_mode" }
+#define D3D12VA_ENCODE_INTRA_REFRESH_OPTIONS \
+ { "intra_refresh_mode", \
+ "Set intra refresh mode", \
+ OFFSET(common.intra_refresh.Mode), AV_OPT_TYPE_INT, \
+ { .i64 = D3D12_VIDEO_ENCODER_INTRA_REFRESH_MODE_NONE }, \
+ D3D12_VIDEO_ENCODER_INTRA_REFRESH_MODE_NONE, \
+ D3D12_VIDEO_ENCODER_INTRA_REFRESH_MODE_ROW_BASED, FLAGS, .unit =
"intra_refresh_mode" }, \
+ D3D12VA_ENCODE_INTRA_REFRESH_MODE(NONE, "Disable intra refresh"), \
+ D3D12VA_ENCODE_INTRA_REFRESH_MODE(ROW_BASED, "Row-based intra refresh"), \
+ { "intra_refresh_duration", \
+ "Number of frames over which to spread intra refresh (0 = GOP size)", \
+ OFFSET(common.intra_refresh.IntraRefreshDuration), AV_OPT_TYPE_INT, \
+ { .i64 = 0 }, 0, INT_MAX, FLAGS }
+
#endif /* AVCODEC_D3D12VA_ENCODE_H */
diff --git a/libavcodec/d3d12va_encode_h264.c b/libavcodec/d3d12va_encode_h264.c
index 967544ea24..8abc9935a1 100644
--- a/libavcodec/d3d12va_encode_h264.c
+++ b/libavcodec/d3d12va_encode_h264.c
@@ -178,7 +178,7 @@ static int
d3d12va_encode_h264_init_sequence_params(AVCodecContext *avctx)
.Codec = D3D12_VIDEO_ENCODER_CODEC_H264,
.InputFormat = hwctx->format,
.RateControl = ctx->rc,
- .IntraRefresh =
D3D12_VIDEO_ENCODER_INTRA_REFRESH_MODE_NONE,
+ .IntraRefresh = ctx->intra_refresh.Mode,
.SubregionFrameEncoding =
D3D12_VIDEO_ENCODER_FRAME_SUBREGION_LAYOUT_MODE_FULL_FRAME,
.ResolutionsListCount = 1,
.pResolutionList = &ctx->resolution,
@@ -549,6 +549,7 @@ static const AVOption d3d12va_encode_h264_options[] = {
HW_BASE_ENCODE_COMMON_OPTIONS,
D3D12VA_ENCODE_COMMON_OPTIONS,
D3D12VA_ENCODE_RC_OPTIONS,
+ D3D12VA_ENCODE_INTRA_REFRESH_OPTIONS,
{ "qp", "Constant QP (for P-frames; scaled by qfactor/qoffset for I/B)",
OFFSET(qp), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 52, FLAGS },
diff --git a/libavcodec/d3d12va_encode_hevc.c b/libavcodec/d3d12va_encode_hevc.c
index 01e5b4cb4c..04f4870907 100644
--- a/libavcodec/d3d12va_encode_hevc.c
+++ b/libavcodec/d3d12va_encode_hevc.c
@@ -250,7 +250,7 @@ static int
d3d12va_encode_hevc_init_sequence_params(AVCodecContext *avctx)
.Codec = D3D12_VIDEO_ENCODER_CODEC_HEVC,
.InputFormat = hwctx->format,
.RateControl = ctx->rc,
- .IntraRefresh =
D3D12_VIDEO_ENCODER_INTRA_REFRESH_MODE_NONE,
+ .IntraRefresh = ctx->intra_refresh.Mode,
.SubregionFrameEncoding =
D3D12_VIDEO_ENCODER_FRAME_SUBREGION_LAYOUT_MODE_FULL_FRAME,
.ResolutionsListCount = 1,
.pResolutionList = &ctx->resolution,
@@ -706,6 +706,7 @@ static const AVOption d3d12va_encode_hevc_options[] = {
HW_BASE_ENCODE_COMMON_OPTIONS,
D3D12VA_ENCODE_COMMON_OPTIONS,
D3D12VA_ENCODE_RC_OPTIONS,
+ D3D12VA_ENCODE_INTRA_REFRESH_OPTIONS,
{ "qp", "Constant QP (for P-frames; scaled by qfactor/qoffset for I/B)",
OFFSET(qp), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 52, FLAGS },
--
2.49.1
_______________________________________________
ffmpeg-devel mailing list -- [email protected]
To unsubscribe send an email to [email protected]
