drm_gtf_mode_complex() derives the blanking duty cycle from the GTF
C/M/K/J coefficients and divides by its complement:
ideal_duty_cycle = GTF_C_PRIME * 1000 -
(GTF_M_PRIME * 1000000 / hfreq_est);
hblank = total_active_pixels * ideal_duty_cycle /
(100000 - ideal_duty_cycle);
ideal_duty_cycle is an unsigned per-mille percentage that must stay below
100000, but nothing enforces it. Via drm_gtf2_mode() the coefficients come
straight from an EDID secondary-GTF range descriptor, so a crafted monitor
(or an edid_override re-probed with DRM_IOCTL_MODE_GETCONNECTOR) can set
ideal_duty_cycle to exactly 100000, making the divisor 0 and trapping with
a divide error; a larger value wraps the unsigned subtraction and yields a
garbage modeline.
Reject a duty cycle of 100% or more, which is never a valid GTF result.
drm_gtf2_mode() and the inferred-mode helpers already handle a NULL return.
Crash:
Oops: divide error: 0000 [#1] SMP KASAN NOPTI
RIP: 0010:drm_gtf_mode_complex (drivers/gpu/drm/drm_modes.c:972)
Call Trace:
drm_gtf2_mode (drivers/gpu/drm/drm_edid.c:3365)
drm_mode_std (drivers/gpu/drm/drm_edid.c:3464)
_drm_edid_connector_add_modes.part.0 (drivers/gpu/drm/drm_edid.c:4024)
drm_edid_connector_add_modes (drivers/gpu/drm/drm_edid.c:7020)
bochs_connector_helper_get_modes (drivers/gpu/drm/tiny/bochs.c:575)
drm_helper_probe_single_connector_modes
(drivers/gpu/drm/drm_probe_helper.c:426)
drm_mode_getconnector (drivers/gpu/drm/drm_connector.c:3376)
...
__x64_sys_ioctl (fs/ioctl.c:597)
do_syscall_64 (arch/x86/entry/syscall_64.c:94)
entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:121)
Kernel panic - not syncing: Fatal exception
Fixes: 26bbdadad356 ("drm/mode: add the GTF algorithm in kernel space")
Reported-by: Weiming Shi <[email protected]>
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Xiang Mei <[email protected]>
---
drivers/gpu/drm/drm_modes.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/drivers/gpu/drm/drm_modes.c b/drivers/gpu/drm/drm_modes.c
index 3f8e025fd6d9..b6f263e9ba86 100644
--- a/drivers/gpu/drm/drm_modes.c
+++ b/drivers/gpu/drm/drm_modes.c
@@ -966,6 +966,10 @@ drm_gtf_mode_complex(struct drm_device *dev, int hdisplay,
int vdisplay,
/* 18.Find the ideal blanking duty cycle from blanking duty cycle */
ideal_duty_cycle = GTF_C_PRIME * 1000 -
(GTF_M_PRIME * 1000000 / hfreq_est);
+ if (ideal_duty_cycle >= 100000) {
+ drm_mode_destroy(dev, drm_mode);
+ return NULL;
+ }
/* 19.Find the number of pixels in the blanking time to the nearest
* double character cell: */
hblank = total_active_pixels * ideal_duty_cycle /
--
2.43.0
