It seems that this issues was fixed now. Linux 6.4.13 (Debian: 6.4.0-4-amd64) and older still had that bug. However with Linux 6.5.3 (6.5.0-1-amd64) and newer it's not reproduceable for me anymore. Updated the issue on gitlab.freedesktop.org accordingly. Don't know which kernel commit fixed it exactly though. Thanks to whoever fixed it!
Regards, Linus