I experienced this exact same issue when I tried to upgrade my home server in April 2022.
Also trying to pass a Conbee II to a HomeAssistant VM, same issues, same dmesg spam. I ended up rolling back to Buster because I had no time to investigate further. -Alejandro