Hello, rmmod failure due to "busy" is annoying indeed, and a bit hard to fix, thus your approach with removing the individual hosts instead seems fine.
If the module isn't available though, I'd consider this a bad testbed. autopkgtest's testbed VM construction scripts (VM, container, cloud) all install the linux-image-extra bits to make sure that the module exists. If this silently gets skipped, it could then be that this functionality just never gets tested, so I'm not too happy about that part. Thanks, Martin