As a workaround, I figured out I can pass to debuild --buildinfo-option="-O"
which makes dpkg-genbuildinfo send the contents of the buildinfo file to stdout instead of creating the file, which means that it does not appear in the .changes file either.
