For the reference of others having this problem: I was able to upgrade by using pg_upgrade instead of pg_upgradecluster, following the instructions here:
https://wiki.postgresql.org/wiki/Using_pg_upgrade_on_Ubuntu/Debian The upgrade maintained the NOT VALID constraints. I could also have used pg_dump; it included the NOT VALID constraints in the dump file. The contraints are added at the end so that the import works.
