I am not a lawyer. That said, you should look up the term "tortious interference" and consult with an attorney. If the other guy knows that the person who granted his "rights" has no authority, you may already have committed a tort by threatening you. A letter from an attorney convincing him that you know your rights and will go after him if he commits a tort against you might do the trick.
Hope this helps. --Mark
