On Wed, Sep 07, 2016 at 11:38:01PM +0200, Sebastian Andrzej Siewior wrote: > On 2016-06-26 12:24:40 [+0200], Kurt Roeckx wrote: > > If you have problems making things work, feel free to contact us. > > HALP! > > It builds against old & new ssl and I am proud what I managed in > rdssl_cert_to_rkey(). However they dereference > EVP_MD->required_pkey_type which vanished in 1.1 and I have no idea what > it was and its purpose was. They don't even assign anything to it in > 1.0.2h.
I think this is the relevant commit: commit 7f572e958b13041056f377a62d3219633cfb1e8a Author: Dr. Stephen Henson <st...@openssl.org> Date: Wed Dec 2 13:57:04 2015 +0000 Remove legacy sign/verify from EVP_MD. Remove sign/verify and required_pkey_type fields of EVP_MD: these are a legacy from when digests were linked to public key types. All signing is now handled by the corresponding EVP_PKEY_METHOD. Only allow supported digest types in RSA EVP_PKEY_METHOD: other algorithms already block unsupported types. Remove now obsolete EVP_dss1() and EVP_ecdsa(). Reviewed-by: Richard Levitte <levi...@openssl.org> The code seems to be doing: The signature algorithm was sha1WithRSAEncryption, so the public key they passed us must have been an RSA key. But from what I understand, what the code really did was: it's an EVP_sha1(), it's an an RSA key. But if it was EVP_sha256(), it was an unknown key type. What they do might have worked in the past by accident. But it clearly doesn't work properly. I don't think there is a way of doing what they want to do. They should really just pass the type of the public key to the function. Kurt