On Fri, Nov 21, 2025 at 09:39:31PM +0000, David Howells wrote:
> Eric Biggers <[email protected]> wrote:
>
> > > > + if (memcmp(ws->ctildeprime, ctilde, params->ctilde_len) != 0)
> > > > + return -EBADMSG;
> > >
> > > Actually, this should return -EKEYREJECTED, not -EBADMSG.
> >
> > Who/what decided that?
>
> I did. When I added RSA support in 2012 for module signing. Note that it
> was originally added as part of crypto/asymmetric_keys/ and was not covered by
> a crypto API. The RSA code has since been moved to crypto/ and is now
> accessed through the crypto API, but it has retained this error code and this
> is also used by other public key algos.
>
> > A lot of the crypto code uses -EBADMSG already.
> > crypto_aead uses it, for example.
>
> ecdsa.c:60: return -EKEYREJECTED;
> ecrdsa.c:111: return -EKEYREJECTED;
> ecrdsa.c:139: return -EKEYREJECTED;
> ecrdsa.c:239: return -EKEYREJECTED;
> rsassa-pkcs1.c:293: return -EKEYREJECTED;
> rsassa-pkcs1.c:295: return -EKEYREJECTED;
crypto/aegis128-core.c:442: return -EBADMSG;
crypto/aegis128-core.c:499: return -EBADMSG;
crypto/algif_aead.c:313: if (err == -EIOCBQUEUED || err
== -EBADMSG || !ret)
crypto/authenc.c:223: return -EBADMSG;
crypto/authencesn.c:220: return -EBADMSG;
crypto/ccm.c:336: err = -EBADMSG;
crypto/ccm.c:384: return -EBADMSG;
crypto/chacha20poly1305.c:90: return -EBADMSG;
crypto/dh.c:207: ret = -EBADMSG;
crypto/dh.c:221: ret = -EBADMSG;
crypto/dh.c:242: ret = -EBADMSG;
crypto/ecdsa.c:37: return -EBADMSG;
crypto/ecrdsa.c:101: return -EBADMSG;
crypto/gcm.c:471: return crypto_memneq(iauth_tag, auth_tag, authsize) ?
-EBADMSG : 0;
crypto/krb5enc.c:259: return -EBADMSG;
crypto/rsa.c:150: ret = -EBADMSG;
crypto/rsa.c:189: ret = -EBADMSG;
crypto/rsassa-pkcs1.c:275: return -EBADMSG;
crypto/rsassa-pkcs1.c:282: return -EBADMSG;
crypto/rsassa-pkcs1.c:286: return -EBADMSG;
crypto/rsassa-pkcs1.c:288: return -EBADMSG;
crypto/testmgr.c:90: * algorithm might result in EINVAL rather than
EBADMSG, due to other
crypto/testmgr.c:2179: (err != vec->crypt_error && !(err == -EBADMSG &&
vec->novrfy))) {
crypto/testmgr.c:2183: vec->crypt_error != 0 && vec->crypt_error
!= -EBADMSG)
crypto/testmgr.c:2184: sprintf(expected_error, "-EBADMSG or
%d",
crypto/testmgr.c:2187: sprintf(expected_error, "-EBADMSG");
include/crypto/aead.h:37: * operation is that the caller should explicitly
check for -EBADMSG of the
include/crypto/aead.h:39: * a breach in the integrity of the message. In
essence, that -EBADMSG error
include/crypto/aead.h:375: * Return: 0 if the cipher operation was successful;
-EBADMSG: The AEAD
That list actually includes the same three files that use -EKEYREJECTED.
It looks like if the signature verification fails "early" it's -EBADMSG,
whereas if it fails "late" it's -EKEYREJECTED? I'm skeptical that
that's a meaningful difference. And it's not like this is documented
either; crypto_sig_verify() just says "error code in case of error".
- Eric
