Hi,
Ludovic Courtès <l...@gnu.org> skribis:
> [10219|3] Peer requested CA: CN=GNUTLS TEST SERVER,OU=GNUTLS,O=FSF,C=GR
> [10219|3] Peer requested CA: CN=GNUTLS INTERMEDIATE TEST
> CA,OU=GNUTLS,O=FSF,C=GR
> [10219|4] checking cert compat with RSA-SHA256
> [10219|3] ASSERT:
> ../../../lib/ext/signature.c[_gnutls_session_sign_algo_enabled]:433
> [10219|4] Signature algorithm RSA-SHA256 is not enabled
> [10219|4] checking cert compat with RSA-PSS-SHA256
> [10219|4]
> throw to `gnutls-error' with args (#<gnutls-error-enum Public key signing has
> failed.> read_from_session_record_port)
> checking cert compat with RSA-PSS-RSAE-SHA256
> [10219|4] HSK[0x55c505382030]: CERTIFICATE was queued [693 bytes]
> [10219|5] REC[0x55c505382030]: Preparing Packet Handshake(22) with length:
> 693 and min pad: 0
> [10219|5] REC[0x55c505382030]: Sent Packet[8906] Handshake(22) in epoch 2 and
> length: 715
> [10219|4] HSK[0x55c505382030]: signing TLS 1.3 handshake data: using
> RSA-PSS-RSAE-SHA256 and PRF: SHA384
> [10219|3] ASSERT: ../../../lib/nettle/pk.c[_rsa_pss_sign_digest_tr]:805
[...]
> In all cases, it’s the client failing to reauthenticate. One example is
> ‘_nettle_rsa_sec_compute_root_tr’ returning 0 and thus leading to
> GNUTLS_E_PK_SIGN_FAILED as we see above.
In the end, it turns out that GMP bignums as used by Nettle are being
overwritten when GC runs. Here’s an example of key material being
overwritten:
--8<---------------cut here---------------start------------->8---
rsa_sec_check_root (m=0x7f3ecbc6d100, x=0x7f3ecbc6d080, pub=0x7ffd2ca5ac90) at
rsa-sign-tr.c:257
257 in rsa-sign-tr.c
(rr) p pub->n._mp_d
$12 = (mp_limb_t *) 0x7f3ecbc6db00
(rr) p *pub->n._mp_d
$13 = 139907683506816
(rr) p *pub->e._mp_d
$14 = 65537
(rr) watch *$12
Hardware watchpoint 5: *$12
(rr) rc
Continuing.
Thread 14 hit Hardware watchpoint 5: *$12
Old value = 139907683506816
New value = 13832525101577573263
0x00007f3ecf963785 in GC_reclaim_generic ()
from /gnu/store/zg126cjicrpm2p6zc08ra5vh4ddag7ww-libgc-8.0.4/lib/libgc.so.1
(rr) bt
#0 0x00007f3ecf963785 in GC_reclaim_generic ()
from /gnu/store/zg126cjicrpm2p6zc08ra5vh4ddag7ww-libgc-8.0.4/lib/libgc.so.1
#1 0x00007f3ecf976fef in GC_generic_malloc_many ()
from /gnu/store/zg126cjicrpm2p6zc08ra5vh4ddag7ww-libgc-8.0.4/lib/libgc.so.1
#2 0x00007f3ecf9776b3 in GC_malloc_kind ()
from /gnu/store/zg126cjicrpm2p6zc08ra5vh4ddag7ww-libgc-8.0.4/lib/libgc.so.1
#3 0x00007f3ecbcf98d2 in _nettle_gmp_alloc (n=n@entry=128) at gmp-glue.c:345
#4 0x00007f3ecbcf454f in rsa_sec_blind (mn=16, m=0x7f3ecbc6d180,
ri=0x7f3ecbc6d000, c=0x7f3ecbc6d100,
random=0x7f3ecbeb61e0 <rnd_nonce_func>, random_ctx=0x0, pub=0x7ffd2ca5ac90)
at rsa-sign-tr.c:175
#5 _nettle_rsa_sec_compute_root_tr (pub=pub@entry=0x7ffd2ca5ac90,
key=key@entry=0x7ffd2ca5acc0,
random_ctx=random_ctx@entry=0x0, random=random@entry=0x7f3ecbeb61e0
<rnd_nonce_func>, x=x@entry=0x7f3ecbc6d100,
m=0x7f3ecbc6d180, mn=16) at rsa-sign-tr.c:330
#6 0x00007f3ecbcf4a74 in nettle_rsa_compute_root_tr
(pub=pub@entry=0x7ffd2ca5ac90, key=key@entry=0x7ffd2ca5acc0,
random_ctx=random_ctx@entry=0x0, random=random@entry=0x7f3ecbeb61e0
<rnd_nonce_func>, x=x@entry=0x7ffd2ca5ac80,
m=m@entry=0x7ffd2ca5abf0) at rsa-sign-tr.c:365
#7 0x00007f3ecbcf5de0 in nettle_rsa_pss_sha256_sign_digest_tr
(pub=0x7ffd2ca5ac90, key=0x7ffd2ca5acc0,
random_ctx=0x0, random=0x7f3ecbeb61e0 <rnd_nonce_func>,
salt_length=<optimized out>,
salt=0xcf1890
"\003\365\204\222t\363B\334\316~S\227Q\tB\217\037\016\314\326\247U9\360\v\214\313\344`\263\212\316\230\305\313",
digest=0xcf1860
"q.\272\251\204\064X\302\fH4\262\263\363\024V5F\225\274\223\360U\fE\237\305$t\004\350u\230",
s=0x7ffd2ca5ac80) at rsa-pss-sha256-sign-tr.c:59
#8 0x00007f3ecbeb8709 in _rsa_pss_sign_digest_tr (rnd_ctx=0x0,
rnd_func=0x7f3ecbeb61e0 <rnd_nonce_func>,
s=0x7ffd2ca5ac80, digest=<optimized out>, salt_size=<optimized out>,
priv=0x7ffd2ca5acc0, pub=0x7ffd2ca5ac90,
dig=<optimized out>) at pk.c:807
#9 _wrap_nettle_pk_sign (algo=<optimized out>, signature=0x7ffd2ca5af30,
vdata=<optimized out>,
pk_params=<optimized out>, sign_params=<optimized out>) at pk.c:1175
#10 0x00007f3ecbe08374 in privkey_sign_and_hash_data (signer=0xce1990,
se=0x7f3ecbf50a20 <sign_algorithms+224>,
data=<optimized out>, signature=0x7ffd2ca5af30, params=0x7ffd2ca5ade0) at
privkey.c:1296
#11 0x00007f3ecbe08658 in gnutls_privkey_sign_data2
(signer=signer@entry=0xce1990, algo=<optimized out>,
flags=flags@entry=0, data=data@entry=0x7ffd2ca5ae50,
signature=signature@entry=0x7ffd2ca5af30) at privkey.c:1194
#12 0x00007f3ecbe1e203 in _gnutls13_handshake_sign_data
(session=session@entry=0xcebd40, cert=<optimized out>,
pkey=0xce1990, context=0x7f3ecbf46210 <srv_ctx>,
signature=signature@entry=0x7ffd2ca5af30,
se=se@entry=0x7f3ecbf50a20 <sign_algorithms+224>) at tls13-sig.c:207
#13 0x00007f3ecbe1d8db in _gnutls13_send_certificate_verify
(session=session@entry=0xcebd40, again=<optimized out>)
at tls13/certificate_verify.c:206
#14 0x00007f3ecbdd2fc1 in _gnutls13_handshake_server
(session=session@entry=0xcebd40) at handshake-tls13.c:464
#15 0x00007f3ecbdde10d in handshake_server (session=<optimized out>) at
handshake.c:3381
#16 gnutls_handshake (session=0xcebd40) at handshake.c:2783
#17 0x00007f3ecbf63e82 in scm_gnutls_handshake (session=<optimized out>) at
core.c:196
--8<---------------cut here---------------end--------------->8---
This is because Guile >= 3.0.1 and >= 2.2.7 changes the GMP allocation
functions such that they go through libgc¹. As a result, libgc may
reuse that memory when it becomes unreachable from its point of view; in
this case, since GnuTLS structures are not scanned by libgc, libgc
doesn’t “see” pointers to those bignums and thus considers they are no
longer reachable.
Where to go from here? Here are options that come to mind:
• Configure Nettle with ‘--enable-mini-gmp’. However, the manual
mentions that it’s “slower” and “more likely to leak side-channel
information” (info "(nettle) Installation").
• Have Guile use mini-GMP; this is not implemented yet.
• In Guile-GnuTLS, arrange so that GnuTLS allocations are made through
libgc. Unfortunately, ‘gnutls_global_set_mem_functions’ was
deprecated in GnuTLS 3.3.0 so this doesn’t look like an option.
• Build Guile with ‘scm_install_gmp_memory_functions = 0’. This would
have a negative impact on the performance of bignum-heavy workloads
such as the compiler itself.
I can’t think of a good workaround. Thoughts?
Ludo’.
¹
https://git.savannah.gnu.org/cgit/guile.git/commit/?id=00fbdfa7345765168e14438eed0b0b8c64c27ab9
