Source: libcrypto++
Version: 5.6.1-9
Severity: normal
Tags: patch
Hi!
libcrypto++ currently fails to build on x32. This is caused by attempts to
use x86 assembly -- the package would build if it used the portable generic
code instead.
Here's a patch. It's a hybrid between disabling assembly and making it work
-- in most places, it works without changes. In one place, it tried to
access a 32-bit variable of type size_t as a 64-bit value, so changing the
type to word64 works. I did disable the assembly in two places -- whirlpool
where it doesn't build (insanely macroized code makes it thoroughly
unobvious) and rijndael which did compile but segfaults in the testsuite.
The patch could be simpler by disabling assembly outright, or more complete
by fixing pieces I disabled, but it's probably ok for a first stab.
-- System Information:
Debian Release: stretch/sid
APT prefers unstable
APT policy: (500, 'unstable')
Architecture: x32 (x86_64)
Kernel: Linux 4.4.0-x32 (SMP w/6 CPU cores)
Locale: LANG=C.UTF-8, LC_CTYPE=C.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: sysvinit (via /sbin/init)
--- libcrypto++-5.6.1.orig/cpu.h
+++ libcrypto++-5.6.1/cpu.h
@@ -96,6 +96,8 @@ _mm_aesdeclast_si128 (__m128i a, __m128i
NAMESPACE_BEGIN(CryptoPP)
+#define X32 (defined __ILP32__ && defined __x86_64__)
+
#if CRYPTOPP_BOOL_X86 || CRYPTOPP_BOOL_X64
#define CRYPTOPP_CPUID_AVAILABLE
--- libcrypto++-5.6.1.orig/rijndael.cpp
+++ libcrypto++-5.6.1/rijndael.cpp
@@ -348,7 +348,7 @@ void Rijndael::Base::UncheckedSetKey(con
void Rijndael::Enc::ProcessAndXorBlock(const byte *inBlock, const byte *xorBlock, byte *outBlock) const
{
-#if CRYPTOPP_BOOL_SSE2_ASM_AVAILABLE || defined(CRYPTOPP_X64_MASM_AVAILABLE) || CRYPTOPP_BOOL_AESNI_INTRINSICS_AVAILABLE
+#if (CRYPTOPP_BOOL_SSE2_ASM_AVAILABLE || defined(CRYPTOPP_X64_MASM_AVAILABLE) || CRYPTOPP_BOOL_AESNI_INTRINSICS_AVAILABLE) && !X32
if (HasSSE2())
{
Rijndael::Enc::AdvancedProcessBlocks(inBlock, xorBlock, outBlock, 16, 0);
@@ -1179,7 +1179,7 @@ size_t Rijndael::Enc::AdvancedProcessBlo
return AESNI_AdvancedProcessBlocks(AESNI_Enc_Block, AESNI_Enc_4_Blocks, (const __m128i *)m_key.begin(), m_rounds, inBlocks, xorBlocks, outBlocks, length, flags);
#endif
-#if CRYPTOPP_BOOL_SSE2_ASM_AVAILABLE || defined(CRYPTOPP_X64_MASM_AVAILABLE)
+#if CRYPTOPP_BOOL_SSE2_ASM_AVAILABLE && !X32 || defined(CRYPTOPP_X64_MASM_AVAILABLE)
if (HasSSE2())
{
if (length < BLOCKSIZE)
--- libcrypto++-5.6.1.orig/vmac.cpp
+++ libcrypto++-5.6.1/vmac.cpp
@@ -735,7 +735,7 @@ size_t VMAC_Base::HashMultipleBlocks(con
return remaining;
}
-static word64 L3Hash(const word64 *input, const word64 *l3Key, size_t len)
+static word64 L3Hash(const word64 *input, const word64 *l3Key, word64 len)
{
word64 rh, rl, t, z=0;
word64 p1 = input[0], p2 = input[1];
--- libcrypto++-5.6.1.orig/whrlpool.cpp
+++ libcrypto++-5.6.1/whrlpool.cpp
@@ -391,7 +391,7 @@ static const word64 Whirlpool_C[4*256+R]
// Whirlpool basic transformation. Transforms state based on block.
void Whirlpool::Transform(word64 *digest, const word64 *block)
{
-#if CRYPTOPP_BOOL_SSE2_ASM_AVAILABLE
+#if CRYPTOPP_BOOL_SSE2_ASM_AVAILABLE && !X32
if (HasISSE())
{
// MMX version has the same structure as C version below
