Hi Ard,
I love your patch! Yet something to improve:
[auto build test ERROR on cryptodev/master]
[also build test ERROR on crypto/master linus/master v5.11-rc1 next-20201223]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--ba
ter
config: x86_64-randconfig-a002-20201231 (attached as .config)
compiler: clang version 12.0.0 (https://github.com/llvm/llvm-project
6b316febb4388764789677f81f03aff373ec35b2)
reproduce (this is a W=1 build):
wget
https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross
e' as documented in
https://git-scm.com/docs/git-format-patch]
url:
https://github.com/0day-ci/linux/commits/Ard-Biesheuvel/crypto-x86-remove-glue-helper-module/20210101-013104
base:
https://git.kernel.org/pub/scm/linux/kernel/git/herbert/cryptodev-2.6.git master
config: x86_64-randconfig-a00
ter
config: i386-randconfig-c001-20201231 (attached as .config)
compiler: gcc-9 (Debian 9.3.0-15) 9.3.0
reproduce (this is a W=1 build):
#
https://github.com/0day-ci/linux/commit/120e62f276c7436572e8a67ecfb9bbb1125bfd8d
git remote add linux-review https://github.com/0day-ci/linux
On Thu, Dec 31, 2020 at 06:23:37PM +0100, Ard Biesheuvel wrote:
> The Camellia, Serpent and Twofish related header files only contain
> declarations that are shared between different implementations of the
> respective algorithms residing under arch/x86/crypto, and none of their
> contents should b
On Thu, Dec 31, 2020 at 06:23:36PM +0100, Ard Biesheuvel wrote:
> All dependencies on the x86 glue helper module have been replaced by
> local instantiations of the new ECB/CBC preprocessor helper macros, so
> the glue helper module can be retired.
>
> Signed-off-by: Ard Biesheuvel
> ---
> arch/
On Thu, Dec 31, 2020 at 06:23:35PM +0100, Ard Biesheuvel wrote:
> Replace the glue helper dependency with implementations of ECB and CBC
> based on the new CPP macros, which avoid the need for indirect calls.
>
> Signed-off-by: Ard Biesheuvel
> ---
> arch/x86/crypto/twofish_avx_glue.c | 73
On Thu, Dec 31, 2020 at 06:23:34PM +0100, Ard Biesheuvel wrote:
> Replace the glue helper dependency with implementations of ECB and CBC
> based on the new CPP macros, which avoid the need for indirect calls.
>
> Signed-off-by: Ard Biesheuvel
> ---
> arch/x86/crypto/cast6_avx_glue.c | 61 ++-
On Thu, Dec 31, 2020 at 06:23:33PM +0100, Ard Biesheuvel wrote:
> Replace the glue helper dependency with implementations of ECB and CBC
> based on the new CPP macros, which avoid the need for indirect calls.
>
> Signed-off-by: Ard Biesheuvel
> ---
> arch/x86/crypto/cast5_avx_glue.c | 184 ++
On Thu, Dec 31, 2020 at 06:23:32PM +0100, Ard Biesheuvel wrote:
> Replace the glue helper dependency with implementations of ECB and CBC
> based on the new CPP macros, which avoid the need for indirect calls.
>
> Signed-off-by: Ard Biesheuvel
> ---
> arch/x86/crypto/serpent_avx2_glue.c | 73
On Thu, Dec 31, 2020 at 06:23:30PM +0100, Ard Biesheuvel wrote:
> The x86 glue helper module has started to show its age:
> - It relies heavily on function pointers to invoke asm helper functions that
> operate on fixed input sizes that are relatively small. This means the
> performance is seve
On Thu, Dec 31, 2020 at 06:23:31PM +0100, Ard Biesheuvel wrote:
> Replace the glue helper dependency with implementations of ECB and CBC
> based on the new CPP macros, which avoid the need for indirect calls.
>
> Signed-off-by: Ard Biesheuvel
> ---
> arch/x86/crypto/camellia_aesni_avx2_glue.c |
On Thu, Dec 31, 2020 at 06:23:28PM +0100, Ard Biesheuvel wrote:
> DES or Triple DES in counter mode is never used in the kernel, so there
> is no point in keeping an accelerated implementation around.
>
> Signed-off-by: Ard Biesheuvel
> ---
> arch/x86/crypto/des3_ede_glue.c | 104 ---
On Thu, Dec 31, 2020 at 06:23:24PM +0100, Ard Biesheuvel wrote:
> CAST5 in CTR mode is never used by the kernel directly, and is highly
> unlikely to be relied upon by dm-crypt or algif_skcipher. So let's drop
> the accelerated CTR mode implementation, and instead, rely on the CTR
> template and th
On Thu, Dec 31, 2020 at 06:23:29PM +0100, Ard Biesheuvel wrote:
> Blowfish in counter mode is never used in the kernel, so there
> is no point in keeping an accelerated implementation around.
>
> Signed-off-by: Ard Biesheuvel
> ---
> arch/x86/crypto/blowfish_glue.c | 107
>
Now that the XTS template can wrap accelerated ECB modes, it can be
used to implement Serpent in XTS mode as well, which turns out to
be at least as fast, and sometimes even faster
Acked-by: Eric Biggers
Signed-off-by: Ard Biesheuvel
---
arch/x86/crypto/serpent-avx-x86_64-asm_64.S | 48 ---
All dependencies on the x86 glue helper module have been replaced by
local instantiations of the new ECB/CBC preprocessor helper macros, so
the glue helper module can be retired.
Signed-off-by: Ard Biesheuvel
---
arch/x86/crypto/Makefile | 2 -
arch/x86/crypto/glue_helper.c
Replace the glue helper dependency with implementations of ECB and CBC
based on the new CPP macros, which avoid the need for indirect calls.
Signed-off-by: Ard Biesheuvel
---
arch/x86/crypto/serpent_avx2_glue.c | 73 +-
arch/x86/crypto/serpent_avx_glue.c | 61 ---
ar
Replace the glue helper dependency with implementations of ECB and CBC
based on the new CPP macros, which avoid the need for indirect calls.
Signed-off-by: Ard Biesheuvel
---
arch/x86/crypto/cast6_avx_glue.c | 61 ++--
crypto/Kconfig | 1 -
2 files changed, 17
Now that the XTS template can wrap accelerated ECB modes, it can be
used to implement CAST6 in XTS mode as well, which turns out to
be at least as fast, and sometimes even faster
Acked-by: Eric Biggers
Signed-off-by: Ard Biesheuvel
---
arch/x86/crypto/cast6-avx-x86_64-asm_64.S | 56 ---
The Camellia, Serpent and Twofish related header files only contain
declarations that are shared between different implementations of the
respective algorithms residing under arch/x86/crypto, and none of their
contents should be used elsewhere. So move the header files into the
same location, and u
Replace the glue helper dependency with implementations of ECB and CBC
based on the new CPP macros, which avoid the need for indirect calls.
Signed-off-by: Ard Biesheuvel
---
arch/x86/crypto/cast5_avx_glue.c | 184 ++--
1 file changed, 17 insertions(+), 167 deletions(-)
diff --g
Replace the glue helper dependency with implementations of ECB and CBC
based on the new CPP macros, which avoid the need for indirect calls.
Signed-off-by: Ard Biesheuvel
---
arch/x86/crypto/twofish_avx_glue.c | 73 +-
arch/x86/crypto/twofish_glue_3way.c | 80 ++-
The x86 glue helper module has started to show its age:
- It relies heavily on function pointers to invoke asm helper functions that
operate on fixed input sizes that are relatively small. This means the
performance is severely impacted by retpolines.
- It goes to great lengths to amortize the
Replace the glue helper dependency with implementations of ECB and CBC
based on the new CPP macros, which avoid the need for indirect calls.
Signed-off-by: Ard Biesheuvel
---
arch/x86/crypto/camellia_aesni_avx2_glue.c | 85 ++--
arch/x86/crypto/camellia_aesni_avx_glue.c | 73 +++
The glue helper's CTR routines are no longer used, so drop them.
Acked-by: Eric Biggers
Signed-off-by: Ard Biesheuvel
---
arch/x86/crypto/glue_helper-asm-avx.S | 45
arch/x86/crypto/glue_helper-asm-avx2.S| 58
arch/x86/crypto/glue_helper.c | 72
Blowfish in counter mode is never used in the kernel, so there
is no point in keeping an accelerated implementation around.
Signed-off-by: Ard Biesheuvel
---
arch/x86/crypto/blowfish_glue.c | 107
crypto/Kconfig | 1 +
2 files changed, 1 insertion(+), 107
Twofish in CTR mode is never used by the kernel directly, and is highly
unlikely to be relied upon by dm-crypt or algif_skcipher. So let's drop
the accelerated CTR mode implementation, and instead, rely on the CTR
template and the bare cipher.
Acked-by: Eric Biggers
Signed-off-by: Ard Biesheuvel
DES or Triple DES in counter mode is never used in the kernel, so there
is no point in keeping an accelerated implementation around.
Signed-off-by: Ard Biesheuvel
---
arch/x86/crypto/des3_ede_glue.c | 104
crypto/Kconfig | 1 +
2 files changed, 1 insertion
Serpent in CTR mode is never used by the kernel directly, and is highly
unlikely to be relied upon by dm-crypt or algif_skcipher. So let's drop
the accelerated CTR mode implementation, and instead, rely on the CTR
template and the bare cipher.
Acked-by: Eric Biggers
Signed-off-by: Ard Biesheuvel
CAST6 in CTR mode is never used by the kernel directly, and is highly
unlikely to be relied upon by dm-crypt or algif_skcipher. So let's drop
the accelerated CTR mode implementation, and instead, rely on the CTR
template and the bare cipher.
Acked-by: Eric Biggers
Signed-off-by: Ard Biesheuvel
-
CAST5 in CTR mode is never used by the kernel directly, and is highly
unlikely to be relied upon by dm-crypt or algif_skcipher. So let's drop
the accelerated CTR mode implementation, and instead, rely on the CTR
template and the bare cipher.
Signed-off-by: Ard Biesheuvel
---
arch/x86/crypto/cast
The glue helper's XTS routines are no longer used, so drop them.
Acked-by: Eric Biggers
Signed-off-by: Ard Biesheuvel
---
arch/x86/crypto/glue_helper-asm-avx.S | 59
arch/x86/crypto/glue_helper-asm-avx2.S| 78 --
arch/x86/crypto/glue_helper.c | 154 ---
Camellia in CTR mode is never used by the kernel directly, and is highly
unlikely to be relied upon by dm-crypt or algif_skcipher. So let's drop
the accelerated CTR mode implementation, and instead, rely on the CTR
template and the bare cipher.
Acked-by: Eric Biggers
Signed-off-by: Ard Biesheuvel
Now that the XTS template can wrap accelerated ECB modes, it can be
used to implement Camellia in XTS mode as well, which turns out to
be at least as fast, and sometimes even faster.
Acked-by: Eric Biggers
Signed-off-by: Ard Biesheuvel
---
arch/x86/crypto/camellia-aesni-avx-asm_64.S | 181
NOTE: this is a follow-up to '[RFC PATCH 00/10] crypto: x86 - remove XTS and
CTR glue helper code' [0].
After applying my performance fixes for AES-NI in XTS mode, the only
remaining users of the x86 glue helper module are the niche algorithms
camellia, cast5/6, serpent and twofish.
It is not cle
Now that the XTS template can wrap accelerated ECB modes, it can be
used to implement Twofish in XTS mode as well, which turns out to
be at least as fast, and sometimes even faster
Acked-by: Eric Biggers
Signed-off-by: Ard Biesheuvel
---
arch/x86/crypto/twofish-avx-x86_64-asm_64.S | 53
The AES-NI driver implements XTS via the glue helper, which consumes
a struct with sets of function pointers which are invoked on chunks
of input data of the appropriate size, as annotated in the struct.
Let's get rid of this indirection, so that we can perform direct calls
to the assembler helper
The XTS asm helper arrangement is a bit odd: the 8-way stride helper
consists of back-to-back calls to the 4-way core transforms, which
are called indirectly, based on a boolean that indicates whether we
are performing encryption or decryption.
Given how costly indirect calls are on x86, let's swi
The AES-NI implementation of XTS was impacted significantly by the retpoline
changes, which is due to the fact that both its asm helper and the chaining
mode glue library use indirect calls for processing small quantitities of
data
So let's fix this, by:
- creating a minimal, backportable fix that
40 matches
Mail list logo
