RE: [PATCH 1/5] net/mlx5/hws: add support for matching on bth_a bit
> RTE_FLOW_ITEM_TYPE_IB_BTH matches an InfiniBand base transport header.
> We extend the match on the acknowledgment bit (BTH_A).
>
> Signed-off-by: Itamar Gozlan
Series-acked-by: Matan Azrad
> ---
> drivers/net/mlx5/hws/mlx5dr_definer.c | 12 ++--
> drivers/net/mlx5/hws/mlx5dr_definer.h | 1 +
> 2 files changed, 11 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/net/mlx5/hws/mlx5dr_definer.c
> b/drivers/net/mlx5/hws/mlx5dr_definer.c
> index 33d0f2d18e..b82af9d102 100644
> --- a/drivers/net/mlx5/hws/mlx5dr_definer.c
> +++ b/drivers/net/mlx5/hws/mlx5dr_definer.c
> @@ -177,7 +177,8 @@ struct mlx5dr_definer_conv_data {
> X(SET_BE32, ipsec_spi, v->hdr.spi,
> rte_flow_item_esp) \
> X(SET_BE32, ipsec_sequence_number, v->hdr.seq,
> rte_flow_item_esp) \
> X(SET, ib_l4_udp_port, UDP_ROCEV2_PORT,
> rte_flow_item_ib_bth) \
> - X(SET, ib_l4_opcode, v->hdr.opcode,
> rte_flow_item_ib_bth)
> + X(SET, ib_l4_opcode, v->hdr.opcode,
> rte_flow_item_ib_bth) \
> + X(SET, ib_l4_bth_a,v->hdr.a,
> rte_flow_item_ib_bth) \
>
> /* Item set function format */
> #define X(set_type, func_name, value, item_type) \ @@ -2148,7 +2149,7 @@
> mlx5dr_definer_conv_item_ib_l4(struct mlx5dr_definer_conv_data *cd,
>
> if (m->hdr.se || m->hdr.m || m->hdr.padcnt || m->hdr.tver ||
> m->hdr.pkey || m->hdr.f || m->hdr.b || m->hdr.rsvd0 ||
> - m->hdr.a || m->hdr.rsvd1 || !is_mem_zero(m->hdr.psn, 3)) {
> + m->hdr.rsvd1 || !is_mem_zero(m->hdr.psn, 3)) {
> rte_errno = ENOTSUP;
> return rte_errno;
> }
> @@ -2167,6 +2168,13 @@ mlx5dr_definer_conv_item_ib_l4(struct
> mlx5dr_definer_conv_data *cd,
> DR_CALC_SET_HDR(fc, ib_l4, qp);
> }
>
> + if (m->hdr.a) {
> + fc = &cd->fc[MLX5DR_DEFINER_FNAME_IB_L4_A];
> + fc->item_idx = item_idx;
> + fc->tag_set = &mlx5dr_definer_ib_l4_bth_a_set;
> + DR_CALC_SET_HDR(fc, ib_l4, ackreq);
> + }
> +
> return 0;
> }
>
> diff --git a/drivers/net/mlx5/hws/mlx5dr_definer.h
> b/drivers/net/mlx5/hws/mlx5dr_definer.h
> index 6b645f4cf0..bf026fa6bb 100644
> --- a/drivers/net/mlx5/hws/mlx5dr_definer.h
> +++ b/drivers/net/mlx5/hws/mlx5dr_definer.h
> @@ -136,6 +136,7 @@ enum mlx5dr_definer_fname {
> MLX5DR_DEFINER_FNAME_OKS2_MPLS4_I,
> MLX5DR_DEFINER_FNAME_IB_L4_OPCODE,
> MLX5DR_DEFINER_FNAME_IB_L4_QPN,
> + MLX5DR_DEFINER_FNAME_IB_L4_A,
> MLX5DR_DEFINER_FNAME_MAX,
> };
>
> --
> 2.38.1
RE: [PATCH v2 1/2] common/mlx5: fix build disabling
> If the dependency common/mlx5 is explicitly disabled, but net/mlx5 is not
> explicitly disabled, Meson will read the full recipe of net/mlx5 and will fail
> when accessing a variable from common/mlx5:
> drivers/net/mlx5/meson.build:76:4: ERROR: Unknown variable
> "mlx5_config".
>
> The solution is to stop parsing net/mlx5 if common/mlx5 is disabled.
> The deps array must be defined before stopping, in order to automatically
> disable the build of net/mlx5 and print the reason.
>
> The same protection is applied to other mlx5 drivers, so it will allow using
> the
> variable mlx5_config in future.
>
> Fixes: 22681deead3e ("net/mlx5/hws: enable hardware steering")
>
> Reported-by: Andrew Rybchenko
> Signed-off-by: Thomas Monjalon
> Tested-by: Andrew Rybchenko
Acked-by: Matan Azrad
RE: [PATCH v2 2/2] common/mlx5: move Meson config initialization and check
> The variable mlx5_config may be used by other mlx5 drivers and should be > always initialized. > By moving its initialization (with configuration file generation), it is made > consistent for Linux and Windows builds. > > And the check of mlx5_config in net/mlx5 is moved at the top of > net/mlx5/hws/meson.build so HWS requirements are in the right context. > > Signed-off-by: Thomas Monjalon > Tested-by: Andrew Rybchenko Acked-by: Matan Azrad
RE: [PATCH] net/mlx5: enable flow aging action
> As the queue-based aging API has been integrated[1], the flow aging action > support in HWS steering code can be enabled now. > > [1]: > https://patchwork.dpdk.org/project/dpdk/cover/20221026214943.3686635- > 1-michae...@nvidia.com/ > > Signed-off-by: Suanming Mou Acked-by: Matan Azrad
RE: [PATCH] net/mlx5/hws: remove deprecated rte_atomic
> The use of rte_atomic functions is deprecated and is not required in HWS
> code. HWS refcounts are used only during control and always under lock.
>
> Fixes: f8c8a6d8440d ("net/mlx5/hws: add action object")
> Signed-off-by: Alex Vesker
Acked-by: Matan Azrad
RE: [PATCH] net/mlx5/hws: fix memory leak on general pool db init
From: Alex Vesker
> On elemend db init we allocated the element_manager which was unused
> and not freed.
>
> Fixes: b4dd7bcb0dcbe ("net/mlx5/hws: add pool and buddy")
> Signed-off-by: Alex Vesker
> Reviewed-by: Erez Shitrit
Acked-by: Matan Azrad
RE: [PATCH] net/mlx5: fix error CQE dumping for vectorized Rx burst
From: Alexander Kozyrev
> There is a dump file with debug information created for an error CQE to help
> with troubleshooting later.
> It starts with the last CQE, which, presumably is the error CQE.
> But this is only true for the scalar Rx burst routing since we handle CQEs
> there
> one by one and detect the error immediately.
> For vectorized Rx bursts, we may already move to another CQE when we
> detect the error since we handle CQEs in batches there.
> Go back to the error CQE in this case to dump proper CQE.
>
> Fixes: 88c0733535 ("net/mlx5: extend Rx completion with error handling")
> Cc: sta...@dpdk.org
>
> Signed-off-by: Alexander Kozyrev
Acked-by: Matan Azrad
RE: [PATCH] net/mlx5: ignore non-critical syndromes for Rx queue
From: Alexander Kozyrev
> For non-fatal syndromes like LOCAL_LENGTH_ERR, the Rx queue reset
> shouldn't be triggered. Rx queue could continue with the next packets
> without any recovery. Only three syndromes warrant Rx queue reset:
> LOCAL_QP_OP_ERR, LOCAL_PROT_ERR and WR_FLUSH_ERR.
> Do not initiate a Rx queue reset in any other cases.
> Skip all non-critical error CQEs and continue with packet processing.
>
> Fixes: 88c0733535 ("net/mlx5: extend Rx completion with error handling")
> Cc: sta...@dpdk.org
>
> Signed-off-by: Alexander Kozyrev
Acked-by: Matan Azrad
RE: [PATCH] net/mlx5: check compressed CQE opcode for an error
From: Alexander Kozyrev
> The CQE opcode is never checked for a compressed CQE in the vectorized Rx
> burst routines. It is assumed that compressed CQEs are always valid and
> skipped error checking.
>
> This is obviously not the case and error CQEs may be compressed together as
> well. Need to check for the MLX5_CQE_RESP_ERR opcode and mark all the
> packets as bad ones in the compression session if it is there.
>
> Note that this issue is not applicable to the scalar Rx burst.
>
> Fixes: 6cb559d67b ("net/mlx5: add vectorized Rx/Tx burst for x86")
> Cc: sta...@dpdk.org
>
> Signed-off-by: Alexander Kozyrev
Acked-by: Matan Azrad
RE: [v2 00/16] net/mlx5/hws: support range and partial hash matching
From: Alex Vesker > ConnectX and BlueField devices which support HWS are capable of > performing advanced match, range-match and hashing operations on packet > headers. This patch-set introduces support for (1) range matching - allows > combining range and exact match over provided item fields using spec, last > and mask. > A useful example for Range and exact match is matching exact IP and port > range. This series also introduces (2) partial hash, which allows using the > same matcher and performing partial hashing in case of multiple match > templates are provided. The matcher create code finds the union and > intersection fields to split the hash and match in a way the HW can support. > This increases the match templates combinations possible although it may > limit the maximum PPS. Since (1) and (2) are not fully support on current HW > the implementation is done on top of a HW like FW API using WQEs. > > v2: > -Fix missed checkpatch comments > > Alex Vesker (16): > net/mlx5/hws: support synchronous drain > net/mlx5/hws: matcher remove AT and MT limitation > net/mlx5/hws: support GTA WQE write using FW command > net/mlx5/hws: add capability query for gen wqe command > net/mlx5/hws: align RTC create command with PRM format > net/mlx5/hws: add send FW match STE using gen WQE > net/mlx5/hws: add send FW range STE WQE > net/mlx5/hws: move matcher size check to function > net/mlx5/hws: support range match > net/mlx5/hws: redesign definer create > net/mlx5/hws: support partial hash > net/mlx5/hws: add range definer creation support > net/mlx5/hws: add FW WQE rule creation logic > net/mlx5/hws: add debug dump support for range and hash > net/mlx5/hws: rename pattern cache object > net/mlx5/hws: cache definer for reuse Series-acked-by: Matan Azrad > drivers/common/mlx5/mlx5_prm.h | 53 +- > drivers/net/mlx5/hws/mlx5dr.h | 6 +- > drivers/net/mlx5/hws/mlx5dr_cmd.c | 72 ++- > drivers/net/mlx5/hws/mlx5dr_cmd.h | 27 +- > drivers/net/mlx5/hws/mlx5dr_context.c | 12 +- > drivers/net/mlx5/hws/mlx5dr_context.h | 1 + > drivers/net/mlx5/hws/mlx5dr_debug.c| 41 +- > drivers/net/mlx5/hws/mlx5dr_debug.h| 4 +- > drivers/net/mlx5/hws/mlx5dr_definer.c | 750 +-- > -- drivers/net/mlx5/hws/mlx5dr_definer.h | 44 +- > drivers/net/mlx5/hws/mlx5dr_internal.h | 2 +- > drivers/net/mlx5/hws/mlx5dr_matcher.c | 279 ++--- > drivers/net/mlx5/hws/mlx5dr_matcher.h | 43 +- > drivers/net/mlx5/hws/mlx5dr_pat_arg.c | 49 +- > drivers/net/mlx5/hws/mlx5dr_pat_arg.h | 6 +- > drivers/net/mlx5/hws/mlx5dr_rule.c | 193 ++- > drivers/net/mlx5/hws/mlx5dr_rule.h | 2 + > drivers/net/mlx5/hws/mlx5dr_send.c | 169 +- > drivers/net/mlx5/hws/mlx5dr_send.h | 24 +- > drivers/net/mlx5/mlx5_flow_hw.c| 2 +- > 20 files changed, 1475 insertions(+), 304 deletions(-) > > -- > 2.18.1
Re: [dpdk-dev] [PATCH] vdpa/mlx5: improve portability of thread naming
From: Thomas Monjalon > The function pthread_setname_np is non-portable, so it may be unavailable in > old glibc or other systems. > The function rte_thread_setname is workarounding portability issues. > > Signed-off-by: Thomas Monjalon Acked-by: Matan Azrad Thanks!
[dpdk-dev] [PATCH] examples/l2fwd-crypto: support cipher multiple data-unit
Patch [1] added a new API to support multiple data-units in cipher
operations.
Add a support for this API:
- Add a new command-line argument to provide the data-unit length.
- Set the length in the cipher xform.
- Validate device capabilities for this feature.
- Pad the AES-XTS operation length to be aligned to the defined data-unit.
[1]: commit d014dddb2d69 ("cryptodev: support multiple cipher data-units")
Signed-off-by: Matan Azrad
---
.../sample_app_ug/l2_forward_crypto.rst | 7 ++-
examples/l2fwd-crypto/main.c | 61 +++
2 files changed, 66 insertions(+), 2 deletions(-)
diff --git a/doc/guides/sample_app_ug/l2_forward_crypto.rst
b/doc/guides/sample_app_ug/l2_forward_crypto.rst
index e2c0f9f1ec..cc772aaec4 100644
--- a/doc/guides/sample_app_ug/l2_forward_crypto.rst
+++ b/doc/guides/sample_app_ug/l2_forward_crypto.rst
@@ -41,8 +41,9 @@ The application requires a number of command line options:
.//examples/dpdk-l2fwd-crypto [EAL options] -- [-p PORTMASK]
[-q NQ] [-s] [-T PERIOD] /
[--cdev_type HW/SW/ANY] [--chain
HASH_CIPHER/CIPHER_HASH/CIPHER_ONLY/HASH_ONLY/AEAD] /
-[--cipher_algo ALGO] [--cipher_op ENCRYPT/DECRYPT] [--cipher_key KEY] /
-[--cipher_key_random_size SIZE] [--cipher_iv IV] [--cipher_iv_random_size
SIZE] /
+[--cipher_algo ALGO] [--cipher_op ENCRYPT/DECRYPT] [--cipher_dataunit_len
SIZE] /
+[--cipher_key KEY] [--cipher_key_random_size SIZE] [--cipher_iv IV] /
+[--cipher_iv_random_size SIZE] /
[--auth_algo ALGO] [--auth_op GENERATE/VERIFY] [--auth_key KEY] /
[--auth_key_random_size SIZE] [--auth_iv IV] [--auth_iv_random_size SIZE] /
[--aead_algo ALGO] [--aead_op ENCRYPT/DECRYPT] [--aead_key KEY] /
@@ -81,6 +82,8 @@ where,
(default is ENCRYPT)
+* cipher_dataunit_len: set the length of the cipher data-unit.
+
* cipher_key: set the ciphering key to be used. Bytes has to be separated
with ":"
* cipher_key_random_size: set the size of the ciphering key,
diff --git a/examples/l2fwd-crypto/main.c b/examples/l2fwd-crypto/main.c
index 4f51616492..22bd47061b 100644
--- a/examples/l2fwd-crypto/main.c
+++ b/examples/l2fwd-crypto/main.c
@@ -182,6 +182,8 @@ struct l2fwd_crypto_params {
unsigned digest_length;
unsigned block_size;
+ uint16_t cipher_dataunit_len;
+
struct l2fwd_iv cipher_iv;
struct l2fwd_iv auth_iv;
struct l2fwd_iv aead_iv;
@@ -434,6 +436,12 @@ l2fwd_simple_crypto_enqueue(struct rte_mbuf *m,
pad_len = cparams->block_size -
(data_len % cparams->block_size);
break;
+ case RTE_CRYPTO_CIPHER_AES_XTS:
+ if (cparams->cipher_dataunit_len != 0 &&
+ (data_len % cparams->cipher_dataunit_len))
+ pad_len = cparams->cipher_dataunit_len -
+ (data_len %
cparams->cipher_dataunit_len);
+ break;
default:
pad_len = 0;
}
@@ -827,6 +835,8 @@ l2fwd_main_loop(struct l2fwd_crypto_options *options)
port_cparams[i].cipher_iv.length);
port_cparams[i].cipher_algo =
options->cipher_xform.cipher.algo;
+ port_cparams[i].cipher_dataunit_len =
+ options->cipher_xform.cipher.dataunit_len;
/* Set IV parameters */
options->cipher_xform.cipher.iv.offset = IV_OFFSET;
options->cipher_xform.cipher.iv.length =
@@ -989,6 +999,7 @@ l2fwd_crypto_usage(const char *prgname)
" --cipher_key_random_size SIZE: size of cipher key when
generated randomly\n"
" --cipher_iv IV (bytes separated with \":\")\n"
" --cipher_iv_random_size SIZE: size of cipher IV when
generated randomly\n"
+ " --cipher_dataunit_len SIZE: length of the algorithm
data-unit\n"
" --auth_algo ALGO\n"
" --auth_op GENERATE / VERIFY\n"
@@ -1215,6 +1226,7 @@ l2fwd_crypto_parse_args_long_options(struct
l2fwd_crypto_options *options,
struct option *lgopts, int option_index)
{
int retval;
+ int val;
if (strcmp(lgopts[option_index].name, "cdev_type") == 0) {
retval = parse_cryptodev_type(&options->type, optarg);
@@ -1245,6 +1257,16 @@ l2fwd_crypto_parse_args_long_options(struct
l2fwd_crypto_options *options,
return -1;
}
+ else if (strcmp(lgopts[option_index].name, "cipher_dataunit_len") == 0)
{
+ retval = pars
[dpdk-dev] [PATCH v2 01/16] common/mlx5: remove redundant spaces in header file
From: Dekel Peled
File drivers/common/mlx5/mlx5_prm.h includes structs representing
data items as defined in PRM document.
Some of these structs were copied as-is from kernel file mlx5_ifc.h.
As result the structs are not all aligned with the same spacing.
This patch removes redundant spaces and new lines from several structs,
to align all structs in mlx5_prm.h to the same format.
Signed-off-by: Dekel Peled
Acked-by: Matan Azrad
---
drivers/common/mlx5/mlx5_prm.h | 334 +++--
1 file changed, 155 insertions(+), 179 deletions(-)
diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h
index efa5ae67bf..da1510ac1e 100644
--- a/drivers/common/mlx5/mlx5_prm.h
+++ b/drivers/common/mlx5/mlx5_prm.h
@@ -953,163 +953,139 @@ enum {
/* Flow counters. */
struct mlx5_ifc_alloc_flow_counter_out_bits {
- u8 status[0x8];
- u8 reserved_at_8[0x18];
- u8 syndrome[0x20];
- u8 flow_counter_id[0x20];
- u8 reserved_at_60[0x20];
+ u8 status[0x8];
+ u8 reserved_at_8[0x18];
+ u8 syndrome[0x20];
+ u8 flow_counter_id[0x20];
+ u8 reserved_at_60[0x20];
};
struct mlx5_ifc_alloc_flow_counter_in_bits {
- u8 opcode[0x10];
- u8 reserved_at_10[0x10];
- u8 reserved_at_20[0x10];
- u8 op_mod[0x10];
- u8 flow_counter_id[0x20];
- u8 reserved_at_40[0x18];
- u8 flow_counter_bulk[0x8];
+ u8 opcode[0x10];
+ u8 reserved_at_10[0x10];
+ u8 reserved_at_20[0x10];
+ u8 op_mod[0x10];
+ u8 flow_counter_id[0x20];
+ u8 reserved_at_40[0x18];
+ u8 flow_counter_bulk[0x8];
};
struct mlx5_ifc_dealloc_flow_counter_out_bits {
- u8 status[0x8];
- u8 reserved_at_8[0x18];
- u8 syndrome[0x20];
- u8 reserved_at_40[0x40];
+ u8 status[0x8];
+ u8 reserved_at_8[0x18];
+ u8 syndrome[0x20];
+ u8 reserved_at_40[0x40];
};
struct mlx5_ifc_dealloc_flow_counter_in_bits {
- u8 opcode[0x10];
- u8 reserved_at_10[0x10];
- u8 reserved_at_20[0x10];
- u8 op_mod[0x10];
- u8 flow_counter_id[0x20];
- u8 reserved_at_60[0x20];
+ u8 opcode[0x10];
+ u8 reserved_at_10[0x10];
+ u8 reserved_at_20[0x10];
+ u8 op_mod[0x10];
+ u8 flow_counter_id[0x20];
+ u8 reserved_at_60[0x20];
};
struct mlx5_ifc_traffic_counter_bits {
- u8 packets[0x40];
- u8 octets[0x40];
+ u8 packets[0x40];
+ u8 octets[0x40];
};
struct mlx5_ifc_query_flow_counter_out_bits {
- u8 status[0x8];
- u8 reserved_at_8[0x18];
- u8 syndrome[0x20];
- u8 reserved_at_40[0x40];
+ u8 status[0x8];
+ u8 reserved_at_8[0x18];
+ u8 syndrome[0x20];
+ u8 reserved_at_40[0x40];
struct mlx5_ifc_traffic_counter_bits flow_statistics[];
};
struct mlx5_ifc_query_flow_counter_in_bits {
- u8 opcode[0x10];
- u8 reserved_at_10[0x10];
- u8 reserved_at_20[0x10];
- u8 op_mod[0x10];
- u8 reserved_at_40[0x20];
- u8 mkey[0x20];
- u8 address[0x40];
- u8 clear[0x1];
- u8 dump_to_memory[0x1];
- u8 num_of_counters[0x1e];
- u8 flow_counter_id[0x20];
+ u8 opcode[0x10];
+ u8 reserved_at_10[0x10];
+ u8 reserved_at_20[0x10];
+ u8 op_mod[0x10];
+ u8 reserved_at_40[0x20];
+ u8 mkey[0x20];
+ u8 address[0x40];
+ u8 clear[0x1];
+ u8 dump_to_memory[0x1];
+ u8 num_of_counters[0x1e];
+ u8 flow_counter_id[0x20];
};
#define MLX5_MAX_KLM_BYTE_COUNT 0x8000u
#define MLX5_MIN_KLM_FIXED_BUFFER_SIZE 0x1000u
-
struct mlx5_ifc_klm_bits {
- u8 byte_count[0x20];
- u8 mkey[0x20];
- u8 address[0x40];
+ u8 byte_count[0x20];
+ u8 mkey[0x20];
+ u8 address[0x40];
};
struct mlx5_ifc_mkc_bits {
- u8 reserved_at_0[0x1];
- u8 free[0x1];
- u8 reserved_at_2[0x1];
- u8 access_mode_4_2[0x3];
- u8 reserved_at_6[0x7];
- u8 relaxed_ordering_write[0x1];
- u8 reserved_at_e[0x1];
- u8 small_fence_on_rdma_read_response[0x1];
- u8 umr_en[0x1];
- u8 a[0x1];
- u8 rw[0x1];
- u8 rr[0x1];
- u8 lw[0x1];
- u8 lr[0x1];
- u8 access_mode_1_0[0x2];
- u8 reserved_at_18[0x8];
-
- u8 qpn[0x18];
- u8 mkey_7_0[0x8];
-
- u8 reserved_at_40[0x20];
-
- u8 length64[0x1];
- u8 bsf_en[0x1];
- u8 sync_umr[0x1
[dpdk-dev] [PATCH v2 00/16] mlx5 common part for crypto driver
The crypto PMD will be supported on Nvidia ConnectX6 The crypto PMD will add the support of encryption and decryption using the AES-XTS symmetric algorithm. The crypto PMD requires rdma-core and uses mlx5 DevX. This is the mlx5 common part that added support for DevX commands needed for crypto driver. Also includes PRM definitions. v2: Add data-path part. Dekel Peled (12): common/mlx5: remove redundant spaces in header file common/mlx5: update GENEVE TLV OPT obj name common/mlx5: optimize read of general obj type caps common/mlx5: add HCA cap for AES-XTS crypto common/mlx5: support general object DEK create op common/mlx5: adjust DevX MKEY fields for crypto common/mlx5: support general obj IMPORT KEK create common/mlx5: support general obj CRYPTO LOGIN create common/mlx5: add crypto BSF struct and defines common/mlx5: support general obj CREDENTIAL create common/mlx5: add crypto register structs and defs common/mlx5: support register write access Shiri Kuzin (2): common/mlx5: share hash list tool common/mlx5: share get ib device match function Suanming Mou (2): common/mlx5: fix cypto bsf attr common/mlx5: add UMR and RDMA write WQE defines drivers/common/mlx5/linux/mlx5_common_os.c | 30 +- drivers/common/mlx5/linux/mlx5_common_os.h | 5 + drivers/common/mlx5/linux/mlx5_nl.c | 2 +- drivers/common/mlx5/meson.build | 15 +- drivers/common/mlx5/mlx5_common.c| 2 +- drivers/common/mlx5/mlx5_common.h| 2 + drivers/common/mlx5/mlx5_common_devx.c | 2 +- drivers/common/mlx5/mlx5_common_log.h| 21 + drivers/common/mlx5/mlx5_common_mp.c | 2 +- drivers/common/mlx5/mlx5_common_mr.c | 2 +- drivers/common/mlx5/mlx5_common_pci.c| 4 +- drivers/common/mlx5/mlx5_common_utils.c | 221 +++ drivers/common/mlx5/mlx5_common_utils.h | 202 +- drivers/common/mlx5/mlx5_devx_cmds.c | 312 +- drivers/common/mlx5/mlx5_devx_cmds.h | 66 ++ drivers/common/mlx5/mlx5_malloc.c| 2 +- drivers/common/mlx5/mlx5_prm.h | 610 +-- drivers/common/mlx5/version.map | 24 +- drivers/common/mlx5/windows/mlx5_common_os.c | 2 +- drivers/common/mlx5/windows/mlx5_glue.c | 2 +- drivers/compress/mlx5/mlx5_compress.c| 30 +- drivers/net/mlx5/mlx5_utils.c| 209 --- drivers/net/mlx5/mlx5_utils.h| 194 +- drivers/regex/mlx5/mlx5_regex.c | 30 +- drivers/vdpa/mlx5/mlx5_vdpa.c| 32 +- 25 files changed, 1317 insertions(+), 706 deletions(-) create mode 100644 drivers/common/mlx5/mlx5_common_log.h create mode 100644 drivers/common/mlx5/mlx5_common_utils.c -- 2.25.1
[dpdk-dev] [PATCH v2 02/16] common/mlx5: update GENEVE TLV OPT obj name
From: Dekel Peled
Rename MLX5_OBJ_TYPE_GENEVE_TLV_OPT as
MLX5_GENERAL_OBJ_TYPE_GENEVE_TLV_OPT, to align with other general
objects names.
Signed-off-by: Dekel Peled
Acked-by: Matan Azrad
---
drivers/common/mlx5/mlx5_devx_cmds.c | 2 +-
drivers/common/mlx5/mlx5_prm.h | 4 ++--
2 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c
b/drivers/common/mlx5/mlx5_devx_cmds.c
index 79fff6457c..831175efc5 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -2265,7 +2265,7 @@ mlx5_devx_cmd_create_geneve_tlv_option(void *ctx,
MLX5_SET(general_obj_in_cmd_hdr, hdr, opcode,
MLX5_CMD_OP_CREATE_GENERAL_OBJECT);
MLX5_SET(general_obj_in_cmd_hdr, hdr, obj_type,
- MLX5_OBJ_TYPE_GENEVE_TLV_OPT);
+MLX5_GENERAL_OBJ_TYPE_GENEVE_TLV_OPT);
MLX5_SET(geneve_tlv_option, opt, option_class,
rte_be_to_cpu_16(class));
MLX5_SET(geneve_tlv_option, opt, option_type, type);
diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h
index da1510ac1e..2e5e42f6e9 100644
--- a/drivers/common/mlx5/mlx5_prm.h
+++ b/drivers/common/mlx5/mlx5_prm.h
@@ -1108,7 +1108,7 @@ enum {
#define MLX5_GENERAL_OBJ_TYPES_CAP_FLOW_METER_ASO \
(1ULL << MLX5_GENERAL_OBJ_TYPE_FLOW_METER_ASO)
#define MLX5_GENERAL_OBJ_TYPES_CAP_GENEVE_TLV_OPT \
- (1ULL << MLX5_OBJ_TYPE_GENEVE_TLV_OPT)
+ (1ULL << MLX5_GENERAL_OBJ_TYPE_GENEVE_TLV_OPT)
enum {
MLX5_HCA_CAP_OPMOD_GET_MAX = 0,
@@ -2402,7 +2402,7 @@ struct mlx5_ifc_create_cq_in_bits {
};
enum {
- MLX5_OBJ_TYPE_GENEVE_TLV_OPT = 0x000b,
+ MLX5_GENERAL_OBJ_TYPE_GENEVE_TLV_OPT = 0x000b,
MLX5_GENERAL_OBJ_TYPE_VIRTQ = 0x000d,
MLX5_GENERAL_OBJ_TYPE_VIRTIO_Q_COUNTERS = 0x001c,
MLX5_GENERAL_OBJ_TYPE_FLEX_PARSE_GRAPH = 0x0022,
--
2.25.1
[dpdk-dev] [PATCH v2 03/16] common/mlx5: optimize read of general obj type caps
From: Dekel Peled
General object types support is indicated in bitmap general_obj_types,
which is part of HCA capabilities list.
Currently this bitmap is read multiple times, and each time a different
bit is extracted.
This patch optimizes the code, reading the bitmap once into a local
variable, and then extracting the required bits.
Signed-off-by: Dekel Peled
Acked-by: Matan Azrad
---
drivers/common/mlx5/mlx5_devx_cmds.c | 21 -
1 file changed, 16 insertions(+), 5 deletions(-)
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c
b/drivers/common/mlx5/mlx5_devx_cmds.c
index 831175efc5..a0bf0d3009 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -647,6 +647,7 @@ mlx5_devx_cmd_query_hca_attr(void *ctx,
uint32_t out[MLX5_ST_SZ_DW(query_hca_cap_out)] = {0};
void *hcattr;
int status, syndrome, rc, i;
+ uint64_t general_obj_types_supported = 0;
MLX5_SET(query_hca_cap_in, in, opcode, MLX5_CMD_OP_QUERY_HCA_CAP);
MLX5_SET(query_hca_cap_in, in, op_mod,
@@ -725,12 +726,22 @@ mlx5_devx_cmd_query_hca_attr(void *ctx,
attr->regex = MLX5_GET(cmd_hca_cap, hcattr, regexp);
attr->regexp_num_of_engines = MLX5_GET(cmd_hca_cap, hcattr,
regexp_num_of_engines);
- attr->flow_hit_aso = !!(MLX5_GET64(cmd_hca_cap, hcattr,
- general_obj_types) &
+ /* Read the general_obj_types bitmap and extract the relevant bits. */
+ general_obj_types_supported = MLX5_GET64(cmd_hca_cap, hcattr,
+general_obj_types);
+ attr->vdpa.valid = !!(general_obj_types_supported &
+ MLX5_GENERAL_OBJ_TYPES_CAP_VIRTQ_NET_Q);
+ attr->vdpa.queue_counters_valid =
+ !!(general_obj_types_supported &
+ MLX5_GENERAL_OBJ_TYPES_CAP_VIRTIO_Q_COUNTERS);
+ attr->parse_graph_flex_node =
+ !!(general_obj_types_supported &
+ MLX5_GENERAL_OBJ_TYPES_CAP_PARSE_GRAPH_FLEX_NODE);
+ attr->flow_hit_aso = !!(general_obj_types_supported &
MLX5_GENERAL_OBJ_TYPES_CAP_FLOW_HIT_ASO);
- attr->geneve_tlv_opt = !!(MLX5_GET64(cmd_hca_cap, hcattr,
- general_obj_types) &
- MLX5_GENERAL_OBJ_TYPES_CAP_GENEVE_TLV_OPT);
+ attr->geneve_tlv_opt = !!(general_obj_types_supported &
+ MLX5_GENERAL_OBJ_TYPES_CAP_GENEVE_TLV_OPT);
+ /* Add reading of other GENERAL_OBJ_TYPES_CAP bits above this line. */
attr->log_max_cq = MLX5_GET(cmd_hca_cap, hcattr, log_max_cq);
attr->log_max_qp = MLX5_GET(cmd_hca_cap, hcattr, log_max_qp);
attr->log_max_cq_sz = MLX5_GET(cmd_hca_cap, hcattr, log_max_cq_sz);
--
2.25.1
[dpdk-dev] [PATCH v2 04/16] common/mlx5: add HCA cap for AES-XTS crypto
From: Dekel Peled
Update the PRM structure and HCA capabilities reading, to include
relevant capabilities for AES-XTS crypto.
Signed-off-by: Dekel Peled
Acked-by: Matan Azrad
---
drivers/common/mlx5/mlx5_devx_cmds.c | 3 +++
drivers/common/mlx5/mlx5_devx_cmds.h | 2 ++
drivers/common/mlx5/mlx5_prm.h | 5 -
3 files changed, 9 insertions(+), 1 deletion(-)
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c
b/drivers/common/mlx5/mlx5_devx_cmds.c
index a0bf0d3009..7ca767944e 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -771,6 +771,9 @@ mlx5_devx_cmd_query_hca_attr(void *ctx,
MLX5_GET(cmd_hca_cap, hcattr, umr_indirect_mkey_disabled);
attr->umr_modify_entity_size_disabled =
MLX5_GET(cmd_hca_cap, hcattr, umr_modify_entity_size_disabled);
+ attr->crypto = MLX5_GET(cmd_hca_cap, hcattr, crypto);
+ if (attr->crypto)
+ attr->aes_xts = MLX5_GET(cmd_hca_cap, hcattr, aes_xts);
if (attr->qos.sup) {
MLX5_SET(query_hca_cap_in, in, op_mod,
MLX5_GET_HCA_CAP_OP_MOD_QOS_CAP |
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h
b/drivers/common/mlx5/mlx5_devx_cmds.h
index 870bdb6b30..28ade5bbc4 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.h
+++ b/drivers/common/mlx5/mlx5_devx_cmds.h
@@ -137,6 +137,8 @@ struct mlx5_hca_attr {
uint32_t qp_ts_format:2;
uint32_t regex:1;
uint32_t reg_c_preserve:1;
+ uint32_t crypto:1; /* Crypto engine is supported. */
+ uint32_t aes_xts:1; /* AES-XTS crypto is supported. */
uint32_t regexp_num_of_engines;
uint32_t log_max_ft_sampler_num:8;
uint32_t geneve_tlv_opt;
diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h
index 2e5e42f6e9..a8fbfbb0f5 100644
--- a/drivers/common/mlx5/mlx5_prm.h
+++ b/drivers/common/mlx5/mlx5_prm.h
@@ -1427,7 +1427,10 @@ struct mlx5_ifc_cmd_hca_cap_bits {
u8 sq_ts_format[0x2];
u8 rq_ts_format[0x2];
u8 reserved_at_444[0x1C];
- u8 reserved_at_460[0x10];
+ u8 reserved_at_460[0x8];
+ u8 aes_xts[0x1];
+ u8 crypto[0x1];
+ u8 reserved_at_46a[0x6];
u8 max_num_eqs[0x10];
u8 reserved_at_480[0x3];
u8 log_max_l2_table[0x5];
--
2.25.1
[dpdk-dev] [PATCH v2 06/16] common/mlx5: adjust DevX MKEY fields for crypto
From: Dekel Peled
MKEY that will be used for crypto purposes must be created with
crypto_en and remote access attributes.
This patch adds support for them in the DevX MKEY context.
Signed-off-by: Dekel Peled
Acked-by: Matan Azrad
---
drivers/common/mlx5/mlx5_devx_cmds.c | 5 +
drivers/common/mlx5/mlx5_devx_cmds.h | 2 ++
drivers/common/mlx5/mlx5_prm.h | 10 +-
3 files changed, 16 insertions(+), 1 deletion(-)
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c
b/drivers/common/mlx5/mlx5_devx_cmds.c
index 742c82cca4..68a10b149a 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -263,6 +263,10 @@ mlx5_devx_cmd_mkey_create(void *ctx,
MLX5_SET(create_mkey_in, in, pg_access, attr->pg_access);
MLX5_SET(mkc, mkc, lw, 0x1);
MLX5_SET(mkc, mkc, lr, 0x1);
+ if (attr->set_remote_rw) {
+ MLX5_SET(mkc, mkc, rw, 0x1);
+ MLX5_SET(mkc, mkc, rr, 0x1);
+ }
MLX5_SET(mkc, mkc, qpn, 0xff);
MLX5_SET(mkc, mkc, pd, attr->pd);
MLX5_SET(mkc, mkc, mkey_7_0, attr->umem_id & 0xFF);
@@ -273,6 +277,7 @@ mlx5_devx_cmd_mkey_create(void *ctx,
MLX5_SET(mkc, mkc, relaxed_ordering_read, attr->relaxed_ordering_read);
MLX5_SET64(mkc, mkc, start_addr, attr->addr);
MLX5_SET64(mkc, mkc, len, attr->size);
+ MLX5_SET(mkc, mkc, crypto_en, attr->crypto_en);
mkey->obj = mlx5_glue->devx_obj_create(ctx, in, in_size_dw * 4, out,
sizeof(out));
if (!mkey->obj) {
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h
b/drivers/common/mlx5/mlx5_devx_cmds.h
index b9ff7ab87d..600577f18a 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.h
+++ b/drivers/common/mlx5/mlx5_devx_cmds.h
@@ -32,6 +32,8 @@ struct mlx5_devx_mkey_attr {
uint32_t relaxed_ordering_write:1;
uint32_t relaxed_ordering_read:1;
uint32_t umr_en:1;
+ uint32_t crypto_en:2;
+ uint32_t set_remote_rw:1;
struct mlx5_klm *klm_array;
int klm_num;
};
diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h
index bc9f58ad03..25f6f8ff00 100644
--- a/drivers/common/mlx5/mlx5_prm.h
+++ b/drivers/common/mlx5/mlx5_prm.h
@@ -1059,7 +1059,15 @@ struct mlx5_ifc_mkc_bits {
u8 relaxed_ordering_read[0x1];
u8 reserved_at_1da[0x1];
u8 log_page_size[0x5];
- u8 reserved_at_1e0[0x20];
+ u8 reserved_at_1e0[0x3];
+ u8 crypto_en[0x2];
+ u8 reserved_at_1e5[0x1b];
+};
+
+/* Range of values for MKEY context crypto_en field. */
+enum {
+ MLX5_MKEY_CRYPTO_DISABLED = 0x0,
+ MLX5_MKEY_CRYPTO_ENABLED = 0x1,
};
struct mlx5_ifc_create_mkey_out_bits {
--
2.25.1
[dpdk-dev] [PATCH v2 05/16] common/mlx5: support general object DEK create op
From: Dekel Peled
Data Encryption Keys (DEKs) are the keys used for data
encryption/decryption operations.
Add reading of DEK support capability.
Add function to create general object type DEK, using DevX API.
Signed-off-by: Dekel Peled
Acked-by: Matan Azrad
---
drivers/common/mlx5/mlx5_devx_cmds.c | 53
drivers/common/mlx5/mlx5_devx_cmds.h | 17 +
drivers/common/mlx5/mlx5_prm.h | 39
drivers/common/mlx5/version.map | 1 +
4 files changed, 110 insertions(+)
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c
b/drivers/common/mlx5/mlx5_devx_cmds.c
index 7ca767944e..742c82cca4 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -741,6 +741,8 @@ mlx5_devx_cmd_query_hca_attr(void *ctx,
MLX5_GENERAL_OBJ_TYPES_CAP_FLOW_HIT_ASO);
attr->geneve_tlv_opt = !!(general_obj_types_supported &
MLX5_GENERAL_OBJ_TYPES_CAP_GENEVE_TLV_OPT);
+ attr->dek = !!(general_obj_types_supported &
+ MLX5_GENERAL_OBJ_TYPES_CAP_DEK);
/* Add reading of other GENERAL_OBJ_TYPES_CAP bits above this line. */
attr->log_max_cq = MLX5_GET(cmd_hca_cap, hcattr, log_max_cq);
attr->log_max_qp = MLX5_GET(cmd_hca_cap, hcattr, log_max_qp);
@@ -2397,3 +2399,54 @@ mlx5_devx_cmd_queue_counter_query(struct mlx5_devx_obj
*dcs, int clear,
*out_of_buffers = MLX5_GET(query_q_counter_out, out, out_of_buffer);
return 0;
}
+
+/**
+ * Create general object of type DEK using DevX API.
+ *
+ * @param[in] ctx
+ * Context returned from mlx5 open_device() glue function.
+ * @param [in] attr
+ * Pointer to DEK attributes structure.
+ *
+ * @return
+ * The DevX object created, NULL otherwise and rte_errno is set.
+ */
+struct mlx5_devx_obj *
+mlx5_devx_cmd_create_dek_obj(void *ctx, struct mlx5_devx_dek_attr *attr)
+{
+ uint32_t in[MLX5_ST_SZ_DW(create_dek_in)] = {0};
+ uint32_t out[MLX5_ST_SZ_DW(general_obj_out_cmd_hdr)] = {0};
+ struct mlx5_devx_obj *dek_obj = NULL;
+ void *ptr = NULL, *key_addr = NULL;
+
+ dek_obj = mlx5_malloc(MLX5_MEM_ZERO, sizeof(*dek_obj),
+ 0, SOCKET_ID_ANY);
+ if (dek_obj == NULL) {
+ DRV_LOG(ERR, "Failed to allocate DEK object data");
+ rte_errno = ENOMEM;
+ return NULL;
+ }
+ ptr = MLX5_ADDR_OF(create_dek_in, in, hdr);
+ MLX5_SET(general_obj_in_cmd_hdr, ptr, opcode,
+MLX5_CMD_OP_CREATE_GENERAL_OBJECT);
+ MLX5_SET(general_obj_in_cmd_hdr, ptr, obj_type,
+MLX5_GENERAL_OBJ_TYPE_DEK);
+ ptr = MLX5_ADDR_OF(create_dek_in, in, dek);
+ MLX5_SET(dek, ptr, key_size, attr->key_size);
+ MLX5_SET(dek, ptr, has_keytag, attr->has_keytag);
+ MLX5_SET(dek, ptr, key_purpose, attr->key_purpose);
+ MLX5_SET(dek, ptr, pd, attr->pd);
+ MLX5_SET64(dek, ptr, opaque, attr->opaque);
+ key_addr = MLX5_ADDR_OF(dek, ptr, key);
+ memcpy(key_addr, (void *)(attr->key), MLX5_CRYPTO_KEY_MAX_SIZE);
+ dek_obj->obj = mlx5_glue->devx_obj_create(ctx, in, sizeof(in),
+ out, sizeof(out));
+ if (dek_obj->obj == NULL) {
+ rte_errno = errno;
+ DRV_LOG(ERR, "Failed to create DEK obj using DevX.");
+ mlx5_free(dek_obj);
+ return NULL;
+ }
+ dek_obj->id = MLX5_GET(general_obj_out_cmd_hdr, out, obj_id);
+ return dek_obj;
+}
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h
b/drivers/common/mlx5/mlx5_devx_cmds.h
index 28ade5bbc4..b9ff7ab87d 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.h
+++ b/drivers/common/mlx5/mlx5_devx_cmds.h
@@ -139,6 +139,7 @@ struct mlx5_hca_attr {
uint32_t reg_c_preserve:1;
uint32_t crypto:1; /* Crypto engine is supported. */
uint32_t aes_xts:1; /* AES-XTS crypto is supported. */
+ uint32_t dek:1; /* General obj type DEK is supported. */
uint32_t regexp_num_of_engines;
uint32_t log_max_ft_sampler_num:8;
uint32_t geneve_tlv_opt;
@@ -435,6 +436,18 @@ struct mlx5_devx_graph_node_attr {
struct mlx5_devx_graph_arc_attr out[MLX5_GRAPH_NODE_ARC_NUM];
};
+/* Encryption key size is up to 1024 bit, 128 bytes. */
+#define MLX5_CRYPTO_KEY_MAX_SIZE 128
+
+struct mlx5_devx_dek_attr {
+ uint32_t key_size:4;
+ uint32_t has_keytag:1;
+ uint32_t key_purpose:4;
+ uint32_t pd:24;
+ uint64_t opaque;
+ uint8_t key[MLX5_CRYPTO_KEY_MAX_SIZE];
+};
+
/* mlx5_devx_cmds.c */
__rte_internal
@@ -587,4 +600,8 @@ int mlx5_devx_cmd_queue_counter_query(struct mlx5_devx_obj
*dcs, int clear,
__rte_internal
struct mlx5_devx_obj *mlx5_devx_cmd_create_flow_meter_aso_obj(void *ctx,
[dpdk-dev] [PATCH v2 07/16] common/mlx5: fix cypto bsf attr
From: Suanming Mou
This commit should be squash to crypto en commit.
Signed-off-by: Suanming Mou
---
drivers/common/mlx5/mlx5_devx_cmds.c | 4
1 file changed, 4 insertions(+)
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c
b/drivers/common/mlx5/mlx5_devx_cmds.c
index 68a10b149a..7e3c8b55fa 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -278,6 +278,10 @@ mlx5_devx_cmd_mkey_create(void *ctx,
MLX5_SET64(mkc, mkc, start_addr, attr->addr);
MLX5_SET64(mkc, mkc, len, attr->size);
MLX5_SET(mkc, mkc, crypto_en, attr->crypto_en);
+ if (attr->crypto_en) {
+ MLX5_SET(mkc, mkc, bsf_en, attr->crypto_en);
+ MLX5_SET(mkc, mkc, bsf_octword_size, 4);
+ }
mkey->obj = mlx5_glue->devx_obj_create(ctx, in, in_size_dw * 4, out,
sizeof(out));
if (!mkey->obj) {
--
2.25.1
[dpdk-dev] [PATCH v2 08/16] common/mlx5: support general obj IMPORT KEK create
From: Dekel Peled
IMPORT_KEK object is used to wrap (encrypt) critical security
parameters, such as other keys and credentials, when those need
to be passed between the device and the software.
This patch add support of IMPORT_KEK object create operation.
Add reading of IMPORT_KEK support capability.
Add function to create general object type IMPORT_KEK, using DevX API.
Signed-off-by: Dekel Peled
Acked-by: Matan Azrad
---
drivers/common/mlx5/mlx5_devx_cmds.c | 50
drivers/common/mlx5/mlx5_devx_cmds.h | 13
drivers/common/mlx5/mlx5_prm.h | 18 ++
drivers/common/mlx5/version.map | 1 +
4 files changed, 82 insertions(+)
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c
b/drivers/common/mlx5/mlx5_devx_cmds.c
index 7e3c8b55fa..afef7a5f63 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -752,6 +752,8 @@ mlx5_devx_cmd_query_hca_attr(void *ctx,
MLX5_GENERAL_OBJ_TYPES_CAP_GENEVE_TLV_OPT);
attr->dek = !!(general_obj_types_supported &
MLX5_GENERAL_OBJ_TYPES_CAP_DEK);
+ attr->import_kek = !!(general_obj_types_supported &
+ MLX5_GENERAL_OBJ_TYPES_CAP_IMPORT_KEK);
/* Add reading of other GENERAL_OBJ_TYPES_CAP bits above this line. */
attr->log_max_cq = MLX5_GET(cmd_hca_cap, hcattr, log_max_cq);
attr->log_max_qp = MLX5_GET(cmd_hca_cap, hcattr, log_max_qp);
@@ -2459,3 +2461,51 @@ mlx5_devx_cmd_create_dek_obj(void *ctx, struct
mlx5_devx_dek_attr *attr)
dek_obj->id = MLX5_GET(general_obj_out_cmd_hdr, out, obj_id);
return dek_obj;
}
+
+/**
+ * Create general object of type IMPORT_KEK using DevX API.
+ *
+ * @param[in] ctx
+ * Context returned from mlx5 open_device() glue function.
+ * @param [in] attr
+ * Pointer to IMPORT_KEK attributes structure.
+ *
+ * @return
+ * The DevX object created, NULL otherwise and rte_errno is set.
+ */
+struct mlx5_devx_obj *
+mlx5_devx_cmd_create_import_kek_obj(void *ctx,
+ struct mlx5_devx_import_kek_attr *attr)
+{
+ uint32_t in[MLX5_ST_SZ_DW(create_import_kek_in)] = {0};
+ uint32_t out[MLX5_ST_SZ_DW(general_obj_out_cmd_hdr)] = {0};
+ struct mlx5_devx_obj *import_kek_obj = NULL;
+ void *ptr = NULL, *key_addr = NULL;
+
+ import_kek_obj = mlx5_malloc(MLX5_MEM_ZERO, sizeof(*import_kek_obj),
+0, SOCKET_ID_ANY);
+ if (import_kek_obj == NULL) {
+ DRV_LOG(ERR, "Failed to allocate IMPORT_KEK object data");
+ rte_errno = ENOMEM;
+ return NULL;
+ }
+ ptr = MLX5_ADDR_OF(create_import_kek_in, in, hdr);
+ MLX5_SET(general_obj_in_cmd_hdr, ptr, opcode,
+MLX5_CMD_OP_CREATE_GENERAL_OBJECT);
+ MLX5_SET(general_obj_in_cmd_hdr, ptr, obj_type,
+MLX5_GENERAL_OBJ_TYPE_IMPORT_KEK);
+ ptr = MLX5_ADDR_OF(create_import_kek_in, in, import_kek);
+ MLX5_SET(import_kek, ptr, key_size, attr->key_size);
+ key_addr = MLX5_ADDR_OF(import_kek, ptr, key);
+ memcpy(key_addr, (void *)(attr->key), MLX5_CRYPTO_KEY_MAX_SIZE);
+ import_kek_obj->obj = mlx5_glue->devx_obj_create(ctx, in, sizeof(in),
+out, sizeof(out));
+ if (import_kek_obj->obj == NULL) {
+ rte_errno = errno;
+ DRV_LOG(ERR, "Failed to create IMPORT_KEK object using DevX.");
+ mlx5_free(import_kek_obj);
+ return NULL;
+ }
+ import_kek_obj->id = MLX5_GET(general_obj_out_cmd_hdr, out, obj_id);
+ return import_kek_obj;
+}
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h
b/drivers/common/mlx5/mlx5_devx_cmds.h
index 600577f18a..6423610dae 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.h
+++ b/drivers/common/mlx5/mlx5_devx_cmds.h
@@ -142,6 +142,7 @@ struct mlx5_hca_attr {
uint32_t crypto:1; /* Crypto engine is supported. */
uint32_t aes_xts:1; /* AES-XTS crypto is supported. */
uint32_t dek:1; /* General obj type DEK is supported. */
+ uint32_t import_kek:1; /* General obj type IMPORT_KEK supported. */
uint32_t regexp_num_of_engines;
uint32_t log_max_ft_sampler_num:8;
uint32_t geneve_tlv_opt;
@@ -450,6 +451,13 @@ struct mlx5_devx_dek_attr {
uint8_t key[MLX5_CRYPTO_KEY_MAX_SIZE];
};
+struct mlx5_devx_import_kek_attr {
+ uint64_t modify_field_select;
+ uint32_t state:8;
+ uint32_t key_size:4;
+ uint8_t key[MLX5_CRYPTO_KEY_MAX_SIZE];
+};
+
/* mlx5_devx_cmds.c */
__rte_internal
@@ -606,4 +614,9 @@ __rte_internal
struct mlx5_devx_obj *
mlx5_devx_cmd_create_dek_obj(void *ctx, struct mlx5_devx_dek_attr *attr);
+__rte_internal
+struct mlx5_devx_obj *
+mlx5_devx_cmd_create_import_kek_obj(void *ctx,
+
[dpdk-dev] [PATCH v2 09/16] common/mlx5: support general obj CRYPTO LOGIN create
From: Dekel Peled
CRYPTO_LOGIN Object is used to login to the device as crypto user
or crypto officer.
Required in order to perform any crypto related control operations.
This patch adds support of CRYPTO_LOGIN object create operation.
Add reading of CRYPTO_LOGIN support capability.
Add function to create general object type CRYPTO_LOGIN, using DevX API.
Signed-off-by: Dekel Peled
Acked-by: Matan Azrad
---
drivers/common/mlx5/mlx5_devx_cmds.c | 54
drivers/common/mlx5/mlx5_devx_cmds.h | 15
drivers/common/mlx5/mlx5_prm.h | 19 ++
drivers/common/mlx5/version.map | 1 +
4 files changed, 89 insertions(+)
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c
b/drivers/common/mlx5/mlx5_devx_cmds.c
index afef7a5f63..5e082ebb78 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -754,6 +754,8 @@ mlx5_devx_cmd_query_hca_attr(void *ctx,
MLX5_GENERAL_OBJ_TYPES_CAP_DEK);
attr->import_kek = !!(general_obj_types_supported &
MLX5_GENERAL_OBJ_TYPES_CAP_IMPORT_KEK);
+ attr->crypto_login = !!(general_obj_types_supported &
+ MLX5_GENERAL_OBJ_TYPES_CAP_CRYPTO_LOGIN);
/* Add reading of other GENERAL_OBJ_TYPES_CAP bits above this line. */
attr->log_max_cq = MLX5_GET(cmd_hca_cap, hcattr, log_max_cq);
attr->log_max_qp = MLX5_GET(cmd_hca_cap, hcattr, log_max_qp);
@@ -2509,3 +2511,55 @@ mlx5_devx_cmd_create_import_kek_obj(void *ctx,
import_kek_obj->id = MLX5_GET(general_obj_out_cmd_hdr, out, obj_id);
return import_kek_obj;
}
+
+/**
+ * Create general object of type CRYPTO_LOGIN using DevX API.
+ *
+ * @param[in] ctx
+ * Context returned from mlx5 open_device() glue function.
+ * @param [in] attr
+ * Pointer to CRYPTO_LOGIN attributes structure.
+ *
+ * @return
+ * The DevX object created, NULL otherwise and rte_errno is set.
+ */
+struct mlx5_devx_obj *
+mlx5_devx_cmd_create_crypto_login_obj(void *ctx,
+ struct mlx5_devx_crypto_login_attr *attr)
+{
+ uint32_t in[MLX5_ST_SZ_DW(create_crypto_login_in)] = {0};
+ uint32_t out[MLX5_ST_SZ_DW(general_obj_out_cmd_hdr)] = {0};
+ struct mlx5_devx_obj *crypto_login_obj = NULL;
+ void *ptr = NULL, *credential_addr = NULL;
+
+ crypto_login_obj = mlx5_malloc(MLX5_MEM_ZERO, sizeof(*crypto_login_obj),
+ 0, SOCKET_ID_ANY);
+ if (crypto_login_obj == NULL) {
+ DRV_LOG(ERR, "Failed to allocate CRYPTO_LOGIN object data");
+ rte_errno = ENOMEM;
+ return NULL;
+ }
+ ptr = MLX5_ADDR_OF(create_crypto_login_in, in, hdr);
+ MLX5_SET(general_obj_in_cmd_hdr, ptr, opcode,
+MLX5_CMD_OP_CREATE_GENERAL_OBJECT);
+ MLX5_SET(general_obj_in_cmd_hdr, ptr, obj_type,
+MLX5_GENERAL_OBJ_TYPE_CRYPTO_LOGIN);
+ ptr = MLX5_ADDR_OF(create_crypto_login_in, in, crypto_login);
+ MLX5_SET(crypto_login, ptr, credential_pointer,
+attr->credential_pointer);
+ MLX5_SET(crypto_login, ptr, session_import_kek_ptr,
+attr->session_import_kek_ptr);
+ credential_addr = MLX5_ADDR_OF(crypto_login, ptr, credential);
+ memcpy(credential_addr, (void *)(attr->credential),
+ MLX5_CRYPTO_LOGIN_CREDENTIAL_SIZE);
+ crypto_login_obj->obj = mlx5_glue->devx_obj_create(ctx, in, sizeof(in),
+ out, sizeof(out));
+ if (crypto_login_obj->obj == NULL) {
+ rte_errno = errno;
+ DRV_LOG(ERR, "Failed to create CRYPTO_LOGIN obj using DevX.");
+ mlx5_free(crypto_login_obj);
+ return NULL;
+ }
+ crypto_login_obj->id = MLX5_GET(general_obj_out_cmd_hdr, out, obj_id);
+ return crypto_login_obj;
+}
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h
b/drivers/common/mlx5/mlx5_devx_cmds.h
index 6423610dae..709e28bfba 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.h
+++ b/drivers/common/mlx5/mlx5_devx_cmds.h
@@ -143,6 +143,7 @@ struct mlx5_hca_attr {
uint32_t aes_xts:1; /* AES-XTS crypto is supported. */
uint32_t dek:1; /* General obj type DEK is supported. */
uint32_t import_kek:1; /* General obj type IMPORT_KEK supported. */
+ uint32_t crypto_login:1; /* General obj type CRYPTO_LOGIN supported. */
uint32_t regexp_num_of_engines;
uint32_t log_max_ft_sampler_num:8;
uint32_t geneve_tlv_opt;
@@ -458,6 +459,15 @@ struct mlx5_devx_import_kek_attr {
uint8_t key[MLX5_CRYPTO_KEY_MAX_SIZE];
};
+#define MLX5_CRYPTO_LOGIN_CREDENTIAL_SIZE 48
+
+struct mlx5_devx_crypto_login_attr {
+ uint64_t modify_field_select;
+ uint32_t credential_pointer:24;
+
[dpdk-dev] [PATCH v2 10/16] common/mlx5: add crypto BSF struct and defines
From: Dekel Peled
This patch adds the struct defining crypto BSF segment of UMR WQE,
and the related value definitions and offsets.
Signed-off-by: Dekel Peled
Acked-by: Matan Azrad
---
drivers/common/mlx5/mlx5_prm.h | 66 ++
1 file changed, 66 insertions(+)
diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h
index a2437faec0..a9dcbfa63c 100644
--- a/drivers/common/mlx5/mlx5_prm.h
+++ b/drivers/common/mlx5/mlx5_prm.h
@@ -1096,6 +1096,72 @@ struct mlx5_ifc_create_mkey_in_bits {
u8 klm_pas_mtt[][0x20];
};
+enum {
+ MLX5_BSF_SIZE_16B = 0x0,
+ MLX5_BSF_SIZE_32B = 0x1,
+ MLX5_BSF_SIZE_64B = 0x2,
+ MLX5_BSF_SIZE_128B = 0x3,
+};
+
+enum {
+ MLX5_BSF_P_TYPE_SIGNATURE = 0x0,
+ MLX5_BSF_P_TYPE_CRYPTO = 0x1,
+};
+
+enum {
+ MLX5_ENCRYPTION_ORDER_ENCRYPTED_WIRE_SIGNATURE = 0x0,
+ MLX5_ENCRYPTION_ORDER_ENCRYPTED_MEMORY_SIGNATURE = 0x1,
+ MLX5_ENCRYPTION_ORDER_ENCRYPTED_RAW_WIRE = 0x2,
+ MLX5_ENCRYPTION_ORDER_ENCRYPTED_RAW_MEMORY = 0x3,
+};
+
+enum {
+ MLX5_ENCRYPTION_STANDARD_AES_XTS = 0x0,
+};
+
+enum {
+ MLX5_BLOCK_SIZE_512B= 0x1,
+ MLX5_BLOCK_SIZE_520B= 0x2,
+ MLX5_BLOCK_SIZE_4096B = 0x3,
+ MLX5_BLOCK_SIZE_4160B = 0x4,
+ MLX5_BLOCK_SIZE_1MB = 0x5,
+ MLX5_BLOCK_SIZE_4048B = 0x6,
+};
+
+#define MLX5_BSF_SIZE_OFFSET 30
+#define MLX5_BSF_P_TYPE_OFFSET 24
+#define MLX5_ENCRYPTION_ORDER_OFFSET 16
+#define MLX5_BLOCK_SIZE_OFFSET 24
+
+struct mlx5_wqe_umr_bsf_seg {
+ /*
+* bs_bpt_eo_es contains:
+* bs bsf_size2 bits at MLX5_BSF_SIZE_OFFSET
+* bpt bsf_p_type 2 bits at MLX5_BSF_P_TYPE_OFFSET
+* eo encryption_order4 bits at MLX5_ENCRYPTION_ORDER_OFFSET
+* es encryption_standard 4 bits at offset 0
+*/
+ uint32_t bs_bpt_eo_es;
+ uint32_t raw_data_size;
+ /*
+* bsp_res contains:
+* bsp crypto_block_size_pointer 8 bits at MLX5_BLOCK_SIZE_OFFSET
+* res reserved 24 bits
+*/
+ uint32_t bsp_res;
+ uint32_t reserved0;
+ uint8_t xts_initial_tweak[16];
+ /*
+* res_dp contains:
+* res reserved 8 bits
+* dp dek_pointer 24 bits at offset 0
+*/
+ uint32_t res_dp;
+ uint32_t reserved1;
+ uint64_t keytag;
+ uint32_t reserved2[4];
+} __rte_packed;
+
enum {
MLX5_GET_HCA_CAP_OP_MOD_GENERAL_DEVICE = 0x0 << 1,
MLX5_GET_HCA_CAP_OP_MOD_ETHERNET_OFFLOAD_CAPS = 0x1 << 1,
--
2.25.1
[dpdk-dev] [PATCH v2 11/16] common/mlx5: share hash list tool
From: Shiri Kuzin In order to use the hash list defined in net in other drivers, the hash list is moved to common utilities. In addition, the log definition was moved from the common utilities to a dedicated new log file in common in order to prevent a conflict. Signed-off-by: Shiri Kuzin Acked-by: Matan Azrad --- drivers/common/mlx5/linux/mlx5_common_os.c | 2 +- drivers/common/mlx5/linux/mlx5_nl.c | 2 +- drivers/common/mlx5/meson.build | 15 +- drivers/common/mlx5/mlx5_common.c| 2 +- drivers/common/mlx5/mlx5_common_devx.c | 2 +- drivers/common/mlx5/mlx5_common_log.h| 21 ++ drivers/common/mlx5/mlx5_common_mp.c | 2 +- drivers/common/mlx5/mlx5_common_mr.c | 2 +- drivers/common/mlx5/mlx5_common_pci.c| 4 +- drivers/common/mlx5/mlx5_common_utils.c | 221 +++ drivers/common/mlx5/mlx5_common_utils.h | 202 - drivers/common/mlx5/mlx5_devx_cmds.c | 2 +- drivers/common/mlx5/mlx5_malloc.c| 2 +- drivers/common/mlx5/version.map | 6 + drivers/common/mlx5/windows/mlx5_common_os.c | 2 +- drivers/common/mlx5/windows/mlx5_glue.c | 2 +- drivers/net/mlx5/mlx5_utils.c| 209 -- drivers/net/mlx5/mlx5_utils.h| 194 +--- 18 files changed, 465 insertions(+), 427 deletions(-) create mode 100644 drivers/common/mlx5/mlx5_common_log.h create mode 100644 drivers/common/mlx5/mlx5_common_utils.c diff --git a/drivers/common/mlx5/linux/mlx5_common_os.c b/drivers/common/mlx5/linux/mlx5_common_os.c index 5cf9576921..fba8245b8b 100644 --- a/drivers/common/mlx5/linux/mlx5_common_os.c +++ b/drivers/common/mlx5/linux/mlx5_common_os.c @@ -15,7 +15,7 @@ #include #include "mlx5_common.h" -#include "mlx5_common_utils.h" +#include "mlx5_common_log.h" #include "mlx5_glue.h" #ifdef MLX5_GLUE diff --git a/drivers/common/mlx5/linux/mlx5_nl.c b/drivers/common/mlx5/linux/mlx5_nl.c index 752c57b33d..f0d04f9473 100644 --- a/drivers/common/mlx5/linux/mlx5_nl.c +++ b/drivers/common/mlx5/linux/mlx5_nl.c @@ -20,7 +20,7 @@ #include #include "mlx5_nl.h" -#include "mlx5_common_utils.h" +#include "../mlx5_common_log.h" #include "mlx5_malloc.h" #ifdef HAVE_DEVLINK #include diff --git a/drivers/common/mlx5/meson.build b/drivers/common/mlx5/meson.build index b5585d2cc3..cd371fe6e1 100644 --- a/drivers/common/mlx5/meson.build +++ b/drivers/common/mlx5/meson.build @@ -9,13 +9,14 @@ endif deps += ['hash', 'pci', 'bus_pci', 'net', 'eal', 'kvargs'] sources += files( -'mlx5_devx_cmds.c', -'mlx5_common.c', -'mlx5_common_mp.c', -'mlx5_common_mr.c', -'mlx5_malloc.c', -'mlx5_common_pci.c', -'mlx5_common_devx.c', + 'mlx5_devx_cmds.c', + 'mlx5_common.c', + 'mlx5_common_mp.c', + 'mlx5_common_mr.c', + 'mlx5_malloc.c', + 'mlx5_common_pci.c', + 'mlx5_common_devx.c', + 'mlx5_common_utils.c', ) cflags_options = [ diff --git a/drivers/common/mlx5/mlx5_common.c b/drivers/common/mlx5/mlx5_common.c index f92f05bda5..d397459a3d 100644 --- a/drivers/common/mlx5/mlx5_common.c +++ b/drivers/common/mlx5/mlx5_common.c @@ -11,7 +11,7 @@ #include "mlx5_common.h" #include "mlx5_common_os.h" -#include "mlx5_common_utils.h" +#include "mlx5_common_log.h" #include "mlx5_common_pci.h" uint8_t haswell_broadwell_cpu; diff --git a/drivers/common/mlx5/mlx5_common_devx.c b/drivers/common/mlx5/mlx5_common_devx.c index d19be122bd..22c8d356c4 100644 --- a/drivers/common/mlx5/mlx5_common_devx.c +++ b/drivers/common/mlx5/mlx5_common_devx.c @@ -12,7 +12,7 @@ #include "mlx5_prm.h" #include "mlx5_devx_cmds.h" -#include "mlx5_common_utils.h" +#include "mlx5_common_log.h" #include "mlx5_malloc.h" #include "mlx5_common.h" #include "mlx5_common_devx.h" diff --git a/drivers/common/mlx5/mlx5_common_log.h b/drivers/common/mlx5/mlx5_common_log.h new file mode 100644 index 00..26b13fedaf --- /dev/null +++ b/drivers/common/mlx5/mlx5_common_log.h @@ -0,0 +1,21 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2019 Mellanox Technologies, Ltd + */ + +#ifndef RTE_PMD_MLX5_COMMON_LOG_H_ +#define RTE_PMD_MLX5_COMMON_LOG_H_ + +#include "mlx5_common.h" + + +extern int mlx5_common_logtype; + +#define MLX5_COMMON_LOG_PREFIX "mlx5_common" +/* Generic printf()-like logging macro with automatic line feed. */ +#define DRV_LOG(level, ...) \ + PMD_DRV_LOG_(level, mlx5_common_logtyp
[dpdk-dev] [PATCH v2 12/16] common/mlx5: share get ib device match function
From: Shiri Kuzin
The get_ib_device_match function iterates over the list of ib devices
returned by the get_device_list glue function and returns the ib device
matching the provided address.
Since this function is in use by several drivers, in this patch we
share the function in common part.
Signed-off-by: Shiri Kuzin
Acked-by: Matan Azrad
---
drivers/common/mlx5/linux/mlx5_common_os.c | 28 +++
drivers/common/mlx5/linux/mlx5_common_os.h | 5
drivers/common/mlx5/mlx5_common.h | 2 ++
drivers/common/mlx5/version.map| 1 +
drivers/compress/mlx5/mlx5_compress.c | 30 +---
drivers/regex/mlx5/mlx5_regex.c| 30 ++--
drivers/vdpa/mlx5/mlx5_vdpa.c | 32 ++
7 files changed, 41 insertions(+), 87 deletions(-)
diff --git a/drivers/common/mlx5/linux/mlx5_common_os.c
b/drivers/common/mlx5/linux/mlx5_common_os.c
index fba8245b8b..037147fe31 100644
--- a/drivers/common/mlx5/linux/mlx5_common_os.c
+++ b/drivers/common/mlx5/linux/mlx5_common_os.c
@@ -16,6 +16,7 @@
#include "mlx5_common.h"
#include "mlx5_common_log.h"
+#include "mlx5_common_os.h"
#include "mlx5_glue.h"
#ifdef MLX5_GLUE
@@ -423,3 +424,30 @@ mlx5_glue_constructor(void)
mlx5_glue = NULL;
}
+struct ibv_device *
+mlx5_os_get_ib_device_match(struct rte_pci_addr *addr)
+{
+ int n;
+ struct ibv_device **ibv_list = mlx5_glue->get_device_list(&n);
+ struct ibv_device *ibv_match = NULL;
+
+ if (ibv_list == NULL) {
+ rte_errno = ENOSYS;
+ return NULL;
+ }
+ while (n-- > 0) {
+ struct rte_pci_addr paddr;
+
+ DRV_LOG(DEBUG, "Checking device \"%s\"..", ibv_list[n]->name);
+ if (mlx5_dev_to_pci_addr(ibv_list[n]->ibdev_path, &paddr) != 0)
+ continue;
+ if (rte_pci_addr_cmp(addr, &paddr) != 0)
+ continue;
+ ibv_match = ibv_list[n];
+ break;
+ }
+ if (ibv_match == NULL)
+ rte_errno = ENOENT;
+ mlx5_glue->free_device_list(ibv_list);
+ return ibv_match;
+}
diff --git a/drivers/common/mlx5/linux/mlx5_common_os.h
b/drivers/common/mlx5/linux/mlx5_common_os.h
index d1c7e3dce0..9fff9cdb83 100644
--- a/drivers/common/mlx5/linux/mlx5_common_os.h
+++ b/drivers/common/mlx5/linux/mlx5_common_os.h
@@ -288,4 +288,9 @@ mlx5_os_free(void *addr)
{
free(addr);
}
+
+__rte_internal
+struct ibv_device *
+mlx5_os_get_ib_device_match(struct rte_pci_addr *addr);
+
#endif /* RTE_PMD_MLX5_COMMON_OS_H_ */
diff --git a/drivers/common/mlx5/mlx5_common.h
b/drivers/common/mlx5/mlx5_common.h
index f3c6beb23b..89aca32305 100644
--- a/drivers/common/mlx5/mlx5_common.h
+++ b/drivers/common/mlx5/mlx5_common.h
@@ -18,6 +18,7 @@
#include "mlx5_prm.h"
#include "mlx5_devx_cmds.h"
+#include "mlx5_common_os.h"
/* Reported driver name. */
#define MLX5_PCI_DRIVER_NAME "mlx5_pci"
@@ -215,6 +216,7 @@ enum mlx5_class {
MLX5_CLASS_VDPA = RTE_BIT64(1),
MLX5_CLASS_REGEX = RTE_BIT64(2),
MLX5_CLASS_COMPRESS = RTE_BIT64(3),
+ MLX5_CLASS_CRYPTO = RTE_BIT64(4),
};
#define MLX5_DBR_SIZE RTE_CACHE_LINE_SIZE
diff --git a/drivers/common/mlx5/version.map b/drivers/common/mlx5/version.map
index d16e484ffa..00df37e81a 100644
--- a/drivers/common/mlx5/version.map
+++ b/drivers/common/mlx5/version.map
@@ -111,6 +111,7 @@ INTERNAL {
mlx5_os_reg_mr;
mlx5_os_umem_reg;
mlx5_os_umem_dereg;
+ mlx5_os_get_ib_device_match; # WINDOWS_NO_EXPORT
mlx5_translate_port_name; # WINDOWS_NO_EXPORT
diff --git a/drivers/compress/mlx5/mlx5_compress.c
b/drivers/compress/mlx5/mlx5_compress.c
index ec3c237512..ff988d26ef 100644
--- a/drivers/compress/mlx5/mlx5_compress.c
+++ b/drivers/compress/mlx5/mlx5_compress.c
@@ -647,34 +647,6 @@ mlx5_compress_dequeue_burst(void *queue_pair, struct
rte_comp_op **ops,
return i;
}
-static struct ibv_device *
-mlx5_compress_get_ib_device_match(struct rte_pci_addr *addr)
-{
- int n;
- struct ibv_device **ibv_list = mlx5_glue->get_device_list(&n);
- struct ibv_device *ibv_match = NULL;
-
- if (ibv_list == NULL) {
- rte_errno = ENOSYS;
- return NULL;
- }
- while (n-- > 0) {
- struct rte_pci_addr paddr;
-
- DRV_LOG(DEBUG, "Checking device \"%s\"..", ibv_list[n]->name);
- if (mlx5_dev_to_pci_addr(ibv_list[n]->ibdev_path, &paddr) != 0)
- continue;
- if (rte_pci_addr_cmp(addr, &paddr) != 0)
- continue;
- ibv_match = ibv_list[n];
- break;
- }
- if (ibv_match == NULL)
-
[dpdk-dev] [PATCH v2 13/16] common/mlx5: support general obj CREDENTIAL create
From: Dekel Peled
CREDENTIAL object is used for any crypto operation in wrapped mode.
This patch add support of CREDENTIAL object create operation.
Add reading of CREDENTIAL support capability.
Add function to create general object type CREDENTIAL, using DevX API.
Signed-off-by: Dekel Peled
Acked-by: Matan Azrad
---
drivers/common/mlx5/mlx5_devx_cmds.c | 53 +++-
drivers/common/mlx5/mlx5_devx_cmds.h | 17 +++--
drivers/common/mlx5/mlx5_prm.h | 23
drivers/common/mlx5/version.map | 1 +
4 files changed, 91 insertions(+), 3 deletions(-)
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c
b/drivers/common/mlx5/mlx5_devx_cmds.c
index c0061741e8..c0a0853c3a 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -754,6 +754,8 @@ mlx5_devx_cmd_query_hca_attr(void *ctx,
MLX5_GENERAL_OBJ_TYPES_CAP_DEK);
attr->import_kek = !!(general_obj_types_supported &
MLX5_GENERAL_OBJ_TYPES_CAP_IMPORT_KEK);
+ attr->credential = !!(general_obj_types_supported &
+ MLX5_GENERAL_OBJ_TYPES_CAP_CREDENTIAL);
attr->crypto_login = !!(general_obj_types_supported &
MLX5_GENERAL_OBJ_TYPES_CAP_CRYPTO_LOGIN);
/* Add reading of other GENERAL_OBJ_TYPES_CAP bits above this line. */
@@ -2512,6 +2514,55 @@ mlx5_devx_cmd_create_import_kek_obj(void *ctx,
return import_kek_obj;
}
+/**
+ * Create general object of type CREDENTIAL using DevX API.
+ *
+ * @param[in] ctx
+ * Context returned from mlx5 open_device() glue function.
+ * @param [in] attr
+ * Pointer to CREDENTIAL attributes structure.
+ *
+ * @return
+ * The DevX object created, NULL otherwise and rte_errno is set.
+ */
+struct mlx5_devx_obj *
+mlx5_devx_cmd_create_credential_obj(void *ctx,
+ struct mlx5_devx_credential_attr *attr)
+{
+ uint32_t in[MLX5_ST_SZ_DW(create_credential_in)] = {0};
+ uint32_t out[MLX5_ST_SZ_DW(general_obj_out_cmd_hdr)] = {0};
+ struct mlx5_devx_obj *credential_obj = NULL;
+ void *ptr = NULL, *credential_addr = NULL;
+
+ credential_obj = mlx5_malloc(MLX5_MEM_ZERO, sizeof(*credential_obj),
+0, SOCKET_ID_ANY);
+ if (credential_obj == NULL) {
+ DRV_LOG(ERR, "Failed to allocate CREDENTIAL object data");
+ rte_errno = ENOMEM;
+ return NULL;
+ }
+ ptr = MLX5_ADDR_OF(create_credential_in, in, hdr);
+ MLX5_SET(general_obj_in_cmd_hdr, ptr, opcode,
+MLX5_CMD_OP_CREATE_GENERAL_OBJECT);
+ MLX5_SET(general_obj_in_cmd_hdr, ptr, obj_type,
+MLX5_GENERAL_OBJ_TYPE_CREDENTIAL);
+ ptr = MLX5_ADDR_OF(create_credential_in, in, credential);
+ MLX5_SET(credential, ptr, credential_role, attr->credential_role);
+ credential_addr = MLX5_ADDR_OF(credential, ptr, credential);
+ memcpy(credential_addr, (void *)(attr->credential),
+ MLX5_CRYPTO_CREDENTIAL_SIZE);
+ credential_obj->obj = mlx5_glue->devx_obj_create(ctx, in, sizeof(in),
+out, sizeof(out));
+ if (credential_obj->obj == NULL) {
+ rte_errno = errno;
+ DRV_LOG(ERR, "Failed to create CREDENTIAL object using DevX.");
+ mlx5_free(credential_obj);
+ return NULL;
+ }
+ credential_obj->id = MLX5_GET(general_obj_out_cmd_hdr, out, obj_id);
+ return credential_obj;
+}
+
/**
* Create general object of type CRYPTO_LOGIN using DevX API.
*
@@ -2551,7 +2602,7 @@ mlx5_devx_cmd_create_crypto_login_obj(void *ctx,
attr->session_import_kek_ptr);
credential_addr = MLX5_ADDR_OF(crypto_login, ptr, credential);
memcpy(credential_addr, (void *)(attr->credential),
- MLX5_CRYPTO_LOGIN_CREDENTIAL_SIZE);
+ MLX5_CRYPTO_CREDENTIAL_SIZE);
crypto_login_obj->obj = mlx5_glue->devx_obj_create(ctx, in, sizeof(in),
out, sizeof(out));
if (crypto_login_obj->obj == NULL) {
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h
b/drivers/common/mlx5/mlx5_devx_cmds.h
index 709e28bfba..811e7a1462 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.h
+++ b/drivers/common/mlx5/mlx5_devx_cmds.h
@@ -143,6 +143,7 @@ struct mlx5_hca_attr {
uint32_t aes_xts:1; /* AES-XTS crypto is supported. */
uint32_t dek:1; /* General obj type DEK is supported. */
uint32_t import_kek:1; /* General obj type IMPORT_KEK supported. */
+ uint32_t credential:1; /* General obj type CREDENTIAL supported. */
uint32_t crypto_login:1; /* General obj type CRYPTO_LOGIN supported. */
uint32_t regexp_num_of_engine
[dpdk-dev] [PATCH v2 14/16] common/mlx5: add crypto register structs and defs
From: Dekel Peled
Encryption key management requires use of several related registers.
This patch adds the relevant structs and values, according to PRM
definitions.
Signed-off-by: Dekel Peled
Acked-by: Matan Azrad
---
drivers/common/mlx5/mlx5_prm.h | 41 ++
1 file changed, 41 insertions(+)
diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h
index 432c8fdb63..c2cd2d9f70 100644
--- a/drivers/common/mlx5/mlx5_prm.h
+++ b/drivers/common/mlx5/mlx5_prm.h
@@ -3307,6 +3307,10 @@ enum {
enum {
MLX5_REGISTER_ID_MTUTC = 0x9055,
+ MLX5_CRYPTO_OPERATIONAL_REGISTER_ID = 0xC002,
+ MLX5_CRYPTO_COMMISSIONING_REGISTER_ID = 0xC003,
+ MLX5_IMPORT_KEK_HANDLE_REGISTER_ID = 0xC004,
+ MLX5_CREDENTIAL_HANDLE_REGISTER_ID = 0xC005,
};
struct mlx5_ifc_register_mtutc_bits {
@@ -3324,6 +3328,43 @@ struct mlx5_ifc_register_mtutc_bits {
#define MLX5_MTUTC_TIMESTAMP_MODE_INTERNAL_TIMER 0
#define MLX5_MTUTC_TIMESTAMP_MODE_REAL_TIME 1
+struct mlx5_ifc_crypto_operational_register_bits {
+ u8 wrapped_crypto_operational[0x1];
+ u8 reserved_at_1[0x1b];
+ u8 kek_size[0x4];
+ u8 reserved_at_20[0x20];
+ u8 credential[0x140];
+ u8 kek[0x100];
+ u8 reserved_at_280[0x180];
+};
+
+struct mlx5_ifc_crypto_commissioning_register_bits {
+ u8 token[0x1]; /* TODO: add size after PRM update */
+};
+
+struct mlx5_ifc_import_kek_handle_register_bits {
+ struct mlx5_ifc_crypto_login_bits crypto_login_object;
+ struct mlx5_ifc_import_kek_bits import_kek_object;
+ u8 reserved_at_200[0x4];
+ u8 write_operation[0x4];
+ u8 import_kek_id[0x18];
+ u8 reserved_at_220[0xe0];
+};
+
+struct mlx5_ifc_credential_handle_register_bits {
+ struct mlx5_ifc_crypto_login_bits crypto_login_object;
+ struct mlx5_ifc_credential_bits credential_object;
+ u8 reserved_at_200[0x4];
+ u8 write_operation[0x4];
+ u8 credential_id[0x18];
+ u8 reserved_at_220[0xe0];
+};
+
+enum {
+ MLX5_REGISTER_ADD_OPERATION = 0x1,
+ MLX5_REGISTER_DELETE_OPERATION = 0x2,
+};
+
struct mlx5_ifc_parse_graph_arc_bits {
u8 start_inner_tunnel[0x1];
u8 reserved_at_1[0x7];
--
2.25.1
[dpdk-dev] [PATCH v2 15/16] common/mlx5: support register write access
From: Dekel Peled
This patch adds support of write operation to NIC registers.
Signed-off-by: Dekel Peled
Acked-by: Matan Azrad
---
drivers/common/mlx5/mlx5_devx_cmds.c | 67 +++-
drivers/common/mlx5/mlx5_devx_cmds.h | 4 ++
drivers/common/mlx5/version.map | 21 ++---
3 files changed, 83 insertions(+), 9 deletions(-)
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c
b/drivers/common/mlx5/mlx5_devx_cmds.c
index c0a0853c3a..0b421933ce 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -12,7 +12,6 @@
#include "mlx5_common_log.h"
#include "mlx5_malloc.h"
-
/**
* Perform read access to the registers. Reads data from register
* and writes ones to the specified buffer.
@@ -61,7 +60,7 @@ mlx5_devx_cmd_register_read(void *ctx, uint16_t reg_id,
uint32_t arg,
if (status) {
int syndrome = MLX5_GET(access_register_out, out, syndrome);
- DRV_LOG(DEBUG, "Failed to access NIC register 0x%X, "
+ DRV_LOG(DEBUG, "Failed to read access NIC register 0x%X, "
"status %x, syndrome = %x",
reg_id, status, syndrome);
return -1;
@@ -74,6 +73,70 @@ mlx5_devx_cmd_register_read(void *ctx, uint16_t reg_id,
uint32_t arg,
return rc;
}
+/**
+ * Perform write access to the registers.
+ *
+ * @param[in] ctx
+ * Context returned from mlx5 open_device() glue function.
+ * @param[in] reg_id
+ * Register identifier according to the PRM.
+ * @param[in] arg
+ * Register access auxiliary parameter according to the PRM.
+ * @param[out] data
+ * Pointer to the buffer containing data to write.
+ * @param[in] dw_cnt
+ * Buffer size in double words (32bit units).
+ *
+ * @return
+ * 0 on success, a negative value otherwise.
+ */
+int
+mlx5_devx_cmd_register_write(void *ctx, uint16_t reg_id, uint32_t arg,
+uint32_t *data, uint32_t dw_cnt)
+{
+ uint32_t in[MLX5_ST_SZ_DW(access_register_in) +
+ MLX5_ACCESS_REGISTER_DATA_DWORD_MAX] = {0};
+ uint32_t out[MLX5_ST_SZ_DW(access_register_out)] = {0};
+ int status, rc;
+ void *ptr;
+
+ MLX5_ASSERT(data && dw_cnt);
+ MLX5_ASSERT(dw_cnt <= MLX5_ACCESS_REGISTER_DATA_DWORD_MAX);
+ if (dw_cnt > MLX5_ACCESS_REGISTER_DATA_DWORD_MAX) {
+ DRV_LOG(ERR, "Data to write exceeds max size");
+ return -1;
+ }
+ MLX5_SET(access_register_in, in, opcode,
+MLX5_CMD_OP_ACCESS_REGISTER_USER);
+ MLX5_SET(access_register_in, in, op_mod,
+MLX5_ACCESS_REGISTER_IN_OP_MOD_WRITE);
+ MLX5_SET(access_register_in, in, register_id, reg_id);
+ MLX5_SET(access_register_in, in, argument, arg);
+ ptr = MLX5_ADDR_OF(access_register_in, in, register_data);
+ memcpy(ptr, data, dw_cnt * sizeof(uint32_t));
+ rc = mlx5_glue->devx_general_cmd(ctx, in, sizeof(in), out, sizeof(out));
+
+ rc = mlx5_glue->devx_general_cmd(ctx, in,
+MLX5_ST_SZ_BYTES(access_register_in) +
+dw_cnt * sizeof(uint32_t),
+out, sizeof(out));
+ if (rc)
+ goto error;
+ status = MLX5_GET(access_register_out, out, status);
+ if (status) {
+ int syndrome = MLX5_GET(access_register_out, out, syndrome);
+
+ DRV_LOG(DEBUG, "Failed to write access NIC register 0x%X, "
+ "status %x, syndrome = %x",
+ reg_id, status, syndrome);
+ return -1;
+ }
+ return 0;
+error:
+ rc = (rc > 0) ? -rc : rc;
+ return rc;
+}
+
/**
* Allocate flow counters via devx interface.
*
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h
b/drivers/common/mlx5/mlx5_devx_cmds.h
index 811e7a1462..ce570ad28a 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.h
+++ b/drivers/common/mlx5/mlx5_devx_cmds.h
@@ -566,6 +566,10 @@ __rte_internal
int mlx5_devx_cmd_register_read(void *ctx, uint16_t reg_id,
uint32_t arg, uint32_t *data, uint32_t dw_cnt);
+__rte_internal
+int mlx5_devx_cmd_register_write(void *ctx, uint16_t reg_id,
+uint32_t arg, uint32_t *data, uint32_t dw_cnt);
+
__rte_internal
struct mlx5_devx_obj *
mlx5_devx_cmd_create_geneve_tlv_option(void *ctx,
diff --git a/drivers/common/mlx5/version.map b/drivers/common/mlx5/version.map
index 1885cb8f6a..1dc2d063ff 100644
--- a/drivers/common/mlx5/version.map
+++ b/drivers/common/mlx5/version.map
@@ -13,8 +13,17 @@ INTERNAL {
mlx5_dev_to_pci_addr; # WINDOWS_NO_EXPORT
mlx5_devx_cmd_alloc_pd;
+ mlx5_devx_alloc_uar;
+
mlx5_devx_cm
[dpdk-dev] [PATCH v2 16/16] common/mlx5: add UMR and RDMA write WQE defines
From: Suanming Mou
This patch adds the struct defining UMR and RDMA write WQEs.
Signed-off-by: Suanming Mou
Signed-off-by: Matan Azrad
---
drivers/common/mlx5/mlx5_prm.h | 187 +
1 file changed, 121 insertions(+), 66 deletions(-)
diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h
index c2cd2d9f70..1ffee5fd56 100644
--- a/drivers/common/mlx5/mlx5_prm.h
+++ b/drivers/common/mlx5/mlx5_prm.h
@@ -412,6 +412,127 @@ struct mlx5_cqe_ts {
uint8_t op_own;
};
+struct mlx5_wqe_rseg {
+ uint64_t raddr;
+ uint32_t rkey;
+ uint32_t reserved;
+} __rte_packed;
+
+#define MLX5_UMRC_IF_OFFSET 31u
+#define MLX5_UMRC_KO_OFFSET 16u
+#define MLX5_UMRC_TO_BS_OFFSET 0u
+
+struct mlx5_wqe_umr_cseg {
+ uint32_t if_cf_toe_cq_res;
+ uint32_t ko_to_bs;
+ uint64_t mkey_mask;
+ uint32_t rsvd1[8];
+} __rte_packed;
+
+struct mlx5_wqe_mkey_cseg {
+ uint32_t fr_res_af_sf;
+ uint32_t qpn_mkey;
+ uint32_t reserved2;
+ uint32_t flags_pd;
+ uint64_t start_addr;
+ uint64_t len;
+ uint32_t bsf_octword_size;
+ uint32_t reserved3[4];
+ uint32_t translations_octword_size;
+ uint32_t res4_lps;
+ uint32_t reserved;
+} __rte_packed;
+
+enum {
+ MLX5_BSF_SIZE_16B = 0x0,
+ MLX5_BSF_SIZE_32B = 0x1,
+ MLX5_BSF_SIZE_64B = 0x2,
+ MLX5_BSF_SIZE_128B = 0x3,
+};
+
+enum {
+ MLX5_BSF_P_TYPE_SIGNATURE = 0x0,
+ MLX5_BSF_P_TYPE_CRYPTO = 0x1,
+};
+
+enum {
+ MLX5_ENCRYPTION_ORDER_ENCRYPTED_WIRE_SIGNATURE = 0x0,
+ MLX5_ENCRYPTION_ORDER_ENCRYPTED_MEMORY_SIGNATURE = 0x1,
+ MLX5_ENCRYPTION_ORDER_ENCRYPTED_RAW_WIRE = 0x2,
+ MLX5_ENCRYPTION_ORDER_ENCRYPTED_RAW_MEMORY = 0x3,
+};
+
+enum {
+ MLX5_ENCRYPTION_STANDARD_AES_XTS = 0x0,
+};
+
+enum {
+ MLX5_BLOCK_SIZE_512B= 0x1,
+ MLX5_BLOCK_SIZE_520B= 0x2,
+ MLX5_BLOCK_SIZE_4096B = 0x3,
+ MLX5_BLOCK_SIZE_4160B = 0x4,
+ MLX5_BLOCK_SIZE_1MB = 0x5,
+ MLX5_BLOCK_SIZE_4048B = 0x6,
+};
+
+#define MLX5_BSF_SIZE_OFFSET 30
+#define MLX5_BSF_P_TYPE_OFFSET 24
+#define MLX5_ENCRYPTION_ORDER_OFFSET 16
+#define MLX5_BLOCK_SIZE_OFFSET 24
+
+struct mlx5_wqe_umr_bsf_seg {
+ /*
+* bs_bpt_eo_es contains:
+* bs bsf_size2 bits at MLX5_BSF_SIZE_OFFSET
+* bpt bsf_p_type 2 bits at MLX5_BSF_P_TYPE_OFFSET
+* eo encryption_order4 bits at MLX5_ENCRYPTION_ORDER_OFFSET
+* es encryption_standard 4 bits at offset 0
+*/
+ uint32_t bs_bpt_eo_es;
+ uint32_t raw_data_size;
+ /*
+* bsp_res contains:
+* bsp crypto_block_size_pointer 8 bits at MLX5_BLOCK_SIZE_OFFSET
+* res reserved 24 bits
+*/
+ uint32_t bsp_res;
+ uint32_t reserved0;
+ uint8_t xts_initial_tweak[16];
+ /*
+* res_dp contains:
+* res reserved 8 bits
+* dp dek_pointer 24 bits at offset 0
+*/
+ uint32_t res_dp;
+ uint32_t reserved1;
+ uint64_t keytag;
+ uint32_t reserved2[4];
+} __rte_packed;
+
+#ifdef PEDANTIC
+#pragma GCC diagnostic ignored "-Wpedantic"
+#endif
+
+struct mlx5_umr_wqe {
+ struct mlx5_wqe_cseg ctr;
+ struct mlx5_wqe_umr_cseg ucseg;
+ struct mlx5_wqe_mkey_cseg mkc;
+ union {
+ struct mlx5_wqe_dseg kseg[0];
+ struct mlx5_wqe_umr_bsf_seg bsf[0];
+ };
+} __rte_packed;
+
+struct mlx5_rdma_write_wqe {
+ struct mlx5_wqe_cseg ctr;
+ struct mlx5_wqe_rseg rseg;
+ struct mlx5_wqe_dseg dseg[0];
+} __rte_packed;
+
+#ifdef PEDANTIC
+#pragma GCC diagnostic error "-Wpedantic"
+#endif
+
/* GGA */
/* MMO metadata segment */
@@ -1096,72 +1217,6 @@ struct mlx5_ifc_create_mkey_in_bits {
u8 klm_pas_mtt[][0x20];
};
-enum {
- MLX5_BSF_SIZE_16B = 0x0,
- MLX5_BSF_SIZE_32B = 0x1,
- MLX5_BSF_SIZE_64B = 0x2,
- MLX5_BSF_SIZE_128B = 0x3,
-};
-
-enum {
- MLX5_BSF_P_TYPE_SIGNATURE = 0x0,
- MLX5_BSF_P_TYPE_CRYPTO = 0x1,
-};
-
-enum {
- MLX5_ENCRYPTION_ORDER_ENCRYPTED_WIRE_SIGNATURE = 0x0,
- MLX5_ENCRYPTION_ORDER_ENCRYPTED_MEMORY_SIGNATURE = 0x1,
- MLX5_ENCRYPTION_ORDER_ENCRYPTED_RAW_WIRE = 0x2,
- MLX5_ENCRYPTION_ORDER_ENCRYPTED_RAW_MEMORY = 0x3,
-};
-
-enum {
- MLX5_ENCRYPTION_STANDARD_AES_XTS = 0x0,
-};
-
-enum {
- MLX5_BLOCK_SIZE_512B= 0x1,
- MLX5_BLOCK_SIZE_520B= 0x2,
- MLX5_BLOCK_SIZE_4096B = 0x3,
- MLX5_BLOCK_SIZE_4160B = 0x4,
- MLX5_BLOCK_SIZE_1MB = 0x5,
- MLX5_BLOCK_SIZE_4048B = 0x6,
-};
-
-#define MLX5_BSF_SIZE_OFFSET 30
-#define MLX5_BSF_P_TYPE_OFFSET 24
-#define MLX5_ENCRYPTION_ORDER_OFFSET 16
-#define MLX5_BLOCK_SIZE_OFFSET 24
-
-struct mlx5_wqe_umr_bsf_seg {
- /*
-
[dpdk-dev] [PATCH v2 00/15] drivers: introduce mlx5 crypto PMD
Add a new PMD for Nvidia devices- crypto PMD. The crypto PMD will be supported on Nvidia ConnectX6 The crypto PMD will add the support of encryption and decryption using the AES-XTS symmetric algorithm. The crypto PMD requires rdma-core and uses mlx5 DevX. v2: Add data-path part. Shiri Kuzin (10): drivers: introduce mlx5 crypto PMD crypto/mlx5: add DEK object management crypto/mlx5: support session operations crypto/mlx5: add basic operations crypto/mlx5: support queue pairs operations crypto/mlx5: add dev stop and start operations crypto/mlx5: add memory region management crypto/mlx5: create login object using DevX crypto/mlx5: adjust to the multiple data unit API crypto/mlx5: set feature flags and capabilities Suanming Mou (5): crypto/mlx5: add keytag device argument crypto/mlx5: add maximum segments device argument crypto/mlx5: add WQE set initialization crypto/mlx5: add enqueue and dequeue operations crypto/mlx5: add statistic get and reset operations MAINTAINERS |4 + doc/guides/cryptodevs/features/mlx5.ini | 37 + doc/guides/cryptodevs/index.rst |1 + doc/guides/cryptodevs/mlx5.rst | 152 doc/guides/rel_notes/release_21_05.rst |5 + drivers/common/mlx5/mlx5_common_pci.c | 14 + drivers/common/mlx5/mlx5_common_pci.h | 21 +- drivers/crypto/meson.build |1 + drivers/crypto/mlx5/meson.build | 27 + drivers/crypto/mlx5/mlx5_crypto.c | 1092 +++ drivers/crypto/mlx5/mlx5_crypto.h | 92 ++ drivers/crypto/mlx5/mlx5_crypto_dek.c | 136 +++ drivers/crypto/mlx5/mlx5_crypto_utils.h | 19 + drivers/crypto/mlx5/version.map |3 + 14 files changed, 1594 insertions(+), 10 deletions(-) create mode 100644 doc/guides/cryptodevs/features/mlx5.ini create mode 100644 doc/guides/cryptodevs/mlx5.rst create mode 100644 drivers/crypto/mlx5/meson.build create mode 100644 drivers/crypto/mlx5/mlx5_crypto.c create mode 100644 drivers/crypto/mlx5/mlx5_crypto.h create mode 100644 drivers/crypto/mlx5/mlx5_crypto_dek.c create mode 100644 drivers/crypto/mlx5/mlx5_crypto_utils.h create mode 100644 drivers/crypto/mlx5/version.map -- 2.25.1
[dpdk-dev] [PATCH v2 01/15] drivers: introduce mlx5 crypto PMD
From: Shiri Kuzin
Add a new PMD for Nvidia devices- crypto PMD.
The crypto PMD will be supported starting Nvidia ConnectX6 and
BlueField2.
The crypto PMD will add the support of encryption and decryption using
the AES-XTS symmetric algorithm.
The crypto PMD requires rdma-core and uses mlx5 DevX.
This patch adds the PCI probing, basic functions, build files and
log utility.
Signed-off-by: Shiri Kuzin
Acked-by: Matan Azrad
---
MAINTAINERS | 4 +
drivers/common/mlx5/mlx5_common_pci.c | 14 ++
drivers/common/mlx5/mlx5_common_pci.h | 21 +-
drivers/crypto/meson.build | 1 +
drivers/crypto/mlx5/meson.build | 26 +++
drivers/crypto/mlx5/mlx5_crypto.c | 272
drivers/crypto/mlx5/mlx5_crypto_utils.h | 19 ++
drivers/crypto/mlx5/version.map | 3 +
8 files changed, 350 insertions(+), 10 deletions(-)
create mode 100644 drivers/crypto/mlx5/meson.build
create mode 100644 drivers/crypto/mlx5/mlx5_crypto.c
create mode 100644 drivers/crypto/mlx5/mlx5_crypto_utils.h
create mode 100644 drivers/crypto/mlx5/version.map
diff --git a/MAINTAINERS b/MAINTAINERS
index 44f3d322ed..707e5a43b4 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1081,6 +1081,10 @@ F: drivers/crypto/octeontx2/
F: doc/guides/cryptodevs/octeontx2.rst
F: doc/guides/cryptodevs/features/octeontx2.ini
+Mellanox mlx5
+M: Matan Azrad
+F: drivers/crypto/mlx5/
+
Null Crypto
M: Declan Doherty
F: drivers/crypto/null/
diff --git a/drivers/common/mlx5/mlx5_common_pci.c
b/drivers/common/mlx5/mlx5_common_pci.c
index 3f16cd21cf..8a47afee20 100644
--- a/drivers/common/mlx5/mlx5_common_pci.c
+++ b/drivers/common/mlx5/mlx5_common_pci.c
@@ -31,6 +31,7 @@ static const struct {
{ .name = "net", .driver_class = MLX5_CLASS_NET },
{ .name = "regex", .driver_class = MLX5_CLASS_REGEX },
{ .name = "compress", .driver_class = MLX5_CLASS_COMPRESS },
+ { .name = "crypto", .driver_class = MLX5_CLASS_CRYPTO },
};
static const unsigned int mlx5_class_combinations[] = {
@@ -38,13 +39,26 @@ static const unsigned int mlx5_class_combinations[] = {
MLX5_CLASS_VDPA,
MLX5_CLASS_REGEX,
MLX5_CLASS_COMPRESS,
+ MLX5_CLASS_CRYPTO,
MLX5_CLASS_NET | MLX5_CLASS_REGEX,
MLX5_CLASS_VDPA | MLX5_CLASS_REGEX,
MLX5_CLASS_NET | MLX5_CLASS_COMPRESS,
MLX5_CLASS_VDPA | MLX5_CLASS_COMPRESS,
MLX5_CLASS_REGEX | MLX5_CLASS_COMPRESS,
+ MLX5_CLASS_NET | MLX5_CLASS_CRYPTO,
+ MLX5_CLASS_VDPA | MLX5_CLASS_CRYPTO,
+ MLX5_CLASS_REGEX | MLX5_CLASS_CRYPTO,
+ MLX5_CLASS_COMPRESS | MLX5_CLASS_CRYPTO,
MLX5_CLASS_NET | MLX5_CLASS_REGEX | MLX5_CLASS_COMPRESS,
MLX5_CLASS_VDPA | MLX5_CLASS_REGEX | MLX5_CLASS_COMPRESS,
+ MLX5_CLASS_NET | MLX5_CLASS_REGEX | MLX5_CLASS_CRYPTO,
+ MLX5_CLASS_VDPA | MLX5_CLASS_REGEX | MLX5_CLASS_CRYPTO,
+ MLX5_CLASS_NET | MLX5_CLASS_COMPRESS | MLX5_CLASS_CRYPTO,
+ MLX5_CLASS_VDPA | MLX5_CLASS_COMPRESS | MLX5_CLASS_CRYPTO,
+ MLX5_CLASS_NET | MLX5_CLASS_REGEX | MLX5_CLASS_COMPRESS |
+ MLX5_CLASS_CRYPTO,
+ MLX5_CLASS_VDPA | MLX5_CLASS_REGEX | MLX5_CLASS_COMPRESS |
+ MLX5_CLASS_CRYPTO,
/* New class combination should be added here. */
};
diff --git a/drivers/common/mlx5/mlx5_common_pci.h
b/drivers/common/mlx5/mlx5_common_pci.h
index de89bb98bc..cb8d2f5f87 100644
--- a/drivers/common/mlx5/mlx5_common_pci.h
+++ b/drivers/common/mlx5/mlx5_common_pci.h
@@ -9,17 +9,18 @@
* @file
*
* RTE Mellanox PCI Driver Interface
- * Mellanox ConnectX PCI device supports multiple class: net,vdpa,regex and
- * compress devices. This layer enables creating such multiple class of devices
- * on a single PCI device by allowing to bind multiple class specific device
- * driver to attach to mlx5_pci driver.
+ * Mellanox ConnectX PCI device supports multiple class:
net,vdpa,regex,compress
+ * and crypto devices. This layer enables creating such multiple class of
+ * devices on a single PCI device by allowing to bind multiple class specific
+ * device driver to attach to mlx5_pci driver.
*
- * ----
- * | mlx5 || mlx5 || mlx5|| mlx5 |
- * | net pmd || vdpa pmd || regex pmd || compress pmd |
- * ----
- * \ \/ /
- * \ \ / /
+ * ---
+ * | mlx5 || mlx5 || mlx5 || mlx5 || mlx5 |
+ * | net || vdpa || regex || compress || crypto |
+ * | pmd || pmd
[dpdk-dev] [PATCH v2 02/15] crypto/mlx5: add DEK object management
From: Shiri Kuzin
A DEK(Data encryption Key) is an mlx5 HW object which represents the
cipher algorithm key.
The DEKs are used during data encryption/decryption operations.
In symmetric algorithms like AES-STS, we use the same DEK for both
encryption and decryption.
Use the mlx5 hash-list tool to manage the DEK objects in the PMD.
Provide the compare, create and destroy functions to manage DEKs in
hash-list and introduce an internal API to setup and unset the DEK
management and to prepare and destroy specific DEK object.
The DEK hash-list will be created in dev_configure routine and
destroyed in dev_close routine.
Signed-off-by: Shiri Kuzin
Acked-by: Matan Azrad
---
drivers/crypto/mlx5/meson.build | 1 +
drivers/crypto/mlx5/mlx5_crypto.c | 44 +
drivers/crypto/mlx5/mlx5_crypto.h | 51 ++
drivers/crypto/mlx5/mlx5_crypto_dek.c | 136 ++
4 files changed, 215 insertions(+), 17 deletions(-)
create mode 100644 drivers/crypto/mlx5/mlx5_crypto.h
create mode 100644 drivers/crypto/mlx5/mlx5_crypto_dek.c
diff --git a/drivers/crypto/mlx5/meson.build b/drivers/crypto/mlx5/meson.build
index 5bf0912766..0666c35094 100644
--- a/drivers/crypto/mlx5/meson.build
+++ b/drivers/crypto/mlx5/meson.build
@@ -11,6 +11,7 @@ fmt_name = 'mlx5_crypto'
deps += ['common_mlx5', 'eal', 'cryptodev']
sources = files(
'mlx5_crypto.c',
+ 'mlx5_crypto_dek.c',
)
cflags_options = [
'-std=c11',
diff --git a/drivers/crypto/mlx5/mlx5_crypto.c
b/drivers/crypto/mlx5/mlx5_crypto.c
index 8a3a307f7b..e7c70c521f 100644
--- a/drivers/crypto/mlx5/mlx5_crypto.c
+++ b/drivers/crypto/mlx5/mlx5_crypto.c
@@ -3,12 +3,9 @@
*/
#include
-#include
#include
+#include
#include
-#include
-#include
-#include
#include
#include
@@ -17,19 +14,10 @@
#include
#include "mlx5_crypto_utils.h"
+#include "mlx5_crypto.h"
#define MLX5_CRYPTO_DRIVER_NAME mlx5_crypto
-#define MLX5_CRYPTO_LOG_NAMEpmd.crypto.mlx5
-
-struct mlx5_crypto_priv {
- TAILQ_ENTRY(mlx5_crypto_priv) next;
- struct ibv_context *ctx; /* Device context. */
- struct rte_pci_device *pci_dev;
- struct rte_cryptodev *crypto_dev;
- void *uar; /* User Access Region. */
- uint32_t pdn; /* Protection Domain number. */
- struct ibv_pd *pd;
-};
+#define MLX5_CRYPTO_LOG_NAME pmd.crypto.mlx5
TAILQ_HEAD(mlx5_crypto_privs, mlx5_crypto_priv) mlx5_crypto_priv_list =
TAILQ_HEAD_INITIALIZER(mlx5_crypto_priv_list);
@@ -48,11 +36,33 @@ static const struct rte_driver mlx5_drv = {
static struct cryptodev_driver mlx5_cryptodev_driver;
+static int
+mlx5_crypto_dev_configure(struct rte_cryptodev *dev,
+ struct rte_cryptodev_config *config __rte_unused)
+{
+ struct mlx5_crypto_priv *priv = dev->data->dev_private;
+
+ if (mlx5_crypto_dek_setup(priv) != 0) {
+ DRV_LOG(ERR, "Dek hash list creation has failed.");
+ return -ENOMEM;
+ }
+ return 0;
+}
+
+static int
+mlx5_crypto_dev_close(struct rte_cryptodev *dev)
+{
+ struct mlx5_crypto_priv *priv = dev->data->dev_private;
+
+ mlx5_crypto_dek_unset(priv);
+ return 0;
+}
+
static struct rte_cryptodev_ops mlx5_crypto_ops = {
- .dev_configure = NULL,
+ .dev_configure = mlx5_crypto_dev_configure,
.dev_start = NULL,
.dev_stop = NULL,
- .dev_close = NULL,
+ .dev_close = mlx5_crypto_dev_close,
.dev_infos_get = NULL,
.stats_get = NULL,
.stats_reset= NULL,
diff --git a/drivers/crypto/mlx5/mlx5_crypto.h
b/drivers/crypto/mlx5/mlx5_crypto.h
new file mode 100644
index 00..4ec67a7e0f
--- /dev/null
+++ b/drivers/crypto/mlx5/mlx5_crypto.h
@@ -0,0 +1,51 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2021 Mellanox Technologies, Ltd
+ */
+
+#ifndef MLX5_CRYPTO_H_
+#define MLX5_CRYPTO_H_
+
+#include
+
+#include
+#include
+
+#include
+
+#define MLX5_CRYPTO_DEK_HTABLE_SZ (1 << 11)
+#define MLX5_CRYPTO_KEY_LENGTH 80
+
+struct mlx5_crypto_priv {
+ TAILQ_ENTRY(mlx5_crypto_priv) next;
+ struct ibv_context *ctx; /* Device context. */
+ struct rte_pci_device *pci_dev;
+ struct rte_cryptodev *crypto_dev;
+ void *uar; /* User Access Region. */
+ uint32_t pdn; /* Protection Domain number. */
+ struct ibv_pd *pd;
+ struct mlx5_hlist *dek_hlist; /* Dek hash list. */
+};
+
+struct mlx5_crypto_dek {
+ struct mlx5_hlist_entry entry; /* Pointer to DEK hash list entry. */
+ struct mlx5_devx_obj *obj; /* Pointer to DEK DevX object. */
+ uint8_t data[MLX5_CRYPTO_KEY_LENGTH]; /* DEK key data. */
+
[dpdk-dev] [PATCH v2 03/15] crypto/mlx5: support session operations
From: Shiri Kuzin
Sessions are used in symmetric transformations in order to prepare
objects and data for packet processing stage.
A mlx5 session includes iv_offset, pointer to mlx5_crypto_dek struct,
bsf_size, bsf_p_type, encryption_order and encryption standard.
Implement the next session operations:
mlx5_crypto_sym_session_get_size- returns the size of the mlx5
session struct.
mlx5_crypto_sym_session_configure- prepares the DEK hash-list
and saves all the session data.
mlx5_crypto_sym_session_clear - destroys the DEK hash-list.
Signed-off-by: Shiri Kuzin
Acked-by: Matan Azrad
---
drivers/crypto/mlx5/mlx5_crypto.c | 96 ++-
1 file changed, 93 insertions(+), 3 deletions(-)
diff --git a/drivers/crypto/mlx5/mlx5_crypto.c
b/drivers/crypto/mlx5/mlx5_crypto.c
index e7c70c521f..c494ed00d1 100644
--- a/drivers/crypto/mlx5/mlx5_crypto.c
+++ b/drivers/crypto/mlx5/mlx5_crypto.c
@@ -3,6 +3,7 @@
*/
#include
+#include
#include
#include
#include
@@ -36,6 +37,24 @@ static const struct rte_driver mlx5_drv = {
static struct cryptodev_driver mlx5_cryptodev_driver;
+struct mlx5_crypto_session {
+ uint32_t bs_bpt_eo_es;
+ /*
+* bsf_size, bsf_p_type, encryption_order and encryption standard,
+* saved in big endian format.
+*/
+ uint32_t iv_offset:16;
+ /* Starting point for Initialisation Vector. */
+ struct mlx5_crypto_dek *dek; /* Pointer to dek struct. */
+ uint32_t dek_id; /* DEK ID */
+} __rte_packed;
+
+static unsigned int
+mlx5_crypto_sym_session_get_size(struct rte_cryptodev *dev __rte_unused)
+{
+ return sizeof(struct mlx5_crypto_session);
+}
+
static int
mlx5_crypto_dev_configure(struct rte_cryptodev *dev,
struct rte_cryptodev_config *config __rte_unused)
@@ -58,6 +77,77 @@ mlx5_crypto_dev_close(struct rte_cryptodev *dev)
return 0;
}
+static int
+mlx5_crypto_sym_session_configure(struct rte_cryptodev *dev,
+ struct rte_crypto_sym_xform *xform,
+ struct rte_cryptodev_sym_session *session,
+ struct rte_mempool *mp)
+{
+ struct mlx5_crypto_priv *priv = dev->data->dev_private;
+ struct mlx5_crypto_session *sess_private_data;
+ struct rte_crypto_cipher_xform *cipher;
+ uint8_t encryption_order;
+ int ret;
+
+ if (unlikely(xform->next != NULL)) {
+ DRV_LOG(ERR, "Xform next is not supported.");
+ return -ENOTSUP;
+ }
+ if (unlikely((xform->type != RTE_CRYPTO_SYM_XFORM_CIPHER) ||
+(xform->cipher.algo != RTE_CRYPTO_CIPHER_AES_XTS))) {
+ DRV_LOG(ERR, "Only AES-XTS algorithm is supported.");
+ return -ENOTSUP;
+ }
+ ret = rte_mempool_get(mp, (void *)&sess_private_data);
+ if (ret != 0) {
+ DRV_LOG(ERR,
+ "Failed to get session %p private data from mempool.",
+ sess_private_data);
+ return -ENOMEM;
+ }
+ cipher = &xform->cipher;
+ sess_private_data->dek = mlx5_crypto_dek_prepare(priv, cipher);
+ if (sess_private_data->dek == NULL) {
+ rte_mempool_put(mp, sess_private_data);
+ DRV_LOG(ERR, "Failed to prepare dek.");
+ return -ENOMEM;
+ }
+ if (cipher->op == RTE_CRYPTO_CIPHER_OP_ENCRYPT)
+ encryption_order = MLX5_ENCRYPTION_ORDER_ENCRYPTED_RAW_MEMORY;
+ else
+ encryption_order = MLX5_ENCRYPTION_ORDER_ENCRYPTED_RAW_WIRE;
+ sess_private_data->bs_bpt_eo_es = rte_cpu_to_be_32
+ (MLX5_BSF_SIZE_64B << MLX5_BSF_SIZE_OFFSET |
+MLX5_BSF_P_TYPE_CRYPTO << MLX5_BSF_P_TYPE_OFFSET |
+encryption_order << MLX5_ENCRYPTION_ORDER_OFFSET |
+MLX5_ENCRYPTION_STANDARD_AES_XTS);
+ sess_private_data->iv_offset = cipher->iv.offset;
+ sess_private_data->dek_id =
+ rte_cpu_to_be_32(sess_private_data->dek->obj->id &
+0xff);
+ set_sym_session_private_data(session, dev->driver_id,
+sess_private_data);
+ DRV_LOG(DEBUG, "Session %p was configured.", sess_private_data);
+ return 0;
+}
+
+static void
+mlx5_crypto_sym_session_clear(struct rte_cryptodev *dev,
+ struct rte_cryptodev_sym_session *sess)
+{
+ struct mlx5_crypto_priv *priv = dev->data->dev_private;
+ struct mlx5_crypto_session *sess_private_data =
+ get_sym_session_private_data(sess, dev->driver_id);
+
+ if (unlikely(sess_private_data == NULL)) {
+
[dpdk-dev] [PATCH v2 04/15] crypto/mlx5: add basic operations
From: Shiri Kuzin
The basic dev control operations are configure, close and get info.
Extended the existing support of configure and close:
-mlx5_crypto_dev_configure- function used to configure device.
-mlx5_crypto_dev_close- function used to close a configured
device.
Added support of get info function:
-mlx5_crypto_dev_infos_get- function used to get specific
information of a device.
Added config struct to user private data with the fields socket id,
number of queue pairs and feature flags to be disabled.
Signed-off-by: Shiri Kuzin
Acked-by: Matan Azrad
---
drivers/crypto/mlx5/mlx5_crypto.c | 46 +++
drivers/crypto/mlx5/mlx5_crypto.h | 1 +
2 files changed, 42 insertions(+), 5 deletions(-)
diff --git a/drivers/crypto/mlx5/mlx5_crypto.c
b/drivers/crypto/mlx5/mlx5_crypto.c
index c494ed00d1..cc11cabbea 100644
--- a/drivers/crypto/mlx5/mlx5_crypto.c
+++ b/drivers/crypto/mlx5/mlx5_crypto.c
@@ -19,6 +19,7 @@
#define MLX5_CRYPTO_DRIVER_NAME mlx5_crypto
#define MLX5_CRYPTO_LOG_NAME pmd.crypto.mlx5
+#define MLX5_CRYPTO_MAX_QPS 1024
TAILQ_HEAD(mlx5_crypto_privs, mlx5_crypto_priv) mlx5_crypto_priv_list =
TAILQ_HEAD_INITIALIZER(mlx5_crypto_priv_list);
@@ -28,6 +29,9 @@ int mlx5_crypto_logtype;
uint8_t mlx5_crypto_driver_id;
+const struct rte_cryptodev_capabilities
+ mlx5_crypto_caps[RTE_CRYPTO_OP_TYPE_UNDEFINED];
+
static const char mlx5_crypto_drv_name[] = RTE_STR(MLX5_CRYPTO_DRIVER_NAME);
static const struct rte_driver mlx5_drv = {
@@ -49,22 +53,47 @@ struct mlx5_crypto_session {
uint32_t dek_id; /* DEK ID */
} __rte_packed;
-static unsigned int
-mlx5_crypto_sym_session_get_size(struct rte_cryptodev *dev __rte_unused)
+static void
+mlx5_crypto_dev_infos_get(struct rte_cryptodev *dev,
+ struct rte_cryptodev_info *dev_info)
{
- return sizeof(struct mlx5_crypto_session);
+ RTE_SET_USED(dev);
+ if (dev_info != NULL) {
+ dev_info->driver_id = mlx5_crypto_driver_id;
+ dev_info->feature_flags = 0;
+ dev_info->capabilities = mlx5_crypto_caps;
+ dev_info->max_nb_queue_pairs = MLX5_CRYPTO_MAX_QPS;
+ dev_info->min_mbuf_headroom_req = 0;
+ dev_info->min_mbuf_tailroom_req = 0;
+ dev_info->sym.max_nb_sessions = 0;
+ /*
+* If 0, the device does not have any limitation in number of
+* sessions that can be used.
+*/
+ }
}
static int
mlx5_crypto_dev_configure(struct rte_cryptodev *dev,
- struct rte_cryptodev_config *config __rte_unused)
+ struct rte_cryptodev_config *config)
{
struct mlx5_crypto_priv *priv = dev->data->dev_private;
+ if (config == NULL) {
+ DRV_LOG(ERR, "Invalid crypto dev configure parameters.");
+ return -EINVAL;
+ }
+ if ((config->ff_disable & RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO) != 0) {
+ DRV_LOG(ERR,
+ "Disabled symmetric crypto feature is not supported.");
+ return -ENOTSUP;
+ }
if (mlx5_crypto_dek_setup(priv) != 0) {
DRV_LOG(ERR, "Dek hash list creation has failed.");
return -ENOMEM;
}
+ priv->dev_config = *config;
+ DRV_LOG(DEBUG, "Device %u was configured.", dev->driver_id);
return 0;
}
@@ -74,9 +103,16 @@ mlx5_crypto_dev_close(struct rte_cryptodev *dev)
struct mlx5_crypto_priv *priv = dev->data->dev_private;
mlx5_crypto_dek_unset(priv);
+ DRV_LOG(DEBUG, "Device %u was closed.", dev->driver_id);
return 0;
}
+static unsigned int
+mlx5_crypto_sym_session_get_size(struct rte_cryptodev *dev __rte_unused)
+{
+ return sizeof(struct mlx5_crypto_session);
+}
+
static int
mlx5_crypto_sym_session_configure(struct rte_cryptodev *dev,
struct rte_crypto_sym_xform *xform,
@@ -153,7 +189,7 @@ static struct rte_cryptodev_ops mlx5_crypto_ops = {
.dev_start = NULL,
.dev_stop = NULL,
.dev_close = mlx5_crypto_dev_close,
- .dev_infos_get = NULL,
+ .dev_infos_get = mlx5_crypto_dev_infos_get,
.stats_get = NULL,
.stats_reset= NULL,
.queue_pair_setup = NULL,
diff --git a/drivers/crypto/mlx5/mlx5_crypto.h
b/drivers/crypto/mlx5/mlx5_crypto.h
index 4ec67a7e0f..5e270d3d5a 100644
--- a/drivers/crypto/mlx5/mlx5_crypto.h
+++ b/drivers/crypto/mlx5/mlx5_crypto.h
@@ -24,6 +24,7 @@ struct mlx5_crypto_priv {
uint32_t pdn; /* Protection Domain number. */
[dpdk-dev] [PATCH v2 05/15] crypto/mlx5: support queue pairs operations
From: Shiri Kuzin
The HW queue pairs are a pair of send queue and receive queue of
independent work queues packed together in one object for the purpose
of transferring data between nodes of a network.
Completion Queue is a FIFO queue of completed work requests.
In crypto driver we use one QP in loopback in order to encrypt and
decrypt data locally without sending it to the wire.
In the configured QP we only use the SQ to perform the encryption and
decryption operations.
Added implementation for the QP setup function which creates the CQ,
creates the QP and changes its state to RTS (ready to send).
Added implementation for the release QP function to release all the QP
resources.
Added the ops structure that contains any operation which is supported
by the cryptodev.
Signed-off-by: Shiri Kuzin
Acked-by: Matan Azrad
---
drivers/crypto/mlx5/mlx5_crypto.c | 124 +-
drivers/crypto/mlx5/mlx5_crypto.h | 11 +++
2 files changed, 133 insertions(+), 2 deletions(-)
diff --git a/drivers/crypto/mlx5/mlx5_crypto.c
b/drivers/crypto/mlx5/mlx5_crypto.c
index cc11cabbea..5a14e05d55 100644
--- a/drivers/crypto/mlx5/mlx5_crypto.c
+++ b/drivers/crypto/mlx5/mlx5_crypto.c
@@ -7,6 +7,7 @@
#include
#include
#include
+#include
#include
#include
@@ -184,6 +185,125 @@ mlx5_crypto_sym_session_clear(struct rte_cryptodev *dev,
DRV_LOG(DEBUG, "Session %p was cleared.", sess_private_data);
}
+static int
+mlx5_crypto_queue_pair_release(struct rte_cryptodev *dev, uint16_t qp_id)
+{
+ struct mlx5_crypto_qp *qp = dev->data->queue_pairs[qp_id];
+
+ if (qp->qp_obj != NULL)
+ claim_zero(mlx5_devx_cmd_destroy(qp->qp_obj));
+ if (qp->umem_obj != NULL)
+ claim_zero(mlx5_glue->devx_umem_dereg(qp->umem_obj));
+ if (qp->umem_buf != NULL)
+ rte_free(qp->umem_buf);
+ mlx5_devx_cq_destroy(&qp->cq_obj);
+ rte_free(qp);
+ dev->data->queue_pairs[qp_id] = NULL;
+ return 0;
+}
+
+static int
+mlx5_crypto_qp2rts(struct mlx5_crypto_qp *qp)
+{
+ /*
+* In Order to configure self loopback, when calling these functions the
+* remote QP id that is used is the id of the same QP.
+*/
+ if (mlx5_devx_cmd_modify_qp_state(qp->qp_obj, MLX5_CMD_OP_RST2INIT_QP,
+ qp->qp_obj->id)) {
+ DRV_LOG(ERR, "Failed to modify QP to INIT state(%u).",
+ rte_errno);
+ return -1;
+ }
+ if (mlx5_devx_cmd_modify_qp_state(qp->qp_obj, MLX5_CMD_OP_INIT2RTR_QP,
+ qp->qp_obj->id)) {
+ DRV_LOG(ERR, "Failed to modify QP to RTR state(%u).",
+ rte_errno);
+ return -1;
+ }
+ if (mlx5_devx_cmd_modify_qp_state(qp->qp_obj, MLX5_CMD_OP_RTR2RTS_QP,
+ qp->qp_obj->id)) {
+ DRV_LOG(ERR, "Failed to modify QP to RTS state(%u).",
+ rte_errno);
+ return -1;
+ }
+ return 0;
+}
+
+static int
+mlx5_crypto_queue_pair_setup(struct rte_cryptodev *dev, uint16_t qp_id,
+const struct rte_cryptodev_qp_conf *qp_conf,
+int socket_id)
+{
+ struct mlx5_crypto_priv *priv = dev->data->dev_private;
+ struct mlx5_devx_qp_attr attr = {0};
+ struct mlx5_crypto_qp *qp;
+ uint16_t log_nb_desc = rte_log2_u32(qp_conf->nb_descriptors);
+ uint32_t umem_size = RTE_BIT32(log_nb_desc) *
+ MLX5_CRYPTO_WQE_SET_SIZE +
+ sizeof(*qp->db_rec) * 2;
+ uint32_t alloc_size = sizeof(*qp);
+ struct mlx5_devx_cq_attr cq_attr = {
+ .uar_page_id = mlx5_os_get_devx_uar_page_id(priv->uar),
+ };
+
+ alloc_size = RTE_ALIGN(alloc_size, RTE_CACHE_LINE_SIZE);
+ alloc_size += sizeof(struct rte_crypto_op *) * RTE_BIT32(log_nb_desc);
+ qp = rte_zmalloc_socket(__func__, alloc_size, RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (qp == NULL) {
+ DRV_LOG(ERR, "Failed to allocate QP memory.");
+ rte_errno = ENOMEM;
+ return -rte_errno;
+ }
+ if (mlx5_devx_cq_create(priv->ctx, &qp->cq_obj, log_nb_desc,
+ &cq_attr, socket_id) != 0) {
+ DRV_LOG(ERR, "Failed to create CQ.");
+ goto error;
+ }
+ qp->umem_buf = rte_zmalloc_socket(__func__, umem_size, 4096, socket_id);
+ if (qp->umem_buf == NULL) {
+ DRV_LOG(ERR, "Failed to allocate QP umem.");
+ rte_errno = ENOMEM;
+ goto error;
+ }
+ qp->umem_obj
[dpdk-dev] [PATCH v2 06/15] crypto/mlx5: add dev stop and start operations
From: Shiri Kuzin
Add the dev_start function that is used to start a configured device.
Add the dev_stop function that is used to stop a configured device.
Both functions set the dev parameter as used and return 0.
Signed-off-by: Shiri Kuzin
Acked-by: Matan Azrad
---
drivers/crypto/mlx5/mlx5_crypto.c | 17 +++--
1 file changed, 15 insertions(+), 2 deletions(-)
diff --git a/drivers/crypto/mlx5/mlx5_crypto.c
b/drivers/crypto/mlx5/mlx5_crypto.c
index 5a14e05d55..495d5448b2 100644
--- a/drivers/crypto/mlx5/mlx5_crypto.c
+++ b/drivers/crypto/mlx5/mlx5_crypto.c
@@ -98,6 +98,19 @@ mlx5_crypto_dev_configure(struct rte_cryptodev *dev,
return 0;
}
+static void
+mlx5_crypto_dev_stop(struct rte_cryptodev *dev)
+{
+ RTE_SET_USED(dev);
+}
+
+static int
+mlx5_crypto_dev_start(struct rte_cryptodev *dev)
+{
+ RTE_SET_USED(dev);
+ return 0;
+}
+
static int
mlx5_crypto_dev_close(struct rte_cryptodev *dev)
{
@@ -306,8 +319,8 @@ mlx5_crypto_queue_pair_setup(struct rte_cryptodev *dev,
uint16_t qp_id,
static struct rte_cryptodev_ops mlx5_crypto_ops = {
.dev_configure = mlx5_crypto_dev_configure,
- .dev_start = NULL,
- .dev_stop = NULL,
+ .dev_start = mlx5_crypto_dev_start,
+ .dev_stop = mlx5_crypto_dev_stop,
.dev_close = mlx5_crypto_dev_close,
.dev_infos_get = mlx5_crypto_dev_infos_get,
.stats_get = NULL,
--
2.25.1
[dpdk-dev] [PATCH v2 07/15] crypto/mlx5: add memory region management
From: Shiri Kuzin
Mellanox user space drivers don't deal with physical addresses as part
of a memory protection mechanism.
The device translates the given virtual address to a physical address
using the given memory key as an address space identifier.
That's why any mbuf virtual address is moved directly to the HW
descriptor(WQE).
The mapping between the virtual address to the physical address is saved
in MR configured by the kernel to the HW.
Each MR has a key that should also be moved to the WQE by the SW.
When the SW sees an unmapped address, it extends the address range and
creates a MR using a system call.
Add memory region cache management:
- 2 level cache per queue-pair - no locks.
- 1 shared cache between all the queues using a lock.
Using this way, the MR key search per data-path address is optimized.
Signed-off-by: Shiri Kuzin
Acked-by: Matan Azrad
---
drivers/crypto/mlx5/mlx5_crypto.c | 20
drivers/crypto/mlx5/mlx5_crypto.h | 3 +++
2 files changed, 23 insertions(+)
diff --git a/drivers/crypto/mlx5/mlx5_crypto.c
b/drivers/crypto/mlx5/mlx5_crypto.c
index 495d5448b2..79e3d3ee45 100644
--- a/drivers/crypto/mlx5/mlx5_crypto.c
+++ b/drivers/crypto/mlx5/mlx5_crypto.c
@@ -209,6 +209,7 @@ mlx5_crypto_queue_pair_release(struct rte_cryptodev *dev,
uint16_t qp_id)
claim_zero(mlx5_glue->devx_umem_dereg(qp->umem_obj));
if (qp->umem_buf != NULL)
rte_free(qp->umem_buf);
+ mlx5_mr_btree_free(&qp->mr_ctrl.cache_bh);
mlx5_devx_cq_destroy(&qp->cq_obj);
rte_free(qp);
dev->data->queue_pairs[qp_id] = NULL;
@@ -288,6 +289,13 @@ mlx5_crypto_queue_pair_setup(struct rte_cryptodev *dev,
uint16_t qp_id,
DRV_LOG(ERR, "Failed to register QP umem.");
goto error;
}
+ if (mlx5_mr_btree_init(&qp->mr_ctrl.cache_bh, MLX5_MR_BTREE_CACHE_N,
+ priv->dev_config.socket_id) != 0) {
+ DRV_LOG(ERR, "Cannot allocate MR Btree for qp %u.",
+ (uint32_t)qp_id);
+ rte_errno = ENOMEM;
+ goto error;
+ }
attr.pd = priv->pdn;
attr.uar_index = mlx5_os_get_devx_uar_page_id(priv->uar);
attr.cqn = qp->cq_obj.cq->id;
@@ -476,6 +484,17 @@ mlx5_crypto_pci_probe(struct rte_pci_driver *pci_drv,
claim_zero(mlx5_glue->close_device(priv->ctx));
return -1;
}
+ if (mlx5_mr_btree_init(&priv->mr_scache.cache,
+MLX5_MR_BTREE_CACHE_N * 2, rte_socket_id()) != 0) {
+ DRV_LOG(ERR, "Failed to allocate shared cache MR memory.");
+ mlx5_crypto_hw_global_release(priv);
+ rte_cryptodev_pmd_destroy(priv->crypto_dev);
+ claim_zero(mlx5_glue->close_device(priv->ctx));
+ rte_errno = ENOMEM;
+ return -rte_errno;
+ }
+ priv->mr_scache.reg_mr_cb = mlx5_common_verbs_reg_mr;
+ priv->mr_scache.dereg_mr_cb = mlx5_common_verbs_dereg_mr;
pthread_mutex_lock(&priv_list_lock);
TAILQ_INSERT_TAIL(&mlx5_crypto_priv_list, priv, next);
pthread_mutex_unlock(&priv_list_lock);
@@ -495,6 +514,7 @@ mlx5_crypto_pci_remove(struct rte_pci_device *pdev)
TAILQ_REMOVE(&mlx5_crypto_priv_list, priv, next);
pthread_mutex_unlock(&priv_list_lock);
if (priv) {
+ mlx5_mr_release_cache(&priv->mr_scache);
mlx5_crypto_hw_global_release(priv);
rte_cryptodev_pmd_destroy(priv->crypto_dev);
claim_zero(mlx5_glue->close_device(priv->ctx));
diff --git a/drivers/crypto/mlx5/mlx5_crypto.h
b/drivers/crypto/mlx5/mlx5_crypto.h
index f5313b89f2..397267d249 100644
--- a/drivers/crypto/mlx5/mlx5_crypto.h
+++ b/drivers/crypto/mlx5/mlx5_crypto.h
@@ -12,6 +12,7 @@
#include
#include
+#include
#define MLX5_CRYPTO_DEK_HTABLE_SZ (1 << 11)
#define MLX5_CRYPTO_KEY_LENGTH 80
@@ -27,6 +28,7 @@ struct mlx5_crypto_priv {
struct ibv_pd *pd;
struct mlx5_hlist *dek_hlist; /* Dek hash list. */
struct rte_cryptodev_config dev_config;
+ struct mlx5_mr_share_cache mr_scache; /* Global shared MR cache. */
};
struct mlx5_crypto_qp {
@@ -36,6 +38,7 @@ struct mlx5_crypto_qp {
void *umem_buf;
volatile uint32_t *db_rec;
struct rte_crypto_op **ops;
+ struct mlx5_mr_ctrl mr_ctrl;
};
struct mlx5_crypto_dek {
--
2.25.1
[dpdk-dev] [PATCH v2 08/15] crypto/mlx5: create login object using DevX
From: Shiri Kuzin
To work with crypto engines that are marked with wrapped_import_method,
a login session is required.
A crypto login object needs to be created using DevX.
The crypto login object contains:
- The credential pointer.
- The import_KEK pointer to be used for all secured information
communicated in crypto commands (key fields), including the
provided credential in this command.
- The credential secret, wrapped by the import_KEK indicated in
this command. Size includes 8 bytes IV for wrapping.
Added devargs for the required login values:
- wcs_file - path to the file containing the credential.
- import_kek_id - the import KEK pointer.
- credential_id - the credential pointer.
Create the login DevX object in pci_probe function and destroy it in
pci_remove.
Destroying the crypto login object means logout.
Signed-off-by: Shiri Kuzin
Acked-by: Matan Azrad
---
drivers/crypto/mlx5/mlx5_crypto.c | 103 ++
drivers/crypto/mlx5/mlx5_crypto.h | 7 ++
2 files changed, 110 insertions(+)
diff --git a/drivers/crypto/mlx5/mlx5_crypto.c
b/drivers/crypto/mlx5/mlx5_crypto.c
index 79e3d3ee45..44038f0e05 100644
--- a/drivers/crypto/mlx5/mlx5_crypto.c
+++ b/drivers/crypto/mlx5/mlx5_crypto.c
@@ -402,6 +402,101 @@ mlx5_crypto_hw_global_prepare(struct mlx5_crypto_priv
*priv)
return 0;
}
+
+static int
+mlx5_crypto_args_check_handler(const char *key, const char *val, void *opaque)
+{
+ struct mlx5_crypto_devarg_params *devarg_prms = opaque;
+ struct mlx5_devx_crypto_login_attr *attr = &devarg_prms->login_attr;
+ unsigned long tmp;
+ FILE *file;
+ int ret;
+ int i;
+
+ if (strcmp(key, "class") == 0)
+ return 0;
+ if (strcmp(key, "wcs_file") == 0) {
+ file = fopen(val, "rb");
+ if (file == NULL) {
+ rte_errno = ENOTSUP;
+ return -rte_errno;
+ }
+ for (i = 0 ; i < MLX5_CRYPTO_CREDENTIAL_SIZE ; i++) {
+ ret = fscanf(file, "%02hhX", &attr->credential[i]);
+ if (ret <= 0) {
+ fclose(file);
+ DRV_LOG(ERR,
+ "Failed to read credential from file.");
+ rte_errno = EINVAL;
+ return -rte_errno;
+ }
+ }
+ fclose(file);
+ devarg_prms->login_devarg = true;
+ return 0;
+ }
+ errno = 0;
+ tmp = strtoul(val, NULL, 0);
+ if (errno) {
+ DRV_LOG(WARNING, "%s: \"%s\" is an invalid integer.", key, val);
+ return -errno;
+ }
+ if (strcmp(key, "import_kek_id") == 0)
+ attr->session_import_kek_ptr = (uint32_t)tmp;
+ else if (strcmp(key, "credential_id") == 0)
+ attr->credential_pointer = (uint32_t)tmp;
+ else
+ DRV_LOG(WARNING, "Invalid key %s.", key);
+ return 0;
+}
+
+static struct mlx5_devx_obj *
+mlx5_crypto_config_login(struct rte_devargs *devargs,
+struct ibv_context *ctx)
+{
+ /*
+* Set credential pointer and session import KEK pointer to a default
+* value of 0.
+*/
+ struct mlx5_crypto_devarg_params login = {
+ .login_devarg = false,
+ .login_attr = {
+ .credential_pointer = 0,
+ .session_import_kek_ptr = 0,
+ }
+ };
+ struct rte_kvargs *kvlist;
+
+ if (devargs == NULL) {
+ DRV_LOG(ERR,
+ "No login devargs in order to enable crypto operations in the device.");
+ rte_errno = EINVAL;
+ return NULL;
+ }
+ kvlist = rte_kvargs_parse(devargs->args, NULL);
+ if (kvlist == NULL) {
+ DRV_LOG(ERR, "Failed to parse devargs.");
+ rte_errno = EINVAL;
+ return NULL;
+ }
+ if (rte_kvargs_process(kvlist, NULL, mlx5_crypto_args_check_handler,
+ &login) != 0) {
+ DRV_LOG(ERR, "Devargs handler function Failed.");
+ rte_kvargs_free(kvlist);
+ rte_errno = EINVAL;
+ return NULL;
+ }
+ rte_kvargs_free(kvlist);
+ if (login.login_devarg == false) {
+ DRV_LOG(ERR,
+ "No login credential devarg in order to enable crypto operations "
+ "in the device.");
+ rte_errno = EINVAL;
+ return NULL;
+ }
+ retur
[dpdk-dev] [PATCH v2 09/15] crypto/mlx5: adjust to the multiple data unit API
From: Shiri Kuzin
In AES-XTS the data to be encrypted\decrypted does not have to be
in multiples of 16B size, the unit of data is called data-unit.
As a result of patch [1] a new field is added to the cipher capability,
called dataunit_set, where the devices can report the range of
supported data-unit sizes.
The new field enables saving the data-unit size in the session
structure to the block size pointer variable in order to support
several data-unit sizes.
[1] https://www.mail-archive.com/dev@dpdk.org/msg205337.html
Signed-off-by: Shiri Kuzin
---
drivers/crypto/mlx5/mlx5_crypto.c | 23 +++
1 file changed, 23 insertions(+)
diff --git a/drivers/crypto/mlx5/mlx5_crypto.c
b/drivers/crypto/mlx5/mlx5_crypto.c
index 44038f0e05..1dcebce04c 100644
--- a/drivers/crypto/mlx5/mlx5_crypto.c
+++ b/drivers/crypto/mlx5/mlx5_crypto.c
@@ -48,6 +48,11 @@ struct mlx5_crypto_session {
* bsf_size, bsf_p_type, encryption_order and encryption standard,
* saved in big endian format.
*/
+ uint32_t bsp_res;
+ /*
+* crypto_block_size_pointer and reserved 24 bits saved in big endian
+* format.
+*/
uint32_t iv_offset:16;
/* Starting point for Initialisation Vector. */
struct mlx5_crypto_dek *dek; /* Pointer to dek struct. */
@@ -171,6 +176,24 @@ mlx5_crypto_sym_session_configure(struct rte_cryptodev
*dev,
MLX5_BSF_P_TYPE_CRYPTO << MLX5_BSF_P_TYPE_OFFSET |
encryption_order << MLX5_ENCRYPTION_ORDER_OFFSET |
MLX5_ENCRYPTION_STANDARD_AES_XTS);
+ switch (xform->cipher.dataunit_len) {
+ case 0:
+ sess_private_data->bsp_res = 0;
+ break;
+ case 512:
+ sess_private_data->bsp_res = rte_cpu_to_be_32
+((uint32_t)MLX5_BLOCK_SIZE_512B <<
+MLX5_BLOCK_SIZE_OFFSET);
+ break;
+ case 4096:
+ sess_private_data->bsp_res = rte_cpu_to_be_32
+((uint32_t)MLX5_BLOCK_SIZE_4096B <<
+MLX5_BLOCK_SIZE_OFFSET);
+ break;
+ default:
+ DRV_LOG(ERR, "Cipher data unit length is not supported.");
+ return -ENOTSUP;
+ }
sess_private_data->iv_offset = cipher->iv.offset;
sess_private_data->dek_id =
rte_cpu_to_be_32(sess_private_data->dek->obj->id &
--
2.25.1
[dpdk-dev] [PATCH v2 10/15] crypto/mlx5: add keytag device argument
From: Suanming Mou
A keytag is a piece of data encrypted together with a DEK.
When a DEK is referenced by an MKEY.bsf through its index, the keytag is
also supplied in the BSF as plaintext. The HW will decrypt the DEK (and
the attached keytag) and will fail the operation if the keytags don't
match.
This commit adds the configuration of the keytag with devargs.
Signed-off-by: Suanming Mou
Signed-off-by: Matan Azrad
---
drivers/crypto/mlx5/mlx5_crypto.c | 50 +--
drivers/crypto/mlx5/mlx5_crypto.h | 3 +-
2 files changed, 30 insertions(+), 23 deletions(-)
diff --git a/drivers/crypto/mlx5/mlx5_crypto.c
b/drivers/crypto/mlx5/mlx5_crypto.c
index 1dcebce04c..9f270e4382 100644
--- a/drivers/crypto/mlx5/mlx5_crypto.c
+++ b/drivers/crypto/mlx5/mlx5_crypto.c
@@ -468,56 +468,52 @@ mlx5_crypto_args_check_handler(const char *key, const
char *val, void *opaque)
attr->session_import_kek_ptr = (uint32_t)tmp;
else if (strcmp(key, "credential_id") == 0)
attr->credential_pointer = (uint32_t)tmp;
+ else if (strcmp(key, "keytag") == 0)
+ devarg_prms->keytag = tmp;
else
DRV_LOG(WARNING, "Invalid key %s.", key);
return 0;
}
-static struct mlx5_devx_obj *
-mlx5_crypto_config_login(struct rte_devargs *devargs,
-struct ibv_context *ctx)
+static int
+mlx5_crypto_parse_devargs(struct rte_devargs *devargs,
+ struct mlx5_crypto_devarg_params *devarg_prms)
{
- /*
-* Set credential pointer and session import KEK pointer to a default
-* value of 0.
-*/
- struct mlx5_crypto_devarg_params login = {
- .login_devarg = false,
- .login_attr = {
- .credential_pointer = 0,
- .session_import_kek_ptr = 0,
- }
- };
+ struct mlx5_devx_crypto_login_attr *attr = &devarg_prms->login_attr;
struct rte_kvargs *kvlist;
+ /* Default values. */
+ attr->credential_pointer = 0;
+ attr->session_import_kek_ptr = 0;
+ devarg_prms->keytag = 0;
if (devargs == NULL) {
DRV_LOG(ERR,
"No login devargs in order to enable crypto operations in the device.");
rte_errno = EINVAL;
- return NULL;
+ return -1;
}
kvlist = rte_kvargs_parse(devargs->args, NULL);
if (kvlist == NULL) {
DRV_LOG(ERR, "Failed to parse devargs.");
rte_errno = EINVAL;
- return NULL;
+ return -1;
}
if (rte_kvargs_process(kvlist, NULL, mlx5_crypto_args_check_handler,
- &login) != 0) {
+ devarg_prms) != 0) {
DRV_LOG(ERR, "Devargs handler function Failed.");
rte_kvargs_free(kvlist);
rte_errno = EINVAL;
- return NULL;
+ return -1;
}
rte_kvargs_free(kvlist);
- if (login.login_devarg == false) {
+ if (devarg_prms->login_devarg == false) {
DRV_LOG(ERR,
"No login credential devarg in order to enable crypto operations "
"in the device.");
rte_errno = EINVAL;
- return NULL;
+ return -1;
}
- return mlx5_devx_cmd_create_crypto_login_obj(ctx, &login.login_attr);
+ return 0;
}
/**
@@ -543,6 +539,7 @@ mlx5_crypto_pci_probe(struct rte_pci_driver *pci_drv,
struct ibv_context *ctx;
struct mlx5_devx_obj *login;
struct mlx5_crypto_priv *priv;
+ struct mlx5_crypto_devarg_params devarg_prms = { 0 };
struct mlx5_hca_attr attr = { 0 };
struct rte_cryptodev_pmd_init_params init_params = {
.name = "",
@@ -551,6 +548,8 @@ mlx5_crypto_pci_probe(struct rte_pci_driver *pci_drv,
.max_nb_queue_pairs =
RTE_CRYPTODEV_PMD_DEFAULT_MAX_NB_QUEUE_PAIRS,
};
+ int ret;
+
RTE_SET_USED(pci_drv);
if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
DRV_LOG(ERR, "Non-primary process type is not supported.");
@@ -580,7 +579,13 @@ mlx5_crypto_pci_probe(struct rte_pci_driver *pci_drv,
rte_errno = ENOTSUP;
return -ENOTSUP;
}
- login = mlx5_crypto_config_login(pci_dev->device.devargs, ctx);
+ ret = mlx5_crypto_parse_devargs(pci_dev->device.devargs, &devarg_prms);
+ if (ret) {
+ DRV_LOG(ERR, "Failed to parse devargs.");
+ return -rte_errno;
+ }
+ login = mlx5_devx_cmd_create_crypto_login_obj(ctx,
+
[dpdk-dev] [PATCH v2 11/15] crypto/mlx5: add maximum segments device argument
From: Suanming Mou
The mlx5 HW crypto operations are done by attaching crypto property
to a memory region. Once done, every access to the memory via the
crypto-enabled memory region will result with in-line encryption or
decryption of the data.
As a result, the design choice is to provide two types of WQEs. One
is UMR WQE which sets the crypto property and the other is rdma write
WQE which sends DMA command to copy data from local MR to remote MR.
The size of the WQEs will be defined by a new devarg called
max_segs_num.
This devarg also defines the maximum segments in mbuf chain that will be
supported for crypto operations.
Signed-off-by: Suanming Mou
Signed-off-by: Matan Azrad
---
drivers/crypto/mlx5/mlx5_crypto.c | 35 +++
drivers/crypto/mlx5/mlx5_crypto.h | 7 +++
2 files changed, 38 insertions(+), 4 deletions(-)
diff --git a/drivers/crypto/mlx5/mlx5_crypto.c
b/drivers/crypto/mlx5/mlx5_crypto.c
index 9f270e4382..5e8e7b63d0 100644
--- a/drivers/crypto/mlx5/mlx5_crypto.c
+++ b/drivers/crypto/mlx5/mlx5_crypto.c
@@ -21,6 +21,7 @@
#define MLX5_CRYPTO_DRIVER_NAME mlx5_crypto
#define MLX5_CRYPTO_LOG_NAME pmd.crypto.mlx5
#define MLX5_CRYPTO_MAX_QPS 1024
+#define MLX5_CRYPTO_MAX_SEGS 56
TAILQ_HEAD(mlx5_crypto_privs, mlx5_crypto_priv) mlx5_crypto_priv_list =
TAILQ_HEAD_INITIALIZER(mlx5_crypto_priv_list);
@@ -464,14 +465,24 @@ mlx5_crypto_args_check_handler(const char *key, const
char *val, void *opaque)
DRV_LOG(WARNING, "%s: \"%s\" is an invalid integer.", key, val);
return -errno;
}
- if (strcmp(key, "import_kek_id") == 0)
+ if (strcmp(key, "max_segs_num") == 0) {
+ if (!tmp || tmp > MLX5_CRYPTO_MAX_SEGS) {
+ DRV_LOG(WARNING, "Invalid max_segs_num: %d, should"
+ " be less than %d.",
+ (uint32_t)tmp, MLX5_CRYPTO_MAX_SEGS);
+ rte_errno = EINVAL;
+ return -rte_errno;
+ }
+ devarg_prms->max_segs_num = (uint32_t)tmp;
+ } else if (strcmp(key, "import_kek_id") == 0) {
attr->session_import_kek_ptr = (uint32_t)tmp;
- else if (strcmp(key, "credential_id") == 0)
+ } else if (strcmp(key, "credential_id") == 0) {
attr->credential_pointer = (uint32_t)tmp;
- else if (strcmp(key, "keytag") == 0)
+ } else if (strcmp(key, "keytag") == 0) {
devarg_prms->keytag = tmp;
- else
+ } else {
DRV_LOG(WARNING, "Invalid key %s.", key);
+ }
return 0;
}
@@ -486,6 +497,7 @@ mlx5_crypto_parse_devargs(struct rte_devargs *devargs,
attr->credential_pointer = 0;
attr->session_import_kek_ptr = 0;
devarg_prms->keytag = 0;
+ devarg_prms->max_segs_num = 8;
if (devargs == NULL) {
DRV_LOG(ERR,
"No login devargs in order to enable crypto operations in the device.");
@@ -626,6 +638,21 @@ mlx5_crypto_pci_probe(struct rte_pci_driver *pci_drv,
priv->mr_scache.reg_mr_cb = mlx5_common_verbs_reg_mr;
priv->mr_scache.dereg_mr_cb = mlx5_common_verbs_dereg_mr;
priv->keytag = rte_cpu_to_be_64(devarg_prms.keytag);
+ priv->max_segs_num = devarg_prms.max_segs_num;
+ priv->umr_wqe_size = sizeof(struct mlx5_wqe_umr_bsf_seg) +
+sizeof(struct mlx5_umr_wqe) +
+RTE_ALIGN(priv->max_segs_num, 4) *
+sizeof(struct mlx5_wqe_dseg);
+ priv->rdmw_wqe_size = sizeof(struct mlx5_rdma_write_wqe) +
+ sizeof(struct mlx5_wqe_dseg) *
+ (priv->max_segs_num <= 2 ? 2 : 2 +
+ RTE_ALIGN(priv->max_segs_num - 2, 4));
+ priv->wqe_set_size = priv->umr_wqe_size + priv->rdmw_wqe_size;
+ priv->wqe_stride = (priv->umr_wqe_size + priv->rdmw_wqe_size) /
+ MLX5_SEND_WQE_BB;
+ priv->max_rdmaw_klm_n = (priv->rdmw_wqe_size -
+sizeof(struct mlx5_rdma_write_wqe)) /
+sizeof(struct mlx5_wqe_dseg);
pthread_mutex_lock(&priv_list_lock);
TAILQ_INSERT_TAIL(&mlx5_crypto_priv_list, priv, next);
pthread_mutex_unlock(&priv_list_lock);
diff --git a/drivers/crypto/mlx5/mlx5_crypto.h
b/drivers/crypto/mlx5/mlx5_crypto.h
index 34c65f9a24..81452bd700 100644
--- a/drivers/crypto/mlx5/mlx5_crypto.h
+++ b/drivers/crypto/mlx5/mlx5_crypto.h
@@ -25,12 +25,18 @@ struct mlx5_crypto_priv {
struct rte_cryptodev *crypto_dev;
void *uar; /* U
[dpdk-dev] [PATCH v2 12/15] crypto/mlx5: add WQE set initialization
From: Suanming Mou
Currently, HW handles the WQEs much faster than the software,
Using the constant WQE set layout can initialize most of the WQE
segments in advanced, and software only needs to configure very
limited segments in datapath. This accelerates the software WQE
organize in datapath.
This commit initializes the fixed WQE set segments.
Signed-off-by: Suanming Mou
Signed-off-by: Matan Azrad
---
drivers/crypto/mlx5/mlx5_crypto.c | 83 +--
drivers/crypto/mlx5/mlx5_crypto.h | 10 +++-
2 files changed, 87 insertions(+), 6 deletions(-)
diff --git a/drivers/crypto/mlx5/mlx5_crypto.c
b/drivers/crypto/mlx5/mlx5_crypto.c
index 5e8e7b63d0..e8f68eb115 100644
--- a/drivers/crypto/mlx5/mlx5_crypto.c
+++ b/drivers/crypto/mlx5/mlx5_crypto.c
@@ -268,6 +268,69 @@ mlx5_crypto_qp2rts(struct mlx5_crypto_qp *qp)
return 0;
}
+static void
+mlx5_crypto_qp_init(struct mlx5_crypto_priv *priv, struct mlx5_crypto_qp *qp)
+{
+ uint32_t i;
+
+ for (i = 0 ; i < qp->entries_n; i++) {
+ struct mlx5_wqe_cseg *cseg = RTE_PTR_ADD(qp->umem_buf, i *
+priv->wqe_set_size);
+ struct mlx5_wqe_umr_cseg *ucseg = (struct mlx5_wqe_umr_cseg *)
+(cseg + 1);
+ struct mlx5_wqe_umr_bsf_seg *bsf =
+ (struct mlx5_wqe_umr_bsf_seg *)(RTE_PTR_ADD(cseg,
+ priv->umr_wqe_size)) - 1;
+ struct mlx5_wqe_rseg *rseg;
+
+ /* Init UMR WQE. */
+ cseg->sq_ds = rte_cpu_to_be_32((qp->qp_obj->id << 8) |
+(priv->umr_wqe_size / MLX5_WSEG_SIZE));
+ cseg->flags = RTE_BE32(MLX5_COMP_ONLY_FIRST_ERR <<
+ MLX5_COMP_MODE_OFFSET);
+ cseg->misc = rte_cpu_to_be_32(qp->mkey[i]->id);
+ ucseg->if_cf_toe_cq_res = RTE_BE32(1u << MLX5_UMRC_IF_OFFSET);
+ ucseg->mkey_mask = RTE_BE64(1u << 0); /* Mkey length bit. */
+ ucseg->ko_to_bs = rte_cpu_to_be_32
+ ((RTE_ALIGN(priv->max_segs_num, 4u) <<
+MLX5_UMRC_KO_OFFSET) | (4 << MLX5_UMRC_TO_BS_OFFSET));
+ bsf->keytag = priv->keytag;
+ /* Init RDMA WRITE WQE. */
+ cseg = RTE_PTR_ADD(cseg, priv->umr_wqe_size);
+ cseg->flags = RTE_BE32((MLX5_COMP_ALWAYS <<
+ MLX5_COMP_MODE_OFFSET) |
+ MLX5_WQE_CTRL_INITIATOR_SMALL_FENCE);
+ rseg = (struct mlx5_wqe_rseg *)(cseg + 1);
+ rseg->rkey = rte_cpu_to_be_32(qp->mkey[i]->id);
+ }
+}
+
+static int
+mlx5_crypto_indirect_mkeys_prepare(struct mlx5_crypto_priv *priv,
+ struct mlx5_crypto_qp *qp)
+{
+ struct mlx5_umr_wqe *umr;
+ uint32_t i;
+ struct mlx5_devx_mkey_attr attr = {
+ .pd = priv->pdn,
+ .umr_en = 1,
+ .crypto_en = 1,
+ .set_remote_rw = 1,
+ .klm_num = RTE_ALIGN(priv->max_segs_num, 4),
+ };
+
+ for (umr = (struct mlx5_umr_wqe *)qp->umem_buf, i = 0;
+ i < qp->entries_n; i++, umr = RTE_PTR_ADD(umr, priv->wqe_set_size)) {
+ attr.klm_array = (struct mlx5_klm *)&umr->kseg[0];
+ qp->mkey[i] = mlx5_devx_cmd_mkey_create(priv->ctx, &attr);
+ if (!qp->mkey[i]) {
+ DRV_LOG(ERR, "Failed to allocate indirect mkey.");
+ return -1;
+ }
+ }
+ return 0;
+}
+
static int
mlx5_crypto_queue_pair_setup(struct rte_cryptodev *dev, uint16_t qp_id,
const struct rte_cryptodev_qp_conf *qp_conf,
@@ -278,7 +341,7 @@ mlx5_crypto_queue_pair_setup(struct rte_cryptodev *dev,
uint16_t qp_id,
struct mlx5_crypto_qp *qp;
uint16_t log_nb_desc = rte_log2_u32(qp_conf->nb_descriptors);
uint32_t umem_size = RTE_BIT32(log_nb_desc) *
- MLX5_CRYPTO_WQE_SET_SIZE +
+ priv->wqe_set_size +
sizeof(*qp->db_rec) * 2;
uint32_t alloc_size = sizeof(*qp);
struct mlx5_devx_cq_attr cq_attr = {
@@ -286,7 +349,9 @@ mlx5_crypto_queue_pair_setup(struct rte_cryptodev *dev,
uint16_t qp_id,
};
alloc_size = RTE_ALIGN(alloc_size, RTE_CACHE_LINE_SIZE);
- alloc_size += sizeof(struct rte_crypto_op *) * RTE_BIT32(log_nb_desc);
+ alloc_size += (sizeof(struct rte_crypto_op *) +
+ sizeof(struct mlx5_devx_obj *)) *
+ RTE_BIT32(log_
[dpdk-dev] [PATCH v2 13/15] crypto/mlx5: add enqueue and dequeue operations
From: Suanming Mou
The crypto operations are done with the WQE set which contains
one UMR WQE and one rdma write WQE. Most segments of the WQE
set are initialized properly during queue setup, only limited
segments are initialized according to the crypto detail in the
datapath process.
This commit adds the enquue and dequeue operations and updates
the WQE set segments accordingly.
Signed-off-by: Suanming Mou
Signed-off-by: Matan Azrad
---
drivers/crypto/mlx5/mlx5_crypto.c | 243 +-
drivers/crypto/mlx5/mlx5_crypto.h | 3 +
2 files changed, 241 insertions(+), 5 deletions(-)
diff --git a/drivers/crypto/mlx5/mlx5_crypto.c
b/drivers/crypto/mlx5/mlx5_crypto.c
index e8f68eb115..08a8c1e925 100644
--- a/drivers/crypto/mlx5/mlx5_crypto.c
+++ b/drivers/crypto/mlx5/mlx5_crypto.c
@@ -268,6 +268,239 @@ mlx5_crypto_qp2rts(struct mlx5_crypto_qp *qp)
return 0;
}
+static __rte_noinline uint32_t
+mlx5_crypto_get_block_size(struct rte_crypto_op *op)
+{
+ uint32_t bl = op->sym->cipher.data.length;
+
+ switch (bl) {
+ case (1 << 20):
+ return RTE_BE32(MLX5_BLOCK_SIZE_1MB << MLX5_BLOCK_SIZE_OFFSET);
+ case (1 << 12):
+ return RTE_BE32(MLX5_BLOCK_SIZE_4096B <<
+ MLX5_BLOCK_SIZE_OFFSET);
+ case (1 << 9):
+ return RTE_BE32(MLX5_BLOCK_SIZE_512B << MLX5_BLOCK_SIZE_OFFSET);
+ default:
+ DRV_LOG(ERR, "Unknown block size: %u.", bl);
+ return UINT32_MAX;
+ }
+}
+
+static __rte_always_inline uint32_t
+mlx5_crypto_klm_set(struct mlx5_crypto_priv *priv, struct mlx5_crypto_qp *qp,
+ struct rte_mbuf *mbuf, struct mlx5_wqe_dseg *klm,
+ uint32_t offset, uint32_t *remain)
+{
+ uint32_t data_len = (rte_pktmbuf_data_len(mbuf) - offset);
+ uintptr_t addr = rte_pktmbuf_mtod_offset(mbuf, uintptr_t, offset);
+
+ if (data_len > *remain)
+ data_len = *remain;
+ *remain -= data_len;
+ klm->bcount = rte_cpu_to_be_32(data_len);
+ klm->pbuf = rte_cpu_to_be_64(addr);
+ klm->lkey = mlx5_mr_addr2mr_bh(priv->pd, 0,
+ &priv->mr_scache, &qp->mr_ctrl, addr,
+ !!(mbuf->ol_flags & EXT_ATTACHED_MBUF));
+ return klm->lkey;
+
+}
+
+static __rte_always_inline uint32_t
+mlx5_crypto_klms_set(struct mlx5_crypto_priv *priv, struct mlx5_crypto_qp *qp,
+struct rte_crypto_op *op, struct rte_mbuf *mbuf,
+struct mlx5_wqe_dseg *klm)
+{
+ uint32_t remain_len = op->sym->cipher.data.length;
+ uint32_t nb_segs = mbuf->nb_segs;
+ uint32_t klm_n = 1;
+
+ /* First mbuf needs to take the cipher offset. */
+ if (unlikely(mlx5_crypto_klm_set(priv, qp, mbuf, klm,
+op->sym->cipher.data.offset, &remain_len) == UINT32_MAX)) {
+ op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+ return 0;
+ }
+ while (remain_len) {
+ nb_segs--;
+ mbuf = mbuf->next;
+ if (unlikely(mbuf == NULL || nb_segs == 0)) {
+ op->status = RTE_CRYPTO_OP_STATUS_INVALID_ARGS;
+ return 0;
+ }
+ if (unlikely(mlx5_crypto_klm_set(priv, qp, mbuf, klm, 0,
+&remain_len) == UINT32_MAX)) {
+ op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+ return 0;
+ }
+ klm_n++;
+ }
+ return klm_n;
+}
+
+static __rte_always_inline int
+mlx5_crypto_wqe_set(struct mlx5_crypto_priv *priv,
+struct mlx5_crypto_qp *qp,
+struct rte_crypto_op *op,
+struct mlx5_umr_wqe *umr)
+{
+ struct mlx5_crypto_session *sess = get_sym_session_private_data
+ (op->sym->session, mlx5_crypto_driver_id);
+ struct mlx5_wqe_cseg *cseg = &umr->ctr;
+ struct mlx5_wqe_mkey_cseg *mkc = &umr->mkc;
+ struct mlx5_wqe_dseg *klms = &umr->kseg[0];
+ struct mlx5_wqe_umr_bsf_seg *bsf = ((struct mlx5_wqe_umr_bsf_seg *)
+ RTE_PTR_ADD(umr, priv->umr_wqe_size)) - 1;
+ uint16_t nop_ds;
+ /* Set UMR WQE. */
+ uint32_t klm_n = mlx5_crypto_klms_set(priv, qp, op,
+ op->sym->m_dst ? op->sym->m_dst : op->sym->m_src, klms);
+
+ if (unlikely(klm_n == 0))
+ return 0;
+ bsf->bs_bpt_eo_es = sess->bs_bpt_eo_es;
+ if (unlikely(!sess->bsp_res)) {
+ bsf->bsp_res = mlx5_crypto_get_block_size(op);
+ if (unlikely(bsf->bsp_res == UINT32_MAX)) {
+ op->sta
[dpdk-dev] [PATCH v2 14/15] crypto/mlx5: add statistic get and reset operations
From: Suanming Mou
This commit adds mlx5 crypto statistic get and reset operations.
Signed-off-by: Suanming Mou
Signed-off-by: Matan Azrad
---
drivers/crypto/mlx5/mlx5_crypto.c | 39 ---
1 file changed, 36 insertions(+), 3 deletions(-)
diff --git a/drivers/crypto/mlx5/mlx5_crypto.c
b/drivers/crypto/mlx5/mlx5_crypto.c
index 08a8c1e925..af8985939e 100644
--- a/drivers/crypto/mlx5/mlx5_crypto.c
+++ b/drivers/crypto/mlx5/mlx5_crypto.c
@@ -437,11 +437,14 @@ mlx5_crypto_enqueue_burst(void *queue_pair, struct
rte_crypto_op **ops,
do {
op = *ops++;
umr = RTE_PTR_ADD(qp->umem_buf, priv->wqe_set_size * qp->pi);
- if (unlikely(mlx5_crypto_wqe_set(priv, qp, op, umr) == 0))
+ if (unlikely(mlx5_crypto_wqe_set(priv, qp, op, umr) == 0)) {
+ qp->stats.enqueue_err_count++;
break;
+ }
qp->ops[qp->pi] = op;
qp->pi = (qp->pi + 1) & mask;
} while (--remain);
+ qp->stats.enqueued_count += nb_ops;
rte_io_wmb();
qp->db_rec[MLX5_SND_DBR] = rte_cpu_to_be_32(qp->db_pi);
rte_wmb();
@@ -458,6 +461,7 @@ mlx5_crypto_cqe_err_handle(struct mlx5_crypto_qp *qp,
struct rte_crypto_op *op)
&qp->cq_obj.cqes[idx];
op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+ qp->stats.dequeue_err_count++;
DRV_LOG(ERR, "CQE ERR:%x.\n", rte_be_to_cpu_32(cqe->syndrome));
}
@@ -497,6 +501,7 @@ mlx5_crypto_dequeue_burst(void *queue_pair, struct
rte_crypto_op **ops,
if (likely(i != 0)) {
rte_io_wmb();
qp->cq_obj.db_rec[0] = rte_cpu_to_be_32(qp->ci);
+ qp->stats.dequeued_count += i;
}
return i;
}
@@ -655,14 +660,42 @@ mlx5_crypto_queue_pair_setup(struct rte_cryptodev *dev,
uint16_t qp_id,
return -1;
}
+static void
+mlx5_crypto_stats_get(struct rte_cryptodev *dev,
+ struct rte_cryptodev_stats *stats)
+{
+ int qp_id;
+
+ for (qp_id = 0; qp_id < dev->data->nb_queue_pairs; qp_id++) {
+ struct mlx5_crypto_qp *qp = dev->data->queue_pairs[qp_id];
+
+ stats->enqueued_count += qp->stats.enqueued_count;
+ stats->dequeued_count += qp->stats.dequeued_count;
+ stats->enqueue_err_count += qp->stats.enqueue_err_count;
+ stats->dequeue_err_count += qp->stats.dequeue_err_count;
+ }
+}
+
+static void
+mlx5_crypto_stats_reset(struct rte_cryptodev *dev)
+{
+ int qp_id;
+
+ for (qp_id = 0; qp_id < dev->data->nb_queue_pairs; qp_id++) {
+ struct mlx5_crypto_qp *qp = dev->data->queue_pairs[qp_id];
+
+ memset(&qp->stats, 0, sizeof(qp->stats));
+ }
+}
+
static struct rte_cryptodev_ops mlx5_crypto_ops = {
.dev_configure = mlx5_crypto_dev_configure,
.dev_start = mlx5_crypto_dev_start,
.dev_stop = mlx5_crypto_dev_stop,
.dev_close = mlx5_crypto_dev_close,
.dev_infos_get = mlx5_crypto_dev_infos_get,
- .stats_get = NULL,
- .stats_reset= NULL,
+ .stats_get = mlx5_crypto_stats_get,
+ .stats_reset= mlx5_crypto_stats_reset,
.queue_pair_setup = mlx5_crypto_queue_pair_setup,
.queue_pair_release = mlx5_crypto_queue_pair_release,
.sym_session_get_size = mlx5_crypto_sym_session_get_size,
--
2.25.1
[dpdk-dev] [PATCH v2 15/15] crypto/mlx5: set feature flags and capabilities
From: Shiri Kuzin Add the supported capabilities to the crypto driver. Add supported feature flags. Add crypto driver documentation. Signed-off-by: Shiri Kuzin Signed-off-by: Matan Azrad --- doc/guides/cryptodevs/features/mlx5.ini | 37 ++ doc/guides/cryptodevs/index.rst | 1 + doc/guides/cryptodevs/mlx5.rst | 152 doc/guides/rel_notes/release_21_05.rst | 5 + drivers/crypto/mlx5/mlx5_crypto.c | 39 +- 5 files changed, 231 insertions(+), 3 deletions(-) create mode 100644 doc/guides/cryptodevs/features/mlx5.ini create mode 100644 doc/guides/cryptodevs/mlx5.rst diff --git a/doc/guides/cryptodevs/features/mlx5.ini b/doc/guides/cryptodevs/features/mlx5.ini new file mode 100644 index 00..a89526add0 --- /dev/null +++ b/doc/guides/cryptodevs/features/mlx5.ini @@ -0,0 +1,37 @@ +; +; Features of a mlx5 crypto driver. +; +; Refer to default.ini for the full list of available PMD features. +; +[Features] +Symmetric crypto = Y +HW Accelerated = Y +In Place SGL = Y +OOP SGL In SGL Out = Y +OOP SGL In LB Out = Y +OOP LB In SGL Out = Y +OOP LB In LB Out = Y +Cipher multiple data units = Y +Cipher wrapped key = Y + +; +; Supported crypto algorithms of a mlx5 crypto driver. +; +[Cipher] +AES XTS (128) = Y +AES XTS (256) = Y + +; +; Supported authentication algorithms of a mlx5 crypto driver. +; +[Auth] + +; +; Supported AEAD algorithms of a mlx5 crypto driver. +; +[AEAD] + +; +; Supported Asymmetric algorithms of a mlx5 crypto driver. +; +[Asymmetric] diff --git a/doc/guides/cryptodevs/index.rst b/doc/guides/cryptodevs/index.rst index 279f56a002..747409c441 100644 --- a/doc/guides/cryptodevs/index.rst +++ b/doc/guides/cryptodevs/index.rst @@ -22,6 +22,7 @@ Crypto Device Drivers octeontx octeontx2 openssl +mlx5 mvsam nitrox null diff --git a/doc/guides/cryptodevs/mlx5.rst b/doc/guides/cryptodevs/mlx5.rst new file mode 100644 index 00..4ccec78be8 --- /dev/null +++ b/doc/guides/cryptodevs/mlx5.rst @@ -0,0 +1,152 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright 2021 Mellanox Technologies, Ltd + +.. include:: + +MLX5 Crypto Driver +== + +The MLX5 crypto driver library +(**librte_crypto_mlx5**) provides support for **Mellanox ConnectX-6** +family adapters. + +Overview + + +The device can provide disk encryption services, allowing data encryption +and decryption towards a disk. Having all encryption/decryption +operations done in a single device can reduce cost and overheads of the related +FIPS certification, as ConnectX-6 is FIPS 140-2 level-2 ready. +The encryption cipher is AES-XTS of 256/512 key size. + +MKEY is a memory region object in the hardware, that holds address translation information and +attributes per memory area. Its ID must be tied to addresses provided to the hardware. +The encryption operations are performed with MKEY read\write transactions, when +the MKEY is configured to perform crypto operations. + +The encryption does not require text to be aligned to the AES block size (128b). + +In order to move the device to crypto operational mode, credential and KEK +(Key Encrypting Key) should be set as the first step. +The credential will be used by the software in order to perform crypto login, and the KEK is +the AES Key Wrap Algorithm (rfc3394) key that will be used for sensitive data +wrapping. +The credential and the AES-XTS keys should be provided to the hardware, as ciphertext +encrypted by the KEK. + +A keytag (64 bits) should be appended to the AES-XTS keys (before wrapping), +and will be validated when the hardware attempts to access it. + +For security reasons and to increase robustness, this driver only deals with virtual +memory addresses. The way resources allocations are handled by the kernel, +combined with hardware specifications that allow handling virtual memory +addresses directly, ensure that DPDK applications cannot access random +physical memory (or memory that does not belong to the current process). + +The PMD uses libibverbs and libmlx5 to access the device firmware or to +access the hardware components directly. +There are different levels of objects and bypassing abilities. +To get the best performances: + +- Verbs is a complete high-level generic API. +- Direct Verbs is a device-specific API. +- DevX allows to access firmware objects. + +Enabling librte_crypto_mlx5 causes DPDK applications to be linked against +libibverbs. + +Mellanox mlx5 PCI device can be probed by a number of different PCI devices, such as +net / vDPA / RegEx. To select the crypto PMD, ``class=crypto`` +should be specified as a device parameter. The crypto device can be probed and +used with other Mellanox classes by adding more options in the class. +For example: ``class=net:crypto`` will probe both the net PMD and the crypto +PMD. + +When crypto engines are defined to work in wrapped import method
Re: [dpdk-dev] [PATCH v2 07/16] common/mlx5: fix cypto bsf attr
Hi Raslan
See below , need to squash this to
" common/mlx5: adjust DevX MKEY fields for crypto"
The one below it...missed that, sorry.
From: Matan Azrad
> From: Suanming Mou
>
> This commit should be squash to crypto en commit.
>
> Signed-off-by: Suanming Mou
> ---
> drivers/common/mlx5/mlx5_devx_cmds.c | 4
> 1 file changed, 4 insertions(+)
>
> diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c
> b/drivers/common/mlx5/mlx5_devx_cmds.c
> index 68a10b149a..7e3c8b55fa 100644
> --- a/drivers/common/mlx5/mlx5_devx_cmds.c
> +++ b/drivers/common/mlx5/mlx5_devx_cmds.c
> @@ -278,6 +278,10 @@ mlx5_devx_cmd_mkey_create(void *ctx,
> MLX5_SET64(mkc, mkc, start_addr, attr->addr);
> MLX5_SET64(mkc, mkc, len, attr->size);
> MLX5_SET(mkc, mkc, crypto_en, attr->crypto_en);
> + if (attr->crypto_en) {
> + MLX5_SET(mkc, mkc, bsf_en, attr->crypto_en);
> + MLX5_SET(mkc, mkc, bsf_octword_size, 4);
> + }
> mkey->obj = mlx5_glue->devx_obj_create(ctx, in, in_size_dw * 4,
> out,
> sizeof(out));
> if (!mkey->obj) {
> --
> 2.25.1
Re: [dpdk-dev] [PATCH v2 00/15] drivers: introduce mlx5 crypto PMD
From: Matan Azrad > Add a new PMD for Nvidia devices- crypto PMD. > The crypto PMD will be supported on Nvidia ConnectX6 The crypto PMD will > add the support of encryption and decryption using the AES-XTS symmetric > algorithm. > The crypto PMD requires rdma-core and uses mlx5 DevX. > > v2: > Add data-path part. > > Shiri Kuzin (10): > drivers: introduce mlx5 crypto PMD > crypto/mlx5: add DEK object management > crypto/mlx5: support session operations > crypto/mlx5: add basic operations > crypto/mlx5: support queue pairs operations > crypto/mlx5: add dev stop and start operations > crypto/mlx5: add memory region management > crypto/mlx5: create login object using DevX > crypto/mlx5: adjust to the multiple data unit API > crypto/mlx5: set feature flags and capabilities > > Suanming Mou (5): > crypto/mlx5: add keytag device argument > crypto/mlx5: add maximum segments device argument > crypto/mlx5: add WQE set initialization > crypto/mlx5: add enqueue and dequeue operations > crypto/mlx5: add statistic get and reset operations Series-acked-by: Matan Azrad > MAINTAINERS |4 + > doc/guides/cryptodevs/features/mlx5.ini | 37 + > doc/guides/cryptodevs/index.rst |1 + > doc/guides/cryptodevs/mlx5.rst | 152 > doc/guides/rel_notes/release_21_05.rst |5 + > drivers/common/mlx5/mlx5_common_pci.c | 14 + > drivers/common/mlx5/mlx5_common_pci.h | 21 +- > drivers/crypto/meson.build |1 + > drivers/crypto/mlx5/meson.build | 27 + > drivers/crypto/mlx5/mlx5_crypto.c | 1092 +++ > drivers/crypto/mlx5/mlx5_crypto.h | 92 ++ > drivers/crypto/mlx5/mlx5_crypto_dek.c | 136 +++ > drivers/crypto/mlx5/mlx5_crypto_utils.h | 19 + > drivers/crypto/mlx5/version.map |3 + > 14 files changed, 1594 insertions(+), 10 deletions(-) create mode 100644 > doc/guides/cryptodevs/features/mlx5.ini > create mode 100644 doc/guides/cryptodevs/mlx5.rst create mode 100644 > drivers/crypto/mlx5/meson.build create mode 100644 > drivers/crypto/mlx5/mlx5_crypto.c create mode 100644 > drivers/crypto/mlx5/mlx5_crypto.h create mode 100644 > drivers/crypto/mlx5/mlx5_crypto_dek.c > create mode 100644 drivers/crypto/mlx5/mlx5_crypto_utils.h > create mode 100644 drivers/crypto/mlx5/version.map > > -- > 2.25.1
Re: [dpdk-dev] [PATCH v2 00/16] mlx5 common part for crypto driver
From: Matan Azrad > The crypto PMD will be supported on Nvidia ConnectX6 The crypto PMD will > add the support of encryption and decryption using the AES-XTS symmetric > algorithm. > The crypto PMD requires rdma-core and uses mlx5 DevX. > This is the mlx5 common part that added support for DevX commands > needed for crypto driver. > Also includes PRM definitions. > > v2: > Add data-path part. > > Dekel Peled (12): > common/mlx5: remove redundant spaces in header file > common/mlx5: update GENEVE TLV OPT obj name > common/mlx5: optimize read of general obj type caps > common/mlx5: add HCA cap for AES-XTS crypto > common/mlx5: support general object DEK create op > common/mlx5: adjust DevX MKEY fields for crypto > common/mlx5: support general obj IMPORT KEK create > common/mlx5: support general obj CRYPTO LOGIN create > common/mlx5: add crypto BSF struct and defines > common/mlx5: support general obj CREDENTIAL create > common/mlx5: add crypto register structs and defs > common/mlx5: support register write access > > Shiri Kuzin (2): > common/mlx5: share hash list tool > common/mlx5: share get ib device match function > > Suanming Mou (2): > common/mlx5: fix cypto bsf attr > common/mlx5: add UMR and RDMA write WQE defines Series-acked-by: Matan Azrad > drivers/common/mlx5/linux/mlx5_common_os.c | 30 +- > drivers/common/mlx5/linux/mlx5_common_os.h | 5 + > drivers/common/mlx5/linux/mlx5_nl.c | 2 +- > drivers/common/mlx5/meson.build | 15 +- > drivers/common/mlx5/mlx5_common.c| 2 +- > drivers/common/mlx5/mlx5_common.h| 2 + > drivers/common/mlx5/mlx5_common_devx.c | 2 +- > drivers/common/mlx5/mlx5_common_log.h| 21 + > drivers/common/mlx5/mlx5_common_mp.c | 2 +- > drivers/common/mlx5/mlx5_common_mr.c | 2 +- > drivers/common/mlx5/mlx5_common_pci.c| 4 +- > drivers/common/mlx5/mlx5_common_utils.c | 221 +++ > drivers/common/mlx5/mlx5_common_utils.h | 202 +- > drivers/common/mlx5/mlx5_devx_cmds.c | 312 +- > drivers/common/mlx5/mlx5_devx_cmds.h | 66 ++ > drivers/common/mlx5/mlx5_malloc.c| 2 +- > drivers/common/mlx5/mlx5_prm.h | 610 +-- > drivers/common/mlx5/version.map | 24 +- > drivers/common/mlx5/windows/mlx5_common_os.c | 2 +- > drivers/common/mlx5/windows/mlx5_glue.c | 2 +- > drivers/compress/mlx5/mlx5_compress.c| 30 +- > drivers/net/mlx5/mlx5_utils.c| 209 --- > drivers/net/mlx5/mlx5_utils.h| 194 +- > drivers/regex/mlx5/mlx5_regex.c | 30 +- > drivers/vdpa/mlx5/mlx5_vdpa.c| 32 +- > 25 files changed, 1317 insertions(+), 706 deletions(-) create mode 100644 > drivers/common/mlx5/mlx5_common_log.h > create mode 100644 drivers/common/mlx5/mlx5_common_utils.c > > -- > 2.25.1
[dpdk-dev] [PATCH v2] vdpa/mlx5: improve interrupt management
The driver should notify the guest for each traffic burst detected by CQ
polling.
The CQ polling trigger is defined by `event_mode` device argument,
either by busy polling on all the CQs or by blocked call to HW
completion event using DevX channel.
Also, the polling event modes can move to blocked call when the
traffic rate is low.
The current blocked call uses the EAL interrupt API suffering a lot
of overhead in the API management and serve all the drivers and
libraries using only single thread.
Use blocking FD of the DevX channel in order to do blocked call
directly by the DevX channel FD mechanism.
Signed-off-by: Matan Azrad
Acked-by: Xueming Li
---
doc/guides/vdpadevs/mlx5.rst| 8 +-
drivers/vdpa/mlx5/mlx5_vdpa.c | 8 +-
drivers/vdpa/mlx5/mlx5_vdpa.h | 8 +-
drivers/vdpa/mlx5/mlx5_vdpa_event.c | 304 +++-
4 files changed, 130 insertions(+), 198 deletions(-)
v2:
Rebased on top of latest updates by Thomas.
diff --git a/doc/guides/vdpadevs/mlx5.rst b/doc/guides/vdpadevs/mlx5.rst
index 1f2ae6fb92..9b2f9f12c7 100644
--- a/doc/guides/vdpadevs/mlx5.rst
+++ b/doc/guides/vdpadevs/mlx5.rst
@@ -129,10 +129,10 @@ Driver options
- ``no_traffic_time`` parameter [int]
- A nonzero value defines the traffic off time, in seconds, that moves the
- driver to no-traffic mode. In this mode the timer events are stopped and
- interrupts are configured to the device in order to notify traffic for the
- driver. Default value is 2s.
+ A nonzero value defines the traffic off time, in polling cycle time units,
+ that moves the driver to no-traffic mode. In this mode the polling is stopped
+ and interrupts are configured to the device in order to notify traffic for
the
+ driver. Default value is 16.
- ``event_core`` parameter [int]
diff --git a/drivers/vdpa/mlx5/mlx5_vdpa.c b/drivers/vdpa/mlx5/mlx5_vdpa.c
index 898e50f807..619e31d61c 100644
--- a/drivers/vdpa/mlx5/mlx5_vdpa.c
+++ b/drivers/vdpa/mlx5/mlx5_vdpa.c
@@ -44,7 +44,7 @@
#define MLX5_VDPA_MAX_RETRIES 20
#define MLX5_VDPA_USEC 1000
-#define MLX5_VDPA_DEFAULT_NO_TRAFFIC_TIME_S 2LLU
+#define MLX5_VDPA_DEFAULT_NO_TRAFFIC_MAX 16LLU
TAILQ_HEAD(mlx5_vdpa_privs, mlx5_vdpa_priv) priv_list =
TAILQ_HEAD_INITIALIZER(priv_list);
@@ -632,7 +632,7 @@ mlx5_vdpa_args_check_handler(const char *key, const char
*val, void *opaque)
} else if (strcmp(key, "event_us") == 0) {
priv->event_us = (uint32_t)tmp;
} else if (strcmp(key, "no_traffic_time") == 0) {
- priv->no_traffic_time_s = (uint32_t)tmp;
+ priv->no_traffic_max = (uint32_t)tmp;
} else if (strcmp(key, "event_core") == 0) {
if (tmp >= (unsigned long)n_cores)
DRV_LOG(WARNING, "Invalid event_core %s.", val);
@@ -658,7 +658,7 @@ mlx5_vdpa_config_get(struct rte_devargs *devargs, struct
mlx5_vdpa_priv *priv)
priv->event_mode = MLX5_VDPA_EVENT_MODE_FIXED_TIMER;
priv->event_us = 0;
priv->event_core = -1;
- priv->no_traffic_time_s = MLX5_VDPA_DEFAULT_NO_TRAFFIC_TIME_S;
+ priv->no_traffic_max = MLX5_VDPA_DEFAULT_NO_TRAFFIC_MAX;
if (devargs == NULL)
return;
kvlist = rte_kvargs_parse(devargs->args, NULL);
@@ -671,7 +671,7 @@ mlx5_vdpa_config_get(struct rte_devargs *devargs, struct
mlx5_vdpa_priv *priv)
priv->event_us = MLX5_VDPA_DEFAULT_TIMER_STEP_US;
DRV_LOG(DEBUG, "event mode is %d.", priv->event_mode);
DRV_LOG(DEBUG, "event_us is %u us.", priv->event_us);
- DRV_LOG(DEBUG, "no traffic time is %u s.", priv->no_traffic_time_s);
+ DRV_LOG(DEBUG, "no traffic max is %u.", priv->no_traffic_max);
}
/**
diff --git a/drivers/vdpa/mlx5/mlx5_vdpa.h b/drivers/vdpa/mlx5/mlx5_vdpa.h
index d93b430c97..722c72b65e 100644
--- a/drivers/vdpa/mlx5/mlx5_vdpa.h
+++ b/drivers/vdpa/mlx5/mlx5_vdpa.h
@@ -120,16 +120,13 @@ struct mlx5_vdpa_priv {
TAILQ_ENTRY(mlx5_vdpa_priv) next;
uint8_t configured;
pthread_mutex_t vq_config_lock;
- uint64_t last_traffic_tic;
+ uint64_t no_traffic_counter;
pthread_t timer_tid;
- pthread_mutex_t timer_lock;
- pthread_cond_t timer_cond;
- volatile uint8_t timer_on;
int event_mode;
int event_core; /* Event thread cpu affinity core. */
uint32_t event_us;
uint32_t timer_delay_us;
- uint32_t no_traffic_time_s;
+ uint32_t no_traffic_max;
uint8_t hw_latency_mode; /* Hardware CQ moderation mode. */
uint16_t hw_max_latency_us; /* Hardware CQ moderation period in usec. */
uint16_t hw_max_pending_comp; /* Hardware CQ moderation counter. */
@@ -146,7 +143,6 @@ struct mlx5_vdpa_priv {
struct mlx5dv_devx_event_channel *eventc;
Re: [dpdk-dev] [PATCH] vdpa/mlx5: improve interrupt management
From: Maxime Coquelin
> Hi Matan,
>
> On 4/11/21 11:07 AM, Thomas Monjalon wrote:
> > 07/04/2021 08:49, Xia, Chenbo:
> >>> Signed-off-by: Matan Azrad
> >>> Acked-by: Xueming Li
> >>
> >> Patch applied to next-virtio/main with conflict resolved.
> >>
> >> Thanks!
> >
> > Sorry it cannot be pulled in the main tree because it breaks
> > compilation with musl libc.
> >
> > It can be fixed in the same way as in
> > 41b5a7a8494e ("vdpa/mlx5: replace pthread functions unavailable in
> > musl") The function pthread_attr_setaffinity_np() must be replaced
> > with
> > pthread_setaffinity_np() if moved after the call to pthread_create().
> >
> > We could add a checkpatch check for forbidden functions like
> > pthread_yield, pthread_attr_setaffinity_np and forbidden
> > includes:
> > ,
> >
> >
>
> Could you please send a new version taking care of Thomas' request.
>
> Thanks,
> Maxime
Done.
Re: [dpdk-dev] [PATCH v2 12/16] common/mlx5: share get ib device match function
Hi Thomas
From: Thomas Monjalon
> 29/04/2021 17:43, Matan Azrad:
> > From: Shiri Kuzin
> >
> > The get_ib_device_match function iterates over the list of ib devices
> > returned by the get_device_list glue function and returns the ib
> > device matching the provided address.
> >
> > Since this function is in use by several drivers, in this patch we
> > share the function in common part.
> >
> > Signed-off-by: Shiri Kuzin
> > Acked-by: Matan Azrad
> > ---
> > --- a/drivers/common/mlx5/linux/mlx5_common_os.c
> > +++ b/drivers/common/mlx5/linux/mlx5_common_os.c
> > +struct ibv_device *
> > +mlx5_os_get_ib_device_match(struct rte_pci_addr *addr)
>
> It would be better as mlx5_get_pci_ibv_device() in
> drivers/common/mlx5/mlx5_common_pci.c
> Just need to forward declare struct ibv_device in
> drivers/common/mlx5/mlx5_common.h This is what is done internally for
> sub-functions.
struct ibv_device Is linux only, defined in Verbs library.
So, no need it in OS common place.
> > --- a/drivers/common/mlx5/mlx5_common.h
> > +++ b/drivers/common/mlx5/mlx5_common.h
> > @@ -18,6 +18,7 @@
> >
> > #include "mlx5_prm.h"
> > #include "mlx5_devx_cmds.h"
> > +#include "mlx5_common_os.h"
> >
> > /* Reported driver name. */
> > #define MLX5_PCI_DRIVER_NAME "mlx5_pci"
> > @@ -215,6 +216,7 @@ enum mlx5_class {
> > MLX5_CLASS_VDPA = RTE_BIT64(1),
> > MLX5_CLASS_REGEX = RTE_BIT64(2),
> > MLX5_CLASS_COMPRESS = RTE_BIT64(3),
> > + MLX5_CLASS_CRYPTO = RTE_BIT64(4),
> > };
>
> This change has nothing to do in this patch I think.
>
Yes, will handle in v3.
> > --- a/drivers/common/mlx5/version.map
> > +++ b/drivers/common/mlx5/version.map
> > @@ -111,6 +111,7 @@ INTERNAL {
> > mlx5_os_reg_mr;
> > mlx5_os_umem_reg;
> > mlx5_os_umem_dereg;
> > + mlx5_os_get_ib_device_match; # WINDOWS_NO_EXPORT
>
> Please keep alphabetical order.
Sure.
Re: [dpdk-dev] [PATCH v2 15/16] common/mlx5: support register write access
From: Thomas Monjalon
> 29/04/2021 17:43, Matan Azrad:
> > From: Dekel Peled
> >
> > This patch adds support of write operation to NIC registers.
> >
> > Signed-off-by: Dekel Peled
> > Acked-by: Matan Azrad
> > ---
> > --- a/drivers/common/mlx5/version.map
> > +++ b/drivers/common/mlx5/version.map
> > @@ -13,8 +13,17 @@ INTERNAL {
> > mlx5_dev_to_pci_addr; # WINDOWS_NO_EXPORT
> >
> > mlx5_devx_cmd_alloc_pd;
> > + mlx5_devx_alloc_uar;
>
> Please keep alphabetical order.
>
> > +
> > mlx5_devx_cmd_create_cq;
> > + mlx5_devx_cmd_create_credential_obj;
> > + mlx5_devx_cmd_create_crypto_login_obj;
> > + mlx5_devx_cmd_create_dek_obj;
> > mlx5_devx_cmd_create_flex_parser;
> > + mlx5_devx_cmd_create_flow_hit_aso_obj;
> > + mlx5_devx_cmd_create_flow_meter_aso_obj;
> > + mlx5_devx_cmd_create_geneve_tlv_option;
> > + mlx5_devx_cmd_create_import_kek_obj;
> > mlx5_devx_cmd_create_qp;
> > mlx5_devx_cmd_create_rq;
> > mlx5_devx_cmd_create_rqt;
> > @@ -24,13 +33,6 @@ INTERNAL {
> > mlx5_devx_cmd_create_tis;
> > mlx5_devx_cmd_create_virtio_q_counters; # WINDOWS_NO_EXPORT
> > mlx5_devx_cmd_create_virtq;
> > - mlx5_devx_cmd_create_flow_hit_aso_obj;
> > - mlx5_devx_cmd_create_flow_meter_aso_obj;
> > - mlx5_devx_cmd_create_geneve_tlv_option;
> > -mlx5_devx_cmd_create_dek_obj;
> > -mlx5_devx_cmd_create_import_kek_obj;
> > -mlx5_devx_cmd_create_credential_obj;
> > -mlx5_devx_cmd_create_crypto_login_obj;
>
> Are they changes of previous patches?
> It seems some cleaning is required in previous patches.
Yes, this one tried to do the cleaning, I will handle it in the first patch (of
this series) accessing this file.
> > mlx5_devx_cmd_destroy;
> > mlx5_devx_cmd_flow_counter_alloc;
> > mlx5_devx_cmd_flow_counter_query; @@ -51,12 +53,17 @@
> INTERNAL {
> > mlx5_devx_cmd_queue_counter_alloc; # WINDOWS_NO_EXPORT
> > mlx5_devx_cmd_queue_counter_query; # WINDOWS_NO_EXPORT
> > mlx5_devx_cmd_register_read;
> > + mlx5_devx_cmd_register_write;
> > mlx5_devx_cmd_wq_query; # WINDOWS_NO_EXPORT
> > mlx5_devx_get_out_command_status;
> > mlx5_devx_alloc_uar; # WINDOWS_NO_EXPORT
> >
> > +
> > mlx5_devx_cq_create;
>
> There is an extra blank line here.
>
>
[dpdk-dev] [PATCH v3 00/15] mlx5 common part for crypto driver
The crypto PMD will be supported on Nvidia ConnectX6. The crypto PMD will add the support of encryption and decryption using the AES-XTS symmetric algorithm. The crypto PMD requires rdma-core and uses mlx5 DevX. This is the mlx5 common part that added support for DevX commands needed for crypto driver. Also includes PRM definitions. v2: Add data-path part. v3: Squash fix commit to the one before it. Change get IB device function name. Rearrange common version.map file. Moved class definition to the driver seires. Dekel Peled (12): common/mlx5: remove redundant spaces in header file common/mlx5: update GENEVE TLV OPT obj name common/mlx5: optimize read of general obj type caps common/mlx5: add HCA cap for AES-XTS crypto common/mlx5: support general object DEK create op common/mlx5: adjust DevX MKEY fields for crypto common/mlx5: support general obj IMPORT KEK create common/mlx5: support general obj CRYPTO LOGIN create common/mlx5: add crypto BSF struct and defines common/mlx5: support general obj CREDENTIAL create common/mlx5: add crypto register structs and defs common/mlx5: support register write access Shiri Kuzin (2): common/mlx5: share hash list tool common/mlx5: share get ib device match function Suanming Mou (1): common/mlx5: add UMR and RDMA write WQE defines drivers/common/mlx5/linux/mlx5_common_os.c | 30 +- drivers/common/mlx5/linux/mlx5_common_os.h | 5 + drivers/common/mlx5/linux/mlx5_nl.c | 2 +- drivers/common/mlx5/meson.build | 15 +- drivers/common/mlx5/mlx5_common.c| 2 +- drivers/common/mlx5/mlx5_common.h| 1 + drivers/common/mlx5/mlx5_common_devx.c | 2 +- drivers/common/mlx5/mlx5_common_log.h| 21 + drivers/common/mlx5/mlx5_common_mp.c | 2 +- drivers/common/mlx5/mlx5_common_mr.c | 2 +- drivers/common/mlx5/mlx5_common_pci.c| 4 +- drivers/common/mlx5/mlx5_common_utils.c | 221 +++ drivers/common/mlx5/mlx5_common_utils.h | 202 +- drivers/common/mlx5/mlx5_devx_cmds.c | 312 +- drivers/common/mlx5/mlx5_devx_cmds.h | 66 ++ drivers/common/mlx5/mlx5_malloc.c| 2 +- drivers/common/mlx5/mlx5_prm.h | 610 +-- drivers/common/mlx5/version.map | 66 +- drivers/common/mlx5/windows/mlx5_common_os.c | 2 +- drivers/common/mlx5/windows/mlx5_glue.c | 2 +- drivers/compress/mlx5/mlx5_compress.c| 30 +- drivers/net/mlx5/mlx5_utils.c| 209 --- drivers/net/mlx5/mlx5_utils.h| 194 +- drivers/regex/mlx5/mlx5_regex.c | 30 +- drivers/vdpa/mlx5/mlx5_vdpa.c| 32 +- 25 files changed, 1337 insertions(+), 727 deletions(-) create mode 100644 drivers/common/mlx5/mlx5_common_log.h create mode 100644 drivers/common/mlx5/mlx5_common_utils.c -- 2.25.1
[dpdk-dev] [PATCH v3 01/15] common/mlx5: remove redundant spaces in header file
From: Dekel Peled
File drivers/common/mlx5/mlx5_prm.h includes structs representing
data items as defined in PRM document.
Some of these structs were copied as-is from kernel file mlx5_ifc.h.
As result the structs are not all aligned with the same spacing.
This patch removes redundant spaces and new lines from several structs,
to align all structs in mlx5_prm.h to the same format.
Signed-off-by: Dekel Peled
Acked-by: Matan Azrad
---
drivers/common/mlx5/mlx5_prm.h | 334 +++--
1 file changed, 155 insertions(+), 179 deletions(-)
diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h
index efa5ae67bf..da1510ac1e 100644
--- a/drivers/common/mlx5/mlx5_prm.h
+++ b/drivers/common/mlx5/mlx5_prm.h
@@ -953,163 +953,139 @@ enum {
/* Flow counters. */
struct mlx5_ifc_alloc_flow_counter_out_bits {
- u8 status[0x8];
- u8 reserved_at_8[0x18];
- u8 syndrome[0x20];
- u8 flow_counter_id[0x20];
- u8 reserved_at_60[0x20];
+ u8 status[0x8];
+ u8 reserved_at_8[0x18];
+ u8 syndrome[0x20];
+ u8 flow_counter_id[0x20];
+ u8 reserved_at_60[0x20];
};
struct mlx5_ifc_alloc_flow_counter_in_bits {
- u8 opcode[0x10];
- u8 reserved_at_10[0x10];
- u8 reserved_at_20[0x10];
- u8 op_mod[0x10];
- u8 flow_counter_id[0x20];
- u8 reserved_at_40[0x18];
- u8 flow_counter_bulk[0x8];
+ u8 opcode[0x10];
+ u8 reserved_at_10[0x10];
+ u8 reserved_at_20[0x10];
+ u8 op_mod[0x10];
+ u8 flow_counter_id[0x20];
+ u8 reserved_at_40[0x18];
+ u8 flow_counter_bulk[0x8];
};
struct mlx5_ifc_dealloc_flow_counter_out_bits {
- u8 status[0x8];
- u8 reserved_at_8[0x18];
- u8 syndrome[0x20];
- u8 reserved_at_40[0x40];
+ u8 status[0x8];
+ u8 reserved_at_8[0x18];
+ u8 syndrome[0x20];
+ u8 reserved_at_40[0x40];
};
struct mlx5_ifc_dealloc_flow_counter_in_bits {
- u8 opcode[0x10];
- u8 reserved_at_10[0x10];
- u8 reserved_at_20[0x10];
- u8 op_mod[0x10];
- u8 flow_counter_id[0x20];
- u8 reserved_at_60[0x20];
+ u8 opcode[0x10];
+ u8 reserved_at_10[0x10];
+ u8 reserved_at_20[0x10];
+ u8 op_mod[0x10];
+ u8 flow_counter_id[0x20];
+ u8 reserved_at_60[0x20];
};
struct mlx5_ifc_traffic_counter_bits {
- u8 packets[0x40];
- u8 octets[0x40];
+ u8 packets[0x40];
+ u8 octets[0x40];
};
struct mlx5_ifc_query_flow_counter_out_bits {
- u8 status[0x8];
- u8 reserved_at_8[0x18];
- u8 syndrome[0x20];
- u8 reserved_at_40[0x40];
+ u8 status[0x8];
+ u8 reserved_at_8[0x18];
+ u8 syndrome[0x20];
+ u8 reserved_at_40[0x40];
struct mlx5_ifc_traffic_counter_bits flow_statistics[];
};
struct mlx5_ifc_query_flow_counter_in_bits {
- u8 opcode[0x10];
- u8 reserved_at_10[0x10];
- u8 reserved_at_20[0x10];
- u8 op_mod[0x10];
- u8 reserved_at_40[0x20];
- u8 mkey[0x20];
- u8 address[0x40];
- u8 clear[0x1];
- u8 dump_to_memory[0x1];
- u8 num_of_counters[0x1e];
- u8 flow_counter_id[0x20];
+ u8 opcode[0x10];
+ u8 reserved_at_10[0x10];
+ u8 reserved_at_20[0x10];
+ u8 op_mod[0x10];
+ u8 reserved_at_40[0x20];
+ u8 mkey[0x20];
+ u8 address[0x40];
+ u8 clear[0x1];
+ u8 dump_to_memory[0x1];
+ u8 num_of_counters[0x1e];
+ u8 flow_counter_id[0x20];
};
#define MLX5_MAX_KLM_BYTE_COUNT 0x8000u
#define MLX5_MIN_KLM_FIXED_BUFFER_SIZE 0x1000u
-
struct mlx5_ifc_klm_bits {
- u8 byte_count[0x20];
- u8 mkey[0x20];
- u8 address[0x40];
+ u8 byte_count[0x20];
+ u8 mkey[0x20];
+ u8 address[0x40];
};
struct mlx5_ifc_mkc_bits {
- u8 reserved_at_0[0x1];
- u8 free[0x1];
- u8 reserved_at_2[0x1];
- u8 access_mode_4_2[0x3];
- u8 reserved_at_6[0x7];
- u8 relaxed_ordering_write[0x1];
- u8 reserved_at_e[0x1];
- u8 small_fence_on_rdma_read_response[0x1];
- u8 umr_en[0x1];
- u8 a[0x1];
- u8 rw[0x1];
- u8 rr[0x1];
- u8 lw[0x1];
- u8 lr[0x1];
- u8 access_mode_1_0[0x2];
- u8 reserved_at_18[0x8];
-
- u8 qpn[0x18];
- u8 mkey_7_0[0x8];
-
- u8 reserved_at_40[0x20];
-
- u8 length64[0x1];
- u8 bsf_en[0x1];
- u8 sync_umr[0x1
[dpdk-dev] [PATCH v3 02/15] common/mlx5: update GENEVE TLV OPT obj name
From: Dekel Peled
Rename MLX5_OBJ_TYPE_GENEVE_TLV_OPT as
MLX5_GENERAL_OBJ_TYPE_GENEVE_TLV_OPT, to align with other general
objects names.
Signed-off-by: Dekel Peled
Acked-by: Matan Azrad
---
drivers/common/mlx5/mlx5_devx_cmds.c | 2 +-
drivers/common/mlx5/mlx5_prm.h | 4 ++--
2 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c
b/drivers/common/mlx5/mlx5_devx_cmds.c
index 79fff6457c..831175efc5 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -2265,7 +2265,7 @@ mlx5_devx_cmd_create_geneve_tlv_option(void *ctx,
MLX5_SET(general_obj_in_cmd_hdr, hdr, opcode,
MLX5_CMD_OP_CREATE_GENERAL_OBJECT);
MLX5_SET(general_obj_in_cmd_hdr, hdr, obj_type,
- MLX5_OBJ_TYPE_GENEVE_TLV_OPT);
+MLX5_GENERAL_OBJ_TYPE_GENEVE_TLV_OPT);
MLX5_SET(geneve_tlv_option, opt, option_class,
rte_be_to_cpu_16(class));
MLX5_SET(geneve_tlv_option, opt, option_type, type);
diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h
index da1510ac1e..2e5e42f6e9 100644
--- a/drivers/common/mlx5/mlx5_prm.h
+++ b/drivers/common/mlx5/mlx5_prm.h
@@ -1108,7 +1108,7 @@ enum {
#define MLX5_GENERAL_OBJ_TYPES_CAP_FLOW_METER_ASO \
(1ULL << MLX5_GENERAL_OBJ_TYPE_FLOW_METER_ASO)
#define MLX5_GENERAL_OBJ_TYPES_CAP_GENEVE_TLV_OPT \
- (1ULL << MLX5_OBJ_TYPE_GENEVE_TLV_OPT)
+ (1ULL << MLX5_GENERAL_OBJ_TYPE_GENEVE_TLV_OPT)
enum {
MLX5_HCA_CAP_OPMOD_GET_MAX = 0,
@@ -2402,7 +2402,7 @@ struct mlx5_ifc_create_cq_in_bits {
};
enum {
- MLX5_OBJ_TYPE_GENEVE_TLV_OPT = 0x000b,
+ MLX5_GENERAL_OBJ_TYPE_GENEVE_TLV_OPT = 0x000b,
MLX5_GENERAL_OBJ_TYPE_VIRTQ = 0x000d,
MLX5_GENERAL_OBJ_TYPE_VIRTIO_Q_COUNTERS = 0x001c,
MLX5_GENERAL_OBJ_TYPE_FLEX_PARSE_GRAPH = 0x0022,
--
2.25.1
[dpdk-dev] [PATCH v3 03/15] common/mlx5: optimize read of general obj type caps
From: Dekel Peled
General object types support is indicated in bitmap general_obj_types,
which is part of HCA capabilities list.
Currently this bitmap is read multiple times, and each time a different
bit is extracted.
This patch optimizes the code, reading the bitmap once into a local
variable, and then extracting the required bits.
Signed-off-by: Dekel Peled
Acked-by: Matan Azrad
---
drivers/common/mlx5/mlx5_devx_cmds.c | 21 -
1 file changed, 16 insertions(+), 5 deletions(-)
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c
b/drivers/common/mlx5/mlx5_devx_cmds.c
index 831175efc5..a0bf0d3009 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -647,6 +647,7 @@ mlx5_devx_cmd_query_hca_attr(void *ctx,
uint32_t out[MLX5_ST_SZ_DW(query_hca_cap_out)] = {0};
void *hcattr;
int status, syndrome, rc, i;
+ uint64_t general_obj_types_supported = 0;
MLX5_SET(query_hca_cap_in, in, opcode, MLX5_CMD_OP_QUERY_HCA_CAP);
MLX5_SET(query_hca_cap_in, in, op_mod,
@@ -725,12 +726,22 @@ mlx5_devx_cmd_query_hca_attr(void *ctx,
attr->regex = MLX5_GET(cmd_hca_cap, hcattr, regexp);
attr->regexp_num_of_engines = MLX5_GET(cmd_hca_cap, hcattr,
regexp_num_of_engines);
- attr->flow_hit_aso = !!(MLX5_GET64(cmd_hca_cap, hcattr,
- general_obj_types) &
+ /* Read the general_obj_types bitmap and extract the relevant bits. */
+ general_obj_types_supported = MLX5_GET64(cmd_hca_cap, hcattr,
+general_obj_types);
+ attr->vdpa.valid = !!(general_obj_types_supported &
+ MLX5_GENERAL_OBJ_TYPES_CAP_VIRTQ_NET_Q);
+ attr->vdpa.queue_counters_valid =
+ !!(general_obj_types_supported &
+ MLX5_GENERAL_OBJ_TYPES_CAP_VIRTIO_Q_COUNTERS);
+ attr->parse_graph_flex_node =
+ !!(general_obj_types_supported &
+ MLX5_GENERAL_OBJ_TYPES_CAP_PARSE_GRAPH_FLEX_NODE);
+ attr->flow_hit_aso = !!(general_obj_types_supported &
MLX5_GENERAL_OBJ_TYPES_CAP_FLOW_HIT_ASO);
- attr->geneve_tlv_opt = !!(MLX5_GET64(cmd_hca_cap, hcattr,
- general_obj_types) &
- MLX5_GENERAL_OBJ_TYPES_CAP_GENEVE_TLV_OPT);
+ attr->geneve_tlv_opt = !!(general_obj_types_supported &
+ MLX5_GENERAL_OBJ_TYPES_CAP_GENEVE_TLV_OPT);
+ /* Add reading of other GENERAL_OBJ_TYPES_CAP bits above this line. */
attr->log_max_cq = MLX5_GET(cmd_hca_cap, hcattr, log_max_cq);
attr->log_max_qp = MLX5_GET(cmd_hca_cap, hcattr, log_max_qp);
attr->log_max_cq_sz = MLX5_GET(cmd_hca_cap, hcattr, log_max_cq_sz);
--
2.25.1
[dpdk-dev] [PATCH v3 04/15] common/mlx5: add HCA cap for AES-XTS crypto
From: Dekel Peled
Update the PRM structure and HCA capabilities reading, to include
relevant capabilities for AES-XTS crypto.
Signed-off-by: Dekel Peled
Acked-by: Matan Azrad
---
drivers/common/mlx5/mlx5_devx_cmds.c | 3 +++
drivers/common/mlx5/mlx5_devx_cmds.h | 2 ++
drivers/common/mlx5/mlx5_prm.h | 5 -
3 files changed, 9 insertions(+), 1 deletion(-)
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c
b/drivers/common/mlx5/mlx5_devx_cmds.c
index a0bf0d3009..7ca767944e 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -771,6 +771,9 @@ mlx5_devx_cmd_query_hca_attr(void *ctx,
MLX5_GET(cmd_hca_cap, hcattr, umr_indirect_mkey_disabled);
attr->umr_modify_entity_size_disabled =
MLX5_GET(cmd_hca_cap, hcattr, umr_modify_entity_size_disabled);
+ attr->crypto = MLX5_GET(cmd_hca_cap, hcattr, crypto);
+ if (attr->crypto)
+ attr->aes_xts = MLX5_GET(cmd_hca_cap, hcattr, aes_xts);
if (attr->qos.sup) {
MLX5_SET(query_hca_cap_in, in, op_mod,
MLX5_GET_HCA_CAP_OP_MOD_QOS_CAP |
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h
b/drivers/common/mlx5/mlx5_devx_cmds.h
index 870bdb6b30..28ade5bbc4 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.h
+++ b/drivers/common/mlx5/mlx5_devx_cmds.h
@@ -137,6 +137,8 @@ struct mlx5_hca_attr {
uint32_t qp_ts_format:2;
uint32_t regex:1;
uint32_t reg_c_preserve:1;
+ uint32_t crypto:1; /* Crypto engine is supported. */
+ uint32_t aes_xts:1; /* AES-XTS crypto is supported. */
uint32_t regexp_num_of_engines;
uint32_t log_max_ft_sampler_num:8;
uint32_t geneve_tlv_opt;
diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h
index 2e5e42f6e9..a8fbfbb0f5 100644
--- a/drivers/common/mlx5/mlx5_prm.h
+++ b/drivers/common/mlx5/mlx5_prm.h
@@ -1427,7 +1427,10 @@ struct mlx5_ifc_cmd_hca_cap_bits {
u8 sq_ts_format[0x2];
u8 rq_ts_format[0x2];
u8 reserved_at_444[0x1C];
- u8 reserved_at_460[0x10];
+ u8 reserved_at_460[0x8];
+ u8 aes_xts[0x1];
+ u8 crypto[0x1];
+ u8 reserved_at_46a[0x6];
u8 max_num_eqs[0x10];
u8 reserved_at_480[0x3];
u8 log_max_l2_table[0x5];
--
2.25.1
[dpdk-dev] [PATCH v3 05/15] common/mlx5: support general object DEK create op
From: Dekel Peled
Data Encryption Keys (DEKs) are the keys used for data
encryption/decryption operations.
Add reading of DEK support capability.
Add function to create general object type DEK, using DevX API.
Arrange common version.map file in alphabetical order.
Signed-off-by: Dekel Peled
Acked-by: Matan Azrad
---
drivers/common/mlx5/mlx5_devx_cmds.c | 53 +++
drivers/common/mlx5/mlx5_devx_cmds.h | 17 +
drivers/common/mlx5/mlx5_prm.h | 39
drivers/common/mlx5/version.map | 55
4 files changed, 140 insertions(+), 24 deletions(-)
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c
b/drivers/common/mlx5/mlx5_devx_cmds.c
index 7ca767944e..742c82cca4 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -741,6 +741,8 @@ mlx5_devx_cmd_query_hca_attr(void *ctx,
MLX5_GENERAL_OBJ_TYPES_CAP_FLOW_HIT_ASO);
attr->geneve_tlv_opt = !!(general_obj_types_supported &
MLX5_GENERAL_OBJ_TYPES_CAP_GENEVE_TLV_OPT);
+ attr->dek = !!(general_obj_types_supported &
+ MLX5_GENERAL_OBJ_TYPES_CAP_DEK);
/* Add reading of other GENERAL_OBJ_TYPES_CAP bits above this line. */
attr->log_max_cq = MLX5_GET(cmd_hca_cap, hcattr, log_max_cq);
attr->log_max_qp = MLX5_GET(cmd_hca_cap, hcattr, log_max_qp);
@@ -2397,3 +2399,54 @@ mlx5_devx_cmd_queue_counter_query(struct mlx5_devx_obj
*dcs, int clear,
*out_of_buffers = MLX5_GET(query_q_counter_out, out, out_of_buffer);
return 0;
}
+
+/**
+ * Create general object of type DEK using DevX API.
+ *
+ * @param[in] ctx
+ * Context returned from mlx5 open_device() glue function.
+ * @param [in] attr
+ * Pointer to DEK attributes structure.
+ *
+ * @return
+ * The DevX object created, NULL otherwise and rte_errno is set.
+ */
+struct mlx5_devx_obj *
+mlx5_devx_cmd_create_dek_obj(void *ctx, struct mlx5_devx_dek_attr *attr)
+{
+ uint32_t in[MLX5_ST_SZ_DW(create_dek_in)] = {0};
+ uint32_t out[MLX5_ST_SZ_DW(general_obj_out_cmd_hdr)] = {0};
+ struct mlx5_devx_obj *dek_obj = NULL;
+ void *ptr = NULL, *key_addr = NULL;
+
+ dek_obj = mlx5_malloc(MLX5_MEM_ZERO, sizeof(*dek_obj),
+ 0, SOCKET_ID_ANY);
+ if (dek_obj == NULL) {
+ DRV_LOG(ERR, "Failed to allocate DEK object data");
+ rte_errno = ENOMEM;
+ return NULL;
+ }
+ ptr = MLX5_ADDR_OF(create_dek_in, in, hdr);
+ MLX5_SET(general_obj_in_cmd_hdr, ptr, opcode,
+MLX5_CMD_OP_CREATE_GENERAL_OBJECT);
+ MLX5_SET(general_obj_in_cmd_hdr, ptr, obj_type,
+MLX5_GENERAL_OBJ_TYPE_DEK);
+ ptr = MLX5_ADDR_OF(create_dek_in, in, dek);
+ MLX5_SET(dek, ptr, key_size, attr->key_size);
+ MLX5_SET(dek, ptr, has_keytag, attr->has_keytag);
+ MLX5_SET(dek, ptr, key_purpose, attr->key_purpose);
+ MLX5_SET(dek, ptr, pd, attr->pd);
+ MLX5_SET64(dek, ptr, opaque, attr->opaque);
+ key_addr = MLX5_ADDR_OF(dek, ptr, key);
+ memcpy(key_addr, (void *)(attr->key), MLX5_CRYPTO_KEY_MAX_SIZE);
+ dek_obj->obj = mlx5_glue->devx_obj_create(ctx, in, sizeof(in),
+ out, sizeof(out));
+ if (dek_obj->obj == NULL) {
+ rte_errno = errno;
+ DRV_LOG(ERR, "Failed to create DEK obj using DevX.");
+ mlx5_free(dek_obj);
+ return NULL;
+ }
+ dek_obj->id = MLX5_GET(general_obj_out_cmd_hdr, out, obj_id);
+ return dek_obj;
+}
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h
b/drivers/common/mlx5/mlx5_devx_cmds.h
index 28ade5bbc4..b9ff7ab87d 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.h
+++ b/drivers/common/mlx5/mlx5_devx_cmds.h
@@ -139,6 +139,7 @@ struct mlx5_hca_attr {
uint32_t reg_c_preserve:1;
uint32_t crypto:1; /* Crypto engine is supported. */
uint32_t aes_xts:1; /* AES-XTS crypto is supported. */
+ uint32_t dek:1; /* General obj type DEK is supported. */
uint32_t regexp_num_of_engines;
uint32_t log_max_ft_sampler_num:8;
uint32_t geneve_tlv_opt;
@@ -435,6 +436,18 @@ struct mlx5_devx_graph_node_attr {
struct mlx5_devx_graph_arc_attr out[MLX5_GRAPH_NODE_ARC_NUM];
};
+/* Encryption key size is up to 1024 bit, 128 bytes. */
+#define MLX5_CRYPTO_KEY_MAX_SIZE 128
+
+struct mlx5_devx_dek_attr {
+ uint32_t key_size:4;
+ uint32_t has_keytag:1;
+ uint32_t key_purpose:4;
+ uint32_t pd:24;
+ uint64_t opaque;
+ uint8_t key[MLX5_CRYPTO_KEY_MAX_SIZE];
+};
+
/* mlx5_devx_cmds.c */
__rte_internal
@@ -587,4 +600,8 @@ int mlx5_devx_cmd_queue_counter_query(struct mlx5_devx_obj
*dcs, int clear,
__rte_internal
s
[dpdk-dev] [PATCH v3 06/15] common/mlx5: adjust DevX MKEY fields for crypto
From: Dekel Peled
MKEY that will be used for crypto purposes must be created with
crypto_en and remote access attributes.
This patch adds support for them in the DevX MKEY context.
Signed-off-by: Dekel Peled
Acked-by: Matan Azrad
---
drivers/common/mlx5/mlx5_devx_cmds.c | 9 +
drivers/common/mlx5/mlx5_devx_cmds.h | 2 ++
drivers/common/mlx5/mlx5_prm.h | 10 +-
3 files changed, 20 insertions(+), 1 deletion(-)
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c
b/drivers/common/mlx5/mlx5_devx_cmds.c
index 742c82cca4..7e3c8b55fa 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -263,6 +263,10 @@ mlx5_devx_cmd_mkey_create(void *ctx,
MLX5_SET(create_mkey_in, in, pg_access, attr->pg_access);
MLX5_SET(mkc, mkc, lw, 0x1);
MLX5_SET(mkc, mkc, lr, 0x1);
+ if (attr->set_remote_rw) {
+ MLX5_SET(mkc, mkc, rw, 0x1);
+ MLX5_SET(mkc, mkc, rr, 0x1);
+ }
MLX5_SET(mkc, mkc, qpn, 0xff);
MLX5_SET(mkc, mkc, pd, attr->pd);
MLX5_SET(mkc, mkc, mkey_7_0, attr->umem_id & 0xFF);
@@ -273,6 +277,11 @@ mlx5_devx_cmd_mkey_create(void *ctx,
MLX5_SET(mkc, mkc, relaxed_ordering_read, attr->relaxed_ordering_read);
MLX5_SET64(mkc, mkc, start_addr, attr->addr);
MLX5_SET64(mkc, mkc, len, attr->size);
+ MLX5_SET(mkc, mkc, crypto_en, attr->crypto_en);
+ if (attr->crypto_en) {
+ MLX5_SET(mkc, mkc, bsf_en, attr->crypto_en);
+ MLX5_SET(mkc, mkc, bsf_octword_size, 4);
+ }
mkey->obj = mlx5_glue->devx_obj_create(ctx, in, in_size_dw * 4, out,
sizeof(out));
if (!mkey->obj) {
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h
b/drivers/common/mlx5/mlx5_devx_cmds.h
index b9ff7ab87d..600577f18a 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.h
+++ b/drivers/common/mlx5/mlx5_devx_cmds.h
@@ -32,6 +32,8 @@ struct mlx5_devx_mkey_attr {
uint32_t relaxed_ordering_write:1;
uint32_t relaxed_ordering_read:1;
uint32_t umr_en:1;
+ uint32_t crypto_en:2;
+ uint32_t set_remote_rw:1;
struct mlx5_klm *klm_array;
int klm_num;
};
diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h
index bc9f58ad03..25f6f8ff00 100644
--- a/drivers/common/mlx5/mlx5_prm.h
+++ b/drivers/common/mlx5/mlx5_prm.h
@@ -1059,7 +1059,15 @@ struct mlx5_ifc_mkc_bits {
u8 relaxed_ordering_read[0x1];
u8 reserved_at_1da[0x1];
u8 log_page_size[0x5];
- u8 reserved_at_1e0[0x20];
+ u8 reserved_at_1e0[0x3];
+ u8 crypto_en[0x2];
+ u8 reserved_at_1e5[0x1b];
+};
+
+/* Range of values for MKEY context crypto_en field. */
+enum {
+ MLX5_MKEY_CRYPTO_DISABLED = 0x0,
+ MLX5_MKEY_CRYPTO_ENABLED = 0x1,
};
struct mlx5_ifc_create_mkey_out_bits {
--
2.25.1
[dpdk-dev] [PATCH v3 07/15] common/mlx5: support general obj IMPORT KEK create
From: Dekel Peled
IMPORT_KEK object is used to wrap (encrypt) critical security
parameters, such as other keys and credentials, when those need
to be passed between the device and the software.
This patch add support of IMPORT_KEK object create operation.
Add reading of IMPORT_KEK support capability.
Add function to create general object type IMPORT_KEK, using DevX API.
Signed-off-by: Dekel Peled
Acked-by: Matan Azrad
---
drivers/common/mlx5/mlx5_devx_cmds.c | 50
drivers/common/mlx5/mlx5_devx_cmds.h | 13
drivers/common/mlx5/mlx5_prm.h | 18 ++
drivers/common/mlx5/version.map | 1 +
4 files changed, 82 insertions(+)
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c
b/drivers/common/mlx5/mlx5_devx_cmds.c
index 7e3c8b55fa..afef7a5f63 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -752,6 +752,8 @@ mlx5_devx_cmd_query_hca_attr(void *ctx,
MLX5_GENERAL_OBJ_TYPES_CAP_GENEVE_TLV_OPT);
attr->dek = !!(general_obj_types_supported &
MLX5_GENERAL_OBJ_TYPES_CAP_DEK);
+ attr->import_kek = !!(general_obj_types_supported &
+ MLX5_GENERAL_OBJ_TYPES_CAP_IMPORT_KEK);
/* Add reading of other GENERAL_OBJ_TYPES_CAP bits above this line. */
attr->log_max_cq = MLX5_GET(cmd_hca_cap, hcattr, log_max_cq);
attr->log_max_qp = MLX5_GET(cmd_hca_cap, hcattr, log_max_qp);
@@ -2459,3 +2461,51 @@ mlx5_devx_cmd_create_dek_obj(void *ctx, struct
mlx5_devx_dek_attr *attr)
dek_obj->id = MLX5_GET(general_obj_out_cmd_hdr, out, obj_id);
return dek_obj;
}
+
+/**
+ * Create general object of type IMPORT_KEK using DevX API.
+ *
+ * @param[in] ctx
+ * Context returned from mlx5 open_device() glue function.
+ * @param [in] attr
+ * Pointer to IMPORT_KEK attributes structure.
+ *
+ * @return
+ * The DevX object created, NULL otherwise and rte_errno is set.
+ */
+struct mlx5_devx_obj *
+mlx5_devx_cmd_create_import_kek_obj(void *ctx,
+ struct mlx5_devx_import_kek_attr *attr)
+{
+ uint32_t in[MLX5_ST_SZ_DW(create_import_kek_in)] = {0};
+ uint32_t out[MLX5_ST_SZ_DW(general_obj_out_cmd_hdr)] = {0};
+ struct mlx5_devx_obj *import_kek_obj = NULL;
+ void *ptr = NULL, *key_addr = NULL;
+
+ import_kek_obj = mlx5_malloc(MLX5_MEM_ZERO, sizeof(*import_kek_obj),
+0, SOCKET_ID_ANY);
+ if (import_kek_obj == NULL) {
+ DRV_LOG(ERR, "Failed to allocate IMPORT_KEK object data");
+ rte_errno = ENOMEM;
+ return NULL;
+ }
+ ptr = MLX5_ADDR_OF(create_import_kek_in, in, hdr);
+ MLX5_SET(general_obj_in_cmd_hdr, ptr, opcode,
+MLX5_CMD_OP_CREATE_GENERAL_OBJECT);
+ MLX5_SET(general_obj_in_cmd_hdr, ptr, obj_type,
+MLX5_GENERAL_OBJ_TYPE_IMPORT_KEK);
+ ptr = MLX5_ADDR_OF(create_import_kek_in, in, import_kek);
+ MLX5_SET(import_kek, ptr, key_size, attr->key_size);
+ key_addr = MLX5_ADDR_OF(import_kek, ptr, key);
+ memcpy(key_addr, (void *)(attr->key), MLX5_CRYPTO_KEY_MAX_SIZE);
+ import_kek_obj->obj = mlx5_glue->devx_obj_create(ctx, in, sizeof(in),
+out, sizeof(out));
+ if (import_kek_obj->obj == NULL) {
+ rte_errno = errno;
+ DRV_LOG(ERR, "Failed to create IMPORT_KEK object using DevX.");
+ mlx5_free(import_kek_obj);
+ return NULL;
+ }
+ import_kek_obj->id = MLX5_GET(general_obj_out_cmd_hdr, out, obj_id);
+ return import_kek_obj;
+}
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h
b/drivers/common/mlx5/mlx5_devx_cmds.h
index 600577f18a..6423610dae 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.h
+++ b/drivers/common/mlx5/mlx5_devx_cmds.h
@@ -142,6 +142,7 @@ struct mlx5_hca_attr {
uint32_t crypto:1; /* Crypto engine is supported. */
uint32_t aes_xts:1; /* AES-XTS crypto is supported. */
uint32_t dek:1; /* General obj type DEK is supported. */
+ uint32_t import_kek:1; /* General obj type IMPORT_KEK supported. */
uint32_t regexp_num_of_engines;
uint32_t log_max_ft_sampler_num:8;
uint32_t geneve_tlv_opt;
@@ -450,6 +451,13 @@ struct mlx5_devx_dek_attr {
uint8_t key[MLX5_CRYPTO_KEY_MAX_SIZE];
};
+struct mlx5_devx_import_kek_attr {
+ uint64_t modify_field_select;
+ uint32_t state:8;
+ uint32_t key_size:4;
+ uint8_t key[MLX5_CRYPTO_KEY_MAX_SIZE];
+};
+
/* mlx5_devx_cmds.c */
__rte_internal
@@ -606,4 +614,9 @@ __rte_internal
struct mlx5_devx_obj *
mlx5_devx_cmd_create_dek_obj(void *ctx, struct mlx5_devx_dek_attr *attr);
+__rte_internal
+struct mlx5_devx_obj *
+mlx5_devx_cmd_create_import_kek_obj(void *ctx,
+
[dpdk-dev] [PATCH v3 08/15] common/mlx5: support general obj CRYPTO LOGIN create
From: Dekel Peled
CRYPTO_LOGIN Object is used to login to the device as crypto user
or crypto officer.
Required in order to perform any crypto related control operations.
This patch adds support of CRYPTO_LOGIN object create operation.
Add reading of CRYPTO_LOGIN support capability.
Add function to create general object type CRYPTO_LOGIN, using DevX API.
Signed-off-by: Dekel Peled
Acked-by: Matan Azrad
---
drivers/common/mlx5/mlx5_devx_cmds.c | 54
drivers/common/mlx5/mlx5_devx_cmds.h | 15
drivers/common/mlx5/mlx5_prm.h | 19 ++
drivers/common/mlx5/version.map | 1 +
4 files changed, 89 insertions(+)
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c
b/drivers/common/mlx5/mlx5_devx_cmds.c
index afef7a5f63..5e082ebb78 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -754,6 +754,8 @@ mlx5_devx_cmd_query_hca_attr(void *ctx,
MLX5_GENERAL_OBJ_TYPES_CAP_DEK);
attr->import_kek = !!(general_obj_types_supported &
MLX5_GENERAL_OBJ_TYPES_CAP_IMPORT_KEK);
+ attr->crypto_login = !!(general_obj_types_supported &
+ MLX5_GENERAL_OBJ_TYPES_CAP_CRYPTO_LOGIN);
/* Add reading of other GENERAL_OBJ_TYPES_CAP bits above this line. */
attr->log_max_cq = MLX5_GET(cmd_hca_cap, hcattr, log_max_cq);
attr->log_max_qp = MLX5_GET(cmd_hca_cap, hcattr, log_max_qp);
@@ -2509,3 +2511,55 @@ mlx5_devx_cmd_create_import_kek_obj(void *ctx,
import_kek_obj->id = MLX5_GET(general_obj_out_cmd_hdr, out, obj_id);
return import_kek_obj;
}
+
+/**
+ * Create general object of type CRYPTO_LOGIN using DevX API.
+ *
+ * @param[in] ctx
+ * Context returned from mlx5 open_device() glue function.
+ * @param [in] attr
+ * Pointer to CRYPTO_LOGIN attributes structure.
+ *
+ * @return
+ * The DevX object created, NULL otherwise and rte_errno is set.
+ */
+struct mlx5_devx_obj *
+mlx5_devx_cmd_create_crypto_login_obj(void *ctx,
+ struct mlx5_devx_crypto_login_attr *attr)
+{
+ uint32_t in[MLX5_ST_SZ_DW(create_crypto_login_in)] = {0};
+ uint32_t out[MLX5_ST_SZ_DW(general_obj_out_cmd_hdr)] = {0};
+ struct mlx5_devx_obj *crypto_login_obj = NULL;
+ void *ptr = NULL, *credential_addr = NULL;
+
+ crypto_login_obj = mlx5_malloc(MLX5_MEM_ZERO, sizeof(*crypto_login_obj),
+ 0, SOCKET_ID_ANY);
+ if (crypto_login_obj == NULL) {
+ DRV_LOG(ERR, "Failed to allocate CRYPTO_LOGIN object data");
+ rte_errno = ENOMEM;
+ return NULL;
+ }
+ ptr = MLX5_ADDR_OF(create_crypto_login_in, in, hdr);
+ MLX5_SET(general_obj_in_cmd_hdr, ptr, opcode,
+MLX5_CMD_OP_CREATE_GENERAL_OBJECT);
+ MLX5_SET(general_obj_in_cmd_hdr, ptr, obj_type,
+MLX5_GENERAL_OBJ_TYPE_CRYPTO_LOGIN);
+ ptr = MLX5_ADDR_OF(create_crypto_login_in, in, crypto_login);
+ MLX5_SET(crypto_login, ptr, credential_pointer,
+attr->credential_pointer);
+ MLX5_SET(crypto_login, ptr, session_import_kek_ptr,
+attr->session_import_kek_ptr);
+ credential_addr = MLX5_ADDR_OF(crypto_login, ptr, credential);
+ memcpy(credential_addr, (void *)(attr->credential),
+ MLX5_CRYPTO_LOGIN_CREDENTIAL_SIZE);
+ crypto_login_obj->obj = mlx5_glue->devx_obj_create(ctx, in, sizeof(in),
+ out, sizeof(out));
+ if (crypto_login_obj->obj == NULL) {
+ rte_errno = errno;
+ DRV_LOG(ERR, "Failed to create CRYPTO_LOGIN obj using DevX.");
+ mlx5_free(crypto_login_obj);
+ return NULL;
+ }
+ crypto_login_obj->id = MLX5_GET(general_obj_out_cmd_hdr, out, obj_id);
+ return crypto_login_obj;
+}
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h
b/drivers/common/mlx5/mlx5_devx_cmds.h
index 6423610dae..709e28bfba 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.h
+++ b/drivers/common/mlx5/mlx5_devx_cmds.h
@@ -143,6 +143,7 @@ struct mlx5_hca_attr {
uint32_t aes_xts:1; /* AES-XTS crypto is supported. */
uint32_t dek:1; /* General obj type DEK is supported. */
uint32_t import_kek:1; /* General obj type IMPORT_KEK supported. */
+ uint32_t crypto_login:1; /* General obj type CRYPTO_LOGIN supported. */
uint32_t regexp_num_of_engines;
uint32_t log_max_ft_sampler_num:8;
uint32_t geneve_tlv_opt;
@@ -458,6 +459,15 @@ struct mlx5_devx_import_kek_attr {
uint8_t key[MLX5_CRYPTO_KEY_MAX_SIZE];
};
+#define MLX5_CRYPTO_LOGIN_CREDENTIAL_SIZE 48
+
+struct mlx5_devx_crypto_login_attr {
+ uint64_t modify_field_select;
+ uint32_t credential_pointer:24;
+
[dpdk-dev] [PATCH v3 09/15] common/mlx5: add crypto BSF struct and defines
From: Dekel Peled
This patch adds the struct defining crypto BSF segment of UMR WQE,
and the related value definitions and offsets.
Signed-off-by: Dekel Peled
Acked-by: Matan Azrad
---
drivers/common/mlx5/mlx5_prm.h | 66 ++
1 file changed, 66 insertions(+)
diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h
index a2437faec0..a9dcbfa63c 100644
--- a/drivers/common/mlx5/mlx5_prm.h
+++ b/drivers/common/mlx5/mlx5_prm.h
@@ -1096,6 +1096,72 @@ struct mlx5_ifc_create_mkey_in_bits {
u8 klm_pas_mtt[][0x20];
};
+enum {
+ MLX5_BSF_SIZE_16B = 0x0,
+ MLX5_BSF_SIZE_32B = 0x1,
+ MLX5_BSF_SIZE_64B = 0x2,
+ MLX5_BSF_SIZE_128B = 0x3,
+};
+
+enum {
+ MLX5_BSF_P_TYPE_SIGNATURE = 0x0,
+ MLX5_BSF_P_TYPE_CRYPTO = 0x1,
+};
+
+enum {
+ MLX5_ENCRYPTION_ORDER_ENCRYPTED_WIRE_SIGNATURE = 0x0,
+ MLX5_ENCRYPTION_ORDER_ENCRYPTED_MEMORY_SIGNATURE = 0x1,
+ MLX5_ENCRYPTION_ORDER_ENCRYPTED_RAW_WIRE = 0x2,
+ MLX5_ENCRYPTION_ORDER_ENCRYPTED_RAW_MEMORY = 0x3,
+};
+
+enum {
+ MLX5_ENCRYPTION_STANDARD_AES_XTS = 0x0,
+};
+
+enum {
+ MLX5_BLOCK_SIZE_512B= 0x1,
+ MLX5_BLOCK_SIZE_520B= 0x2,
+ MLX5_BLOCK_SIZE_4096B = 0x3,
+ MLX5_BLOCK_SIZE_4160B = 0x4,
+ MLX5_BLOCK_SIZE_1MB = 0x5,
+ MLX5_BLOCK_SIZE_4048B = 0x6,
+};
+
+#define MLX5_BSF_SIZE_OFFSET 30
+#define MLX5_BSF_P_TYPE_OFFSET 24
+#define MLX5_ENCRYPTION_ORDER_OFFSET 16
+#define MLX5_BLOCK_SIZE_OFFSET 24
+
+struct mlx5_wqe_umr_bsf_seg {
+ /*
+* bs_bpt_eo_es contains:
+* bs bsf_size2 bits at MLX5_BSF_SIZE_OFFSET
+* bpt bsf_p_type 2 bits at MLX5_BSF_P_TYPE_OFFSET
+* eo encryption_order4 bits at MLX5_ENCRYPTION_ORDER_OFFSET
+* es encryption_standard 4 bits at offset 0
+*/
+ uint32_t bs_bpt_eo_es;
+ uint32_t raw_data_size;
+ /*
+* bsp_res contains:
+* bsp crypto_block_size_pointer 8 bits at MLX5_BLOCK_SIZE_OFFSET
+* res reserved 24 bits
+*/
+ uint32_t bsp_res;
+ uint32_t reserved0;
+ uint8_t xts_initial_tweak[16];
+ /*
+* res_dp contains:
+* res reserved 8 bits
+* dp dek_pointer 24 bits at offset 0
+*/
+ uint32_t res_dp;
+ uint32_t reserved1;
+ uint64_t keytag;
+ uint32_t reserved2[4];
+} __rte_packed;
+
enum {
MLX5_GET_HCA_CAP_OP_MOD_GENERAL_DEVICE = 0x0 << 1,
MLX5_GET_HCA_CAP_OP_MOD_ETHERNET_OFFLOAD_CAPS = 0x1 << 1,
--
2.25.1
[dpdk-dev] [PATCH v3 10/15] common/mlx5: share hash list tool
From: Shiri Kuzin In order to use the hash list defined in net in other drivers, the hash list is moved to common utilities. In addition, the log definition was moved from the common utilities to a dedicated new log file in common in order to prevent a conflict. Signed-off-by: Shiri Kuzin Acked-by: Matan Azrad --- drivers/common/mlx5/linux/mlx5_common_os.c | 2 +- drivers/common/mlx5/linux/mlx5_nl.c | 2 +- drivers/common/mlx5/meson.build | 15 +- drivers/common/mlx5/mlx5_common.c| 2 +- drivers/common/mlx5/mlx5_common_devx.c | 2 +- drivers/common/mlx5/mlx5_common_log.h| 21 ++ drivers/common/mlx5/mlx5_common_mp.c | 2 +- drivers/common/mlx5/mlx5_common_mr.c | 2 +- drivers/common/mlx5/mlx5_common_pci.c| 4 +- drivers/common/mlx5/mlx5_common_utils.c | 221 +++ drivers/common/mlx5/mlx5_common_utils.h | 202 - drivers/common/mlx5/mlx5_devx_cmds.c | 2 +- drivers/common/mlx5/mlx5_malloc.c| 2 +- drivers/common/mlx5/version.map | 6 + drivers/common/mlx5/windows/mlx5_common_os.c | 2 +- drivers/common/mlx5/windows/mlx5_glue.c | 2 +- drivers/net/mlx5/mlx5_utils.c| 209 -- drivers/net/mlx5/mlx5_utils.h| 194 +--- 18 files changed, 465 insertions(+), 427 deletions(-) create mode 100644 drivers/common/mlx5/mlx5_common_log.h create mode 100644 drivers/common/mlx5/mlx5_common_utils.c diff --git a/drivers/common/mlx5/linux/mlx5_common_os.c b/drivers/common/mlx5/linux/mlx5_common_os.c index 5cf9576921..fba8245b8b 100644 --- a/drivers/common/mlx5/linux/mlx5_common_os.c +++ b/drivers/common/mlx5/linux/mlx5_common_os.c @@ -15,7 +15,7 @@ #include #include "mlx5_common.h" -#include "mlx5_common_utils.h" +#include "mlx5_common_log.h" #include "mlx5_glue.h" #ifdef MLX5_GLUE diff --git a/drivers/common/mlx5/linux/mlx5_nl.c b/drivers/common/mlx5/linux/mlx5_nl.c index 752c57b33d..f0d04f9473 100644 --- a/drivers/common/mlx5/linux/mlx5_nl.c +++ b/drivers/common/mlx5/linux/mlx5_nl.c @@ -20,7 +20,7 @@ #include #include "mlx5_nl.h" -#include "mlx5_common_utils.h" +#include "../mlx5_common_log.h" #include "mlx5_malloc.h" #ifdef HAVE_DEVLINK #include diff --git a/drivers/common/mlx5/meson.build b/drivers/common/mlx5/meson.build index e78b4f47bc..c7ce4c7045 100644 --- a/drivers/common/mlx5/meson.build +++ b/drivers/common/mlx5/meson.build @@ -9,13 +9,14 @@ endif deps += ['hash', 'pci', 'bus_pci', 'net', 'eal', 'kvargs'] sources += files( -'mlx5_devx_cmds.c', -'mlx5_common.c', -'mlx5_common_mp.c', -'mlx5_common_mr.c', -'mlx5_malloc.c', -'mlx5_common_pci.c', -'mlx5_common_devx.c', + 'mlx5_devx_cmds.c', + 'mlx5_common.c', + 'mlx5_common_mp.c', + 'mlx5_common_mr.c', + 'mlx5_malloc.c', + 'mlx5_common_pci.c', + 'mlx5_common_devx.c', + 'mlx5_common_utils.c', ) cflags_options = [ diff --git a/drivers/common/mlx5/mlx5_common.c b/drivers/common/mlx5/mlx5_common.c index f92f05bda5..d397459a3d 100644 --- a/drivers/common/mlx5/mlx5_common.c +++ b/drivers/common/mlx5/mlx5_common.c @@ -11,7 +11,7 @@ #include "mlx5_common.h" #include "mlx5_common_os.h" -#include "mlx5_common_utils.h" +#include "mlx5_common_log.h" #include "mlx5_common_pci.h" uint8_t haswell_broadwell_cpu; diff --git a/drivers/common/mlx5/mlx5_common_devx.c b/drivers/common/mlx5/mlx5_common_devx.c index d19be122bd..22c8d356c4 100644 --- a/drivers/common/mlx5/mlx5_common_devx.c +++ b/drivers/common/mlx5/mlx5_common_devx.c @@ -12,7 +12,7 @@ #include "mlx5_prm.h" #include "mlx5_devx_cmds.h" -#include "mlx5_common_utils.h" +#include "mlx5_common_log.h" #include "mlx5_malloc.h" #include "mlx5_common.h" #include "mlx5_common_devx.h" diff --git a/drivers/common/mlx5/mlx5_common_log.h b/drivers/common/mlx5/mlx5_common_log.h new file mode 100644 index 00..26b13fedaf --- /dev/null +++ b/drivers/common/mlx5/mlx5_common_log.h @@ -0,0 +1,21 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2019 Mellanox Technologies, Ltd + */ + +#ifndef RTE_PMD_MLX5_COMMON_LOG_H_ +#define RTE_PMD_MLX5_COMMON_LOG_H_ + +#include "mlx5_common.h" + + +extern int mlx5_common_logtype; + +#define MLX5_COMMON_LOG_PREFIX "mlx5_common" +/* Generic printf()-like logging macro with automatic line feed. */ +#define DRV_LOG(level, ...) \ + PMD_DRV_LOG_(level, mlx5_common_logtyp
[dpdk-dev] [PATCH v3 11/15] common/mlx5: share get ib device match function
From: Shiri Kuzin
The get_ib_device_match function iterates over the list of ib devices
returned by the get_device_list glue function and returns the ib device
matching the provided address.
Since this function is in use by several drivers, in this patch we
share the function in common part.
Signed-off-by: Shiri Kuzin
Acked-by: Matan Azrad
---
drivers/common/mlx5/linux/mlx5_common_os.c | 28 +++
drivers/common/mlx5/linux/mlx5_common_os.h | 5
drivers/common/mlx5/mlx5_common.h | 1 +
drivers/common/mlx5/version.map| 1 +
drivers/compress/mlx5/mlx5_compress.c | 30 +---
drivers/regex/mlx5/mlx5_regex.c| 30 ++--
drivers/vdpa/mlx5/mlx5_vdpa.c | 32 ++
7 files changed, 40 insertions(+), 87 deletions(-)
diff --git a/drivers/common/mlx5/linux/mlx5_common_os.c
b/drivers/common/mlx5/linux/mlx5_common_os.c
index fba8245b8b..ea0b71e425 100644
--- a/drivers/common/mlx5/linux/mlx5_common_os.c
+++ b/drivers/common/mlx5/linux/mlx5_common_os.c
@@ -16,6 +16,7 @@
#include "mlx5_common.h"
#include "mlx5_common_log.h"
+#include "mlx5_common_os.h"
#include "mlx5_glue.h"
#ifdef MLX5_GLUE
@@ -423,3 +424,30 @@ mlx5_glue_constructor(void)
mlx5_glue = NULL;
}
+struct ibv_device *
+mlx5_os_get_ibv_device(struct rte_pci_addr *addr)
+{
+ int n;
+ struct ibv_device **ibv_list = mlx5_glue->get_device_list(&n);
+ struct ibv_device *ibv_match = NULL;
+
+ if (ibv_list == NULL) {
+ rte_errno = ENOSYS;
+ return NULL;
+ }
+ while (n-- > 0) {
+ struct rte_pci_addr paddr;
+
+ DRV_LOG(DEBUG, "Checking device \"%s\"..", ibv_list[n]->name);
+ if (mlx5_dev_to_pci_addr(ibv_list[n]->ibdev_path, &paddr) != 0)
+ continue;
+ if (rte_pci_addr_cmp(addr, &paddr) != 0)
+ continue;
+ ibv_match = ibv_list[n];
+ break;
+ }
+ if (ibv_match == NULL)
+ rte_errno = ENOENT;
+ mlx5_glue->free_device_list(ibv_list);
+ return ibv_match;
+}
diff --git a/drivers/common/mlx5/linux/mlx5_common_os.h
b/drivers/common/mlx5/linux/mlx5_common_os.h
index d1c7e3dce0..72d6bf828b 100644
--- a/drivers/common/mlx5/linux/mlx5_common_os.h
+++ b/drivers/common/mlx5/linux/mlx5_common_os.h
@@ -288,4 +288,9 @@ mlx5_os_free(void *addr)
{
free(addr);
}
+
+__rte_internal
+struct ibv_device *
+mlx5_os_get_ibv_device(struct rte_pci_addr *addr);
+
#endif /* RTE_PMD_MLX5_COMMON_OS_H_ */
diff --git a/drivers/common/mlx5/mlx5_common.h
b/drivers/common/mlx5/mlx5_common.h
index f3c6beb23b..1fbefe0fa6 100644
--- a/drivers/common/mlx5/mlx5_common.h
+++ b/drivers/common/mlx5/mlx5_common.h
@@ -18,6 +18,7 @@
#include "mlx5_prm.h"
#include "mlx5_devx_cmds.h"
+#include "mlx5_common_os.h"
/* Reported driver name. */
#define MLX5_PCI_DRIVER_NAME "mlx5_pci"
diff --git a/drivers/common/mlx5/version.map b/drivers/common/mlx5/version.map
index 40b0d713ac..5706c6bbef 100644
--- a/drivers/common/mlx5/version.map
+++ b/drivers/common/mlx5/version.map
@@ -126,6 +126,7 @@ INTERNAL {
mlx5_os_alloc_pd;
mlx5_os_dealloc_pd;
mlx5_os_dereg_mr;
+ mlx5_os_get_ibv_device; # WINDOWS_NO_EXPORT
mlx5_os_reg_mr;
mlx5_os_umem_dereg;
mlx5_os_umem_reg;
diff --git a/drivers/compress/mlx5/mlx5_compress.c
b/drivers/compress/mlx5/mlx5_compress.c
index ec3c237512..0581dee505 100644
--- a/drivers/compress/mlx5/mlx5_compress.c
+++ b/drivers/compress/mlx5/mlx5_compress.c
@@ -647,34 +647,6 @@ mlx5_compress_dequeue_burst(void *queue_pair, struct
rte_comp_op **ops,
return i;
}
-static struct ibv_device *
-mlx5_compress_get_ib_device_match(struct rte_pci_addr *addr)
-{
- int n;
- struct ibv_device **ibv_list = mlx5_glue->get_device_list(&n);
- struct ibv_device *ibv_match = NULL;
-
- if (ibv_list == NULL) {
- rte_errno = ENOSYS;
- return NULL;
- }
- while (n-- > 0) {
- struct rte_pci_addr paddr;
-
- DRV_LOG(DEBUG, "Checking device \"%s\"..", ibv_list[n]->name);
- if (mlx5_dev_to_pci_addr(ibv_list[n]->ibdev_path, &paddr) != 0)
- continue;
- if (rte_pci_addr_cmp(addr, &paddr) != 0)
- continue;
- ibv_match = ibv_list[n];
- break;
- }
- if (ibv_match == NULL)
- rte_errno = ENOENT;
- mlx5_glue->free_device_list(ibv_list);
- return ibv_match;
-}
-
static void
mlx5_compress_hw_global_release(struct mlx5_compress_priv *priv)
{
@@ -774,7 +746,7 @@ mlx5_compress_pci_probe(struct rte_pci_d
[dpdk-dev] [PATCH v3 12/15] common/mlx5: support general obj CREDENTIAL create
From: Dekel Peled
CREDENTIAL object is used for any crypto operation in wrapped mode.
This patch add support of CREDENTIAL object create operation.
Add reading of CREDENTIAL support capability.
Add function to create general object type CREDENTIAL, using DevX API.
Signed-off-by: Dekel Peled
Acked-by: Matan Azrad
---
drivers/common/mlx5/mlx5_devx_cmds.c | 53 +++-
drivers/common/mlx5/mlx5_devx_cmds.h | 17 +++--
drivers/common/mlx5/mlx5_prm.h | 23
drivers/common/mlx5/version.map | 1 +
4 files changed, 91 insertions(+), 3 deletions(-)
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c
b/drivers/common/mlx5/mlx5_devx_cmds.c
index c0061741e8..c0a0853c3a 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -754,6 +754,8 @@ mlx5_devx_cmd_query_hca_attr(void *ctx,
MLX5_GENERAL_OBJ_TYPES_CAP_DEK);
attr->import_kek = !!(general_obj_types_supported &
MLX5_GENERAL_OBJ_TYPES_CAP_IMPORT_KEK);
+ attr->credential = !!(general_obj_types_supported &
+ MLX5_GENERAL_OBJ_TYPES_CAP_CREDENTIAL);
attr->crypto_login = !!(general_obj_types_supported &
MLX5_GENERAL_OBJ_TYPES_CAP_CRYPTO_LOGIN);
/* Add reading of other GENERAL_OBJ_TYPES_CAP bits above this line. */
@@ -2512,6 +2514,55 @@ mlx5_devx_cmd_create_import_kek_obj(void *ctx,
return import_kek_obj;
}
+/**
+ * Create general object of type CREDENTIAL using DevX API.
+ *
+ * @param[in] ctx
+ * Context returned from mlx5 open_device() glue function.
+ * @param [in] attr
+ * Pointer to CREDENTIAL attributes structure.
+ *
+ * @return
+ * The DevX object created, NULL otherwise and rte_errno is set.
+ */
+struct mlx5_devx_obj *
+mlx5_devx_cmd_create_credential_obj(void *ctx,
+ struct mlx5_devx_credential_attr *attr)
+{
+ uint32_t in[MLX5_ST_SZ_DW(create_credential_in)] = {0};
+ uint32_t out[MLX5_ST_SZ_DW(general_obj_out_cmd_hdr)] = {0};
+ struct mlx5_devx_obj *credential_obj = NULL;
+ void *ptr = NULL, *credential_addr = NULL;
+
+ credential_obj = mlx5_malloc(MLX5_MEM_ZERO, sizeof(*credential_obj),
+0, SOCKET_ID_ANY);
+ if (credential_obj == NULL) {
+ DRV_LOG(ERR, "Failed to allocate CREDENTIAL object data");
+ rte_errno = ENOMEM;
+ return NULL;
+ }
+ ptr = MLX5_ADDR_OF(create_credential_in, in, hdr);
+ MLX5_SET(general_obj_in_cmd_hdr, ptr, opcode,
+MLX5_CMD_OP_CREATE_GENERAL_OBJECT);
+ MLX5_SET(general_obj_in_cmd_hdr, ptr, obj_type,
+MLX5_GENERAL_OBJ_TYPE_CREDENTIAL);
+ ptr = MLX5_ADDR_OF(create_credential_in, in, credential);
+ MLX5_SET(credential, ptr, credential_role, attr->credential_role);
+ credential_addr = MLX5_ADDR_OF(credential, ptr, credential);
+ memcpy(credential_addr, (void *)(attr->credential),
+ MLX5_CRYPTO_CREDENTIAL_SIZE);
+ credential_obj->obj = mlx5_glue->devx_obj_create(ctx, in, sizeof(in),
+out, sizeof(out));
+ if (credential_obj->obj == NULL) {
+ rte_errno = errno;
+ DRV_LOG(ERR, "Failed to create CREDENTIAL object using DevX.");
+ mlx5_free(credential_obj);
+ return NULL;
+ }
+ credential_obj->id = MLX5_GET(general_obj_out_cmd_hdr, out, obj_id);
+ return credential_obj;
+}
+
/**
* Create general object of type CRYPTO_LOGIN using DevX API.
*
@@ -2551,7 +2602,7 @@ mlx5_devx_cmd_create_crypto_login_obj(void *ctx,
attr->session_import_kek_ptr);
credential_addr = MLX5_ADDR_OF(crypto_login, ptr, credential);
memcpy(credential_addr, (void *)(attr->credential),
- MLX5_CRYPTO_LOGIN_CREDENTIAL_SIZE);
+ MLX5_CRYPTO_CREDENTIAL_SIZE);
crypto_login_obj->obj = mlx5_glue->devx_obj_create(ctx, in, sizeof(in),
out, sizeof(out));
if (crypto_login_obj->obj == NULL) {
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h
b/drivers/common/mlx5/mlx5_devx_cmds.h
index 709e28bfba..811e7a1462 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.h
+++ b/drivers/common/mlx5/mlx5_devx_cmds.h
@@ -143,6 +143,7 @@ struct mlx5_hca_attr {
uint32_t aes_xts:1; /* AES-XTS crypto is supported. */
uint32_t dek:1; /* General obj type DEK is supported. */
uint32_t import_kek:1; /* General obj type IMPORT_KEK supported. */
+ uint32_t credential:1; /* General obj type CREDENTIAL supported. */
uint32_t crypto_login:1; /* General obj type CRYPTO_LOGIN supported. */
uint32_t regexp_num_of_engine
[dpdk-dev] [PATCH v3 13/15] common/mlx5: add crypto register structs and defs
From: Dekel Peled
Encryption key management requires use of several related registers.
This patch adds the relevant structs and values, according to PRM
definitions.
Signed-off-by: Dekel Peled
Acked-by: Matan Azrad
---
drivers/common/mlx5/mlx5_prm.h | 41 ++
1 file changed, 41 insertions(+)
diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h
index 432c8fdb63..c2cd2d9f70 100644
--- a/drivers/common/mlx5/mlx5_prm.h
+++ b/drivers/common/mlx5/mlx5_prm.h
@@ -3307,6 +3307,10 @@ enum {
enum {
MLX5_REGISTER_ID_MTUTC = 0x9055,
+ MLX5_CRYPTO_OPERATIONAL_REGISTER_ID = 0xC002,
+ MLX5_CRYPTO_COMMISSIONING_REGISTER_ID = 0xC003,
+ MLX5_IMPORT_KEK_HANDLE_REGISTER_ID = 0xC004,
+ MLX5_CREDENTIAL_HANDLE_REGISTER_ID = 0xC005,
};
struct mlx5_ifc_register_mtutc_bits {
@@ -3324,6 +3328,43 @@ struct mlx5_ifc_register_mtutc_bits {
#define MLX5_MTUTC_TIMESTAMP_MODE_INTERNAL_TIMER 0
#define MLX5_MTUTC_TIMESTAMP_MODE_REAL_TIME 1
+struct mlx5_ifc_crypto_operational_register_bits {
+ u8 wrapped_crypto_operational[0x1];
+ u8 reserved_at_1[0x1b];
+ u8 kek_size[0x4];
+ u8 reserved_at_20[0x20];
+ u8 credential[0x140];
+ u8 kek[0x100];
+ u8 reserved_at_280[0x180];
+};
+
+struct mlx5_ifc_crypto_commissioning_register_bits {
+ u8 token[0x1]; /* TODO: add size after PRM update */
+};
+
+struct mlx5_ifc_import_kek_handle_register_bits {
+ struct mlx5_ifc_crypto_login_bits crypto_login_object;
+ struct mlx5_ifc_import_kek_bits import_kek_object;
+ u8 reserved_at_200[0x4];
+ u8 write_operation[0x4];
+ u8 import_kek_id[0x18];
+ u8 reserved_at_220[0xe0];
+};
+
+struct mlx5_ifc_credential_handle_register_bits {
+ struct mlx5_ifc_crypto_login_bits crypto_login_object;
+ struct mlx5_ifc_credential_bits credential_object;
+ u8 reserved_at_200[0x4];
+ u8 write_operation[0x4];
+ u8 credential_id[0x18];
+ u8 reserved_at_220[0xe0];
+};
+
+enum {
+ MLX5_REGISTER_ADD_OPERATION = 0x1,
+ MLX5_REGISTER_DELETE_OPERATION = 0x2,
+};
+
struct mlx5_ifc_parse_graph_arc_bits {
u8 start_inner_tunnel[0x1];
u8 reserved_at_1[0x7];
--
2.25.1
[dpdk-dev] [PATCH v3 14/15] common/mlx5: support register write access
From: Dekel Peled
This patch adds support of write operation to NIC registers.
Signed-off-by: Dekel Peled
Acked-by: Matan Azrad
---
drivers/common/mlx5/mlx5_devx_cmds.c | 67 +++-
drivers/common/mlx5/mlx5_devx_cmds.h | 4 ++
drivers/common/mlx5/version.map | 1 +
3 files changed, 70 insertions(+), 2 deletions(-)
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c
b/drivers/common/mlx5/mlx5_devx_cmds.c
index c0a0853c3a..0b421933ce 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -12,7 +12,6 @@
#include "mlx5_common_log.h"
#include "mlx5_malloc.h"
-
/**
* Perform read access to the registers. Reads data from register
* and writes ones to the specified buffer.
@@ -61,7 +60,7 @@ mlx5_devx_cmd_register_read(void *ctx, uint16_t reg_id,
uint32_t arg,
if (status) {
int syndrome = MLX5_GET(access_register_out, out, syndrome);
- DRV_LOG(DEBUG, "Failed to access NIC register 0x%X, "
+ DRV_LOG(DEBUG, "Failed to read access NIC register 0x%X, "
"status %x, syndrome = %x",
reg_id, status, syndrome);
return -1;
@@ -74,6 +73,70 @@ mlx5_devx_cmd_register_read(void *ctx, uint16_t reg_id,
uint32_t arg,
return rc;
}
+/**
+ * Perform write access to the registers.
+ *
+ * @param[in] ctx
+ * Context returned from mlx5 open_device() glue function.
+ * @param[in] reg_id
+ * Register identifier according to the PRM.
+ * @param[in] arg
+ * Register access auxiliary parameter according to the PRM.
+ * @param[out] data
+ * Pointer to the buffer containing data to write.
+ * @param[in] dw_cnt
+ * Buffer size in double words (32bit units).
+ *
+ * @return
+ * 0 on success, a negative value otherwise.
+ */
+int
+mlx5_devx_cmd_register_write(void *ctx, uint16_t reg_id, uint32_t arg,
+uint32_t *data, uint32_t dw_cnt)
+{
+ uint32_t in[MLX5_ST_SZ_DW(access_register_in) +
+ MLX5_ACCESS_REGISTER_DATA_DWORD_MAX] = {0};
+ uint32_t out[MLX5_ST_SZ_DW(access_register_out)] = {0};
+ int status, rc;
+ void *ptr;
+
+ MLX5_ASSERT(data && dw_cnt);
+ MLX5_ASSERT(dw_cnt <= MLX5_ACCESS_REGISTER_DATA_DWORD_MAX);
+ if (dw_cnt > MLX5_ACCESS_REGISTER_DATA_DWORD_MAX) {
+ DRV_LOG(ERR, "Data to write exceeds max size");
+ return -1;
+ }
+ MLX5_SET(access_register_in, in, opcode,
+MLX5_CMD_OP_ACCESS_REGISTER_USER);
+ MLX5_SET(access_register_in, in, op_mod,
+MLX5_ACCESS_REGISTER_IN_OP_MOD_WRITE);
+ MLX5_SET(access_register_in, in, register_id, reg_id);
+ MLX5_SET(access_register_in, in, argument, arg);
+ ptr = MLX5_ADDR_OF(access_register_in, in, register_data);
+ memcpy(ptr, data, dw_cnt * sizeof(uint32_t));
+ rc = mlx5_glue->devx_general_cmd(ctx, in, sizeof(in), out, sizeof(out));
+
+ rc = mlx5_glue->devx_general_cmd(ctx, in,
+MLX5_ST_SZ_BYTES(access_register_in) +
+dw_cnt * sizeof(uint32_t),
+out, sizeof(out));
+ if (rc)
+ goto error;
+ status = MLX5_GET(access_register_out, out, status);
+ if (status) {
+ int syndrome = MLX5_GET(access_register_out, out, syndrome);
+
+ DRV_LOG(DEBUG, "Failed to write access NIC register 0x%X, "
+ "status %x, syndrome = %x",
+ reg_id, status, syndrome);
+ return -1;
+ }
+ return 0;
+error:
+ rc = (rc > 0) ? -rc : rc;
+ return rc;
+}
+
/**
* Allocate flow counters via devx interface.
*
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h
b/drivers/common/mlx5/mlx5_devx_cmds.h
index 811e7a1462..ce570ad28a 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.h
+++ b/drivers/common/mlx5/mlx5_devx_cmds.h
@@ -566,6 +566,10 @@ __rte_internal
int mlx5_devx_cmd_register_read(void *ctx, uint16_t reg_id,
uint32_t arg, uint32_t *data, uint32_t dw_cnt);
+__rte_internal
+int mlx5_devx_cmd_register_write(void *ctx, uint16_t reg_id,
+uint32_t arg, uint32_t *data, uint32_t dw_cnt);
+
__rte_internal
struct mlx5_devx_obj *
mlx5_devx_cmd_create_geneve_tlv_option(void *ctx,
diff --git a/drivers/common/mlx5/version.map b/drivers/common/mlx5/version.map
index 04b2179b2c..c630696213 100644
--- a/drivers/common/mlx5/version.map
+++ b/drivers/common/mlx5/version.map
@@ -53,6 +53,7 @@ INTERNAL {
mlx5_devx_cmd_queue_counter_alloc; # WINDOWS_NO_EXPORT
mlx5_devx_cmd_queue_counter_query; # WINDOWS_NO_EXPORT
mlx5_devx_cmd_r
[dpdk-dev] [PATCH v3 15/15] common/mlx5: add UMR and RDMA write WQE defines
From: Suanming Mou
This patch adds the struct defining UMR and RDMA write WQEs.
Signed-off-by: Suanming Mou
Signed-off-by: Matan Azrad
---
drivers/common/mlx5/mlx5_prm.h | 187 +
1 file changed, 121 insertions(+), 66 deletions(-)
diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h
index c2cd2d9f70..1ffee5fd56 100644
--- a/drivers/common/mlx5/mlx5_prm.h
+++ b/drivers/common/mlx5/mlx5_prm.h
@@ -412,6 +412,127 @@ struct mlx5_cqe_ts {
uint8_t op_own;
};
+struct mlx5_wqe_rseg {
+ uint64_t raddr;
+ uint32_t rkey;
+ uint32_t reserved;
+} __rte_packed;
+
+#define MLX5_UMRC_IF_OFFSET 31u
+#define MLX5_UMRC_KO_OFFSET 16u
+#define MLX5_UMRC_TO_BS_OFFSET 0u
+
+struct mlx5_wqe_umr_cseg {
+ uint32_t if_cf_toe_cq_res;
+ uint32_t ko_to_bs;
+ uint64_t mkey_mask;
+ uint32_t rsvd1[8];
+} __rte_packed;
+
+struct mlx5_wqe_mkey_cseg {
+ uint32_t fr_res_af_sf;
+ uint32_t qpn_mkey;
+ uint32_t reserved2;
+ uint32_t flags_pd;
+ uint64_t start_addr;
+ uint64_t len;
+ uint32_t bsf_octword_size;
+ uint32_t reserved3[4];
+ uint32_t translations_octword_size;
+ uint32_t res4_lps;
+ uint32_t reserved;
+} __rte_packed;
+
+enum {
+ MLX5_BSF_SIZE_16B = 0x0,
+ MLX5_BSF_SIZE_32B = 0x1,
+ MLX5_BSF_SIZE_64B = 0x2,
+ MLX5_BSF_SIZE_128B = 0x3,
+};
+
+enum {
+ MLX5_BSF_P_TYPE_SIGNATURE = 0x0,
+ MLX5_BSF_P_TYPE_CRYPTO = 0x1,
+};
+
+enum {
+ MLX5_ENCRYPTION_ORDER_ENCRYPTED_WIRE_SIGNATURE = 0x0,
+ MLX5_ENCRYPTION_ORDER_ENCRYPTED_MEMORY_SIGNATURE = 0x1,
+ MLX5_ENCRYPTION_ORDER_ENCRYPTED_RAW_WIRE = 0x2,
+ MLX5_ENCRYPTION_ORDER_ENCRYPTED_RAW_MEMORY = 0x3,
+};
+
+enum {
+ MLX5_ENCRYPTION_STANDARD_AES_XTS = 0x0,
+};
+
+enum {
+ MLX5_BLOCK_SIZE_512B= 0x1,
+ MLX5_BLOCK_SIZE_520B= 0x2,
+ MLX5_BLOCK_SIZE_4096B = 0x3,
+ MLX5_BLOCK_SIZE_4160B = 0x4,
+ MLX5_BLOCK_SIZE_1MB = 0x5,
+ MLX5_BLOCK_SIZE_4048B = 0x6,
+};
+
+#define MLX5_BSF_SIZE_OFFSET 30
+#define MLX5_BSF_P_TYPE_OFFSET 24
+#define MLX5_ENCRYPTION_ORDER_OFFSET 16
+#define MLX5_BLOCK_SIZE_OFFSET 24
+
+struct mlx5_wqe_umr_bsf_seg {
+ /*
+* bs_bpt_eo_es contains:
+* bs bsf_size2 bits at MLX5_BSF_SIZE_OFFSET
+* bpt bsf_p_type 2 bits at MLX5_BSF_P_TYPE_OFFSET
+* eo encryption_order4 bits at MLX5_ENCRYPTION_ORDER_OFFSET
+* es encryption_standard 4 bits at offset 0
+*/
+ uint32_t bs_bpt_eo_es;
+ uint32_t raw_data_size;
+ /*
+* bsp_res contains:
+* bsp crypto_block_size_pointer 8 bits at MLX5_BLOCK_SIZE_OFFSET
+* res reserved 24 bits
+*/
+ uint32_t bsp_res;
+ uint32_t reserved0;
+ uint8_t xts_initial_tweak[16];
+ /*
+* res_dp contains:
+* res reserved 8 bits
+* dp dek_pointer 24 bits at offset 0
+*/
+ uint32_t res_dp;
+ uint32_t reserved1;
+ uint64_t keytag;
+ uint32_t reserved2[4];
+} __rte_packed;
+
+#ifdef PEDANTIC
+#pragma GCC diagnostic ignored "-Wpedantic"
+#endif
+
+struct mlx5_umr_wqe {
+ struct mlx5_wqe_cseg ctr;
+ struct mlx5_wqe_umr_cseg ucseg;
+ struct mlx5_wqe_mkey_cseg mkc;
+ union {
+ struct mlx5_wqe_dseg kseg[0];
+ struct mlx5_wqe_umr_bsf_seg bsf[0];
+ };
+} __rte_packed;
+
+struct mlx5_rdma_write_wqe {
+ struct mlx5_wqe_cseg ctr;
+ struct mlx5_wqe_rseg rseg;
+ struct mlx5_wqe_dseg dseg[0];
+} __rte_packed;
+
+#ifdef PEDANTIC
+#pragma GCC diagnostic error "-Wpedantic"
+#endif
+
/* GGA */
/* MMO metadata segment */
@@ -1096,72 +1217,6 @@ struct mlx5_ifc_create_mkey_in_bits {
u8 klm_pas_mtt[][0x20];
};
-enum {
- MLX5_BSF_SIZE_16B = 0x0,
- MLX5_BSF_SIZE_32B = 0x1,
- MLX5_BSF_SIZE_64B = 0x2,
- MLX5_BSF_SIZE_128B = 0x3,
-};
-
-enum {
- MLX5_BSF_P_TYPE_SIGNATURE = 0x0,
- MLX5_BSF_P_TYPE_CRYPTO = 0x1,
-};
-
-enum {
- MLX5_ENCRYPTION_ORDER_ENCRYPTED_WIRE_SIGNATURE = 0x0,
- MLX5_ENCRYPTION_ORDER_ENCRYPTED_MEMORY_SIGNATURE = 0x1,
- MLX5_ENCRYPTION_ORDER_ENCRYPTED_RAW_WIRE = 0x2,
- MLX5_ENCRYPTION_ORDER_ENCRYPTED_RAW_MEMORY = 0x3,
-};
-
-enum {
- MLX5_ENCRYPTION_STANDARD_AES_XTS = 0x0,
-};
-
-enum {
- MLX5_BLOCK_SIZE_512B= 0x1,
- MLX5_BLOCK_SIZE_520B= 0x2,
- MLX5_BLOCK_SIZE_4096B = 0x3,
- MLX5_BLOCK_SIZE_4160B = 0x4,
- MLX5_BLOCK_SIZE_1MB = 0x5,
- MLX5_BLOCK_SIZE_4048B = 0x6,
-};
-
-#define MLX5_BSF_SIZE_OFFSET 30
-#define MLX5_BSF_P_TYPE_OFFSET 24
-#define MLX5_ENCRYPTION_ORDER_OFFSET 16
-#define MLX5_BLOCK_SIZE_OFFSET 24
-
-struct mlx5_wqe_umr_bsf_seg {
- /*
-
[dpdk-dev] [PATCH v3 00/15] drivers: introduce mlx5 crypto PMD
Add a new PMD for Nvidia devices- crypto PMD. The crypto PMD will be supported on Nvidia ConnectX6 The crypto PMD will add the support of encryption and decryption using the AES-XTS symmetric algorithm. The crypto PMD requires rdma-core and uses mlx5 DevX. v2: Add data-path part. v3: Rebase. Shiri Kuzin (10): drivers: introduce mlx5 crypto PMD crypto/mlx5: add DEK object management crypto/mlx5: support session operations crypto/mlx5: add basic operations crypto/mlx5: support queue pairs operations crypto/mlx5: add dev stop and start operations crypto/mlx5: add memory region management crypto/mlx5: create login object using DevX crypto/mlx5: adjust to the multiple data unit API crypto/mlx5: set feature flags and capabilities Suanming Mou (5): crypto/mlx5: add keytag device argument crypto/mlx5: add maximum segments device argument crypto/mlx5: add WQE set initialization crypto/mlx5: add enqueue and dequeue operations crypto/mlx5: add statistic get and reset operations MAINTAINERS |4 + doc/guides/cryptodevs/features/mlx5.ini | 37 + doc/guides/cryptodevs/index.rst |1 + doc/guides/cryptodevs/mlx5.rst | 152 doc/guides/rel_notes/release_21_05.rst |5 + drivers/common/mlx5/mlx5_common.h |1 + drivers/common/mlx5/mlx5_common_pci.c | 14 + drivers/common/mlx5/mlx5_common_pci.h | 21 +- drivers/crypto/meson.build |1 + drivers/crypto/mlx5/meson.build | 27 + drivers/crypto/mlx5/mlx5_crypto.c | 1092 +++ drivers/crypto/mlx5/mlx5_crypto.h | 92 ++ drivers/crypto/mlx5/mlx5_crypto_dek.c | 136 +++ drivers/crypto/mlx5/mlx5_crypto_utils.h | 19 + drivers/crypto/mlx5/version.map |3 + 15 files changed, 1595 insertions(+), 10 deletions(-) create mode 100644 doc/guides/cryptodevs/features/mlx5.ini create mode 100644 doc/guides/cryptodevs/mlx5.rst create mode 100644 drivers/crypto/mlx5/meson.build create mode 100644 drivers/crypto/mlx5/mlx5_crypto.c create mode 100644 drivers/crypto/mlx5/mlx5_crypto.h create mode 100644 drivers/crypto/mlx5/mlx5_crypto_dek.c create mode 100644 drivers/crypto/mlx5/mlx5_crypto_utils.h create mode 100644 drivers/crypto/mlx5/version.map -- 2.25.1
[dpdk-dev] [PATCH v3 01/15] drivers: introduce mlx5 crypto PMD
From: Shiri Kuzin
Add a new PMD for Nvidia devices- crypto PMD.
The crypto PMD will be supported starting Nvidia ConnectX6 and
BlueField2.
The crypto PMD will add the support of encryption and decryption using
the AES-XTS symmetric algorithm.
The crypto PMD requires rdma-core and uses mlx5 DevX.
This patch adds the PCI probing, basic functions, build files and
log utility.
Signed-off-by: Shiri Kuzin
Acked-by: Matan Azrad
---
MAINTAINERS | 4 +
drivers/common/mlx5/mlx5_common.h | 1 +
drivers/common/mlx5/mlx5_common_pci.c | 14 ++
drivers/common/mlx5/mlx5_common_pci.h | 21 +-
drivers/crypto/meson.build | 1 +
drivers/crypto/mlx5/meson.build | 26 +++
drivers/crypto/mlx5/mlx5_crypto.c | 272
drivers/crypto/mlx5/mlx5_crypto_utils.h | 19 ++
drivers/crypto/mlx5/version.map | 3 +
9 files changed, 351 insertions(+), 10 deletions(-)
create mode 100644 drivers/crypto/mlx5/meson.build
create mode 100644 drivers/crypto/mlx5/mlx5_crypto.c
create mode 100644 drivers/crypto/mlx5/mlx5_crypto_utils.h
create mode 100644 drivers/crypto/mlx5/version.map
diff --git a/MAINTAINERS b/MAINTAINERS
index b40d8ae266..165474c91f 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1078,6 +1078,10 @@ F: drivers/crypto/octeontx2/
F: doc/guides/cryptodevs/octeontx2.rst
F: doc/guides/cryptodevs/features/octeontx2.ini
+Mellanox mlx5
+M: Matan Azrad
+F: drivers/crypto/mlx5/
+
Null Crypto
M: Declan Doherty
F: drivers/crypto/null/
diff --git a/drivers/common/mlx5/mlx5_common.h
b/drivers/common/mlx5/mlx5_common.h
index 1fbefe0fa6..89aca32305 100644
--- a/drivers/common/mlx5/mlx5_common.h
+++ b/drivers/common/mlx5/mlx5_common.h
@@ -216,6 +216,7 @@ enum mlx5_class {
MLX5_CLASS_VDPA = RTE_BIT64(1),
MLX5_CLASS_REGEX = RTE_BIT64(2),
MLX5_CLASS_COMPRESS = RTE_BIT64(3),
+ MLX5_CLASS_CRYPTO = RTE_BIT64(4),
};
#define MLX5_DBR_SIZE RTE_CACHE_LINE_SIZE
diff --git a/drivers/common/mlx5/mlx5_common_pci.c
b/drivers/common/mlx5/mlx5_common_pci.c
index 3f16cd21cf..8a47afee20 100644
--- a/drivers/common/mlx5/mlx5_common_pci.c
+++ b/drivers/common/mlx5/mlx5_common_pci.c
@@ -31,6 +31,7 @@ static const struct {
{ .name = "net", .driver_class = MLX5_CLASS_NET },
{ .name = "regex", .driver_class = MLX5_CLASS_REGEX },
{ .name = "compress", .driver_class = MLX5_CLASS_COMPRESS },
+ { .name = "crypto", .driver_class = MLX5_CLASS_CRYPTO },
};
static const unsigned int mlx5_class_combinations[] = {
@@ -38,13 +39,26 @@ static const unsigned int mlx5_class_combinations[] = {
MLX5_CLASS_VDPA,
MLX5_CLASS_REGEX,
MLX5_CLASS_COMPRESS,
+ MLX5_CLASS_CRYPTO,
MLX5_CLASS_NET | MLX5_CLASS_REGEX,
MLX5_CLASS_VDPA | MLX5_CLASS_REGEX,
MLX5_CLASS_NET | MLX5_CLASS_COMPRESS,
MLX5_CLASS_VDPA | MLX5_CLASS_COMPRESS,
MLX5_CLASS_REGEX | MLX5_CLASS_COMPRESS,
+ MLX5_CLASS_NET | MLX5_CLASS_CRYPTO,
+ MLX5_CLASS_VDPA | MLX5_CLASS_CRYPTO,
+ MLX5_CLASS_REGEX | MLX5_CLASS_CRYPTO,
+ MLX5_CLASS_COMPRESS | MLX5_CLASS_CRYPTO,
MLX5_CLASS_NET | MLX5_CLASS_REGEX | MLX5_CLASS_COMPRESS,
MLX5_CLASS_VDPA | MLX5_CLASS_REGEX | MLX5_CLASS_COMPRESS,
+ MLX5_CLASS_NET | MLX5_CLASS_REGEX | MLX5_CLASS_CRYPTO,
+ MLX5_CLASS_VDPA | MLX5_CLASS_REGEX | MLX5_CLASS_CRYPTO,
+ MLX5_CLASS_NET | MLX5_CLASS_COMPRESS | MLX5_CLASS_CRYPTO,
+ MLX5_CLASS_VDPA | MLX5_CLASS_COMPRESS | MLX5_CLASS_CRYPTO,
+ MLX5_CLASS_NET | MLX5_CLASS_REGEX | MLX5_CLASS_COMPRESS |
+ MLX5_CLASS_CRYPTO,
+ MLX5_CLASS_VDPA | MLX5_CLASS_REGEX | MLX5_CLASS_COMPRESS |
+ MLX5_CLASS_CRYPTO,
/* New class combination should be added here. */
};
diff --git a/drivers/common/mlx5/mlx5_common_pci.h
b/drivers/common/mlx5/mlx5_common_pci.h
index de89bb98bc..cb8d2f5f87 100644
--- a/drivers/common/mlx5/mlx5_common_pci.h
+++ b/drivers/common/mlx5/mlx5_common_pci.h
@@ -9,17 +9,18 @@
* @file
*
* RTE Mellanox PCI Driver Interface
- * Mellanox ConnectX PCI device supports multiple class: net,vdpa,regex and
- * compress devices. This layer enables creating such multiple class of devices
- * on a single PCI device by allowing to bind multiple class specific device
- * driver to attach to mlx5_pci driver.
+ * Mellanox ConnectX PCI device supports multiple class:
net,vdpa,regex,compress
+ * and crypto devices. This layer enables creating such multiple class of
+ * devices on a single PCI device by allowing to bind multiple class specific
+ * device driver to attach to mlx5_pci driver.
*
- * ----
- * | mlx5 || mlx5 || mlx5|| mlx5 |
- * | net pmd || vdpa
[dpdk-dev] [PATCH v3 04/15] crypto/mlx5: add basic operations
From: Shiri Kuzin
The basic dev control operations are configure, close and get info.
Extended the existing support of configure and close:
-mlx5_crypto_dev_configure- function used to configure device.
-mlx5_crypto_dev_close- function used to close a configured
device.
Added support of get info function:
-mlx5_crypto_dev_infos_get- function used to get specific
information of a device.
Added config struct to user private data with the fields socket id,
number of queue pairs and feature flags to be disabled.
Signed-off-by: Shiri Kuzin
Acked-by: Matan Azrad
---
drivers/crypto/mlx5/mlx5_crypto.c | 46 +++
drivers/crypto/mlx5/mlx5_crypto.h | 1 +
2 files changed, 42 insertions(+), 5 deletions(-)
diff --git a/drivers/crypto/mlx5/mlx5_crypto.c
b/drivers/crypto/mlx5/mlx5_crypto.c
index 32f5077066..cec21dbea7 100644
--- a/drivers/crypto/mlx5/mlx5_crypto.c
+++ b/drivers/crypto/mlx5/mlx5_crypto.c
@@ -19,6 +19,7 @@
#define MLX5_CRYPTO_DRIVER_NAME mlx5_crypto
#define MLX5_CRYPTO_LOG_NAME pmd.crypto.mlx5
+#define MLX5_CRYPTO_MAX_QPS 1024
TAILQ_HEAD(mlx5_crypto_privs, mlx5_crypto_priv) mlx5_crypto_priv_list =
TAILQ_HEAD_INITIALIZER(mlx5_crypto_priv_list);
@@ -28,6 +29,9 @@ int mlx5_crypto_logtype;
uint8_t mlx5_crypto_driver_id;
+const struct rte_cryptodev_capabilities
+ mlx5_crypto_caps[RTE_CRYPTO_OP_TYPE_UNDEFINED];
+
static const char mlx5_crypto_drv_name[] = RTE_STR(MLX5_CRYPTO_DRIVER_NAME);
static const struct rte_driver mlx5_drv = {
@@ -49,22 +53,47 @@ struct mlx5_crypto_session {
uint32_t dek_id; /* DEK ID */
} __rte_packed;
-static unsigned int
-mlx5_crypto_sym_session_get_size(struct rte_cryptodev *dev __rte_unused)
+static void
+mlx5_crypto_dev_infos_get(struct rte_cryptodev *dev,
+ struct rte_cryptodev_info *dev_info)
{
- return sizeof(struct mlx5_crypto_session);
+ RTE_SET_USED(dev);
+ if (dev_info != NULL) {
+ dev_info->driver_id = mlx5_crypto_driver_id;
+ dev_info->feature_flags = 0;
+ dev_info->capabilities = mlx5_crypto_caps;
+ dev_info->max_nb_queue_pairs = MLX5_CRYPTO_MAX_QPS;
+ dev_info->min_mbuf_headroom_req = 0;
+ dev_info->min_mbuf_tailroom_req = 0;
+ dev_info->sym.max_nb_sessions = 0;
+ /*
+* If 0, the device does not have any limitation in number of
+* sessions that can be used.
+*/
+ }
}
static int
mlx5_crypto_dev_configure(struct rte_cryptodev *dev,
- struct rte_cryptodev_config *config __rte_unused)
+ struct rte_cryptodev_config *config)
{
struct mlx5_crypto_priv *priv = dev->data->dev_private;
+ if (config == NULL) {
+ DRV_LOG(ERR, "Invalid crypto dev configure parameters.");
+ return -EINVAL;
+ }
+ if ((config->ff_disable & RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO) != 0) {
+ DRV_LOG(ERR,
+ "Disabled symmetric crypto feature is not supported.");
+ return -ENOTSUP;
+ }
if (mlx5_crypto_dek_setup(priv) != 0) {
DRV_LOG(ERR, "Dek hash list creation has failed.");
return -ENOMEM;
}
+ priv->dev_config = *config;
+ DRV_LOG(DEBUG, "Device %u was configured.", dev->driver_id);
return 0;
}
@@ -74,9 +103,16 @@ mlx5_crypto_dev_close(struct rte_cryptodev *dev)
struct mlx5_crypto_priv *priv = dev->data->dev_private;
mlx5_crypto_dek_unset(priv);
+ DRV_LOG(DEBUG, "Device %u was closed.", dev->driver_id);
return 0;
}
+static unsigned int
+mlx5_crypto_sym_session_get_size(struct rte_cryptodev *dev __rte_unused)
+{
+ return sizeof(struct mlx5_crypto_session);
+}
+
static int
mlx5_crypto_sym_session_configure(struct rte_cryptodev *dev,
struct rte_crypto_sym_xform *xform,
@@ -153,7 +189,7 @@ static struct rte_cryptodev_ops mlx5_crypto_ops = {
.dev_start = NULL,
.dev_stop = NULL,
.dev_close = mlx5_crypto_dev_close,
- .dev_infos_get = NULL,
+ .dev_infos_get = mlx5_crypto_dev_infos_get,
.stats_get = NULL,
.stats_reset= NULL,
.queue_pair_setup = NULL,
diff --git a/drivers/crypto/mlx5/mlx5_crypto.h
b/drivers/crypto/mlx5/mlx5_crypto.h
index 4ec67a7e0f..5e270d3d5a 100644
--- a/drivers/crypto/mlx5/mlx5_crypto.h
+++ b/drivers/crypto/mlx5/mlx5_crypto.h
@@ -24,6 +24,7 @@ struct mlx5_crypto_priv {
uint32_t pdn; /* Protection Domain number. */
[dpdk-dev] [PATCH v3 02/15] crypto/mlx5: add DEK object management
From: Shiri Kuzin
A DEK(Data encryption Key) is an mlx5 HW object which represents the
cipher algorithm key.
The DEKs are used during data encryption/decryption operations.
In symmetric algorithms like AES-STS, we use the same DEK for both
encryption and decryption.
Use the mlx5 hash-list tool to manage the DEK objects in the PMD.
Provide the compare, create and destroy functions to manage DEKs in
hash-list and introduce an internal API to setup and unset the DEK
management and to prepare and destroy specific DEK object.
The DEK hash-list will be created in dev_configure routine and
destroyed in dev_close routine.
Signed-off-by: Shiri Kuzin
Acked-by: Matan Azrad
---
drivers/crypto/mlx5/meson.build | 1 +
drivers/crypto/mlx5/mlx5_crypto.c | 44 +
drivers/crypto/mlx5/mlx5_crypto.h | 51 ++
drivers/crypto/mlx5/mlx5_crypto_dek.c | 136 ++
4 files changed, 215 insertions(+), 17 deletions(-)
create mode 100644 drivers/crypto/mlx5/mlx5_crypto.h
create mode 100644 drivers/crypto/mlx5/mlx5_crypto_dek.c
diff --git a/drivers/crypto/mlx5/meson.build b/drivers/crypto/mlx5/meson.build
index fd00283665..9cb3bd214f 100644
--- a/drivers/crypto/mlx5/meson.build
+++ b/drivers/crypto/mlx5/meson.build
@@ -11,6 +11,7 @@ fmt_name = 'mlx5_crypto'
deps += ['common_mlx5', 'eal', 'cryptodev']
sources = files(
'mlx5_crypto.c',
+ 'mlx5_crypto_dek.c',
)
cflags_options = [
'-std=c11',
diff --git a/drivers/crypto/mlx5/mlx5_crypto.c
b/drivers/crypto/mlx5/mlx5_crypto.c
index ffbce5d68a..2bdfb1a10f 100644
--- a/drivers/crypto/mlx5/mlx5_crypto.c
+++ b/drivers/crypto/mlx5/mlx5_crypto.c
@@ -3,12 +3,9 @@
*/
#include
-#include
#include
+#include
#include
-#include
-#include
-#include
#include
#include
@@ -17,19 +14,10 @@
#include
#include "mlx5_crypto_utils.h"
+#include "mlx5_crypto.h"
#define MLX5_CRYPTO_DRIVER_NAME mlx5_crypto
-#define MLX5_CRYPTO_LOG_NAMEpmd.crypto.mlx5
-
-struct mlx5_crypto_priv {
- TAILQ_ENTRY(mlx5_crypto_priv) next;
- struct ibv_context *ctx; /* Device context. */
- struct rte_pci_device *pci_dev;
- struct rte_cryptodev *crypto_dev;
- void *uar; /* User Access Region. */
- uint32_t pdn; /* Protection Domain number. */
- struct ibv_pd *pd;
-};
+#define MLX5_CRYPTO_LOG_NAME pmd.crypto.mlx5
TAILQ_HEAD(mlx5_crypto_privs, mlx5_crypto_priv) mlx5_crypto_priv_list =
TAILQ_HEAD_INITIALIZER(mlx5_crypto_priv_list);
@@ -48,11 +36,33 @@ static const struct rte_driver mlx5_drv = {
static struct cryptodev_driver mlx5_cryptodev_driver;
+static int
+mlx5_crypto_dev_configure(struct rte_cryptodev *dev,
+ struct rte_cryptodev_config *config __rte_unused)
+{
+ struct mlx5_crypto_priv *priv = dev->data->dev_private;
+
+ if (mlx5_crypto_dek_setup(priv) != 0) {
+ DRV_LOG(ERR, "Dek hash list creation has failed.");
+ return -ENOMEM;
+ }
+ return 0;
+}
+
+static int
+mlx5_crypto_dev_close(struct rte_cryptodev *dev)
+{
+ struct mlx5_crypto_priv *priv = dev->data->dev_private;
+
+ mlx5_crypto_dek_unset(priv);
+ return 0;
+}
+
static struct rte_cryptodev_ops mlx5_crypto_ops = {
- .dev_configure = NULL,
+ .dev_configure = mlx5_crypto_dev_configure,
.dev_start = NULL,
.dev_stop = NULL,
- .dev_close = NULL,
+ .dev_close = mlx5_crypto_dev_close,
.dev_infos_get = NULL,
.stats_get = NULL,
.stats_reset= NULL,
diff --git a/drivers/crypto/mlx5/mlx5_crypto.h
b/drivers/crypto/mlx5/mlx5_crypto.h
new file mode 100644
index 00..4ec67a7e0f
--- /dev/null
+++ b/drivers/crypto/mlx5/mlx5_crypto.h
@@ -0,0 +1,51 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2021 Mellanox Technologies, Ltd
+ */
+
+#ifndef MLX5_CRYPTO_H_
+#define MLX5_CRYPTO_H_
+
+#include
+
+#include
+#include
+
+#include
+
+#define MLX5_CRYPTO_DEK_HTABLE_SZ (1 << 11)
+#define MLX5_CRYPTO_KEY_LENGTH 80
+
+struct mlx5_crypto_priv {
+ TAILQ_ENTRY(mlx5_crypto_priv) next;
+ struct ibv_context *ctx; /* Device context. */
+ struct rte_pci_device *pci_dev;
+ struct rte_cryptodev *crypto_dev;
+ void *uar; /* User Access Region. */
+ uint32_t pdn; /* Protection Domain number. */
+ struct ibv_pd *pd;
+ struct mlx5_hlist *dek_hlist; /* Dek hash list. */
+};
+
+struct mlx5_crypto_dek {
+ struct mlx5_hlist_entry entry; /* Pointer to DEK hash list entry. */
+ struct mlx5_devx_obj *obj; /* Pointer to DEK DevX object. */
+ uint8_t data[MLX5_CRYPTO_KEY_LENGTH]; /* DEK key data. */
+
[dpdk-dev] [PATCH v3 05/15] crypto/mlx5: support queue pairs operations
From: Shiri Kuzin
The HW queue pairs are a pair of send queue and receive queue of
independent work queues packed together in one object for the purpose
of transferring data between nodes of a network.
Completion Queue is a FIFO queue of completed work requests.
In crypto driver we use one QP in loopback in order to encrypt and
decrypt data locally without sending it to the wire.
In the configured QP we only use the SQ to perform the encryption and
decryption operations.
Added implementation for the QP setup function which creates the CQ,
creates the QP and changes its state to RTS (ready to send).
Added implementation for the release QP function to release all the QP
resources.
Added the ops structure that contains any operation which is supported
by the cryptodev.
Signed-off-by: Shiri Kuzin
Acked-by: Matan Azrad
---
drivers/crypto/mlx5/mlx5_crypto.c | 124 +-
drivers/crypto/mlx5/mlx5_crypto.h | 11 +++
2 files changed, 133 insertions(+), 2 deletions(-)
diff --git a/drivers/crypto/mlx5/mlx5_crypto.c
b/drivers/crypto/mlx5/mlx5_crypto.c
index cec21dbea7..8c3417ee96 100644
--- a/drivers/crypto/mlx5/mlx5_crypto.c
+++ b/drivers/crypto/mlx5/mlx5_crypto.c
@@ -7,6 +7,7 @@
#include
#include
#include
+#include
#include
#include
@@ -184,6 +185,125 @@ mlx5_crypto_sym_session_clear(struct rte_cryptodev *dev,
DRV_LOG(DEBUG, "Session %p was cleared.", sess_private_data);
}
+static int
+mlx5_crypto_queue_pair_release(struct rte_cryptodev *dev, uint16_t qp_id)
+{
+ struct mlx5_crypto_qp *qp = dev->data->queue_pairs[qp_id];
+
+ if (qp->qp_obj != NULL)
+ claim_zero(mlx5_devx_cmd_destroy(qp->qp_obj));
+ if (qp->umem_obj != NULL)
+ claim_zero(mlx5_glue->devx_umem_dereg(qp->umem_obj));
+ if (qp->umem_buf != NULL)
+ rte_free(qp->umem_buf);
+ mlx5_devx_cq_destroy(&qp->cq_obj);
+ rte_free(qp);
+ dev->data->queue_pairs[qp_id] = NULL;
+ return 0;
+}
+
+static int
+mlx5_crypto_qp2rts(struct mlx5_crypto_qp *qp)
+{
+ /*
+* In Order to configure self loopback, when calling these functions the
+* remote QP id that is used is the id of the same QP.
+*/
+ if (mlx5_devx_cmd_modify_qp_state(qp->qp_obj, MLX5_CMD_OP_RST2INIT_QP,
+ qp->qp_obj->id)) {
+ DRV_LOG(ERR, "Failed to modify QP to INIT state(%u).",
+ rte_errno);
+ return -1;
+ }
+ if (mlx5_devx_cmd_modify_qp_state(qp->qp_obj, MLX5_CMD_OP_INIT2RTR_QP,
+ qp->qp_obj->id)) {
+ DRV_LOG(ERR, "Failed to modify QP to RTR state(%u).",
+ rte_errno);
+ return -1;
+ }
+ if (mlx5_devx_cmd_modify_qp_state(qp->qp_obj, MLX5_CMD_OP_RTR2RTS_QP,
+ qp->qp_obj->id)) {
+ DRV_LOG(ERR, "Failed to modify QP to RTS state(%u).",
+ rte_errno);
+ return -1;
+ }
+ return 0;
+}
+
+static int
+mlx5_crypto_queue_pair_setup(struct rte_cryptodev *dev, uint16_t qp_id,
+const struct rte_cryptodev_qp_conf *qp_conf,
+int socket_id)
+{
+ struct mlx5_crypto_priv *priv = dev->data->dev_private;
+ struct mlx5_devx_qp_attr attr = {0};
+ struct mlx5_crypto_qp *qp;
+ uint16_t log_nb_desc = rte_log2_u32(qp_conf->nb_descriptors);
+ uint32_t umem_size = RTE_BIT32(log_nb_desc) *
+ MLX5_CRYPTO_WQE_SET_SIZE +
+ sizeof(*qp->db_rec) * 2;
+ uint32_t alloc_size = sizeof(*qp);
+ struct mlx5_devx_cq_attr cq_attr = {
+ .uar_page_id = mlx5_os_get_devx_uar_page_id(priv->uar),
+ };
+
+ alloc_size = RTE_ALIGN(alloc_size, RTE_CACHE_LINE_SIZE);
+ alloc_size += sizeof(struct rte_crypto_op *) * RTE_BIT32(log_nb_desc);
+ qp = rte_zmalloc_socket(__func__, alloc_size, RTE_CACHE_LINE_SIZE,
+ socket_id);
+ if (qp == NULL) {
+ DRV_LOG(ERR, "Failed to allocate QP memory.");
+ rte_errno = ENOMEM;
+ return -rte_errno;
+ }
+ if (mlx5_devx_cq_create(priv->ctx, &qp->cq_obj, log_nb_desc,
+ &cq_attr, socket_id) != 0) {
+ DRV_LOG(ERR, "Failed to create CQ.");
+ goto error;
+ }
+ qp->umem_buf = rte_zmalloc_socket(__func__, umem_size, 4096, socket_id);
+ if (qp->umem_buf == NULL) {
+ DRV_LOG(ERR, "Failed to allocate QP umem.");
+ rte_errno = ENOMEM;
+ goto error;
+ }
+ qp->umem_obj
[dpdk-dev] [PATCH v3 03/15] crypto/mlx5: support session operations
From: Shiri Kuzin
Sessions are used in symmetric transformations in order to prepare
objects and data for packet processing stage.
A mlx5 session includes iv_offset, pointer to mlx5_crypto_dek struct,
bsf_size, bsf_p_type, encryption_order and encryption standard.
Implement the next session operations:
mlx5_crypto_sym_session_get_size- returns the size of the mlx5
session struct.
mlx5_crypto_sym_session_configure- prepares the DEK hash-list
and saves all the session data.
mlx5_crypto_sym_session_clear - destroys the DEK hash-list.
Signed-off-by: Shiri Kuzin
Acked-by: Matan Azrad
---
drivers/crypto/mlx5/mlx5_crypto.c | 96 ++-
1 file changed, 93 insertions(+), 3 deletions(-)
diff --git a/drivers/crypto/mlx5/mlx5_crypto.c
b/drivers/crypto/mlx5/mlx5_crypto.c
index 2bdfb1a10f..32f5077066 100644
--- a/drivers/crypto/mlx5/mlx5_crypto.c
+++ b/drivers/crypto/mlx5/mlx5_crypto.c
@@ -3,6 +3,7 @@
*/
#include
+#include
#include
#include
#include
@@ -36,6 +37,24 @@ static const struct rte_driver mlx5_drv = {
static struct cryptodev_driver mlx5_cryptodev_driver;
+struct mlx5_crypto_session {
+ uint32_t bs_bpt_eo_es;
+ /*
+* bsf_size, bsf_p_type, encryption_order and encryption standard,
+* saved in big endian format.
+*/
+ uint32_t iv_offset:16;
+ /* Starting point for Initialisation Vector. */
+ struct mlx5_crypto_dek *dek; /* Pointer to dek struct. */
+ uint32_t dek_id; /* DEK ID */
+} __rte_packed;
+
+static unsigned int
+mlx5_crypto_sym_session_get_size(struct rte_cryptodev *dev __rte_unused)
+{
+ return sizeof(struct mlx5_crypto_session);
+}
+
static int
mlx5_crypto_dev_configure(struct rte_cryptodev *dev,
struct rte_cryptodev_config *config __rte_unused)
@@ -58,6 +77,77 @@ mlx5_crypto_dev_close(struct rte_cryptodev *dev)
return 0;
}
+static int
+mlx5_crypto_sym_session_configure(struct rte_cryptodev *dev,
+ struct rte_crypto_sym_xform *xform,
+ struct rte_cryptodev_sym_session *session,
+ struct rte_mempool *mp)
+{
+ struct mlx5_crypto_priv *priv = dev->data->dev_private;
+ struct mlx5_crypto_session *sess_private_data;
+ struct rte_crypto_cipher_xform *cipher;
+ uint8_t encryption_order;
+ int ret;
+
+ if (unlikely(xform->next != NULL)) {
+ DRV_LOG(ERR, "Xform next is not supported.");
+ return -ENOTSUP;
+ }
+ if (unlikely((xform->type != RTE_CRYPTO_SYM_XFORM_CIPHER) ||
+(xform->cipher.algo != RTE_CRYPTO_CIPHER_AES_XTS))) {
+ DRV_LOG(ERR, "Only AES-XTS algorithm is supported.");
+ return -ENOTSUP;
+ }
+ ret = rte_mempool_get(mp, (void *)&sess_private_data);
+ if (ret != 0) {
+ DRV_LOG(ERR,
+ "Failed to get session %p private data from mempool.",
+ sess_private_data);
+ return -ENOMEM;
+ }
+ cipher = &xform->cipher;
+ sess_private_data->dek = mlx5_crypto_dek_prepare(priv, cipher);
+ if (sess_private_data->dek == NULL) {
+ rte_mempool_put(mp, sess_private_data);
+ DRV_LOG(ERR, "Failed to prepare dek.");
+ return -ENOMEM;
+ }
+ if (cipher->op == RTE_CRYPTO_CIPHER_OP_ENCRYPT)
+ encryption_order = MLX5_ENCRYPTION_ORDER_ENCRYPTED_RAW_MEMORY;
+ else
+ encryption_order = MLX5_ENCRYPTION_ORDER_ENCRYPTED_RAW_WIRE;
+ sess_private_data->bs_bpt_eo_es = rte_cpu_to_be_32
+ (MLX5_BSF_SIZE_64B << MLX5_BSF_SIZE_OFFSET |
+MLX5_BSF_P_TYPE_CRYPTO << MLX5_BSF_P_TYPE_OFFSET |
+encryption_order << MLX5_ENCRYPTION_ORDER_OFFSET |
+MLX5_ENCRYPTION_STANDARD_AES_XTS);
+ sess_private_data->iv_offset = cipher->iv.offset;
+ sess_private_data->dek_id =
+ rte_cpu_to_be_32(sess_private_data->dek->obj->id &
+0xff);
+ set_sym_session_private_data(session, dev->driver_id,
+sess_private_data);
+ DRV_LOG(DEBUG, "Session %p was configured.", sess_private_data);
+ return 0;
+}
+
+static void
+mlx5_crypto_sym_session_clear(struct rte_cryptodev *dev,
+ struct rte_cryptodev_sym_session *sess)
+{
+ struct mlx5_crypto_priv *priv = dev->data->dev_private;
+ struct mlx5_crypto_session *sess_private_data =
+ get_sym_session_private_data(sess, dev->driver_id);
+
+ if (unlikely(sess_private_data == NULL)) {
+
[dpdk-dev] [PATCH v3 06/15] crypto/mlx5: add dev stop and start operations
From: Shiri Kuzin
Add the dev_start function that is used to start a configured device.
Add the dev_stop function that is used to stop a configured device.
Both functions set the dev parameter as used and return 0.
Signed-off-by: Shiri Kuzin
Acked-by: Matan Azrad
---
drivers/crypto/mlx5/mlx5_crypto.c | 17 +++--
1 file changed, 15 insertions(+), 2 deletions(-)
diff --git a/drivers/crypto/mlx5/mlx5_crypto.c
b/drivers/crypto/mlx5/mlx5_crypto.c
index 8c3417ee96..538fe5ce56 100644
--- a/drivers/crypto/mlx5/mlx5_crypto.c
+++ b/drivers/crypto/mlx5/mlx5_crypto.c
@@ -98,6 +98,19 @@ mlx5_crypto_dev_configure(struct rte_cryptodev *dev,
return 0;
}
+static void
+mlx5_crypto_dev_stop(struct rte_cryptodev *dev)
+{
+ RTE_SET_USED(dev);
+}
+
+static int
+mlx5_crypto_dev_start(struct rte_cryptodev *dev)
+{
+ RTE_SET_USED(dev);
+ return 0;
+}
+
static int
mlx5_crypto_dev_close(struct rte_cryptodev *dev)
{
@@ -306,8 +319,8 @@ mlx5_crypto_queue_pair_setup(struct rte_cryptodev *dev,
uint16_t qp_id,
static struct rte_cryptodev_ops mlx5_crypto_ops = {
.dev_configure = mlx5_crypto_dev_configure,
- .dev_start = NULL,
- .dev_stop = NULL,
+ .dev_start = mlx5_crypto_dev_start,
+ .dev_stop = mlx5_crypto_dev_stop,
.dev_close = mlx5_crypto_dev_close,
.dev_infos_get = mlx5_crypto_dev_infos_get,
.stats_get = NULL,
--
2.25.1
[dpdk-dev] [PATCH v3 08/15] crypto/mlx5: create login object using DevX
From: Shiri Kuzin
To work with crypto engines that are marked with wrapped_import_method,
a login session is required.
A crypto login object needs to be created using DevX.
The crypto login object contains:
- The credential pointer.
- The import_KEK pointer to be used for all secured information
communicated in crypto commands (key fields), including the
provided credential in this command.
- The credential secret, wrapped by the import_KEK indicated in
this command. Size includes 8 bytes IV for wrapping.
Added devargs for the required login values:
- wcs_file - path to the file containing the credential.
- import_kek_id - the import KEK pointer.
- credential_id - the credential pointer.
Create the login DevX object in pci_probe function and destroy it in
pci_remove.
Destroying the crypto login object means logout.
Signed-off-by: Shiri Kuzin
Acked-by: Matan Azrad
---
drivers/crypto/mlx5/mlx5_crypto.c | 103 ++
drivers/crypto/mlx5/mlx5_crypto.h | 7 ++
2 files changed, 110 insertions(+)
diff --git a/drivers/crypto/mlx5/mlx5_crypto.c
b/drivers/crypto/mlx5/mlx5_crypto.c
index b95aea0068..18b1a6be88 100644
--- a/drivers/crypto/mlx5/mlx5_crypto.c
+++ b/drivers/crypto/mlx5/mlx5_crypto.c
@@ -402,6 +402,101 @@ mlx5_crypto_hw_global_prepare(struct mlx5_crypto_priv
*priv)
return 0;
}
+
+static int
+mlx5_crypto_args_check_handler(const char *key, const char *val, void *opaque)
+{
+ struct mlx5_crypto_devarg_params *devarg_prms = opaque;
+ struct mlx5_devx_crypto_login_attr *attr = &devarg_prms->login_attr;
+ unsigned long tmp;
+ FILE *file;
+ int ret;
+ int i;
+
+ if (strcmp(key, "class") == 0)
+ return 0;
+ if (strcmp(key, "wcs_file") == 0) {
+ file = fopen(val, "rb");
+ if (file == NULL) {
+ rte_errno = ENOTSUP;
+ return -rte_errno;
+ }
+ for (i = 0 ; i < MLX5_CRYPTO_CREDENTIAL_SIZE ; i++) {
+ ret = fscanf(file, "%02hhX", &attr->credential[i]);
+ if (ret <= 0) {
+ fclose(file);
+ DRV_LOG(ERR,
+ "Failed to read credential from file.");
+ rte_errno = EINVAL;
+ return -rte_errno;
+ }
+ }
+ fclose(file);
+ devarg_prms->login_devarg = true;
+ return 0;
+ }
+ errno = 0;
+ tmp = strtoul(val, NULL, 0);
+ if (errno) {
+ DRV_LOG(WARNING, "%s: \"%s\" is an invalid integer.", key, val);
+ return -errno;
+ }
+ if (strcmp(key, "import_kek_id") == 0)
+ attr->session_import_kek_ptr = (uint32_t)tmp;
+ else if (strcmp(key, "credential_id") == 0)
+ attr->credential_pointer = (uint32_t)tmp;
+ else
+ DRV_LOG(WARNING, "Invalid key %s.", key);
+ return 0;
+}
+
+static struct mlx5_devx_obj *
+mlx5_crypto_config_login(struct rte_devargs *devargs,
+struct ibv_context *ctx)
+{
+ /*
+* Set credential pointer and session import KEK pointer to a default
+* value of 0.
+*/
+ struct mlx5_crypto_devarg_params login = {
+ .login_devarg = false,
+ .login_attr = {
+ .credential_pointer = 0,
+ .session_import_kek_ptr = 0,
+ }
+ };
+ struct rte_kvargs *kvlist;
+
+ if (devargs == NULL) {
+ DRV_LOG(ERR,
+ "No login devargs in order to enable crypto operations in the device.");
+ rte_errno = EINVAL;
+ return NULL;
+ }
+ kvlist = rte_kvargs_parse(devargs->args, NULL);
+ if (kvlist == NULL) {
+ DRV_LOG(ERR, "Failed to parse devargs.");
+ rte_errno = EINVAL;
+ return NULL;
+ }
+ if (rte_kvargs_process(kvlist, NULL, mlx5_crypto_args_check_handler,
+ &login) != 0) {
+ DRV_LOG(ERR, "Devargs handler function Failed.");
+ rte_kvargs_free(kvlist);
+ rte_errno = EINVAL;
+ return NULL;
+ }
+ rte_kvargs_free(kvlist);
+ if (login.login_devarg == false) {
+ DRV_LOG(ERR,
+ "No login credential devarg in order to enable crypto operations "
+ "in the device.");
+ rte_errno = EINVAL;
+ return NULL;
+ }
+ retur
[dpdk-dev] [PATCH v3 07/15] crypto/mlx5: add memory region management
From: Shiri Kuzin
Mellanox user space drivers don't deal with physical addresses as part
of a memory protection mechanism.
The device translates the given virtual address to a physical address
using the given memory key as an address space identifier.
That's why any mbuf virtual address is moved directly to the HW
descriptor(WQE).
The mapping between the virtual address to the physical address is saved
in MR configured by the kernel to the HW.
Each MR has a key that should also be moved to the WQE by the SW.
When the SW sees an unmapped address, it extends the address range and
creates a MR using a system call.
Add memory region cache management:
- 2 level cache per queue-pair - no locks.
- 1 shared cache between all the queues using a lock.
Using this way, the MR key search per data-path address is optimized.
Signed-off-by: Shiri Kuzin
Acked-by: Matan Azrad
---
drivers/crypto/mlx5/mlx5_crypto.c | 20
drivers/crypto/mlx5/mlx5_crypto.h | 3 +++
2 files changed, 23 insertions(+)
diff --git a/drivers/crypto/mlx5/mlx5_crypto.c
b/drivers/crypto/mlx5/mlx5_crypto.c
index 538fe5ce56..b95aea0068 100644
--- a/drivers/crypto/mlx5/mlx5_crypto.c
+++ b/drivers/crypto/mlx5/mlx5_crypto.c
@@ -209,6 +209,7 @@ mlx5_crypto_queue_pair_release(struct rte_cryptodev *dev,
uint16_t qp_id)
claim_zero(mlx5_glue->devx_umem_dereg(qp->umem_obj));
if (qp->umem_buf != NULL)
rte_free(qp->umem_buf);
+ mlx5_mr_btree_free(&qp->mr_ctrl.cache_bh);
mlx5_devx_cq_destroy(&qp->cq_obj);
rte_free(qp);
dev->data->queue_pairs[qp_id] = NULL;
@@ -288,6 +289,13 @@ mlx5_crypto_queue_pair_setup(struct rte_cryptodev *dev,
uint16_t qp_id,
DRV_LOG(ERR, "Failed to register QP umem.");
goto error;
}
+ if (mlx5_mr_btree_init(&qp->mr_ctrl.cache_bh, MLX5_MR_BTREE_CACHE_N,
+ priv->dev_config.socket_id) != 0) {
+ DRV_LOG(ERR, "Cannot allocate MR Btree for qp %u.",
+ (uint32_t)qp_id);
+ rte_errno = ENOMEM;
+ goto error;
+ }
attr.pd = priv->pdn;
attr.uar_index = mlx5_os_get_devx_uar_page_id(priv->uar);
attr.cqn = qp->cq_obj.cq->id;
@@ -476,6 +484,17 @@ mlx5_crypto_pci_probe(struct rte_pci_driver *pci_drv,
claim_zero(mlx5_glue->close_device(priv->ctx));
return -1;
}
+ if (mlx5_mr_btree_init(&priv->mr_scache.cache,
+MLX5_MR_BTREE_CACHE_N * 2, rte_socket_id()) != 0) {
+ DRV_LOG(ERR, "Failed to allocate shared cache MR memory.");
+ mlx5_crypto_hw_global_release(priv);
+ rte_cryptodev_pmd_destroy(priv->crypto_dev);
+ claim_zero(mlx5_glue->close_device(priv->ctx));
+ rte_errno = ENOMEM;
+ return -rte_errno;
+ }
+ priv->mr_scache.reg_mr_cb = mlx5_common_verbs_reg_mr;
+ priv->mr_scache.dereg_mr_cb = mlx5_common_verbs_dereg_mr;
pthread_mutex_lock(&priv_list_lock);
TAILQ_INSERT_TAIL(&mlx5_crypto_priv_list, priv, next);
pthread_mutex_unlock(&priv_list_lock);
@@ -495,6 +514,7 @@ mlx5_crypto_pci_remove(struct rte_pci_device *pdev)
TAILQ_REMOVE(&mlx5_crypto_priv_list, priv, next);
pthread_mutex_unlock(&priv_list_lock);
if (priv) {
+ mlx5_mr_release_cache(&priv->mr_scache);
mlx5_crypto_hw_global_release(priv);
rte_cryptodev_pmd_destroy(priv->crypto_dev);
claim_zero(mlx5_glue->close_device(priv->ctx));
diff --git a/drivers/crypto/mlx5/mlx5_crypto.h
b/drivers/crypto/mlx5/mlx5_crypto.h
index f5313b89f2..397267d249 100644
--- a/drivers/crypto/mlx5/mlx5_crypto.h
+++ b/drivers/crypto/mlx5/mlx5_crypto.h
@@ -12,6 +12,7 @@
#include
#include
+#include
#define MLX5_CRYPTO_DEK_HTABLE_SZ (1 << 11)
#define MLX5_CRYPTO_KEY_LENGTH 80
@@ -27,6 +28,7 @@ struct mlx5_crypto_priv {
struct ibv_pd *pd;
struct mlx5_hlist *dek_hlist; /* Dek hash list. */
struct rte_cryptodev_config dev_config;
+ struct mlx5_mr_share_cache mr_scache; /* Global shared MR cache. */
};
struct mlx5_crypto_qp {
@@ -36,6 +38,7 @@ struct mlx5_crypto_qp {
void *umem_buf;
volatile uint32_t *db_rec;
struct rte_crypto_op **ops;
+ struct mlx5_mr_ctrl mr_ctrl;
};
struct mlx5_crypto_dek {
--
2.25.1
[dpdk-dev] [PATCH v3 10/15] crypto/mlx5: add keytag device argument
From: Suanming Mou
A keytag is a piece of data encrypted together with a DEK.
When a DEK is referenced by an MKEY.bsf through its index, the keytag is
also supplied in the BSF as plaintext. The HW will decrypt the DEK (and
the attached keytag) and will fail the operation if the keytags don't
match.
This commit adds the configuration of the keytag with devargs.
Signed-off-by: Suanming Mou
Signed-off-by: Matan Azrad
---
drivers/crypto/mlx5/mlx5_crypto.c | 50 +--
drivers/crypto/mlx5/mlx5_crypto.h | 3 +-
2 files changed, 30 insertions(+), 23 deletions(-)
diff --git a/drivers/crypto/mlx5/mlx5_crypto.c
b/drivers/crypto/mlx5/mlx5_crypto.c
index 8cc29ced21..73cca8136b 100644
--- a/drivers/crypto/mlx5/mlx5_crypto.c
+++ b/drivers/crypto/mlx5/mlx5_crypto.c
@@ -468,56 +468,52 @@ mlx5_crypto_args_check_handler(const char *key, const
char *val, void *opaque)
attr->session_import_kek_ptr = (uint32_t)tmp;
else if (strcmp(key, "credential_id") == 0)
attr->credential_pointer = (uint32_t)tmp;
+ else if (strcmp(key, "keytag") == 0)
+ devarg_prms->keytag = tmp;
else
DRV_LOG(WARNING, "Invalid key %s.", key);
return 0;
}
-static struct mlx5_devx_obj *
-mlx5_crypto_config_login(struct rte_devargs *devargs,
-struct ibv_context *ctx)
+static int
+mlx5_crypto_parse_devargs(struct rte_devargs *devargs,
+ struct mlx5_crypto_devarg_params *devarg_prms)
{
- /*
-* Set credential pointer and session import KEK pointer to a default
-* value of 0.
-*/
- struct mlx5_crypto_devarg_params login = {
- .login_devarg = false,
- .login_attr = {
- .credential_pointer = 0,
- .session_import_kek_ptr = 0,
- }
- };
+ struct mlx5_devx_crypto_login_attr *attr = &devarg_prms->login_attr;
struct rte_kvargs *kvlist;
+ /* Default values. */
+ attr->credential_pointer = 0;
+ attr->session_import_kek_ptr = 0;
+ devarg_prms->keytag = 0;
if (devargs == NULL) {
DRV_LOG(ERR,
"No login devargs in order to enable crypto operations in the device.");
rte_errno = EINVAL;
- return NULL;
+ return -1;
}
kvlist = rte_kvargs_parse(devargs->args, NULL);
if (kvlist == NULL) {
DRV_LOG(ERR, "Failed to parse devargs.");
rte_errno = EINVAL;
- return NULL;
+ return -1;
}
if (rte_kvargs_process(kvlist, NULL, mlx5_crypto_args_check_handler,
- &login) != 0) {
+ devarg_prms) != 0) {
DRV_LOG(ERR, "Devargs handler function Failed.");
rte_kvargs_free(kvlist);
rte_errno = EINVAL;
- return NULL;
+ return -1;
}
rte_kvargs_free(kvlist);
- if (login.login_devarg == false) {
+ if (devarg_prms->login_devarg == false) {
DRV_LOG(ERR,
"No login credential devarg in order to enable crypto operations "
"in the device.");
rte_errno = EINVAL;
- return NULL;
+ return -1;
}
- return mlx5_devx_cmd_create_crypto_login_obj(ctx, &login.login_attr);
+ return 0;
}
/**
@@ -543,6 +539,7 @@ mlx5_crypto_pci_probe(struct rte_pci_driver *pci_drv,
struct ibv_context *ctx;
struct mlx5_devx_obj *login;
struct mlx5_crypto_priv *priv;
+ struct mlx5_crypto_devarg_params devarg_prms = { 0 };
struct mlx5_hca_attr attr = { 0 };
struct rte_cryptodev_pmd_init_params init_params = {
.name = "",
@@ -551,6 +548,8 @@ mlx5_crypto_pci_probe(struct rte_pci_driver *pci_drv,
.max_nb_queue_pairs =
RTE_CRYPTODEV_PMD_DEFAULT_MAX_NB_QUEUE_PAIRS,
};
+ int ret;
+
RTE_SET_USED(pci_drv);
if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
DRV_LOG(ERR, "Non-primary process type is not supported.");
@@ -580,7 +579,13 @@ mlx5_crypto_pci_probe(struct rte_pci_driver *pci_drv,
rte_errno = ENOTSUP;
return -ENOTSUP;
}
- login = mlx5_crypto_config_login(pci_dev->device.devargs, ctx);
+ ret = mlx5_crypto_parse_devargs(pci_dev->device.devargs, &devarg_prms);
+ if (ret) {
+ DRV_LOG(ERR, "Failed to parse devargs.");
+ return -rte_errno;
+ }
+ login = mlx5_devx_cmd_create_crypto_login_obj(ctx,
+
[dpdk-dev] [PATCH v3 09/15] crypto/mlx5: adjust to the multiple data unit API
From: Shiri Kuzin
In AES-XTS the data to be encrypted\decrypted does not have to be
in multiples of 16B size, the unit of data is called data-unit.
As a result of patch [1] a new field is added to the cipher capability,
called dataunit_set, where the devices can report the range of
supported data-unit sizes.
The new field enables saving the data-unit size in the session
structure to the block size pointer variable in order to support
several data-unit sizes.
[1] https://www.mail-archive.com/dev@dpdk.org/msg205337.html
Signed-off-by: Shiri Kuzin
---
drivers/crypto/mlx5/mlx5_crypto.c | 23 +++
1 file changed, 23 insertions(+)
diff --git a/drivers/crypto/mlx5/mlx5_crypto.c
b/drivers/crypto/mlx5/mlx5_crypto.c
index 18b1a6be88..8cc29ced21 100644
--- a/drivers/crypto/mlx5/mlx5_crypto.c
+++ b/drivers/crypto/mlx5/mlx5_crypto.c
@@ -48,6 +48,11 @@ struct mlx5_crypto_session {
* bsf_size, bsf_p_type, encryption_order and encryption standard,
* saved in big endian format.
*/
+ uint32_t bsp_res;
+ /*
+* crypto_block_size_pointer and reserved 24 bits saved in big endian
+* format.
+*/
uint32_t iv_offset:16;
/* Starting point for Initialisation Vector. */
struct mlx5_crypto_dek *dek; /* Pointer to dek struct. */
@@ -171,6 +176,24 @@ mlx5_crypto_sym_session_configure(struct rte_cryptodev
*dev,
MLX5_BSF_P_TYPE_CRYPTO << MLX5_BSF_P_TYPE_OFFSET |
encryption_order << MLX5_ENCRYPTION_ORDER_OFFSET |
MLX5_ENCRYPTION_STANDARD_AES_XTS);
+ switch (xform->cipher.dataunit_len) {
+ case 0:
+ sess_private_data->bsp_res = 0;
+ break;
+ case 512:
+ sess_private_data->bsp_res = rte_cpu_to_be_32
+((uint32_t)MLX5_BLOCK_SIZE_512B <<
+MLX5_BLOCK_SIZE_OFFSET);
+ break;
+ case 4096:
+ sess_private_data->bsp_res = rte_cpu_to_be_32
+((uint32_t)MLX5_BLOCK_SIZE_4096B <<
+MLX5_BLOCK_SIZE_OFFSET);
+ break;
+ default:
+ DRV_LOG(ERR, "Cipher data unit length is not supported.");
+ return -ENOTSUP;
+ }
sess_private_data->iv_offset = cipher->iv.offset;
sess_private_data->dek_id =
rte_cpu_to_be_32(sess_private_data->dek->obj->id &
--
2.25.1
[dpdk-dev] [PATCH v3 11/15] crypto/mlx5: add maximum segments device argument
From: Suanming Mou
The mlx5 HW crypto operations are done by attaching crypto property
to a memory region. Once done, every access to the memory via the
crypto-enabled memory region will result with in-line encryption or
decryption of the data.
As a result, the design choice is to provide two types of WQEs. One
is UMR WQE which sets the crypto property and the other is rdma write
WQE which sends DMA command to copy data from local MR to remote MR.
The size of the WQEs will be defined by a new devarg called
max_segs_num.
This devarg also defines the maximum segments in mbuf chain that will be
supported for crypto operations.
Signed-off-by: Suanming Mou
Signed-off-by: Matan Azrad
---
drivers/crypto/mlx5/mlx5_crypto.c | 35 +++
drivers/crypto/mlx5/mlx5_crypto.h | 7 +++
2 files changed, 38 insertions(+), 4 deletions(-)
diff --git a/drivers/crypto/mlx5/mlx5_crypto.c
b/drivers/crypto/mlx5/mlx5_crypto.c
index 73cca8136b..6de44398bd 100644
--- a/drivers/crypto/mlx5/mlx5_crypto.c
+++ b/drivers/crypto/mlx5/mlx5_crypto.c
@@ -21,6 +21,7 @@
#define MLX5_CRYPTO_DRIVER_NAME mlx5_crypto
#define MLX5_CRYPTO_LOG_NAME pmd.crypto.mlx5
#define MLX5_CRYPTO_MAX_QPS 1024
+#define MLX5_CRYPTO_MAX_SEGS 56
TAILQ_HEAD(mlx5_crypto_privs, mlx5_crypto_priv) mlx5_crypto_priv_list =
TAILQ_HEAD_INITIALIZER(mlx5_crypto_priv_list);
@@ -464,14 +465,24 @@ mlx5_crypto_args_check_handler(const char *key, const
char *val, void *opaque)
DRV_LOG(WARNING, "%s: \"%s\" is an invalid integer.", key, val);
return -errno;
}
- if (strcmp(key, "import_kek_id") == 0)
+ if (strcmp(key, "max_segs_num") == 0) {
+ if (!tmp || tmp > MLX5_CRYPTO_MAX_SEGS) {
+ DRV_LOG(WARNING, "Invalid max_segs_num: %d, should"
+ " be less than %d.",
+ (uint32_t)tmp, MLX5_CRYPTO_MAX_SEGS);
+ rte_errno = EINVAL;
+ return -rte_errno;
+ }
+ devarg_prms->max_segs_num = (uint32_t)tmp;
+ } else if (strcmp(key, "import_kek_id") == 0) {
attr->session_import_kek_ptr = (uint32_t)tmp;
- else if (strcmp(key, "credential_id") == 0)
+ } else if (strcmp(key, "credential_id") == 0) {
attr->credential_pointer = (uint32_t)tmp;
- else if (strcmp(key, "keytag") == 0)
+ } else if (strcmp(key, "keytag") == 0) {
devarg_prms->keytag = tmp;
- else
+ } else {
DRV_LOG(WARNING, "Invalid key %s.", key);
+ }
return 0;
}
@@ -486,6 +497,7 @@ mlx5_crypto_parse_devargs(struct rte_devargs *devargs,
attr->credential_pointer = 0;
attr->session_import_kek_ptr = 0;
devarg_prms->keytag = 0;
+ devarg_prms->max_segs_num = 8;
if (devargs == NULL) {
DRV_LOG(ERR,
"No login devargs in order to enable crypto operations in the device.");
@@ -626,6 +638,21 @@ mlx5_crypto_pci_probe(struct rte_pci_driver *pci_drv,
priv->mr_scache.reg_mr_cb = mlx5_common_verbs_reg_mr;
priv->mr_scache.dereg_mr_cb = mlx5_common_verbs_dereg_mr;
priv->keytag = rte_cpu_to_be_64(devarg_prms.keytag);
+ priv->max_segs_num = devarg_prms.max_segs_num;
+ priv->umr_wqe_size = sizeof(struct mlx5_wqe_umr_bsf_seg) +
+sizeof(struct mlx5_umr_wqe) +
+RTE_ALIGN(priv->max_segs_num, 4) *
+sizeof(struct mlx5_wqe_dseg);
+ priv->rdmw_wqe_size = sizeof(struct mlx5_rdma_write_wqe) +
+ sizeof(struct mlx5_wqe_dseg) *
+ (priv->max_segs_num <= 2 ? 2 : 2 +
+ RTE_ALIGN(priv->max_segs_num - 2, 4));
+ priv->wqe_set_size = priv->umr_wqe_size + priv->rdmw_wqe_size;
+ priv->wqe_stride = (priv->umr_wqe_size + priv->rdmw_wqe_size) /
+ MLX5_SEND_WQE_BB;
+ priv->max_rdmaw_klm_n = (priv->rdmw_wqe_size -
+sizeof(struct mlx5_rdma_write_wqe)) /
+sizeof(struct mlx5_wqe_dseg);
pthread_mutex_lock(&priv_list_lock);
TAILQ_INSERT_TAIL(&mlx5_crypto_priv_list, priv, next);
pthread_mutex_unlock(&priv_list_lock);
diff --git a/drivers/crypto/mlx5/mlx5_crypto.h
b/drivers/crypto/mlx5/mlx5_crypto.h
index 34c65f9a24..81452bd700 100644
--- a/drivers/crypto/mlx5/mlx5_crypto.h
+++ b/drivers/crypto/mlx5/mlx5_crypto.h
@@ -25,12 +25,18 @@ struct mlx5_crypto_priv {
struct rte_cryptodev *crypto_dev;
void *uar; /* U
[dpdk-dev] [PATCH v3 12/15] crypto/mlx5: add WQE set initialization
From: Suanming Mou
Currently, HW handles the WQEs much faster than the software,
Using the constant WQE set layout can initialize most of the WQE
segments in advanced, and software only needs to configure very
limited segments in datapath. This accelerates the software WQE
organize in datapath.
This commit initializes the fixed WQE set segments.
Signed-off-by: Suanming Mou
Signed-off-by: Matan Azrad
---
drivers/crypto/mlx5/mlx5_crypto.c | 83 +--
drivers/crypto/mlx5/mlx5_crypto.h | 10 +++-
2 files changed, 87 insertions(+), 6 deletions(-)
diff --git a/drivers/crypto/mlx5/mlx5_crypto.c
b/drivers/crypto/mlx5/mlx5_crypto.c
index 6de44398bd..7bffe08bfe 100644
--- a/drivers/crypto/mlx5/mlx5_crypto.c
+++ b/drivers/crypto/mlx5/mlx5_crypto.c
@@ -268,6 +268,69 @@ mlx5_crypto_qp2rts(struct mlx5_crypto_qp *qp)
return 0;
}
+static void
+mlx5_crypto_qp_init(struct mlx5_crypto_priv *priv, struct mlx5_crypto_qp *qp)
+{
+ uint32_t i;
+
+ for (i = 0 ; i < qp->entries_n; i++) {
+ struct mlx5_wqe_cseg *cseg = RTE_PTR_ADD(qp->umem_buf, i *
+priv->wqe_set_size);
+ struct mlx5_wqe_umr_cseg *ucseg = (struct mlx5_wqe_umr_cseg *)
+(cseg + 1);
+ struct mlx5_wqe_umr_bsf_seg *bsf =
+ (struct mlx5_wqe_umr_bsf_seg *)(RTE_PTR_ADD(cseg,
+ priv->umr_wqe_size)) - 1;
+ struct mlx5_wqe_rseg *rseg;
+
+ /* Init UMR WQE. */
+ cseg->sq_ds = rte_cpu_to_be_32((qp->qp_obj->id << 8) |
+(priv->umr_wqe_size / MLX5_WSEG_SIZE));
+ cseg->flags = RTE_BE32(MLX5_COMP_ONLY_FIRST_ERR <<
+ MLX5_COMP_MODE_OFFSET);
+ cseg->misc = rte_cpu_to_be_32(qp->mkey[i]->id);
+ ucseg->if_cf_toe_cq_res = RTE_BE32(1u << MLX5_UMRC_IF_OFFSET);
+ ucseg->mkey_mask = RTE_BE64(1u << 0); /* Mkey length bit. */
+ ucseg->ko_to_bs = rte_cpu_to_be_32
+ ((RTE_ALIGN(priv->max_segs_num, 4u) <<
+MLX5_UMRC_KO_OFFSET) | (4 << MLX5_UMRC_TO_BS_OFFSET));
+ bsf->keytag = priv->keytag;
+ /* Init RDMA WRITE WQE. */
+ cseg = RTE_PTR_ADD(cseg, priv->umr_wqe_size);
+ cseg->flags = RTE_BE32((MLX5_COMP_ALWAYS <<
+ MLX5_COMP_MODE_OFFSET) |
+ MLX5_WQE_CTRL_INITIATOR_SMALL_FENCE);
+ rseg = (struct mlx5_wqe_rseg *)(cseg + 1);
+ rseg->rkey = rte_cpu_to_be_32(qp->mkey[i]->id);
+ }
+}
+
+static int
+mlx5_crypto_indirect_mkeys_prepare(struct mlx5_crypto_priv *priv,
+ struct mlx5_crypto_qp *qp)
+{
+ struct mlx5_umr_wqe *umr;
+ uint32_t i;
+ struct mlx5_devx_mkey_attr attr = {
+ .pd = priv->pdn,
+ .umr_en = 1,
+ .crypto_en = 1,
+ .set_remote_rw = 1,
+ .klm_num = RTE_ALIGN(priv->max_segs_num, 4),
+ };
+
+ for (umr = (struct mlx5_umr_wqe *)qp->umem_buf, i = 0;
+ i < qp->entries_n; i++, umr = RTE_PTR_ADD(umr, priv->wqe_set_size)) {
+ attr.klm_array = (struct mlx5_klm *)&umr->kseg[0];
+ qp->mkey[i] = mlx5_devx_cmd_mkey_create(priv->ctx, &attr);
+ if (!qp->mkey[i]) {
+ DRV_LOG(ERR, "Failed to allocate indirect mkey.");
+ return -1;
+ }
+ }
+ return 0;
+}
+
static int
mlx5_crypto_queue_pair_setup(struct rte_cryptodev *dev, uint16_t qp_id,
const struct rte_cryptodev_qp_conf *qp_conf,
@@ -278,7 +341,7 @@ mlx5_crypto_queue_pair_setup(struct rte_cryptodev *dev,
uint16_t qp_id,
struct mlx5_crypto_qp *qp;
uint16_t log_nb_desc = rte_log2_u32(qp_conf->nb_descriptors);
uint32_t umem_size = RTE_BIT32(log_nb_desc) *
- MLX5_CRYPTO_WQE_SET_SIZE +
+ priv->wqe_set_size +
sizeof(*qp->db_rec) * 2;
uint32_t alloc_size = sizeof(*qp);
struct mlx5_devx_cq_attr cq_attr = {
@@ -286,7 +349,9 @@ mlx5_crypto_queue_pair_setup(struct rte_cryptodev *dev,
uint16_t qp_id,
};
alloc_size = RTE_ALIGN(alloc_size, RTE_CACHE_LINE_SIZE);
- alloc_size += sizeof(struct rte_crypto_op *) * RTE_BIT32(log_nb_desc);
+ alloc_size += (sizeof(struct rte_crypto_op *) +
+ sizeof(struct mlx5_devx_obj *)) *
+ RTE_BIT32(log_
[dpdk-dev] [PATCH v3 14/15] crypto/mlx5: add statistic get and reset operations
From: Suanming Mou
This commit adds mlx5 crypto statistic get and reset operations.
Signed-off-by: Suanming Mou
Signed-off-by: Matan Azrad
---
drivers/crypto/mlx5/mlx5_crypto.c | 39 ---
1 file changed, 36 insertions(+), 3 deletions(-)
diff --git a/drivers/crypto/mlx5/mlx5_crypto.c
b/drivers/crypto/mlx5/mlx5_crypto.c
index fd0afb9fb2..896ca60866 100644
--- a/drivers/crypto/mlx5/mlx5_crypto.c
+++ b/drivers/crypto/mlx5/mlx5_crypto.c
@@ -437,11 +437,14 @@ mlx5_crypto_enqueue_burst(void *queue_pair, struct
rte_crypto_op **ops,
do {
op = *ops++;
umr = RTE_PTR_ADD(qp->umem_buf, priv->wqe_set_size * qp->pi);
- if (unlikely(mlx5_crypto_wqe_set(priv, qp, op, umr) == 0))
+ if (unlikely(mlx5_crypto_wqe_set(priv, qp, op, umr) == 0)) {
+ qp->stats.enqueue_err_count++;
break;
+ }
qp->ops[qp->pi] = op;
qp->pi = (qp->pi + 1) & mask;
} while (--remain);
+ qp->stats.enqueued_count += nb_ops;
rte_io_wmb();
qp->db_rec[MLX5_SND_DBR] = rte_cpu_to_be_32(qp->db_pi);
rte_wmb();
@@ -458,6 +461,7 @@ mlx5_crypto_cqe_err_handle(struct mlx5_crypto_qp *qp,
struct rte_crypto_op *op)
&qp->cq_obj.cqes[idx];
op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+ qp->stats.dequeue_err_count++;
DRV_LOG(ERR, "CQE ERR:%x.\n", rte_be_to_cpu_32(cqe->syndrome));
}
@@ -497,6 +501,7 @@ mlx5_crypto_dequeue_burst(void *queue_pair, struct
rte_crypto_op **ops,
if (likely(i != 0)) {
rte_io_wmb();
qp->cq_obj.db_rec[0] = rte_cpu_to_be_32(qp->ci);
+ qp->stats.dequeued_count += i;
}
return i;
}
@@ -655,14 +660,42 @@ mlx5_crypto_queue_pair_setup(struct rte_cryptodev *dev,
uint16_t qp_id,
return -1;
}
+static void
+mlx5_crypto_stats_get(struct rte_cryptodev *dev,
+ struct rte_cryptodev_stats *stats)
+{
+ int qp_id;
+
+ for (qp_id = 0; qp_id < dev->data->nb_queue_pairs; qp_id++) {
+ struct mlx5_crypto_qp *qp = dev->data->queue_pairs[qp_id];
+
+ stats->enqueued_count += qp->stats.enqueued_count;
+ stats->dequeued_count += qp->stats.dequeued_count;
+ stats->enqueue_err_count += qp->stats.enqueue_err_count;
+ stats->dequeue_err_count += qp->stats.dequeue_err_count;
+ }
+}
+
+static void
+mlx5_crypto_stats_reset(struct rte_cryptodev *dev)
+{
+ int qp_id;
+
+ for (qp_id = 0; qp_id < dev->data->nb_queue_pairs; qp_id++) {
+ struct mlx5_crypto_qp *qp = dev->data->queue_pairs[qp_id];
+
+ memset(&qp->stats, 0, sizeof(qp->stats));
+ }
+}
+
static struct rte_cryptodev_ops mlx5_crypto_ops = {
.dev_configure = mlx5_crypto_dev_configure,
.dev_start = mlx5_crypto_dev_start,
.dev_stop = mlx5_crypto_dev_stop,
.dev_close = mlx5_crypto_dev_close,
.dev_infos_get = mlx5_crypto_dev_infos_get,
- .stats_get = NULL,
- .stats_reset= NULL,
+ .stats_get = mlx5_crypto_stats_get,
+ .stats_reset= mlx5_crypto_stats_reset,
.queue_pair_setup = mlx5_crypto_queue_pair_setup,
.queue_pair_release = mlx5_crypto_queue_pair_release,
.sym_session_get_size = mlx5_crypto_sym_session_get_size,
--
2.25.1
[dpdk-dev] [PATCH v3 13/15] crypto/mlx5: add enqueue and dequeue operations
From: Suanming Mou
The crypto operations are done with the WQE set which contains
one UMR WQE and one rdma write WQE. Most segments of the WQE
set are initialized properly during queue setup, only limited
segments are initialized according to the crypto detail in the
datapath process.
This commit adds the enquue and dequeue operations and updates
the WQE set segments accordingly.
Signed-off-by: Suanming Mou
Signed-off-by: Matan Azrad
---
drivers/crypto/mlx5/mlx5_crypto.c | 243 +-
drivers/crypto/mlx5/mlx5_crypto.h | 3 +
2 files changed, 241 insertions(+), 5 deletions(-)
diff --git a/drivers/crypto/mlx5/mlx5_crypto.c
b/drivers/crypto/mlx5/mlx5_crypto.c
index 7bffe08bfe..fd0afb9fb2 100644
--- a/drivers/crypto/mlx5/mlx5_crypto.c
+++ b/drivers/crypto/mlx5/mlx5_crypto.c
@@ -268,6 +268,239 @@ mlx5_crypto_qp2rts(struct mlx5_crypto_qp *qp)
return 0;
}
+static __rte_noinline uint32_t
+mlx5_crypto_get_block_size(struct rte_crypto_op *op)
+{
+ uint32_t bl = op->sym->cipher.data.length;
+
+ switch (bl) {
+ case (1 << 20):
+ return RTE_BE32(MLX5_BLOCK_SIZE_1MB << MLX5_BLOCK_SIZE_OFFSET);
+ case (1 << 12):
+ return RTE_BE32(MLX5_BLOCK_SIZE_4096B <<
+ MLX5_BLOCK_SIZE_OFFSET);
+ case (1 << 9):
+ return RTE_BE32(MLX5_BLOCK_SIZE_512B << MLX5_BLOCK_SIZE_OFFSET);
+ default:
+ DRV_LOG(ERR, "Unknown block size: %u.", bl);
+ return UINT32_MAX;
+ }
+}
+
+static __rte_always_inline uint32_t
+mlx5_crypto_klm_set(struct mlx5_crypto_priv *priv, struct mlx5_crypto_qp *qp,
+ struct rte_mbuf *mbuf, struct mlx5_wqe_dseg *klm,
+ uint32_t offset, uint32_t *remain)
+{
+ uint32_t data_len = (rte_pktmbuf_data_len(mbuf) - offset);
+ uintptr_t addr = rte_pktmbuf_mtod_offset(mbuf, uintptr_t, offset);
+
+ if (data_len > *remain)
+ data_len = *remain;
+ *remain -= data_len;
+ klm->bcount = rte_cpu_to_be_32(data_len);
+ klm->pbuf = rte_cpu_to_be_64(addr);
+ klm->lkey = mlx5_mr_addr2mr_bh(priv->pd, 0,
+ &priv->mr_scache, &qp->mr_ctrl, addr,
+ !!(mbuf->ol_flags & EXT_ATTACHED_MBUF));
+ return klm->lkey;
+
+}
+
+static __rte_always_inline uint32_t
+mlx5_crypto_klms_set(struct mlx5_crypto_priv *priv, struct mlx5_crypto_qp *qp,
+struct rte_crypto_op *op, struct rte_mbuf *mbuf,
+struct mlx5_wqe_dseg *klm)
+{
+ uint32_t remain_len = op->sym->cipher.data.length;
+ uint32_t nb_segs = mbuf->nb_segs;
+ uint32_t klm_n = 1;
+
+ /* First mbuf needs to take the cipher offset. */
+ if (unlikely(mlx5_crypto_klm_set(priv, qp, mbuf, klm,
+op->sym->cipher.data.offset, &remain_len) == UINT32_MAX)) {
+ op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+ return 0;
+ }
+ while (remain_len) {
+ nb_segs--;
+ mbuf = mbuf->next;
+ if (unlikely(mbuf == NULL || nb_segs == 0)) {
+ op->status = RTE_CRYPTO_OP_STATUS_INVALID_ARGS;
+ return 0;
+ }
+ if (unlikely(mlx5_crypto_klm_set(priv, qp, mbuf, klm, 0,
+&remain_len) == UINT32_MAX)) {
+ op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+ return 0;
+ }
+ klm_n++;
+ }
+ return klm_n;
+}
+
+static __rte_always_inline int
+mlx5_crypto_wqe_set(struct mlx5_crypto_priv *priv,
+struct mlx5_crypto_qp *qp,
+struct rte_crypto_op *op,
+struct mlx5_umr_wqe *umr)
+{
+ struct mlx5_crypto_session *sess = get_sym_session_private_data
+ (op->sym->session, mlx5_crypto_driver_id);
+ struct mlx5_wqe_cseg *cseg = &umr->ctr;
+ struct mlx5_wqe_mkey_cseg *mkc = &umr->mkc;
+ struct mlx5_wqe_dseg *klms = &umr->kseg[0];
+ struct mlx5_wqe_umr_bsf_seg *bsf = ((struct mlx5_wqe_umr_bsf_seg *)
+ RTE_PTR_ADD(umr, priv->umr_wqe_size)) - 1;
+ uint16_t nop_ds;
+ /* Set UMR WQE. */
+ uint32_t klm_n = mlx5_crypto_klms_set(priv, qp, op,
+ op->sym->m_dst ? op->sym->m_dst : op->sym->m_src, klms);
+
+ if (unlikely(klm_n == 0))
+ return 0;
+ bsf->bs_bpt_eo_es = sess->bs_bpt_eo_es;
+ if (unlikely(!sess->bsp_res)) {
+ bsf->bsp_res = mlx5_crypto_get_block_size(op);
+ if (unlikely(bsf->bsp_res == UINT32_MAX)) {
+ op->sta
[dpdk-dev] [PATCH v3 15/15] crypto/mlx5: set feature flags and capabilities
From: Shiri Kuzin Add the supported capabilities to the crypto driver. Add supported feature flags. Add crypto driver documentation. Signed-off-by: Shiri Kuzin Signed-off-by: Matan Azrad --- doc/guides/cryptodevs/features/mlx5.ini | 37 ++ doc/guides/cryptodevs/index.rst | 1 + doc/guides/cryptodevs/mlx5.rst | 152 doc/guides/rel_notes/release_21_05.rst | 5 + drivers/crypto/mlx5/mlx5_crypto.c | 39 +- 5 files changed, 231 insertions(+), 3 deletions(-) create mode 100644 doc/guides/cryptodevs/features/mlx5.ini create mode 100644 doc/guides/cryptodevs/mlx5.rst diff --git a/doc/guides/cryptodevs/features/mlx5.ini b/doc/guides/cryptodevs/features/mlx5.ini new file mode 100644 index 00..a89526add0 --- /dev/null +++ b/doc/guides/cryptodevs/features/mlx5.ini @@ -0,0 +1,37 @@ +; +; Features of a mlx5 crypto driver. +; +; Refer to default.ini for the full list of available PMD features. +; +[Features] +Symmetric crypto = Y +HW Accelerated = Y +In Place SGL = Y +OOP SGL In SGL Out = Y +OOP SGL In LB Out = Y +OOP LB In SGL Out = Y +OOP LB In LB Out = Y +Cipher multiple data units = Y +Cipher wrapped key = Y + +; +; Supported crypto algorithms of a mlx5 crypto driver. +; +[Cipher] +AES XTS (128) = Y +AES XTS (256) = Y + +; +; Supported authentication algorithms of a mlx5 crypto driver. +; +[Auth] + +; +; Supported AEAD algorithms of a mlx5 crypto driver. +; +[AEAD] + +; +; Supported Asymmetric algorithms of a mlx5 crypto driver. +; +[Asymmetric] diff --git a/doc/guides/cryptodevs/index.rst b/doc/guides/cryptodevs/index.rst index 279f56a002..747409c441 100644 --- a/doc/guides/cryptodevs/index.rst +++ b/doc/guides/cryptodevs/index.rst @@ -22,6 +22,7 @@ Crypto Device Drivers octeontx octeontx2 openssl +mlx5 mvsam nitrox null diff --git a/doc/guides/cryptodevs/mlx5.rst b/doc/guides/cryptodevs/mlx5.rst new file mode 100644 index 00..4ccec78be8 --- /dev/null +++ b/doc/guides/cryptodevs/mlx5.rst @@ -0,0 +1,152 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright 2021 Mellanox Technologies, Ltd + +.. include:: + +MLX5 Crypto Driver +== + +The MLX5 crypto driver library +(**librte_crypto_mlx5**) provides support for **Mellanox ConnectX-6** +family adapters. + +Overview + + +The device can provide disk encryption services, allowing data encryption +and decryption towards a disk. Having all encryption/decryption +operations done in a single device can reduce cost and overheads of the related +FIPS certification, as ConnectX-6 is FIPS 140-2 level-2 ready. +The encryption cipher is AES-XTS of 256/512 key size. + +MKEY is a memory region object in the hardware, that holds address translation information and +attributes per memory area. Its ID must be tied to addresses provided to the hardware. +The encryption operations are performed with MKEY read\write transactions, when +the MKEY is configured to perform crypto operations. + +The encryption does not require text to be aligned to the AES block size (128b). + +In order to move the device to crypto operational mode, credential and KEK +(Key Encrypting Key) should be set as the first step. +The credential will be used by the software in order to perform crypto login, and the KEK is +the AES Key Wrap Algorithm (rfc3394) key that will be used for sensitive data +wrapping. +The credential and the AES-XTS keys should be provided to the hardware, as ciphertext +encrypted by the KEK. + +A keytag (64 bits) should be appended to the AES-XTS keys (before wrapping), +and will be validated when the hardware attempts to access it. + +For security reasons and to increase robustness, this driver only deals with virtual +memory addresses. The way resources allocations are handled by the kernel, +combined with hardware specifications that allow handling virtual memory +addresses directly, ensure that DPDK applications cannot access random +physical memory (or memory that does not belong to the current process). + +The PMD uses libibverbs and libmlx5 to access the device firmware or to +access the hardware components directly. +There are different levels of objects and bypassing abilities. +To get the best performances: + +- Verbs is a complete high-level generic API. +- Direct Verbs is a device-specific API. +- DevX allows to access firmware objects. + +Enabling librte_crypto_mlx5 causes DPDK applications to be linked against +libibverbs. + +Mellanox mlx5 PCI device can be probed by a number of different PCI devices, such as +net / vDPA / RegEx. To select the crypto PMD, ``class=crypto`` +should be specified as a device parameter. The crypto device can be probed and +used with other Mellanox classes by adding more options in the class. +For example: ``class=net:crypto`` will probe both the net PMD and the crypto +PMD. + +When crypto engines are defined to work in wrapped import method
Re: [dpdk-dev] [PATCH] net/mlx5/linux: fix missing firmware version copying
From: Kamil Vojanec > This patch fixes a bug where firmware version was not copied from > ibv_device_attr structure into mlx5_dev_attr structure, resulting in inability > to read firmware version. > > Signed-off-by: Kamil Vojanec Acked-by: Matan Azrad > --- > drivers/net/mlx5/linux/mlx5_os.c | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/drivers/net/mlx5/linux/mlx5_os.c > b/drivers/net/mlx5/linux/mlx5_os.c > index a8cc5f311..fa35c06e7 100644 > --- a/drivers/net/mlx5/linux/mlx5_os.c > +++ b/drivers/net/mlx5/linux/mlx5_os.c > @@ -152,6 +152,7 @@ mlx5_os_get_dev_attr(void *ctx, struct > mlx5_dev_attr *device_attr) #ifdef HAVE_IBV_DEVICE_TUNNEL_SUPPORT > device_attr->tunnel_offloads_caps = dv_attr.tunnel_offloads_caps; > #endif > + strlcpy(device_attr->fw_ver, attr_ex.orig_attr.fw_ver, > + sizeof(device_attr->fw_ver)); > > return err; > } > -- > 2.17.1
Re: [dpdk-dev] [PATCH] common/mlx5: fix Netlink port name attribute padding
From: Slava Ovsiienko
> On some kernels the string attributes within Netlink reply messages might be
> not padded with zeroes (in cases when string length is aligned with 4-byte
> boundary).
> This caused wrong physical port names recognition and
> mlx5 PMD load failures.
>
> Fixes: 30a86157f6d5 ("net/mlx5: support PF representor")
> Cc: sta...@dpdk.org
>
> Signed-off-by: Viacheslav Ovsiienko
Acked-by: Matan Azrad
Re: [dpdk-dev] [PATCH v3] net/mlx5: add TCP and IPv6 to supported flow items list in Windows
From: Tal Shnaiderman > WINOF2 2.70 Windows kernel driver allows DevX rule creation of types TCP > and IPv6. > > Added the types to the supported items in mlx5_flow_os_item_supported > to allow them to be created in the PMD. > > Added description of new rules support in Windows kernel driver WINOF2 > 2.70 to the mlx5 driver guide. > > Signed-off-by: Tal Shnaiderman Acked-by: Matan Azrad
Re: [dpdk-dev] [PATCH 1/3] regex/mlx5: fix memory region unregistration
From: Michael Baum
> The issue can cause illegal physical address access while a huge-page A is
> released and huge-page B is allocated on the same virtual address.
> The old MR can be matched using the virtual address of huge-page B but the
> HW will access the physical address of huge-page A which is no more part of
> the DPDK process.
>
> Register a driver callback for memory event in order to free out all the MRs
> of
> memory that is going to be freed from the dpdk process.
>
> Fixes: cda883bbb655 ("regex/mlx5: add dynamic memory registration to
> datapath")
> Cc: sta...@dpdk.org
>
> Signed-off-by: Michael Baum
> ---
>
> This series depends on this patch:
> https://patchwork.dpdk.org/project/dpdk/patch/20210628150614.1769507-
> 1-michae...@nvidia.com/
> Please don't apply it only before this patch is integrated.
>
> drivers/regex/mlx5/mlx5_regex.c | 55
> drivers/regex/mlx5/mlx5_regex.h | 2 +
> drivers/regex/mlx5/mlx5_regex_fastpath.c | 39 +++--
> 3 files changed, 92 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/regex/mlx5/mlx5_regex.c
> b/drivers/regex/mlx5/mlx5_regex.c index dcb2ced88e..0f12d94d7e 100644
> --- a/drivers/regex/mlx5/mlx5_regex.c
> +++ b/drivers/regex/mlx5/mlx5_regex.c
> @@ -11,6 +11,7 @@
> #include
>
> #include
> +#include
> #include
> #include
> #include
> @@ -24,6 +25,10 @@
>
> int mlx5_regex_logtype;
>
> +TAILQ_HEAD(regex_mem_event, mlx5_regex_priv) mlx5_mem_event_list
> =
> +
> TAILQ_HEAD_INITIALIZER(mlx5_mem_event_list);
> +static pthread_mutex_t mem_event_list_lock =
> PTHREAD_MUTEX_INITIALIZER;
> +
> const struct rte_regexdev_ops mlx5_regexdev_ops = {
> .dev_info_get = mlx5_regex_info_get,
> .dev_configure = mlx5_regex_configure, @@ -82,6 +87,40 @@
> mlx5_regex_get_name(char *name, struct rte_pci_device *pci_dev
> __rte_unused)
> pci_dev->addr.devid, pci_dev->addr.function); }
>
> +/**
> + * Callback for memory event.
> + *
> + * @param event_type
> + * Memory event type.
> + * @param addr
> + * Address of memory.
> + * @param len
> + * Size of memory.
> + */
> +static void
> +mlx5_regex_mr_mem_event_cb(enum rte_mem_event event_type,
> const void *addr,
> +size_t len, void *arg __rte_unused) {
> + struct mlx5_regex_priv *priv;
> +
> + /* Must be called from the primary process. */
> + MLX5_ASSERT(rte_eal_process_type() == RTE_PROC_PRIMARY);
> + switch (event_type) {
> + case RTE_MEM_EVENT_FREE:
> + pthread_mutex_lock(&mem_event_list_lock);
> + /* Iterate all the existing mlx5 devices. */
> + TAILQ_FOREACH(priv, &mlx5_mem_event_list,
> mem_event_cb)
> + mlx5_free_mr_by_addr(&priv->mr_scache,
> + priv->ctx->device->name,
> + addr, len);
> + pthread_mutex_unlock(&mem_event_list_lock);
> + break;
> + case RTE_MEM_EVENT_ALLOC:
> + default:
> + break;
> + }
> +}
> +
> static int
> mlx5_regex_pci_probe(struct rte_pci_driver *pci_drv __rte_unused,
>struct rte_pci_device *pci_dev)
> @@ -193,6 +232,15 @@ mlx5_regex_pci_probe(struct rte_pci_driver
> *pci_drv __rte_unused,
> rte_errno = ENOMEM;
> goto error;
> }
> + /* Register callback function for global shared MR cache
> management. */
> + if (TAILQ_EMPTY(&mlx5_mem_event_list))
> +
> rte_mem_event_callback_register("MLX5_MEM_EVENT_CB",
> +
> mlx5_regex_mr_mem_event_cb,
> + NULL);
> + /* Add device to memory callback list. */
> + pthread_mutex_lock(&mem_event_list_lock);
> + TAILQ_INSERT_TAIL(&mlx5_mem_event_list, priv, mem_event_cb);
> + pthread_mutex_unlock(&mem_event_list_lock);
> DRV_LOG(INFO, "RegEx GGA is %s.",
> priv->has_umr ? "supported" : "unsupported");
> return 0;
> @@ -225,6 +273,13 @@ mlx5_regex_pci_remove(struct rte_pci_device
> *pci_dev)
> return 0;
> priv = dev->data->dev_private;
> if (priv) {
> + /* Remove from memory callback device list. */
> + pthread_mutex_lock(&mem_event_list_lock);
> + TAILQ_REMOVE(&mlx5_mem_event_list, priv,
> mem_event_cb);
> + pthread_mutex_unlock(&mem_event_list_lock);
> + if (TAILQ_EMPTY(&mlx5_mem_event_list))
> +
> rte_mem_event_callback_unregister("MLX5_MEM_EVENT_CB",
> + NULL);
> if (priv->pd)
> mlx5_glue->dealloc_pd(priv->pd);
> if (priv->uar)
> diff --git a/drivers/regex/mlx5/mlx5_regex.h
> b/drivers/regex/mlx5/mlx5_regex.h index 51a2101e53..61f59ba873 100644
> --- a/drivers/regex/mlx5/mlx5_regex.h
> +++ b/drivers/regex/mlx5/mlx5_regex.h
> @@ -70,6 +70,8 @@ struct mlx5_regex_priv {
Re: [dpdk-dev] [PATCH] net/mlx5: fix match MPLS over GRE with key
From: Jack Min
> Currently PMD needs previous layer information in order to set
> corresponding match field for MPLSoGRE or MPLSoUDP.
>
> GRE_KEY item is missing as supported previous layer when translate item
> MPLS, which causes flow[1] cannot match MPLS over GRE traffic.
>
> According to RFC4023, MPLS over GRE tunnel with optional key field needs to
> be supported too.
>
> By adding missing GRE_KEY as supported previous layer fix problem.
>
> [1]:
> flow create 0 ingress pattern eth / ipv6 / gre k_bit is 1 / gre_key / mpls
> label is
> 966138 / end actions queue index 1 / mark id 0xa / end
>
> Fixes: a7a0365565a4 ("net/mlx5: match GRE key and present bits")
> Cc: sta...@dpdk.org
>
> Signed-off-by: Xiaoyu Min
Acked-by: Matan Azrad
Re: [dpdk-dev] [PATCH 3/3] net/bonding: start ethdev prior to setting 8023ad flow
From: Havlík Martin
> Dne 2021-06-23 09:04, Min Hu (Connor) napsal:
> > 在 2021/6/22 17:25, Martin Havlik 写道:
> >> When dedicated queues are enabled, mlx5 PMD fails to install RTE
> >> Flows if the underlying ethdev is not started:
> >> bond_ethdev_8023ad_flow_set(267) - bond_ethdev_8023ad_flow_set:
> port
> >> not started (slave_port=0 queue_id=1)
> >>
> > Why mlx5 PMD doing flow create relys on port started ?
> > I noticed other PMDs did not has that reliance.
> >
> After looking into it, I really can't answer this mlx5 centered question.
> Closest related info we found so far is the 5th point in
> https://doc.dpdk.org/guides/prog_guide/rte_flow.html#caveats
> but it only specifies it's the application's responsibility and that flow
> rules are
> assumed invalid after port stop/close/restart but doesn't say anything about
> vs where
> the
> former is the point of failure on mlx5.
> I'm addressing the maintainers for mlx5, who might know a bit more on the
> topic.
From rte_ethdev.h
* Please note that some configuration is not stored between calls to
* rte_eth_dev_stop()/rte_eth_dev_start(). The following configuration will
* be retained:
*
* - MTU
* - flow control settings
* - receive mode configuration (promiscuous mode, all-multicast mode,
* hardware checksum mode, RSS/VMDQ settings etc.)
* - VLAN filtering configuration
* - default MAC address
* - MAC addresses supplied to MAC address array
* - flow director filtering mode (but not filtering rules)
* - NIC queue statistics mappings
Mlx5 assumes flows are allowed to be configured only after rte_eth_dev_start().
Before start \ after stop - no flow is valid anymore.
Matan
> >> Signed-off-by: Martin Havlik
> >> Cc: Jan Viktorin
> >> ---
> >> drivers/net/bonding/rte_eth_bond_pmd.c | 26
> >> ++
> >> 1 file changed, 18 insertions(+), 8 deletions(-)
> >>
> >> diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c
> >> b/drivers/net/bonding/rte_eth_bond_pmd.c
> >> index a6755661c..fea3bc537 100644
> >> --- a/drivers/net/bonding/rte_eth_bond_pmd.c
> >> +++ b/drivers/net/bonding/rte_eth_bond_pmd.c
> >> @@ -1818,25 +1818,35 @@ slave_configure(struct rte_eth_dev
> >> *bonded_eth_dev,
> >> rte_flow_destroy(slave_eth_dev->data->port_id,
> >>
> >> internals-
> >mode4.dedicated_queues.flow[slave_eth_dev->data->port_id],
> >> &flow_error);
> >> +}
> >> + /* Start device */
> >> +errval = rte_eth_dev_start(slave_eth_dev->data->port_id);
> >> +if (errval != 0) {
> >> +RTE_BOND_LOG(ERR, "rte_eth_dev_start: port=%u, err (%d)",
> >> +slave_eth_dev->data->port_id, errval);
> >> +return -1;
> >> +}
> >> +
> >> +if (internals->mode == BONDING_MODE_8023AD &&
> >> +internals->mode4.dedicated_queues.enabled == 1)
> >> + {
> >> errval = bond_ethdev_8023ad_flow_set(bonded_eth_dev,
> >> slave_eth_dev->data->port_id);
> >> if (errval != 0) {
> >> RTE_BOND_LOG(ERR,
> >> "bond_ethdev_8023ad_flow_set: port=%d, err
> >> (%d)",
> >> slave_eth_dev->data->port_id, errval);
> >> +
> >> +errval =
> >> rte_eth_dev_stop(slave_eth_dev->data->port_id);
> >> +if (errval < 0) {
> >> +RTE_BOND_LOG(ERR,
> >> +"rte_eth_dev_stop: port=%d, err (%d)",
> >> +slave_eth_dev->data->port_id, errval);
> >> +}
> >> return errval;
> >> }
> >> }
> >> - /* Start device */
> >> -errval = rte_eth_dev_start(slave_eth_dev->data->port_id);
> >> -if (errval != 0) {
> >> -RTE_BOND_LOG(ERR, "rte_eth_dev_start: port=%u, err (%d)",
> >> -slave_eth_dev->data->port_id, errval);
> >> -return -1;
> >> -}
> >> -
> >> /* If RSS is enabled for bonding, synchronize RETA */
> >> if (bonded_eth_dev->data->dev_conf.rxmode.mq_mode &
> >> ETH_MQ_RX_RSS) {
> >> int i;
> >>
Re: [dpdk-dev] [PATCH] dmadev: introduce DMA device library
From: Chengwen Feng
> This patch introduces 'dmadevice' which is a generic type of DMA
> device.
>
> The APIs of dmadev library exposes some generic operations which can
> enable configuration and I/O with the DMA devices.
>
Did you consider RTE_COMP_ALGO_NULL xform in compressdev library?
> Signed-off-by: Chengwen Feng
> ---
> MAINTAINERS | 4 +
> config/rte_config.h | 3 +
> lib/dmadev/meson.build | 6 +
> lib/dmadev/rte_dmadev.c | 438 +
> lib/dmadev/rte_dmadev.h | 919
> +++
> lib/dmadev/rte_dmadev_core.h | 98 +
> lib/dmadev/rte_dmadev_pmd.h | 210 ++
> lib/dmadev/version.map | 32 ++
> lib/meson.build | 1 +
> 9 files changed, 1711 insertions(+)
> create mode 100644 lib/dmadev/meson.build
> create mode 100644 lib/dmadev/rte_dmadev.c
> create mode 100644 lib/dmadev/rte_dmadev.h
> create mode 100644 lib/dmadev/rte_dmadev_core.h
> create mode 100644 lib/dmadev/rte_dmadev_pmd.h
> create mode 100644 lib/dmadev/version.map
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 4347555..2019783 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -496,6 +496,10 @@ F: drivers/raw/skeleton/
> F: app/test/test_rawdev.c
> F: doc/guides/prog_guide/rawdev.rst
>
> +Dma device API
> +M: Chengwen Feng
> +F: lib/dmadev/
> +
>
> Memory Pool Drivers
> ---
> diff --git a/config/rte_config.h b/config/rte_config.h
> index 590903c..331a431 100644
> --- a/config/rte_config.h
> +++ b/config/rte_config.h
> @@ -81,6 +81,9 @@
> /* rawdev defines */
> #define RTE_RAWDEV_MAX_DEVS 64
>
> +/* dmadev defines */
> +#define RTE_DMADEV_MAX_DEVS 64
> +
> /* ip_fragmentation defines */
> #define RTE_LIBRTE_IP_FRAG_MAX_FRAG 4
> #undef RTE_LIBRTE_IP_FRAG_TBL_STAT
> diff --git a/lib/dmadev/meson.build b/lib/dmadev/meson.build
> new file mode 100644
> index 000..c918dae
> --- /dev/null
> +++ b/lib/dmadev/meson.build
> @@ -0,0 +1,6 @@
> +# SPDX-License-Identifier: BSD-3-Clause
> +# Copyright(c) 2021 HiSilicon Limited.
> +
> +sources = files('rte_dmadev.c')
> +headers = files('rte_dmadev.h', 'rte_dmadev_pmd.h')
> +indirect_headers += files('rte_dmadev_core.h')
> diff --git a/lib/dmadev/rte_dmadev.c b/lib/dmadev/rte_dmadev.c
> new file mode 100644
> index 000..a94e839
> --- /dev/null
> +++ b/lib/dmadev/rte_dmadev.c
> @@ -0,0 +1,438 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright 2021 HiSilicon Limited.
> + */
> +
> +#include
> +#include
> +#include
> +#include
> +
> +#include
> +#include
> +#include
> +#include
> +#include
> +#include
> +#include
> +#include
> +
> +#include "rte_dmadev.h"
> +#include "rte_dmadev_pmd.h"
> +
> +struct rte_dmadev rte_dmadevices[RTE_DMADEV_MAX_DEVS];
> +
> +uint16_t
> +rte_dmadev_count(void)
> +{
> + uint16_t count = 0;
> + uint16_t i;
> +
> + for (i = 0; i < RTE_DMADEV_MAX_DEVS; i++) {
> + if (rte_dmadevices[i].attached)
> + count++;
> + }
> +
> + return count;
> +}
> +
> +int
> +rte_dmadev_get_dev_id(const char *name)
> +{
> + uint16_t i;
> +
> + if (name == NULL)
> + return -EINVAL;
> +
> + for (i = 0; i < RTE_DMADEV_MAX_DEVS; i++)
> + if ((strcmp(rte_dmadevices[i].name, name) == 0) &&
> + (rte_dmadevices[i].attached == RTE_DMADEV_ATTACHED))
> + return i;
> +
> + return -ENODEV;
> +}
> +
> +int
> +rte_dmadev_socket_id(uint16_t dev_id)
> +{
> + struct rte_dmadev *dev;
> +
> + RTE_DMADEV_VALID_DEVID_OR_ERR_RET(dev_id, -EINVAL);
> + dev = &rte_dmadevices[dev_id];
> +
> + return dev->socket_id;
> +}
> +
> +int
> +rte_dmadev_info_get(uint16_t dev_id, struct rte_dmadev_info *dev_info)
> +{
> + struct rte_dmadev *dev;
> + int diag;
> +
> + RTE_DMADEV_VALID_DEVID_OR_ERR_RET(dev_id, -EINVAL);
> + RTE_FUNC_PTR_OR_ERR_RET(dev_info, -EINVAL);
> +
> + dev = &rte_dmadevices[dev_id];
> +
> + RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_info_get, -
> ENOTSUP);
> +
> + memset(dev_info, 0, sizeof(struct rte_dmadev_info));
> + diag = (*dev->dev_ops->dev_info_get)(dev, dev_info);
> + if (diag != 0)
> + return diag;
> +
> + dev_info->device = dev->device;
> + dev_info->driver_name = dev->driver_name;
> + dev_info->socket_id = dev->socket_id;
> +
> + return 0;
> +}
> +
> +int
> +rte_dmadev_configure(uint16_t dev_id, const struct rte_dmadev_conf
> *dev_conf)
> +{
> + struct rte_dmadev *dev;
> + int diag;
> +
> + RTE_DMADEV_VALID_DEVID_OR_ERR_RET(dev_id, -EINVAL);
> + RTE_FUNC_PTR_OR_ERR_RET(dev_conf, -EINVAL);
> +
> + dev = &rte_dmadevices[dev_id];
> +
> + RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->dev_configure, -
> ENOTSUP);
> +
> + if (dev->started) {
> + RTE_DMADEV_LOG(ERR,
> + "dev
Re: [dpdk-dev] [PATCH] dmadev: introduce DMA device library
Hi From: fengchengwen > On 2021/7/4 23:21, Matan Azrad wrote: > > > > > > From: Chengwen Feng > >> This patch introduces 'dmadevice' which is a generic type of DMA > >> device. > >> > >> The APIs of dmadev library exposes some generic operations which can > >> enable configuration and I/O with the DMA devices. > >> > > Did you consider RTE_COMP_ALGO_NULL xform in compressdev library? > > > > em, I just looked at the code. > > The RTE_COMP_ALGO_NULL is a small feature of the compression device. > and currently only mlx5 and isal support it. Yes, but what that is mean? If more drivers support DMA operations they can add the support there, no? > Also the compressdev dataplane API relatively complicated to do just DMA > copy. You snipped more comments I wrote below 😊 Maybe it is related > So I think we need a separate driver framework for the DMA device. Need to consider deprecation in compressdev if so > thanks > > [snip] > > > > > > > . > >
Re: [dpdk-dev] [PATCH] dmadev: introduce DMA device library
From: fengchengwen > On 2021/7/6 14:50, Matan Azrad wrote: > > Hi > > > > From: fengchengwen > >> On 2021/7/4 23:21, Matan Azrad wrote: > >>> > >>> > >>> From: Chengwen Feng > >>>> This patch introduces 'dmadevice' which is a generic type of DMA > >>>> device. > >>>> > >>>> The APIs of dmadev library exposes some generic operations which > >>>> can enable configuration and I/O with the DMA devices. > >>>> > >>> Did you consider RTE_COMP_ALGO_NULL xform in compressdev library? > >>> > >> > >> em, I just looked at the code. > >> > >> The RTE_COMP_ALGO_NULL is a small feature of the compression device. > >> and currently only mlx5 and isal support it. > > > > Yes, but what that is mean? > > If more drivers support DMA operations they can add the support there, > no? > > > > You mean to expand directly on compressdev ? > I think it hard to expand, and may break the compressdev concept. Maybe, what do you need to expand? Also maybe your expansion is related also to compress, finally both are mem-to-mem offload. > > > >> Also the compressdev dataplane API relatively complicated to do just > >> DMA copy. > > > > You snipped more comments I wrote below 😊 > > Maybe it is related > > Sorry, I just skipped. > > 'Did you consider also mbuf API usage for memory descriptor?' > ---One scenario of the DMA is vhost-net, which src or dst could be mbuf, but > the peer were not mbuf. so here we use raw fields. Did you consider using external\attached mbuf for this case? Did you also consider raw API in cryptodev library? > > > >> So I think we need a separate driver framework for the DMA device. > > > > Need to consider deprecation in compressdev if so > > > >> thanks > >> > >> [snip] > >> > >>> > >>> > >>> . > >>> > >
Re: [dpdk-dev] [PATCH 0/3] MPLS RSS fixs
From: Xiaoyu Min > This patch set fixs several problems of MPLS RSS: > > - removed MPLSoIP(HW doesn't support) related code > - support MPLSoUDP, MPLSoGRE RSS expansion > - reduced number of expanded sub-flows for inner MPLS > > Xiaoyu Min (3): > net/mlx5: fix MPLS over IP > net/mlx5: fix MPLS RSS expansion > net/mlx5: limit inner RSS expansion for MPLS Series-acked by Matan Azrad > drivers/net/mlx5/mlx5_flow.c| 31 +-- > drivers/net/mlx5/mlx5_flow_dv.c | 3 --- > 2 files changed, 21 insertions(+), 13 deletions(-) > > -- > 2.32.0
Re: [dpdk-dev] [PATCH 0/2] fix RSS expansion missing items
From: Xiaoyu Min > This patch set fixs the missing items in RSS expansion. > > Xiaoyu Min (2): > net/mlx5: fix missing RSS expandable items > net/mlx5: fix missing RSS expansion of IPv6 frag > Series-acked -by: Matan Azrad > drivers/net/mlx5/mlx5_flow.c | 10 +- > 1 file changed, 9 insertions(+), 1 deletion(-) > > -- > 2.32.0
Re: [PATCH v5 1/7] net/mlx5: add LWM support for Rxq
Series-acked-by: Matan Azrad mailto:ma...@nvidia.com>>
קבל Outlook עבור Android<https://aka.ms/AAb9ysg>
From: Spike Du
Sent: Tuesday, June 7, 2022 3:59:36 PM
To: Matan Azrad ; Slava Ovsiienko ;
Ori Kam ; NBU-Contact-Thomas Monjalon (EXTERNAL)
Cc: andrew.rybche...@oktetlabs.ru ;
step...@networkplumber.org ;
m...@smartsharesystems.com ; dev@dpdk.org
; Raslan Darawsheh
Subject: [PATCH v5 1/7] net/mlx5: add LWM support for Rxq
Add lwm(Limit WaterMark) field to Rxq object which indicates the percentage
of RX queue size used by HW to raise LWM event to the user.
Allow LWM setting in modify_rq command.
Allow the LWM configuration dynamically by adding RDY2RDY state change.
Signed-off-by: Spike Du
---
drivers/net/mlx5/mlx5.h | 1 +
drivers/net/mlx5/mlx5_devx.c | 13 -
drivers/net/mlx5/mlx5_devx.h | 1 +
drivers/net/mlx5/mlx5_rx.h | 1 +
4 files changed, 15 insertions(+), 1 deletion(-)
diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h
index ef755ee..305edff 100644
--- a/drivers/net/mlx5/mlx5.h
+++ b/drivers/net/mlx5/mlx5.h
@@ -1395,6 +1395,7 @@ enum mlx5_rxq_modify_type {
MLX5_RXQ_MOD_RST2RDY, /* modify state from reset to ready. */
MLX5_RXQ_MOD_RDY2ERR, /* modify state from ready to error. */
MLX5_RXQ_MOD_RDY2RST, /* modify state from ready to reset. */
+ MLX5_RXQ_MOD_RDY2RDY, /* modify state from ready to ready. */
};
enum mlx5_txq_modify_type {
diff --git a/drivers/net/mlx5/mlx5_devx.c b/drivers/net/mlx5/mlx5_devx.c
index 4b48f94..c918a50 100644
--- a/drivers/net/mlx5/mlx5_devx.c
+++ b/drivers/net/mlx5/mlx5_devx.c
@@ -62,7 +62,7 @@
* @return
* 0 on success, a negative errno value otherwise and rte_errno is set.
*/
-static int
+int
mlx5_devx_modify_rq(struct mlx5_rxq_priv *rxq, uint8_t type)
{
struct mlx5_devx_modify_rq_attr rq_attr;
@@ -76,6 +76,11 @@
case MLX5_RXQ_MOD_RST2RDY:
rq_attr.rq_state = MLX5_RQC_STATE_RST;
rq_attr.state = MLX5_RQC_STATE_RDY;
+ if (rxq->lwm) {
+ rq_attr.modify_bitmask |=
+ MLX5_MODIFY_RQ_IN_MODIFY_BITMASK_WQ_LWM;
+ rq_attr.lwm = rxq->lwm;
+ }
break;
case MLX5_RXQ_MOD_RDY2ERR:
rq_attr.rq_state = MLX5_RQC_STATE_RDY;
@@ -85,6 +90,12 @@
rq_attr.rq_state = MLX5_RQC_STATE_RDY;
rq_attr.state = MLX5_RQC_STATE_RST;
break;
+ case MLX5_RXQ_MOD_RDY2RDY:
+ rq_attr.rq_state = MLX5_RQC_STATE_RDY;
+ rq_attr.state = MLX5_RQC_STATE_RDY;
+ rq_attr.modify_bitmask |=
MLX5_MODIFY_RQ_IN_MODIFY_BITMASK_WQ_LWM;
+ rq_attr.lwm = rxq->lwm;
+ break;
default:
break;
}
diff --git a/drivers/net/mlx5/mlx5_devx.h b/drivers/net/mlx5/mlx5_devx.h
index a95207a..ebd1da4 100644
--- a/drivers/net/mlx5/mlx5_devx.h
+++ b/drivers/net/mlx5/mlx5_devx.h
@@ -11,6 +11,7 @@
int mlx5_txq_devx_modify(struct mlx5_txq_obj *obj,
enum mlx5_txq_modify_type type, uint8_t dev_port);
void mlx5_txq_devx_obj_release(struct mlx5_txq_obj *txq_obj);
+int mlx5_devx_modify_rq(struct mlx5_rxq_priv *rxq, uint8_t type);
extern struct mlx5_obj_ops devx_obj_ops;
diff --git a/drivers/net/mlx5/mlx5_rx.h b/drivers/net/mlx5/mlx5_rx.h
index e715ed6..25a5f2c 100644
--- a/drivers/net/mlx5/mlx5_rx.h
+++ b/drivers/net/mlx5/mlx5_rx.h
@@ -175,6 +175,7 @@ struct mlx5_rxq_priv {
struct mlx5_devx_rq devx_rq;
struct rte_eth_hairpin_conf hairpin_conf; /* Hairpin configuration. */
uint32_t hairpin_status; /* Hairpin binding status. */
+ uint32_t lwm:16;
};
/* External RX queue descriptor. */
--
1.8.3.1
RE: [PATCH V2] vdpa/mlx5: return correct error code after rte_intr_instance_alloc failed
Hi Shiqi
From: 835703...@qq.com <835703...@qq.com>
> From: Shiqi Liu <835703...@qq.com>
>
> To Maxime,
> I am so sorry too late to reply your comment.My name is Shiqi Liu.
> I got stuck in how to reply the comment before.Now, it seenms to be
> ok!Maybe there will be some mistakes I have taken , when I reply,please tell
> me how to do it correctly!
> Thanks,
> Shiqi Liu
>
> diff --git a/drivers/vdpa/mlx5/mlx5_vdpa_event.c
> b/drivers/vdpa/mlx5/mlx5_vdpa_event.c
> index 7167a98db0..6223afaae8 100644
> --- a/drivers/vdpa/mlx5/mlx5_vdpa_event.c
> +++ b/drivers/vdpa/mlx5/mlx5_vdpa_event.c
> @@ -395,6 +395,7 @@ mlx5_vdpa_err_event_setup(struct mlx5_vdpa_priv
> *priv)
> rte_intr_instance_alloc(RTE_INTR_INSTANCE_F_SHARED);
> if (priv->err_intr_handle == NULL) {
> DRV_LOG(ERR, "Fail to allocate intr_handle");
> + rte_errno = ENOMEM;
Shouldn't rte_intr_instance_alloc set it?
> goto error;
> }
> if (rte_intr_fd_set(priv->err_intr_handle, priv->err_chnl->fd))
> --
RE: [PATCH v7] app/testpmd: add Host Shaper command
From: Spike Du > Add command line options to support host shaper configure. > - Command syntax: > mlx5 set port host_shaper avail_thresh_triggered <0|1> rate > > > - Example commands: > To enable avail_thresh_triggered on port 1 and disable current host > shaper: > testpmd> mlx5 set port 1 host_shaper avail_thresh_triggered 1 rate 0 > > To disable avail_thresh_triggered and current host shaper on port 1: > testpmd> mlx5 set port 1 host_shaper avail_thresh_triggered 0 rate 0 > > The rate unit is 100Mbps. > To disable avail_thresh_triggered and configure a shaper of 5Gbps on port 1: > testpmd> mlx5 set port 1 host_shaper avail_thresh_triggered 0 rate 50 > > Add sample code to handle rxq available descriptor threshold event, it delays > a while so that rxq empties, then disables host shaper and rearms available > descriptor threshold event. > > Signed-off-by: Spike Du Acked-by: Matan Azrad
RE: [PATCH] vdpa/mlx5: return correct error code after rte_intr_instance_alloc failed
Hi
I put here again the comment the putted on another version of this commit.
I saw it is already integrated but still not in master.
But please look.
From: Maxime Coquelin
> Hi newsky647,
>
> On 6/12/22 16:09, 835703...@qq.com wrote:
> > From: newsky647 <835703...@qq.com>
> >
> > After function rte_intr_instance_alloc failed, we should return ENOMEM
> > for error code.
> >
> > Fixes: 5fe068bf7a2 ("vdpa/mlx5: reuse resources in reconfiguration")
> >
> > Signed-off-by: newsky647 <835703...@qq.com>
> > ---
> > drivers/vdpa/mlx5/mlx5_vdpa_event.c | 1 +
> > 1 file changed, 1 insertion(+)
> >
> > diff --git a/drivers/vdpa/mlx5/mlx5_vdpa_event.c
> > b/drivers/vdpa/mlx5/mlx5_vdpa_event.c
> > index 7167a98db0..6223afaae8 100644
> > --- a/drivers/vdpa/mlx5/mlx5_vdpa_event.c
> > +++ b/drivers/vdpa/mlx5/mlx5_vdpa_event.c
> > @@ -395,6 +395,7 @@ mlx5_vdpa_err_event_setup(struct mlx5_vdpa_priv
> *priv)
> > rte_intr_instance_alloc(RTE_INTR_INSTANCE_F_SHARED);
> > if (priv->err_intr_handle == NULL) {
> > DRV_LOG(ERR, "Fail to allocate intr_handle");
> > + rte_errno = ENOMEM;
Shouldn't rte_intr_instance_alloc set it?
> > goto error;
> > }
> > if (rte_intr_fd_set(priv->err_intr_handle, priv->err_chnl->fd))
>
> The patch looks good to me, but I would need your real name for the sign-off.
>
> Reviewed-by: Maxime Coquelin
>
> Thanks,
> Maxime
RE: [PATCH 1/3] vdpa/mlx5: remove unnecessary null check
From: Stephen Hemminger > Found by nullfree.cocci. > > Signed-off-by: Stephen Hemminger Acked-by: Matan Azrad
