On Tue, Mar 26, 2019 at 02:51:25PM +0100, Thomas Monjalon wrote: > 26/03/2019 10:21, Christian Ehrhardt: > > > > > > > > rdma-core configures modprobe for ib/rdma modules, installs some > > > > helper daemons and installs some udev rules. It certainly makes > > > > administration of the server an easier task, but it isn't *required* > > > > when configuring a Debian server for rdma and user-space ibverbs. > > > > > > I didn't think it is required for mlx5 as the ethernet driver > > > autoloads the rdma modules itself.. > > > > Below is the log that I got from Mellanox (thanks Alaa and Thomas. > > To me the messages related to ib_uverbs seem interesting. > > > > @Jason - this is triggered by DPDK. In regard to the autoloading - did > > you refer to DPDKs net_mlx5 or the kernels net_mlx5 in e.g. > > mlx5_core.ko? > > I only found [1] which does not seem right, can you point at the > > autoloading feature (a commit maybe) that you meant (keep in mind this > > is DPDK 17.11.5)? > > [1]: https://doc.dpdk.org/guides/nics/mlx5.html#compilation-options > > There is no autoloading of kernel drivers when using DPDK.
It looks like it is only done if the mlx5 NIC driver detects some kind of RDMA mode.. > > @Allaa/Thomas - due to the messages in the log, could you try if > > instead of installing rdma-core just doing the modprobe would be > > enough to resolve the problem? > > $ modprobe ib_uverbs # instead of the apt install > > Yes, manual modprobe partially replaces rdma-core packages. > > These are the files we identified as required from the rdma-core packages: > /etc/modprobe.d/mlx4.conf > /etc/rdma/modules/rdma.conf > /lib/udev/rules.d/90-rdma-hw-modules.rules > /lib/systemd/system/rdma-hw.target > > The first file (mlx4.conf) is a template helping to get the right config > by commenting out a line. > The others files are for autoloading of the kernel drivers. > > I think you should include the rdma-core package when using DPDK. > It can be a dependency of the DPDK package, or a dependency of libibverbs. > Does it make sense to install libibverbs without rdma-core configuration? Many rdma drivers do properly autoload and don't need the udev magic. I have a long term plan to avoid needing this, but nobody is working on it right now. Jason
