> > > > rdma-core configures modprobe for ib/rdma modules, installs some > > helper daemons and installs some udev rules. It certainly makes > > administration of the server an easier task, but it isn't *required* > > when configuring a Debian server for rdma and user-space ibverbs. > > I didn't think it is required for mlx5 as the ethernet driver > autoloads the rdma modules itself..
Below is the log that I got from Mellanox (thanks Alaa and Thomas. To me the messages related to ib_uverbs seem interesting. @Jason - this is triggered by DPDK. In regard to the autoloading - did you refer to DPDKs net_mlx5 or the kernels net_mlx5 in e.g. mlx5_core.ko? I only found [1] which does not seem right, can you point at the autoloading feature (a commit maybe) that you meant (keep in mind this is DPDK 17.11.5)? @Allaa/Thomas - due to the messages in the log, could you try if instead of installing rdma-core just doing the modprobe would be enough to resolve the problem? $ modprobe ib_uverbs # instead of the apt install [1]: https://doc.dpdk.org/guides/nics/mlx5.html#compilation-options FYI - The test log from Mellanox showing that they need rdma-core to run properly: 1 - Running testpmd with installing any thing root@qa-l-vrt-111-124:~# testpmd -c 0xF -n 4 -w 0000:00:06.0 -w 0000:00:07.0 -- --txd=256 --rxd=256 --rxq=1 --txq=1 --nb-cores=3 --i EAL: Detected 4 lcore(s) EAL: Probing VFIO support... EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using unreliable clock cycles ! EAL: PCI device 0000:00:06.0 on NUMA socket -1 EAL: Invalid NUMA socket, default to 0 EAL: probe driver: 15b3:1016 net_mlx5 net_mlx5: cannot list devices, is ib_uverbs loaded? EAL: Requested device 0000:00:06.0 cannot be used EAL: PCI device 0000:00:07.0 on NUMA socket -1 EAL: Invalid NUMA socket, default to 0 EAL: probe driver: 15b3:1016 net_mlx5 net_mlx5: cannot list devices, is ib_uverbs loaded? EAL: Requested device 0000:00:07.0 cannot be used EAL: No probed ethernet devices Interactive-mode selected USER1: create a new mbuf pool <mbuf_pool_socket_0>: n=171456, size=2176, socket=0 Done 2- Installing rdma-core root@qa-l-vrt-111-124:~# apt-get install rdma-core Reading package lists... Done Building dependency tree Reading state information... Done The following NEW packages will be installed: rdma-core 0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded. Need to get 56.8 kB of archives. After this operation, 195 kB of additional disk space will be used. Get:1 http://us.archive.ubuntu.com/ubuntu bionic-updates/universe amd64 rdma-core amd64 17.1-1ubuntu0.1 [56.8 kB] Fetched 56.8 kB in 1s (79.4 kB/s) Selecting previously unselected package rdma-core. (Reading database ... 91266 files and directories currently installed.) Preparing to unpack .../rdma-core_17.1-1ubuntu0.1_amd64.deb ... Unpacking rdma-core (17.1-1ubuntu0.1) ... Processing triggers for ureadahead (0.100.0-20) ... Processing triggers for systemd (237-3ubuntu10.15) ... Processing triggers for man-db (2.8.3-2ubuntu0.1) ... Setting up rdma-core (17.1-1ubuntu0.1) ... rdma-hw.target is a disabled or a static unit, not starting it. rdma-ndd.service is a disabled or a static unit, not starting it. Processing triggers for ureadahead (0.100.0-20) ... Processing triggers for systemd (237-3ubuntu10.15) ... 3- Running testpmd again after installing rdma-core root@qa-l-vrt-111-124:~# testpmd -c 0xF -n 4 -w 0000:00:06.0 -w 0000:00:07.0 -- --txd=256 --rxd=256 --rxq=1 --txq=1 --nb-cores=3 --i EAL: Detected 4 lcore(s) EAL: Probing VFIO support... EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using unreliable clock cycles ! EAL: PCI device 0000:00:06.0 on NUMA socket -1 EAL: Invalid NUMA socket, default to 0 EAL: probe driver: 15b3:1016 net_mlx5 EAL: PCI device 0000:00:07.0 on NUMA socket -1 EAL: Invalid NUMA socket, default to 0 EAL: probe driver: 15b3:1016 net_mlx5 Interactive-mode selected USER1: create a new mbuf pool <mbuf_pool_socket_0>: n=171456, size=2176, socket=0 Configuring Port 0 (socket 0) Port 0: 26:D4:B6:45:70:D7 Configuring Port 1 (socket 0) Port 1: BA:52:9F:30:6E:BF Checking link statuses... Done
