Hi,
after creating a new subsystem this subsystem and its namespaces works:
# ceph nvmeof subsystem list
+---------------------------------+------------------+--------------------+---------------+-------+--------------+--------------+--------------+-------------------+
|Nqn |Serial Number |Model Number
|Namespace Count|Subtype|Max Namespaces|Has Dhchap Key|Allow Any Host|Created
Without Key|
+---------------------------------+------------------+--------------------+---------------+-------+--------------+--------------+--------------+-------------------+
|nqn.2025-08.com.example:HOSA.hosa|Ceph30844860597194|Ceph bdev Controller|2
|NVMe |128 |False |True |False
|
|nqn.2025-08.com.example:TEST |Ceph44670645393042|Ceph bdev Controller|1
|NVMe |32 |False |True |False
|
+---------------------------------+------------------+--------------------+---------------+-------+--------------+--------------+--------------+-------------------+
# ceph nvmeof listener list nqn.2025-08.com.example:TEST
+-------------+------+--------------+------+------+-------+
|Host Name |Trtype|Traddr |Secure|Adrfam|Trsvcid|
+-------------+------+--------------+------+------+-------+
|b236asbrce101|TCP |10.105.141.195|False |ipv4 |4420 |
|b236asbrce102|TCP |10.105.141.196|False |ipv4 |4420 |
|b236asbrce103|TCP |10.105.141.197|False |ipv4 |4420 |
|b236asbrce104|TCP |10.105.141.198|False |ipv4 |4420 |
+-------------+------+--------------+------+------+-------+
# ceph nvmeof namespace list nqn.2025-08.com.example:TEST
+-----------------------------------------+---------------------------+--------+--------+--------------+----------+-----------+-----------+------------+-------------+------------+-----+----+------------------------------------+----------------------------+-----------+-------------------+---------+
|Bdev Name |Rbd Image |Rbd
Pool|Lb Group|Rbd Image Size|Block Size|R/W Ios/Sec|R/W Mbs/Sec|Read
Mbs/Sec|Write Mbs/Sec|Auto Visible|Hosts|Nsid|Uuid
|Ns Subsystem Nqn |Trash Image|Disable Auto Resize|Read Only|
+-----------------------------------------+---------------------------+--------+--------+--------------+----------+-----------+-----------+------------+-------------+------------+-----+----+------------------------------------+----------------------------+-----------+-------------------+---------+
|bdev_b9b0259a-3702-415b-b223-0946cd91eaa3|nvme_nvmeof_hosa_w2h0v7yb8q|nvmeof
|8 |150GB |512B |0 |0 |0 |0
|True |[] |1
|b9b0259a-3702-415b-b223-0946cd91eaa3|nqn.2025-08.com.example:TEST|False
|False |False |
+-----------------------------------------+---------------------------+--------+--------+--------------+----------+-----------+-----------+------------+-------------+------------+-----+----+------------------------------------+----------------------------+-----------+-------------------+---------+
On the initiator:
# nvme connect-all -t tcp -a 10.105.141.195 -s 8009 -v
warning: use hostid which does not match uuid in hostnqn
nvme0: nqn.2014-08.org.nvmexpress.discovery connected
nvme1: nqn.2025-08.com.example:HOSA.hosa connected
nvme2: nqn.2025-08.com.example:HOSA.hosa connected
nvme3: nqn.2025-08.com.example:HOSA.hosa connected
nvme4: nqn.2025-08.com.example:HOSA.hosa connected
nvme5: nqn.2025-08.com.example:TEST connected
nvme6: nqn.2025-08.com.example:TEST connected
nvme7: nqn.2025-08.com.example:TEST connected
nvme8: nqn.2025-08.com.example:TEST connected
nvme0: nqn.2014-08.org.nvmexpress.discovery disconnected
# nvme list -v
Subsystem Subsystem-NQN
Controllers
----------------
--------------------------------------------------------------------------------------------
----------------
nvme-subsys1 nqn.2025-08.com.example:HOSA.hosa
nvme1, nvme2, nvme3, nvme4
nvme-subsys5 nqn.2025-08.com.example:TEST
nvme5, nvme6, nvme7, nvme8
Device SN MN FR
TxPort Address Slot Subsystem Namespaces
-------- -------------------- ---------------------------------------- --------
------ -------------- ------ ------------ ----------------
nvme1 Ceph30844860597194 Ceph bdev Controller 25.05
tcp traddr=10.105.141.195,trsvcid=4420,src_addr=10.105.141.212
nvme-subsys1
nvme2 Ceph30844860597194 Ceph bdev Controller 25.05
tcp traddr=10.105.141.196,trsvcid=4420,src_addr=10.105.141.212
nvme-subsys1
nvme3 Ceph30844860597194 Ceph bdev Controller 25.05
tcp traddr=10.105.141.197,trsvcid=4420,src_addr=10.105.141.212
nvme-subsys1
nvme4 Ceph30844860597194 Ceph bdev Controller 25.05
tcp traddr=10.105.141.198,trsvcid=4420,src_addr=10.105.141.212
nvme-subsys1
nvme5 Ceph44670645393042 Ceph bdev Controller 25.05
tcp traddr=10.105.141.195,trsvcid=4420,src_addr=10.105.141.212
nvme-subsys5 nvme5n1
nvme6 Ceph44670645393042 Ceph bdev Controller 25.05
tcp traddr=10.105.141.196,trsvcid=4420,src_addr=10.105.141.212
nvme-subsys5 nvme5n1
nvme7 Ceph44670645393042 Ceph bdev Controller 25.05
tcp traddr=10.105.141.197,trsvcid=4420,src_addr=10.105.141.212
nvme-subsys5 nvme5n1
nvme8 Ceph44670645393042 Ceph bdev Controller 25.05
tcp traddr=10.105.141.198,trsvcid=4420,src_addr=10.105.141.212
nvme-subsys5 nvme5n1
Device Generic NSID Usage Format
Controllers
------------ ------------ ---------- --------------------------
---------------- ----------------
/dev/nvme5n1 /dev/ng5n1 0x1 161.06 GB / 161.06 GB 512 B + 0 B
nvme5, nvme6, nvme7, nvme8
# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
sda 8:0 0 60G 0 disk
├─sda1 8:1 0 1G 0 part /boot/efi
├─sda2 8:2 0 2G 0 part /boot
└─sda3 8:3 0 56.9G 0 part
├─system-swap 252:0 0 100M 0 lvm [SWAP]
└─system-root 252:1 0 56.8G 0 lvm /var/lib/containers/storage/overlay
/
sr0 11:0 1 4M 0 rom
nvme5n1 259:9 0 150G 0 disk
So there must be a slight difference in the config of the subsystems /
namespaces
between v19 and v20.
Regards
--
Robert Sander
Linux Consultant
Heinlein Consulting GmbH
Schwedter Str. 8/9b, 10119 Berlin
https://www.heinlein-support.de
Tel: +49 30 405051 - 0
Fax: +49 30 405051 - 19
Amtsgericht Berlin-Charlottenburg - HRB 220009 B
Geschäftsführer: Peer Heinlein - Sitz: Berlin
_______________________________________________
ceph-users mailing list -- [email protected]
To unsubscribe send an email to [email protected]
