Test Procedure with KVM guests + iPXE
=====================================
- 2 guests: iSCSI target/server and iSCSI initiator/client.
- 1 bridge for iSCSI traffic (virbr-iscsi, new), static ip.
- 1 bridge for internet access (virbr0, exists), dhcp ip.
- Release/Installer: Debian Buster Alpha 5 (netinst)
Host:
----
Configure the iSCSI bridge and QEMU access in the host:
# ip link add dev virbr-iscsi type bridge
# ip link set dev virbr-iscsi up
# echo 'allow virbr-iscsi' >>/etc/qemu/bridge.conf
iSCSI target:
------------
This guest serves an iSCSI target with one LUN
in iSCSI NIC with IP 10.0.0.1 for IP 10.0.0.2.
$ qemu-system-x86_64 \
-nodefaults \
-enable-kvm \
-smp 2 -m 4096 \
-serial stdio \
-nographic -vga none \
-netdev bridge,id=bridge-world,br=virbr0 \
-netdev bridge,id=bridge-iscsi,br=virbr-iscsi \
-device
virtio-net-pci,netdev=bridge-world,id=nic-world,mac=52:54:00:00:00:11
\
-device
virtio-net-pci,netdev=bridge-iscsi,id=nic-iscsi,mac=52:54:00:00:00:22
\
-drive file=debian-iscsi-target.qcow2,if=virtio \
-drive
file=debian-buster-DI-alpha5-amd64-netinst.iso,media=cdrom,read-only,if=scsi
# lsb_release -d
Description: Debian GNU/Linux buster/sid
Configure iSCSI NIC:
# cat <<EOF >/etc/network/interfaces.d/nic-iscsi
iface ens4 inet static
address 10.0.0.1/24
EOF
# ifup ens4
# ip addr list ens4
3: ens4: <...>
link/ether 52:54:00:00:00:22 brd ff:ff:ff:ff:ff:ff
inet 10.0.0.1/24 brd 10.0.0.255 scope global ens4
...
Configure iSCSI target/lun:
# apt-get install -y tgt
# mkdir /var/lib/iscsi
# dd if=/dev/zero of=/var/lib/iscsi/disk bs=1 count=0 seek=4G
# tgtadm --lld iscsi --op new --mode target --tid 1 -T
iqn.2019-03.com.example:target1
# tgtadm --lld iscsi --op new --mode logicalunit --tid 1 --lun 1
-b /var/lib/iscsi/disk
# tgtadm --lld iscsi --op bind --mode target --tid 1 -I 10.0.0.2
# tgt-admin --dump >/etc/tgt/conf.d/target1.conf
iSCSI initiator:
---------------
This guest first boots iPXE to configure iBFT,
and then boots/chainloads to debian-installer.
Later we install the patched disk-detect udeb.
(The netboot installer didn't find modules no
matter what, so this uses netinst iso's files)
$ wget http://boot.ipxe.org/ipxe.lkrn
$ cp
debian-buster-DI-alpha5-amd64-netinst.iso:install.amd/{vmlinuz,initrd.gz}
.
$ python3 -m http.server &
Serving HTTP on 0.0.0.0 port 8000 ...
$ qemu-system-x86_64 \
-nodefaults \
-enable-kvm \
-smp 2 -m 4096 \
-serial stdio \
-vga virtio \
-display vnc=0.0.0.0:1 \
-netdev bridge,id=bridge-world,br=virbr0 \
-netdev bridge,id=bridge-iscsi,br=virbr-iscsi \
-device
virtio-net-pci,netdev=bridge-world,id=nic-world,mac=52:54:00:00:00:01
\
-device
virtio-net-pci,netdev=bridge-iscsi,id=nic-iscsi,mac=52:54:00:00:00:02
\
-drive
file=debian-buster-DI-alpha5-amd64-netinst.iso,media=cdrom,read-only,if=scsi
\
-kernel ipxe.lkrn
Connect to VNC for iPXE shell:
$ vncviewer <address>:1
iPXE <...>
<wait for NIC address detection>
Press Ctrl-B for iPXE command line.
^B
iPXE>
Configure iSCSI NIC:
iPXE> ifopen net1
iPXE> set net1/ip 10.0.0.2
iPXE> set net1/netmask 255.255.255.0
Configure iBFT: (iSCSI portal 10.0.0.1, LUN 1 on target iqn.<...>:target1)
iPXE> sanhook iscsi:10.0.0.1:::1:iqn.2019-03.com.example:target1
Registered SAN device 0x80
Boot the installer
(add option 'disk-detect/ibft/enable=true' for installer
and option 'iscsi_auto' for system to boot with iBFT):
iPXE> ifopen net0
iPXE> kernel http://192.168.122.1:8000/vmlinuz initrd=initrd.gz
disk-detect/ibft/enable=true --- console=ttyS0 iscsi_auto
iPXE> initrd http://192.168.122.1:8000/initrd.gz
iPXE> boot
Back to serial console.
Proceed with the installer.
In 'Users and passwords' dialogs, select 'Go back', and 'Execute a
shell', and 'Continue'.
~ # wget 192.168.122.1:8000/disk-detect_1.136+ibft1_amd64.udeb
~ # udpkg --unpack disk-detect_1.136+ibft1_amd64.udeb
~ # debconf-get disk-detect/ibft/enable
true
(Use this if you need it.)
~ # debconf-set disk-detect/ibft/enable true
Start another installer menu with the new debconf templates/question:
~ # debconf -o d-i /usr/bin/main-menu
Proceed with the installer.
In the 'Partition disks' dialog, the iSCSI LUN should be present:
SCSI4 (0,0,1) (sda) - 4.3 GB IET VIRTUAL-DISK
Proceed with the installer.
System reboots.
Back to VNC console, go again to iPXE shell:
$ vncviewer <address>:1
iPXE <...>
<wait for NIC address detection>
Press Ctrl-B for iPXE command line.
^B
iPXE>
Configure iSCSI NIC:
iPXE> ifopen net1
iPXE> set net1/ip 10.0.0.2
iPXE> set net1/netmask 255.255.255.0
Boot from iSCSI (iSCSI portal 10.0.0.1, LUN 1 on target iqn.<...>:target1)
iPXE> sanboot iscsi:10.0.0.1:::1:iqn.2019-03.com.example:target1
Registered SAN device 0x80
Booting from SAN device 0x80
Back to serial console.
Watch the system boot:
GNU GRUB version 2.02+dfsg1-12
+----------------------------------------------------------------------------+
|*Debian GNU/Linux
|
| Advanced options for Debian GNU/Linux
|
|
|
<...>
The highlighted entry will be executed automatically in 0s.
Loading Linux 4.19.0-2-amd64 ...
Loading initial ramdisk ...
Setting up software interface ens5
iscsistart: Logging into iqn.2019-03.com.example:target1 10.0.0.1:3260,1
iscsistart: can not connect to iSCSI daemon (111)!
iscsistart: version 2.0-874
iscsistart: Connection1:0 to [target:
iqn.2019-03.com.example:target1, portal: 10.0.0.1,3260] through
[iface: default] is operational now
iscsistart: TargetName not set. Exiting iscsistart
/dev/sda1: clean, 35475/262144 files, 364064/1048064 blocks
Debian GNU/Linux buster/sid debian-iscsi-initiator ttyS0
debian-iscsi-initiator login: root
Password:
root@debian-iscsi-initiator:~# mount | grep -w /
/dev/sda1 on / type ext4 (rw,relatime,errors=remount-ro)
root@debian-iscsi-initiator:~# ls -ld /sys/block/sda
<...> /sys/block/sda ->
../devices/platform/host3/session1/target3:0:0/3:0:0:1/block/sda
root@debian-iscsi-initiator:~# cat
/sys/devices/platform/host3/session1/iscsi_session/session1/targetname
iqn.2019-03.com.example:target1
With just this patch, boot depends on the 'iscsi_auto' option.
There are more patches for partman-iscsi to do that automatically.
syslog snippet
==============
...
Mar 15 14:43:35 kernel: [ 0.005211] iBFT found at 0x9e520.
...
Mar 15 14:49:42 disk-detect: insmod
/lib/modules/4.19.0-1-amd64/kernel/drivers/scsi/iscsi_boot_sysfs.ko
Mar 15 14:49:42 disk-detect: insmod
/lib/modules/4.19.0-1-amd64/kernel/drivers/firmware/iscsi_ibft.ko
Mar 15 14:49:42 kernel: [ 371.250451] iBFT detected.
Mar 15 14:49:42 disk-detect: # BEGIN RECORD 2.0-874
Mar 15 14:49:42 disk-detect: iface.initiatorname =
iqn.2010-04.org.ipxe:00000000-0000-0000-0000-000000000000
Mar 15 14:49:42 disk-detect: iface.hwaddress = 52:54:00:00:00:02
Mar 15 14:49:42 disk-detect: iface.bootproto = STATIC
Mar 15 14:49:42 disk-detect: iface.ipaddress = 10.0.0.2
Mar 15 14:49:42 disk-detect: iface.subnet_mask = 255.255.255.0
Mar 15 14:49:42 disk-detect: iface.primary_dns = 192.168.122.1
Mar 15 14:49:42 disk-detect: iface.vlan_id = 0
Mar 15 14:49:42 disk-detect: iface.net_ifacename = ens5
Mar 15 14:49:42 disk-detect: node.name = iqn.2019-03.com.example:target1
Mar 15 14:49:42 disk-detect: node.conn[0].address = 10.0.0.1
Mar 15 14:49:42 disk-detect: node.conn[0].port = 3260
Mar 15 14:49:42 disk-detect: node.boot_lun = 01000000
Mar 15 14:49:42 disk-detect: # END RECORD
Mar 15 14:49:42 disk-detect: Setting up software interface ens5
Mar 15 14:49:42 disk-detect: iscsistart: version 2.0-874
Mar 15 14:49:42 kernel: [ 371.266976] Loading iSCSI transport class v2.0-870.
Mar 15 14:49:42 kernel: [ 371.274601] iscsi: registered transport (tcp)
Mar 15 14:49:42 kernel: [ 371.276222] scsi host3: iSCSI Initiator over TCP/IP
Mar 15 14:49:42 disk-detect: iscsistart: Connection1:0 to [target:
iqn.2019-03.com.example:target1, portal: 10.0.0.1,3260] through
[iface: default] is operational now
Mar 15 14:49:42 kernel: [ 371.284243] scsi 3:0:0:0: RAID
IET Controller 0001 PQ: 0 ANSI: 5
Mar 15 14:49:42 kernel: [ 371.289197] scsi 3:0:0:1: Direct-Access
IET VIRTUAL-DISK 0001 PQ: 0 ANSI: 5
Mar 15 14:49:42 disk-detect: iscsistart: Logging into
iqn.2019-03.com.example:target1 10.0.0.1:3260,1
Mar 15 14:49:42 kernel: [ 371.295427] sd 3:0:0:1: Power-on or device
reset occurred
Mar 15 14:49:42 kernel: [ 371.298133] sd 3:0:0:1: [sda] 8388608
512-byte logical blocks: (4.29 GB/4.00 GiB)
Mar 15 14:49:42 kernel: [ 371.298136] sd 3:0:0:1: [sda] 4096-byte
physical blocks
Mar 15 14:49:42 kernel: [ 371.298555] sd 3:0:0:1: [sda] Write Protect is off
Mar 15 14:49:42 kernel: [ 371.298558] sd 3:0:0:1: [sda] Mode Sense: 69 00 10 08
Mar 15 14:49:42 kernel: [ 371.299453] sd 3:0:0:1: [sda] Write cache:
enabled, read cache: enabled, supports DPO and FUA
Mar 15 14:49:42 kernel: [ 371.312823] sd 3:0:0:1: [sda] Attached SCSI disk
Mar 15 14:49:42 disk-detect: iBFT disk detection finished.
...
--
Mauricio Faria de Oliveira
