Hello,
I have updated the README.debian further with my own experiences and I
also 0006-README.Debian-add-notes-about-networking-issues-see-.patch
since it mismatched with my README.debian and there for would not patch
(missing --minimal in the command on line 64).
Cheers,
Harrie
--- a/debian/README.Debian
+++ b/debian/README.Debian
@@ -1,4 +1,4 @@
-xen-api for Debian author's notes
+XCP-XAPI for Debian author's notes
---------------------------------
This is the XCP toolstack ported to Debian. Installing these packages will turn
@@ -54,20 +54,23 @@
a virtual machine network interface, a storage repository, an iso image, etc.
When we write:
-network=$(xe network-list bridge=xenbr0 --minimal)
-that means we want the $network shell script variable to contain the UUID of
+NETWORK=$(xe network-list bridge=xenbr0 --minimal)
+
+that means we want the $NETWORK shell script variable to contain the UUID of
the network attached to our xenbr0, but you could also type:
+
xe network-list bridge=xenbr0
and copy/paste the information somewhere for later use as well.
-1/ Install Debian wheezy
+1/ Install Debian Wheezy
-------------------------
-Make sure you use LVM, and leave enough free space on the VG to be able to host
-VMs. What I use is a 2GB RAID1 rootfs on 2 disks, then a 990 GB RAID10 array on
-which I have LVM as a host for swap (1GB), usr (8 to 16 GB), var (the amount of
-RAM + 4GB), tmp (1GB).
+Install Debian Wheezy on your machine. Make sure you use LVM, and
+leave enough free space on the VG to be able to host VMs. What I use is a
+2GB RAID1 rootfs on 2 disks, then a 990 GB RAID10 array on which I have LVM
+as a host for swap (1GB), usr (8 to 16 GB), var (the amount of RAM + 4GB),
+tmp (1GB).
Important note: For performance reasons, it is recommended to use a 32 bits
dom0 kernel (with of course a 32 bits userland OS), and a 64 bits hypervisor
@@ -77,14 +80,13 @@
Also, make sure you install the 686-bigmem (otherwised called as -pae in SID),
if you want your dom0 to be able to run on more than 1GB.
-2/ Upgrade to SID or testing
-----------------------------
-XCP is currently only in SID (not even in Wheezy yet as of writing, but it
-should soon migrate).
+VirtualBox: If you are using VirtualBox to run Debian make sure you enable a
+host only network adaptor. If not you might encounter an authentication error
+when running xe commands.
-3/ Install Xen and XCP
+2/ Install Xen and XCP
----------------------
-3.1/ Installing XCP itself.
+2.1/ Installing XCP itself.
You can just do:
apt-get install xcp-xapi
@@ -95,17 +97,28 @@
apt-get install xcp-xapi xcp-vncterm xcp-guest-templates \
linux-headers-686-pae bridge-utils qemu-keymaps qemu-utils libc6-xen
-3.2/ Samba or NFS
+2.2/ Set default Toolstack
+Switch to using xapi instead of xm or xend to control Xen, you do that with the following:
+
+sed -i "s/TOOLSTACK=.*/TOOLSTACK=xapi/" /etc/default/xen
+
+It is later planned to change this in Debian with 3 switches: xm, xl and xe.
+Since this is not yet implemented, please refer to the /etc/default/xen file
+for an eventual explanation.
+
+2.3/ Samba or NFS
With XCP you can boot guests on iso images. To do that, you will need to use
special iso SR (XCP uses Storage Repositories, or SR), for iso images. The .iso
images can be stored either using NFS or Samba. So you should install support
for one of them if you plan booting on iso images. Note that you can still
-install Debian guests without using an iso SR. Anyway, for Samba, you will need
-to do:
+install Debian guests without using an iso SR.
+
+Anyway, for Samba, you will need to do:
apt-get install cifs-utils
-The following samba config will serve these images in the folder /iso-images:
+Applyling the following samba configuration in /etc/samba/smb.conf will serve
+these images in the folder /iso-images:
[isoimg]
comment = XCP iso image SR
@@ -115,7 +128,7 @@
browseable = yes
guest ok = yes
-3.3/ Bash completion
+2.4/ Bash completion
Since you'll be using xe, I would highly recommend that you install
bash-completion as well. The xcp-xe package contains a
/etc/bash_completion.d/xe file with all the commands. So do:
@@ -125,39 +138,45 @@
and uncomment the use of it in /etc/bash.bashrc so that everyone in the
system may use completion.
-4/ Configuring Xen and XCP
---------------------------
-4.1/ Switch using xapi instead of xm and xend to control Xen
-As of writing, you should do that:
-sed -i "s/TOOLSTACK=.*/TOOLSTACK=xapi/" /etc/default/xen
+2.5/ Reboot
+Reboot and choose XEN 4.1-amd64 in the grub menu. To ensure Xen
+is running you can do the following:
+
+cat /proc/xen/capabilities
+
+If everything is working correctly this should output "control_d".
+To check if xapi is running:
+
+/etc/init.d/xcp-xapi status
-It is later planned to change this in Debian with 3 switches: xm, xl and xe.
-Since this is not yet implemented, please refer to the /etc/default/xen
-file for an eventual explanation.
+This should output "[ ok ] xapi is running.".
-4.2/ Configure network to use a bridge xenbr0
+3/ Configuring Xen and XCP
+--------------------------
+3.1/ Configure network to use a bridge xenbr0
You need to deliberately clobber your interfaces file like so:
-cp -n /etc/network/interfaces /etc/network/interfaces.bak-xcp &&
+cp -n /etc/network/interfaces /etc/network/interfaces.bak-xcp
cat > /etc/network/interfaces << EOF
auto lo
iface lo inet loopback
EOF
-and then use xe to configure the network:
+Note that you will need to be running Xen and have xe as your TOOLSTACK (see
+2.2) to be able to execute xe and configure your network:
-PIF_UUID=`xe pif-list device=eth0 --minimal`
+PIF_UUID=$(xe pif-list device=eth0 --minimal)
xe pif-reconfigure-ip mode=static DNS=1.2.3.4 \
ip=12.34.56.78 netmask=255.255.255.0 gateway=12.34.56.1 \
uuid=$PIF_UUID
Note that it will survive reboots. If you use dhcp, then simply use:
-PIF_UUID=`xe pif-list device=eth0`
+PIF_UUID=$(xe pif-list device=eth0)
xe pif-reconfigure-ip uuid=$PIF_UUID mode=dhcp
-IMPORTANT BUG:
+3.2/ IMPORTANT BUG:
*** pif-reconfigure-ip silently converts your netmask to /32 ***
@@ -179,9 +198,7 @@
g) try rebooting, verify networking is good before doing any real work
with the server
-Note that you will need to be running Xen and have xe as your TOOLSTACK (see
-below) to be able to execute xe and configure your network.
-
+3.3/ Listen on public IP adress
If you want xe to listen on the public IP address, you will need to have your
MANAGEMENT_INTERFACE variable set as the default bridge interface in the
/etc/xcp/inventory file. Here is an example:
@@ -190,61 +207,66 @@
BUILD_NUMBER='0'
CURRENT_INTERFACES='xenbr0'
CONTROL_DOMAIN_UUID='de90f922-8bd8-176e-ea9a-4efece397b73'
-MANAGEMENT_INTERFACE='xenbr0'
+MANAGEMENT_INTERFACE='xenbr0'roo
INSTALLATION_UUID='7f4f3149-2e9d-7bb5-bb99-fd51420a8cd6'
-4.3/ Set Xen as default in GRUB
-Hackish way:
-cd /etc/grub.d
-mv 10_linux 25_linux
+3.4/ Set Xen as default in GRUB
+Using deb package to move xen to boot first:
-Or using a more definitive way:
dpkg-divert --divert /etc/grub.d/08_linux_xen --rename /etc/grub.d/20_linux_xen
-Then set reasonable defaults for the dom0 RAM, and boot only from Xen:
+3.5/ Set dom0 Defaults
+Set reasonable defaults in /etc/default/grub for the dom0 RAM and Virtual Central
+Processing Units, and boot only from Xen:
-echo "
-# Start dom0 with less RAM
-GRUB_CMDLINE_XEN_DEFAULT=\"dom0_mem=1024M\"
+# Start dom0 with 1024 MB of
+GRUB_CMDLINE_XEN_DEFAULT="dom0_mem=1024M,max:1024M"
# Disable OS prober to prevent virtual machines on logical volumes from
# appearing in the boot menu.
GRUB_DISABLE_OS_PROBER=true
-" >>/etc/default/grub
update-grub2 or dpkg-reconfigure grub-pc (the later is needed to check that
you've installed Grub on both MBR of your freshly installed Debian anyway).
-4.4/ Reboot (under Xen)
+VirtualBox: Setting dom0 memory won't work properly so use at your own risk.
+
+3.6/ Reboot (under Xen)
Since you've just configure to run under Xen and with the correct $TOOLSTACK
above, you should be booting under the correct environment.
-5/ Setting-up storage repositories (SRs)
-5.1/ VM SR
+4/ Setting-up storage repositories (SRs)
+4.1/ VM SR
For your virtual machines, XCP uses storage repositories. To create a new
one, you need a big block device on your server. For example, if you use
LVM, you could do something like that to create it:
-lvcreate -L500G -nsr0 node1234vg0
+
+lvcreate -L500G -n <name> <volume group>
to create a 500 GB LVM, then tell XCP to use it:
-xe sr-create type=ext name-label=node1234sr0 \
- device-config:device=/dev/mapper/node1234vg0-sr0
+
+xe sr-create type=ext name-label=<volume group> \
+ device-config:device=/dev/mapper/<volume group>-<name>
By default, this will print the new UUID of the SR, but you can also get it
with:
-SR=`xe sr-list type=ext --minimal`
+
+xe sr-list type=ext --minimal
You should then attach it as the default for the pool of storage for XCP:
-pool=`xe pool-list --minimal`
-xe pool-param-set uuid=$pool default-SR=$SR
+
+SR=`xe sr-list type=ext --minimal`
+POOL=`xe pool-list --minimal`
+xe pool-param-set uuid=$POOL default-SR=$SR
If later, you want to use that SR with openstack, you should also do:
+
xe sr-param-set uuid=$SR other-config:i18n-key=local-storage
this will, by default, use mkfs.ext3, so expect it to take some time
if you gave xe sr-create a big partition.
-5.2/ ISO SR using samba
+4.2/ ISO SR using samba
If you have some ISO images that you want to use in your VMs, then you
should export them through NFS or a samba share, then do as below for
samba:
@@ -255,9 +277,11 @@
right?).
as like this for NFS:
-xe-mount-iso-sr nfs-server:/path/to/isos
+
+/usr/lib/xcp/bin/xe-mount-iso-sr <nfs-server>:/path/to/isos
Once you've done that, you will be able to do:
+
xe cd-list
and see the list of available iso images. Note that there's some sort of
@@ -271,21 +295,26 @@
or
apt-get install nfs-common
-6/ Using XCP: let the fun begins
+5/ Using XCP: let the fun begins
--------------------------------
-6.1/ Choosing a template
+5.1/ Choosing a template
Now that you have XCP up and running on your server, let's play and setup some
-virtual machines.
+virtual machines. You can list available VM templates with:
+
+xe template-list
+
+once you've done your choice, you will need to do something like this:
-You can list available VM templates with "xe template-list", once you've done
-your choice, you will need to do something like this:
-template=$(xe template-list name-label="Debian Squeeze 6.0 (32-bit)" --minimal)
+TEMPLATE=$(xe template-list name-label="Debian Wheezy 7.0 (32-bit)" --minimal)
Creating a VM instance may be done using:
-vm=$(xe vm-install template=$template new-name-label=debian)
+
+VM=$(xe vm-install template=$TEMPLATE new-name-label=debian)
Note that if you have installed bash completion, then you can do:
+
xe vm-install template=[ TAB ]
+
to show the list of templates. You could use both a real template name, or
its UUID.
@@ -293,64 +322,89 @@
way so you don't have to worry about them (eg: if your VM needs hardware
virtualization, for example, or if we should use pygrub, etc.).
-6.2/ Configuration for our VM before it starts
+5.2/ Configuration for our VM before it starts
Get the UID of your xenbr0 network to use it when creating the vif below:
-network=$(xe network-list bridge=xenbr0 --minimal)
+
+NETWORK=$(xe network-list bridge=xenbr0 --minimal)
And use it to attach a new virtual network interface to your VM:
-vif=$(xe vif-create vm-uuid=$vm network-uuid=$network device=0)
+
+VIF=$(xe vif-create vm-uuid=$VM network-uuid=$NETWORK device=0)
Tell the VM that we want to boot using a debian repo:
-xe vm-param-set uuid=$vm \
+
+xe vm-param-set uuid=$VM \
other-config:install-repository=http://ftp.uk.debian.org/debian
Let's say you wanted to use an iso image rather than the ftp repository,
which image would be in your iso SR (see above), then you could have done
this way:
+
xe vm-cd-add vm=windows cd-name=wheezy.iso device=3
Before you start it, you can set its memory to 1GB:
+
xe vm-memory-limits-set static-min=1024MiB static-max=1024MiB \
dynamic-min=1024MiB dynamic-max=1024MiB \
- vm=$vm
+ vm=$VM
If you want to resize the HDD of a VM, you should first get the UUID
of its VDI:
-xe vm-disk-list vm=$vm
+xe vm-disk-list vm=$VM
Then resize its VDI:
xe vdi-resize uuid=<UUID-OF-THE-VDI> disk-size=10GiB
-6.3/ Cloning the newly created VM
+5.3/ Cloning the newly created VM
Since we've done quite some work, it could be possible to clone our virtual
machine to later do another install:
-xe vm-clone new-name-label=wheezy-32-bits-1024mb vm=$vm
-6.4/ Life and death of a VM
+xe vm-clone new-name-label=wheezy-32-bits-1024mb vm=$VM
+
+5.4/ Life and death of a VM
Finally start the VM:
-xe vm-start uuid=$vm
+
+xe vm-start uuid=$VM
And access its console:
-xe console vm=$vm
+xe console vm=$VM
+
+If you want to exit the console, use Ctrl + ']' to quit.
When you're done with the VM, you can do:
-xe vm-shutdown uuid=$vm
+
+xe vm-shutdown uuid=$VM
And if you want to completely get rid of the VM, we do:
-xe vm-destroy uuid=$vm
-7/ Pool of servers
+xe vm-destroy uuid=$VM
+
+5.5/ Acces to VM console
+For more information on the different options to acces your VM console:
+
+http://wiki.xen.org/wiki/Xen_Cloud_Platform:_Access_to_VM_console
+
+6/ Pool of servers
------------------
If you want to add a new machine in the pool, then:
+
xe pool-join master-address=<ip-of-your-other-dom0> \
master-username=<username> master-password=<pass>
-8/ XCP with Openstack
----------------------
+7/ More information & help
+-------------------
+More information on XCP-XAPI can be found on the xen.org wiki:
+
+http://wiki.xen.org/wiki/
+
+Or you can try the IRC / mailinglist:
+
+IRC Freenode: #xen-api
+Mail: xen-...@lists.xen.org
999/ Feedback
-------------
-This quick howto was written by Thomas Goirand <z...@debian.org>, please send
+This quick howto was written by Thomas Goirand <z...@debian.org>, please send
me feedback, corrections and improvements!
>From 50507a5f3c1219333807e6465fa2bc761c231042 Mon Sep 17 00:00:00 2001
From: Daniel Pocock <dan...@pocock.com.au>
Date: Wed, 6 Mar 2013 23:13:02 +0100
Subject: [PATCH 6/6] README.Debian: add notes about networking issues, see
bugs #695221 and #702337
---
debian/README.Debian | 66 +++++++++++++++++++++++---------------------------
1 file changed, 30 insertions(+), 36 deletions(-)
diff --git a/debian/README.Debian b/debian/README.Debian
index 19c1979..36e7744 100644
--- a/debian/README.Debian
+++ b/debian/README.Debian
@@ -136,55 +136,49 @@ Since this is not yet implemented, please refer to the /etc/default/xen
file for an eventual explanation.
4.2/ Configure network to use a bridge xenbr0
-The following will convert a static IP configuration on eth0 to a bridged
-physical interface using xenbr0:
-sed -i "s/allow-hotplug eth0/auto xenbr0/" /etc/network/interfaces
-sed -i "s/iface eth0/iface xenbr0/"/etc/network/interfaces
-echo " bridge_ports eth0" >>/etc/network/interfaces
-In other words, the following /etc/network/interfaces would do:
+You need to deliberately clobber your interfaces file like so:
+cp -n /etc/network/interfaces /etc/network/interfaces.bak-xcp &&
+cat > /etc/network/interfaces << EOF
auto lo
iface lo inet loopback
+EOF
-auto xenbr0
-iface xenbr0 inet static
- bridge_ports eth0
- address 12.34.56.78
- netmask 255.255.255.0
- network 12.34.56.0
- broadcast 12.34.56.255
- gateway 12.34.56.1
-
-If you are using DHCP, then this will fit you:
-auto lo xenbr0
-iface lo inet loopback
-
-iface eth0 inet manual
-
-iface xenbr0 inet dhcp
- bridge_ports eth0
-
-Note that if you are using openvswitch together with XCP, and the xcp-networkd
-daemon, it is overly important that you do setup your xenbr0 as per above,
-because otherwise, you will loose all network connectivity on next reboot,
-as a xenbr0 interface will be setup automatically, and your eth0 will *not*
-be automatically set in the forwarding state (eg: it will stay disabled, which
-means no connectivity).
-
-If you intend to run XCP using Open vSwitch (that is, if your
-/etc/xcp/network.conf contains "openvswitch" and not "bridge), then you should
-tell XCP about your network configuration:
+and then use xe to configure the network:
PIF_UUID=`xe pif-list device=eth0 --minimal`
-xe pif-reconfigure-ip uuid=$PIF_UUID mode=static \
- IP=1.2.3.4 netmask=255.255.255.0 gateway=1.2.3.1
+xe pif-reconfigure-ip mode=static DNS=1.2.3.4 \
+ ip=12.34.56.78 netmask=255.255.255.0 gateway=12.34.56.1 \
+ uuid=$PIF_UUID
Note that it will survive reboots. If you use dhcp, then simply use:
PIF_UUID=`xe pif-list device=eth0`
xe pif-reconfigure-ip uuid=$PIF_UUID mode=dhcp
+IMPORTANT BUG:
+
+ *** pif-reconfigure-ip silently converts your netmask to /32 ***
+
+ http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=695221
+
+If you have a /29 network, for example, pif-reconfigure-ip will
+configure your host using a /32 netmask. This almost always
+leads to a complete loss of connectivity on the next boot.
+
+Therefore, as a workaround, it is necessary to do the following:
+
+a) run the pif-reconfigure-ip anyway, giving it your valid netmask
+b) stop xcp-networkd
+c) manually edit /var/lib/xcp/networkd.db
+d) look for the bits (e.g. 24 or 32) in the file, change it to the actual
+ prefix length of your subnet (e.g. 29 for a /29 subnet)
+e) save the file
+f) start xcp-networkd again
+g) try rebooting, verify networking is good before doing any real work
+ with the server
+
4.3/ Set Xen as default in GRUB
cd /etc/grub.d
mv 10_linux 25_linux
--
1.7.10.4
