How to Install Placeholder Virtual Machines

The XGE needs placeholder (PH) virtual machines (VM) on every physical node. The number of PH-VMs depends of the number of installed CPU cores on the nodes, i.e. if a node has 2 CPUs, each with 2 cores (total of 4 cores) the XGE needs 4 PH-VMs on that node. To ease the installation and deployment of PH-VMs the XGE provides a set of utilities.

If you use Torque as RM, its pbs_mom daemon is installed within the initial ramdisk of the PH-VMs by default, thus no local file system is needed. The actual installed Torque version is 2.3.10.

Configure host names and MAC addresses

Every VM needs its own unique MAC address/host name and the XGE needs to know these addresses. Be sure to chose a private MAC address (see the  Xen Networking HOWTO for further information). All information (Host name of the physical node, MAC address of the physical node, MAC of the VM, host name of the VM) must be registered into the /opt/xge/etc/nodes.conf file.

The first section named [physical_nodes] contains one line per physical node present in the network. Every line contains the host name (resolvable by DNS) and the MAC address. The second section named [mac_addresses] contains one line per virtual node present in the network. Every line contains the host name (resolvable by DNS) and the MAC address. The third section named [virtual_machines] contains the mapping between the physical nodes and the virtual nodes. Every line contains the host name (resolvable by DNS) of the virtual nodes and the host name of the physical node. It is possible that multiple virtual nodes run on one physical node.

An example might look like this:

[physical_nodes]
node01  = 00:19:d1:e5:9d:b7
pc12856 = 00:19:d1:12:2a:79

# Host names and MAC addresses of all virtual (!) nodes you want to use with
# the XGE
# Format:  host name = MAC address
[mac_addresses]
node01c0 = 1A:00:00:00:00:64
node01c1 = 1A:00:00:00:00:65
node02c0 = 1A:00:00:00:00:66
node02c1 = 1A:00:00:00:00:67

# Mapping between the virtual and physical nodes, i.e. which virtual node
# runs on which physical node.  Make sure that you do not mix up this table!
# Format: Host name virtual machine = host name physical machine
[virtual_machines]
node01c0 = node01
node01c1 = node01
node02c0 = pc12856
node02c1 = pc12856

Edit the basic placeholder VM settings

All basic PH-VMs related configuration is stored in /opt/xge/etc/placeholder.conf. Check and edit this file if needed. If your system architecture cannot be determined by the setupph.sh script (see next section) or if you want to create placeholder machines for a different architecture than the current one, set the arch value to either 32 or 64. Furthermore you can specify the amount of RAM needed for the PH-VMs. If you use Torque, be sure to set the Import option of the [torque] section to True.

Set up the base placeholder VM

---

NOTE:

If you successfully used setup.sh, you do not need to perform the following steps because the script automatically downloads the PL VMs for you.

---

# cd /opt/xge/util/placeholder/

Download the PH-VMs image tarballs (containing kernel and a ramdisk) and put them in the placeholder directory. Chose either the 32 or the 64-bit version.

# wget http://ds.mathematik.uni-marburg.de/~matthias/xge/xge_phutil_32bit.tar.gz
--2010-05-04 10:35:12--  http://ds.mathematik.uni-marburg.de/~matthias/xge/xge_phutil_32bit.tar.gz
[...]
2010-05-04 10:35:13 (11.2 MB/s) - `xge_phutil_32bit.tar.gz' saved [3850712/3850712]

Now run the setupph.py script to prepare the initial ramdisk for the PH-VMs. NOTE: If you chose to use Torque and did not specify the correct host name of your Torque Server in /opt/xge/etc/placeholder.conf, the PH-VMs won't be able to find the server!

# ./setupph.py 
Determine system architecture.
Architecture is 32bit.
Read configuration options.
   -> install torque: yes
   -> torque server: suse
   -> kernel: /opt/xge/util/ph/boot/vmlinuz26-xen
   -> ramdisk: /opt/xge/util/ph/boot/initrd26-xen
Search tarball in /opt/xge/util/placeholder.
Tarball /opt/xge/util/placeholder/xge_phutil_32bit.tar.gz found.
Extract tarball.
Customize initial ramdisk ...
   Extract initramfs ...   done.
   Write pbs_mom configuration file ...  done.
   Compress initramfs ...  done.
done.
Install placeholder ...
   Copy kernel to /opt/xge/util/ph/boot/vmlinuz26-xen ...  done.
   Copy ramdisk to /opt/xge/util/ph/boot/initrd26-xen ...  done.
done.
Clean up.
Placeholder successfully installed.

Configure the required services

To be able to use the PH-VMs (and of course the real VMs also) you need to configure a certain set of services. This includes:

• All users need to be present in all VMs (use LDAP+PAM for this)
• All VMs must have an IP address
• All VMs must have network access to reach the headnode
• All VMs must be able to resolve their own hostname (use DNS)

Boot the PH-VMs

After the installation successfully finished, you can start the PH-VMs with the bootph.py script in the same directory.

# ./bootphs.py 
Run bootscript local!
Using config file "/dev/null".
Started domain suse-vm (id=14)
done.

Customized the PH-VMs

Usually you should not need to modify/customize the provided PH-VM images. In case you need to modify it anyway, follow the steps in the tutorial.

Installation

Return to Installation section.