How to prepare disk images for BitTorrent deployment

Install libtorrent

If you want to use the BitTorrent mode for image deployment you need to install libtorrent-rasterbar including python bindings on all physical hosts specified in the nodes.conf (including seed hosts). If your Linux distribution offers a software package this should be the preferred way to install the library. Otherwise you will need to compile it from source. The source can be obtained from the downloads section on the  libtorrent website. See  here for instructions on how to build and install libtorrent-rasterbar.

Make your disk image known to the XGE

Create the following directories on all physical hosts:

• /images
• /imagepool

Once you have configured etc/nodes.conf, you can use the createDirectories.py script in util/placeholder/ to create the directories.

Copy your VM disk image

Go to your seed host (usually this will either be your XGE head node or a separate host specified as seeder in the nodes.conf) and create a user directory in the imagepool.

# mkdir /imagepool/exampleuser

Copy the disk image to this directory. Suppose, the disk image is stored in your home directory on the XGE head node.

# ls /home/exampleuser | grep image
mydiskimage

If the head node serves as seeder, just copy the file to the new location.

# cp /home/exampleuser/mydiskimage /imagepool/exampleuser

Otherwise, copy the image e.g. via scp.

# scp /home/exampleuser/mydiskimage root@your.seed.host:/imagepool/exampleuser

Note, that these commands must be done with root privileges.

Adjust the images.conf

If you don't already have such a config file, copy the sample one in `/opt/xge/etc/images.conf.sample and adjust it to your needs.

# cd /opt/xge/etc
# cp images.conf.sample images.conf

Insert a reference to the new disk image into your images.conf.

# User who owns the following image
[exampleuser]
# Image location relative to the imagepool (specified in xge.conf)
image = exampleuser/mydiskimage

Set up a tracker

You will need a BitTorrent tracker for using the BitTorrent deployment. It is recommended to install an own tracker somewhere reachable from your XGE cluster. There are many free tracker implementation you can use, for example  opentracker or  peertracker. Follow the install instructions from the tracker of your choice and install it to your system. Remember the announce URL configured with your tracker, you will need it later.

For testing purposes you can alternatively use a publicly available tracker on the Internet.  Trackon provides an up-to-date list of usable trackers. Another option is  openbittorrent.

Create a torrent file and a resume file for your image

Download the torrentcreator tarball and extract it. Chose either the 32 or the 64-bit version according to your system architecture.

# wget http://ds.mathematik.uni-marburg.de/~matthias/xge/tc_32bit.tar.gz
--2010-05-05 13:01:41--  http://ds.mathematik.uni-marburg.de/~matthias/xge/tc_32bit.tar.gz
[...]
2010-05-05 13:01:41 (1.02 MB/s) - “tc_32bit.tar.gz” saved [44136/44136]
# tar xzvf tc_32bit.tar.gz
torrentcreator

Copy the torrentcreator binary to /opt/xge/bin.

# cp torrentcreator /opt/xge/bin/
# which torrentcreator
/opt/xge/bin/torrentcreator

Available options:

# torrentcreator
At least one of -r and -t must be specified.
Usage: [OPTIONS] <target directory or filename>

Options:
-t              : Create a torrentfile.
-r              : Create a resumefile.
-a announce-url : This option is mandatory if -t is specified.
-d resumefile   : Optional. Use resumefile instead of target.fastresume.
-o torrentfile  : Optional. Use torrentfile instead of target.torrent.
-p piece-size   : Optional. Set piece-size to piece-size kb. Default is 256.
-h              : Print this help.

Go to the directory where you just installed your new disk image.

# cd /imagepool/exampleuser
# ls
mydiskimage

Use the torrentcreator to create a torrent file and a resume file. Specify the announce URL from your BitTorrent tracker or from any open tracker you want to use after the -a flag.

# torrentcreator -rt -a http://example.tracker.com/announce mydiskimage
Torrentfile successfully created.
Resumefile successfully created.

If the torrentcreator cannot be found on your system, make sure that /opt/xge/bin is on your path and that opt/xge is mounted via NFS on your seed host (if different from your head node).

# echo $PATH
/bin:/usr/bin:/sbin:/usr/sbin: [...] :/opt/xge/bin