CentosWorkstation

From MDWiki
Revision as of 07:48, 11 May 2009 by Matt (talk | contribs) (→‎TODO)
Jump to navigationJump to search

How to make a Centos Workstation (VM)

Warning Do not install the Xen virtualisation package within the virtual Centos as it will crash during boot when running inside the VirtualBox VM!

A virtual machine makes installation and testing faster and easier, especially since it makes it easy to make and revert to snapshots.

Building the VM (Virtual Machine)

For serving the image to lychee, VirtualBox needs to be run as root because the NAT port forwarding from 873/TCP on the host to 873/TCP on the virtual machine is only possible in Linux if the VirtualBox processes are run by root. It is probably best to run VirtualBox as root for other stages, just for VirtualBox's own internal consistency.

  • Inside VirtualBox (F7|C5) server, add users to vboxgroup (/etc/group), run VirtualBox, register product, create new virtual machine
    • "Centos Workstation"
    • Linux, Red Hat 64-bit
    • 1536MB RAM
    • Boot hard disk (Primary master)
      • Create new hard disk
        • Dynamically expanding storage
        • Location: /data/virtualbox/CentosHDD.vdi
        • Size: 40.0 GB
  • This creates a $HOME/.VirtualBox directory, which contains all the information about the VM
  • Further settings
    • General: Enable 3D acceleration
    • Hard Disks: Enable Additional Controller: SATA (AHCI). Slot: SATA Port 0
    • CD/DVD-ROM: Mount Centos 5.3 DVD iso

Installing Centos

  • Start VM
  • Press Enter at Centos installation screen
  • Skip media check
  • Ignore colour mode message
  • Centos Installation screen (GUI)
    • Language: English
    • US English
    • Click OK to erase ALL DATA on device sda VBOX HARDDISK 40955MB? Yes
    • Create custom layout
    • Partitioning
      • Note: I found that making a separate /usr partition stuffed up systemimager.
      • /dev/sda1, /boot, ext3, 100MB, Fixed size, Force to be a primary partition
      • /dev/sda2, /, ext3, 36000MB, Fixed size, Force to be a primary partition
      • /dev/sda3, swap, 2048MB, Fixed size, Force to be a primary partition
      • /dev/sda4, /data, ext3, Fill to maximum allowable size, Force to be a primary partition
    • The Grub boot loader will be installed on /dev/sda
      • (tick) CentOS /dev/sda2
    • Network
      • Active on boot, eth0, DHCP, Auto
    • Hostname:
      • Automatically via DHCP
    • Time
      • Australia/Brisbane time
      • System clock uses UTC
    • Root password
      • <standard workstation root password>
    • Installation (only the following!)
      • Desktop - Gnome
      • Desktop - KDE
      • (tick) Customize later
    • Begin!
    • Wait ~20 minutes
    • Reboot (unmounting ISO)
  • Post Installation Configuration
    • Firewall: Disabled
    • SELinux: Disabled
    • NTP: Enabled, 130.102.2.15, 130.102.128.43, Synchronise system clock, Local time source
    • Create User: Use Network Login: LDAP for both User Information & Authentication:
      • LDAP Search Base DN: dc=md,dc=smms,dc=uq,dc=edu,dc=au
      • LDAP Server: ldap://lychee.md.smms.uq.edu.au/
    • Sound should be OK
    • Reboot
    • Power off VM
  • Turn off VirtualBox

Installing Imaging System

After installation of the base system, we need to perform some one-time steps to set it up to allow it to be imaged to real machines. VM stands for commands run on Virtual Centos (Virtual Machine), H stands for commands run on the host, such as commands to the VirtualBox application itself.

  • H: Forward port 873/TCP on the host to the VM
    • Note: e1000 below is because the virtual Intel 1Gb adapter is used (this is the default). Check that this is indeed the case.
    • VBoxManage setextradata "Centos Workstation" "VBoxInternal/Devices/e1000/0/LUN#0/Config/rsync/Protocol" TCP
    • VBoxManage setextradata "Centos Workstation" "VBoxInternal/Devices/e1000/0/LUN#0/Config/rsync/GuestPort" 873
    • VBoxManage setextradata "Centos Workstation" "VBoxInternal/Devices/e1000/0/LUN#0/Config/rsync/HostPort" 873
  • H: Forward port 2022/TCP on the host to 22/TCP on the VM (useful)
    • VBoxManage setextradata "Centos Workstation" "VBoxInternal/Devices/e1000/0/LUN#0/Config/ssh/Protocol" TCP
    • VBoxManage setextradata "Centos Workstation" "VBoxInternal/Devices/e1000/0/LUN#0/Config/ssh/GuestPort" 22
    • VBoxManage setextradata "Centos Workstation" "VBoxInternal/Devices/e1000/0/LUN#0/Config/ssh/HostPort" 2022
  • H: Start VirtualBox
  • H: Start Centos Workstation
  • VM: chkconfig yum-updatesd off
  • VM: scp -r lychee.md.smms.uq.edu.au:/etc/yum.repos.d/\* /etc/yum.repos.d/
  • VM: yum install --nogpgcheck rpmforge-release
  • VM: yum install perl-AppConfig
  • VM: rpm -iv systemconfigurator-2.2.11-1.noarch.rpm systemimager-common-4.1.6-1.noarch.rpm systemimager-client-4.1.6-1.noarch.rpm systemimager-x86_64initrd_template-4.1.6-1.noarch.rpm
  • VM: We don't need to open port 873/TCP (for serving the image) as no firewall is turned on
  • VM: Modify the virtual Centos to include another kernel in /boot, which is the kernel used for systemimager. Grub will be later modified to present this kernel as an option to boot into.
    • scp -r lychee.md.smms.uq.edu.au:'/usr/share/systemimager/boot/x86_64/standard/{kernel,initrd.img}' /boot/
  • H: Make sure 873/TCP is open if a firewall is enabled

Editing Centos' initrd.img to read real workstation HDD drives

Installing Centos on a given machine will affect which drivers are stored in the initrd.img file that is included by the kernel as it boots. These drivers are used to give the kernel access to the hard disk/filesystems which store the / partitions, etc. The virtual machine probably has different hardware to the real workstation, so the drivers needed for the real workstation must be added to the initrd.img file in our Centos.

  • VM: Determine running kernel version and extract corresponding initrd file:
export KVERSION=`uname -r`
echo $KVERSION
mkdir -p /tmp/$KVERSION && cd /tmp/$KVERSION && gunzip -c /boot/initrd-$KVERSION.img | cpio -iv
  • VM: Copy in required drivers into directory structure. Our real workstations use a different SATA controller to the ahci.ko and ata_piix.ko modules used for the VM. This was discovered by exploring the contents of a Centos installation (initrd.img file) on a real workstation.
cp /lib/modules/$KVERSION/kernel/drivers/ata/sata_nv.ko ./lib/
chmod 600 ./lib/sata_nv.ko
  • VM: Edit init, after "insmod /lib/ata_piix.ko" insert:
echo "Loading sata_nv.ko module"
insmod /lib/sata_nv.ko

Repackage initrd.img file

  • VM: Shutdown to avoid repackaging initrd.img while kernel is running
    • shutdown -h now
[Create System Rescue CD VM]
  • Create new VirtualBox VM
    • "System Rescue CD"
    • Linux/Gentoo (64 bit)
    • 512 MB RAM
    • Boot Hard Disk
      • Use existing: CentosHDD.vdi
  • Settings: CD/DVD ROM
Load System Rescue CD VM and Repackage initrd.img
  • boot: <Enter>
  • Keyboard: <Enter>
  • mkdir /mnt/sda1 /mnt/sda2
  • mount -t ext3 /dev/sda1 /mnt/sda1
  • mount -t ext3 /dev/sda2 /mnt/sda2
  • cd /mnt/sda2/tmp/$KVERSION # where $KVERSION is defined as above
  • find ./ | cpio -o -H newc | gzip -9 > /mnt/sda1/initrd-$KVERSION.img.new
  • cd /mnt/sda1
  • mv initrd-$KVERSION.img{,.old}
  • mv initrd-$KVERSION.img{.new,}
  • shutdown -h now

Creating Initial Image and Sending to Image Server (Initial Image uses Bootable CD)

The initial image ("centos_golden_client") is for imaging a real workstation for the first time, as it performs a full disk partitioning. It shouldn't need to be regenerated unless Centos undergoes a major revision. The update image ("centos_golden_client_update", which boots from Grub on an imaged workstation) does not perform a full partitioning (only /boot, / and swap are repartitioned; /data is unchaged). How this is created is described in a later section. VM stands for commands run on Virtual Centos (Virtual Machine) and IS stands for commands to the image server.

  • VM: Prepare the image for image server, currently on lychee (IP Address 152.98.195.188). This initiates an rsync server process.
    • /usr/sbin/si_prepareclient --server 152.98.195.188
      • Continue? y
      • Continue? y
  • IS: Grab the image from the virtual Centos running on the VirtualBox VM on the host computer (this case uqmd13, which has IP address 152.98.195.144).
    • /usr/sbin/si_getimage -golden-client 152.98.195.144 -image centos_golden_client -ip-assignment static -autodetect-disks
      • Continue? y
      • [An image named centos_golden_client already exists ... Update? y]
      • Press <Enter> to continue ... <Enter>
      • [Update autoinstall script? y]
      • [Run si_clusterconfig? n]
  • VM: No longer needed
  • IS: Fix disk partitioning information
    • Edit: /systemimager/images/centos_golden_client/etc/systemimager/autoinstallscript.conf
      • Alter partitioning to use full disk (should be at least 250 GB):
<disk dev="/dev/sda" label_type="msdos" unit_of_measurement="%">             
    <!--                                                                       
      This disk's output was brought to you by the partition tool "parted",    
      and by the numbers 4 and 5 and the letter Q.                             
    -->                                                                        
    <part  num="1"  size="0.1"  p_type="primary"  p_name="-"  flags="boot" />  
    <part  num="2"  size="20"  p_type="primary"  p_name="-"  flags="-" />      
    <part  num="3"  size="1"  p_type="primary"  p_name="-"  flags="-" />       
    <part  num="4"  size="*"  p_type="primary"  p_name="-"  flags="-" />
  </disk>
    • Recreate installer script:
      • /usr/sbin/si_mkautoinstallscript -image centos_golden_client -force -ip-assignment static -post-install beep
  • IS: Make bootable CD that will download the image from the image server
    • Put a writable CD into the image server
    • Generate ISO
      • /usr/bin/si_mkautoinstallcd --out-file /tmp/systemimager.iso --kernel /usr/share/systemimager/boot/x86_64/standard/kernel --initrd /usr/share/systemimager/boot/x86_64/standard/initrd.img --append "IMAGESERVER=152.98.195.188 IMAGENAME=centos_golden_client SKIP_LOCAL_CFG=y"
    • Burn ISO to CD (to find the CD recorder device run: cdrecord -scanbus)
      • cdrecord -v -eject speed=16 dev=1,0,0 /tmp/systemimager.iso
  • IS: Create the file /var/lib/systemimager/scripts/post-install/98centos_golden_client.grubfix containing the following text:
cat >> /boot/grub/grub.conf <<EOF
title Update Image (SystemImager) (/data is preserved)
    root (hd0,0)
    kernel /kernel noexec=off root=/dev/ram ramdisk_blocksize=1024 ramdisk_size=80000 IMAGESERVER=152.98.195.188 IMAGENAME=centos_golden_client_update SKIP_LOCAL_CFG=y
    initrd /initrd.img
EOF
  • IS: Start rsync server
    • To check status: /etc/init.d/systemimager-server-rsyncd status
    • To start: /etc/init.d/systemimager-server-rsyncd start
  • IS: The initial image is now ready to be served to the real workstations

Creating Update Image and Sending to Image Server (Update Image boots from Grub)

These steps are for creating and sending the update image ("centos_golden_client_update"), which can be installed on a real workstation from the grub menu once the initial image has been loaded. These steps can be run any number of times. This image does not perform a full partitioning on the real workstation local disk (only /boot, / and swap are repartitioned; /data is unchaged). VM stands for commands run on Virtual Centos (Virtual Machine) and IS stands for commands to the image server.

  • VM: Edit /etc/systemimager/updateclient.local.exclude to specify any directories the contents of which should not be copied onto the real workstation, such as the contents to external nfs mounts (e.g. /melon1/*)
  • VM: If not created already, make /nvidia directory for installing NVIDIA driver
  • VM: Prepare the image for image server, currently on lychee (IP Address 152.98.195.188). This initiates an rsync server process. The partitioning on disk /dev/sda is excluded
    • /usr/sbin/si_prepareclient --server 152.98.195.188 -e sda
      • Continue? y
      • Continue? y
  • VM: Check /etc/systemconfig/systemconfig.conf to see it uses the right initrd.img file in the INITRD setting. I have seen it pick up the /boot/initrd-$KVERSION.img.old file before. Change it to the right one if not correct.
  • IS: Grab the image from the virtual Centos running on the VirtualBox VM on the host computer (this case uqmd13, which has IP address 152.98.195.144). Also defined are the excluded contents of certain directories.
    • /usr/sbin/si_getimage -exclude /nfs4importsguava/\* -exclude /nfs4importslychee/\* -exclude /data1/\* -exclude /data2/\* -exclude /data3/\* -exclude /data4/\* -exclude /data5/\* -exclude /data6/\* -exclude /data7/\* -exclude /data8/\* -exclude /data9/\* -exclude /data10/\* -exclude /melon1/\* -exclude /melon2/\* -exclude /marksw/\* -exclude /opt/\* -exclude /home1/\* -exclude /home2/\* -golden-client 152.98.195.144 -image centos_golden_client_update -ip-assignment static
      • Continue? y
      • [An image named centos_golden_client_update already exists ... Update? y]
      • Press <Enter> to continue ... <Enter>
      • [Update autoinstall script? y]
      • [Run si_clusterconfig? n]
  • VM: No longer needed
  • IS: Fix disk partitioning information
    • Edit /systemimager/images/centos_golden_client_update/etc/systemimager/autoinstallscript.conf to prevent repartitioning/reformatting of /data
      • Add format="no" into tag also containing real_dev="/dev/sda4"
    • Recreate installer script:
      • /usr/sbin/si_mkautoinstallscript -image centos_golden_client_update -force -ip-assignment static -post-install reboot
  • IS: Create the file /var/lib/systemimager/scripts/post-install/98centos_golden_client_update.grubfix containing the following text:
cat >> /boot/grub/grub.conf <<EOF
title Update Image (SystemImager) (/data is preserved)
    root (hd0,0)
    kernel /kernel noexec=off root=/dev/ram ramdisk_blocksize=1024 ramdisk_size=80000 IMAGESERVER=152.98.195.188 IMAGENAME=centos_golden_client_update SKIP_LOCAL_CFG=y
    initrd /initrd.img
EOF
  • IS: Create the file /var/lib/systemimager/scripts/post-install/97centos_golden_client_update.nvidia containing the following text, replacing $KVERSION with the actual version given by uname -r within the VM:
uname () { echo $KVERSION; }
sh /nvidia/NVIDIA-Linux-x86_64-180.51-pkg2.run -a -q -X
  • IS: Start rsync server
    • To check status: /etc/init.d/systemimager-server-rsyncd status
    • To start: /etc/init.d/systemimager-server-rsyncd start
  • IS: The update image is now ready to be served to the real workstations

Software

Kernel Updates

Note: Any kernel updates will probably require also rebuilding initrd.img as described above (#Editing Centos' initrd.img to read real workstation HDD drives). As such it is suggested that the system is usually updated without any kernel updates:

  • yum update --exclude=kernel

Yum

  • kernel-devel kernel-headers # Make sure these match the current kernel
  • gcc
  • zsh
  • lyx
  • fonts-chinese
  • kile
  • rxvt
  • xterm
  • nedit
  • grace
  • vlc
  • w3m
  • iftop
  • htop
  • rpm-build

Other

  • /usr/local is copied from the Fedora 7 workstations created by Mitch and AJ

Printer

  • Go to System->Administration->Printing
    • New Printer
      • Name: "MDPrinter"
      • AppSocket/HP JetDirect: Hostname: 172.23.213.43
      • Select Printer from Database: HP
        • Color LaserJet 4650 (Postscript)
    • More settings (MDPrinter):
      • Settings->Description: "MDGroupPrinter"
      • Settings->Make default Printer
      • Policies->Error policy->Abort job
      • Installable options->Duplex Unit
      • Printer options->Media size: A4
      • Paper source->Tray 2

Pymol

  • After pymol is installed by yum it has to be fixed.
  1. Download source
  2. In source directory, run python setup.py build fix (Dirty hack): As the one above gives the python shared object in 32-bit, we need to rebuild it. This involves running, inside the corresponding pymol source (v 1.1): python setup.py build. Then, copying the libpymol.so built to /usr/lib/python2.5/site-packages/pymol/_cmd.so.

NFS Mounts

The following instructions describe how to set up NFS mounts. The NFS mounts only work properly on a real machine as the port forwarding isn't yet set up for the VM.

  • Add to /etc/fstab:
# IMPORT NFS4SHARES
guava:/ /nfs4importsguava       nfs4    rw,rsize=8192,wsize=8192,timeo=14,intr
lychee:/        /nfs4importslychee      nfs4    rw,rsize=8192,wsize=8192,timeo=14,intr
# GUAVA DISKS
/nfs4importsguava/data3 /data3  none    bind    0       0
/nfs4importsguava/data1 /data1  none    bind    0       0
/nfs4importsguava/data2 /data2  none    bind    0       0
/nfs4importsguava/data6 /data6  none    bind    0       0
# LYCHEE DISKS
/nfs4importslychee/data7        /data7  none    bind    0       0
/nfs4importslychee/home         /home   none    bind    0       0
/nfs4importslychee/home1        /home1  none    bind    0       0
/nfs4importslychee/home2        /home2  none    bind    0       0
/nfs4importslychee/data5        /data5  none    bind    0       0
/nfs4importslychee/marksw       /marksw none    bind    0       0
/nfs4importslychee/opt       /opt none    bind    0       0
# GET THE OLD GRAPE DISKS 
grape:/data1    /data8  nfs     rw,rsize=32768,wsize=32768,nfsvers=3    0       0
grape:/data2    /data9  nfs     rw,rsize=32768,wsize=32768,nfsvers=3    0       0
grape:/data3    /data10 nfs     rw,rsize=32768,wsize=32768,nfsvers=3    0       0
# MELON DISKS
melon:/melon1   /melon1 nfs4    rw,rsize=8192,wsize=8192,timeo=14,intr
melon:/melon2   /melon2 nfs4    rw,rsize=8192,wsize=8192,timeo=14,intr
  • Create mount points
    • mkdir -p /nfs4importsguava /nfs4importslychee /data1 /data2 /data3 /data4 /data5 /data6 /data7 /data8 /data9 /data10 /melon1 /melon2 /marksw /opt /home1 /home2
  • Add details to /etc/systemimager/updateclient.local.exclude:
/nfs4importsguava/*
/nfs4importslychee/*
/data1/*
/data2/*
/data3/*
/data4/*
/data5/*
/data6/*
/data7/*
/data8/*
/data9/*
/data10/*
/melon1/*
/melon2/*
/marksw/*
/opt/*
/home1/*
/home2/*
  • Add to /etc/idmapd.conf after [General]
Domain = md.smms.uq.edu.au
  • Change in /etc/idmapd.conf nobody to nfsnobody
  • Reboot

How to make a Centos Workstation (Real Machine)

Note: During imaging, there may be an error about kernel failing in a memory allocation. This can be ignored.

Initial Image ("centos_golden_client")

Warning! Make sure that the local disk does not have anything important on it as it will be wiped

Update Image ("centos_golden_client_update")

Note: The /data partition will not be changed

  • Boot from Update Image (SystemImager) (/data is preserved) in grub menu

NVIDIA Graphics Driver

The NVIDIA driver will have to be installed every time the computer is imaged. It is not required, but improves performance for things like VMD.

  • Log in to Centos over ssh/console (i.e. not through its gdm)
    • init 3
    • cd /tmp
    • wget NVIDIA-Linux-x86_64-180.51-pkg2.run
    • sh NVIDIA-Linux-x86_64-180.51-pkg2.run
      • License? Accept
      • Download kernel interface? Yes
      • No matching interface found. OK
      • Install compabtible 32-bit libraries? Yes
      • (TODO.....) init 5

How to move the VirtualBox VM to another computer

  • Move the $HOME/.VirtualBox directory and the /data/virtualbox directory. This is how it can be backed up also.
Retrieved from "http://compbio.biosci.uq.edu.au/mediawiki/index.php?title=CentosWorkstation&oldid=12138"