Linux Installation on Intel® iSCSI Boot

This document explains how to install and boot Linux on an iSCSI target LU using opensource linux-iSCSI initiator.

The example shows how to perform this process for SUSE* Linux Enterprise Server 9-SP3.

The Red Hat* Enterprise Linux 4 process can be found here.

Prerequisites:

  1. A host system to install Linux on.
  2. A driver disk compatible with the host system and the diskless target (either USB key or floppy disk).
  3. The SUSE Linux Enterprise Server 9-SP3 install CDs.
  4. linux-iscsi files (this requirement will be satisfied by Section I)
    1. iscsi.ko
    2. iscsid
    3. initiatorname.iscsi
    4. iscsi.conf

Section I : Creating the Linux-iSCSI driver

This is the recommended method for creating the driver disk.

  1. On the system running SUSE Linux Enterprise Server 9-SP3, insert and mount your media to be used as the driver disk.
  2. YaST -> Add/Remove Software.
  3. Search "kernel"
  4. Select "kernel-default" and install the package.
  5. Search "iscsi"
  6. Select "linux-iscsi" and install the package.
  7. Copy the following files to your mounted media
    1. /lib/modules/2.6.5-7.244-default/extra/iscsi.ko
    2. /sbin/iscsid
    3. /etc/initiatorname.iscsi
    4. /etc/iscsi.conf
NOTES: Configure the initiatorname.iscsi and iscsi.conf files according to your Target configuration. To do so you will need to read the README provided by the linux-iscsi source code. You must at least configure the DiscoveryAddress in the iscsi.conf.

If your target supports advanced features such as CHAP, you will need to make sure that the iscsi.conf and initiatorname.iscsi match your target configuration at this point. These files will be used throughout the install and boot process.

Alternatively you may create the driver disk using a different host machine other than SUSE Linux Enterprise Server 9-SP3. You will need to compile the iscsi.ko and iscsid using the 2.6.5-7.244-default kernel. You can learn to do this by reading the README provided with the linux-iscsi source code, which is available on the Internet.

Section II : Installing to the iSCSI target system

NOTE: You need an Intel® iSCSI Boot capable adapter to connect to your target and boot the OS from the iSCSI Target Disk.

Requirements:

  1. Diskless host compatible with the driver disk.
  2. The driver disk created in Section I.
  3. The full set of CDs for both SUSE Linux Enterprise Server 9 and SUSE Linux Enterprise Server 9-SP3.

Procedure:

  1. Boot from SUSE Linux Enterprise Server 9-SP3 installation CD1.
    1. Select "installation", but before pressing "enter", please enter the following boot option.

      netsetup=1

  2. When it again asks for CD1 insert SUSE Linux Enterprise Server 9 CD1.
  3. When you are prompted to accept the license agreement, press Ctrl-Alt-F2 to switch to the console
    1. mount the driver disk
    2. copy the following files to /etc
      1. initiatorname.iscsi
      2. iscsi.conf
    3. copy the following files to /tmp
      1. iscsi.ko
      2. iscsid
    4. unmount your driver disk and remove the driver disk
    5. cd /tmp
    6. insmod iscsi.ko
    7. execute iscsid
  4. Verify that you are connected to the target

Ctrl-Alt-F4 will display something along the lines of

"iSCSI target bus 0 target 0 = iqn.XYZ"

"Attached scsi disk ..."

  1. Press Alt-F7 or Ctrl+Alt+F7 to go back and continue installation. Accept the license agreement and continue until you reach the "Installation Settings" Dialog.
  2. Select the Software packages to install
    1. Click "Software".
    2. Choose the software you wish to install.
    3. Select "Detailed selection". Change the value of the "Filter" drop-down box to "search".
    4. Search for "iSCSI".
    5. Make sure the "linux-iscsi" checkbox is selected.
    6. Make sure the "Autocheck" checkbox is selected.
    7. Click "accept" and "continue" if prompted.
  3. Change some of the Booting options
    1. Click "Booting"
    2. Select "Disk Order"
    3. Ensure that the disk that you are installing the OS on is the top disk in the list.
  4. When the installation completes, the user is warned that a reboot will occur, it is accompanied by a 10 second countdown. Press "Stop" to stop the system reboot.
  5. Press Ctrl-Alt-F2 to go to console
    1. cp -fp /etc/initiatorname.iscsi /etc/iscsi.conf /mnt/etc/
    2. Run "chroot /mnt"
    3. Edit line 10 of the file /etc/sysconfig/kernel. Append "e1000" & "iscsi" driver to INITRD_MODULES variable.

      Example: INITRD_MODULES="... e1000 iscsi".

    4. Make the following changes to /sbin/mkinitrd

      Add to line 1197 (after the "elif test -x $root_dir/sbin/iscsid $tmp_mnt/sbin/iscsid")

      cp -fp $root_dir/etc/iscsi.conf $root_dir/etc/initiatorname.iscsi $tmp_mnt/etc/

      Add the following to line 1830 (before the line "if has_module iscsi ; then")

      echo "iscsi_boot=1" >> $linuxrc

      Revise the "if has_module iscsi ; then" statement at line 1899. It should look like this:

      if has_module iscsi ; then

      cat_linuxrc <<-'EOF'

      | echo "Starting iSCSI"

      | iscsid

      | usleep 5000000

      |

      | # clear dhcp_mode so it looks like we're disk booted...

      | dhcp_mode=

      EOF

      fi

    5. mkinitrd -D ethX
    6. Press Ctrl-D
  6. Press Alt-F7 (Alt-F1 if using text mode installation) to return to installation screen.
    1. Select "OK" to reboot.
    2. Remove the install CD.
  7. After the reboot iscsi-boot you will see YaST prompting you for various configuration steps. Please skip the network-configurations and continue with the rest of the configurations.
  8. When you have iscsi-booted your system please edit /etc/rc.d/network. Add the following line to the script to prevent it from trying to reset our iscsi-ethernet interface. Please add carefully

    test "$a" = ethX && continue;

    immediately after line number 169 where ethX is the ethernet interface used for iSCSI communication.
    NOTE: After the system has booted iscsid will be running. The user may want to add something like "kill `pidof iscsid`" to an init script. iscsid is only needed for discovery and can be killed in this way without problems.

Red Hat Enterprise Linux 4 Installation:

This process has three phases. First, before installation, you will need to create a DUD (Driver Update Disk). This disk will be used during the install process. Second, you will need to install Red Hat Enterprise Linux 4 U3 on the iSCSI Target using the normal installation process, but with some extra steps inserted. Third you will need to modify the initrd.img file created by the installation process. Once each of these phases has been completed the user will be able to boot from the newly installed Red Hat Enterprise Linux 4 U3 iSCSI Target.

Prerequisites

  1. A computer with Red Hat Enterprise Linux 4 U3 installed on it as described in the following section. This system will be referred to as the build machine.
    1. The system will be used to create a Driver Update Disk (DUD) and will need to have a 3.5' floppy drive or a USB 3.5' floppy drive.
  2. iSCSI initiator source.
  3. Source code for any dependency modules. iscsi initiator may depend on any other kernel modules.
  4. The iscsiboot.tar from the Intel provided media.
  5. A blank 3.5' disk.
NOTE: The system used to connect to the iSCSI target durring the install process must not have any local disks installed. It should also not have any USB storage devices attached when the install starts. The intention is to get anaconda (the Red Hat installer) to see no disks in the system and stop. This will allow us to install the initiator and discover the iSCSI LUs. If there is any media installed anaconda will continue and attempt to install to the locally connected device.

Phase I - Creating Driver Update Diskette

  1. Install Red Hat Enterprise Linux 4 U3 on your build machine. To avoid dependency problems "Customize software packages to be installed" and select "Everything" at the bottom of the "Package Group Selection".
  2. Install the dkms-2.0.13-1.noarch.rpm (http://linux.dell.com/dkms/) on the build machine if it did not come with your distribution.
  3. Install the following packages.
    1. Use the following table to install required files. Execute the "COMMAND" on the specified file at the "LOCATION" and confirm that the "OUTPUT" was created correctly.

       

      COMMAND                     LOCATION OUTPUT
      rpm -ivh iscsi-initiator-utils-4.0.3.0-3.src.rpm Red Hat Enterprise Linux 4 U3 Source CD3 /usr/src/redhat/SPECS/iscsi-initiator-utils.spec
      rpm -ivh kernel-2.6.9-34.EL.src.rpm Red Hat Enterprise Linux 4 U3 Source CD4 /usr/src/redhat/SPECS/kernel-2.6.spec
      rpm -ivh kernel-devel-2.6.9-34.EL.i686.rpm Red Hat Enterprise Linux 4 U3 CD3 /usr/src/kernels/2.6.9-34.EL-i686/
      rpm -ivh kernel-smp-devel-2.6.9-34.EL.i686.rpm Red Hat Enterprise Linux 4 U3 CD3 /usr/src/kernels/2.6.9-34.EL-smp-i686/
      rpm -ivh kernel-hugemem-devel-2.6.9-34.EL.i686.rpm Red Hat Enterprise Linux 4 U3 CD3 /usr/src/kernels/2.6.9-34.EL-hugemem-i686
      rpmbuild -bp --target `uname -m` kernel-2.6.spec /usr/src/redhat/SPECS /usr/src/redhat/BUILD/kernel-2.6.9/
      rpmbuild -bp --target `uname -m` iscsi-initiator-utils.spec /usr/src/redhat/SPECS /usr/src/redhat/BUILD/iscsi-initiator-utils-4.0.3.0

       

  1. Copy and untar iscsiboot.tar.
  1. cp iscsiboot.tar /home
  2. cd /home
  3. tar -xvf iscsiboot.tar
  1. Insert the formatted blank diskette into the build machine.
  2. Run the make_disk_image script (run without any flags for usage).
    1. cd /home/iscsiboot/
    2. ./make_disk_image -k /usr/src/redhat/BUILD/kernel-2.6.9/linux-2.6.9 -i

/usr/src/redhat/BUILD/iscsi-initiator-utils-4.0.3.0/linux-iscsi-4.0.3.0

-v 2.6.9-34.EL

This process uses dd to write the Driver Update Disk you may specify the device node with the -t option or the script will prompt you. Keep in mind that you must use the device node and not a mounted partition.

Phase II - Installation to iSCSI disk

NOTE: This assumes you have flashed and configured your iSCSI firmware to connect to your target and iscsi firmware was able to connect to the iscsi target.
  1. Begin installation. Remember no local storage devices can be connected at this point, including USB floppy devices.
  2. Install iSCSI Driver
    1. If the system that you're using to install Red Hat Enterprise Linux 4 U3 is diskless, you will be asked if you'd like to configure the disks.
    2. On the "Devices" dialog select "Add Device".
    3. If your driver disk uses the USB interface you will need to choose the "USB Mass Storage driver for Linux (usb-storage)".
    4. Insert your driver disk.
    5. Return to the "Select Device Driver to Load", press F2 and follow the dialogs to install the Cisco Driver.
    6. The "Select Device Driver to Load" dialog should reappear. Select "Cisco Systems iSCSI initiator (iscsi_sfnet)" and press "OK".
    7. DO NOT PRESS "DONE".
  3. Discover the target disks
    1. Press Ctrl+Alt+f2 to switch to command line mode.
    2. Make a mount point directory and mount the driver disk. (The mount point can be anything but /mnt)

Floppy

mkdir /mnt/floppy

mount /dev/fd0 /mnt/floppy.

USB Floppy Drive

dmesg

Examine the output of dmesg and determine what /dev node is reserved for your USB drive (i.e. /dev/sdX)

mkdir /mnt/usb

mknod /dev/sdX b 8 0

mount /dev/sdX /mnt/usb

  1. Copy files to a temporary location

mkdir /tmp/iscsi

cp -fr /mnt/usb/* /tmp/iscsi/

umount /mnt/usb

  1. Run the "preinstall/iscsipreinstall" script from the mount point.

/mnt/usb/preinstall/iscsipreinstall

  1. Type "dmesg" to verify that discovery occurred correctly. It should show that drives were recently discovered.
  2. Press Ctrl+Alt+F1 and select "Done".
  3. Continue with the installation.
NOTE: When partitioning the iSCSI LUs ensure that your boot partition is not distributed across physical partitions on different iSCSI LUs. The Intel iSCSI Option ROM will only expose one iSCSI LU and if /boot is distributed across multiple LUs it will not be visible durring the boot process.
  1. Configure the boot order
    1. On the "Boot Loader Configuration" select "Configure advanced boot loader options"
    2. Change the drive order so that your target disk is on the top of the list.
    3. Continue with remainder of installation.
    4. When the install is complete DO NOT press "Reboot".
  2. Make some final post install changes
    1. Instead of pressing "Reboot" press Ctrl-Alt-F2 to switch back to the console.
    2. Run the post install script and unmount your driver disk.

/tmp/iscsi/postinstall/iscsipostinstall

  1. Ensure the network is not shutdown durring halt
  1. Edit the /mnt/sysimage/etc/init.d/halt and scroll to the bottom.
  2. Change the line stating HALTARGS="-i -d" to HALTARGS="-d"
  1. Return to the GUI and reboot
  1. Press Ctl+Alt+F7 to return to installation screen and select "Reboot".
  1. On your first boot if Kudzu asks you if you'd like to configure your NICs please select to "ignore" the device.
NOTE: After the system has booted iscsid will be running. The user may want to add something like "kill `pidof iscsid`" to an init script. iscsid is only needed for discovery and can be killed in this way without problems.