Search This Blog

There was an error in this gadget

Saturday, May 3, 2008

A poor man's guide for creating iscsi targets without using external USB hard disks

A special need..


Those of you who have a need for exposing iscsi targets to other machines for discovery, but who do not want to invest in an external USB hard disk OR who do not want to have a USB external hard disk connected to a desktop 24x7, there is a better way of creating and exposing iscsi target logical volumes.

This method does not need installing openfiler either as noted in my earlier article Combining Openfiler and Virtualbox (Ubuntu guest OS on windows host).

A simple solution..


The method is simple: create logical volumes based on SCSI devices on a Ubuntu Hardy Heron 8.04 installation using virtualbox.
The advantage of using Ubuntu is that it creates the hard disk devices as scsi devices (with the naming convention /dev/sd*, instead of /dev/hd*). I have seen that in other unix OS like SuSE linux, the local disks are listed as /dev/hd* (IDE). Ubuntu seems to have better disk drivers.

If you use Hardy Heron 8.04 Ubuntu, it is *really* easy to get iscsitarget package working. If you try to make the iscsi-target package work with Gutsy Gibbon 7.x release of Ubuntu, there is a very good chance that you will run into lot of compilation issues. I went down this path myself and later realized that a lot of bugs existed for Gutsy Gibbon release. (e.g. https://bugs.launchpad.net/ubuntu/gutsy/+source/iscsitarget/+bug/160104, http://ubuntuforums.org/showthread.php?t=692651) and eventually found https://bugs.launchpad.net/ubuntu/+source/iscsitarget/+bug/145539 which said that the module was fixed in Hardy Heron 8.04.

So instead of breaking my head over making iscsi-target package work in Gutsy Gibbon 2.6.22, I decided to give Hardy Heron 8.04 (still beta) a try.

Thankfully, with a little effort, I was able to make it work. In this article, I present a simplistic scenario in which we create three logical volumes that can be easily discovered by another virtualbox iscsi initiator machine(s) using open-iscsi package. This is the beauty of the entire approach.

Here is a block diagram of the end configuration:

Click here for a bigger view


Iscsi support in Hardy Heron 8.04 Ubuntu..


The first thing to understand is that as per the published features of Hardy Heron 8.04 Ubuntu at https://wiki.ubuntu.com/HardyHeron/Beta#head-da07b62e1e43afd0bef06ab8b60d2502c734a0f9, the iscsi support is enabled out of the box if we add iscsi=true in the boot options during the installation.

Click here for a bigger view



While installing Hardy 8.04 OS using virtualbox or any other virtualization software being used, remember to add iscsi=true in the boot options (after pressing F6)

Click here for a bigger view


Configuring iscsi targets..


I would like to give credit to a really awesome link I found through google: http://www.linuxconfig.org/Linux_lvm_-_Logical_Volume_Manager, which I rate as a one of the best article s for doing this!

Some other relevant links are:

https://help.ubuntu.com/community/SettingUpLVM-WithoutACleanInstall
http://t3flyers.wordpress.com/2007/04/24/logical-volume-manager-on-ubuntu-feisty-704/
http://www.howtoforge.com/linux_lvm

After Ubuntu installation, the output of uname -a looks like this:
HARDY# uname -a
Linux gverma-laptop 2.6.24-16-generic #1 SMP Thu Apr 10 13:23:42 UTC 2008 i686 GNU/Linux

And the output of fdisk -l looks like this:
HARDY# fdisk -l

Disk /dev/sda: 16.5 GB, 16592666624 bytes
255 heads, 63 sectors/track, 2017 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x0004ccc2
Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1        1927    15478596   83  Linux
/dev/sda2            1928        2017      722925    5  Extended
/dev/sda5            1928        2017      722893+  82  Linux swap / Solaris

At this point, I added two more hard disks to the virtualbox virtual machine. Virtualbox 1.5.6 allows upto three hard disk devices to be attached to a machine.

Click here for a bigger view



After booting up HARDY again, /dev/sdb and /dev/sdc were also visible in the output of fdisk -l. Using fdisk, I created a single partition in both devices that spanned the entire device, resulting in /dev/sdb1 and /dev/sdc1:
HARDY# fdisk -l

...
Disk /dev/sdb: 21.4 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0xd310045f

Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1        2610    20964793+  83  Linux

Disk /dev/sdc: 21.4 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x93d4c692

Device Boot      Start         End      Blocks   Id  System
/dev/sdc1               1        2610    20964793+  83  Linux

Also make sure that you configure host only networking so that other virtual machines (scsi initiators) are able to ssh/telnet into HARDY. Please look at article to understand how host only networking can be setup: Virtualbox Case Study: Making host only networking work between two Ubuntu Guest OS (virtual machine) on Windows Vista host



Getting the right packages installed..


Now, we need to create the target physical volume, virtual group, and logical volume, in that order. These logical volumes will serve as SCSI targets that can be discovered by other iscsi initiator machines or virtual machines as per your configuration.

Make sure you reload the package definitions from the configured repositories (get the latest packages that are published from the ubuntu repositories). You can do this from the Synaptic Package Manager:

Click here for a bigger view



Now, you should install the lvm2 package in Synaptic Package Manager:

Click here for a bigger view


Create the physical volumes:


HARDY# lvm

lvm> pvcreate /dev/sdb1
Physical volume "/dev/sdb1" successfully created
lvm> pvcreate /dev/sdc1
Physical volume "/dev/sdc1" successfully created
lvm> pvdisplay
--- NEW Physical volume ---
PV Name               /dev/sdb1
VG Name
PV Size               19.99 GB
Allocatable           NO
PE Size (KByte)       0
Total PE              0
Free PE               0
Allocated PE          0
PV UUID               OHKhOq-AMG7-XtYz-7CrP-q2VH-2b53-tW0yMO

--- NEW Physical volume ---
PV Name               /dev/sdc1
VG Name
PV Size               19.99 GB
Allocatable           NO
PE Size (KByte)       0
Total PE              0
Free PE               0
Allocated PE          0
PV UUID               M9qXXB-GVxC-FZjI-9Zb7-IhcH-rbhb-vQ3aek

Create the virtual group:


lvm> vgcreate vg /dev/sdb1 /dev/sdc1
Volume group "vg" successfully created
lvm> vgdisplay
--- Volume group ---
VG Name               vg
System ID
Format                lvm2
Metadata Areas        2
Metadata Sequence No  1
VG Access             read/write
VG Status             resizable
MAX LV                0
Cur LV                0
Open LV               0
Max PV                0
Cur PV                2
Act PV                2
VG Size               39.98 GB
PE Size               4.00 MB
Total PE              10236
Alloc PE / Size       0 / 0
Free  PE / Size       10236 / 39.98 GB
VG UUID               GkUMNq-3atR-qKTK-lG0b-gM0n-budV-Uc4lH8

lvm>

Create logial volumes:


lvm> lvcreate -L 1G -n ocr vg
Logical volume "ocr" created
lvm> lvcreate -L 1G -n vote vg
Logical volume "vote" created
lvm> lvcreate -L 35G -n asm vg
Logical volume "asm" created

If you get an error message saying:
/proc/misc: No entry for device-mapper
you can get around it by issuing sudo modprobe dm-mod

lvm> lvdisplay
--- Logical volume ---
LV Name                /dev/vg/ocr
VG Name                vg
LV UUID                a2ARIQ-11dn-kdoB-cHCd-gtoR-aokO-HGqoo4
LV Write Access        read/write
LV Status              available
# open                 0
LV Size                1.00 GB
Current LE             256
Segments               1
Allocation             inherit
Read ahead sectors     0
Block device           254:0

--- Logical volume ---
LV Name                /dev/vg/vote
VG Name                vg
LV UUID                1DqYL0-Ptsx-9kmE-UHmn-PmE9-ebVm-bfr0r0
LV Write Access        read/write
LV Status              available
# open                 0
LV Size                1.00 GB
Current LE             256
Segments               1
Allocation             inherit
Read ahead sectors     0
Block device           254:1

--- Logical volume ---
LV Name                /dev/vg/asm
VG Name                vg
LV UUID                UKWcPz-aNGl-VdPm-51wa-Ewvm-ahvD-Y7N57v
LV Write Access        read/write
LV Status              available
# open                 0
LV Size                35.00 GB
Current LE             8960
Segments               2
Allocation             inherit
Read ahead sectors     0
Block device           254:2

lvm>

Now, you can check the logical volume devices like this:
HARDY# ls -l /dev/vg
total 0
lrwxrwxrwx 1 root root 18 2008-05-03 02:42 asm -> /dev/mapper/vg-asm
lrwxrwxrwx 1 root root 18 2008-05-03 02:41 ocr -> /dev/mapper/vg-ocr
lrwxrwxrwx 1 root root 19 2008-05-03 02:41 vote -> /dev/mapper/vg-vote

You can also check the newly created devices in the /dev/disk/* directories on HARDY:
HARDY# ls -l /dev/disk/*
/dev/disk/by-id:
..
lrwxrwxrwx 1 root root 19 2008-05-03 02:42 dm-name-vg-asm -> ../../mapper/vg-asm
lrwxrwxrwx 1 root root 19 2008-05-03 02:41 dm-name-vg-ocr -> ../../mapper/vg-ocr
lrwxrwxrwx 1 root root 20 2008-05-03 02:41 dm-name-vg-vote -> ../../mapper/vg-vote
..

/dev/disk/by-path:
total 0
lrwxrwxrwx 1 root root  9 2008-05-02 22:30 pci-0000:00:01.1-scsi-0:0:1:0 -> ../../sdb
lrwxrwxrwx 1 root root 10 2008-05-03 02:34 pci-0000:00:01.1-scsi-0:0:1:0-part1 -> ../../sdb1
lrwxrwxrwx 1 root root  9 2008-05-02 22:30 pci-0000:00:01.1-scsi-1:0:1:0 -> ../../sdc
lrwxrwxrwx 1 root root 10 2008-05-03 02:35 pci-0000:00:01.1-scsi-1:0:1:0-part1 -> ../../sdc1

You need to install the iscsitarget package using Synaptic Package Manager:

Click here for a bigger view



Now, you need to configure the /etc/ietd.conf file with the Target names and entries for each logical volume. The target names just need to unique within the network and are totally upto your imagination.
HARDY# more /etc/ietd.conf
Target iqn.2001-04.com.ubuntu:scsi.disk.vg.vote
Lun 0 Path=/dev/vg/vote,Type=fileio
Target iqn.2001-04.com.ubuntu:scsi.disk.vg.ocr
Lun 0 Path=/dev/vg/ocr,Type=fileio
Target iqn.2001-04.com.ubuntu:scsi.disk.vg.asm
Lun 0 Path=/dev/vg/asm,Type=fileio

To enforce these entries, we need to restart the iscsitarget service manually now.
Keep in mind that there is no dash (-) between iscsi and target in the name of the service, as in openfiler 2.2 (openfiler's IET implementation service has the name iscsi-target)

HARDY# /etc/init.d/iscsitarget restart
Removing iSCSI enterprise target devices: succeeded.
Stopping iSCSI enterprise target service: succeeded.
Removing iSCSI enterprise target modules: succeeded.
Starting iSCSI enterprise target service: succeeded.

To double check if the logical volumes have been discovered and published, you can check content of /proc/net/iet/volume on HARDY:
::::::::::::::
/proc/net/iet/volume
::::::::::::::
tid:3 name:iqn.2001-04.com.ubuntu:scsi.disk.vg.asm
lun:0 state:0 iotype:fileio iomode:wt path:/dev/vg/asm
tid:2 name:iqn.2001-04.com.ubuntu:scsi.disk.vg.ocr
lun:0 state:0 iotype:fileio iomode:wt path:/dev/vg/ocr
tid:1 name:iqn.2001-04.com.ubuntu:scsi.disk.vg.vote
lun:0 state:0 iotype:fileio iomode:wt path:/dev/vg/vote

If you do a more of /proc/net/iet/session, you can clearly see that no iscsi-initiator machine have connected to HARDY yet (there are no session sub-entries under the volume name):
HARDY# more /proc/net/iet/session
tid:3 name:iqn.2001-04.com.ubuntu:scsi.disk.vg.asm
tid:2 name:iqn.2001-04.com.ubuntu:scsi.disk.vg.ocr
tid:1 name:iqn.2001-04.com.ubuntu:scsi.disk.vg.vote

Here is a quick check of the physical volume, volume group, and logical volume :

HARDY# lvm
lvm> lvscan
ACTIVE '/dev/vg/ocr' [1.00 GB] inherit
ACTIVE '/dev/vg/vote' [1.00 GB] inherit
ACTIVE '/dev/vg/asm' [35.00 GB] inherit

lvm> pvscan
PV /dev/sdb1 VG vg lvm2 [19.99 GB / 0 free]
PV /dev/sdc1 VG vg lvm2 [19.99 GB / 2.98 GB free]
Total: 2 [39.98 GB] / in use: 2 [39.98 GB] / in no VG: 0 [0 ]

lvm> vgscan
Reading all physical volumes. This may take a while...
Found volume group "vg" using metadata type lvm2
lvm>

Discovering the iscsi targets from another iscsi initiator machine..


We will assume that the iscsi initiator machine name is GUTSY and it has the open-iscsi package installed (this also installs iscsiadm utility along with it). We will also assume that the IP of HARDY (iscsi target) is 192.168.0.7.
You can leave the majority default values in /etc/iscsi.conf. Make sure you disable the CHAP authentication parameter values since we have not configured them in the scsi target.

Discover the new iscsi targets in HARDY:
GUTSY# iscsiadm -m discovery -t st -p 192.168.0.7
192.168.0.7:3260,1 iqn.2001-04.com.ubuntu:scsi.disk.vg.asm
192.168.0.7:3260,1 iqn.2001-04.com.ubuntu:scsi.disk.vg.ocr
192.168.0.7:3260,1 iqn.2001-04.com.ubuntu:scsi.disk.vg.vote

Verify the newly discovered target machine:
GUTSY# sudo iscsiadm -m discovery
192.168.0.6:3260 via sendtargets
192.168.0.7:3260 via sendtargets  --> the new iscsi target got added to the local database

Check the combination of discovered iscsi-target and logical volumes (the new ones are in BLUE color):
GUTSY# sudo iscsiadm -m node
192.168.0.6:3260,1 iqn.2006-01.com.openfiler:openfiler.ocr
192.168.0.6:3260,1 iqn.2006-01.com.openfiler:openfiler.vote
192.168.0.6:3260,1 iqn.2006-01.com.openfiler:openfiler.asm
192.168.0.7:3260,1 iqn.2001-04.com.ubuntu:scsi.disk.vg.asm
192.168.0.7:3260,1 iqn.2001-04.com.ubuntu:scsi.disk.vg.ocr
192.168.0.7:3260,1 iqn.2001-04.com.ubuntu:scsi.disk.vg.vote

But, at this point, we have no active sessions connected to these targets:
GUTSY# iscsiadm -m session
iscsiadm: No active sessions.

Establish connection to the scsi targets from the initiator:
GUTSY# iscsiadm -m node \
-T iqn.2001-04.com.ubuntu:scsi.disk.vg.asm \
-p 192.168.0.7 -l
Login session [iface: default, target: iqn.2001-04.com.ubuntu:scsi.disk.vg.asm, portal: 192.168.0.7,3260]
GUTSY# iscsiadm -m node \
-T iqn.2001-04.com.ubuntu:scsi.disk.vg.vote \
-p 192.168.0.7 -l
Login session [iface: default, target: iqn.2001-04.com.ubuntu:scsi.disk.vg.vote, portal: 192.168.0.7,3260]
GUTSY# iscsiadm -m node \
-T iqn.2001-04.com.ubuntu:scsi.disk.vg.ocr \
-p 192.168.0.7 -l
Login session [iface: default, target: iqn.2001-04.com.ubuntu:scsi.disk.vg.ocr, portal: 192.168.0.7,3260]

Verify the newly formed connections:
GUTSY# iscsiadm -m session
tcp: [1] 192.168.0.7:3260,1 iqn.2001-04.com.ubuntu:scsi.disk.vg.asm
tcp: [2] 192.168.0.7:3260,1 iqn.2001-04.com.ubuntu:scsi.disk.vg.vote
tcp: [3] 192.168.0.7:3260,1 iqn.2001-04.com.ubuntu:scsi.disk.vg.ocr

It would be useful to enable automatic startup/discovery of these volumes on server and client. For this, you need to do the following:
  • On server(where the logical volumes are created):

HARDY# sudo gedit /etc/iscsi/iscsid.conf

Change node.startup=manual to: node.startup = Automatic
GUTSY# sudo /etc/init.d/open-iscsi restart

  • On client

GUTSY# sudo iscsiadm -m discovery -t st -p <ipaddress>
GUTSY# sudo iscsiadm -m node
-T iqn.2001-04.com.ubuntu:scsi.disk.vg.asm
-p <ipaddress> -o update
-n node.conn[0].startup -v automatic
GUTSY# sudo /etc/init.d/open-iscsi restart

  • or reboot both the systems if required (make sure server boots first). To check if the scsi devices are automatically setup and list all your scsi iqns:

GUTSY # sudo iscsiadm -m session

Now assumed that the TCP scsi connections have been formed, let us check if the scsi devices are visible from fdisk -l:
GUTSY# fdisk -l

..
Disk /dev/sdb: 37.5 GB, 37580963840 bytes
64 heads, 32 sectors/track, 35840 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Disk identifier: 0x00000000

Disk /dev/sdb doesn't contain a valid partition table

Disk /dev/sdc: 1073 MB, 1073741824 bytes
34 heads, 61 sectors/track, 1011 cylinders
Units = cylinders of 2074 * 512 = 1061888 bytes
Disk identifier: 0x00000000

Disk /dev/sdc doesn't contain a valid partition table

Disk /dev/sdd: 1073 MB, 1073741824 bytes
34 heads, 61 sectors/track, 1011 cylinders
Units = cylinders of 2074 * 512 = 1061888 bytes
Disk identifier: 0x00000000

Disk /dev/sdd doesn't contain a valid partition table
GUTSY#

The newly discovered devices can also be checked under the contents of /dev/disk/*:
GUTSY# ls -l /dev/disk/*
/dev/disk/by-id:
total 0
..
lrwxrwxrwx 1 root root  9 2008-05-02 23:05 scsi-149455400000000000000000001000000795600000e000000 -> ../../sdc
lrwxrwxrwx 1 root root  9 2008-05-02 23:05 scsi-1494554000000000000000000020000004f5600000e000000 -> ../../sdd
lrwxrwxrwx 1 root root  9 2008-05-02 23:05 scsi-149455400000000000000000003000000a45600000e000000 -> ../../sdb
..

/dev/disk/by-path:
total 0
lrwxrwxrwx 1 root root  9 2008-05-02 23:05 ip-192.168.0.7:3260-iscsi-iqn.2001-04.com.ubuntu:scsi.disk.vg.asm-lun-0 -> ../../sdb
lrwxrwxrwx 1 root root  9 2008-05-02 23:05 ip-192.168.0.7:3260-iscsi-iqn.2001-04.com.ubuntu:scsi.disk.vg.ocr-lun-0 -> ../../sdd
lrwxrwxrwx 1 root root  9 2008-05-02 23:05 ip-192.168.0.7:3260-iscsi-iqn.2001-04.com.ubuntu:scsi.disk.vg.vote-lun-0 -> ../../sdc
..

Relevant messages in the system log of scsi initiator (GUTSY)


# dmesg
...
[ 1156.290868] scsi2 : iSCSI Initiator over TCP/IP
[ 1156.607026] scsi 2:0:0:0: Direct-Access     IET      VIRTUAL-DISK     0    PQ: 0 ANSI: 4
[ 1156.607026] sd 2:0:0:0: [sdb] 73400320 512-byte hardware sectors (37581 MB)
[ 1156.607026] sd 2:0:0:0: [sdb] Write Protect is off
[ 1156.607026] sd 2:0:0:0: [sdb] Mode Sense: 77 00 00 08
[ 1156.607026] sd 2:0:0:0: [sdb] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[ 1156.607111] sd 2:0:0:0: [sdb] 73400320 512-byte hardware sectors (37581 MB)
[ 1156.608275] sd 2:0:0:0: [sdb] Write Protect is off
[ 1156.608293] sd 2:0:0:0: [sdb] Mode Sense: 77 00 00 08
[ 1156.610635] sd 2:0:0:0: [sdb] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[ 1156.610665]  sdb: unknown partition table
[ 1156.619320] sd 2:0:0:0: [sdb] Attached SCSI disk
[ 1156.619398] sd 2:0:0:0: Attached scsi generic sg2 type 0
[ 1163.865083] scsi3 : iSCSI Initiator over TCP/IP
[ 1164.132972] scsi 3:0:0:0: Direct-Access     IET      VIRTUAL-DISK     0    PQ: 0 ANSI: 4
[ 1164.135287] sd 3:0:0:0: [sdc] 2097152 512-byte hardware sectors (1074 MB)
[ 1164.136424] sd 3:0:0:0: [sdc] Write Protect is off
[ 1164.136438] sd 3:0:0:0: [sdc] Mode Sense: 77 00 00 08
[ 1164.138002] sd 3:0:0:0: [sdc] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[ 1164.140341] sd 3:0:0:0: [sdc] 2097152 512-byte hardware sectors (1074 MB)
[ 1164.141443] sd 3:0:0:0: [sdc] Write Protect is off
[ 1164.141460] sd 3:0:0:0: [sdc] Mode Sense: 77 00 00 08
[ 1164.144746] sd 3:0:0:0: [sdc] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[ 1164.144809]  sdc: unknown partition table
[ 1164.154241] sd 3:0:0:0: [sdc] Attached SCSI disk
[ 1164.154290] sd 3:0:0:0: Attached scsi generic sg3 type 0
[ 1170.533158] scsi4 : iSCSI Initiator over TCP/IP
[ 1170.798343] scsi 4:0:0:0: Direct-Access     IET      VIRTUAL-DISK     0    PQ: 0 ANSI: 4
[ 1170.798343] sd 4:0:0:0: [sdd] 2097152 512-byte hardware sectors (1074 MB)
[ 1170.801229] sd 4:0:0:0: [sdd] Write Protect is off
[ 1170.801263] sd 4:0:0:0: [sdd] Mode Sense: 77 00 00 08
[ 1170.806093] sd 4:0:0:0: [sdd] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[ 1170.809875] sd 4:0:0:0: [sdd] 2097152 512-byte hardware sectors (1074 MB)
[ 1170.812347] sd 4:0:0:0: [sdd] Write Protect is off
[ 1170.812380] sd 4:0:0:0: [sdd] Mode Sense: 77 00 00 08
[ 1170.816757] sd 4:0:0:0: [sdd] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[ 1170.816757]  sdd: unknown partition table
[ 1170.828375] sd 4:0:0:0: [sdd] Attached SCSI disk
[ 1170.828446] sd 4:0:0:0: Attached scsi generic sg4 type 0

Cross checking incoming sessions at scsi target (HARDY)..


In the meanwhile, in HARDY, the incoming sessions can be checked by:
HARDY# more /proc/net/iet/*
::::::::::::::
/proc/net/iet/session
::::::::::::::
tid:3 name:iqn.2001-04.com.ubuntu:scsi.disk.vg.asm
sid:281474997486080 initiator:iqn.1993-08.org.debian:01:950a218cdd1
cid:0 ip:192.168.0.6 state:active hd:none dd:none
tid:2 name:iqn.2001-04.com.ubuntu:scsi.disk.vg.ocr
sid:844424984461824 initiator:iqn.1993-08.org.debian:01:950a218cdd1
cid:0 ip:192.168.0.6 state:active hd:none dd:none
tid:1 name:iqn.2001-04.com.ubuntu:scsi.disk.vg.vote
sid:562949990973952 initiator:iqn.1993-08.org.debian:01:950a218cdd1
cid:0 ip:192.168.0.6 state:active hd:none dd:none
::::::::::::::
/proc/net/iet/volume
::::::::::::::
tid:3 name:iqn.2001-04.com.ubuntu:scsi.disk.vg.asm
lun:0 state:0 iotype:fileio iomode:wt path:/dev/vg/asm
tid:2 name:iqn.2001-04.com.ubuntu:scsi.disk.vg.ocr
lun:0 state:0 iotype:fileio iomode:wt path:/dev/vg/ocr
tid:1 name:iqn.2001-04.com.ubuntu:scsi.disk.vg.vote
lun:0 state:0 iotype:fileio iomode:wt path:/dev/vg/vote
root@gverma-laptop:/home/gverma#

Conclusion..


Congratulations! You have been successful in creating a poor man's scsi target and discovered them using open source softwares in an easy manner. Now GUTSY can read/write to /dev/vg/asm etc logical volumes. For more machines to do the same thing, similar setup is needed on them and they should also be access these volumes in a clustered fashion. This kind of setup is most beneficial for creating your own Oracle 10g RAC sandbox environment or for doing installs over shared disk.
In this case, we did not format the discovered logical volumes in ext3 or OCFS2 file system, because it is assumed that these volumes will be used as raw devices only.


Appendix


Content contributed by Sundar Mahadevan, who tenaciously resolved weird scsi target issues and graciously published this for the benefit of similar seekers):


The following helps newbies:

For iscsi to be implemented, you need to have iscsitarget installed onthe machine from where the logical volumes are going to be created and you need to have open-iscsi (initiator) installed on the machine that has to access the already created logical volumes.

When I tried this setup, I was getting some really weird errors. In this appendix, I outline my experience and offer the solution that ultimately worked for me. The intent of this sharing is to prevent anxiety for other people who may be in similar situations (I remember it was quite frustrating for me).

System1 : ubuntu 8.10
created /dev/vg1/asm, /dev/vg1/ocr and /dev/vg1/vote on the secondary
hard drive with lvm2
has iscsitarget and iscsitarget-source installed

System2: ubuntu 8.10
has open-iscsi installed

Connection: static (crossover cable between the 2 systems)

Error: Only one logical volume of the entire size of /dev/vg1 was discovered by the initiator! I should have got three disks like /dev/sda, sdb, sdc , but instead I got /dev/sdd or /dev/sdc.

Then i tried to make my connection as dhcp (with router connecting the 2 systems)

Error:
Apr 17 11:40:42 sunny1 iscsid: session
[iqn.2009-09.com.ubuntu:asm,192.168.2.3,3260] already running.
Apr 17 11:40:44 sunny1 kernel: [ 5661.360412] sd 2:0:0:0: [sdb]
Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK,SUGGEST_OK
Apr 17 11:40:44 sunny1 kernel: [ 5661.360436] end_request: I/O error,
dev sdb, sector 0
Apr 17 11:40:44 sunny1 kernel: [ 5661.360449] __ratelimit: 2 callbacks
suppressed
Apr 17 11:40:44 sunny1 kernel: [ 5661.360457] Buffer I/O error on
device sdb, logical block 0
Apr 17 11:40:44 sunny1 kernel: [ 5661.360471] Buffer I/O error on
device sdb, logical block 1
Apr 17 11:40:44 sunny1 kernel: [ 5661.360478] Buffer I/O error on
device sdb, logical block 2
Apr 17 11:40:44 sunny1 kernel: [ 5661.360486] Buffer I/O error on
device sdb, logical block 3
Apr 17 11:40:44 sunny1 kernel: [ 5661.370342] sd 2:0:0:0: [sdb]
Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK,SUGGEST_OK
Apr 17 11:40:44 sunny1 kernel: [ 5661.370359] end_request: I/O error,
dev sdb, sector 0
Apr 17 11:40:44 sunny1 kernel: [ 5661.370369] Buffer I/O error on
device sdb, logical block 0
Apr 17 11:40:44 sunny1 kernel: [ 5661.375106] sd 2:0:0:0: [sdb]
Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK,SUGGEST_OK
Apr 17 11:40:44 sunny1 kernel: [ 5661.375125] end_request: I/O error,
dev sdb, sector 33554424
Apr 17 11:40:44 sunny1 kernel: [ 5661.375135] Buffer I/O error on
device sdb, logical block 4194303
Apr 17 11:40:44 sunny1 kernel: [ 5661.379983] sd 2:0:0:0: [sdb]

Also i have been getting the error below from the beginning when i enter iscsiadm -m session ( from the client/initiator side)
iscsiadm: could not get host for sid 1.
iscsiadm: could not get host_no for session 6.
iscsiadm: could not find session info for session1
iscsiadm: can not get list of active sessions (6)

The were few bugs reported for the above error and the solution was to upgrade to 2.0.870 while my version was 2.0.865 I then did a fresh install on the client/initator side (ubuntu 9.04) and then installed open-iscsi 2.0.870 and just executed these commands:
iscsiadm -m discovery t st -p 192.168.2.3
iscsiadm -m node -T iqn.2001-04.com.ubuntu:scsi.disk.vg.asm -p 192.168.2.3 -l

and that was it!!...all logical partitions were detected from the client/initiator.

1 comment:

  1. Great article thanks! Do you know if the Ubuntu iSCSI targets will work with Windows Server 2008 R2 Clustering? I have used OpenFiler for 2003 Clustering but it does not seem to work with 08.

    ReplyDelete