Virtualbox shared folder mounting issue: Missing module on opensuse 10.3

The Version combination for this scenario..

This problem was encountered with the following combination:

• Virtualbox 1.6.4


• Host OS: windows XP service pack 2


• Guest OS: OpenSuSE 10.3

A shared virtual folder was defined for the opensuse virtual machine like this:

In the guest OS, a mount point was defined like this:

linux-jo4o:~ # mkdir -p /ddrive

Now when the mount of the shared folder was attempted, this error was received:

inux-jo4o:~ # mount -t vboxsf d_drive /ddrive
/sbin/mount.vboxsf: mounting failed with the error: No such device

An unlikely solution...

It seems that a critical module required for shared mount points was not loaded in memory. Here, we can see that only one module with the vbox pattern in it's name is loaded:

linux-jo4o:~ # lsmod | grep vbox
vboxadd                56232  3

So lets load the vboxsvfs module into memory:

linux-jo4o:~ # modprobe vboxvfs

After that, we can verify that it is really loaded:

linux-jo4o:~ # lsmod | grep vbox
vboxvfs                38184  0
vboxadd                56232  4 vboxvfs

Now, let us mount the shared folder:

linux-jo4o:~ #  mount -t vboxsf d_drive /ddrive

Awesome! That was smooth. Lets check if the shared folder really got mounted or not:

linux-jo4o:~ # mount
/dev/sda2 on / type ext3 (rw,acl,user_xattr)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
debugfs on /sys/kernel/debug type debugfs (rw)
udev on /dev type tmpfs (rw)
devpts on /dev/pts type devpts (rw,mode=0620,gid=5)
/dev/sda3 on /home type ext3 (rw,acl,user_xattr)
/dev/sr0 on /media/VBOXADDITIONS_1.6.4_33808 type iso9660 (ro,nosuid,nodev,uid=0)
d_drive on /ddrive type vboxsf (rw)
linux-jo4o:~ #

Yippee!

I can share files with my windows laptop now. Now that was some adventure, eh :)

To make this more permanent, there has to be a way of loading this module automatically. Maybe I can put the modprobe command in /etc/rc.d/boot.local file, but then maybe there is a better way. I leave that to you to figure out.

Package dependency for installing virtualbox guest additions on Open SuSE 10.3

The problem at hand..

Well, I had to get the shared folder feature working on Open SuSE 10.3 virtual machine for a variety of reasons, and in turn, I had to install guest additions on virtualbox 1.6.4. There were a bunch of package dependencies even before coming to this error, and unfortunately, I dont have the entire series of command that I issued, but in this posting, I will attempt to present the solution of the immediate error that I was facing.

The KERN_INCL variable had to be set to the /usr/include directory:

linux-jo4o:/usr/src/linux # env | grep KER
KERN_INCL=/usr/include

After mounting the guest additions CD and running the VboxAdditions.run command as root, the kernel was not able to be compiled and this was visible in the install log:

linux-jo4o:~/addons # more /var/log/vboxadd-install.log

Installing VirtualBox 1.6.4 Guest Additions, built Tue Jul 29 20:13:59 CEST 2008

Testing the setup of the guest system

Building a test kernel module...

make KBUILD_VERBOSE=1 -C /lib/modules/2.6.22.5-31-default/build SUBDIRS=/tmp/sel
fgz1883915647/module/test SRCROOT=/tmp/selfgz1883915647/module/test modules
make[1]: *** No rule to make target `modules'.  Stop.
make: *** [vboxadd_test] Error 2

Building the test kernel module failed.
Giving up due to the problems mentioned above.
linux-jo4o:~/addons #

Initial thoughts and a potential solution

Initially, I thought that maybe it was a problem due to missing symbolic link to the modules directory (??), but when I posted this question on the vbox-users mailing list (which I think is more effective than the forums at times), I got a response saying that maybe I should check if the kernel-source package was installed or not.

To even find the package, we needed the main SOURCE repository added in the YAST setup. This is what the installed online repositories looked like:



When I ran the following command as root, it was clear that this was the missing link:

linux-jo4o:~ # zypper install kernel-source
* Reading repository 'Main Repository (NON-OSS)' cache
* Reading repository 'openSUSE-10.3-OSS-KDE 10.3' cache
* Reading repository 'Main Repository (Sources)' cache
* Reading repository 'Main Repository (OSS)' cache
* Reading installed packages [100%]

The following NEW package is going to be installed:
  kernel-source 

Overall download size: 49.2 M. After the operation, additional 243.4 M will be used.
Continue? [yes/no]: yes
Downloading package kernel-source-2.6.22.5-31.i586, 49.2 M (243.4 M unpacked)
Downloading: media
* Downloading [100%]
Downloading: kernel-source-2.6.22.5-31.i586.rpm
* Downloading [100%]
* Installing: kernel-source-2.6.22.5-31 [100%]
linux-jo4o:~ #

Now, when I tried installing the guest additions, it was able to install them and recompile the kernel JUST fine. Sweet!

linux-jo4o:~ # /media/VBOXADDITIONS_1.6.4_33808/VboxAdditions.run
Verifying archive integrity... All good.
Uncompressing
VirtualBox 1.6.4 Guest Additions for Linux
installation.............................................................................................................................................................................
VirtualBox 1.6.4 Guest Additions installation
Building the VirtualBox Guest Additions kernel module...
Building the shared folder support kernel module...
Installing the VirtualBox Guest Additions...

Successfully installed the VirtualBox Guest Additions.
You must restart your guest system in order to complete the installation.
linux-jo4o:~ #

Sharing windows folders on unix hosts using samba filesystem (smbfs) using virtualbox

The need for windows sharing..

A few days ago, I wanted to do a Oracle 12i installation on a x86 linux distribution using virtualbox. The media was available on an external hard disk, but I thought that instead of copying the 30+ G of media on unix virtual machine, why not just make it shared between the windows host OS and unix guest OS.

The quest for sharing..

And so began my quest for sharing files between windows and virtualbox unix OS (SuSE linux). I discovered that for sharing folders using virtualbox, the guest OS additions have to be installed.

I have seen that upgrading the kernel breaks the guest OS additions at times. After upgrading the guest OS from SuSE linux 9 to SuSE linux 9.3, the guest additions stopped working. The same problem also exists with vmware tools.

So first, I created a shared folder in the virtual machine's definition:

Click here for a bigger view



After starting the Guest OS (Ubuntu),

root@gverma-laptop:/home/gverma# mkdir /mnt/winshare

root@gverma-laptop:/home/gverma# mount -t vboxsf public /mnt/winshare

Mind you, the filesystem type is NOT vboxfs, it is vboxsf (s and f are interchanged). If you do not take note of this, you might end up wasting your time for nothing.

Now, we can check that the shared folder did indeed mount:

root@gverma-laptop:/home/gverma# mount | grep vbox
public on /mnt/winshare type vboxsf (rw)

root@gverma-laptop:/home/gverma# ls /mnt/winshare
CyberLink  desktop.ini  Downloads  Music     Recorded TV         Videos
Desktop    Documents    Favorites  Pictures  StarzEntertainment  Vongo

Let it be mentioned here that the vboxadd and vboxvfs (note that it has vfs after vbox) modules needed to be loaded for this to work:

root@gverma-laptop:/home/gverma# lsmod | grep vb
vboxvfs                 42432  1
vboxadd                 24872  9 vboxvfs

The Caveat..

But, as every cloud has a silver lining, there was a caveat here for my problem.

The above case was just a demonstration of how the shared folder could have worked fine. I actually then went on to mount some linux media for Oracle Applications 12i for x86 intel platform. After the directory mounted all right, I had a really hard time adding the execute permissions to the rapidwiz files. Even after doing persistent # chmod -R +x commands, the shell scripts under the media were not showing the "x" execute flag.

This was very frustrating. It was probably just another virtualbox bug.

So, then I started looking around for more options. I had heard that many people had used samba with success and hence I started scouring the internet (Google Zindabaad! - Long live, Google!). Came across a link that explained how to do it in a succinct manner.

The good thing about sharing files from windows to unix using samba file system (smbfs) was that was that I did not need to run samba on unix at all.

Doing it using samba filesystem..

So here are some quick and easy steps for sharing a windows share using smbfs:

First things first. You need to create a network share for the folder that you want to share. In my case, I wanted to share an external hard disk that had the media for Oracle Applications 12i. So, I did a network share of windows folder (e.g. \\GAURAV-PC\media)

On the guest OS (Ubuntu), did the following:

# mkdir -p /mnt/media

# smbmount //GAURAV-PC/media /mnt/media -o username=gaurav,password=welcome

Please note that // (forward slashes) worked for me. Some guides on the internet say that you need to use \\ (back slashes)

Note here that GAURAV-PC hostname was defined in the local hosts file to a resolvable IP address in the LAN. The Ubuntu Virtual machine was connected to the windows Host through a host only networking type interface, that allowed the host and guest OS to talk to each other.

# more /etc/hosts
GAURAV-PC 192.168.0.2

Please note here that a ping to GAURAV-PC HAS TO work when you give the smbmount command. I wasted many hours trying to run strace on this command, just because it was hanging. The issue was that due to some strange happenstance, the ping to GAURAV-PC (windows host) was hanging.

The workaround was to ping the Ubuntu virtual machine (say 192.168.0.7) first from the host OS (windows), after which the ping to GAURAV-PC 192.168.0.2 started working from Ubuntu Guest OS. Strange, strange, I agree, but I am just a guy looking for solutions! I have seen this behaviour in Windows Vista home premium edition - host OS.

Make sure the smbfs module is loaded in the guest OS. If not, then load it using # modprobe smbfs:

root@gverma-laptop:/mnt/media/oracle_12.0.4/startCD/Disk1/rapidwiz# lsmod | grep smbfs
smbfs                  66296  0

Note that the Samba server does not need to run:

root@gverma-laptop:/mnt/media/oracle_12.0.4/startCD/Disk1/rapidwiz# ps -ef | grep smb
root      6116  5616  0 00:00:00 grep smb

This was my samba configuration file:

root@gverma-laptop:/mnt/media/oracle_12.0.4/startCD/Disk1/rapidwiz# more /usr/share/samba/smb.conf
[global]
workgroup = WORKGROUP
security = share
usershare path =  /var/lib/samba/usershare
usershare max shares = 100
usershare allow guests = yes
usershare owner only = yes
wins support = no
netbios name = GAURAV-PC

I am not exactly sure if setting the netbios name parameter was a major factor in making it work or not, but I guess it certainly did not hurt.

Now when I run mount command, the windows network share was showing up there just fine:

root@gverma-laptop:/mnt/media/oracle_12.0.4/startCD/Disk1/rapidwiz# mount
/dev/sda1 on / type ext3 (rw,relatime,errors=remount-ro)
proc on /proc type proc (rw,noexec,nosuid,nodev)
/sys on /sys type sysfs (rw,noexec,nosuid,nodev)
varrun on /var/run type tmpfs (rw,noexec,nosuid,nodev,mode=0755)
varlock on /var/lock type tmpfs (rw,noexec,nosuid,nodev,mode=1777)
udev on /dev type tmpfs (rw,mode=0755)
devshm on /dev/shm type tmpfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
lrm on /lib/modules/2.6.24-16-generic/volatile type tmpfs (rw)
securityfs on /sys/kernel/security type securityfs (rw)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
nfsd on /proc/fs/nfsd type nfsd (rw)
gvfs-fuse-daemon on /home/gverma/.gvfs type  fuse.gvfs-fuse-daemon (rw,nosuid,nodev,user=gverma)
//GAURAV-PC/media on /mnt/media type cifs (rw,mand)

Thankfully, I was able to make the unix shell scripts executable after doing this and launch the rapid wizard for 12i installation. That's all I wanted.

Conclusion

Well, so there you have it. If you are using windows host OS and you upgraded your guest OS kernel, the guest additions may not work due to compilation errors. In such a case, you are better off using the samba file system as its much easier to deal with and not too much of a pain to setup.

iSCSI: no LUNs detected for session when using openfiler

A common problem...

If you are using openfiler, at some point of time, you are bound to reboot the openfiler OS/machine. After the reboot, a common problem is that openfiler forgets or loses the logical volume/volume group information.

So, as a workaround, you need to issue the following commands:

################################
# to discover the volume groups
################################
# vgscan   

#################################
# to discover the logical volumes
#################################
# lvscan  

##################################
# to discover the physical volumes
##################################
# pvscan

Then, you can display them using the commands vgdisplay, lvdisplay, pvdisplay. Similarly, there is a whole suite of commands in the same location.

Then, you STILL need to activate the discovered logical volumes using the -ay switch of lvchange:

# lvchange -ay openfiler/asm
# lvchange -ay openfiler/ocr
# lvchange -ay openfiler/vote

Now, if you run the lvscan or lvdisplay command, the status of the Logical volumes will show as ACTIVATED.

Remember..

AFTER you do this, You will STILL need to restart the iscsi-target service on openfiler machine.

openfiler $> service iscsi-target restart

If you fail to do this, you will see these messages in the dmesg output on the iscsi-initiator node (not the openfiler machine) - iSCSI: no LUNs detected for session. This message can be very confusing, especially since you will see that the session to the iscsi target WAS established:

openfiler $> dmesg
...
...
iSCSI: bus 0 target 0 = iqn.2006-01.com.openfiler:openfiler.vote
iSCSI: bus 0 target 0 portal 0 = address 10.143.213.248 port 3260 group 1
iSCSI: starting timer thread at 286340
iSCSI: bus 0 target 0 trying to establish session to portal 0, address 10.143.213.248 port 3260 group 1
iSCSI: bus 0 target 1 = iqn.2006-01.com.openfiler:openfiler.ocr
iSCSI: bus 0 target 1 portal 0 = address 10.143.213.248 port 3260 group 1
iSCSI: bus 0 target 1 trying to establish session to portal 0, address 10.143.213.248 port 3260 group 1
iSCSI: bus 0 target 0 established session #1, portal 0, address 10.143.213.248 port 3260 group 1
iSCSI: bus 0 target 2 = iqn.2006-01.com.openfiler:openfiler.asm
iSCSI: bus 0 target 2 portal 0 = address 10.143.213.248 port 3260 group 1
iSCSI: bus 0 target 1 established session #1, portal 0, address 10.143.213.248 port 3260 group 1
iSCSI: bus 0 target 2 trying to establish session to portal 0, address 10.143.213.248 port 3260 group 1
iSCSI: bus 0 target 3 = iqn.2006-01.com.openfiler:openfiler.test
iSCSI: bus 0 target 3 portal 0 = address 10.143.213.248 port 3260 group 1
iSCSI: bus 0 target 3 trying to establish session to portal 0, address 10.143.213.248 port 3260 group 1
iSCSI: bus 0 target 2 established session #1, portal 0, address 10.143.213.248 port 3260 group 1
iSCSI: no LUNs detected for session (bus 0 target 0) to iqn.2006-01.com.openfiler:openfiler.vote
iSCSI: no LUNs detected for session (bus 0 target 1) to iqn.2006-01.com.openfiler:openfiler.ocr
iSCSI: no LUNs detected for session (bus 0 target 2) to iqn.2006-01.com.openfiler:openfiler.asm
iSCSI: bus 0 target 3 established session #1, portal 0, address 10.143.213.248 port 3260 group 1

Virtualbox How to: Gotchas involved in making a USB external hard disk device work on windows host

Preface

Sometime back, I installed Openfiler on a external hard disk (manufacturer: Iomega)that I had at home (running Windows Vista Home Premium). This was meant to be a proof of concept and pretty soon, I had to bring it to office desktop (running Windows XP SP2) to make it work with some other virtualbox virtual machines that I had built in my spare time.

Troubles in recognizing the USB external hard disk on windows XP

However, the problem that I faced on Windows XP SP2 host operating system was that the openfiler Virtual machine (using virtualbox 1.5.6 of course) would not recognize the USB external hard disk at all! The same USB device was working fine on windows vista.

After struggling with this problem for a while, I found a quick workaround (Not sure if it is documented in the user manual or not) and also tested some reset steps on windows, which seem to work with the Iomega external USB hard disk.

These steps should also be relevant to other hard disk models in general.

Some more details...

In my case, windows had detected the USB drive and had even installed the Virtualbox driver for it. USB support was enabled for the openfiler virtual machine in the settings. The external hard disk was selected, but still no iscsi drive showed up on doing a fdisk -l

When to know a device is being accessed..

Well, if the device makes a whirring noise and the light blinks or comes on, it is enough to know that the device is being accessed internally and something good happened in terms of making the USB device work.

The setup that worked..

However, this setup worked after a little tinkering:

Firstly, make sure the USB controller and EHCI controller is enabled. Then choose the USB external hard disk device from the list of all USB devices and enabled it.

Click here for a bigger view



Make sure no other Virtual machine is accessing the same USB device. From what I have seen, if the same device is accessed by other virtualbox VMs, an exclusive lock is acquired on it, which makes it difficult to be used by other virtual machines.

This is true with windows too. Windows will stop seeing the USB hard disk when it is being used by a virtualbox virtual machine. At least, I have seen this behaviour with virtualbox 1.5.6

Click here for a bigger view



Make sure that Windows is able to detect the USB device first. If not, then you need to troubleshoot that first. Usually, powering off the device, unplugging it from the USB port and restarting the windows PC/desktop/laptop works great. (Hey, its windows).

You want to see something like this in your system tray to be sure of this.



Now you know for a fact that windows, your guest Operating system, detected the USB drive fine. So now, there is a good chance that virtualbox will detect it too.

Fire it up..

Power up the virtual machine and check if the USB device got detected. A simple fdisk -l will show it as a iscsi device, with names like /dev/sda, /dev/sdb or so. If you do not see anything of the sort, you can be certain that the device did not get detected. Another visual way to know is to see if the light on USB device comes up or not. No light, no access.

The manual workaround if the device is still now seen after powering up the virtual machine ..

However, there is still another trick up our sleeve that can be tried.

Click here for a bigger view



At the bottom message bar of the virtual machine, right click on the USB plug icon. You will see that a list of the detected USB devices comes up. Select the external hard disk. Unless another virtual machine is running that is using the same USB device, there is a very good chance that the light on the hard disk will light up like this:



At this point, you may also see a windows dialog box saying that a virtualbox USB device was detected and the driver needs to be loaded:

Double check before you congratulate yourself..

All right! So it looks like your device got accessed after all.

There are many ways to confirm if it really got detected by the virtual machine as a scsi device or not:

1) Check the fdisk -l command's output:

Click here for a bigger view



2) Check the contents of /proc/scsi/scsi file:

Click here for a bigger view



3) Check the content of /var/log/messages or the output of dmesg command:

Click here for a bigger view

A quick Recap...

Once again, if this does not work, just go back to the basics:

1) Power off the USB device (when applicable) and take it out

2) Reboot windows

3) Plug in the USB device

4) Make sure windows detects it

5) Make sure the virtual machine has the USB device enabled in its setup

6) Start up the virtual machine. Make sure no other virtual machine is using the same USB drive.

7) If you do not see the USB device, force it to be attached by right click on USB plug icon in bottom bar of virtualbox VM application window

Hope that helps.

Combining Openfiler and Virtualbox (Ubuntu guest OS on windows host)

Preface - Celebrating Openfiler

Ever since I came to know about openfiler, a free open source network attached storage appliance, I could not wait to get started on it! Compared to similar open source products in the internet like FreeNAS, openfiler has much better reviews from the user community.

In this article, we will talk about how we can leverage openfiler along with an Ubuntu virtual machine running on windows host. The fact that the virtual machine is running on windows is immaterial, as most of the material covered in this article will deal with making openfiler shared devices work with a unix distribution, namely Ubuntu, in our case. The steps discussed should not be largely different for any other unix operating system.

The ingredients

One of the main ingredients for doing this setup is to know about how to make host only networking work in Virtualbox. For this, I will recommend you to go through the article Virtualbox Case Study: Making host only networking work between two Ubuntu Guest OS (virtual machine) on Windows Vista host, that was posted just before this one. It has detailed steps as to how two virtual machines can be made to talk to each other, with internet working too.


So, going along with this idea, you need to do host only networking between a Unix virtual machine and an openfiler installation virtual machine. For installing openfiler, detailed how to graphical instructions are available at http://openfiler.com/learn/how-to/graphical-installation. If you are a text person, consider going through the text instructions at http://openfiler.com/learn/how-to/text-based-installation.

The other important ingredient that we need is an external USB hard disk to attach with the openfiler virtual machine like this:

click here for a bigger picture

I discovered that after the openfiler virtual machine booted up, windows vista premium home edition was not able to discover the USB hard disk in explorer. After I shutdown openfiler virtual machine, windows would detect the USB external drive.

I also noticed was that if I specified the USB mouse device in the virtualbox USB devices setup of the unix machine, it was not able to access it! (The touchpad was still working though). So eventually, I just unchecked the USB mouse device for Ubuntu virtual machine and everything was ok.

The Openfiler Virtualbox VM setup example

Click here to see a bigger picture



The Ubuntu unix Virtualbox VM setup example

Click here to see a bigger picture



After the openfiler installation, you can see that it installs a linux 2.6.x OS, which can be brought up like any other linux installation.



Once you boot up the operating system, you can invoke the web administration GUI tool using http://<IP of openfiler server>:446 like this (openfiler/password is the default login):

Click here for a bigger picture

A Caveat if you do not want to use the entire external USB drive as network attached storage

Since I could not afford to dedicate my entire 300GB USB external drive for this experiment, I had to find a way of working with a part of it. Thankfully, with the help of gparted live CD, I was able to resize the FAT32/NTFS windows partition in external drive to 175G and created another partition of 125G with the remaining space. If you have never used gparted (GNOME partition editor), I must tell you that you simply have to try it. It is free, its versatile and its simply amazing.

Please note here that I DID NOT format the 125G partition as ext3 or any other format. This was important since the openfiler was not able to see the second physical volume in the USB drive otherwise as a iSCSI device otherwise.

Once this was done, I found that the openfiler administrator GUI was having a hard time creating a physical volume and a volume group based on the second partition carved in USB drive.

It seems that the GUI assumes that the entire attached drive should be available for its manipulations.

To get around this problem, I had to use the command line interface commands, which was good in a way, as I was able to learn many useful commands that are employed by the pretty front end. Using fdisk -l, it could be seen that the USB external drive was discovered as /dev/sda by openfiler OS.

# pvcreate /dev/sda2
Physical volume "/dev/sda2" successfully created

Click here for the bigger picture

Let it be understood that after any changes on openfiler, it is required to restart the iSCSI service. This can be done either by "# service iscsi-target restart" command or from the GUI: Services->Enable/Disable (Disable and Enable the iSCSI service forcibly to achieve the same result).

# vgcreate openfiler /dev/sda2
Volume group "vg" successfully created

Click here for a bigger picture (note that the Web UI 
does not show the PVs yet, but its still OK)

After this, we need to create the logical volumes. Another weird thing I noticed was that if I used the lvcreate commane line interface to create logical volumes, they were not showing up in the web admin utility, even after restarting the iSCSI server on openfiler OS.

So I created three of them from the web administration utility (Hey, whatever works):

/dev/openfiler/ocr
/dev/openfiler/vote
/dev/openfiler/asm

This is how the web GUI showed them now:

Click here for a bigger view



The openfiler:/etc/ietd.conf file now has contents like this:

Defining Local networks

Be informed that another variation here is to define a local network consisting of IP subnet, which would essentially decide which machines can be allowed to discover the LUNs in network attached storage. Using the GUI, this is achieved through General->Local Networks.



Subsequently, you need can control allow/deny access to each logical volume from the GUI by updating the property of respective logical volumes (be aware that this creates /etc/initiators.allow and /etc/initiators.deny files in the openfiler OS):

Click here for a bigger view

I did not use any local networks, as I got burnt by giving an incorrect subnet and that was causing the discovery of LUNs to not work from ubuntu VM. So I just kept it plain and simple by allowing any machine in the LAN to discover the LUNs.

CHAP authentication

There is also something called CHAP authentication (the username and passwords can be set using the iscsiadm command) to further qualify iscsi initiator/target LUN discovery. This is a topic that I have not explored fully at this point, so I did not enable incoming/outgoing CHAP authentication either in openfiler OS (iSCSI target) or the ubuntu virtual machine (iSCSI initiator).

This was another aspect of open-iscsi that burnt me while trying to discover target LUNs from ubuntu VM, so I just steered clear of it, for the time being.

There is a fantastic How to on how to configure open-iscsi using CHAP authentication at http://en.opensuse.org/Open-iSCSI_and_SUSE_Linux. I would strongly recommend you to read and digest it.

Understanding how LUN discovery works with open-iscsi

open-iscsi is a robust, well performing implementation of iSCSI device package that is very much in vogue and is being adopted by various unix flavours.

It runs a daemon in the background called iscsid. Open-iSCSI utility keeps a persistent configuration of target LUNs and initiator nodes in a database. The iscsiadm utility is a command-line tool to manage (update, delete, insert, query) the persistent database.

The database contains two tables:

- Discovery table (/etc/iscsi/send_targets);
- Node table (/etc/iscsi/nodes).

You can install the open-iscsi package using either Synaptic package manager or "sudo apt-get install open-iscsi" command in Ubuntu. For my case, the apt-get command was somehow not able to to refer to ubuntu repositories, but thankfully Synaptic package manager worked fine.

open-iscsi works on a client server model. Initiators like the Ubuntu virtual machine will send discovery requests to SCSI targets and access them by creating login sessions. Till the duration of the session, the initiator can access the discovered targets (LUNs). Simply put, that is it.

This can be made a little complicated by adding CHAP authentication into the mix.

Each Initiator has a unique name, which can be got by checking the contents of /etc/iscsi/initiatorname.iscsi:

gverma@gverma-laptop:~$ sudo more /etc/iscsi/initiatorname.iscsi
## DO NOT EDIT OR REMOVE THIS FILE!
## If you remove this file, the iSCSI daemon will not start.
## If you change the InitiatorName, existing access control lists
## may reject this initiator.  The InitiatorName must be unique
## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.
InitiatorName=iqn.1993-08.org.debian:01:8211251a31ff

Making iscsi work

After installation of open-iscsi, I made sure that /etc/iscsi/iscsid.conf had the defaults configured, without CHAP authentication. The defaults for the rest are usually ok, which is what I went along with. After any changes, its important to restart open-iscsi service:

# sudo /etc/init.d/open-iscsi restart

I also disabled CHAP authentication (both incoming and outgoing users) for each logical volume from the openfiler GUI administrator utility.

The IP of openfiler VM was 192.168.0.6 and that Ubuntu VM was 192.168.0.4

Some initial problems..

gverma@gverma-laptop:~$ sudo iscsiadm -m discovery -t
st -p 192.168.0.6
iscsiadm: Login failed to authenticate with target
iscsiadm: discovery login to 192.168.0.6 rejected:
initiator error (02/01), non-retryable, giving up

To debug this, we can use the -d switch:

gverma@gverma-laptop:~$ sudo iscsiadm -m discovery -d -t st -p 192.168.0.6

discovery.startup = manual
discovery.type = sendtargets
discovery.sendtargets.address = 192.168.0.6
discovery.sendtargets.port = 3260
discovery.sendtargets.auth.authmethod = None
discovery.sendtargets.auth.username = gverma
discovery.sendtargets.auth.password = ********
discovery.sendtargets.auth.username_in = <empty>
discovery.sendtargets.auth.password_in = <empty>
discovery.sendtargets.timeo.login_timeout = 15
discovery.sendtargets.reopen_max = 5
discovery.sendtargets.timeo.auth_timeout = 45
discovery.sendtargets.timeo.active_timeout = 30
discovery.sendtargets.timeo.idle_timeout = 60
discovery.sendtargets.iscsi.MaxRecvDataSegmentLength = 32768

It seemed that some sort of authentication was still being used. To overcome this issue, I commented CHAP authentication for all discovery modes in /etc/iscsi/iscsid.conf and restarted open-iscsi service.

Then we needed to discover the targets on 192.168.0.6 (openfiler VM):

gverma@gverma-laptop:~$ sudo iscsiadm -m discovery
-t st -p 192.168.0.6
192.168.0.6:3260,1 iqn.2006-01.com.openfiler:openfiler.asm
192.168.0.6:3260,1 iqn.2006-01.com.openfiler:openfiler.ocr
192.168.0.6:3260,1 iqn.2006-01.com.openfiler:openfiler.vote

gverma@gverma-laptop:~$ sudo iscsiadm -m discovery
192.168.0.6:3260 via sendtargets

gverma@gverma-laptop:~$ sudo iscsiadm -m node
192.168.0.6:3260,1 iqn.2006-01.com.openfiler:openfiler.ocr
192.168.0.6:3260,1 iqn.2006-01.com.openfiler:openfiler.vote
192.168.0.6:3260,1 iqn.2006-01.com.openfiler:openfiler.asm

Now that the targets from Ubuntu node were discovered, it was needed to login to each of them:

gverma@gverma-laptop:~$ sudo iscsiadm -m node
-T iqn.2006-01.com.openfiler:openfiler.ocr -p 192.168.0.6 -l

Login session [iface: default, target: iqn.2006-01.com.openfiler:openfiler.ocr,
portal: 192.168.0.6,3260]
iscsiadm: initiator reported error (5 - encountered iSCSI login failure)
iscsiadm: Could not execute operation on all records. Err 107.

This error was because I had setup a local network for qualifying initiators to connect to LUNs and has basically given a wrong subnet in the openfiler setup. When I removed the local network from openfiler setup, removed openfiler:/etc/initiators.allow and /etc/initiators.deny files, and restarted ietd service, the command went through.

Now, we can also setup the target LUNs to startup/attach automatically on iscsi service restart on initiator (this will make the LUNs visible from the iscsi initiator by creating a session to the iscsi target). This can be done by influencing the node.startup property value to automatic in the iscsid database:

gverma@gverma-laptop:~$ sudo iscsiadm -m node \
-T iqn.2006-01.com.openfiler:openfiler.ocr -p 192.168.0.6 \
--op update -n node.startup -v automatic
gverma@gverma-laptop:~$ sudo iscsiadm -m node \
-T iqn.2006-01.com.openfiler:openfiler.vote -p 192.168.0.6 \
--op update -n node.startup -v automatic
gverma@gverma-laptop:~$ sudo iscsiadm -m node \
-T iqn.2006-01.com.openfiler:openfiler.asm -p 192.168.0.6 \
--op update -n node.startup -v automatic

Now, when we restart the open-iscsi service, it can be seen that the targets LUNs are attaching to the initiator:

gverma@gverma-laptop:~$ sudo /etc/init.d/open-iscsi restart
* Disconnecting iSCSI targets                                                                                         [ OK ]
* Stopping iSCSI initiator service                                                                                    [ OK ]
* Starting iSCSI initiator service iscsid                                                                             [ OK ]
* Setting up iSCSI targets
Login session [iface: default, target: iqn.2006-01.com.openfiler:openfiler.ocr,
portal: 192.168.0.6,3260]
Login session [iface: default, target: iqn.2006-01.com.openfiler:openfiler.vote,
portal: 192.168.0.6,3260]
Login session [iface: default, target: iqn.2006-01.com.openfiler:openfiler.asm,
portal: 192.168.0.6,3260]                                                                                               [ OK ]

You can verify the login sessions with this command:

gverma@gverma-laptop:~$ sudo iscsiadm -m session
tcp: [4] 192.168.0.6:3260,1 iqn.2006-01.com.openfiler:openfiler.ocr
tcp: [5] 192.168.0.6:3260,1 iqn.2006-01.com.openfiler:openfiler.vote
tcp: [6] 192.168.0.6:3260,1 iqn.2006-01.com.openfiler:openfiler.asm

If you logout, the sessions wont be visible anymore:

gverma@gverma-laptop:~$ sudo iscsiadm -m node \
-T iqn.2006-01.com.openfiler:openfiler.asm -p 192.168.0.6 --logout
Logout session [sid: 1, target: iqn.2006-01.com.openfiler:openfiler.ocr,
portal: 192.168.0.6,3260]
Logout session [sid: 2, target: iqn.2006-01.com.openfiler:openfiler.vote,
portal: 192.168.0.6,3260]
Logout session [sid: 3, target: iqn.2006-01.com.openfiler:openfiler.asm,
portal: 192.168.0.6,3260]
gverma@gverma-laptop:~$ sudo iscsiadm -m session
iscsiadm: No active sessions.

You can login back again now:

gverma@gverma-laptop:~$ sudo iscsiadm -m node \
-T iqn.2006-01.com.openfiler:openfiler.asm -p 192.168.0.6 --login
Login session [iface: default, target: iqn.2006-01.com.openfiler:openfiler.ocr,
portal: 192.168.0.6,3260]
Login session [iface: default, target: iqn.2006-01.com.openfiler:openfiler.vote,
portal: 192.168.0.6,3260]
Login session [iface: default, target: iqn.2006-01.com.openfiler:openfiler.asm,
portal: 192.168.0.6,3260]
gverma@gverma-laptop:~$ sudo iscsiadm -m session
tcp: [4] 192.168.0.6:3260,1 iqn.2006-01.com.openfiler:openfiler.ocr
tcp: [5] 192.168.0.6:3260,1 iqn.2006-01.com.openfiler:openfiler.vote
tcp: [6] 192.168.0.6:3260,1 iqn.2006-01.com.openfiler:openfiler.asm

This is how the current mapping of the attached SAN LUNs and device name is got for the current session:

gverma@gverma-laptop:~$ ls -l /dev/disk/by-path
lrwxrwxrwx 1 root root  9 2008-04-16 22:02 ..
ip-192.168.0.6:3260-iscsi-iqn.2006-01.com.openfiler:openfiler.asm-lun-0 -> ../../sdd
lrwxrwxrwx 1 root root  9 2008-04-16 22:02 ..
ip-192.168.0.6:3260-iscsi-iqn.2006-01.com.openfiler:openfiler.ocr-lun-0 -> ../../sdb
lrwxrwxrwx 1 root root  9 2008-04-16 22:02 ..
ip-192.168.0.6:3260-iscsi-iqn.2006-01.com.openfiler:openfiler.vote-lun-0 -> ../../sdc
lrwxrwxrwx 1 root root  9 2008-04-16 12:48 pci-0000:00:01.1-scsi-0:0:0:0 -> ../../sda
lrwxrwxrwx 1 root root 10 2008-04-16 12:48 pci-0000:00:01.1-scsi-0:0:0:0-part1 -> ../../sda1
lrwxrwxrwx 1 root root 10 2008-04-16 12:48 pci-0000:00:01.1-scsi-0:0:0:0-part2 -> ../../sda2
lrwxrwxrwx 1 root root 10 2008-04-16 12:48 pci-0000:00:01.1-scsi-0:0:0:0-part5 -> ../../sda5
lrwxrwxrwx 1 root root 10 2008-04-16 12:48 pci-0000:00:01.1-scsi-1:0:0:0 -> ../../scd0

Paydirt: Seeing the LUNs on Ubuntu VM

This could be seen in the /var/log/messages:

Apr 16 21:53:19 gverma-laptop kernel: [18263.996005] scsi 42:0:0:0: Direct-Access Openfile Virtual disk 0 PQ: 0 ANSI: 4
Apr 16 21:53:19 gverma-laptop kernel: [18264.000393] sd 42:0:0:0: [sdb] 2097152 512-byte hardware sectors (1074 MB)
Apr 16 21:53:19 gverma-laptop kernel: [18264.002548] sd 42:0:0:0: [sdb] Write Protect is off
Apr 16 21:53:19 gverma-laptop kernel: [18264.004196] sd 42:0:0:0: [sdb] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
Apr 16 21:53:19 gverma-laptop kernel: [18264.004196] sd 42:0:0:0: [sdb] 2097152 512-byte hardware sectors (1074 MB)
Apr 16 21:53:19 gverma-laptop kernel: [18264.004196] sd 42:0:0:0: [sdb] Write Protect is off
Apr 16 21:53:19 gverma-laptop kernel: [18264.009630] sd 42:0:0:0: [sdb] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
Apr 16 21:53:19 gverma-laptop kernel: [18264.009718] sdb: unknown partition table
Apr 16 21:53:19 gverma-laptop kernel: [18264.030856] sd 42:0:0:0: [sdb] Attached SCSI disk
Apr 16 21:53:19 gverma-laptop kernel: [18264.030974] sd 42:0:0:0: Attached scsi generic sg2 type 0
Apr 16 21:53:19 gverma-laptop kernel: [18264.292677] scsi43 : iSCSI Initiator over TCP/IP
Apr 16 21:53:19 gverma-laptop kernel: [18264.553516] scsi 43:0:0:0: Direct-Access Openfile Virtual disk 0 PQ: 0 ANSI: 4
Apr 16 21:53:19 gverma-laptop kernel: [18264.555463] sd 43:0:0:0: [sdc] 2097152 512-byte hardware sectors (1074 MB)
Apr 16 21:53:19 gverma-laptop kernel: [18264.556540] sd 43:0:0:0: [sdc] Write Protect is off
Apr 16 21:53:19 gverma-laptop kernel: [18264.559456] sd 43:0:0:0: [sdc] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
Apr 16 21:53:19 gverma-laptop kernel: [18264.561681] sd 43:0:0:0: [sdc] 2097152 512-byte hardware sectors (1074 MB)
Apr 16 21:53:19 gverma-laptop kernel: [18264.564112] sd 43:0:0:0: [sdc] Write Protect is off
Apr 16 21:53:19 gverma-laptop kernel: [18264.566617] sd 43:0:0:0: [sdc] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
Apr 16 21:53:19 gverma-laptop kernel: [18264.566719] sdc: unknown partition table
Apr 16 21:53:19 gverma-laptop kernel: [18264.585774] sd 43:0:0:0: [sdc] Attached SCSI disk
Apr 16 21:53:19 gverma-laptop kernel: [18264.585872] sd 43:0:0:0: Attached scsi generic sg3 type 0
Apr 16 21:53:20 gverma-laptop kernel: [18264.847763] scsi44 : iSCSI Initiator over TCP/IP
Apr 16 21:53:20 gverma-laptop kernel: [18265.112437] scsi 44:0:0:0: Direct-Access Openfile Virtual disk 0 PQ: 0 ANSI: 4
Apr 16 21:53:20 gverma-laptop kernel: [18265.112437] sd 44:0:0:0: [sdd] 125829120 512-byte hardware sectors (64425 MB)
Apr 16 21:53:20 gverma-laptop kernel: [18265.112437] sd 44:0:0:0: [sdd] Write Protect is off
Apr 16 21:53:20 gverma-laptop kernel: [18265.113012] sd 44:0:0:0: [sdd] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
Apr 16 21:53:20 gverma-laptop kernel: [18265.115823] sd 44:0:0:0: [sdd] 125829120 512-byte hardware sectors (64425 MB)
Apr 16 21:53:20 gverma-laptop kernel: [18265.117126] sd 44:0:0:0: [sdd] Write Protect is off
Apr 16 21:53:20 gverma-laptop kernel: [18265.119686] sd 44:0:0:0: [sdd] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
Apr 16 21:53:20 gverma-laptop kernel: [18265.119786] sdd: unknown partition table
Apr 16 21:53:20 gverma-laptop kernel: [18265.134147] sd 44:0:0:0: [sdd] Attached SCSI disk
Apr 16 21:53:20 gverma-laptop kernel: [18265.134235] sd 44:0:0:0: Attached scsi generic sg4 type 0

The SAN devices were visible on Ubuntu VM now:

gverma@gverma-laptop:~$ sudo fdisk -l
Disk /dev/sda: 5906 MB, 5906628608 bytes
255 heads, 63 sectors/track, 718 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x000eb831
Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1         680     5462068+  83  Linux
/dev/sda2             681         718      305235    5  Extended
/dev/sda5             681         718      305203+  82  Linux swap / Solaris
Disk /dev/sdb: 1073 MB, 1073741824 bytes
34 heads, 61 sectors/track, 1011 cylinders
Units = cylinders of 2074 * 512 = 1061888 bytes
Disk identifier: 0x00000000
Disk /dev/sdb doesn't contain a valid partition table
Disk /dev/sdc: 1073 MB, 1073741824 bytes
34 heads, 61 sectors/track, 1011 cylinders
Units = cylinders of 2074 * 512 = 1061888 bytes
Disk identifier: 0x00000000
Disk /dev/sdc doesn't contain a valid partition table
Disk /dev/sdd: 64.4 GB, 64424509440 bytes
64 heads, 32 sectors/track, 61440 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Disk identifier: 0x00000000
Disk /dev/sdd doesn't contain a valid partition table

Gotcha: Logical Volumes lost after reboot of openfiler

One common caveat that I noticed was that the logical volumes were lost after a reboot of the openfiler virtual machine. The issue seems to be related to detection of USB devices while bringing up the linux OS.

Anyways, to get around it, I did the following each time (better put this in /etc/rc.local)

# pvscan
# vgscan
# lvscan

Activate the logical volumes, otherwise, although sudo /etc/init.d/open-iscsi restart will show that initiator discovers target LUNs, BUT they will not show up in fdisk -l and when you do lvdisplay on openfiler machine, the status will show as NOT available.

# lvchange -ay openfiler/asm
# lvchange -ay openfiler/ocr
# lvchange -ay openfiler/vote
# pvscan
# vgscan
# lvscan

You can verify the status of the logical/physical volumes and volume groups now:

# lvdisplay
# pvdisplay
# vgdisplay

Pending Areas to explore

• One of the pending topics to explore is unique device labeling using udev. This will prevent LUN name renaming from /dev/sdb to /dev/sdd all of a sudden, should you happen to add a new Logical volume in the SAN or restart the open-iscsi server on initiator.

I tried setting it up using some other examples on the internet, but it did not quite work out. It also seems that devlabel is passe' and udev is favored in most unix distributions.

• Mounting the SAN device as a filesystem on initiator machine.

Conclusion

Well, its not as if I covered the whole nine yards, but it was a start and I hope to complete the remaining topics soon. When I do so, I will cover that in more details either in this article or in a separate one.

In the meanwhile, if you have any feedback, feel free to leave a message or email me at gaurav _ verma 22 at yahoo DOT com.

Virtualbox Case Study: Making host only networking work between two Ubuntu Guest OS (virtual machine) on Windows Vista host

Preface

In the past articles, I have talked about how to make internal networking work between two virtual machines built using Virtualbox. In this article, we will see seeing a configuration of host only networking between two virtual machines built on virtualbox.

The advantages of configuring host only networking are :

1) Internet works (yes!)

2) You get an IP on the LAN for each virtual machine. Yes, this is really possible. So this means that you can have 2 more real LAN IPs coming out of a single windows desktop/laptop in the LAN. Isn't that amazing?

3) Due to 2), your host machine can also ping or access your virtual machine

4) Each virtual machine can access the other virtual machine. E.g. ping/ssh/telnet into it.

For most purposes, this kind of setup should be optimal or sufficient. Hey, if you can access your machines in the LAN and internet works from them, that should be good enough. And, its for free.

Here is a bird's eye overview of the end setup..

click here to enlarge this diagram

The Virtualbox Network Adapter configuration

We start with the Virtualbox Network Adapter configuration. We create a host only network interface on each virtual machine definition.

For Virtual machine1 - Gutsy:



For Virtual machine2 - gutsy2:



Please note here that while this article primarily deals with setting up host only networking, in the previous article, I had setup internal networking too, so these VMs also have a Virtual Host Interface 2 for internal networking within themselves.

Install the Guest Operating system on the virtual machines

Now, do the installation of the guest operating system in the virtual machine and setup the two network interfaces like this.

Note here that I chose static IP 192.168.0.5 because my windows machine (host OS) had the IP 192.168.0.3 and the IP had to be in the 192.168.0.x subnet. My windows host was behind a Netgear router.


We need to make sure we choose non-conflicting IPs for the host only interface otherwise you will get an IP conflict in your LAN domain. For the other machine, the eth0 static IP was 192.168.0.4.

An interesting fact here was that if I chose DHCP for eth0 in the VM, i was getting an IP of 192.168.0.3, the same as that of my windows host!! This is another reason why I had to choose static IP address.

For eth1, i chose a random IP in the subnet 192.168.2.x for no particular reason. The eth1 IP of the other virtual machine was 192.168.2.1

Make sure to restart the network after doing this network setup to bring it into effect.

Do not bother about the SIOCADDRT: No such process error:

gverma@gverma-laptop:~$ sudo /etc/init.d/networking restart
[sudo] password for gverma:
* Reconfiguring network interfaces...
SIOCADDRT: No such process
Failed to bring up eth1.
[ OK ]

For the benefit of the reader, the DNS and host information is also reproduced here, although the only thing of significance should be the DNS and default gateway, both of which were set to be the same as that on windows host:

This is how the network configuration on the other virtual machine looked like:

For those who are interested, here is the output of the route command:

gverma@gverma-laptop:~$ route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
192.168.2.0     *               255.255.255.0   U     0      0        0 eth1
192.168.0.0     *               255.255.255.0   U     0      0        0 eth0
link-local      *               255.255.0.0     U     1000   0        0 eth0
default         192.168.0.1     0.0.0.0         UG    100    0        0 eth0

A quick side note for Windows Vista

Here is a quick side note for Vista or windows in general. It is preferable to disable the UAC (user access control) setting and also disable the firewall on the host windows machine.



Disable the User Access Control (UAC) feature on vista to make your life easier:

Network bridging: the key to make host only networking work

The KEY to make host only networking work on windows is to bridge the real working network interface (either wireless connection of a hard wired ethernet) with the virtual adapter network interface. You can do this by selecting two network interfaces, right click, choose bridge and voila, there you have it.

A network bridge is nothing but a simplified concept of joining two connections into one. Its like joining two rivers into a bigger river. When water will flow into the bigger river, it will flow into both the rivers. At least, this is how I understand it.

ALSO, we have to enable promiscuous packet routing mode in ALL the member network interfaces that are in the bridge. This is an important setup to make it work.

Initially, I had struggled a lot to make host only networking work by enabling promiscuous for only the Virtualbox Adapter in the bridge, but was not successful. That is when I read a cryptic posting on the Virtualbox forum saying that it has to be done for all the member bridge interfaces; and thats when it worked.

A note of caution

When we add an adapter to the bridge, the connection of the main wireless network is lost for a moment, but it is re-enabled back. This is also covered in the user manual in secion 6.3 (Virtualbox version 1.5.6)

Warning: Setting up Host Interface Networking requires changes to your
host’s network configuration, which will cause the host to lose its network
connection. Do not change network settings on remote or production systems
unless you know what you are doing.

Later on, I assigned Virtualbox Host Interface 2 (on windows) to NAT0 (eth0) of Virtual machine 2 and bridged it to the wireless network connection too:

After the bridging, this is how the ipconfig output looked like (the IP of the windows host is 192.168.0.3):

Now, we were one BIG, HAPPY Family. Make sure the connection status shows as connected.

Testing the connections - Internet works!

At this point, the internet was working from the virtual machines:

Another real quick check is to use the wget utility:

gverma@gverma-laptop:~$ wget yahoo.com
--18:32:56--  http://yahoo.com/
=> `index.html'
Resolving yahoo.com... 216.109.112.135, 66.94.234.13
Connecting to yahoo.com|216.109.112.135|:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: http://www.yahoo.com/ [following]
--18:32:57--  http://www.yahoo.com/
=> `index.html'
Resolving www.yahoo.com... 69.147.114.210
Connecting to www.yahoo.com|69.147.114.210|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 9,490 (9.3K) [text/html]

100%[=====================================================>] 9,490         --.--K/s

18:32:57 (258.06 KB/s) - `index.html' saved [9490/9490]

Testing the connections - Host only networking

Now comes the acid test of whether the virtual machines and hosts can see each other.

Windows could see the individual virtual machines:

C:\Users\gaurav> ping 192.168.0.4
Pinging 192.168.0.4 with 32 bytes of data:

Reply from 192.168.0.4: bytes=32 time<1ms TTL=64
Reply from 192.168.0.4: bytes=32 time<1ms TTL=64
Reply from 192.168.0.4: bytes=32 time<1ms TTL=64
Reply from 192.168.0.4: bytes=32 time<1ms TTL=64

Ping statistics for 192.168.0.4:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 0ms, Maximum = 0ms, Average = 0ms

C:\Users\gaurav> ping 192.168.0.5
Pinging 192.168.0.5 with 32 bytes of data:

Reply from 192.168.0.5: bytes=32 time<1ms TTL=128
Reply from 192.168.0.5: bytes=32 time<1ms TTL=128
Reply from 192.168.0.5: bytes=32 time<1ms TTL=128
Reply from 192.168.0.5: bytes=32 time<1ms TTL=128

Ping statistics for 192.168.0.5:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 0ms, Maximum = 0ms, Average = 0ms

Pinging the windows host IP from inside the VMs worked beautifully:



And, pinging to the other virtual machine from each VM worked beautifully too:



Taking this a step further, since ssh was enabled on both the virtual machines, I was able to login to the individual virtual machines also:

gverma@gverma-laptop:~$ ps -ef | grep ssh
gverma    4684  4643  0 21:27 ?        00:00:00 /usr/bin/ssh-agent x-session-manager
root      5422     1  0 22:20 ?        00:00:00 /usr/sbin/sshd
gverma    5456  4864  0 22:23 pts/0    00:00:00 grep ssh
gverma@gverma-desktop:~$ ssh 192.168.0.5

gverma@192.168.0.5's password:
Linux gverma-desktop 2.6.22-14-generic #1 SMP Sun Oct 14 23:05:12 GMT 2007 i686

The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.
Last login: Mon Apr 14 22:20:55 2008 from gverma-laptop.local
gverma@gverma-desktop:~$

gverma@gverma-desktop:~$ ssh 192.168.0.4
The authenticity of host '192.168.0.4 (192.168.0.4)' can't be established.
RSA key fingerprint is a8:29:91:97:7d:99:37:6e:31:f1:06:ec:04:39:78:d7.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.0.4' (RSA) to the list of known hosts.

gverma@192.168.0.4's password:

Linux gverma-laptop 2.6.22-14-generic #1 SMP Sun Oct 14 23:05:12 GMT 2007 i686

The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.
Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.

Private networks

To make things better, the private networks were working too (thanks to the good karma accumulated in the previous article - Case study: Configuring Internal networking work for talking two linux guest OS (Ubuntu) on windows vista host ) :

gverma@gverma-laptop:~$ ping 192.168.2.1
PING 192.168.2.1 (192.168.2.1) 56(84) bytes of data.
64 bytes from 192.168.2.1: icmp_seq=1 ttl=64 time=0.200 ms
64 bytes from 192.168.2.1: icmp_seq=2 ttl=64 time=1.16 ms
64 bytes from 192.168.2.1: icmp_seq=3 ttl=64 time=0.000 ms
64 bytes from 192.168.2.1: icmp_seq=4 ttl=64 time=0.073 ms
--- 192.168.2.1 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3006ms
rtt min/avg/max/mdev = 0.000/0.358/1.160/0.468 ms

gverma@gverma-laptop:~$ ping 192.168.2.2
PING 192.168.2.2 (192.168.2.2) 56(84) bytes of data.
64 bytes from 192.168.2.2: icmp_seq=1 ttl=64 time=4.49 ms
64 bytes from 192.168.2.2: icmp_seq=2 ttl=64 time=0.368 ms
64 bytes from 192.168.2.2: icmp_seq=3 ttl=64 time=1.65 ms
64 bytes from 192.168.2.2: icmp_seq=4 ttl=64 time=3.15 ms
64 bytes from 192.168.2.2: icmp_seq=5 ttl=64 time=1.51 ms

A moment of Triumph

So here we are. Let us take a moment to sit back, relax and let reality sink it. It is really working! I am reminded of the umpteen times I had gone to forums.virtualbox.org to find the answer to this riddle and here it is, solved.

I hope this guide is of use to someone else who is trying to make a similar configuration work or just trying to understand -- what is it that we can achieve from host only networking??

Nothing could be sweeter, nothing.

Case study: Making Internal networking for talking between two linux guest OS (Ubuntu) on windows vista host

Preface

For the past few days, I had been struggling with making internal networking work on two (or more) linux guest OSes on Windows host using virtualbox.

The advantage of this setup is we can setup an internal networking lab between two or more nods using regular windows, which can be used for a variety of purposes. My primary purpose behind this setup was to simulate an environment for implementing Oracle 10gR2 RAC cluster.

As we know, setting up 10g RAC on a node needs a public network and a private network. The public network is achieved right off the bat with the NAT network configuration in Virtualbox, but the internal network setup is one that is pretty tricky.

I will give credit to both http://www.virtualbox.org/wiki/Testing_Networks and http://blogs.sun.com/manoj/entry/netowkring_with_virtualbox for the ideas tested, but more so to http://www.virtualbox.org/wiki/Testing_Networks

That being said, I would daresay that the content authoring in Virtualbox user manual about setting up internal networking on windows host deserves a little more attention. I was able to find several posts on the internet, including a particular Virtualbox wiki page, that dealt with making host only networking or internal networking work on Ubuntu host, but was not able to find the same for windows host.

Lets get to the setup now..

This is how the GUI setup looks like internal network (Adapter 1 - 2nd adapter) for the two virtual machines:

The caveat...

The first thing which we need to understand that is the Virtualbox GUI for setting up internal network is BROKEN!

You need to set it up using the command line interface (VboxManage command) Also, be aware that the internal network settings are reset by the GUI if any of the VM settings are changed, so you better do this AFTER you are done with all the other VM changes.

As per the User manual, the default internal network name is "intnet", but you can choose any other name. I chose the intnet for no particular reason.

This is what we need to do with the VboxManage command. (here, we assume that there are two guest OSes - "Gutsy" and "gutsy2"). Mind you, the virtual machine name is case-sensitive in the VboxManage command.

c:\Program Files\Innotek> VboxManage modifyvm Gutsy -nic2 intnet

c:\Program Files\Innotek> VboxManage modifyvm Gutsy -intnet2 intnet

c:\Program Files\Innotek> VboxManage modifyvm gutsy2 -nic2 intnet

c:\Program Files\Innotek> VboxManage modifyvm gutsy2 -intnet2 intnet

Default Gateway and DNS of eth1 interfaces - (used for private network as per our semantics)

If you already have the eth0 in the guest OS (linux) as NAT, it will automatically acquire a DHCP assigned IP in the subnet 10.0.2.x, and will have a default gateway of 10.0.2.2. The names server will be 10.0.2.3. With this setup, internet will work on the linux guest OS, (provided it is works on the windows host too).



We will need to make the default gateway of the private network (eth1) in guest OS (linux) as 10.0.2.2 too -- whichever is causing the internet to work.

Private IPs should be the same subnet...

Another thing to notice here is that all the guest OSes that are intended to be connected through internal network should have the private interface IP (eth1) in the SAME subnet. You can choose any private subnet like 192.168.x.x or 10.10.x.x:

Here are three valid combinations of eth1 IPs:

a) Gutsy could have 192.168.2.1, and gutsy2 could have 192.168.2.2

b) Gutsy could have 192.168.3.2, and gutsy2 could have 192.168.3.4

c) Gutsy could have 10.10.1.1, and gutsy2 could have 10.10.1.2

( I hope you get the idea).

For the sake of illustration, I chose this combination of private IPs:

• 192.168.2.1 (for linux guest OS 1 - Gutsy)


• 192.168.2.2 (for linux guest OS 1 - gutsy2)

Restart networking

After doing this, you need to restart the networking on the linux host. On Ubuntu, it is done with the sudo /etc/init.d/networking restart command:

gverma@gverma-desktop:~$ sudo /etc/init.d/networking restart
[sudo] password for gverma: ********
* Reconfiguring network interfaces...
SIOCADDRT: No such process
Failed to bring up eth1.
[ OK ]

Overview of the networking configuration

Here is how the ifconfig for Gutsy - linux guest OS 1 looks like (as you can see the eth0 IP has been acquired in 10.0.2.x subnet by the DHCP server):

And this is how it looks of Gutsy2 (as you can see, the eth0 IP has been acquired in 10.0.2.x subnet by the DHCP server):

By mistake, I have put the same image for the ifconfig output (that of Gutsy) for Gutsy2  as well. The hardware MAC address for eth1/0 should be different and so should be the the IP address for eth1 -- it should be 192.168.2.1. The eth0 IP for Gutsy2 would still be 10.0.2.x as it was of NAT type. -- Thanks, Gaurav

A variation...

Let us consider a variation here.

If the Virtual machines have only 1 virtualbox network adapter defined of type internal network, then the internal static IP within the Virtual machine would have needed to use a default gateway == host's IP default gateway.

For example, if the windows host had an IP of 192.168.0.4 (say), by virtue of being behind a router, then the default gateway of eth0 interface in in the virtual machine should be 192.168.0.1. (Again, this is assuming ONLY if you do not have a NAT type virtual interface defined for the VM.)



In my case, since I had two virtual network interfaces defined (one NAT and one internal networking type) for the VMs, I had to make the default gateway of the eth1 (private interface) of guest OS the same as that of eth0 -- which was 10.0.2.2.

The default gateway can be verified by the output of route command.

Interesting Trivia

In addition, the media state of the Virtual TAP Adapters in windows host were showing up as media disconnected, but still the internet was working for the Virtual machines AND the pings to each other were working. So, don't be fooled by the status of Virtulbox Adapters in the windows host.

gverma@gverma-desktop:~$ ping 192.168.2.1
PING 192.168.2.2 (192.168.2.2) 56(84) bytes of data.
64 bytes from 192.168.2.2: icmp_seq=1 ttl=64 time=0.031 ms
64 bytes from 192.168.2.2: icmp_seq=2 ttl=64 time=0.030 ms
64 bytes from 192.168.2.2: icmp_seq=3 ttl=64 time=0.109 ms
64 bytes from 192.168.2.2: icmp_seq=4 ttl=64 time=0.031 ms
64 bytes from 192.168.2.2: icmp_seq=5 ttl=64 time=0.036 ms
64 bytes from 192.168.2.2: icmp_seq=6 ttl=64 time=0.033 ms
64 bytes from 192.168.2.2: icmp_seq=7 ttl=64 time=0.110 ms

--- 192.168.2.2 ping statistics ---
7 packets transmitted, 7 received, 0% packet loss, time 5998ms
rtt min/avg/max/mdev = 0.030/0.054/0.110/0.035 ms

gverma@gverma-desktop:~$ ping 192.168.2.2
PING 192.168.2.1 (192.168.2.1) 56(84) bytes of data.
64 bytes from 192.168.2.1: icmp_seq=1 ttl=64 time=0.398 ms
64 bytes from 192.168.2.1: icmp_seq=2 ttl=64 time=0.304 ms
64 bytes from 192.168.2.1: icmp_seq=3 ttl=64 time=0.380 ms
64 bytes from 192.168.2.1: icmp_seq=4 ttl=64 time=7.20 ms
64 bytes from 192.168.2.1: icmp_seq=5 ttl=64 time=0.407 ms
64 bytes from 192.168.2.1: icmp_seq=6 ttl=64 time=0.272 ms
64 bytes from 192.168.2.1: icmp_seq=7 ttl=64 time=3.42 ms
64 bytes from 192.168.2.1: icmp_seq=8 ttl=64 time=0.411 ms
64 bytes from 192.168.2.1: icmp_seq=9 ttl=64 time=0.398 ms

--- 192.168.2.1 ping statistics ---
9 packets transmitted, 9 received, 0% packet loss, time 8004ms
rtt min/avg/max/mdev = 0.272/1.465/7.200/2.241 ms

Viola! The same result can be seen on gutsy2 too.

Well done!

Congratulations on setting up a private internal network between multiple virtual machines using Virtualbox. I hope this tutorial is of use to someone exploring the virtualbox tool. I personally feel it is a lot leaner and powerful, as compared to the vmware's size. It is also more robust performance wise.

If you feel this was helpful, send me a note at gaurav_verma two two [at] yahoo dot com or just leave me a comment.