Case study: Making Internal networking for talking between two linux guest OS (Ubuntu) on windows vista host

Preface

For the past few days, I had been struggling with making internal networking work on two (or more) linux guest OSes on Windows host using virtualbox.

The advantage of this setup is we can setup an internal networking lab between two or more nods using regular windows, which can be used for a variety of purposes. My primary purpose behind this setup was to simulate an environment for implementing Oracle 10gR2 RAC cluster.

As we know, setting up 10g RAC on a node needs a public network and a private network. The public network is achieved right off the bat with the NAT network configuration in Virtualbox, but the internal network setup is one that is pretty tricky.

I will give credit to both http://www.virtualbox.org/wiki/Testing_Networks and http://blogs.sun.com/manoj/entry/netowkring_with_virtualbox for the ideas tested, but more so to http://www.virtualbox.org/wiki/Testing_Networks

That being said, I would daresay that the content authoring in Virtualbox user manual about setting up internal networking on windows host deserves a little more attention. I was able to find several posts on the internet, including a particular Virtualbox wiki page, that dealt with making host only networking or internal networking work on Ubuntu host, but was not able to find the same for windows host.

Lets get to the setup now..

This is how the GUI setup looks like internal network (Adapter 1 - 2nd adapter) for the two virtual machines:

The caveat...

The first thing which we need to understand that is the Virtualbox GUI for setting up internal network is BROKEN!

You need to set it up using the command line interface (VboxManage command) Also, be aware that the internal network settings are reset by the GUI if any of the VM settings are changed, so you better do this AFTER you are done with all the other VM changes.

As per the User manual, the default internal network name is "intnet", but you can choose any other name. I chose the intnet for no particular reason.

This is what we need to do with the VboxManage command. (here, we assume that there are two guest OSes - "Gutsy" and "gutsy2"). Mind you, the virtual machine name is case-sensitive in the VboxManage command.

c:\Program Files\Innotek> VboxManage modifyvm Gutsy -nic2 intnet

c:\Program Files\Innotek> VboxManage modifyvm Gutsy -intnet2 intnet

c:\Program Files\Innotek> VboxManage modifyvm gutsy2 -nic2 intnet

c:\Program Files\Innotek> VboxManage modifyvm gutsy2 -intnet2 intnet

Default Gateway and DNS of eth1 interfaces - (used for private network as per our semantics)

If you already have the eth0 in the guest OS (linux) as NAT, it will automatically acquire a DHCP assigned IP in the subnet 10.0.2.x, and will have a default gateway of 10.0.2.2. The names server will be 10.0.2.3. With this setup, internet will work on the linux guest OS, (provided it is works on the windows host too).



We will need to make the default gateway of the private network (eth1) in guest OS (linux) as 10.0.2.2 too -- whichever is causing the internet to work.

Private IPs should be the same subnet...

Another thing to notice here is that all the guest OSes that are intended to be connected through internal network should have the private interface IP (eth1) in the SAME subnet. You can choose any private subnet like 192.168.x.x or 10.10.x.x:

Here are three valid combinations of eth1 IPs:

a) Gutsy could have 192.168.2.1, and gutsy2 could have 192.168.2.2

b) Gutsy could have 192.168.3.2, and gutsy2 could have 192.168.3.4

c) Gutsy could have 10.10.1.1, and gutsy2 could have 10.10.1.2

( I hope you get the idea).

For the sake of illustration, I chose this combination of private IPs:

• 192.168.2.1 (for linux guest OS 1 - Gutsy)


• 192.168.2.2 (for linux guest OS 1 - gutsy2)

Restart networking

After doing this, you need to restart the networking on the linux host. On Ubuntu, it is done with the sudo /etc/init.d/networking restart command:

gverma@gverma-desktop:~$ sudo /etc/init.d/networking restart
[sudo] password for gverma: ********
* Reconfiguring network interfaces...
SIOCADDRT: No such process
Failed to bring up eth1.
[ OK ]

Overview of the networking configuration

Here is how the ifconfig for Gutsy - linux guest OS 1 looks like (as you can see the eth0 IP has been acquired in 10.0.2.x subnet by the DHCP server):

And this is how it looks of Gutsy2 (as you can see, the eth0 IP has been acquired in 10.0.2.x subnet by the DHCP server):

By mistake, I have put the same image for the ifconfig output (that of Gutsy) for Gutsy2  as well. The hardware MAC address for eth1/0 should be different and so should be the the IP address for eth1 -- it should be 192.168.2.1. The eth0 IP for Gutsy2 would still be 10.0.2.x as it was of NAT type. -- Thanks, Gaurav

A variation...

Let us consider a variation here.

If the Virtual machines have only 1 virtualbox network adapter defined of type internal network, then the internal static IP within the Virtual machine would have needed to use a default gateway == host's IP default gateway.

For example, if the windows host had an IP of 192.168.0.4 (say), by virtue of being behind a router, then the default gateway of eth0 interface in in the virtual machine should be 192.168.0.1. (Again, this is assuming ONLY if you do not have a NAT type virtual interface defined for the VM.)



In my case, since I had two virtual network interfaces defined (one NAT and one internal networking type) for the VMs, I had to make the default gateway of the eth1 (private interface) of guest OS the same as that of eth0 -- which was 10.0.2.2.

The default gateway can be verified by the output of route command.

Interesting Trivia

In addition, the media state of the Virtual TAP Adapters in windows host were showing up as media disconnected, but still the internet was working for the Virtual machines AND the pings to each other were working. So, don't be fooled by the status of Virtulbox Adapters in the windows host.

gverma@gverma-desktop:~$ ping 192.168.2.1
PING 192.168.2.2 (192.168.2.2) 56(84) bytes of data.
64 bytes from 192.168.2.2: icmp_seq=1 ttl=64 time=0.031 ms
64 bytes from 192.168.2.2: icmp_seq=2 ttl=64 time=0.030 ms
64 bytes from 192.168.2.2: icmp_seq=3 ttl=64 time=0.109 ms
64 bytes from 192.168.2.2: icmp_seq=4 ttl=64 time=0.031 ms
64 bytes from 192.168.2.2: icmp_seq=5 ttl=64 time=0.036 ms
64 bytes from 192.168.2.2: icmp_seq=6 ttl=64 time=0.033 ms
64 bytes from 192.168.2.2: icmp_seq=7 ttl=64 time=0.110 ms

--- 192.168.2.2 ping statistics ---
7 packets transmitted, 7 received, 0% packet loss, time 5998ms
rtt min/avg/max/mdev = 0.030/0.054/0.110/0.035 ms

gverma@gverma-desktop:~$ ping 192.168.2.2
PING 192.168.2.1 (192.168.2.1) 56(84) bytes of data.
64 bytes from 192.168.2.1: icmp_seq=1 ttl=64 time=0.398 ms
64 bytes from 192.168.2.1: icmp_seq=2 ttl=64 time=0.304 ms
64 bytes from 192.168.2.1: icmp_seq=3 ttl=64 time=0.380 ms
64 bytes from 192.168.2.1: icmp_seq=4 ttl=64 time=7.20 ms
64 bytes from 192.168.2.1: icmp_seq=5 ttl=64 time=0.407 ms
64 bytes from 192.168.2.1: icmp_seq=6 ttl=64 time=0.272 ms
64 bytes from 192.168.2.1: icmp_seq=7 ttl=64 time=3.42 ms
64 bytes from 192.168.2.1: icmp_seq=8 ttl=64 time=0.411 ms
64 bytes from 192.168.2.1: icmp_seq=9 ttl=64 time=0.398 ms

--- 192.168.2.1 ping statistics ---
9 packets transmitted, 9 received, 0% packet loss, time 8004ms
rtt min/avg/max/mdev = 0.272/1.465/7.200/2.241 ms

Viola! The same result can be seen on gutsy2 too.

Well done!

Congratulations on setting up a private internal network between multiple virtual machines using Virtualbox. I hope this tutorial is of use to someone exploring the virtualbox tool. I personally feel it is a lot leaner and powerful, as compared to the vmware's size. It is also more robust performance wise.

If you feel this was helpful, send me a note at gaurav_verma two two [at] yahoo dot com or just leave me a comment.

Migrating from vmware to Virtualbox: Making NAT networking type work with SuSE Linux as guest OS and Windows Host

In the past two weeks, I have discovered a lot about Virtualbox. I wanted an easy alternative to vmware server/player and I had heard at a friend's party that virtualbox was another virtualization software in the market and it was free. Vmware workstation needs a paid license, so I decided to create a working virtual machine in which internet would work.

A difficult virtualbox journey.. from vmware

Intially, I dabbled with all sorts of network setups, but nothing seemed to work.

In the starting, I used static IPs -- 192.168.203.21 for eth0 and 10.10.10.21 for eth1, mostly because I had seen these IPs being used for NAT and Host only network types in Vmware. (later on, I realized that the network functionalities of Vmware and virtualbox are not the same. So host only in vmware may not be the same as host only in virtualbox).

This very setting was working for the SLES9 OS install done in Vmware, with the same setting in virtualbox, the network would not work.

The route command was taking a long time to print the output.

The "route -n" command was coming out with output instantly though, which clearly pointed that the issue was with the name resolution or accessing the names server.

Trivia...

During my efforts to make NAT networking work, on checking the c:\> ipconfig /all output in windows, I could see that the status of Virtualbox TAP adapter (first network interface) was showing as "media disconnected".

I started reading different posts on the internet frantically and came across one very cryptic post at http://eligere.wordpress.com/2008/03/12/virtualbox-windows-tap-disconnected-message-remove/ , which had an interesting hack to make the status of Virtualbox TAP adapter as always connected or with 0.0.0.0 IP when the VM was not started.

The solution was to open up the Device manager in windows, right click on the Virtualbox TAP adapter, go to advanced tab, and select "Always connected" value for the Media Status option.

After this, the media state always shows as connected with a DHCP IP assigned to it in the subnet 169.254.xx.xx. If you read http://en.wikipedia.org/wiki/Zeroconf, you will come to know that 169.254.0.0 - 169.254.255.255 is a special range reserved for link-local set of addresses.

After making this change, the networking in the SLES9 guest OS STILL DID NOT work!!

Random ramblings...

While searching on the virtualbox forums, I found a lot of posts talking of making Ubuntu work with Virtualbox.

Well, I thought. At least ubuntu seems to work, if not SuSE. So, I decided to virtualbox with Ubuntu guest OS on a windows vista at home.

I discovered that Ubuntu created a DHCP network interface for the first virtualbox TAP adapter, with an IP of 10.0.2.3, and networking was working just fine out of the box!

Trying DHCP for eth0 on SuSE linux..

This lead me on to think that Virtualbox seemed to have a liking for DHCP networking interfaces, rather than static IPs.

Using the "yast2" utility in the previous SLES9 virtual machine, I changed the eth0 network interface setting to DHCP and found that the ifconfig output had assigned an IP of 10.0.2.15 to eth0.

What's more, the route command was not hanging anymore!

Somehow, a default gateway of 10.0.2.2 was being taken up by the OS. (although /etc/resolv.conf had an entry of 192.168.203.2 for the nameserver. Now, that was strange.)

But, internet was not working yet.

Making internet work on SLES9 VM using virtualbox (in office intranet)....

To make networking work on SLES9 virtual machine, I still had do the following things:

1) Copied the DNS servers from the windows host into /etc/resolv.conf . This could have been done through the yast2 utility too.

If I did not do this, then the route command was printing the expected output, but the nslookup for external websites like yahoo.com was failing or hanging.

2) Changed default gateway as 10.0.2.2 through yast2 utility

3) Since I was behind in the office network, I had to setup the corporate proxy too through Yast2. It should be visible in the http_proxy environment variable in the SLES9 guest OS. This was crucial to make networking work. If this was not set, the ARP name resolution was happening, but the connection was hanging.

A really easy way to check the networking connection was to use $ wget yahoo.com, which would tell the entire story very well.

4) Another thing You COULD consider doing (but MAY NOT be necessary) could be syncing up (typing in the same MAC id) the MAC address of the Windows TAP Adapater in the Network setting of the Virtual machine definition in Virtualbox (before the VM starts). This may be relevant if you generated a new MAC id of the virtual network interface or you recreated a new virtualbox network interface.

You may also have to change the new MAC id in name of the file /etc/sysconfig/network/ifcfg-eth-id-<MAC:ID> , which controls the internal names of the network interface.

I started going on this path and did this consistency naming for the first network interface, but did not bother doing so for the other one, and even then networking was working just fine.

Experimenting with 2 NAT type Virtualbox TAP adapters

Then I tried configuring the guest OS VM with 2 NATs and this is how the ifconfig output came up like:

The route command came up like:

I was able to browse external websites ....

An important learning about how NAT works in Virtualbox..

Summing it up, I made an important discovery (for me) about how NAT works in virtualbox.

In reality, the NAT setup was really, really easy and worked for most people. To make it work, all that was needed to be done was let the Guest OS get an IP in the subnet 10.0.2.x through DHCP, have a default gateway of 10.0.2.2 and a names server of 10.0.2.3. With this, everything was hunky dory.

Had I chosen multiple NAT type Virtualbox TAP adapters for the Virtual machine (you can choose upto 4), the third and fourth ones would have acquired IPs in the subnet 10.0.3.x and 10.0.4.x. This is assuming eth2 and eth3 would have been of type DHCP.

Extending this concept to Static IPs..

When I discussed this finding with Srikanth, the sysadmi of my wife's company, he said that since DHCP server is giving an IP in the subnet range 10.0.2.x, one should be able to setup any static IP value for that network interface in the same range!

The more I thought about it, the more it seemed to make sense. And then, try it out I did. And work it did. (Yes, sir). I put eth0 as static ip of 10.0.2.5, default gateway 10.0.2.2, names server 10.0.2.3 and networking worked fine.

So basically, each NAT interface allows you to choose any IP in the range 10.0.N.x, where N >=2 and x ranges from 4 and 255 (leaving off 10.0.N.[1-3] for special functions of default gateway/names server).

Redemption..

Having experimented and dealt in depth with the NAT network interface of virtualbox, I was happy to have learnt something of value in the networking arena, which was an area of improvement for me.

What's more, I learnt that its easier to make networking work in Virtualbox than it is in VMware, provided you let the OS choose the IP in its desired magical subnet -- 10.0.N.x.