The case is that I have a server on which I want to run some sort of virtualization - I don't think it's appropriate to discuss why virtualization is a go (if you think so, search the web for articles regarding this subject - there are more than plenty discussing this matter). I need at least to host a web server and a mail server - and it would be great to be able to have some sort of playground, too. My mind's almost made up in regards to the software for the web server and mail server: Ubuntu Server (or maybe FreeBSD - time will show ). But what about the virtualization software?

Besides ESXi I'm comtemplating Xen and VMware Server. Why those three - why not Microsoft's new software, Hyper-V? Well, I'm not going to use Microsoft servers so I don't really want to mix this into the environment. Besides that, Hyper-V is quite new compared to the others and still misses some features (and testing, I guess - but it's a statement without proof).

Since this server with virtualization is a hobby-project I don't want to spend too much money on software - preferably none. On top of this I like the idea of open source software. Because of this, my interest for ESXi came the moment I read that I would become free (as in money, not beer).

My server is hosted at a Hetzner (http://www.hetzner.de/), so in order to install ESXi I need to order something called LARA at a price of 25€ (it's a way of making it possible to install custom software like ESXi since they "only" provides recoverbility of different Linux flavours). As mentioned this is needed to install ESXi, whereas both Xen and VMware Server run on ordinary server software, e.g. Ubuntu Server, which doesn't require a LARA-setup (and hence not the 25€).

Comparison

The already mentioned arguments are just practical matter - what about the software? I have certain requirements such as mirroring of my data (raid 1), but this would yield an expend of several euro each month as it would cost if it should be done in real hardware raid. Because of that I'm willing to offer CPU-time and instead use software raid, i.e. mdadm in the world of Linux and FreeBSD.

Well, they all have both advantages and disadvantages in regards to my requirements:

• VMware ESXi, see [1]
  • Advantages
    • Only a 32 MB footprint ought to yield a good performance
    • The technology (ESX) is widely used and mature (but refer to the disadvantages to view this in perspective)
  • Disadvantages
    • Not open source
    • Would require a LARA-setup to install on a hosted server
    • Doesn't support software raid
    • To quote [1]:
      

      VMware states that the ESX Server product runs on "bare metal". In contrast to other VMware products, it does not run atop a third-party operating system, but instead includes its own kernel. In ESX version 3 and older, a Linux kernel is started first and is used to load a variety of specialized virtualization components, including VMware's 'vmkernel' component.

    • The full virtualization technique is ancient [2] - or to quote [2] for some of the explanation:
      

      Of course, those with the most market share will continue to sell their solutions as "more mature" and/or "enterprise ready" while continuing to improve their tools. Unfortunately, they will continue to lean on an outdated, albeit refined technology core. The core may continue to evolve, but the approach is fundamentally less efficient, and will therefore never achieve the performance of the more logical solution. It reminds me of the ice farmers' response to the refrigerator - rather than evolving their business, they tried to find better, more efficient ways to make ice, and ultimately went out of business because the technology simply wasn’t as good.

• VMware Server
  • Advantages
    • Looks easy and straight-forward with the flashy tools and wizards - seems very user friendly
    • The technology is widely used and mature (but refer to the disadvantages to view this in perspective)
  • Disadvantages
    • The same as for VMware ESXi (quotes inclusive), exept for the LARA-requirement
    • Requires an underlaying operating system (but if this doesn't run other services this doesn't seem to be a huge problem)
• Xen, see [3]
  • Advantages
    • Open source
    • The virtualization technique is called integrated virtualization or paravirtualization. This means that unlike with VMware ESXi and Server, the virtualized operating systems are aware that they are being virtualized. This means that there's no need for a SBE-mechanism (Scan-Before-Execution) to handle special or privileged CPU instructions. This is an expensive operation because every instruction has to be scanned before the execution of it, and making that expensive scan unnecessary yields greater performance
  • Disadvantages
    • As far as I've experienced, the learning-curve can seem more steep than both the one of VMware ESXi and Server (but I've also used VMware Workstation for some time). But that something is harder doesn't mean that it isn't as good . But I mainly think it's because of quite a lot new concepts - after getting my hands dirty, I'm actually very impressed of the simplicity of Xen! For reference, navigate to [4] and review the installtion process - it's not that complex after all.
    • Requires an underlaying operating system (but if this doesn't run other services this doesn't seem to be a huge problem)

Conclusion

After quite a lot reading on the web (much more than refered to in this blog entry), I've finally decided!

If I had hardware raid, the battle between ESXi and Xen was very, very close! I would not be able to distinctly answer whether I would use ESXi or Xen. In this regard I see ESXi as an old, experienced giant and Xen as a new actor (although they in no ways are inmature!). If the short-term period is weighted the most, I'd go with ESXi, but if I should you think a on the future, I'd pick Xen.

With software raid there's really no doubt after this reseach: Xen is here to stay - and I look forward to join this field of evolution!

Disclaimer: virtualization is a huge area, and I've not nearly said enough about it, nor did I compared the products in-depth for this to be a complete comparison! But besides the points in this entry, I've read a lot on the web and had a lecture in virtualization. Altogether I've been convienced to go for it with Xen, based on my requirements and needs. Maybe yours a different or have read other sites saying something else. After all there is not a definte right answer for which virtualization software to use, but the least you can do is some research on the basis of your requirements, needs, hardware etc. (and since you're reading this, you've probably already started ).

Within a week I'll post a detailed guide of how to install Xen on Ubuntu Server 8.04 with Ubuntu Server 8.04, FreeBSD, and Windows as guests - with a complex disk setup including software raid mirroring (raid 1) and LVM.

Once again: if I've missed or misunderstood something, please make me aware of it by leaving a comment!

Sources

[1]: http://en.wikipedia.org/wiki/ESXi
[2]: http://servervirtualization.blogs.techtarget.com/.../is-xen-ready-.../
[3]: http://en.wikipedia.org/wiki/Xen
[4]: http://www.howtoforge.com/high-performance-xen-on-ubuntu-8.04-amd64

In order to make it work create a machine with a 20GB SCSI disk (defaults: non-independent etc.) and chose the downloaded ISO-file with ESXi - the other options of the machine is not important because the vmx-file is to be overwritten, but you have to create the disk. Please be sure that VT is enabled - if not VMware will complain (to enable it go to your BIOS, enable it, shut down the computer entirely wait a few seconds and turn it on again). Now close the tab in Workstation , save a copy of the vmx-file, and open the original vmx-file in notepad (or equivalent) and replace its contents with:

config.version = "8"
virtualHW.version = "6"
scsi0.present = "TRUE"
scsi0.virtualDev = "lsilogic"
memsize = "1024"
scsi0:0.present = "TRUE"
scsi0:0.fileName = "Other.vmdk"
ide1:0.present = "TRUE"
ide1:0.fileName = "C:\VMware-VMvisor-InstallerCD-3.5.0_Update_2-103909.i386.iso"
ide1:0.deviceType = "cdrom-image"
floppy0.autodetect = "TRUE"
ethernet0.present = "TRUE"
ethernet0.virtualDev = "e1000"
ethernet0.wakeOnPcktRcv = "FALSE"
usb.present = "TRUE"
ehci.present = "TRUE"
sound.present = "TRUE"
sound.fileName = "-1"
sound.autodetect = "TRUE"
svga.autodetect = "TRUE"
pciBridge0.present = "TRUE"
mks.keyboardFilter = "allow"
displayName = "esxi 3.5"
guestOS = "rhel3"
nvram = "Other.nvram"
deploymentPlatform = "windows"
virtualHW.productCompatibility = "hosted"
tools.upgrade.policy = "useGlobal"
 
monitor_control.restrict_backdoor = "TRUE"
monitor_control.vt32 = "TRUE"
ide1:0.autodetect = "TRUE"
floppy0.fileName = "A:"
isolation.tools.hgfs.disable = "TRUE"
 
ethernet0.addressType = "generated"
uuid.location = "56 4d ed e0 c0 75 12 c6-f7 8d a4 27 12 c8 bc 05"
uuid.bios = "56 4d ed e0 c0 75 12 c6-f7 8d a4 27 12 c8 bc 05"
scsi0:0.redo = ""
pciBridge0.pciSlotNumber = "17"
scsi0.pciSlotNumber = "16"
ethernet0.pciSlotNumber = "32"
sound.pciSlotNumber = "33"
ehci.pciSlotNumber = "34"
ethernet0.generatedAddress = "00:0c:29:c8:bc:05"
ethernet0.generatedAddressOffset = "0"
tools.remindInstall = "TRUE"

Now change change scsi0:0.fileName, ide1:0.fileName, nvram, ethernet0.generatedAddress, uuid.location, and uuid.bios to the correct values (as in the copy of the vmx-file). Then open the machine in VMware Workstation and change the network to "Host-only". And voila - now you should be able to install ESXi!