Here are some quick performance results from VMware servers with 10 Gigabit Ethernet cards (from Sun with an Intel 82598 controller. The servers were Sun X4150 with Dual-Quadcore processors.
Network tests
I fired up two test VMs with Suse Linux Enterprise Desktop 10 (64 bit) and used netperf to measure TCP bandwidth between them (the exact command used was netperf -H <target ip> -t TCP_STREAM -C -c -l 60 -f M).
VM to VM (both VMs placed on the same physical server): 230 MB/s
VM to VM (each VM on a different physical server using 10 GigE through a single switch): 250 MB/s
VM to a physical server with a 10 GigE card: 540 MB/s
Physical server to VM: 290 MB/s
Looks like there is a bottleneck somewhere on the receive side but the performance is sufficient even for the most demanding (business) applications. Remember: this is out-of-the-box performance, no tuning.
Disk performance to a NFS based datastore on a Netapp
Here I simple used IOzone in the Linux-VM to write and read several GBs of data to a virtual disk (formatted with XFS). The virtual disk was placed on a NFS datastore (Netapp 6070 with Ontap 7.2.4, 2×16 disk FC aggregate).
Write: 160 MB/s
Read: 160 MB/s (200 MB/s from filer cache)
I think these numbers can be improved as neither the filer nor the server were fully utilized, so I expect more VMs will bring additional performance. Perhaps the receive-side problem mentioned above also plays a role.
Summary
Generally this is exactly what we expected: to be able to provide good performance to many virtual servers which do network and/or disk-I/O only periodically without overprovisioning the whole infrastructure.
At the moment there is exactly one dual-port 10 GigE card in every VMware server and there are two physical connections to two different switches - a much cleaner setup than the 12 (!) GigE interfaces per server we had before 
How do your numbers look like? Comments welcome (moderated).
8 comments ↓
Where you using the vmware tools driver in Suse? What about send and recieve buffers? Jumbo frames? What switches are you connecting to?
Did you enable VMDq on the NIC?
http://www.huliq.com/36119/intel-server-adapters-based-on-intel-82598-10-gigabit-ethernet-controller
Im very interested in 10Gbit cards since we’re also looking at NetApp for our storage and don’t want to add 10 NICs to every server…
VMware tools were enabled, no jumbo frames. I didn’t do anything to the default buffer size and the switch was a HP ProCurve 6400cl (8-Port 10 GigE).
Any idea how to “enable” VMDq? I didn’t see any options in VMware. BTW the link you referenced mentions 8 VMs on a server - I used only a single one. It might be interesting to redo the test with more VMs
Sorry, regarding VMDq it was this link:
http://viroptics.blogspot.com/2008/03/intel-10gbit-nic-with-vmdq-doubles.html
I suppose VMDq might need another driver from Intel…
Do you install any hardware agent on the Sun x4150 server or only use the ILO for hardware monitoring?
Im looking at the x4150 or the x4450 but are a little bit worried about the hardware monitoring.
[…] de rest van de benchmark hier: http://21stcenturystorage.cebis.net « VMware: Top Things you should know […]
Big README folks
- All results are *without* VMDq - here is how to enable it:
http://kb.vmware.com/selfservice/search.do?cmd=displayKC&docType=kc&externalId=1004278
.. but I’ll have to file a RFC to our VMware admin before I can retest. Stay tuned
Why didn’t you include “Physical server to Physical server” in the list ?
Mainly because the “physical” servers are completely different machines using different network cards (Myricom).
I re-did the tests after enabling NetQueue in VMware - the results for a *single* VM are pretty much the same. With multiple processes netperf can send up to 910 MB/s to multiple physical machines but the receive side is still at 250 M/s.
You must log in to post a comment.