VMware Relaxes Benchmark Policy

While you were trying to decide which virtualization solution to go with (VMware, Microsoft, Xen, Parallels, etc.), did you search for published benchmarking results to try and compare the different products? If so, did you find any? My guess is probably not. The reason being, VMware has had a pretty strict policy about publishing benchmarks. After the release of VMware Workstation 3.2, they introduced a clause in the End User License Agreement (EULA) stating you cannot publish benchmarking results from their products. With such an agreement in place, you can understand why finding such a benchmark comparison would prove difficult.

But are things about to change? According to the blog site run by Richard Garsthagen (the Technical Marketing Manager for VMware EMEA), it appears as though the release of VMware Infrastructure 3 may be ushering in a new benchmarking policy at VMware. Although everyone reads a software package's EULA before clicking that little "I accept" checkbox during the installation, Richard offers up a snippet that explains the policy change:

You may use the Software to conduct internal performance testing and benchmarking studies, the results of which you (and not unauthorized third parties) may publish or publicly disseminate; provided that VMware has reviewed and approved of the methodology, assumptions and other parameters of the study. Please contact VMware at benchmark@vmware.com to request such review.

I am a little disappointed that VMware didn't go further with relaxing the policy. With the battle cry for standardization that has been going on in the virtualization community as of late, would it not have made more sense to agree on a set of benchmarking tools and a common testing methodology, rather than asking to review and approve each study being conducted?

In any case, I hope this opens up a whole new arena for people to publish and share their benchmarking results of various virtualization products across a number of different hardware platforms. It will be interesting to see the results.

Posted by David Marshall on June 28, 2006 07:52 PM