The SPECvirt Datacenter 2021 Benchmark — A User's Story

By David Schmidt, Virtualization Committee Chair

John, an IT Manager at a rapidly growing financial services company, needed to specify the hardware required for the company's new private cloud deployment. His goal was to ensure a sufficient number of virtual machines to support a significant increase in the firm's development team.

John had an RFP out to three manufacturers for the servers he would need, and each had provided him with multiple test systems highlighting their proposed hardware and software solutions. Although he wanted to thoroughly stress the components of these systems to help him determine the best value, he was concerned that it could take several weeks, perhaps a month or two, to set up and run the various tools he had in his lab.

One of those tools was the SPEC VIRT_SC® 2013 benchmark. While it was an effective and vendor-agnostic benchmark, it did require significant configuration and tuning, so John went to the SPEC website to see if there was an updated version of it. He was pleased to find the SPECvirt Datacenter 2021 benchmark, which offered several benefits compared to the 2013 version – and to any of the other tools John had in his arsenal.

The multi-host SPECvirt® Datacenter 2021 benchmark measures the performance of a scaled-out datacenter using simulated and real-life workloads to measure the overall efficiency of virtualization solutions and their management environments. The benchmark models typical, modern-day usage of virtualized infrastructure, such as virtual machine (VM) resource provisioning, cross-node load balancing including management operations such as VM migrations, and VM power on/off. It exercises datacenter operations under load, and it dynamically provisions new workload tiles via a VM template or by powering on existing VMs. As load reaches maximum capacity of the cluster, hosts are added to the cluster to measure scheduler efficiency.

Of particular interest to John was that the SPECvirt Datacenter 2021 benchmark makes benchmarking faster and easier. Other tools require manually creating the VM, installing the operating system into it, adjusting specific OS tuning settings, installing workload applications, and generating the workload data. This process is time-consuming, complicated and highly prone to error. To address this, the SPECvirt Datacenter 2021 benchmark is delivered as a pre-built appliance template containing the software and scripts needed to create the controller, workload driver clients, and workload VMs for the benchmark's operation. The software is pre-loaded and pre-configured, minimizing the need for intervention and reducing implementation time and effort.

Because of John's past experience with the SPEC VIRT_SC 2013 benchmark and SPEC's continued reputation for delivering high-quality benchmark updates, he quickly purchased a license for the SPECvirt Datacenter 2021 benchmark. He downloaded and imported the template into his hypervisor cluster containing the servers to be tested and into the cluster to be used for the client drivers. After creating the director VM from the template on his client cluster, he was able to utilize the internal scripts to configure the director and create his client driver VMs. He ran built-in setup scripts to deploy from the director all the workload VMs needed for the benchmark onto one set of servers, storage, and networks. By adjusting the number of tiles to run, then running the benchmark until his target configuration no longer achieved the required quality of service, he determined how much capacity the configuration could handle. He then ran the benchmark on all the other test configurations and determined which supported the greatest number of VMs. John was also able to modify other provided scripts to call user-defined commands to collect additional performance-monitoring data from his hypervisor for more detailed analysis.

Because there was so little manual setup and configuring to do, SPECvirt Datacenter 2021 benchmark saved John untold hours of work. With the optimal configuration for the planned number of VMs determined, he quickly made a purchasing decision, slashing the time required for building out his new private cloud environment.

[Back to SPEC blog]