Standard Performance Evaluation Corporation

SPEC Releases New Benchmark for Evaluating Java Virtual Machine (JVM) Server Performance

WARRENTON, Va., June 5, 2000 – The Standard Performance Evaluation Corp. (SPEC) has released SPECjbb2000, a new benchmark for evaluating the performance of servers running typical Java business applications. The benchmark can be used across several versions of UNIX, Windows/NT, Linux and other operating systems.

SPECjbb2000 represents an order processing application for a wholesale supplier. Written in Java, it was developed with participation from computer vendors, system integrators, universities, research organizations, publishers and consultants. Systems integrators and end users can use the benchmark to evaluate performance of hardware and software aspects of Java Virtual Machine (JVM) servers. Hardware vendors can use SPECjbb2000 to analyze their platform’s scalability when running Java applications. Software vendors can use it to evaluate the efficiency of their JVMs, just-in-time compilers (JITs), garbage collectors, thread implementations, and operating systems.

SPECjbb2000 – the "jbb" stands for Java business benchmark – is implemented as a Java program emulating a three-tier client/server system with emphasis on the middle tier. All three tiers are implemented within a single JVM. The tiers mimic a typical business application, where users in tier one generate inputs that result in the execution of business logic in the middle tier (tier two), which in turn calls to a database on the third tier. SPECjbb2000 does not depend on any package beyond the Java run-time environment (JRE).

"SPEC is continuing what it started with its Java client benchmark, SPECjvm98," says Kaivalya Dixit, SPEC president. "There are other Java server benchmarks available, but only SPECjbb2000 provides SPEC’s methodology for ensuring consistent, repeatable results."

Modeling Business Applications
"Companies worldwide are developing Java application frameworks for deploying commercial systems," says Anne Troop, chair of the SPECjbb2000 project committee. "SPECjbb2000 is an excellent yardstick to compare servers that use business applications written in Java."

SPECjbb2000 models a wholesale company, with warehouses that serve a number of districts. Customers initiate a set of operations such as placing new orders or requesting the status of an existing order. Additional operations are generated within the company, such as processing orders for delivery, entering customer payments, and checking stock levels.

SPECjbb2000 assigns one active customer per warehouse. A warehouse is a unit of about 25MB of data stored in binary trees (Btrees). Warehouses map directly to Java threads. As the number of warehouses increases during the full benchmark run, so does the number of threads.

While SPECjbb2000 is not a complete OLTP benchmark, it is a good stand-in for a large business application. The benchmark exercises the implementations of JVM, JIT, garbage collection, threads and some aspects of the operating system. It also measures the performance of CPUs, caches, memory hierarchy and the scalability of shared memory processors (SMPs).

Dependable Results
SPECjbb2000 measures the throughput of the underlying Java platform, which is the rate at which business operations are performed per second. The metric is a composite number derived by averaging peak performance throughput with the results from adding warehouses one at a time until there are twice as many warehouses as server processors.<</p>

A typical benchmark run takes about three minutes per warehouse. SPECjbb2000 measures throughput in a fixed amount of time, so faster machines do more work in the allotted time. Testing by SPEC on a variety of platforms shows that results are consistent and repeatable.

Vendor Reporting
SPECjbb2000 performance results are available on SPEC’s web site at http://www.spec.org. SPECjbb2000 licensees may publish their own results in accordance with SPEC’s run and reporting rules. To run SPECjbb2000, users need a machine with a minimum of 256MB physical memory and a JVM environment supporting Java or Java 2 API. JVMs generally need 32MB or more of disk space for the installed software and the benchmark itself requires 10MB of disk space.

Availability
SPECjbb2000 is available now on CD-ROM from SPEC for $400. Discounts are available for eligible universities and non-profit organizations.

SPECjbb2000 was developed by SPEC's Open Systems Group (OSG). SPEC is a non-profit corporation formed to establish, maintain and endorse a standardized set of relevant benchmarks that can be applied to the newest generation of high-performance computers. The organization’s membership includes more than 30 computer vendors, systems integrators, publishers, consultants, and leading universities and research centers from around the world. The SPEC Web site http://www.spec.org offers, without cost, the broadest set of performance ratings for evaluating computer systems. In addition to OSG, SPEC benchmark development groups include the High-Performance Group (HPG), and the Graphics Performance Characterization (GPC) Group. For more information on SPECjbb2000 and other benchmarks, contact SPEC, 6585 Merchant Place, Suite 100, Warrenton, Va. 20187; phone: 540-349-7878; fax: 540-349-5992; e-mail: info@spec.org; http://www.spec.org.

Press Contacts:
Bob Cramblitt or Ellen Gooch
Cramblitt & Company
tel. 1-919-481-4599
[email protected]