Standard Performance Evaluation Corporation

SPEC Workstation Benchmark

SPEC/GWPG (Graphics & Workstation Performance Group) is an umbrella organization for autonomous project groups that develop graphics and workstation benchmarks and performance reporting procedures. The group supports the development of a range of graphics and workstation benchmarks that have a value to the user and vendor communities and is always looking for new groups that realize the value of working under the SPEC/GWPG charter. To this end, SPEC/GWPG is building a workstation benchmark that represents the performance characteristics of workstation applications, without requiring the full application and associated licensing to be installed on the system under test.

What is SPEC/GWPG Looking For?

In keeping with the SPEC/GWPG philosophy of characterizing performance of workstations running applications, not synthetic workloads, the group is looking for applications, algorithms, and workloads that are representative of those used by typical workstations customers in CAD/CAE, digital media and entertainment, finance, health sciences, energy and other prominent markets. The SPEC/GWPG group has established the following goals to define performance:

• To permit standardized performance measurement and evaluation by creating unambiguous, vendor-neutral measures for product evaluation and comparison.
• To contribute to the coherence of the field of performance measurement and evaluation so that vendors will be better able to present well-defined measures and customers will be better able to compare and evaluate vendors' products and environments.
• To provide mechanisms that enable vendors, customers and others to perform their own performance measurements and evaluations.
• To provide software releases to the public in a timely fashion.

Criteria for selecting tests for the Workstation Benchmark Suite

• Application or algorithm must be used by real users, or derived from a representative application from the desktop workstation market.
• Workload generated from the models must be typical for the software with respect to memory usage, scalability with number of cores and core frequencies.
• Capable of solving problems or varying sizes with an emphasis on large (>10GB) problems.
• Generate reproducible results.
• Portable to multiple hardware architectures.
• Can be compiled with the Microsoft Visual C++ compiler.

Benefits of supplying application code for the SPEC Workstation Benchmark

• The code will become the target for hardware vendors’ optimizations that will likely speed performance for the real-world application.
• The application will receive worldwide media coverage through SPEC press releases and as part of a benchmark that will be used widely for product reviews in online and print publications.
• ISVs will garner goodwill from customers who will be able to use the benchmark for workstation performance evaluation based on their actual application.
• The application will receive high-profile exposure on the SPEC website, visited by nearly one million annually.
• The application will receive direct exposure from the thousands expected to download the benchmark

How to Participate

• Step One: submitter must fill out application provided by SPEC/GWPG. You will need to supply the following information
• Name and contact information
• Name of the program/application
• Description of what the program does, how it is used, and by whom
• Description of how the program was compiled - compiler, compiler flags, libraries, etc (only Windows binaries will be accepted)
• A statement declaring that the Submitter has the authority to give SPEC the right to distribute the program/application as part of a SPEC benchmark suite
• Upon receipt of the proposal, a SPEC/GWPG member will acknowledge its receipt within 7 days. Within 4 weeks the SPEC/GWPG committee will notify the submitter as to whether the committee wishes to further evaluate the proposal.
• If SPEC GWPG wishes to further evaluate the proposal the submitter must provide a signed “Permission to Use” form for the submitted code. At the option of the submitter, source code may be provided to other SPEC/GWPG members for the purpose of further optimization, analysis and/or integration into the SPEC workstation benchmark .
• Step two:
• Submitter must provide the compiled code along with a workload to drive the application. The code must be compatible with Windows 7 64-bit.
• The submitter must also provide a description of how to run the application and workload, and the expected results.
• Step Three:
• If the program and workload meet all of the selection criteria, it may be recommended for inclusion in the SPEC/GWPG workstation benchmark.
• If it is included in the benchmark suite, the submitter will get a hardy (though possibly virtual) pat on the back and their name in the credits portion of the benchmark.
• SPEC/GWPG may choose at any time, at its sole discretion, to discontinue work on a submission.

For more information, contact [email protected].