SPEC Search Programs — How You Can Contribute to the Future of Computing

By Mathew Colgrove, SPEC Communications Committee Chair

SPEC, a nonprofit organization, develops benchmarks that evaluate the performance and energy consumption of the newest generation of computing systems. These benchmarks help hardware vendors gauge how their products perform against the competition and target the areas that need improvement. They also enable buyers to make reliable comparisons between products, so they can purchase the right ones for their needs.

SPEC believes that the most effective computing benchmarks are developed based on how various user communities run actual applications. To enable us to do this, SPEC regularly conducts Search Programs that encourage those outside of SPEC to contribute applications, workloads, or models that will enable us to build more comprehensive and more applicable benchmarks that will better serve their communities.

We currently have two ongoing Search Programs.

SPEC CPU® v8 Benchmark Search Program

The SPEC CPU v8 Benchmark Search Program is designed to help SPEC locate applications, complete with source code and datasets, that can be used for the next CPU-intensive benchmark suite, which is currently under development within SPEC and designated the SPEC CPU® v8 benchmark. SPEC CPU benchmarks provide performance metrics for comparing systems using known compute-intensive workloads, with emphasis on the system's processors, memory hierarchy, and compilers.

If you have a compute-intensive application that fits the criteria for a good candidate, please consider sharing it with us. For each step of the process that a submission passes, SPEC offers compensation to the Program Submitter. A submission that passes all the steps and is included in the next SPEC CPU benchmark suite will receive up to $9000 overall and a license for the new benchmark suite when released.

The SPEC CPU v8 Benchmark Search Program is currently scheduled to stop accepting contributions on March 13, 2023. To see the complete Search Program rules, as well as the complete criteria for good application candidates, visit the SPEC CPU® v8 Benchmark Search Program page.

SPEChpc Weak Scaling Benchmark Search Program

The SPEChpc Weak Scaling Benchmark Search Program is designed to help SPEC locate applications that can be used as components of a new SPEChpc benchmark. The SPEChpc Weak Scaling benchmark will be used to evaluate the throughput performance of large-scale HPC-based systems by allowing users of the benchmarks to size the workload for their system.

A good candidate application would include MPI+OpenMP, MPI+OpenACC or both, is portable across multiple systems, and can scale to an arbitrary size. If your application is a good fit, please consider sharing it with us. For each step of the process that a submission passes, SPEC/HPG offers compensation to the Program Submitter. A submission that passes all the steps and is included in the next SPEChpc benchmark suite will receive $5000 overall and a license for the new benchmark suite when released.

The SPEChpc Weak Scaling Benchmark Search Program is currently scheduled to stop accepting contributions on July 13, 2023. To see the complete Search Program rules, visit the SPEChpc Weak Scaling Benchmark Search Program page.

Conclusion

Your contributions to SPEC will enable us to improve our benchmarks, which in turn will drive computing system performance improvements in future products. It’s a true win-win for users and hardware vendors! For nearly 35 years, SPEC has helped improve each generation of computing solutions. Here is your chance to be a part of that ongoing effort.

[Back to SPEC blog]