SPEC Delivers Major Update to SPECaccel Benchmark Suite

GAINESVILLE, Va., Nov. 9, 2023 — The Standard Performance Evaluation Corporation (SPEC), the trusted global leader in computing benchmarks, today announced the availability of the SPECaccel 2023 benchmark suite, a major update to the SPEC ACCEL v1.4 benchmark. The SPECaccel 2023 benchmark suite measures the performance of key system components using computationally intensive parallel applications running under the OpenACC and OpenMP target offloading APIs. The updated suite includes larger workloads appropriate for modern accelerators, as well as expanded support for OpenMP target offload, such as metadirectives, which enables more relevant performance measurement of today's high performance systems.

The SPECaccel 2023 benchmark suite uses many of the original benchmarks from SPEC ACCEL V1.4, containing 12 application benchmarks ported to use OpenACC, OpenMP using "loop" constructs, OpenMP with "distribute" constructs favoring collapsing all loop levels, or OpenMP with "distribute" applied to outer loops and "simd" applied to inner loops. Users choose the model based on the compiler and accelerator they are using.

"SPECaccel 2023 is a much anticipated suite for not only research but also for teaching purposes. Engaging students with a benchmark suite developed collaboratively by academic researchers, industry and lab partners is key as it exposes them to the orchestration of the suite, usage of directives, the intense experimentation and evaluation of results on more than just one platform, the numerous decisions that are made along the way before an application could be bundled into the suite," said Dr. Sunita Chandrasekaran, Associate Professor, Dept. of CIS, University of Delaware. "The SPECaccel 2023 benchmark suite, just like the previous versions, is very useful to stress test development versions of OpenMP offload and OpenACC compilers and file bugs sooner than later. Overall, such a comprehensive suite is time-critical for software that forever plays catch-up with the continuously evolving hardware."

The SPECaccel 2023 benchmark suite is typically used as:

• A straightforward tool for calibrating and objectively comparing the performance of various accelerators, which can be important when making purchasing decisions or doing regression testing.
• A research tool for developers to better understand how OpenMP and OpenACC implementation decisions impact performance.
• A training tool for how to use OpenACC and OpenMP Offload to achieve high levels of performance. For example, instructors may use it to train students in the use of profilers or have students modify the directives to understand how they impact performance. An instructor can also remove part of the directives from the source and assign students to port the code.

New in the SPECaccel 2023 Benchmark Suite:

• New workloads using up to 16 GB of device memory, can better utilize modern accelerators.
• The OpenACC and OpenMP ports have been merged into a single source, and results from different models are directly comparable. Users can select which model to use for optimal performance and compare compiler implementations between models.
• OpenMP now includes three selectable methods controlled via OpenMP's new metadirective feature: "loop" construct, "distribute/parallel" construct collapsing nested loops, and "simd" on inner loops. These methods allow for differences in compiler implementation and architectures and help reduce bias in how OpenMP is applied. Users are also able to modify the directive in peak to allow for custom tuning.
• Alternate source files (Src.alt) are available that provide support for OpenMP compilers that have not yet implemented metadirectives.

"In the highly complex world of high performance computing, it's extremely difficult to benchmark accelerator performance across systems because most code is optimized for vendor-specific solutions, making accurate and reliable comparisons impossible," said Ron Lieberman, SPEC High Performance Group Chair. "The SPECaccel 2023 benchmark includes realistic workloads derived from scientific applications that can be run on any system to provide a reliable comparison of accelerator performance."

Included with the SPECaccel 2023 Benchmark Suite:

• Source code for the benchmarks
• Data sets
• A toolset for compiling, running, validating and reporting on the benchmarks
• Pre-compiled tools for Linux on a variety of architectures (x86_64, Power, ARM)
• Source code for the SPECaccel 2023 tools, for systems not covered by the pre-compiled tools
• Documentation
• Run and reporting rules

Available for Immediate Download

The SPECaccel 2023 benchmark suite is available for immediate download from SPEC for $2,000. Academic and research users can apply for a no-cost license. Visit SPEC for more information.

About SPEC

SPEC is a non-profit organization that establishes, maintains and endorses standardized benchmarks and tools to evaluate performance for the newest generation of computing systems. Its membership comprises more than 120 leading computer hardware and software vendors, educational institutions, research organizations, and government agencies worldwide.

###

Media contact:
Brigit Valencia
360.597.4516
[email protected]

Images available upon request.

SPEC® and SPECaccel® are trademarks of the Standard Performance Evaluation Corporation. All other product and company names herein may be trademarks of their registered owners.