SPEC CPU®2017 Floating Point Speed Result

Copyright 2017-2020 Standard Performance Evaluation Corporation

Hewlett Packard Enterprise (Test Sponsor: HPE)

Superdome Flex 280
(2.80 GHz, Intel Xeon Gold 6328HL)

SPECspeed®2017_fp_base = 18600

SPECspeed®2017_fp_peak = 18700

CPU2017 License: 3 Test Date: Nov-2020
Test Sponsor: HPE Hardware Availability: Nov-2020
Tested by: HPE Software Availability: Apr-2020

Benchmark result graphs are available in the PDF report.

Hardware
CPU Name: Intel Xeon Gold 6328HL
  Max MHz: 4300
  Nominal: 2800
Enabled: 64 cores, 4 chips
Orderable: 2, 4, 8 chip(s)
Cache L1: 32 KB I + 32 KB D on chip per core
  L2: 1 MB I+D on chip per core
  L3: 22 MB I+D on chip per chip
  Other: None
Memory: 3 TB (24 x 128 GB 4Rx4 PC4-3200AA-L,
running at 2933)
Storage: 1 x 1 TB SATA HDD, 7.2K RPM
Other: None
Software
OS: Red Hat Enterprise Linux 8.2 (Ootpa)
Kernel 4.18.0-193.el8.x86_64
Compiler: C/C++: Version 19.1.1.217 of Intel C/C++
Compiler Build 20200306 for Linux;
Fortran: Version 19.1.1.217 of Intel Fortran
Compiler Build 20200306 for Linux;
Parallel: Yes
Firmware: HPE Firmware Bundle Version 1.0.142 released Oct-2020
File System: xfs
System State: Run level 3 (multi-user)
Base Pointers: 64-bit
Peak Pointers: 64-bit
Other: jemalloc memory allocator V5.0.1;
HPE Foundation Software 2.4,
Build 734.0820.200723T0100.a.rhel82hpe-200723T0100
Power Management: BIOS set to prefer performance at the cost of additional power usage

Results Table

Benchmark Base Peak
Threads Seconds Ratio Seconds Ratio Seconds Ratio Threads Seconds Ratio Seconds Ratio Seconds Ratio
SPECspeed®2017_fp_base 18600
SPECspeed®2017_fp_peak 18700
Results appear in the order in which they were run. Bold underlined text indicates a median measurement.
603.bwaves_s 64 68.2 8650 68.8 8580 69.0 8550 64 68.2 8650 68.8 8580 69.0 8550
607.cactuBSSN_s 64 78.0 2140 78.7 2120 79.3 2100 64 78.0 2140 78.7 2120 79.3 2100
619.lbm_s 64 37.8 1390 38.6 1360 37.8 1390 64 37.8 1390 38.6 1360 37.8 1390
621.wrf_s 64 1010 1310 1010 1300 1010 1310 64 1010 1310 1010 1310 1000 1320
627.cam4_s 64 56.9 1560 56.7 1560 56.5 1570 64 56.9 1560 56.7 1560 56.5 1570
628.pop2_s 64 2370 50.2 2360 50.3 2360 50.3 64 2370 50.2 2360 50.3 2360 50.3
638.imagick_s 64 93.8 1540 94.3 1530 93.9 1540 64 93.8 1540 94.3 1530 93.9 1540
644.nab_s 64 46.1 3790 46.1 3790 46.3 3770 64 43.9 3980 43.7 4000 43.7 4000
649.fotonik3d_s 64 71.3 1280 71.5 1280 71.5 1280 64 71.4 1280 71.1 1280 70.9 1290
654.roms_s 64 63.5 2480 62.3 2530 61.6 2550 64 63.5 2480 62.3 2530 61.6 2550

Operating System Notes

 Stack size set to unlimited using "ulimit -s unlimited"
 Transparent Huge Pages enabled by default
 Prior to runcpu invocation
 Filesystem page cache synced and cleared with:
 sync; echo 3>       /proc/sys/vm/drop_caches
 Tuned-adm profile was set to Throughput-Performance using "tuned-adm profile throughput-performance"

 HPE Foundation Software (HFS) is a collection of software packages designed to make the HPE Superdome Flex family of
 servers easier to use for customers. This software is compatible with RHEL, SLES, and Oracle Linux only. It is
 highly recommended all users install HFS for the Superdome Flex system. More details, and a revision history list, can
 be found at: https://support.hpe.com/hpsc/swd/public/detail?swItemId=MTX_b48de5f6a8a041f0ae985825a5#tab-history

Environment Variables Notes

Environment variables set by runcpu before the start of the run:
KMP_AFFINITY = "granularity=fine,compact"
LD_LIBRARY_PATH = "/home/cpu2017/lib/intel64:/home/cpu2017/je5.0.1-64"
MALLOC_CONF = "retain:true"
OMP_STACKSIZE = "192M"

General Notes

 Binaries compiled on a system with 1x Intel Core i9-7980XE CPU + 64GB RAM
 memory using Redhat Enterprise Linux 8.0
 NA: The test sponsor attests, as of date of publication, that CVE-2017-5754 (Meltdown)
 is mitigated in the system as tested and documented.
 Yes: The test sponsor attests, as of date of publication, that CVE-2017-5753 (Spectre variant 1)
 is mitigated in the system as tested and documented.
 Yes: The test sponsor attests, as of date of publication, that CVE-2017-5715 (Spectre variant 2)
 is mitigated in the system as tested and documented.
 jemalloc, a general purpose malloc implementation
 built with the RedHat Enterprise 7.5, and the system compiler gcc 4.8.5
 sources available from jemalloc.net or https://github.com/jemalloc/jemalloc/releases

Platform Notes

 BIOS Configuration:
  Workload Profile set to HPC
  Intel Hyper-Threading set to Disabled
  Workload Profile set to Custom
  	Minimum Processor Idle Power Core C-State set to C6 State
   Minimum Processor Idle Power Package C-State set to Package C6 (non-retention) State
   LLC Prefetch set to Enabled

 Sysinfo program /home/cpu2017/bin/sysinfo
 Rev: r6365 of 2019-08-21 295195f888a3d7edb1e6e46a485a0011
 running on ch-620.fchst.rdlabs.hpecorp.net Wed Dec  2 21:36:16 2020

 SUT (System Under Test) info as seen by some common utilities.
 For more information on this section, see
    https://www.spec.org/cpu2017/Docs/config.html#sysinfo

 From /proc/cpuinfo
    model name : Intel(R) Xeon(R) Gold 6328HL CPU @ 2.80GHz
       4  "physical id"s (chips)
       64 "processors"
    cores, siblings (Caution: counting these is hw and system dependent. The following
    excerpts from /proc/cpuinfo might not be reliable.  Use with caution.)
       cpu cores : 16
       siblings  : 16
       physical 0: cores 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
       physical 1: cores 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
       physical 2: cores 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
       physical 3: cores 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

 From lscpu:
      Architecture:        x86_64
      CPU op-mode(s):      32-bit, 64-bit
      Byte Order:          Little Endian
      CPU(s):              64
      On-line CPU(s) list: 0-63
      Thread(s) per core:  1
      Core(s) per socket:  16
      Socket(s):           4
      NUMA node(s):        4
      Vendor ID:           GenuineIntel
      CPU family:          6
      Model:               85
      Model name:          Intel(R) Xeon(R) Gold 6328HL CPU @ 2.80GHz
      Stepping:            11
      CPU MHz:             3242.222
      CPU max MHz:         4300.0000
      CPU min MHz:         1000.0000
      BogoMIPS:            5600.13
      Virtualization:      VT-x
      L1d cache:           32K
      L1i cache:           32K
      L2 cache:            1024K
      L3 cache:            22528K
      NUMA node0 CPU(s):   0-15
      NUMA node1 CPU(s):   16-31
      NUMA node2 CPU(s):   32-47
      NUMA node3 CPU(s):   48-63
      Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
      pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp
      lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid
      aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16
      xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave
      avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3
      invpcid_single intel_ppin ssbd mba ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi
      flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm
      cqm mpx rdt_a avx512f avx512dq rdseed adx smap clflushopt clwb intel_pt avx512cd
      avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total
      cqm_mbm_local avx512_bf16 dtherm ida arat pln pts pku ospke avx512_vnni md_clear
      flush_l1d arch_capabilities

 /proc/cpuinfo cache data
    cache size : 22528 KB

 From numactl --hardware  WARNING: a numactl 'node' might or might not correspond to a
 physical chip.
   available: 4 nodes (0-3)
   node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
   node 0 size: 772622 MB
   node 0 free: 765888 MB
   node 1 cpus: 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
   node 1 size: 774139 MB
   node 1 free: 769175 MB
   node 2 cpus: 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47
   node 2 size: 774112 MB
   node 2 free: 770953 MB
   node 3 cpus: 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63
   node 3 size: 773619 MB
   node 3 free: 772514 MB
   node distances:
   node   0   1   2   3
     0:  10  16  16  24
     1:  16  10  24  16
     2:  16  24  10  16
     3:  24  16  16  10

 From /proc/meminfo
    MemTotal:       3168761828 kB
    HugePages_Total:       0
    Hugepagesize:       2048 kB

 /usr/bin/lsb_release -d
    Red Hat Enterprise Linux release 8.2 (Ootpa)

 From /etc/*release* /etc/*version*
    hpe-foundation-release: HPE Foundation Software 2.4, Build
    734.0820.200723T0100.a.rhel82hpe-200723T0100
    os-release:
       NAME="Red Hat Enterprise Linux"
       VERSION="8.2 (Ootpa)"
       ID="rhel"
       ID_LIKE="fedora"
       VERSION_ID="8.2"
       PLATFORM_ID="platform:el8"
       PRETTY_NAME="Red Hat Enterprise Linux 8.2 (Ootpa)"
       ANSI_COLOR="0;31"
    redhat-release: Red Hat Enterprise Linux release 8.2 (Ootpa)
    system-release: Red Hat Enterprise Linux release 8.2 (Ootpa)
    system-release-cpe: cpe:/o:redhat:enterprise_linux:8.2:ga

 uname -a:
    Linux ch-620.fchst.rdlabs.hpecorp.net 4.18.0-193.el8.x86_64 #1 SMP Fri Mar 27 14:35:58
    UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

 Kernel self-reported vulnerability status:

 itlb_multihit:                            Not affected
 CVE-2018-3620 (L1 Terminal Fault):        Not affected
 Microarchitectural Data Sampling:         Not affected
 CVE-2017-5754 (Meltdown):                 Not affected
 CVE-2018-3639 (Speculative Store Bypass): Mitigation: Speculative Store Bypass disabled
                                           via prctl and seccomp
 CVE-2017-5753 (Spectre variant 1):        Mitigation: usercopy/swapgs barriers and __user
                                           pointer sanitization
 CVE-2017-5715 (Spectre variant 2):        Mitigation: Enhanced IBRS, IBPB: conditional,
                                           RSB filling
 tsx_async_abort:                          Not affected

 run-level 3 Dec 2 06:58

 SPEC is set to: /home/cpu2017
    Filesystem            Type  Size  Used Avail Use% Mounted on
    /dev/mapper/rhel-home xfs   876G  190G  687G  22% /home

 From /sys/devices/virtual/dmi/id
     BIOS:    HPE Bundle:1.0.142 SFW:008.000.189.000.2010080501 10/08/2020
     Vendor:  HPE
     Product: Superdome Flex 280
     Product Family: 1590PID02020001
     Serial:  5UF0130953

 Additional information from dmidecode follows.  WARNING: Use caution when you interpret
 this section. The 'dmidecode' program reads system data which is "intended to allow
 hardware to be accurately determined", but the intent may not be met, as there are
 frequent changes to hardware, firmware, and the "DMTF SMBIOS" standard.
   Memory:
     24x Hynix HMABAGL7ABR4N-XN 128 GB 4 rank 3200
     24x NO DIMM NO DIMM

 (End of data from sysinfo program)

Compiler Version Notes

==============================================================================
C               | 619.lbm_s(base, peak) 638.imagick_s(base, peak)
                | 644.nab_s(base, peak)
------------------------------------------------------------------------------
Intel(R) C Intel(R) 64 Compiler for applications running on Intel(R) 64,
  Version 19.1.1.217 Build 20200306
Copyright (C) 1985-2020 Intel Corporation.  All rights reserved.
------------------------------------------------------------------------------

==============================================================================
C++, C, Fortran | 607.cactuBSSN_s(base, peak)
------------------------------------------------------------------------------
Intel(R) C++ Intel(R) 64 Compiler for applications running on Intel(R) 64,
  Version 19.1.1.217 Build 20200306
Copyright (C) 1985-2020 Intel Corporation.  All rights reserved.
Intel(R) C Intel(R) 64 Compiler for applications running on Intel(R) 64,
  Version 19.1.1.217 Build 20200306
Copyright (C) 1985-2020 Intel Corporation.  All rights reserved.
Intel(R) Fortran Intel(R) 64 Compiler for applications running on Intel(R)
  64, Version 19.1.1.217 Build 20200306
Copyright (C) 1985-2020 Intel Corporation.  All rights reserved.
------------------------------------------------------------------------------

==============================================================================
Fortran         | 603.bwaves_s(base, peak) 649.fotonik3d_s(base, peak)
                | 654.roms_s(base, peak)
------------------------------------------------------------------------------
Intel(R) Fortran Intel(R) 64 Compiler for applications running on Intel(R)
  64, Version 19.1.1.217 Build 20200306
Copyright (C) 1985-2020 Intel Corporation.  All rights reserved.
------------------------------------------------------------------------------

==============================================================================
Fortran, C      | 621.wrf_s(base, peak) 627.cam4_s(base, peak)
                | 628.pop2_s(base, peak)
------------------------------------------------------------------------------
Intel(R) Fortran Intel(R) 64 Compiler for applications running on Intel(R)
  64, Version 19.1.1.217 Build 20200306
Copyright (C) 1985-2020 Intel Corporation.  All rights reserved.
Intel(R) C Intel(R) 64 Compiler for applications running on Intel(R) 64,
  Version 19.1.1.217 Build 20200306
Copyright (C) 1985-2020 Intel Corporation.  All rights reserved.
------------------------------------------------------------------------------

Base Compiler Invocation

C benchmarks:

 icc 

Fortran benchmarks:

 ifort 

Benchmarks using both Fortran and C:

 ifort   icc 

Benchmarks using Fortran, C, and C++:

 icpc   icc   ifort 

Base Portability Flags

603.bwaves_s:  -DSPEC_LP64 
607.cactuBSSN_s:  -DSPEC_LP64 
619.lbm_s:  -DSPEC_LP64 
621.wrf_s:  -DSPEC_LP64   -DSPEC_CASE_FLAG   -convert big_endian 
627.cam4_s:  -DSPEC_LP64   -DSPEC_CASE_FLAG 
628.pop2_s:  -DSPEC_LP64   -DSPEC_CASE_FLAG   -convert big_endian   -assume byterecl 
638.imagick_s:  -DSPEC_LP64 
644.nab_s:  -DSPEC_LP64 
649.fotonik3d_s:  -DSPEC_LP64 
654.roms_s:  -DSPEC_LP64 

Base Optimization Flags

C benchmarks:

 -m64   -std=c11   -xCORE-AVX512   -ipo   -O3   -no-prec-div   -qopt-prefetch   -ffinite-math-only   -qopt-mem-layout-trans=4   -qopenmp   -DSPEC_OPENMP   -mbranches-within-32B-boundaries 

Fortran benchmarks:

 -m64   -Wl,-z,muldefs   -DSPEC_OPENMP   -xCORE-AVX512   -ipo   -O3   -no-prec-div   -qopt-prefetch   -ffinite-math-only   -qopt-mem-layout-trans=4   -qopenmp   -nostandard-realloc-lhs   -mbranches-within-32B-boundaries   -L/usr/local/jemalloc64-5.0.1/lib   -ljemalloc 

Benchmarks using both Fortran and C:

 -m64   -std=c11   -Wl,-z,muldefs   -xCORE-AVX512   -ipo   -O3   -no-prec-div   -qopt-prefetch   -ffinite-math-only   -qopt-mem-layout-trans=4   -qopenmp   -DSPEC_OPENMP   -mbranches-within-32B-boundaries   -nostandard-realloc-lhs   -L/usr/local/jemalloc64-5.0.1/lib   -ljemalloc 

Benchmarks using Fortran, C, and C++:

 -m64   -std=c11   -Wl,-z,muldefs   -xCORE-AVX512   -ipo   -O3   -no-prec-div   -qopt-prefetch   -ffinite-math-only   -qopt-mem-layout-trans=4   -qopenmp   -DSPEC_OPENMP   -mbranches-within-32B-boundaries   -nostandard-realloc-lhs   -L/usr/local/jemalloc64-5.0.1/lib   -ljemalloc 

Peak Compiler Invocation

C benchmarks:

 icc 

Fortran benchmarks:

 ifort 

Benchmarks using both Fortran and C:

 ifort   icc 

Benchmarks using Fortran, C, and C++:

 icpc   icc   ifort 

Peak Portability Flags

Same as Base Portability Flags

Peak Optimization Flags

C benchmarks:

619.lbm_s:  basepeak = yes 
638.imagick_s:  basepeak = yes 
644.nab_s:  -m64   -std=c11   -Wl,-z,muldefs   -xCORE-AVX512   -ipo   -O3   -no-prec-div   -qopt-prefetch   -ffinite-math-only   -qopt-mem-layout-trans=4   -qopenmp   -DSPEC_OPENMP   -mbranches-within-32B-boundaries   -L/usr/local/jemalloc64-5.0.1/lib   -ljemalloc 

Fortran benchmarks:

603.bwaves_s:  basepeak = yes 
649.fotonik3d_s:  -m64   -Wl,-z,muldefs   -prof-gen(pass 1)   -prof-use(pass 2)   -DSPEC_SUPPRESS_OPENMP   -DSPEC_OPENMP   -ipo   -xCORE-AVX512   -O3   -no-prec-div   -qopt-prefetch   -ffinite-math-only   -qopt-mem-layout-trans=4   -qopenmp   -nostandard-realloc-lhs   -mbranches-within-32B-boundaries   -L/usr/local/jemalloc64-5.0.1/lib   -ljemalloc 
654.roms_s:  basepeak = yes 

Benchmarks using both Fortran and C:

621.wrf_s:  -m64   -std=c11   -Wl,-z,muldefs   -prof-gen(pass 1)   -prof-use(pass 2)   -ipo   -xCORE-AVX512   -O3   -no-prec-div   -qopt-prefetch   -ffinite-math-only   -qopt-mem-layout-trans=4   -DSPEC_SUPPRESS_OPENMP   -qopenmp   -DSPEC_OPENMP   -mbranches-within-32B-boundaries   -nostandard-realloc-lhs   -L/usr/local/jemalloc64-5.0.1/lib   -ljemalloc 
627.cam4_s:  basepeak = yes 
628.pop2_s:  basepeak = yes 

Benchmarks using Fortran, C, and C++:

607.cactuBSSN_s:  basepeak = yes 

The flags files that were used to format this result can be browsed at
http://www.spec.org/cpu2017/flags/HPE-Platform-Flags-Intel-V1.3-CLX-revC.html,
http://www.spec.org/cpu2017/flags/Intel-ic19.1u1-official-linux64_revA.html.

You can also download the XML flags sources by saving the following links:
http://www.spec.org/cpu2017/flags/HPE-Platform-Flags-Intel-V1.3-CLX-revC.xml,
http://www.spec.org/cpu2017/flags/Intel-ic19.1u1-official-linux64_revA.xml.