Performance Summary:

Validation Errors:

•   No Validation Errors Found

Physical Configuration:

Virtual Configuration:

Notes:

Physical System Notes

• BIOS settings used:
• Power Profile set to Maximum Performance
• Thermal Configuration set to Maximum Cooling
• Turbo Mode:Enable
• The test sponsor attests, as of date of publication, that CVE-2017-5754 (Meltdown) is mitigated in the system as tested and documented:
• No: (host, guests)
• The test sponsor attests, as of date of publication, that CVE-2017-5753 (Spectre variant 1) is mitigated in the system as tested and documented:
• No: (host, guests)
• The test sponsor attests, as of date of publication, that CVE-2017-5715 (Spectre variant 2) is mitigated in the system as tested and documented:
• No: (host, guests)

Storage Notes

• 2 x 300GB internal SAS drivers were connected to the UIS-RAIN-1000-M2 controller,built as RAID1 and used for OS
• 2 x D3700 disk enclosures were connected to the HP Smart Array P431 controllers and P441 controllers
• One D3700 contained 25 x 200 GB MLC SSD drives
• Enclosure was partitioned and used as follows:
• Create 1 x RAID5 array with 13 x SSD drives,and used for Tile1-7 VMs, including vm image and datastore
• Create 1 x RAID5 array with 12 x SSD drives,and used for Tile8-12 VMs, including vm image and datastore
• The other D3700 contained 25 x 200 GB MLC SSD drives
• Enclosure was partitioned and used as follows:
• Create 3 x RAID5 arrays,and two have 8 x SSD drives,and the third has 9 x SSD drivers.
• One is used for Tile13-16 VMs including vm image and datastore, the second for Tile17-20 VMs including vm image and datastore, and the third for Tile21-26 VMs including vm images and datastore.

Virtualization Software Notes

• The following lines were added to /etc/sysctl.conf:
• kernel.core_uses_pid = 1
• kernel.panic_on_oops = 1
• kernel.sched_min_granularity_ns = 10000000
• kernel.sched_wakeup_granularity_ns = 15000000
• kernel.sched_latency_ns = 20000000
• kernel.sched_migration_cost_ns = 150000
• kernel.numa_balancing = 0
• vm.dirty_background_ratio = 5
• vm.dirty_ratio = 10
• vm.swappiness = 0
• net.ipv6.conf.all.disable_ipv6 = 1
• net.core.netdev_max_backlog = 500000
• fs.inotify.max_user_watches = 65536
• net.ipv4.conf.default.promote_secondaries = 1
• net.ipv4.conf.all.promote_secondaries = 1
• net.core.somaxconn = 8192
• net.ipv4.tcp_max_tw_buckets = 500000
• net.ipv4.conf.default.arp_filter = 1
• fs.aio-max-nr = 70656
• The following boot options were used:
• quiet nmi_watchdog=0 intel_iommu=on iommu=pt selinux=0 clocksource=tsc highres=off tsc=reliable rdshell nosoftlockup intel_idle.max_cstate=0 mce=ignore_ce ipv6.disable=1 nohz=on nohz_full=1-87 rcunocbs=1-87 cgroup_disable=memory,cpu,cpuacct,blkio,hugetlb
• The following services are disabled:
• bootlogd grub-common ipmievd logd openvswitch-switch restore-pci-driver restore-sriov stop-bootlogd stop-bootlogd-single sudo urandom virtlogd x11-common
• Network configuration is as follows:
• 1 Intel Corporation 82599EB 10-Gigabit Network Connection for internal communication between WEBserver & INFRAserver of tile10 and tile24, a maximum of 16 virtual functions were configured
• 1 Intel Corporation 82599EB 10-Gigabit Network Connection for internal communication between other WEBserver & INFRAserver of other tiles, a maximum of 63 virtual functions were configured
• 4 Intel Corporation 82599EB 10-Gigabit Network Connection for the Web, App, DB, Mail, Infra and Batch VMs, a maximum of 63 virtual functions were configured.
• All the guest VMs use Virtual Functions
• Each VM is given a virtual function that is routed to the NUMA node to which the VM's VCPUs are pinned
• VMs are bound to host CPUs via their XML configuration in the following manner:
• All VMs of tiles 1-12, infraserver25-26, webserver25-26 are bound to NUMA node 0
• All VMs of tiles 13-24 are bound to NUMA node 1
• mailserver25-26, batchserver25-26, appserver25-26, dbserver25 are bound to all cpus
• App and DB VMs have a cputune shares value of 16384, Infra and Mail VMs have a cputune shares value of 64, Batch VMs have a cputune shares value of 512, Web VMs have a cputune shares value of 3072
• VM configurations are as follows:
• All VMs are managed with libvirt and created using virsh define & virsh start
• All VMs are created with the following xml template, exceptions noted below template.
• <domain type='kvm'>
• <name>$vm_name</name>
• <memory>$vm_memory</memory>
• <currentMemory>$vm_memory</currentMemory>
• <memtune>
• <priority>>$mem_priority </priority>
• </memtune>
• <vcpu cpuset='$host-cpu-list'>$vm_cpu</vcpu>
• <cputune>
• <shares>$vm_cpushare</shares>
• </cputune>
• <numatune>
• <memory mode='strict' nodeset='$vm_node'/>
• </numatune>
• <os>
• <type arch='x86_64' machine='pc-i440fx-2.7'>hvm</type>
• </os>
• <features>
• <acpi/>
• <apic/>
• <pae/>
• </features>
• <cpu mode='custom' match='exact'>
• <model fallback='forbid'>SandyBridge</model>
• <vendor>Intel</vendor>
• <topology sockets='1' cores='$vm_cpu' threads='1'/>
• <gurantee unit='MHz'>0</gurantee>
• </cpu>
• <clock offset='utc'>
• <timer name='pit' tickpolicy='delay'/>
• </clock>
• <on_poweroff>destroy</on_poweroff>
• <on_reboot>restart</on_reboot>
• <on_crash>destroy</on_crash>
• <tools upgrade='manual'/>
• <devices>
• <emulator>/usr/bin/kvm</emulator>
• <disk type='block' device='disk'>
• <driver name='qemu' type='raw' cache='none' io='native'/>
• <source dev='$vm_disk' />
• <target dev='vda' bus='virtio'/>
• </disk>
• <interface type='hostdev' managed='yes'>
• <mac address='$mac_address'/>
• <driver name='vfio'/>
• <label>'ethx' </label>
• <source>
• <address type='pci' domain='0x0000' bus='$bus' slot='$slot' function='$function'/>
• </source>
• </interface>
• <memballoon model='none'>
• </devices>
• </domain>
• The following section is repeated for the second PCI device used in the infra and web VMs:
• <interface type='hostdev' managed='yes'>
• <mac address='$mac_address'/>
• <driver name='vfio'/>
• <label>'ethx' </label>
• <source>
• <address type='pci' domain='0x0000' bus='$bus' slot='$slot' function='$function'/>
• </source>
• </interface>
• The following section is repeated for the second disk used in the infra, mail and db VMs for their data:
• <disk type='block' device='disk'>
• <driver name='qemu' type='raw' cache='none' io='native'/>
• <source dev='$vm_disk' />
• <target dev='vdb' bus='virtio'/>
• </disk>
• VMs' nice values were setted after startup via the renice command to the following settings:
• app='-19'
• db='-19'
• web='19'
• infra='19'
• mail='19'
• batch='-20'

Web Server VM Notes

• The following boot options were used:
• nosoftlockup intel_idle.max_cstate=0 mce=ignore_ce selinux=0 audit=0 elevator=noop clocksource=tsc tsc=reliable watchdog=0 hpet=disable nohz=on cgroup_disable=memory,cpu,blkio
• The following services are disabled:
• abrtd,atd,htcacheclean,httpd,ip6tables,iptables,irqbalance,kdump,mdmonitor,netconsole,netfs,nfs,nfslock,nginx-debug,ntpd,ntpdate,psacct,rdisc,restorecond,rngd,rpcgssd,rpcsvcgssd,saslauthd,smartd,blk-availability,jexec, lvm2-monitor
• /etc/fstab changes:
• root file-system options: noatime,barrier=0
• nfs file-system defaults options: async,noatime,actimeo=14400,nocto
• MTU=9000 for eth1 (internal network).
• Contents for nginx configuration file /etc/nginx/nginx.conf:
• user nginx;
• worker_processes 4;
• worker_cpu_affinity 00000001 00000001 00000010 00000010;
• worker_rlimit_nofile 65535;
• pid /var/run/nginx.pid;
• events {
• use epoll;
• worker_connections 65535;
• multi_accept on;
• accept_mutex off;
• }
• http {
• include /etc/nginx/mime.types;
• default_type application/octet-stream;
• log_format common '$remote_addr - $remote_user [$time_local] $request'
• '"$status" $body_bytes_sent ';
• access_log /var/log/access.log common gzip buffer=4m flush=1m;
• sendfile on;
• sendfile_max_chunk 1m;
• tcp_nopush on;
• tcp_nodelay on;
• keepalive_timeout 120;
• open_file_cache max=204800 inactive=20s;
• open_file_cache_min_uses 1;
• open_file_cache_valid 30s;
• charset utf-8;
• server_names_hash_bucket_size 128;
• client_header_buffer_size 2k;
• large_client_header_buffers 4 4k;
• include /etc/nginx/conf.d/*.conf;
• }
• Contents of nginx configuration file /etc/nginx/conf.d/default.conf:
• server {
• listen 80 backlog=8192;
• server_name localhost;
• location / {
• root /var/www/html;
• index index.html index.htm index.php;
• error_page 500 502 503 504 /50x.html;
• location = /50x.html {
• root /usr/share/nginx/html;
• }
• location ~ .*\.(html|htm|gif|jpg|jpeg|bmp|png|ico|txt|js|css)$
• {
• root /var/www/html;
• expires 100d;
• }
• location ~ ^/images/ {
• root /var/www/html/support;
• expires 100d;
• }
• location ~ \.php$ {
• root html;
• fastcgi_pass unix:/dev/shm/php-cgi.sock;
• fastcgi_index index.php;
• fastcgi_param SCRIPT_FILENAME /var/www/html$fastcgi_script_name;
• include fastcgi_params;
• }
• }
• Contents of nginx configuration file /etc/nginx/conf.d/example_ssl.conf:
• server {
• listen 443 ssl;
• server_name localhost;
• ssl on;
• ssl_certificate /etc/pki/tls/certs/localhost.crt;
• ssl_certificate_key /etc/pki/tls/private/localhost.key;
• ssl_protocols SSLv3 TLSv1 TLSv1.1 TLSv1.2;
• ssl_session_cache shared:SSL:1m;
• ssl_session_timeout 5m;
• ssl_ciphers ALL:!kEDH!ADH:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP;
• ssl_prefer_server_ciphers on;
• location / {
• root /var/www/html;
• index index.html index.htm index.php;
• }
• location ~ .*\.(html|htm|gif|jpg|jpeg|bmp|png|ico|txt|js|css)$
• {
• root /var/www/html;
• expires 100d;
• }
• location ~ ^/images/ {
• root /var/www/html/support;
• expires 100d;
• }
• location ~ \.php$ {
• root html;
• fastcgi_pass unix:/dev/shm/php-cgi.sock;
• fastcgi_index index.php;
• fastcgi_param SCRIPT_FILENAME /var/www/html$fastcgi_script_name;
• include fastcgi_params;
• }
• }
• The following lines were changed in /etc/php-fpm.d/www.conf:
• listen = /dev/shm/php-cgi.sock
• listen.owner = nginx
• listen.group = nginx
• user = nginx
• group = nginx
• pm = static
• pm.max_children = 64
• Following items were added to /etc/sysctl.conf:
• fs.file-max = 500000
• net.core.somaxconn = 81920
• net.ipv4.tcp_max_tw_buckets = 1500000
• net.ipv4.conf.default.arp_filter = 1
• net.ipv4.conf.all.arp_filter = 1
• net.ipv4.tcp_tw_reuse = 1
• net.ipv4.tcp_keepalive_time = 120
• net.ipv4.tcp_keepalive_intvl = 30
• net.ipv4.tcp_timestamps = 0
• vm.swappiness = 0
• net.ipv4.tcp_fin_timeout=60
• net.ipv4.tcp_tw_recycle=1
• The following items were added to /etc/bashrc:
• ulimit -n 100000 > /dev/null 2>&1
• The tuned-0.2.19 package was installed and activated with virtual-guest profile
• ixgbevf module option used: InterruptThrottleRate=2000
• /etc/security/limits.conf changes:
• hard and soft nofile set to 102400
• hard and soft nproc set to 102400

Application Server VM Notes

• The following boot options were used:
• nosoftlockup intel_idle.max_cstate=0 mce=ignore_ce selinux=0 audit=0 elevator=noop clocksource=tsc tsc=reliable watchdog=0 hpet=disable cgroup_disable=memory,cpu,blkio
• The following services are disabled:
• abrt-ccpp,abrtd,acpid,atd,auditd,ip6tables,iptables,irqbalance,kdump,lvm2-monitor,mdmonitor,netconsole,netfs,ntpd,ntpdate,psacct,quota_nld,rdisc,restorecond,rngd,saslauthd,smartd,blk-availability
• /etc/fstab changes:
• root file-system options: barrier=0
• There are two WebSphere profiles, AppSrv01 for SPECjAppServer2004 and AppSrv02 for SPEC Emulator
• Heap size for AppSrv01 has been changed to 2048 MB
• Following items were added to /etc/sysctl.conf:
• fs.file-max = 500000
• net.core.somaxconn = 8192
• net.ipv4.tcp_max_tw_buckets = 500000
• net.ipv4.conf.default.arp_filter = 1
• net.ipv4.conf.all.arp_filter = 1
• vm.swappiness=0
• The following items were added to /etc/bashrc:
• ulimit -n 100000 > /dev/null 2>&1
• The tuned-0.2.19 package was installed and activated with virtual-guest profile
• ixgbevf module option used: InterruptThrottleRate=7000
• /etc/security/limits.conf changes:
• hard and soft memlock set to 50000000
• hard and soft nproc set to 100000
• * - nofile 1048576

Mail Server VM Notes

• The following boot options were used:
• nosoftlockup intel_idle.max_cstate=0 mce=ignore_ce selinux=0 audit=0 elevator=noop clocksource=tsc tsc=reliable watchdog=0 hpet=disable nohz=on cgroup_disable=memory,cpu,blkio
• The following services are disabled:
• abrtd,atd,ip6tables,iptables,irqbalance,kdump,mdmonitor,netconsole,netfs,ntpd,ntpdate,psacct,rdisc,restorecond,rngd,saslauthd,smartd,svnserve,blk-availability,jexec,lvm2-monitor
• /etc/fstab changes:
• root file-system options: barrier=0
• datadisk file-system defaults options:noatime,barrier=0,data=writeback
• Dovecot was compiled with default options and the following dovecot.conf was used:
• dict {
• }
• !include conf.d/*.conf
• base_dir = /var/run/dovecot/
• disable_plaintext_auth = no
• log_path = /var/log/dovecot.log
• mail_location = dbox:~/
• mail_debug = no
• mmap_disable = yes
• mail_fsync = never
• mdbox_rotate_size = 8192
• passdb {
• args = scheme=plain /usr/local/etc/dovecot/passwd.dovecot
• driver = passwd-file
• }
• postmaster_address = [email protected]
• service auth {
• user = root
• }
• service imap-login {
• inet_listener imap {
• port = 143
• }
• process_min_avail = 10
• service_count = 0
• }
• service imap {
• process_limit = 1024
• }
• ssl = no
• ssl_cert = </etc/pki/dovecot/certs/dovecot.pem
• ssl_key = </etc/pki/dovecot/private/dovecot.pem
• userdb {
• driver = passwd
• }
• Following items were added to /etc/sysctl.conf:
• net.ipv6.conf.all.disable_ipv6=1
• fs.file-max = 500000
• net.core.somaxconn = 8192
• net.ipv4.tcp_max_tw_buckets = 500000
• net.ipv4.conf.default.arp_filter = 1
• net.ipv4.conf.all.arp_filter = 1
• net.core.optmem_max = 1000000
• net.core.rmem_default = 1000000
• net.core.rmem_max = 1000000
• net.core.wmem_default = 1000000
• net.core.wmem_max = 1000000
• net.ipv4.tcp_mem = 10000000 10000000 10000000
• net.ipv4.tcp_rmem = 10000000 10000000 10000000
• net.ipv4.tcp_wmem = 10000000 10000000 10000000
• The following items were added to /etc/bashrc:
• ulimit -n 100000 > /dev/null 2>&1
• ulimit -u 10240
• The tuned-0.2.19 package was installed and activated with virtual-guest profile
• ixgbevf module option used: InterruptThrottleRate=1000
• /etc/security/limits.conf changes:
• hard and soft nofile set to 102400
• hard and soft nproc set to 102400

Database Server VM Notes

• The following boot options were used:
• nosoftlockup mce=ignore_ce selinux=0 audit=0 elevator=noop clocksource=tsc tsc=reliable watchdog=0 hpet=disable cgroup_disable=memory,cpu,blkio
• The following services are disabled:
• abrtd,atd,ip6tables,iptables,irqbalance,kdump,mdmonitor,mysql,netconsole,netfs,ntpd,ntpdate,psacct,rdisc,restorecond,rngd,saslauthd,smartd,blk-availability,lvm2-monitor
• /etc/fstab changes:
• root file-system options: barrier=0
• datadisk file-system defaults options:noatime,nodiratime,nobarrier
• Following items were added to /etc/sysctl.conf:
• fs.file-max = 500000
• net.core.somaxconn = 32384
• net.ipv4.tcp_max_tw_buckets = 1000000
• net.ipv4.conf.default.arp_filter = 1
• net.ipv4.conf.all.arp_filter = 1
• vm.swappiness=0
• vm.nr_hugepages = 12288
• vm.hugetlb_shm_group = 901
• vm.nr_overcommit_hugepages=512
• kernel.sem = 500 64000 400 512
• kernel.shmmax = 137438953472
• kernel.shmmni = 11118096
• kernel.shmall = 8294967296
• kernel.randomize_va_space=0
• The following items were added to /etc/bashrc:
• ulimit -n 100000 > /dev/null 2>&1
• The tuned-0.2.19 package was installed and activated with virtual-guest profile
• ixgbevf module option used: InterruptThrottleRate=7000
• Database tuning options:
• db2 alter bufferpool IBMDEFAULTBP size 5500000
• db2 alter tablespace roll_space NO FILE SYSTEM CACHING
• db2 alter tablespace C_cust_space NO FILE SYSTEM CACHING
• db2 alter tablespace C_supp_space NO FILE SYSTEM CACHING
• db2 alter tablespace C_site_space NO FILE SYSTEM CACHING
• db2 alter tablespace C_parts_space NO FILE SYSTEM CACHING
• db2 alter tablespace C_custinv_space NO FILE SYSTEM CACHING
• db2 alter tablespace M_parts_space NO FILE SYSTEM CACHING
• db2 alter tablespace M_bom_space NO FILE SYSTEM CACHING
• db2 alter tablespace M_wo_space NO FILE SYSTEM CACHING
• db2 alter tablespace M_lo_space NO FILE SYSTEM CACHING
• db2 alter tablespace M_inv_space NO FILE SYSTEM CACHING
• db2 alter tablespace O_cust_space NO FILE SYSTEM CACHING
• db2 alter tablespace O_ords_space NO FILE SYSTEM CACHING
• db2 alter tablespace O_ordl_space NO FILE SYSTEM CACHING
• db2 alter tablespace O_item_space NO FILE SYSTEM CACHING
• db2 alter tablespace S_comp_space NO FILE SYSTEM CACHING
• db2 alter tablespace S_sc_space NO FILE SYSTEM CACHING
• db2 alter tablespace S_po_space NO FILE SYSTEM CACHING
• db2 alter tablespace S_site_space NO FILE SYSTEM CACHING
• db2 alter tablespace S_supp_space NO FILE SYSTEM CACHING
• db2 alter tablespace U_space NO FILE SYSTEM CACHING
• db2 update database config for specdb using LOGFILSIZ 65536
• db2 update db cfg for specdb using logprimary 54 logsecond 4
• db2 update dbm cfg using DIAGLEVEL 0
• db2 update dbm cfg using NOTIFYLEVEL 0
• db2 update dbm cfg using MAX_QUERYDEGREE 1
• db2 update dbm cfg using CPUSPEED 2.676618e-07
• db2 update db cfg for specdb using CATALOGCACHE_SZ 2560
• db2 update db cfg for specdb using LOGBUFSZ 2560
• db2 update db cfg for specdb using UTIL_HEAP_SZ 364378
• db2 update db cfg for specdb using LOCKTIMEOUT 60
• db2 update db cfg for specdb using LOGFILSIZ 65536
• db2 update db cfg for specdb using LOGPRIMARY 54
• db2 update db cfg for specdb using MINCOMMIT 10
• db2 update db cfg for specdb using AUTO_RUNSTATS OFF
• db2 update db cfg for specdb using DATABASE_MEMORY 6100000
• /etc/security/limits.conf changes:
• db2inst1 soft data -1
• db2inst1 hard data -1
• db2inst1 soft nofile 65536
• db2inst1 hard nofile 65536
• db2inst1 soft fsize -1
• db2inst1 hard fsize -1
• db2inst1 soft nproc -1
• db2inst1 hard nproc -1
• * - nofile 1048576

Infraserver VM Notes

• The following boot options were used:
• nosoftlockup intel_idle.max_cstate=0 mce=ignore_ce selinux=0 audit=0 elevator=noop clocksource=tsc tsc=reliable watchdog=0 hpet=disable nohz=on cgroup_disable=memory,cpu,blkio
• The following services are disabled:
• abrtd,atd,autofs,htcacheclean,ip6tables,iptables,irqbalance,kdump,mdmonitor,netconsole,nfslock,ntpd,ntpdate,oddjobd,psacct,rdisc,restorecond,rngd,rpcgssd,rpcsvcgssd,saslauthd,smartd,sssd,svnserve,winbind,ypbind,blk-availability,jexec,lvm2-monitor
• /etc/fstab changes:
• root file-system options: barrier=0,noatime
• datadisk file-system defaults options:noatime,nodiratime,nobarrier
• MTU=9000 for eth1 (internal network).
• /etc/httpd/conf/httpd.conf changes:
• Timeout 20
• KeepAlive On
• MaxKeepAliveRequests 0
• KeepAliveTimeout 60
• <IfModule prefork.c>
• StartServers 20
• MinSpareServers 20
• MaxSpareServers 20
• ServerLimit 4096
• MaxClients 4096
• MaxRequestsPerChild 640000
• </IfModule>
• <IfModule worker.c>
• StartServers 20
• MaxClients 4096
• MaxRequestsPerChild 640000
• </IfModule>
• Listen 81
• ServerName besim:80
• CustomLog logs/access_log common
• /etc/httpd/conf/httpd.conf add:
• LoadModule file_cache_module modules/mod_file_cache.so
• LoadModule mem_cache_module modules/mod_mem_cache.so
• LoadModule fastcgi_module modules/mod_fastcgi.so
• LoadModule file_cache_module /usr/lib64/httpd/modules/mod_file_cache.so
• ScriptAlias /fcgi-bin/ "/var/www/fcgi-bin/"
• FastCgiIpcDir /etc/httpd/fastcgi
• <Directory "/var/www/fcgi-bin">
• AllowOverride None
• Options +ExecCGI -Includes
• SetHandler fastcgi-script
• Order allow,deny
• Allow from all
• </Directory>
• AddHandler fastcgi-script fcgi
• FastCgiServer /var/www/fcgi-bin/besim_fcgi.fcgi -processes 500
• Following items were added to /etc/sysctl.conf:
• fs.file-max = 500000
• net.core.somaxconn = 81920
• net.ipv4.tcp_max_tw_buckets = 1500000
• net.ipv4.conf.default.arp_filter = 1
• net.ipv4.conf.all.arp_filter = 1
• net.ipv4.tcp_tw_reuse=1
• net.ipv4.tcp_keepalive_time = 120
• net.ipv4.tcp_keepalive_intvl = 30
• net.ipv4.tcp_timestamps = 0
• net.ipv4.tcp_syn_retries = 4
• net.ipv4.tcp_retries2 = 7
• net.ipv4.conf.all.arp_notify = 1
• net.ipv4.conf.default.arp_notify = 1
• net.ipv4.conf.eth0.arp_notify = 1
• net.ipv4.conf.lo.arp_notify = 1
• vm.swappiness=0
• The following items were added to /etc/bashrc:
• ulimit -n 100000 > /dev/null 2>&1
• The tuned-0.2.19 package was installed and activated with virtual-guest profile
• ixgbevf module option used: InterruptThrottleRate=2000
• /etc/security/limits.conf changes:
• hard and soft nofile set to 102400
• hard and soft nproc set to 102400

Batch Server VM Notes

• The following boot options were used:
• nosoftlockup intel_idle.max_cstate=0 mce=ignore_ce selinux=0 audit=0 elevator=noop clocksource=tsc tsc=reliable watchdog=0 highres=off hpet=disable nohz=on cgroup_disable=memory,cpu,blkio
• The following services are disabled:
• abrtd,atd,ip6tables,iptables,irqbalance,kdump,mdmonitor,netconsole,netfs,ntpd,ntpdate,psacct,rdisc,restorecond,rngd,saslauthd,smartd,svnserve,blk-availability,lvm2-monitor
• Following items were added to /etc/sysctl.conf:
• fs.file-max = 500000
• net.core.somaxconn = 8192
• net.ipv4.tcp_max_tw_buckets = 500000
• net.ipv4.conf.default.arp_filter = 1
• net.ipv4.conf.all.arp_filter = 1
• The tuned-0.2.19 package was installed and activated with virtual-guest profile
• ixgbevf module option used: InterruptThrottleRate=1000
• /etc/security/limits.conf changes:
• hard and soft nofile set to 102400
• hard and soft nproc set to 102400

Client Driver Notes

• The client systems VMs using Red Hat Enterprise Linux 6.5.
• Two virtual clients were used for each tile,One virtual client per tile ran the webserver workload and the other virtual client per tile ran the other workloads.
• The first virtual client is the prime controller.
• Each prime client VM has 10240 MB memory, and a 20 GB disk,1 ixgbevf device and 1 virtio_blk device are used.
• Each prime client has 4 VCPUs.
• Each webserver client VM has 5120 MB memory, and a 20 GB disk,1 ixgbevf device and 1 virtio_blk device are used.
• Each web client has 4 VCPUs.

Other Notes

• A H3C S5830 24-port switch was used to connect client systems to SUT.
• The 10GbE ports were attached to a dual port 10 GbE SFP+ expansion module.

Performance Details: