Performance Summary:

Validation Errors:

•   No Validation Errors Found

Physical Configuration:

Virtual Configuration:

Notes:

Physical System Notes

• BIOS settings used:
• Power Profile set to Maximum Performance
• Thermal Configuration set to Maximum Cooling
• Turbo Mode:Enable
• The test sponsor attests, as of date of publication, that CVE-2017-5754 (Meltdown) is mitigated in the system as tested and documented:
• No: (host, guests)
• The test sponsor attests, as of date of publication, that CVE-2017-5753 (Spectre variant 1) is mitigated in the system as tested and documented:
• No: (host, guests)
• The test sponsor attests, as of date of publication, that CVE-2017-5715 (Spectre variant 2) is mitigated in the system as tested and documented:
• No: (host, guests)
• The test sponsor attests, as of date of publication, that CVE-2018-3620 and CVE-2018-3646 (Foreshadow) are mitigated in the system as tested and documented:
• No: (host, guests)

Storage Notes

• 8 x 500G SEAGATE ST600MM0088 were built as RAID5 and used for OS
• 3 x 3.2TB Huawei Corporation Device 0123 (rev 45) were used for vm images and datastores:
• The first device that located in node0 was partitioned as 12 LUNs, Tile 1-12 VMs including vm images and datastores were located on the device separately.
• The second device that located in node1 was partitioned as 7 LUNs, Tile 13-19 VMs including vm images and datastores were located on the device separately.
• The third device that located in node1 was partitioned as 8 LUNs, Tile 20-27 VMs including vm images and datastores were located on the device separately.
• There are total 27 LUNs,each LUN contained one tile vm images and datasores.

Virtualization Software Notes

• The following lines were added to /etc/sysctl.conf(especially some temporary fine-tuned configurations were added via the sysctl -w command:kernel.sched_min_granularity_ns = 20000000,kernel.sched_wakeup_granularity_ns = 15000000,kernel.sched_latency_ns = 80000000,kernel.sched_migration_cost_ns = 0):
• net.ipv6.conf.all.disable_ipv6 = 1
• net.ipv4.icmp_echo_ignore_broadcasts = 1
• net.ipv4.conf.all.rp_filter = 1
• fs.inotify.max_user_watches = 65536
• net.ipv4.conf.default.promote_secondaries = 1
• net.ipv4.conf.all.promote_secondaries = 1
• net.core.somaxconn = 8192
• net.ipv4.tcp_max_tw_buckets = 500000
• net.ipv4.conf.default.arp_filter = 1
• net.ipv4.conf.all.arp_filter = 1
• net.core.netdev_max_backlog = 500000
• fs.aio-max-nr = 70656
• vm.swappiness = 0
• vm.dirty_ratio = 10
• vm.dirty_background_ratio = 5
• kernel.sched_min_granularity_ns = 80000000
• kernel.sched_wakeup_granularity_ns = 150000000
• kernel.sched_latency_ns = 160000000
• kernel.sched_migration_cost_ns = 500000
• The following boot options were used:
• oops=panic net.ifnames=0 biosdevname=0 noexec=on efi=old_map nmi_watchdog=0 spectre_v2=off intel_iommu=on iommu=pt selinux=0 pci=realloc nohz=on highres=off hpet=disable audit=0 elevator=deadline nopku uksm=disable clocksource=tsc tsc=reliable cgroup_disable=cpu,cpuacct,memory,blkio,hugetlb rdshell nosoftlockup intel_idle.max_cstate=0 mce=ignore_ce nopti ipv6.disable=1 nohz_full=1-21,23-87 rcunocbs=1-21,23-87 LANG=en_US.UTF-8
• The following services are disabled:
• arp-ethers, auditd, autovt@, blk-availability, brltty, console-getty, console-shell, cpupower, crond, debug-shell, dm-event, dnsmasq, ebtables, euleros-security, fcoe, getosstat, gssproxy, guest_kbox_ram, haveged, hirmd, hotpatch, ipmi, ipmievd, irqbalance, iscsi, iscsid, iscsiuio, kbox, kdump, lldpad, lvm2-lvmetad, lvm2-lvmpolld, mcelog, multipathd, nfs-server, nfs, ntpd, ntpdate, netconsole, o2cb, openvswitch, p2pgroup, pmqos-static, radvd, ras-mc-ctl, rasdaemon, rdisc, rdma, rhel-dmesg, rhel-domainname, rsyncd, saslauthd, serial-getty@, sigma-serial, smartd, sntp, sysalarm, sysmonitor, sysstat, systemd, target, tcsd, uvp-memoc, UVPHostd, vBMC_agentd
• A maximum of 336 virtual functions were configured and 196 virtual functions were used for the Web, App, DB, Mail Infra and Batch VMs.
• VMs are bound to host CPUs via their XML configurations in the following manner:
• All VMs of tiles 1-12 are bound to NUMA node 0. (The batch VMs for tiles 1-12 had their VCPUs pinned to cpuset=20-21,64-65)
• All VMs of tiles 13-27 (except the dbserver25, appserver25, appserver27, infraserver25, mailserver25, webserver25 are bound to NUMA node 0) are bound to NUMA node 1. (The VCPU of mailserver27 is pinned to node 0, while the memory is pinned to node 1. The batch VMs for tiles 13, 14, 17, 18, 21, 22, 25, and 26 had their VCPUs pinned to cpuset=41-43,85-87, and the batch VMs for tiles 15, 16, 19, 20, 23, and 24 had their VCPUs pinned to cpuset=42-43,86-87)
• VM configurations are as follows:
• All VMs are managed with libvirt and created using virsh define and virsh start.
• All VM kernels were rebuilt that their type of Preemption Model were replaced as No Forced Preemption.
• All VMs are created with the following xml template, exceptions noted below template. (Each VM's xml template is at SUT/VM-xml directory):
• <domain type='kvm'>
• <name>$vm_name</name>
• <memory>$vm_memory</memory>
• <currentMemory>$vm_memory</currentMemory>
• <vcpu placement='static' cpuset='$host-cpu-list'>$vm_cpu</vcpu>
• <cputune>
• <vcpupin vcpu='0' cpuset='$host-cpu-list'/>
• <vcpupin vcpu='1' cpuset='$host-cpu-list'/>
• </cputune>
• <cpu mode='custom' match='exact' check='full'>
• <model fallback='forbid'>SandyBridge</model>
• <vendor>Intel</vendor>
• <topology sockets=$sockets cores=$cores threads=$threads/>
• <feature policy='require' name='vme'/>
• <feature policy='require' name='hypervisor'/>
• <feature policy='require' name='arat'/>
• <feature policy='require' name='xsaveopt'/>
• </cpu>
• <numatune>
• <memory mode='strict' nodeset='$vm_node'/>
• </numatune>
• <os>
• <type arch='x86_64' machine='pc'>hvm</type>
• <boot dev='hd'/>
• </os>
• <features>
• <apic eoi='on'/>
• <pae/>
• <acpi/>
• </features>
• <clock offset='utc'>
• <timer name='rtc' tickpolicy='catchup' track='guest'/>
• <timer name='hpet' present='no'/>
• <timer name='pit' tickpolicy='delay'/>
• </clock>
• <on_poweroff>destroy</on_poweroff>
• <on_reboot>restart</on_reboot>
• <on_crash>restart</on_crash>
• <devices>
• <emulator>/usr/bin/qemu-kvm</emulator>
• <disk type='block' device='disk'>
• <driver name='qemu' type='raw' cache='none' io='native'/>
• <source dev='$vm_disk' />
• <target dev='vda' bus='virtio'/>
• </disk>
• <hostdev mode='subsystem' type='pci' managed='yes'>
• <source>
• <address domain='0x0000' bus='$bus' slot='$slot' function='$function'/>
• </source>
• </hostdev>
• <graphics type='vnc' port='-1' autoport='yes' listen='0.0.0.0'>
• <listen type='address' address='0.0.0.0'/>
• </graphics>
• <memballoon model='none'>
• <alias name='balloon0'/>
• </memballoon>
• </devices>
• <qemu:commandline>
• <qemu:arg value='-machine'>
• <qemu:arg value='smm=off'>
• </qemu:commandline>
• </domain>
• The following section is repeated for the second PCI device used in the web and infra VMs:
• <hostdev mode='subsystem' type='pci' managed='yes'>
• <source>
• <address domain='0x0000' bus='$bus' slot='$slot' function='$function'/>
• </source>
• </hostdev>
• The following section is repeated for the second disk used in the db,mail and infra VMs for their data:
• <disk type='block' device='disk'>
• <driver name='qemu' type='raw' cache='none' io='native'/>
• <source dev='$vm_disk' />
• <target dev='vdb' bus='virtio'/>
• </disk>
• VMs' nice values were setted after startup via the renice command to the following settings:
• app='-9'
• db='-12'
• web='0'
• infra='4'
• mail='7'
• batch='3'

Web Server VM Notes

• The following boot options were used:
• lapic=notscdeadline rhgb quiet nosoftlockup intel_idle.max_cstate=0 mce=ignore_ce selinux=0 audit=0 elevator=noop clocksource=tsc tsc=reliable watchdog=0 highres=off hpet=disable nohz=off ipv6.disable=1 ipv6.disable_ipv6=1 cgroup_disable=memory,cpu,blkio
• The following services are disabled:
• abrt-ccpp abrtd acpid atd auditd autofs certmonger cgconfig cpuspeed crond cgred cups haldaemon htcacheclean httpd ip6tables iptables kdump mcelogd messagebus netconsole netfs nginx-debug ntpd ntpdate numad oddjobd portreserve postfix psacct quota_nld rdisc restorecond rhnsd rhsmcertd rngd rpcgssd rpcidmapd rpcsvcgssd saslauthd smartd sssd sysstat winbind ypbind
• /etc/fstab changes:
• root file-system options: barrier=0,noatime
• nfs file-system defaults options: noatime,actimeo=14400,nocto
• MTU=9000 for eth1 (internal network).
• The MTU is set after all VMs are started through the following command: ifconfig eth1 mtu 9000 (the script setmtu.sh runs this command and the script is at SUT directory).
• Contents of nginx configuration file /etc/nginx/nginx.conf
• user nginx;
• worker_processes 4;
• worker_cpu_affinity 00000001 00000001 00000010 00000010;
• worker_rlimit_nofile 65535;
• pid /var/run/nginx.pid;
• events {
• use epoll;
• worker_connections 65535;
• multi_accept on;
• accept_mutex off;
• }
• http {
• include /etc/nginx/mime.types;
• default_type application/octet-stream;
• log_format common '$remote_addr - $remote_user [$time_local] $request'
• '"$status" $body_bytes_sent ';
• access_log /var/log/nginx/access.log common gzip buffer=4m flush=1m;
• sendfile on;
• sendfile_max_chunk 1m;
• tcp_nopush on;
• tcp_nodelay on;
• keepalive_timeout 120;
• open_file_cache max=204800 inactive=20s;
• open_file_cache_min_uses 1;
• open_file_cache_valid 30s;
• charset utf-8;
• server_names_hash_bucket_size 128;
• client_header_buffer_size 2k;
• large_client_header_buffers 4 4k;
• include /etc/nginx/conf.d/*.conf;
• }
• Contents of nginx configuration file /etc/nginx/conf.d/default.conf
• server {
• listen 80 backlog=8192;
• server_name localhost;
• location / {
• root /var/www/html;
• index index.html index.htm index.php;
• }
• error_page 500 502 503 504 /50x.html;
• location = /50x.html {
• root /usr/share/nginx/html;
• }
• location ~ .*\.(html|htm|gif|jpg|jpeg|bmp|png|ico|txt|js|css)$
• {
• root /var/www/html;
• expires 100d;
• }
• location ~ ^/images/ {
• root /var/www/html/support;
• expires 100d;
• }
• location ~ \.php$ {
• root html;
• fastcgi_pass unix:/dev/shm/php-cgi.sock;
• fastcgi_index index.php;
• fastcgi_param SCRIPT_FILENAME /var/www/html$fastcgi_script_name;
• include fastcgi_params;
• }
• }
• Contents of nginx configuration file /etc/nginx/conf.d/example_ssl.conf
• server {
• listen 443 ssl;
• server_name localhost;
• ssl on;
• ssl_certificate /etc/pki/tls/certs/localhost.crt;
• ssl_certificate_key /etc/pki/tls/private/localhost.key;
• ssl_session_cache shared:SSL:1m;
• ssl_session_timeout 5m;
• ssl_protocols SSLv2 SSLv3 TLSv1 TLSv1.1 TLSv1.2;
• ssl_ciphers ALL:!kEDH!ADH:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP;
• ssl_prefer_server_ciphers on;
• location / {
• root /var/www/html;
• index index.html index.htm index.php;
• }
• location ~ .*\.(html|htm|gif|jpg|jpeg|bmp|png|ico|txt|js|css)$
• {
• root /var/www/html;
• expires 100d;
• }
• location ~ ^/images/ {
• root /var/www/html/support;
• expires 100d;
• }
• location ~ \.php$ {
• root html;
• fastcgi_pass unix:/dev/shm/php-cgi.sock;
• fastcgi_index index.php;
• fastcgi_param SCRIPT_FILENAME /var/www/html$fastcgi_script_name;
• include fastcgi_params;
• }
• }
• The following lines were changed in /etc/php-fpm.d/www.conf:
• listen = /dev/shm/php-cgi.sock
• listen.owner = nginx
• listen.group = nginx
• user = nginx
• group = nginx
• pm = static
• pm.max_children = 72
• listen.backlog = 8192
• pm.start_servers = 20
• request_terminate_timeout = 0
• rlimit_files = 40960
• Following items were added to /etc/sysctl.conf:
• fs.file-max = 500000
• net.core.somaxconn = 81920
• net.ipv4.tcp_max_tw_buckets = 1500000
• net.ipv4.conf.default.arp_filter = 1
• net.ipv4.conf.all.arp_filter = 1
• net.ipv4.tcp_tw_reuse = 1
• net.ipv4.tcp_keepalive_time = 120
• net.ipv4.tcp_keepalive_intvl = 30
• net.ipv4.tcp_timestamps = 0
• vm.swappiness = 0
• net.ipv4.tcp_fin_timeout=60
• net.ipv4.tcp_tw_recycle=1
• net.ipv4.ip_forward = 0
• net.ipv4.conf.default.rp_filter = 1
• net.ipv4.conf.default.accept_source_route = 0
• kernel.sysrq = 0
• kernel.core_uses_pid = 1
• net.ipv4.tcp_syncookies = 1
• kernel.msgmnb = 65536
• kernel.msgmax = 65536
• net.ipv4.tcp_max_syn_backlog = 8192
• net.core.netdev_max_backlog = 8192
• net.bridge.bridge-nf-call-ip6tables = 0
• net.bridge.bridge-nf-call-iptables = 0
• net.bridge.bridge-nf-call-arptables = 0
• kernel.shmmax = 68719476736
• kernel.shmall = 4294967296
• The following items were added to /etc/bashrc:
• ulimit -n 1048576 > /dev/null 2>&1
• ulimit -u 1000000 > /dev/null 2>&1
• The tuned-0.2.19 package was installed and activated with virtual-guest profile
• /etc/security/limits.conf changes:
• * soft memlock unlimited
• * hard memlock unlimited

Application Server VM Notes

• The following boot options were used:
• lapic=notscdeadline rhgb quiet nosoftlockup intel_idle.max_cstate=0 mce=ignore_ce selinux=0 audit=0 elevator=noop clocksource=tsc tsc=reliable watchdog=0 highres=off hpet=disable nohz=off ipv6.disable=1 ipv6.disable_ipv6=1 cgroup_disable=memory,cpu,blkio
• The following services are disabled:
• abrt-ccpp abrtd acpid atd auditd autofs certmonger cgconfig cpuspeed crond cgred cups haldaemon ip6tables iptables kdump mcelogd messagebus netconsole netfs nfs nfslock ntpd ntpdate numad oddjobd portreserve postfix psacct quota_nld rdisc restorecond rhnsd rhsmcertd rngd rpcgssd rpcidmapd rpcsvcgssd saslauthd smartd sssd sysstat winbind ypbind
• /etc/fstab changes:
• root file-system options: barrier=0
• Following items were added to /etc/sysctl.conf:
• fs.file-max = 500000
• net.core.somaxconn = 8192
• net.ipv4.tcp_max_tw_buckets = 500000
• net.ipv4.conf.default.arp_filter = 1
• net.ipv4.conf.all.arp_filter = 1
• vm.swappiness=0
• The tuned-0.2.19 package was installed and activated with virtual-guest profile
• /etc/security/limits.conf changes:
• root hard memlock unlimited
• root soft memlock unlimited
• - nofile 1048576
• The following items were added to /etc/bashrc:
• ulimit -n 1048576 > /dev/null 2>&1
• ulimit -u 1000000 > /dev/null 2>&1
• JAVA_HOME=/opt/IBM/WebSphere/AppServer/java
• export JAVA_HOME
• PATH=$JAVA_HOME/bin:$PATH
• export PATH
• Update the JVM from 1.6.0 which is installed with the Websphere 8.5.5 to 1.7.0 which is download from IBM WebSite

Mail Server VM Notes

• The following boot options were used:
• lapic=notscdeadline rhgb quiet nosoftlockup intel_idle.max_cstate=0 mce=ignore_ce selinux=0 audit=0 elevator=noop clocksource=tsc tsc=reliable watchdog=0 highres=off hpet=disable nohz=off ipv6.disable=1 ipv6.disable_ipv6=1 cgroup_disable=memory,cpu,blkio
• The following services are disabled:
• abrt-ccpp abrtd acpid atd auditd autofs certmonger cgconfig cpuspeed crond cgred cups haldaemon ip6tables iptables irqbalance kdump mcelogd messagebus netconsole netfs nfs nfslock ntpd ntpdate numad oddjobd portreserve postfix psacct quota_nld restorecond rdisc rhnsd rhsmcertd rngd rpcgssd rpcidmapd rpcsvcgssd saslauthd smartd sssd sysstat winbind ypbind
• /etc/fstab changes:
• root file-system options: barrier=0
• datadisk file-system defaults options:noatime,barrier=0,data=writeback
• Dovecot was compiled with default options and the following dovecot.conf was used:
• base_dir = /var/run/dovecot/
• ssl = no
• disable_plaintext_auth = no
• listen = *
• log_path = /var/log/dovecot
• mail_debug = no
• mmap_disable = yes
• mail_fsync = never
• mail_location = dbox:/mailstore/dovecot/maildir/%u
• mdbox_rotate_size = 8192
• passdb {
• args = /etc/passwd.dovecot
• driver = passwd-file
• }
• protocols = imap
• service auth {
• user = root
• }
• service imap-login {
• inet_listener imap {
• port = 143
• }
• process_min_avail = 10
• }
• userdb {
• driver = passwd
• }
• Following items were added to /etc/sysctl.conf:
• net.ipv6.conf.all.disable_ipv6=1
• fs.file-max = 500000
• net.core.somaxconn = 8192
• net.ipv4.tcp_max_tw_buckets = 500000
• net.ipv4.conf.default.arp_filter = 1
• net.ipv4.conf.all.arp_filter = 1
• net.core.optmem_max = 1000000
• net.core.rmem_default = 1000000
• net.core.rmem_max = 1000000
• net.core.wmem_default = 1000000
• net.core.wmem_max = 1000000
• net.ipv4.tcp_mem = 10000000 10000000 10000000
• net.ipv4.tcp_rmem = 10000000 10000000 10000000
• net.ipv4.tcp_wmem = 10000000 10000000 10000000
• The following items were added to /etc/bashrc:
• ulimit -n 1048576 > /dev/null 2>&1
• ulimit -u 1000000 > /dev/null 2>&1
• The tuned-0.2.19 package was installed and activated with virtual-guest profile
• /etc/security/limits.conf changes:
• * soft memlock unlimited
• * soft memlock unlimited

Database Server VM Notes

• The following boot options were used:
• lapic=notscdeadline rhgb quiet nosoftlockup intel_idle.max_cstate=0 mce=ignore_ce selinux=0 audit=0 elevator=noop clocksource=tsc tsc=reliable watchdog=0 highres=off hpet=disable nohz=off ipv6.disable=1 ipv6.disable_ipv6=1 cgroup_disable=memory,cpu,blkio
• The following services are disabled:
• abrtd atd auditd autofs cgconfig cpuspeed cgred cups dnsmasq ip6tables iptables kdump mcelogd netconsole netfs nfs nfslock ntpd ntpdate numad oddjobd portreserve postfix psacct quota_nld restorecond rdisc rhnsd rhsmcertd rngd rpcidmapd rpcsvcgssd saslauthd smartd sssd sysstat winbind wpa_supplicant ypbind
• /etc/fstab changes:
• datadisk file-system defaults options:noatime,barrier=0,data=writeback
• root file-system defaults options:barrier=0
• Database tuning options:
• db2 update dbm cfg using CPUSPEED 3.306409e-07
• db2 update db cfg for specdb using DATABASE_MEMORY 5260000
• db2set DB2_LARGE_PAGE_MEM=DB; db2set DB2_PINNED_BP=ON
• alter bufferpool IBMDEFAULTBP size 5500000
• alter tablespace roll_space NO FILE SYSTEM CACHING
• alter tablespace C_cust_space NO FILE SYSTEM CACHING
• alter tablespace C_supp_space NO FILE SYSTEM CACHING
• alter tablespace C_site_space NO FILE SYSTEM CACHING
• alter tablespace C_parts_space NO FILE SYSTEM CACHING
• alter tablespace C_custinv_space NO FILE SYSTEM CACHING
• alter tablespace M_parts_space NO FILE SYSTEM CACHING
• alter tablespace M_bom_space NO FILE SYSTEM CACHING
• alter tablespace M_wo_space NO FILE SYSTEM CACHING
• alter tablespace M_lo_space NO FILE SYSTEM CACHING
• alter tablespace M_inv_space NO FILE SYSTEM CACHING
• alter tablespace O_cust_space NO FILE SYSTEM CACHING
• alter tablespace O_ords_space NO FILE SYSTEM CACHING
• alter tablespace O_ordl_space NO FILE SYSTEM CACHING
• alter tablespace O_item_space NO FILE SYSTEM CACHING
• alter tablespace S_comp_space NO FILE SYSTEM CACHING
• alter tablespace S_sc_space NO FILE SYSTEM CACHING
• alter tablespace S_po_space NO FILE SYSTEM CACHING
• alter tablespace S_site_space NO FILE SYSTEM CACHING
• alter tablespace S_supp_space NO FILE SYSTEM CACHING
• alter tablespace U_space NO FILE SYSTEM CACHING
• db2 update database config for specdb using LOGFILSIZ 65536
• db2 update db cfg for specdb using logprimary 54 logsecond 10
• Following items were added to /etc/sysctl.conf:
• kernel.msgmni = 1024
• kernel.shmmax = 137438953472
• kernel.shmmni = 11118096
• kernel.shmall = 8294967296
• fs.file-max = 500000
• net.core.somaxconn = 32384
• net.ipv4.tcp_max_tw_buckets = 1000000
• net.ipv4.conf.default.arp_filter = 1
• net.ipv4.conf.all.arp_filter = 1
• vm.swappiness=0
• vm.nr_hugepages = 10288
• vm.hugetlb_shm_group = 901
• vm.nr_overcommit_hugepages=512
• kernel.sem = 500 64000 400 512
• kernel.randomize_va_space=0
• /etc/security/limits.conf changes:
• db2inst1 soft data -1
• db2inst1 hard data -1
• db2inst1 soft nofile 65536
• db2inst1 hard nofile 65536
• db2inst1 soft fsize -1
• db2inst1 hard fsize -1
• db2inst1 soft nproc -1
• db2inst1 hard nproc -1
• * - nofile 1048576

Infraserver VM Notes

• The following boot options were used:
• lapic=notscdeadline rhgb quiet nosoftlockup intel_idle.max_cstate=0 mce=ignore_ce selinux=0 audit=0 elevator=noop clocksource=tsc tsc=reliable watchdog=0 highres=off hpet=disable nohz=off ipv6.disable=1 ipv6.disable_ipv6=1 cgroup_disable=memory,cpu,blkio
• The following services are disabled:
• abrt-ccpp abrtd acpid atd auditd autofs certmonger cgconfig cpuspeed crond cgred cups haldaemon htcacheclean ip6tables iptables irqbalance kdump mcelogd messagebus netconsole netfs ntpd ntpdate numad oddjobd portreserve postfix psacct quota_nld rdisc restorecond rhnsd rhsmcertd rngd rpcgssd rpcidmapd rpcsvcgssd saslauthd smartd sssd sysstat winbind ypbind
• /etc/fstab changes:
• root file-system options: barrier=0,noatime
• datadisk file-system defaults options:noatime,barrier=0,data=writeback
• MTU=9000 for eth1 (internal network).
• The MTU is set after all VMs are started through the following command: ifconfig eth1 mtu 9000 (the script setmtu.sh runs this command and the script is at SUT directory).
• httpd.conf changes:
• ## KeepAliveTimeout 15
• ##Include conf.d/*.conf
• Timeout 20
• KeepAlive on
• MaxKeepAliveRequests 0
• StartServers 20
• MinSpareServers 20
• ServerLimit 4096
• MaxClients 4096
• MaxRequestsPerChild 640000
• Listen 81
• ServerName besim:80
• CustomLog logs/access_log common
• httpd.conf adds:
• ScriptAlias /fcgi-bin/ "/var/www/fcgi-bin/"
• FastCgilpcDir /etc/httpd/fastcgi
• <Directory "/var/www/fcgi-bin" >
• AllowOverride None
• Options +ExecCGI -Includes
• SetHandler fastcgi-script
• Order allow,deny
• Allow from all
• </Directory>
• Addhandler fastcgi-script fcgi
• FastCgiServer /var/www/fcgi-bin/besim_fcgi.fcgi -processes 100
• The following httpd modules were loaded:
• file_cache mem_cache fastcgi
• Following items were added to /etc/sysctl.conf:
• fs.file-max = 500000
• net.core.somaxconn = 81920
• net.ipv4.tcp_max_tw_buckets = 1500000
• net.ipv4.conf.default.arp_filter = 1
• net.ipv4.conf.all.arp_filter = 1
• net.ipv4.tcp_tw_reuse = 1
• net.ipv4.tcp_keepalive_time = 120
• net.ipv4.tcp_keepalive_intvl = 30
• net.ipv4.tcp_timestamps = 0
• net.ipv4.tcp_syn_retries = 4
• net.ipv4.tcp_retries2 = 7
• vm.swappiness = 0
• A temporary fine-tuned configuration is added via the sysctl -w command:net.ipv4.tcp_timestamps=1
• The following items were added to /etc/bashrc:
• ulimit -n 1048576 > /dev/null 2>&1
• ulimit -u 1000000 > /dev/null 2>&1
• The tuned-0.2.19 package was installed and activated with virtual-guest profile
• /etc/security/limits.conf changes:
• * soft memlock unlimited
• * hard memlock unlimited
• NFS export options:
• rw,async,no_subtree_check,crossmnt,no_root_squash

Batch Server VM Notes

• The following boot options were used:
• lapic=notscdeadline rhgb quiet nosoftlockup intel_idle.max_cstate=0 mce=ignore_ce selinux=0 audit=0 elevator=noop clocksource=tsc tsc=reliable watchdog=0 highres=off hpet=disable nohz=off ipv6.disable=1 ipv6.disable_ipv6=1 cgroup_disable=memory,cpu,blkio
• The following services are disabled:
• abrt-ccpp abrtd acpid atd auditd autofs certmonger cgconfig cgred cpuspeed crond cups haldaemon ip6tables iptables irqbalance kdump mcelogd messagebus netconsole netfs nfs nfslock ntpd ntpdate numad oddjobd portreserve postfix psacct quota_nld rdisc restorecond rhnsd rhsmcertd rngd rpcgssd rpcidmapd rpcsvcgssd saslauthd smartd sssd sysstat winbind ypbind
• /etc/fstab changes:
• root file-system options: barrier=0
• Following items were added to /etc/sysctl.conf:
• fs.file-max = 500000
• net.core.somaxconn = 8192
• net.ipv4.tcp_max_tw_buckets = 500000
• net.ipv4.conf.default.arp_filter = 1
• net.ipv4.conf.all.arp_filter = 1
• The following items were added to /etc/bashrc:
• ulimit -n 1048576 > /dev/null 2>&1
• ulimit -u 1000000 > /dev/null 2>&1
• The tuned-0.2.19 package was installed and activated with virtual-guest profile
• /etc/security/limits.conf changes:
• * soft memlock unlimited
• * hard memlock unlimited

Client Driver Notes

• The client systems VMs using Red Hat Enterprise Linux 6.4. And the kernel was rebuilt that its type of Preemption Model was replaced as No Forced Preemption.
• Two virtual clients were used for each tile,One virtual client per tile ran the webserver workload and the other virtual client per tile ran the other workloads.
• The first virtual client is the prime controller.
• Each prime client VM has 10240 MB memory(except client1 has 12288MB memory, and client26 has 8192MB memory), and a 20 GB disk,1 ixgbevf device and 1 virtio_blk device are used.
• Each prime client has 4 VCPUs.
• Each webserver client VM has 10240 MB memory(except wclient20, wclient21, wclient22, wclient23, wclient24, wclient25, wclient26 and wclient27 have 8192MB memory), and a 20 GB disk,1 ixgbevf device and 1 virtio_blk device are used.
• Each webserver client has 4 VCPUs.

Other Notes

• A Quidway S9303 36-port switch was used to connect client systems to SUT. The 10GbE ports were attached to a dual port 10 GbE SFP+ expansion module.

Performance Details: