Many common questions are addressed in this FAQ or in the Chauffeur WDK
documentation on the Chauffeur WDK Information (https://www.spec.org/chauffeur-wdk) web page.
A1: Is this a Benchmark? Is Chauffeur WDK replacing
SPECpower_ssj2008 or SERT?
No. This is a development tool for creating new worklets for the
Chauffeur framework
A2: Can I use Chauffeur WDK to make numeric
comparisons?
No. New worklets developed with Chauffeur WDK can be used for
research purposes. The license prohibits competitive comparisons
that promote one product over another.
A3: Where can I find the documentation?
Documentation is provided with the Chauffeur WDK distribution in
the "doc" directory.
A4: Are updates to documents available?
Yes, the most recent documents are on the SPEC website on the
Chauffeur WDK Information (https://www.spec.org/chauffeur-wdk)
web page.
A5: Can I "turn off" compliance checking?
Compliance checks will always be performed at the end of a run.
When re-generating a report manually, you can use the "-s"
parameter on the Reporter to skip validation checks. In this case,
the run will automatically be marked invalid, but no additional
validation will be performed.
A6: What is a "graduated workload"?
A graduated workload operates at several levels, usually stepping
up or down by some regular value.
A7: Why did SPEC choose to implement a graduated
workload?
Server power consumption has been shown to be significantly
different at low load levels versus high load levels, and server
loads vary widely from hour to hour and day to day.
Since no definitive surveys exist, and all data centers are
different, SPEC has chosen to provide graduated loads across a set
of load levels to portray performance and power consumption in
several reasonable ranges.
A8: Does Chauffeur WDK support measuring systems with DC-DC
power supplies? (DC=Direct Current)
No.
DC powered servers can not be compared to AC (Alternating Current)
powered servers because the AC to DC power conversion cost (energy
loss) is not present in DC powered servers.
Although physically possible, SPEC has neither developed nor tested
methods for DC system measurement.
The Chauffeur WDK can run on DC powered servers, but does not support
power measurements on these servers.
A9: Is there a "User Guide" that can walk me through
set up and running Chauffeur WDK?
Yes. That document is the "Chauffeur WDK User Guide".
See below
A10: Can I get documentation without purchasing Chauffeur
WDK?
Yes. The User Guide and other documents are available on the SPEC
web site on the Chauffeur WDK Information
(https://www.spec.org/chauffeur-wdk)
web page.
A11: What are the key documents I should read before I get started?
- SPEC License
- SPEC Power and Performance Methodology
- Chauffeur WDK User Guide
- Measurement Setup Guide
- SERT Design Documents
- FAQ (this document)
- Run and Reporting Rules
A12: Does Chauffeur WDK include the latest version of SPEC Power and Temperature Daemon (PTDaemon)?
Each Chauffeur WDK release includes the latest PTD version
available at that time. Please check to see if a newer PTD is
available here: https://www.spec.org/power/docs/SPECpower-Device_List.html
A13: Are updates to Chauffeur WDK and PTDaemon provided free
of charge?
SPEC provides minor bug fix updates without charge. See
https://www.spec.org/chauffeur-wdk
for information regarding any available updates. Licensees of
Chauffeur WDK 1.x are eligible to receive a free upgrade until October
31, 2017.
D1: Is a power analyzer required to run the Chauffeur
WDK?
No. The Chauffeur WDK may be used for performance tests without a
power analyzer. A power analyzer is required for measuring energy
efficiency.
D2: Can the Chauffeur WDK support multiple power
analyzers?
Yes. Multiple power analyzers and multiple temperature sensors are
supported.
D3: What power analyzers have been tested with the Chauffeur
WDK?
A list of SPEC Accepted (supported and compliant) power measuring
devices are documented on the SPEC web site at: Accepted Measurement
Devices (https://www.spec.org/power/docs/SPECpower-Device_List.html)
The PTDaemon program can be used to display a list of power and
temperature measuring devices along with their device codes.
For SPEC Accepted devices: Run the PTDaemon with the -h parameter, or
zero parameters.
D4: Will my power analyzer work with the SPEC PTDaemon
even though my device does not appear on the accepted devices list
shown on the SPEC web site at Accepted Measurement Devices
(https://www.spec.org/power/docs/SPECpower-Device_List.html)?
No, only devices on the SPEC Accepted Device List are expected to
produce valid results.
If your device is not present on the devices list, it is unlikely that
PTDaemon will successfully read data.
Communication protocols are quite different among the various
manufacturers and models of power analyzers.
D5: Will the SPEC PTDaemon
connect to and read my power analyzer even though my device is not on
the tested list?
It is unlikely that PTDaemon will successfully read a meter not on the
"tested" list.
See the responses to the FAQ just above.
D6: How can I get my power analyzer included in the SPEC
accepted devices list?
Basically, documentation must show that the device has sufficient
accuracy, code must be written to enable SPEC's PTDaemon software to
read measured values from the device, and then testing must be
performed to show that the device does meet SPEC's requirements in
actual usage.
D7: No data cable came with my power analyzer (RS-232), what
now?
Each brand/model power analyzer may require a different
cable. Some experimentation might be required.
Some power analyzers will work with a "null modem" cable.
One combination that has worked with some devices is a standard
"Modem" cable with the addition of a "NULL MODEM"
adapter.
The above combination can also be plugged into a RS-232 to USB
converter.
Some data cable information can be found in the Measurement Setup Guide
(https://www.spec.org/power/docs/SPEC-Power_Measurement_Setup_Guide.pdf)
D8: My power analyzer supports only RS-232 for data logging;
my control system has no serial ports; neither DB-9 nor DB-25 -- What
do I do?
There are devices that convert RS-232 to USB, that cost $10-14
(US$).
Several brands of serial to USB converters have been found to work
well.
With Windows, install the software per manufacturer instructions.
Then, use the device manager to see what "COM" port number
was assigned.
Edit the runpower.bat file; change the default DEVICE_PORT number.
Note that the COM port assignment on your system may change if you
disconnect and re-connect a device or change USB connections.
D9: The PTDaemon software can't connect to my power analyzer.
What is wrong?
First, ensure the analyzer is communicating on the data cable with the
host machine.
The best way to verify this is to first use the vendor supplied
software to read the analyzer.
Most problems, such as cable not connected, wrong cable, incorrect device settings and so on, can be resolved with this method.
If the vendor supplied software shows the correct readings, ensure that
the following are correct:
Ensure the following are set correctly in runpower.bat/runtemp.bat
in the PTDaemon subdirectory:
- NETWORK_PORT set correctly: 8888 for power analyzer 1, 8889 for
temp sensor.
- DEVICE set to a value that reflects your connected Power
Analyzer (run the ptd executable for a device list).
- DEVICE_PORT is set correctly (COM1, COM2, etc.)
More details on device setup can be found in the Measurement Setup
Guide (https://www.spec.org/power/docs/SPEC-Power_Measurement_Setup_Guide.pdf)
D10: Why does my power analyzer display unexpected watts
readings?
This is likely to be traced back to how the analyzer is connected or
the device settings.
More details on device setup can be found in the Measurement Setup
Guide (https://www.spec.org/power/docs/SPEC-Power_Measurement_Setup_Guide.pdf)
Instructions will also be found in the power analyzer documentation or
on the vendor web site. You may also contact the device vendor for
further technical support.
D11: How do I run without a power analyzer?
The PTDaemon has a sensor type of "dummy".
This is the best temporary fix if you will add or use a sensor in other runs.
The device code for "dummy" power analyzer is 1000.
Alternatively, edit listeners.xml, find each
PowerAnalyzerListener and set
<listener enabled="false">
F1: I am having problems running the correct JVM on my SUT or
controller. How do I ensure that the Chauffeur WDK is using a
specific JVM?
By default, the host.bat/.sh (SUT) or director.bat/.sh (Controller)
will expect to find the Java executable in the system PATH. If a
suitable Java version is not found, the test will fail.
Some Java versions in Microsoft Windows may not automatically insert
themselves into the system PATH during installation.
Linux systems normally require an explicit path to the Java executable
in the host.sh file or in the system PATH environmental variable.
Please check the host.bat/.sh or director.bat/.sh file on the problem
system, and ensure that the set JAVA=java line is
modified to point directly to the JVM executable you intend to run on
the SUT.
When switching between multiple JVM vendors or versions, modifying the
set JAVA= line to point to the correct version is the
preferred method.
- Example modification in host.bat on a Microsoft Windows system:
set JAVA=c:\Hotspot-1.7.0_u15\bin\java.exe
- Example modification in host.sh on a Linux system:
set JAVA=/usr/local/IBM-J9/jre/bin/java
F2: For Chauffeur WDK configurations running more than 99 JVM
clients on the SUT the following error messages will show up in
"client.<#>.out" files for # greater than 99 and no
"log.<#>.txt" files are generated:
Can't get load log handler "java.util.logging.FileHandler"
java.io.IOException: Couldn't get lock for log.%u.txt
These error messages can be ignored. All client processes will proceed
and finish their tasks correctly including those, which generate above
error messages. For clients 100 and above you won't find the
"log.<#>.txt" files but
"client.<#>.out" files only.
F3: Why am I getting invalid results with messages that
relate to my power analyzer?
INVALID: samples with unknown uncertainty exceeding the threshold
INVALID: The voltage reported was above the maximum allowed voltage
INVALID: The calibration date for the power analyzer is over 1 year before the test date
Uncertainty and voltage errors relate to how the power analyzer ranges
are set for the test. Please refer to the below links for instructions:
F4: Why do I only get a results.xml.partial file in the results
directory after a full run?
If the user attempts to view the results before final processing has
been completed, this can occur. It could also be caused by a failure
during post-processing of the results file. Please review the file
director.log on the controller. The error messages often point to the
location in the results file where the error occurred.
F5: What is the "coefficient of variation between
clients"?
Coefficient of variation is a mathematical term that measures the
variability of a set (defined as the standard deviation divided by the
mean). Each Java instance that is run on the SUT is referred to as a
client. Chauffeur WDK worklets have different limits on the maximum
coefficient of variation between clients, as a way of ensuring that
the test is run in a stable and repeatable way.
Product and service names mentioned herein may be the trademarks of their respective owners.
Copyright ©1988-2017 Standard Performance Evaluation Corporation (SPEC). All rights reserved.
