standard performance evaluation corporation (SPEC)

The Standard Performance Evaluation Corporation (SPEC), an association of computer manufacturers founded in 1988, has set itself the task of standardizing objective and comparable evaluation criteria and measurement methods, so-called benchmarks, for the computing performance ofcomputers. SPEC is a non-profit organization to which all well-known manufacturers have joined. The more than 60 member companies include Apple, Compac, Dell, Fujitsu Siemens, Hewlett Packard, IBM, Intel, Microsoft, Sun and others.

SPEC is composed of three groups: The Open Systems Group (OSG), the High Performance Group( HPG), and the Graphics Performance Characterization Group (GPC), which are dedicated to various computer systems and subject areas.

One of the most significant SPEC activities is the definition of benchmarks to determine computing performance. The definitions are application, hardware and software related and always carry the designation SPECxxxx; for example SPECjvm for Java Virtual Machine, SPECmark for benchmark, SPECweb for web server or SPECint for integer and SPECfp for floating point. The last two evaluation criteria are used for the comparability of processor performance.

Overview of various SPEC benchmarks:

SPEC89 is the designation for the CPU89 benchmark.

SPEC92 is the name for the CPU92 benchmark.

SPEC95 is the designation for the CPU95 benchmark.

SPECchem96 is the official name for applications of SPEChpg96: Computer applications in the chemical industry.

SPEChpg96 were the first benchmarks published by the High Performance Group (HPG) within SPEC. These covered seismic applications, among others.

SPECjvm2008 is the current benchmark for Java Virtual Machine( JVM).

SPECmarks are the metrics for the original benchmark CPU89. In the meantime, the designation is also used for CPU95ratio.

SPECrate is the metric for throughput and is based on the CPU95 benchmark.

SPECseis96 is the designation for seismological applications of SPEChpg96.

SPECweb96 is the first benchmark of SPEC for web servers. This benchmark measures the ability to handle http/1.0GET requests from multiple external clients.

SPECweb99 is one of the current benchmarks for web servers, replacing SPECweb96.

SPECweb2005_Banking: The SPECweb2005_Banking benchmark simulates typical encrypted requests to an onlinedatabase via HTTPS. These queries can be logins and logouts, booking transactions or account queries.

SPECweb2005_Ecommerce is a test in which products are searched for and purchased on websites.

SPECweb2005_Support tests support websites. This includes downloading drivers and documents.

SPEC MPI2007: A benchmark set and rule set for 32 and 64 bit architectures. SPEC MPI2007 supports all major 32- and 64-bit architectures on Linux, Unix and Windows. The dataset scales up to 512 processor cores.

SPEC has defined four performance measurements for the CPU2006 benchmark:

SPECint_2006: This performance test determines the performance of the central processing units for integers. The test is performed with single thread.

SPECint_rate_2006: This is also a performance test with integer numbers, in contrast to the previous test, the programs work in parallel.

SPECfp_2006: This test is used to determine performance with floating point operations.

SPECfp_rate_2006: In contrast to the previous test, the programs work in parallel. The performance of multi-core processors can be determined with this test.

SPECPower is a benchmark used to measure the performance values and characteristics of servers. This test compares the performance of different servers in order to draw conclusions for improving efficiency. This benchmark targets hardware vendors, as well as the IT industry and computer manufacturers.