Previous Table of Contents Next


A pseudo-code definition of the debit-credit workload is given in Figure 4.5. There are four record types in the database: account, teller, branch, and history. History records are 50 bytes; others are 100 bytes. Each transaction consists of reading a 10byte message from a block mode terminal connected via X.25. The system performs presentation services to map the input for a

Begin-Transaction
     Read message from the terminal (100 bytes)
     Rewrite account                (100 bytes, random)
     Write history                  (50 bytes, sequential)
     Rewrite teller                 (100 bytes, random)
     Rewrite branch                 (100 bytes, random)
     Write message to the terminal  (200 bytes)
Commit-Transaction

FIGURE 4.5 Debit-credit transaction pseudo-code.

COBOL program, which in turn uses a database system to debit a bank account, do the standard double-entry bookkeeping, and then reply to the terminal. Fifteen percent of the transactions require access to remote branch databases; others are local.

Due to increased popularity of the transaction processing systems, an effort is underway to define transaction processing benchmarks more precisely. A council of several transaction processing system vendors, users, and consultants, called the Transactions Processing Performance Council (TPC), was formed in August 1988. Its first benchmark, called TPC Benchmark A, is a variant of the debit-credit benchmark (see TPC 1989). In TPC benchmark, the throughput is measured in terms of TPS such that 90% of all transactions provide 2 seconds or less response time.

4.6.8 SPEC Benchmark Suite

The Systems Performance Evaluation Cooperative (SPEC) is a nonprofit corporation formed by leading computer vendors to develop a standardized set of benchmarks. Release 1.0 of the SPEC benchmark suite (see SPEC 1990) consists of the following 10 benchmarks drawn from various engineering and scientific applications:

1.  GCC: The time for the GNU C Compiler to convert 19 preprocessed source files into assembly language output is measured. This benchmark is representative of a software engineering environment and measures the compiling efficiency of a system.
2.  Espresso: Espresso is an Electronic Design Automation (EDA) tool that performs heuristic boolean function minimization for Programmable Logic Arrays (PLAs). The elapsed time to run a set of seven input models is measured.
3.  Spice 2g6: Spice, another representative of the EDA environment, is a widely used analog circuit simulation tool. The time to simulate a bipolar circuit is measured.
4.  Doduc: This is a synthetic benchmark that performs a Monte Carlo simulation of certain aspects of a nuclear reactor. Because of its iterative structure and abundance of short branches and compact loops, it tests the cache memory effectiveness.
5.  NASA7: This is a collection of seven floating-point intensive kernels performing matrix operations on double-precision data.
6.  LI: The elapsed time to solve the popular 9-queens problem by the LISP interpreter is measured.
7.  Eqntom: This benchmark translates a logical representation of a boolean equation to a truth table.
8.  Matrix300: This performs various matrix operations using several LINPACK routines on matrices of size 300 × 300. The code uses double-precision floating-point arithmetic and is highly vectorizable.
9.  Fpppp: This is a quantum chemistry benchmark that performs two electron integral derivatives using double-precision floating-point FORTRAN. It is difficult to vectorize.
10.  Tomcatv: This is a vectorized mesh generation program using double-precision floating-point FORTRAN. Since it is highly vectorizable, substantial speedups have been observed on several shared-memory multiprocessor systems.

These benchmarks, which stress primarily the CPU, Floating Point Unit (FPU), and to some extent the memory subsystem, are meant for comparing CPU speeds. Benchmarks to compare I/O and other subsystems may be included in future releases.

The elapsed time to run two copies of a benchmark on each of the N processors of a system (a total of 2N copies) is measured and compared with the time to run two copies of the benchmark on a reference system (which is VAX-11/780 for Release 1.0). For each benchmark, the ratio of the time on the reference system and the system under test is reported as SPECthruput using a notation of #CPU@Ratio. For example, a system with three CPUs taking only 1/15th as long as the reference system on GCC benchmark has a SPECthruput of 3@15. The ratio is a measure of the per processor throughput relative to the reference system. The aggregate throughput for all processors of a multiprocessor system can be obtained by multiplying the ratio by the number of processors. For example, the aggregate throughput for the above system is 45.

The geometric mean (discussed in Section 12.5) of the time ratios for single copies of the 10 benchmarks is used to indicate the overall performance for the suite and is called SPECmark. Again, the notation #CPU@Ratio is used to report the SPECmark.

The TPC and SPEC are the beginning of a new trend in the industry to develop standard benchmarks for comparing all types of computer systems including networks, image processing systems, and databases.

EXERCISES

4.1  Select an area of computer systems (for example, processor design, networks, operating systems, or databases), review articles on performance evaluation in that area, and make a list of benchmarks used in those articles.
4.2  Implement the Sieve workload in a language of your choice, run it on systems available to you, and report the results.


Previous Table of Contents Next

Copyright © John Wiley & Sons, Inc.