Previous | Table of Contents | Next |
If you cant convince them, confuse them.
Trumans Law
Ratios provide good opportunities for playing performance games with competitors. Ratios have a numerator and a denominator. The denominator is also called a base. Two ratios with different bases are not comparable. There are, however, many examples in published literature where computer scientists have knowingly or unknowingly compared ratios with different bases. The technique of using ratios with incomparable bases and combining them to ones advantage is called a ratio game. There are many different ways in which ratios can be used. Some of these are explicit and others are implicit in the sense that it is not obvious that a ratio has been taken. Some of these techniques and the strategies that are used for winning such games are described in this chapter. Learning about such games is helpful, not because we recommend their usage, but because their knowledge will help us protect ourselves from being victimized by others.
The simplest way in which ratio games are played is by presenting the performance of two (or more) systems on a variety of workloads, taking a ratio of performance for each workload, and then using the average ratio to show that ones proposed system is better than the alternative.
An example of such a case was presented in Section 1.2, where three ways to average the performance of two systems A and B were compared. To recapitulate, these three ways are as follows:
It was shown that by appropriate choice of the base system one could reverse the conclusion about which of the two systems is better.
The following case study illustrates the game with real measurements.
TABLE 11.1 A Comparison of 6502 and 8080 Processors | ||||||
---|---|---|---|---|---|---|
Raw Measurements | With 6502 as a Base | With 8080 as a Base | ||||
System | System | System | ||||
Benchmark | 6502 | 8080 | 6502 | 8080 | 6502 | 8080 |
Block | 41.16 | 51.50 | 1.00 | 1.25 | 0.80 | 1.00 |
Sieve | 63.17 | 48.08 | 1.00 | 0.76 | 1.31 | 1.00 |
Sum | 104.33 | 99.58 | 2.00 | 2.01 | 2.11 | 2.00 |
Average | 52.17 | 49.79 | 1.00 | 1.01 | 1.06 | 1.00 |
Ratio games can be played even when more than two systems or benchmarks are involved. The following case study illustrates one such case.
Previous | Table of Contents | Next |