TABLE 11.4 Performance or Two Network Architectures
|
|
Network
| Throughput
| Response
|
|
A
| 10
| 2
|
B
| 4
| 1
|
|
TABLE 11.5 Using a Ratio Metric
|
|
System
| Throughput
| Response
| Power
|
|
A
| 10
| 2
| 5
|
B
| 4
| 1
| 4
|
|
- Example 11.1 Throughput and response time were measured on two network architectures. The results are tabulated in Table 11.4. Notice that network A has higher throughput (an HB metric) but also has a higher response time (an LB metric).
The designers of network A suggest that the right metric to compare the networks is by power, which is defined as the ratio of throughput and response time. The transformed results are shown in Table 11.5. From this table, one would conclude that network A is better than network B.
11.3 USING RELATIVE PERFORMANCE ENHANCEMENT
Sometimes the performance metric is already specified. In this case, it is possible to show that relative increase in the performance is better using one type of enhancement than another provided the two are tried on different machines.
- Example 11.2 Two floating-point accelerators A and B are to be compared. MFLOPS measured on two machines with and without the accelerators are tabulated in Table 11.6.
The right way to compare the two accelerators would be to try them out on a single machine. But since B seems to be a popular accelerator and A does not yet work on the same machine as B, the designers of A can exploit the technique of using incomparable bases, as shown in Table 11.7. Conclusion: A gives a higher improvement than B.
TABLE 11.6 MFLOPS of Two Processors with and without Accelerators
|
|
Alternative
| Without
| With
|
|
A on X
| 2
| 4
|
B on Y
| 3
| 5
|
|
TABLE 11.7 Improvement Due to an Accelerator
|
|
Alternative
| Without
| With
| Ratio
|
|
A on X
| 2
| 4
| 2.00
|
B on Y
| 3
| 5
| 1.66
|
|
11.4 RATIO GAMES WITH PERCENTAGES
Percentages are basically ratios. They allow playing ratio games in ways that do not look like ratios.
- Example 11.3 Two experiments were repeatedly conducted on two systems. Each experiment either passed or failed (system either met the specified performance goal or did not). The results are tabulated in the first two rows of Table 11.8.
One alternative to compare the two systems is to take each experiment individually, as shown in Figure 11.1a. The conclusion from this figure is that System B is better than System A in both experiments. Another alternative is to add the results of the two experiments as shown in the last row of Table 11.8 and plotted in Figure 11.1b. The conclusion in this case is that System A is better than system B.
Actually both alternatives have the problem of incomparable bases. In alternative 1, the base is the total number of times the experiment is repeated on a system, which is different for the two systems. In alternative 2, the base is the sum of repetitions of the two experiments together, which is also different for the two systems.
TABLE 11.8 Two Tests on Two Systems
|
System A
|
| System B
|
Test
| Total
| Pass
| %Pass
|
| Test
| Total
| Pass
| %Pass
|
|
|
|
1
| 300
| 60
| 20
|
| 1
| 32
| 8
| 25
|
2
| 50
| 2
| 4
|
| 2
| 500
| 40
| 8
|
|
|
|
| 350
| 62
| 20.6
|
|
| 532
| 48
| 9
|
|
FIGURE 11.1 Ratio games with percentages.
Percentages are misused in several other ways. For example, they can be used to have a psychological impact on the reader. Large numbers attract immediate attention. A 1000% improvement in performance sounds more impressive than an 11-time improvement. This is particularly useful when the performance before the improvement as well as the absolute increase in the performance are not impressive. Instead of saying that the network throughput improved from 1 packet per second to 11 packets per second (which are both unimpressive), it is better to say that the performance was improved by 1000%.