Preface

This book is intended to be a comprehensive textbook for the computer professionals who design, procure, or use computer systems. These professionals may be proficient in their own field (design, procurement, application, and so forth) while knowing little about queueing theory, statistics, experimental design, or workload characterization. Performance analysis is a key step in the design and procurement of new computer systems including processors, languages, operating systems, networking architectures, or database systems. In all cases, it is necessary to compare performance of new computer systems with that of similar systems. This book provides these professionals with performance analysis techniques that are simple and straightforward and require a minimal level of statistical sophistication.

A computer systems performance analyst must understand computer systems as well as various analysis techniques such as statistics, probability theory, experimental design, simulation, and queueing theory. The purpose of this book is to give basic modeling, simulation, and analysis background to systems analysts so that they are able to understand performance terminology, correctly design performance experiments, use simple queueing or simulation models, and interpret results. Often performance analysts are of two basic types: those who can measure but cannot model and those who can model but cannot measure. In practice, one needs to use both measurement as well as modeling techniques to solve performance problems. This book stresses both aspects of the field.

There are many books on computer systems performance. These books discuss only one or two aspects of performance analysis, with a majority of the books being queueing theoretic. Queueing theory is admittedly a helpful tool, but the knowledge of simulation, measurement techniques, data analysis, and experimental design is invaluable. I wrote this book because there is no book available that emphasizes and integrates all these aspects of performance analysis. Experimental design techniques that help reduce analysis costs are currently a totally neglected area in performance analysis books. As a performance analyst with several design teams at Digital Equipment Corporation and as an instructor of a course on performance analysis at Massachusetts Institute of Technology, I have been involved in the performance analysis of a wide variety of computer systems and subsystems. This book is a compendium of the techniques that were used in these analyses.

Analytical simplicity is a valuable aspect of this book. Sophisticated queueing theoretic models and statistical techniques are of interest to performance specialists, but in the majority of cases simple analysis yields sufficiently useful results. Complex analyses are not feasible, for example, in an industrial environment where there are considerable time restraints and deadlines. Simple models and analyses that are easily understood expedite decisions and solutions within a design team that may be composed of team members of varying degree of analytical sophistication. In spite of my attempt to keep the mathematical rigor to a minimum, some portions, particularly those involving derivations or proofs of results, may be a bit too intricate for a few readers. In such cases, it is recommended that the reader skip the particular portion during the first pass, try to understand the examples illustrating the application of results, and then come back to the derivation, if necessary.

Another interesting aspect of this book is that it discusses common mistakes and games in various steps in performance studies. The book contains lists of common mistakes in performance evaluation, benchmarking, data presentation, data analysis, experimental design, and simulation. While most of these mistakes are a result of a lack of knowledge of proper techniques, there are a number of tricks that some analysts knowingly use to show the superiority of their systems. A knowledge of such tricks, called games, will help protect the readers from being victimized by such analysts. The book discusses several games including those in benchmarking, analyzing data, and presenting results.

The analysis techniques discussed in the book have been illustrated using examples and case studies—all from the field of computer systems. This is an important aspect of this book. Textbooks on statistics and experimental designs using examples from other fields such as agriculture do not interest computer scientists as much as this book does. The case studies are from actual computer system design projects. To illustrate the use and misuse of various techniques in the literature, several examples of analyses published in technical journals are also presented. Overall, there are more than 150 examples and case studies in the book.

Important techniques and results have been summarized in “boxes.” There are more than 30 such boxes. It is expected that after a first reading, most readers would use these boxes for a quick reference.

The book consists of 36 chapters. Most chapters are organized so that each chapter can be presented in 45 minutes with some time left for discussion of exercises and their solutions in a typical 55-minute session of the class. This makes the book ideally suited for a one- or two-semester course. If the course is taught in one semester, some elementary material related to statistics and some of the advanced queueing techniques can be omitted or assigned for student reading. Most chapters also have exercises that can be assigned as homework.

Parts of this book were used as course notes in a graduate seminar on computer systems performance at the Massachusetts Institute of Technology. The students were system oriented with their primary goal being to design new computer systems. All of the students had previously taken courses in computer architectures, programming languages, and operating systems. Few had taken any advance courses on statistics, queueing theory, or probability. One of the course requirements included a project that required a performance analysis of their system. The projects were carried out on such diverse computer systems as LISP machines, data flow architectures, database querying packages, network protocols, and computer-aided design (CAD) tools.

Writing this book has been a monumental task. This being the first edition, many errors may have gone unnoticed. If you notice any errors or if you have any suggestions for improvement, please write to me care of the publisher.

RAJ JAIN

Littleton, Massachusetts