- Preface
- Acknowledgments
- PART IAN OVERVIEW OF PERFORMANCE EVALUATION
- CHAPTER 1INTRODUCTION
- 1.1 OUTLINE OF TOPICS
- 1.2 THE ART OF PERFORMANCE EVALUATION
- 1.3 PROFESSIONAL ORGANIZATIONS, JOURNALS, AND CONFERENCES
- 1.4 PERFORMANCE PROJECTS
- EXERCISE
- CHAPTER 2COMMON MISTAKES AND HOW TO AVOID THEM
- 2.1 COMMON MISTAKES IN PERFORMANCE EVALUATION
- 2.2 A SYSTEMATIC APPROACH TO PERFORMANCE EVALUATION
- EXERCISES
- CHAPTER 3SELECTION OF TECHNIQUES AND METRICS
- 3.1 SELECTING AN EVALUATION TECHNIQUE
- 3.2 SELECTING PERFORMANCE METRICS
- 3.3 COMMONLY USED PERFORMANCE METRICS
- 3.4 UTILITY CLASSIFICATION OF PERFORMANCE METRICS
- 3.5 SETTING PERFORMANCE REQUIREMENTS
- EXERCISES
- FURTHER READING FOR PART I
- PART IIMEASUREMENT TECHNIQUES AND TOOLS
- CHAPTER 4TYPES OF WORKLOADS
- 4.1 ADDITION INSTRUCTION
- 4.2 INSTRUCTION MIXES
- 4.3 KERNELS
- 4.4 SYNTHETIC PROGRAMS
- 4.5 APPLICATION BENCHMARKS
- 4.6 POPULAR BENCHMARKS
- 4.6.1 Sieve
- 4.6.2 Ackermanns Function
- 4.6.3 Whetstone
- 4.6.4 LINPACK
- 4.6.5 Dhrystone
- 4.6.6 Lawrence Livermore Loops
- 4.6.7 Debit-Credit Benchmark
- 4.6.8 SPEC Benchmark Suite
- EXERCISES
- CHAPTER 5THE ART OF WORKLOAD SELECTION
- 5.1 SERVICES EXERCISED
- 5.2 LEVEL OF DETAIL
- 5.3 REPRESENTATIVENESS
- 5.4 TIMELINESS
- 5.5 OTHER CONSIDERATIONS IN WORKLOAD SELECTION
- EXERCISES
- CHAPTER 6WORKLOAD CHARACTERIZATION TECHNIQUES
- 6.1 TERMINOLOGY
- 6.2 AVERAGING
- 6.3 SPECIFYING DISPERSION
- 6.4 SINGLE-PARAMETER HISTOGRAMS
- 6.5 MULTIPARAMETER HISTOGRAMS
- 6.6 PRINCIPAL-COMPONENT ANALYSIS
- 6.7 MARKOV MODELS
- 6.8 CLUSTERING
- 6.8.1 Sampling
- 6.8.2 Parameter Selection
- 6.8.3 Transformation
- 6.8.4 Outliers
- 6.8.5 Data Scaling
- 6.8.6 Distance Metric
- 6.8.7 Clustering Techniques
- 6.8.8 Minimum Spanning Tree Method
- 6.8.9 Cluster Interpretation
- 6.8.10 Problems with Clustering
- EXERCISES
- CHAPTER 7MONITORS
- 7.1 MONITOR TERMINOLOGY
- 7.2 MONITOR CLASSIFICATION
- 7.3 SOFTWARE MONITORS
- 7.3.1 Issues In Software Monitor Design
- 7.4 HARDWARE MONITORS
- 7.5 SOFTWARE VERSUS HARDWARE MONITORS
- 7.6 FIRMWARE AND HYBRID MONITORS
- 7.7 DISTRIBUTED-SYSTEM MONITORS
- 7.7.1 Observation
- 7.7.2 Collection
- 7.7.3 Analysis
- 7.7.4 Presentation
- 7.7.5 Interpretation
- 7.7.6 Console Functions
- EXERCISES
- CHAPTER 8PROGRAM EXECUTION MONITORS AND ACCOUNTING LOGS
- 8.1 PROGRAM EXECUTION MONITORS
- 8.1.1 Issues In Designing a Program Execution Monitor
- 8.2 TECHNIQUES FOR IMPROVING PROGRAM PERFORMANCE
- 8.3 ACCOUNTING LOGS
- 8.4 ANALYSIS AND INTERPRETATION OF ACCOUNTING LOG DATA
- 8.5 USING ACCOUNTING LOGS TO ANSWER COMMONLY ASKED QUESTIONS
- EXERCISE
- CHAPTER 9CAPACITY PLANNING AND BENCHMARKING
- 9.1 STEPS IN CAPACITY PLANNING AND MANAGEMENT
- 9.2 PROBLEMS IN CAPACITY PLANNING
- 9.3 COMMON MISTAKES IN BENCHMARKING
- 9.4 BENCHMARKING GAMES
- 9.5 LOAD DRIVERS
- 9.6 REMOTE-TERMINAL EMULATION
- 9.7 COMPONENTS OF AN RTE
- 9.8 LIMITATIONS OF CURRENT RTES
- EXERCISES
- CHAPTER 10THE ART OF DATA PRESENTATION
- 10.1 TYPES OF VARIABLES
- 10.2 GUIDELINES FOR PREPARING GOOD GRAPHIC CHARTS
- 10.3 COMMON MISTAKES IN PREPARING CHARTS
- 10.4 PICTORIAL GAMES
- 10.5 GANTT CHARTS
- 10.6 KIVIAT GRAPHS
- 10.6.1 Shapes of Kiviat Graphs
- 10.6.2 Application of Kiviat Graphs to Other Systems
- 10.7 SCHUMACHER CHARTS
- 10.8 DECISION MAKERS GAMES
- CHAPTER 11RATIO GAMES
- 11.1 CHOOSING AN APPROPRIATE BASE SYSTEM
- 11.2 USING AN APPROPRIATE RATIO METRIC
- 11.3 USING RELATIVE PERFORMANCE ENHANCEMENT
- 11.4 RATIO GAMES WITH PERCENTAGES
- 11.5 STRATEGIES FOR WINNING A RATIO GAME
- 11.6 CORRECT ANALYSIS
- EXERCISES
- FURTHER READING FOR PART II
- PART IIIPROBABILITY THEORY AND STATISTICS
- CHAPTER 12SUMMARIZING MEASURED DATA
- 12.1 BASIC PROBABILITY AND STATISTICS CONCEPTS
- 12.2 SUMMARIZING DATA BY A SINGLE NUMBER
- 12.3 SELECTING AMONG THE MEAN, MEDIAN, AND MODE
- 12.4 COMMON MISUSES OF MEANS
- 12.5 GEOMETRIC MEAN
- 12.6 HARMONIC MEAN
- 12.7 MEAN OF A RATIO
- 12.8 SUMMARIZING VARIABILITY
- 12.9 SELECTING THE INDEX OF DISPERSION
- 12.10 DETERMINING DISTRIBUTION OF DATA
- EXERCISES
- CHAPTER 13COMPARING SYSTEMS USING SAMPLE DATA
- 13.1 SAMPLE VERSUS POPULATION
- 13.2 CONFIDENCE INTERVAL FOR THE MEAN
- 13.3 TESTING FOR A ZERO MEAN
- 13.4 COMPARING TWO ALTERNATIVES
- 13.4.1 Paired Observations
- 13.4.2 Unpaired Observations
- 13.4.3 Approximate Visual Test
- 13.5 WHAT CONFIDENCE LEVEL TO USE
- 13.6 HYPOTHESIS TESTING VERSUS CONFIDENCE INTERVALS
- 13.7 ONE-SIDED CONFIDENCE INTERVALS
- 13.8 CONFIDENCE INTERVALS FOR PROPORTIONS
- 13.9 DETERMINING SAMPLE SIZE
- 13.9.1 Sample Size for Determining Mean
- 13.9.2 Sample Size for Determining Proportions
- 13.9.3 Sample Size for Comparing Two Alternatives
- CHAPTER 14SIMPLE LINEAR REGRESSION MODELS
- 14.1 DEFINITION OF A GOOD MODEL
- 14.2 ESTIMATION OF MODEL PARAMETERS
- 14.5 CONFIDENCE INTERVALS FOR REGRESSION PARAMETERS
- 14.6 CONFIDENCE INTERVALS FOR PREDICTIONS
- EXERCISES
- CHAPTER 15OTHER REGRESSION MODELS
- 15.1 MULTIPLE LINEAR REGRESSION MODELS
- 15.1.1 Analysis of Variance
- 15.1.2 Problem of Multicollinearity
- 15.2 REGRESSION WITH CATEGORICAL PREDICTORS
- 15.3 CURVILINEAR REGRESSION
- 15.4 TRANSFORMATIONS
- 15.5 OUTLIERS
- 15.6 COMMON MISTAKES IN REGRESSION
- EXERCISES
- FURTHER READING FOR PART III
- PART IVEXPERIMENTAL DESIGN AND ANALYSIS
- CHAPTER 16INTRODUCTION TO EXPERIMENTAL DESIGN
- 16.1 TERMINOLOGY
- 16.2 COMMON MISTAKES IN EXPERIMENTATION
- 16.3 TYPES OF EXPERIMENTAL DESIGNS
- 16-3.1 Simple Designs
- 16.3.2 Full Factorial Design
- 16.3.3 Fractional Factorial Designs
- EXERCISE
- CHAPTER 172k FACTORIAL DESIGNS
- 17.1 22 FACTORIAL DESIGNS
- 17.2 COMPUTATION OF EFFECTS
- 17.3 SIGN TABLE METHOD FOR CALCULATING EFFECTS
- 17.4 ALLOCATION OF VARIATION
- 17.5 GENERAL 2k FACTORIAL DESIGNS
- EXERCISE
- CHAPTER 182kr FACTORIAL DESIGNS WITH REPLICATIONS
- 18.1 22r FACTORIAL DESIGNS
- 18.2 COMPUTATION OF EFFECTS
- 18.3 ESTIMATION OF EXPERIMENTAL ERRORS
- 18.4 ALLOCATION OF VARIATION
- 18.5 CONFIDENCE INTERVALS FOR EFFECTS
- 18.6 CONFIDENCE INTERVALS FOR PREDICTED RESPONSES
- 18.7 VISUAL TESTS FOR VERIFYING THE ASSUMPTIONS
- 18.9 GENERAL 2kr FACTORIAL DESIGN
- EXERCISE
- CHAPTER 192kp FRACTIONAL FACTORIAL DESIGNS
- 19.1 PREPARING THE SIGN TABLE FOR A 2kp DESIGN
- 19.2 CONFOUNDING
- 19.3 ALGEBRA OF CONFOUNDING
- 19.4 DESIGN RESOLUTION
- EXERCISES
- CHAPTER 20ONE-FACTOR EXPERIMENTS
- 20.1 MODEL
- 20.2 COMPUTATION OF EFFECTS
- 20.3 ESTIMATING EXPERIMENTAL ERRORS
- 20.4 ALLOCATION OF VARIATION
- 20.5 ANALYSIS OF VARIANCE
- 20.6 VISUAL DIAGNOSTIC TESTS
- 20.7 CONFIDENCE INTERVALS FOR EFFECTS
- 20.8 UNEQUAL SAMPLE SIZES
- EXERCISE
- CHAPTER 21TWO-FACTOR FULL FACTORIAL DESIGN WITHOUT REPLICATIONS
- 21.1 MODEL
- 21.2 COMPUTATION OF EFFECTS
- 21.3 ESTIMATING EXPERIMENTAL ERRORS
- 21.4 ALLOCATION OF VARIATION
- 21.5 ANALYSIS OF VARIANCE
- 21.6 CONFIDENCE INTERVALS FOR EFFECTS
- 21.7 MULTIPLICATIVE MODELS FOR TWO-FACTOR EXPERIMENTS
- 21.8 MISSING OBSERVATIONS
- EXERCISES
- CHAPTER 22TWO-FACTOR FULL FACTORIAL DESIGN WITH REPLICATIONS
- 22.1 MODEL
- 22.2 COMPUTATION OF EFFECTS
- 22.3 COMPUTATION OF ERRORS
- 22.4 ALLOCATION OF VARIATION
- 22.5 ANALYSIS OF VARIANCE
- 22.6 CONFIDENCE INTERVALS FOR EFFECTS
- CHAPTER 23GENERAL FULL FACTORIAL DESIGNS WITH k FACTORS
- 23.1 MODEL
- 23.2 ANALYSIS OF A GENERAL DESIGN
- 23.3 INFORMAL METHODS
- 23.3.1 Observation Method
- 23.3.2 Ranking Method
- 23.3.3 Range Method
- EXERCISES
- FURTHER READING FOR PART IV
- PART VSIMULATION
- CHAPTER 24INTRODUCTION TO SIMULATION
- 24.1 COMMON MISTAKES IN SIMULATION
- 24.2 OTHER CAUSES OF SIMULATION ANALYSIS FAILURE
- 24.3 TERMINOLOGY
- 24.4 SELECTING A LANGUAGE FOR SIMULATION
- 24.5 TYPES OF SIMULATIONS
- 24.5.1 Monte Carlo Simulation
- 24.5.2 Trace-Driven Simulation
- 24.5.3 Discrete-Event Simulations
- 24.6 EVENT-SET ALGORITHMS
- EXERCISES
- CHAPTER 25ANALYSIS OF SIMULATION RESULTS
- 25.1 MODEL VERIFICATION TECHNIQUES
- 25.1.1 Top-Down Modular Design
- 25.1.2 Antibugging
- 25.1.3 Structured Walk-Through
- 25.1.4 Deterministic Models
- 25.1.5 Run Simplified Cases
- 25.1.6 Trace
- 25.1.7 On-Line Graphic Displays
- 25.1.8 Continuity Test
- 25.1.9 Degeneracy Tests
- 25.1.10 Consistency Tests
- 25.1.11 Seed Independence
- 25.2 MODEL VALIDATION TECHNIQUES
- 25.2.1 Expert Intuition
- 25.2.2 Real-System Measurements
- 25.2.3 Theoretical Results
- 25.3 TRANSIENT REMOVAL
- 25.3.1 Long Runs
- 25.3.2 Proper Initialization
- 25.3.3 Truncation
- 25.3.4 Initial Date Deletion
- 25.3.5 Moving Average of Independent Replications
- 25.3.6 Batch Means
- 25.4 TERMINATING SIMULATIONS
- 25.5 STOPPING CRITERIA: VARIANCE ESTIMATION
- 25.5.1 Independent Replications
- 25.5.2 Batch Means
- 25.5.3 Method of Regeneration
- 25.6 VARIANCE REDUCTION
- EXERCISES
- CHAPTER 26RANDOM-NUMBER GENERATION
- 26.1 DESIRED PROPERTIES OF A GOOD GENERATOR
- 26.2 LINEAR-CONGRUENTIAL GENERATORS
- 26.2.1 Multiplicative LCG
- 26.2.2 Multiplicative LCG with m = 2k
- 26.2.3 Multiplicative LCG with m ≠ 2k
- 26.3 TAUSWORTHE GENERATORS
- 26.4 EXTENDED FIBONACCI GENERATORS
- 26.5 COMBINED GENERATORS
- 26.6 A SURVEY OF RANDOM-NUMBER GENERATORS
- 26.7 SEED SELECTION
- 26.8 MYTHS ABOUT RANDOM-NUMBER GENERATION
- EXERCISES
- CHAPTER 27TESTING RANDOM-NUMBER GENERATORS
- 27.1 CHI-SQUARE TEST
- 27.2 KOLMOGOROV-SMIRNOV TEST
- 27.3 SERIAL-CORRELATION TEST
- 27.4 TWO-LEVEL TESTS
- 27.5 k-DIMENSIONAL UNIFORMITY OR k-DISTRIBUTIVITY
- 27.6 SERIAL TEST
- 27.7 SPECTRAL TEST
- EXERCISES
- CHAPTER 28RANDOM-VARIATE GENERATION
- 28.1 INVERSE TRANSFORMATION
- 28.2 REJECTION
- 28.3 COMPOSITION
- 28.4 CONVOLUTION
- 28.5 CHARACTERIZATION
- EXERCISE
- CHAPTER 29COMMONLY USED DISTRIBUTIONS
- 29.1 BERNOULLI DISTRIBUTION
- 29.2 BETA DISTRIBUTION
- 29.3 BINOMIAL DISTRIBUTION
- 29.4 CHI-SQUARE DISTRIBUTION
- 29.5 ERLANG DISTRIBUTION
- 29.6 EXPONENTIAL DISTRIBUTION
- 29.7 F DISTRIBUTION
- 29.8 GAMMA DISTRIBUTION
- 29.9 GEOMETRIC DISTRIBUTION
- 29.10 LOGNORMAL DISTRIBUTION
- 29.11 NEGATIVE BINOMIAL DISTRIBUTION
- 29.12 NORMAL DISTRIBUTION
- 29.13 PARETO DISTRIBUTION
- 29.14 PASCAL DISTRIBUTION
- 29.15 POISSON DISTRIBUTION
- 29.16 STUDENTS t DISTRIBUTION
- 29.17 UNIFORM DISTRIBUTION (CONTINUOUS)
- 29.18 UNIFORM DISTRIBUTION (DISCRETE)
- 29.19 WEIBULL DISTRIBUTION
- 29.20 RELATIONSHIPS AMONG DISTRIBUTIONS
- EXERCISES
- FURTHER READING FOR PART V
- CURRENT AREAS OF RESEARCH IN SIMULATION
- PART VIQUEUEING MODELS
- CHAPTER 30INTRODUCTION TO QUEUEING THEORY
- 30.1 QUEUEING NOTATION
- 30.2 RULES FOR ALL QUEUES
- 30.3 LITTLES LAW
- 30.4 TYPES OF STOCHASTIC PROCESSES
- EXERCISES
- CHAPTER 31ANALYSIS OF A SINGLE QUEUE
- 31.1 BIRTH-DEATH PROCESSES
- 31.2 M/M/1 QUEUE
- 31.3 M/M/m QUEUE
- 31.4 M/M/m/B OUEUE WITH FINITE BUFFERS
- 31.5 RESULTS FOR OTHER QUEUEING SYSTEMS
- EXERCISES
- CHAPTER 32QUEUEING NETWORKS
- 32.1 OPEN AND CLOSED QUEUEING NETWORKS
- 32.2 PRODUCT FORM NETWORKS
- 32.3 QUEUEING NETWORK MODELS OF COMPUTER SYSTEMS
- EXERCISE
- CHAPTER 33OPERATIONAL LAWS
- 33.1 UTILIZATION LAW
- 33.2 FORCED FLOW LAW
- 33.3 LITTLE'S LAW
- 33.4 GENERAL RESPONSE TIME LAW
- 33.5 INTERACTIVE RESPONSE TIME LAW
- 33.6 BOTTLENECK ANALYSIS
- EXERCISES
- CHAPTER 34MEAN-VALUE ANALYSIS AND RELATED TECHNIQUES
- 34.1 ANALYSIS OF OPEN QUEUEING NETWORKS
- 34.2 MEAN-VALUE ANALYSIS
- 34.3 APPROXIMATE MVA
- 34.4 BALANCED JOB BOUNDS
- EXERCISES
- CHAPTER 35CONVOLUTION ALGORITHM
- 35.1 DISTRIBUTION OF JOBS IN A SYSTEM
- 35.2 CONVOLUTION ALGORITHM FOR COMPUTING G(N)
- 35.3 COMPUTING PERFORMANCE USING G(N)
- 35.4 TIMESHARING SYSTEMS
- EXERCISES
- CHAPTER 36HIERARCHICAL DECOMPOSITION OF LARGE QUEUEING NETWORKS
- 36.1 LOAD-DEPENDENT SERVICE CENTERS
- 36.2 HIERARCHICAL DECOMPOSITION
- 36.3 LIMITATIONS OF QUEUEING THEORY
- EXERCISES
- FURTHER READING FOR PART VI
- SYMBOLS FREQUENTLY USED IN QUEUEING ANALYSIS
References
Appendix A
Appendix B
Appendix C
Subject Index