|
Part I Fundamentals of Probability and Probability Distributions |
|
|
|
|
3 | (28) |
|
1.1 Random Experiments and Events |
|
|
3 | (3) |
|
|
6 | (2) |
|
1.2.1 Variations and Permutations |
|
|
6 | (1) |
|
1.2.2 Combinations Without Repetition |
|
|
7 | (1) |
|
1.2.3 Combinations with Repetition |
|
|
8 | (1) |
|
1.3 Properties of Probability |
|
|
8 | (3) |
|
1.4 Conditional Probability |
|
|
11 | (7) |
|
|
14 | (2) |
|
|
16 | (2) |
|
|
18 | (13) |
|
1.5.1 Boltzmann, Bose-Einstein and Fermi-Dirac Distributions |
|
|
18 | (1) |
|
|
19 | (2) |
|
1.5.3 Independence of Events in Particle Detection |
|
|
21 | (1) |
|
1.5.4 Searching for the Lost Plane |
|
|
22 | (1) |
|
1.5.5 The Monty Hall Problem * |
|
|
22 | (3) |
|
1.5.6 Bayes Formula in Medical Diagnostics |
|
|
25 | (2) |
|
1.5.7 One-Dimensional Random Walk * |
|
|
27 | (2) |
|
|
29 | (2) |
|
2 Probability Distributions |
|
|
31 | (34) |
|
|
31 | (4) |
|
2.1.1 Composition of the Dirac Delta with a Function |
|
|
33 | (2) |
|
|
35 | (1) |
|
2.3 Discrete and Continuous Distributions |
|
|
36 | (1) |
|
|
37 | (1) |
|
2.5 One-Dimensional Discrete Distributions |
|
|
37 | (2) |
|
2.6 One-Dimensional Continuous Distributions |
|
|
39 | (2) |
|
2.7 Transformation of Random Variables |
|
|
41 | (4) |
|
2.7.1 What If the Inverse of y = h(x) Is Not Unique? |
|
|
44 | (1) |
|
2.8 Two-Dimensional Discrete Distributions |
|
|
45 | (2) |
|
2.9 Two-Dimensional Continuous Distributions |
|
|
47 | (3) |
|
2.10 Transformation of Variables in Two and More Dimensions |
|
|
50 | (6) |
|
|
56 | (9) |
|
2.11.1 Black-Body Radiation |
|
|
56 | (1) |
|
2.11.2 Energy Losses of Particles in a Planar Detector |
|
|
57 | (1) |
|
2.11.3 Computing Marginal Probability Densities from a Joint Density |
|
|
58 | (2) |
|
2.11.4 Independence of Random Variables in Two Dimensions |
|
|
60 | (1) |
|
2.11.5 Transformation of Variables in Two Dimensions |
|
|
61 | (2) |
|
2.11.6 Distribution of Maximal and Minimal Values |
|
|
63 | (1) |
|
|
64 | (1) |
|
3 Special Continuous Probability Distributions |
|
|
65 | (28) |
|
|
65 | (2) |
|
3.2 Exponential Distribution |
|
|
67 | (3) |
|
3.2.1 Is the Decay of Unstable States Truly Exponential? |
|
|
70 | (1) |
|
3.3 Normal (Gauss) Distribution |
|
|
70 | (4) |
|
3.3.1 Standardized Normal Distribution |
|
|
71 | (2) |
|
3.3.2 Measure of Peak Separation |
|
|
73 | (1) |
|
|
74 | (1) |
|
|
75 | (2) |
|
3.5.1 Estimating the Maximum x in the Sample |
|
|
77 | (1) |
|
|
77 | (2) |
|
|
79 | (1) |
|
3.8 Student's Distribution |
|
|
79 | (1) |
|
|
80 | (1) |
|
|
80 | (13) |
|
3.10.1 In-Flight Decay of Neutral Pions |
|
|
80 | (3) |
|
3.10.2 Product of Uniformly Distributed Variables |
|
|
83 | (1) |
|
3.10.3 Joint Distribution of Exponential Variables |
|
|
84 | (1) |
|
3.10.4 Integral of Maxwell Distribution over Finite Range |
|
|
85 | (1) |
|
3.10.5 Decay of Unstable States and the Hyper-exponential Distribution |
|
|
86 | (3) |
|
3.10.6 Nuclear Decay Chains and the Hypo-exponential Distribution |
|
|
89 | (2) |
|
|
91 | (2) |
|
|
93 | (30) |
|
4.1 Expected (Average, Mean) Value |
|
|
93 | (2) |
|
|
95 | (1) |
|
|
96 | (2) |
|
4.4 Expected Values of Functions of Random Variables |
|
|
98 | (2) |
|
4.4.1 Probability Densities in Quantum Mechanics |
|
|
99 | (1) |
|
4.5 Variance and Effective Deviation |
|
|
100 | (1) |
|
4.6 Complex Random Variables |
|
|
101 | (1) |
|
|
102 | (4) |
|
4.7.1 Moments of the Cauchy Distribution |
|
|
105 | (1) |
|
4.8 Two- and d-dimensional Generalizations |
|
|
106 | (5) |
|
4.8.1 Multivariate Normal Distribution |
|
|
110 | (1) |
|
4.8.2 Correlation Does Not Imply Causality |
|
|
111 | (1) |
|
4.9 Propagation of Errors |
|
|
111 | (4) |
|
4.9.1 Multiple Functions and Transformation of the Co variance Matrix |
|
|
113 | (2) |
|
|
115 | (8) |
|
4.10.1 Expected Device Failure Time |
|
|
115 | (1) |
|
4.10.2 Covariance of Continuous Random Variables |
|
|
116 | (1) |
|
4.10.3 Conditional Expected Values of Two-Dimensional Distributions |
|
|
117 | (1) |
|
4.10.4 Expected Values of Hyper- and Hypo-exponential Variables |
|
|
117 | (2) |
|
4.10.5 Gaussian Noise in an Electric Circuit |
|
|
119 | (1) |
|
4.10.6 Error Propagation in a Measurement of the Momentum Vector * |
|
|
120 | (1) |
|
|
121 | (2) |
|
5 Special Discrete Probability Distributions |
|
|
123 | (20) |
|
5.1 Binomial Distribution |
|
|
123 | (5) |
|
5.1.1 Expected Value and Variance |
|
|
126 | (2) |
|
5.2 Multinomial Distribution |
|
|
128 | (1) |
|
5.3 Negative Binomial (Pascal) Distribution |
|
|
129 | (1) |
|
5.3.1 Negative Binomial Distribution of Order k |
|
|
129 | (1) |
|
5.4 Normal Approximation of the Binomial Distribution |
|
|
130 | (2) |
|
|
132 | (3) |
|
|
135 | (8) |
|
5.6.1 Detection Efficiency |
|
|
135 | (1) |
|
5.6.2 The Newsboy Problem * |
|
|
136 | (2) |
|
5.6.3 Time to Critical Error |
|
|
138 | (2) |
|
5.6.4 Counting Events with an Inefficient Detector |
|
|
140 | (1) |
|
5.6.5 Influence of Primary Ionization on Spatial Resolution * |
|
|
140 | (2) |
|
|
142 | (1) |
|
6 Stable Distributions and Random Walks |
|
|
143 | (34) |
|
6.1 Convolution of Continuous Distributions |
|
|
143 | (4) |
|
6.1.1 The Effect of Convolution on Distribution Moments |
|
|
146 | (1) |
|
6.2 Convolution of Discrete Distributions |
|
|
147 | (2) |
|
6.3 Central Limit Theorem |
|
|
149 | (4) |
|
6.3.1 Proof of the Central Limit Theorem |
|
|
150 | (3) |
|
6.4 Stable Distributions * |
|
|
153 | (2) |
|
6.5 Generalized Central Limit Theorem * |
|
|
155 | (1) |
|
6.6 Extreme-Value Distributions * |
|
|
156 | (6) |
|
6.6.1 Fisher--Tippett--Gnedenko Theorem |
|
|
158 | (1) |
|
6.6.2 Return Values and Return Periods |
|
|
159 | (2) |
|
6.6.3 Asymptotics of Minimal Values |
|
|
161 | (1) |
|
6.7 Discrete-Time Random Walks * |
|
|
162 | (3) |
|
|
163 | (2) |
|
6.8 Continuous-Time Random Walks * |
|
|
165 | (2) |
|
|
167 | (10) |
|
6.9.1 Convolutions with the Normal Distribution |
|
|
167 | (1) |
|
6.9.2 Spectral Line Width |
|
|
168 | (1) |
|
6.9.3 Random Structure of Polymer Molecules |
|
|
169 | (2) |
|
6.9.4 Scattering of Thermal Neutrons in Lead |
|
|
171 | (1) |
|
6.9.5 Distribution of Extreme Values of Normal Variables * |
|
|
172 | (2) |
|
|
174 | (3) |
|
Part II Determination of Distribution Parameters |
|
|
|
7 Statistical Inference from Samples |
|
|
177 | (26) |
|
7.1 Statistics and Estimators |
|
|
178 | (6) |
|
7.1.1 Sample Mean and Sample Variance |
|
|
179 | (5) |
|
7.2 Three Important Sample Distributions |
|
|
184 | (4) |
|
7.2.1 Sample Distribution of Sums and Differences |
|
|
184 | (1) |
|
7.2.2 Sample Distribution of Variances |
|
|
185 | (1) |
|
7.2.3 Sample Distribution of Variance Ratios |
|
|
186 | (2) |
|
|
188 | (4) |
|
7.3.1 Confidence Interval for Sample Mean |
|
|
188 | (3) |
|
7.3.2 Confidence Interval for Sample Variance |
|
|
191 | (1) |
|
7.3.3 Confidence Region for Sample Mean and Variance |
|
|
191 | (1) |
|
7.4 Outliers and Robust Measures of Mean and Variance |
|
|
192 | (3) |
|
|
193 | (1) |
|
7.4.2 Distribution of Sample Median (and Sample Quantiles) |
|
|
194 | (1) |
|
|
195 | (3) |
|
7.5.1 Linear (Pearson) Correlation |
|
|
195 | (1) |
|
7.5.2 Non-parametric (Spearman) Correlation |
|
|
196 | (2) |
|
|
198 | (5) |
|
7.6.1 Estimator of Third Moment |
|
|
198 | (1) |
|
7.6.2 Unbiasedness of Poisson Variable Estimators |
|
|
199 | (1) |
|
7.6.3 Concentration of Mercury in Fish |
|
|
199 | (2) |
|
7.6.4 Dosage of Active Ingredient |
|
|
201 | (1) |
|
|
201 | (2) |
|
8 Maximum-Likelihood Method |
|
|
203 | (24) |
|
|
203 | (1) |
|
8.2 Principle of Maximum Likelihood |
|
|
204 | (2) |
|
8.3 Variance of Estimator |
|
|
206 | (3) |
|
8.3.1 Limit of Large Samples |
|
|
207 | (2) |
|
8.4 Efficiency of Estimator |
|
|
209 | (3) |
|
|
212 | (2) |
|
8.6 Simultaneous Determination of Multiple Parameters |
|
|
214 | (3) |
|
8.6.1 General Method for Arbitrary (Small or Large) Samples |
|
|
214 | (1) |
|
8.6.2 Asymptotic Method (Large Samples) |
|
|
215 | (2) |
|
|
217 | (2) |
|
8.7.1 Alternative Likelihood Regions |
|
|
218 | (1) |
|
|
219 | (8) |
|
8.8.1 Lifetime of Particles in Finite Detector |
|
|
219 | (2) |
|
8.8.2 Device Failure Due to Corrosion |
|
|
221 | (1) |
|
8.8.3 Distribution of Extreme Rainfall |
|
|
222 | (2) |
|
8.8.4 Tensile Strength of Glass Fibers |
|
|
224 | (1) |
|
|
224 | (3) |
|
9 Method of Least Squares |
|
|
227 | (32) |
|
|
228 | (14) |
|
9.1.1 Fitting a Polynomial, Known Uncertainties |
|
|
230 | (2) |
|
9.1.2 Fitting Observations with Unknown Uncertainties |
|
|
232 | (3) |
|
9.1.3 Confidence Intervals for Optimal Parameters |
|
|
235 | (1) |
|
9.1.4 How "Good" Is the Fit? |
|
|
236 | (1) |
|
9.1.5 Regression with Orthogonal Polynomials * |
|
|
236 | (1) |
|
9.1.6 Fitting a Straight Line |
|
|
237 | (3) |
|
9.1.7 Fitting a Straight Line with Uncertainties in both Coordinates |
|
|
240 | (1) |
|
|
240 | (2) |
|
9.1.9 Are We Allowed to Simply Discard Some Data? |
|
|
242 | (1) |
|
9.2 Linear Regression for Binned Data |
|
|
242 | (3) |
|
9.3 Linear Regression with Constraints |
|
|
245 | (3) |
|
9.4 General Linear Regression by Singular-Value Decomposition * |
|
|
248 | (1) |
|
9.5 Robust Linear Regression |
|
|
249 | (1) |
|
9.6 Non-linear Regression |
|
|
250 | (3) |
|
|
253 | (6) |
|
9.7.1 Two Gaussians on Exponential Background |
|
|
253 | (1) |
|
9.7.2 Time Dependence of the Pressure Gradient |
|
|
254 | (1) |
|
9.7.3 Thermal Expansion of Copper |
|
|
255 | (1) |
|
9.7.4 Electron Mobility in Semiconductor |
|
|
255 | (1) |
|
9.7.5 Quantum Defects in Iodine Atoms |
|
|
256 | (1) |
|
9.7.6 Magnetization in Superconductor |
|
|
257 | (1) |
|
|
257 | (2) |
|
10 Statistical Tests: Verifying Hypotheses |
|
|
259 | (24) |
|
|
259 | (5) |
|
10.2 Parametric Tests for Normal Variables |
|
|
264 | (5) |
|
10.2.1 Test of Sample Mean |
|
|
264 | (1) |
|
10.2.2 Test of Sample Variance |
|
|
265 | (1) |
|
10.2.3 Comparison of Two Sample Means, σ2x = σ2y |
|
|
265 | (1) |
|
10.2.4 Comparison of Two Sample Means, σ2x ≠ σ2y |
|
|
266 | (1) |
|
10.2.5 Comparison of Two Sample Variances |
|
|
267 | (2) |
|
|
269 | (2) |
|
10.3.1 Comparing Two Sets of Binned Data |
|
|
271 | (1) |
|
10.4 Kolmogorov--Smirnov Test |
|
|
271 | (5) |
|
10.4.1 Comparison of Two Samples |
|
|
274 | (1) |
|
10.4.2 Other Tests Based on Empirical Distribution Functions |
|
|
275 | (1) |
|
|
276 | (7) |
|
10.5.1 Test of Mean Decay Time |
|
|
276 | (2) |
|
10.5.2 Pearson's Test for Two Histogrammed Samples |
|
|
278 | (1) |
|
|
279 | (1) |
|
|
279 | (1) |
|
|
280 | (3) |
|
Part III Special Applications of Probability |
|
|
|
11 Entropy and Information * |
|
|
283 | (24) |
|
11.1 Measures of Information and Entropy |
|
|
283 | (5) |
|
11.1.1 Entropy of Infinite Discrete Probability Distribution |
|
|
285 | (1) |
|
11.1.2 Entropy of a Continuous Probability Distribution |
|
|
286 | (1) |
|
11.1.3 Kullback--Leibler Distance |
|
|
287 | (1) |
|
11.2 Principle of Maximum Entropy |
|
|
288 | (1) |
|
11.3 Discrete Distributions with Maximum Entropy |
|
|
289 | (8) |
|
11.3.1 Lagrange Formalism for Discrete Distributions |
|
|
289 | (2) |
|
11.3.2 Distribution with Prescribed Mean and Maximum Entropy |
|
|
291 | (1) |
|
11.3.3 Maxwell--Boltzmann Distribution |
|
|
292 | (2) |
|
11.3.4 Relation Between Information and Thermodynamic Entropy |
|
|
294 | (1) |
|
11.3.5 Bose--Einstein Distribution |
|
|
295 | (1) |
|
11.3.6 Fermi--Dirac Distribution |
|
|
296 | (1) |
|
11.4 Continuous Distributions with Maximum Entropy |
|
|
297 | (1) |
|
11.5 Maximum-Entropy Spectral Analysis |
|
|
298 | (9) |
|
11.5.1 Calculating the Lagrange Multipliers |
|
|
300 | (2) |
|
11.5.2 Estimating the Spectrum |
|
|
302 | (2) |
|
|
304 | (3) |
|
|
307 | (18) |
|
12.1 Discrete-Time (Classical) Markov Chains |
|
|
308 | (5) |
|
12.1.1 Long-Time Characteristics of Markov Chains |
|
|
309 | (4) |
|
12.2 Continuous-Time Markov Processes |
|
|
313 | (12) |
|
12.2.1 Markov Propagator and Its Moments |
|
|
314 | (2) |
|
12.2.2 Time Evolution of the Moments |
|
|
316 | (1) |
|
|
317 | (1) |
|
12.2.4 Ornstein--Uhlenbeck Process |
|
|
318 | (5) |
|
|
323 | (2) |
|
13 The Monte--Carlo Method |
|
|
325 | (22) |
|
13.1 Historical Introduction and Basic Idea |
|
|
325 | (3) |
|
13.2 Numerical Integration |
|
|
328 | (5) |
|
13.2.1 Advantage of Monte--Carlo Methods over Quadrature Formulas |
|
|
332 | (1) |
|
|
333 | (6) |
|
13.3.1 Importance Sampling |
|
|
333 | (4) |
|
13.3.2 The Monte--Carlo Method with Quasi-Random Sequences |
|
|
337 | (2) |
|
13.4 Markov--Chain Monte Carlo * |
|
|
339 | (8) |
|
13.4.1 Metropolis--Hastings Algorithm |
|
|
339 | (6) |
|
|
345 | (2) |
|
14 Stochastic Population Modeling |
|
|
347 | (14) |
|
|
347 | (1) |
|
|
348 | (3) |
|
14.3 Modeling Births and Deaths |
|
|
351 | (6) |
|
|
352 | (1) |
|
14.3.2 General Solution in the Case λn = Nλ, μn = Nμ, λ ≠ μ |
|
|
353 | (1) |
|
14.3.3 General Solution in the Case λn = Nλ, μn = Nμ, λ = μ |
|
|
353 | (1) |
|
14.3.4 Extinction Probability |
|
|
353 | (1) |
|
14.3.5 Moments of the Distribution P(t) in the Case λn = nλ, μn = nμ |
|
|
354 | (3) |
|
14.4 Concluding Example: Rabbits and Foxes |
|
|
357 | (4) |
|
|
359 | (2) |
Appendix A Probability as Measure * |
|
361 | (4) |
Appendix B Generating and Characteristic Functions + |
|
365 | (16) |
Appendix C Random Number Generators |
|
381 | (14) |
Appendix D Tables of Distribution Quantiles |
|
395 | (14) |
Index |
|
409 | |