These notes describe how to average and fit numerical data that have been obtained either by simulation or measurement. Following an introduction on how to estimate various average values, they discuss how to determine error bars on those estimates, and how to proceed for combinations of measured values. Techniques for fitting data to a given set of models will be described in the second part of these notes. This primer equips readers to properly derive the results covered, presenting the content in a style suitable for a physics audience. It also includes scripts in python, perl and gnuplot for performing a number of tasks in data analysis and fitting, thereby providing readers with a useful reference guide.
|
|
|
1 | (2) |
|
|
|
1 | (2) |
|
2 Averages and Error Bars |
|
|
3 | (18) |
|
|
|
3 | (6) |
|
|
|
9 | (12) |
|
|
|
10 | (3) |
|
|
|
13 | (2) |
|
|
|
15 | (4) |
|
2.2.4 Jackknife or Bootstrap? |
|
|
19 | (1) |
|
|
|
19 | (2) |
|
3 Fitting Data to a Model |
|
|
21 | (34) |
|
3.1 Fitting to a Straight Line |
|
|
23 | (2) |
|
3.2 Fitting to a Polynomial |
|
|
25 | (1) |
|
|
|
25 | (5) |
|
|
|
30 | (2) |
|
3.5 Fitting to a Non-linear Model |
|
|
32 | (1) |
|
|
|
32 | (4) |
|
3.7 Confidence Limits by Resampling the Data |
|
|
36 | (1) |
|
3.8 A Tale of Two Probabilities. When Can One Rule Out a Fit? |
|
|
37 | (2) |
|
3.9 Model Selection (i.e. How to Avoid Over-Fitting): Maximum Likelihood Versus Bayes |
|
|
39 | (16) |
|
|
|
42 | (2) |
|
3.9.2 A Bayesian Approach |
|
|
44 | (8) |
|
3.9.3 Conclusions for Model Selection |
|
|
52 | (1) |
|
|
|
53 | (2) |
| Appendix A Central Limit Theorem |
|
55 | (6) |
| Appendix B The Number of Degrees of Freedom |
|
61 | (2) |
| Appendix C The Chi-squared Distribution and the Goodness of Fit Parameter Q |
|
63 | (4) |
| Appendix D Asymptotic Standard Error and How to Get Correct Error Bars from Gnuplot |
|
67 | (4) |
| Appendix E The Distribution of Fitted Parameters Determined from Simulated Datasets |
|
71 | (2) |
| Appendix F The Distribution of Fitted Parameters from Repeated Sets of Measurements |
|
73 | (2) |
| Appendix G Fitting Correlated Data |
|
75 | (2) |
| Appendix H Scripts for Some Data Analysis and Fitting Tasks |
|
77 | |
Research Professor of Physics. D. Phil., Oxford University 1973 Honor and Awards: 2014 Martin-Gutzwiller-Fellowship at the Max Planck Institute for the Physics of Complex Systems, Dresden 2012 Member of the American Academy of Arts and Sciences 2012 Humboldt Research Award (Humboldt Prize) of the Alexander von Humboldt Foundation 2009 Aneesur Rahman Prize of the American Physical Society for Computational Physics 2008 Outstanding Referee of the American Physical Society 2004-2005 Outstanding Faculty Award of the Division of Physical and Biological Sciences. 2003-2004 Excellence in Teaching Award of the University of California, Santa Cruz 1989, Fellow of the American Physical Society 1985 Maxwell Medal and Prize from the Institute of Physics (UK)
