Preface |
|
xiii | |
Acknowledgments |
|
xix | |
|
|
xxi | |
|
|
1 | (34) |
|
|
1 | (2) |
|
1.2 Population-Based Discriminants |
|
|
3 | (5) |
|
|
8 | (5) |
|
1.4 Sample-Based Discriminants |
|
|
13 | (3) |
|
1.4.1 Quadratic Discriminants |
|
|
14 | (1) |
|
1.4.2 Linear Discriminants |
|
|
15 | (1) |
|
1.4.3 Kernel Discriminants |
|
|
16 | (1) |
|
|
16 | (4) |
|
1.6 Other Classification Rules |
|
|
20 | (5) |
|
1.6.1 k-Nearest-Neighbor Rules |
|
|
20 | (1) |
|
1.6.2 Support Vector Machines |
|
|
21 | (1) |
|
|
22 | (1) |
|
1.6.4 Classification Trees |
|
|
23 | (1) |
|
|
24 | (1) |
|
|
25 | (10) |
|
|
28 | (7) |
|
|
35 | (42) |
|
2.1 Error Estimation Rules |
|
|
35 | (3) |
|
|
38 | (5) |
|
2.2.1 Deviation Distribution |
|
|
39 | (2) |
|
|
41 | (1) |
|
2.2.3 Conditional Expectation |
|
|
41 | (1) |
|
|
42 | (1) |
|
2.2.5 Confidence Intervals |
|
|
42 | (1) |
|
2.3 Test-Set Error Estimation |
|
|
43 | (3) |
|
|
46 | (2) |
|
|
48 | (7) |
|
|
55 | (2) |
|
2.7 Convex Error Estimation |
|
|
57 | (4) |
|
2.8 Smoothed Error Estimation |
|
|
61 | (2) |
|
2.9 Bolstered Error Estimation |
|
|
63 | (14) |
|
2.9.1 Gaussian-Bolstered Error Estimation |
|
|
67 | (1) |
|
2.9.2 Choosing the Amount of Bolstering |
|
|
68 | (3) |
|
2.9.3 Calibrating the Amount of Bolstering |
|
|
71 | (2) |
|
|
73 | (4) |
|
|
77 | (20) |
|
3.1 Empirical Deviation Distribution |
|
|
77 | (2) |
|
|
79 | (3) |
|
3.3 Impact on Feature Selection |
|
|
82 | (2) |
|
3.4 Multiple-Data-Set Reporting Bias |
|
|
84 | (2) |
|
|
86 | (6) |
|
3.6 Performance Reproducibility |
|
|
92 | (5) |
|
|
94 | (3) |
|
4 Error Estimation For Discrete Classification |
|
|
97 | (18) |
|
|
98 | (3) |
|
4.1.1 Resubstitution Error |
|
|
98 | (1) |
|
4.1.2 Leave-One-Out Error |
|
|
98 | (1) |
|
4.1.3 Cross-Validation Error |
|
|
99 | (1) |
|
|
99 | (2) |
|
4.2 Small-Sample Performance |
|
|
101 | (9) |
|
|
101 | (2) |
|
|
103 | (2) |
|
4.2.3 Deviation Variance, RMS, and Correlation |
|
|
105 | (1) |
|
|
106 | (2) |
|
4.2.5 Complete Enumeration Approach |
|
|
108 | (2) |
|
4.3 Large-Sample Performance |
|
|
110 | (5) |
|
|
114 | (1) |
|
|
115 | (30) |
|
5.1 Mixture Sampling Versus Separate Sampling |
|
|
115 | (4) |
|
5.2 Sample-Based Discriminants Revisited |
|
|
119 | (1) |
|
|
120 | (1) |
|
|
121 | (4) |
|
5.4.1 Resubstitution Error |
|
|
121 | (1) |
|
5.4.2 Leave-One-Out Error |
|
|
122 | (1) |
|
5.4.3 Cross-Validation Error |
|
|
122 | (2) |
|
|
124 | (1) |
|
|
125 | (11) |
|
|
125 | (3) |
|
5.5.2 Resubstitution Error |
|
|
128 | (2) |
|
5.5.3 Leave-One-Out Error |
|
|
130 | (2) |
|
5.5.4 Cross-Validation Error |
|
|
132 | (1) |
|
|
133 | (3) |
|
5.6 Higher-Order Moments of Error Rates |
|
|
136 | (4) |
|
|
136 | (1) |
|
5.6.2 Resubstitution Error |
|
|
137 | (2) |
|
5.6.3 Leave-One-Out Error |
|
|
139 | (1) |
|
5.7 Sampling Distribution of Error Rates |
|
|
140 | (5) |
|
5.7.1 Resubstitution Error |
|
|
140 | (1) |
|
5.7.2 Leave-One-Out Error |
|
|
141 | (1) |
|
|
142 | (3) |
|
6 Gaussian Distribution Theory: Univariate Case |
|
|
145 | (34) |
|
|
146 | (1) |
|
6.2 Univariate Discriminant |
|
|
147 | (1) |
|
|
148 | (6) |
|
|
148 | (3) |
|
6.3.2 Resubstitution Error |
|
|
151 | (1) |
|
6.3.3 Leave-One-Out Error |
|
|
152 | (1) |
|
|
152 | (2) |
|
6.4 Higher-Order Moments of Error Rates |
|
|
154 | (12) |
|
|
154 | (3) |
|
6.4.2 Resubstitution Error |
|
|
157 | (3) |
|
6.4.3 Leave-One-Out Error |
|
|
160 | (5) |
|
|
165 | (1) |
|
6.5 Sampling Distributions of Error Rates |
|
|
166 | (13) |
|
6.5.1 Marginal Distribution of Resubstitution Error |
|
|
166 | (3) |
|
6.5.2 Marginal Distribution of Leave-One-Out Error |
|
|
169 | (5) |
|
6.5.3 Joint Distribution of Estimated and True Errors |
|
|
174 | (2) |
|
|
176 | (3) |
|
7 Gaussian Distribution Theory: Multivariate Case |
|
|
179 | (42) |
|
7.1 Multivariate Discriminants |
|
|
179 | (1) |
|
|
180 | (19) |
|
7.2.1 Statistical Representations |
|
|
181 | (13) |
|
7.2.2 Computational Methods |
|
|
194 | (5) |
|
|
199 | (22) |
|
7.3.1 Expected Error Rates |
|
|
200 | (7) |
|
7.3.2 Second-Order Moments of Error Rates |
|
|
207 | (11) |
|
|
218 | (3) |
|
8 Bayesian MMSE Error Estimation |
|
|
221 | (38) |
|
8.1 The Bayesian MMSE Error Estimator |
|
|
222 | (4) |
|
8.2 Sample-Conditioned MSE |
|
|
226 | (1) |
|
8.3 Discrete Classification |
|
|
227 | (11) |
|
8.4 Linear Classification of Gaussian Distributions |
|
|
238 | (8) |
|
|
246 | (7) |
|
|
253 | (2) |
|
|
255 | (2) |
|
|
257 | (2) |
|
A Basic Probability Review |
|
|
259 | (18) |
|
A.1 Sample Spaces and Events |
|
|
259 | (1) |
|
A.2 Definition of Probability |
|
|
260 | (1) |
|
A.3 Borel-Cantelli Lemmas |
|
|
261 | (1) |
|
A.4 Conditional Probability |
|
|
262 | (1) |
|
|
263 | (2) |
|
A.6 Discrete Random Variables |
|
|
265 | (1) |
|
|
266 | (2) |
|
A.8 Conditional Expectation |
|
|
268 | (1) |
|
|
269 | (1) |
|
A.10 Vector Random Variables |
|
|
270 | (1) |
|
A.11 The Multivariate Gaussian |
|
|
271 | (2) |
|
A.12 Convergence of Random Sequences |
|
|
273 | (2) |
|
|
275 | (2) |
|
B Vapnik-Chervonenkis Theory |
|
|
277 | (8) |
|
|
277 | (1) |
|
|
278 | (1) |
|
B.3 VC Theory of Classification |
|
|
279 | (3) |
|
B.3.1 Linear Classification Rules |
|
|
279 | (1) |
|
B.3.2 kNN Classification Rule |
|
|
280 | (1) |
|
B.3.3 Classification Trees |
|
|
280 | (1) |
|
|
281 | (1) |
|
|
281 | (1) |
|
|
281 | (1) |
|
B.4 Vapnik-Chervonenkis Theorem |
|
|
282 | (3) |
|
|
285 | (6) |
Bibliography |
|
291 | (10) |
Author Index |
|
301 | (4) |
Subject Index |
|
305 | |