List of Figures |
|
xxii | |
List of Tables |
|
xxvii | |
Preface |
|
xxix | |
Acknowledgments |
|
xxxiv | |
About the Author |
|
xxxvi | |
Part I: Introduction |
|
1 | (36) |
|
Chapter 1 Introduction to Making Sense of Numbers |
|
|
2 | (11) |
|
|
2 | (1) |
|
Common Tasks Involving Numbers |
|
|
3 | (1) |
|
Plausibility of Numeric Values |
|
|
4 | (1) |
|
Challenges in Making Sense of Numbers |
|
|
5 | (4) |
|
|
8 | (1) |
|
How We Learn to Make Sense of Numbers |
|
|
9 | (4) |
|
Chapter 2 Foundational Concepts for Quantitative Research |
|
|
13 | (24) |
|
Terminology for Quantitative Research |
|
|
13 | (4) |
|
|
17 | (4) |
|
|
19 | (1) |
|
|
19 | (2) |
|
Goals Of Quantitative Research |
|
|
21 | (4) |
|
|
21 | (1) |
|
|
22 | (1) |
|
|
22 | (1) |
|
|
23 | (2) |
|
|
25 | (1) |
|
Report and Interpret Numbers |
|
|
26 | (2) |
|
Specify Direction and Magnitude |
|
|
28 | (10) |
|
|
28 | (3) |
|
Magnitude of an Association |
|
|
31 | (6) |
Part II: How Topic, Measurement, And Context Help Make Sense Of Numbers |
|
37 | (76) |
|
Chapter 3 Topic and Conceptualization |
|
|
38 | (21) |
|
|
38 | (2) |
|
|
40 | (11) |
|
How Change or Variation in a Definition Affects the Scope |
|
|
45 | (5) |
|
Anecdotes Are Not Definitions |
|
|
50 | (1) |
|
How Topic and Scope Affect Plausibility |
|
|
51 | (3) |
|
Concepts With Limits on Their Possible Values |
|
|
52 | (1) |
|
Positive and Negative Values |
|
|
53 | (1) |
|
Other Limits on Numeric Values |
|
|
53 | (1) |
|
How Topic and Perspective Affect Optimal Values |
|
|
54 | (5) |
|
How Topic Affects Optimal Value |
|
|
54 | (1) |
|
How Perspective Affects Optimal Value |
|
|
55 | (4) |
|
|
59 | (33) |
|
|
59 | (3) |
|
Factors Affecting Operationalization |
|
|
62 | (3) |
|
Expected Audience and Use |
|
|
62 | (1) |
|
|
63 | (1) |
|
|
64 | (1) |
|
Measurement Issues for Secondary Data and Non-research Data Sources |
|
|
65 | (1) |
|
|
65 | (4) |
|
|
66 | (2) |
|
|
68 | (1) |
|
|
69 | (4) |
|
|
70 | (1) |
|
|
70 | (2) |
|
|
72 | (1) |
|
Data Collection and Level of Measurement |
|
|
73 | (8) |
|
How Question Format Affects Measurement |
|
|
75 | (6) |
|
|
75 | (1) |
|
|
75 | (2) |
|
Single-Response and Multiple-Response Questions |
|
|
77 | (1) |
|
Mutually Exclusive and Exhaustive Categories |
|
|
78 | (1) |
|
|
79 | (1) |
|
|
80 | (1) |
|
How Measurement Affects Plausibility |
|
|
81 | (1) |
|
Level and Range of Possible Values |
|
|
81 | (1) |
|
|
82 | (1) |
|
Reliability and Validity of Numeric Measures |
|
|
82 | (10) |
|
|
82 | (2) |
|
|
84 | (9) |
|
|
85 | (1) |
|
|
86 | (1) |
|
|
86 | (1) |
|
|
87 | (5) |
|
|
92 | (21) |
|
|
92 | (1) |
|
How Context Affects Plausibility |
|
|
93 | (5) |
|
Context as Limiting the Set of Cases Studied |
|
|
96 | (1) |
|
Other Dimensions of Context |
|
|
97 | (1) |
|
How Context Affects Measurement |
|
|
98 | (1) |
|
Population Versus Study Sample |
|
|
98 | (2) |
|
|
100 | (1) |
|
|
101 | (4) |
|
|
103 | (2) |
|
Level of Analysis and Fallacy of Level |
|
|
105 | (8) |
Part III: Exhibits For Communicating Numeric Information |
|
113 | (82) |
|
Chapter 6 Working With Tables |
|
|
114 | (35) |
|
Criteria for Effective Tables |
|
|
114 | (1) |
|
|
114 | (1) |
|
|
115 | (1) |
|
|
115 | (10) |
|
|
116 | (3) |
|
|
118 | (1) |
|
|
118 | (1) |
|
Types of Measures or Statistics |
|
|
118 | (1) |
|
|
119 | (1) |
|
|
120 | (1) |
|
|
120 | (1) |
|
|
120 | (1) |
|
|
121 | (1) |
|
|
121 | (1) |
|
More Advanced Table Features |
|
|
122 | (3) |
|
|
122 | (3) |
|
Organizing Data in Tables and Charts |
|
|
125 | (12) |
|
|
126 | (1) |
|
Organizing Data in Tables to Accompany a Written Description |
|
|
127 | (3) |
|
|
127 | (2) |
|
|
129 | (1) |
|
Organizing Variables by Their Role in an Analysis |
|
|
130 | (1) |
|
Organizing Data in Tables Intended for Data Lookup |
|
|
130 | (5) |
|
|
130 | (1) |
|
Order of Items as Collected From Source |
|
|
130 | (4) |
|
Multiple Criteria for Organizing Tables |
|
|
134 | (1) |
|
Organizing Data in Three-Way Tables |
|
|
135 | (2) |
|
|
137 | (5) |
|
|
137 | (13) |
|
Direction of Percentage Calculations |
|
|
138 | (2) |
|
Percentages for Two-Category Variables |
|
|
140 | (2) |
|
Considerations for Creating Tables |
|
|
142 | (7) |
|
Portrait Versus Landscape Layout |
|
|
142 | (1) |
|
|
143 | (1) |
|
Digits and Decimal Places |
|
|
143 | (6) |
|
Chapter 7 Working With Charts and Visualizations |
|
|
149 | (46) |
|
Criteria for Effective Charts and Visualizations |
|
|
150 | (2) |
|
|
150 | (1) |
|
|
151 | (1) |
|
|
151 | (1) |
|
Visual Perception Principles |
|
|
152 | (4) |
|
|
152 | (1) |
|
|
152 | (4) |
|
Anatomy of a Chart or Visualization |
|
|
156 | (5) |
|
|
156 | (2) |
|
Axis Titles and Axis Labels |
|
|
158 | (1) |
|
|
158 | (1) |
|
|
158 | (1) |
|
|
159 | (1) |
|
|
159 | (1) |
|
Reading Data From a Chart or Visualization |
|
|
159 | (2) |
|
Charts and Visualizations for Specific Tasks |
|
|
161 | (21) |
|
|
161 | (4) |
|
Text Visualization for One Large Number |
|
|
161 | (1) |
|
|
162 | (1) |
|
|
163 | (2) |
|
|
165 | (3) |
|
|
165 | (2) |
|
|
167 | (1) |
|
Comparing Two or More Numbers |
|
|
168 | (4) |
|
Categorical Independent Variables |
|
|
168 | (2) |
|
Continuous Independent Variables |
|
|
170 | (2) |
|
|
172 | (8) |
|
|
172 | (1) |
|
Multiple Stacked Bar Charts |
|
|
173 | (1) |
|
|
174 | (2) |
|
|
176 | (1) |
|
|
177 | (1) |
|
|
177 | (3) |
|
|
180 | (1) |
|
|
180 | (2) |
|
|
182 | (2) |
|
|
182 | (1) |
|
Linear and Logarithmic Axis Scales |
|
|
183 | (1) |
|
Common Errors in Chart Creation |
|
|
184 | (12) |
|
Incorrect Chart for Level of Measurement |
|
|
184 | (1) |
|
|
184 | (1) |
|
|
185 | (2) |
|
Incorrect Spacing of Values on an Axis |
|
|
187 | (2) |
|
|
189 | (7) |
|
Pie With Too Many Categories |
|
|
189 | (1) |
|
Use of 3-D and Angled or Tilted Perspective |
|
|
189 | (6) |
Part IV: Making Sense Of Numbers From Mathematical And Statistical Methods |
|
195 | (110) |
|
Chapter 8 Comparison Values, Contrast Sizes, and Standards |
|
|
196 | (33) |
|
Reference Groups and Comparison Values |
|
|
196 | (5) |
|
Criteria for Identifying Comparison Values |
|
|
197 | (4) |
|
Comparisons Across Groups |
|
|
197 | (3) |
|
|
200 | (1) |
|
Standards, Thresholds, and Target Values |
|
|
201 | (11) |
|
|
201 | (1) |
|
|
202 | (4) |
|
Benchmarks, Targets, and Objectives |
|
|
206 | (1) |
|
|
207 | (1) |
|
|
208 | (3) |
|
Criteria for Choosing External Comparison Values or Standards |
|
|
211 | (1) |
|
Contrast Sizes for Quantitative Variables |
|
|
212 | (4) |
|
|
212 | (2) |
|
When a One-Unit Contrast Is Too Big |
|
|
213 | (1) |
|
When a One-Unit Contrast Is Too Small |
|
|
214 | (1) |
|
Criteria for Identifying Contrast Sizes |
|
|
214 | (2) |
|
Theoretical Criteria for Contrast Size |
|
|
214 | (1) |
|
Empirical Criteria for Contrast Size |
|
|
215 | (1) |
|
Considerations for Comparability |
|
|
216 | (13) |
|
|
216 | (2) |
|
|
216 | (1) |
|
|
217 | (1) |
|
|
218 | (3) |
|
|
218 | (1) |
|
Comparison Across Location |
|
|
219 | (1) |
|
Comparison Across Units, Place, and Time |
|
|
219 | (2) |
|
Comparability of Standards |
|
|
221 | (8) |
|
Chapter 9 Numbers, Comparisons, and Calculations |
|
|
229 | (31) |
|
Numeric Measures of Level |
|
|
230 | (5) |
|
|
230 | (1) |
|
|
231 | (1) |
|
|
232 | (1) |
|
|
233 | (1) |
|
|
233 | (1) |
|
|
234 | (1) |
|
Plausibility Criteria for Measures of Level |
|
|
235 | (1) |
|
Measures of Position in a Ranked List |
|
|
235 | (5) |
|
|
235 | (5) |
|
|
240 | (1) |
|
Plausibility Criteria for Measures of Position |
|
|
240 | (2) |
|
Mathematical Calculations |
|
|
242 | (8) |
|
|
242 | (1) |
|
|
243 | (2) |
|
|
244 | (1) |
|
Percentage or Proportion of a Whole |
|
|
245 | (1) |
|
|
246 | (1) |
|
Relationship between Ratio and Percentage Difference |
|
|
247 | (1) |
|
|
247 | (3) |
|
Percentage of Versus Percentile Versus Percentage Difference or Change |
|
|
248 | (2) |
|
Plausibility Criteria for Results of Calculations |
|
|
250 | (1) |
|
How Level of Measurement Affects Valid Types of Comparison |
|
|
250 | (6) |
|
|
250 | (3) |
|
|
253 | (3) |
|
Choosing Types of Comparisons |
|
|
256 | (4) |
|
Influence of Topic and Field |
|
|
257 | (3) |
|
Chapter 10 Distributions and Associations |
|
|
260 | (45) |
|
Distributions of Single Variables |
|
|
261 | (13) |
|
Measures of Central Tendency |
|
|
261 | (2) |
|
|
261 | (1) |
|
|
262 | (1) |
|
|
262 | (1) |
|
|
263 | (6) |
|
|
263 | (2) |
|
|
265 | (1) |
|
|
265 | (1) |
|
|
266 | (2) |
|
|
268 | (1) |
|
|
269 | (2) |
|
Position in a Distribution: Standardized Score or Z-Score |
|
|
271 | (3) |
|
Plausibility Criteria for Univariate Statistics |
|
|
274 | (3) |
|
Tables and Charts for Presenting Distributions |
|
|
277 | (10) |
|
Portraying Distribution of Categorical Variables |
|
|
277 | (6) |
|
|
279 | (1) |
|
|
280 | (3) |
|
Portraying Distribution of Continuous Variables |
|
|
283 | (4) |
|
Associations Between Two or More Variables |
|
|
287 | (8) |
|
|
289 | (1) |
|
|
290 | (3) |
|
Difference in Means or ANOVA |
|
|
293 | (2) |
|
|
295 | (2) |
|
Two-Way Difference in Means or Two-Way ANOVA |
|
|
295 | (1) |
|
Three-Way Cross-Tabulations |
|
|
296 | (1) |
|
Plausibility Criteria for Bivariate and Three-Way Statistics |
|
|
297 | (2) |
|
Comparisons by Level of Measurement, Revisited |
|
|
299 | (6) |
Part V: Assessing The Quality Of Numeric Estimates |
|
305 | (120) |
|
|
306 | (41) |
|
|
307 | (1) |
|
Time Structure of Study Designs |
|
|
308 | (5) |
|
|
308 | (1) |
|
|
309 | (4) |
|
Repeated Cross-Sectional Studies |
|
|
309 | (1) |
|
|
310 | (1) |
|
|
311 | (2) |
|
|
313 | (12) |
|
Probability Sampling Methods |
|
|
314 | (6) |
|
|
315 | (1) |
|
Stratified Random Sampling |
|
|
316 | (1) |
|
Disproportionate Sampling |
|
|
317 | (1) |
|
|
318 | (1) |
|
Multistage Random Sampling |
|
|
319 | (1) |
|
Non-probability Sampling Methods |
|
|
320 | (4) |
|
|
321 | (1) |
|
|
321 | (2) |
|
|
323 | (1) |
|
|
324 | (1) |
|
|
325 | (3) |
|
|
325 | (2) |
|
Attrition From Longitudinal Studies |
|
|
327 | (1) |
|
|
328 | (5) |
|
|
329 | (1) |
|
Respondents Who Lack Knowledge |
|
|
330 | (1) |
|
Missing Values vs. Not Applicable |
|
|
331 | (2) |
|
|
333 | (4) |
|
|
333 | (1) |
|
|
334 | (1) |
|
|
335 | (1) |
|
|
336 | (1) |
|
Respondents Who Lack Knowledge |
|
|
336 | (1) |
|
|
337 | (10) |
|
Primary and Secondary Data |
|
|
337 | (2) |
|
Data Collected for Research Purposes |
|
|
339 | (2) |
|
|
339 | (1) |
|
|
339 | (1) |
|
Other Sources of Research Data |
|
|
340 | (1) |
|
Data Collected for Non-research Purposes |
|
|
341 | (7) |
|
|
341 | (1) |
|
|
341 | (6) |
|
|
347 | (40) |
|
|
348 | (2) |
|
Why Does Causality Matter? |
|
|
348 | (1) |
|
Association Does not Equal Causation |
|
|
349 | (1) |
|
Criteria for Assessing Causality |
|
|
350 | (11) |
|
|
350 | (2) |
|
|
352 | (3) |
|
Study Design and Time Order |
|
|
355 | (1) |
|
|
356 | (2) |
|
|
358 | (1) |
|
|
359 | (1) |
|
Summary of Assessing Causality |
|
|
360 | (1) |
|
|
361 | (9) |
|
Randomization Into Treatment and Control Groups |
|
|
362 | (3) |
|
|
365 | (2) |
|
|
367 | (2) |
|
Colloquial vs. Research Meaning of "Experiment" |
|
|
369 | (1) |
|
|
370 | (6) |
|
Variation in the Independent Variable |
|
|
371 | (1) |
|
|
372 | (2) |
|
|
374 | (1) |
|
Other Threats to Internal Validity |
|
|
375 | (1) |
|
Research Strategies for Assessing Confounding |
|
|
376 | (4) |
|
Randomize to Remove Confounders |
|
|
376 | (1) |
|
Measure and Take Into Account Potential Confounders |
|
|
377 | (3) |
|
Random Sampling vs. Random Assignment |
|
|
380 | (2) |
|
Implications of Causality for Quantitative Research |
|
|
382 | (5) |
|
Chapter 13 Uncertainty of Numeric Estimates |
|
|
387 | (38) |
|
What Is Statistical Uncertainty? |
|
|
388 | (1) |
|
|
389 | (3) |
|
|
392 | (7) |
|
|
392 | (1) |
|
Margin of Error and Confidence Level |
|
|
393 | (3) |
|
|
396 | (3) |
|
Criteria for Making Sense of Measures of Uncertainty |
|
|
399 | (1) |
|
|
399 | (3) |
|
Basics of Hypothesis Testing |
|
|
402 | (6) |
|
Step 1: Write a Hypothesis |
|
|
403 | (1) |
|
|
403 | (1) |
|
Step 3: Specify the Significance Level |
|
|
404 | (1) |
|
Step 4: Assess Statistical Significance Using the p-Value |
|
|
404 | (1) |
|
Use of Confidence Intervals for Hypothesis Testing |
|
|
405 | (2) |
|
Criteria for Making Sense of Hypothesis-Testing Results |
|
|
407 | (1) |
|
Drawbacks of Traditional Hypothesis Testing |
|
|
408 | (2) |
|
Interpreting Inferential Statistics for Bivariate and Three-Way Procedures |
|
|
410 | (16) |
|
Bivariate Statistical Results |
|
|
411 | (5) |
|
|
411 | (2) |
|
|
413 | (1) |
|
Difference in Means or ANOVA |
|
|
414 | (2) |
|
|
416 | (11) |
|
Three-Way Cross-Tabulations |
|
|
416 | (2) |
|
|
418 | (2) |
|
|
420 | (5) |
Part VI: Pulling It All Together |
|
425 | (78) |
|
Chapter 14 Communicating Quantitative Research |
|
|
426 | (43) |
|
Tools for Presenting Quantitative Research |
|
|
427 | (5) |
|
|
428 | (1) |
|
|
429 | (1) |
|
General Shape or Precise Values? |
|
|
429 | (2) |
|
|
431 | (1) |
|
|
431 | (1) |
|
Expository Writing Techniques |
|
|
432 | (5) |
|
Using Paragraphs to Organize Ideas |
|
|
432 | (1) |
|
|
433 | (1) |
|
|
434 | (1) |
|
Using Evidentiary Sentences |
|
|
434 | (1) |
|
Using Transition Sentences |
|
|
435 | (2) |
|
Writing About Numbers in Particular |
|
|
437 | (3) |
|
|
437 | (1) |
|
Reporting and Interpreting Numbers |
|
|
437 | (1) |
|
Expressing Direction and Magnitude |
|
|
437 | (1) |
|
Specifying the Comparison Value |
|
|
438 | (1) |
|
Using Vocabulary and Calculations to Express Shape and Size |
|
|
438 | (2) |
|
Conveying the Type of Measure or Calculation |
|
|
440 | (10) |
|
|
440 | (3) |
|
|
440 | (2) |
|
Percentage or Proportion of a Whole |
|
|
442 | (1) |
|
Ratio Between Two Concepts |
|
|
442 | (1) |
|
|
443 | (1) |
|
|
443 | (7) |
|
|
446 | (1) |
|
|
447 | (2) |
|
Percentage Difference or Change |
|
|
449 | (1) |
|
Writing About Distributions |
|
|
450 | (4) |
|
|
451 | (1) |
|
|
452 | (2) |
|
|
453 | (1) |
|
|
453 | (1) |
|
Writing About Associations |
|
|
454 | (4) |
|
Cross-Sectional Comparisons |
|
|
454 | (1) |
|
Nominal Independent Variables |
|
|
454 | (1) |
|
Ordinal Independent Variables |
|
|
455 | (1) |
|
|
455 | (2) |
|
Comparison Against a Benchmark |
|
|
457 | (1) |
|
Writing About Complex Patterns |
|
|
458 | (5) |
|
|
459 | (2) |
|
|
461 | (1) |
|
|
461 | (2) |
|
Content and Structure of Research Formats |
|
|
463 | (6) |
|
|
463 | (1) |
|
|
463 | (1) |
|
|
464 | (5) |
|
Chapter 15 The Role of Research Methods in Making Sense of Numbers |
|
|
469 | (34) |
|
|
470 | (1) |
|
|
470 | (7) |
|
What Is Practical Importance? |
|
|
470 | (1) |
|
Practical Significance vs. Statistical Significance |
|
|
471 | (9) |
|
What Questions Can Statistical Significance Answer? |
|
|
471 | (1) |
|
What Questions Can't Statistical Significance Answer? |
|
|
472 | (5) |
|
Importance of a Numeric Finding: The Big Picture |
|
|
477 | (3) |
|
How Study Design, Measurement, and Sample Size Affect "Importance" |
|
|
480 | (11) |
|
|
480 | (4) |
|
|
484 | (1) |
|
|
485 | (2) |
|
|
487 | (1) |
|
Relationships Among Dimensions of "Importance" |
|
|
488 | (2) |
|
|
490 | (1) |
|
Making Sense of Numbers in Quantitative Research Tasks |
|
|
491 | (13) |
|
Reading About and Assessing Others' Use of Numeric Information |
|
|
496 | (1) |
|
|
497 | (2) |
|
Conducting Mathematical and Elementary Statistical Analysis |
|
|
499 | (1) |
|
Communicating Quantitative Information |
|
|
500 | (3) |
Appendixes |
|
503 | (41) |
|
Appendix A: Why and How to Create New Variables |
|
|
504 | (18) |
|
Why New Variables Might Be Needed |
|
|
504 | (3) |
|
Transformations of Numbers |
|
|
507 | (4) |
|
|
507 | (1) |
|
|
508 | (1) |
|
|
509 | (1) |
|
Logarithmic Transformations |
|
|
510 | (1) |
|
|
511 | (3) |
|
|
514 | (1) |
|
|
514 | (1) |
|
Other Types of Composite Measures |
|
|
515 | (1) |
|
New Categorical Variables |
|
|
515 | (7) |
|
Categorical Variable From a Continuous Variable |
|
|
515 | (1) |
|
Simplified Categorical Variable From a Detailed Categorical Variable |
|
|
515 | (1) |
|
New Categorical Variable From Two Categorical Variables |
|
|
516 | (1) |
|
|
517 | (1) |
|
Indicator Variables From a Multiple-Response Item |
|
|
517 | (5) |
|
Appendix B: Sampling Weights |
|
|
522 | (5) |
|
The Purpose of Sampling Weights |
|
|
522 | (1) |
|
Sampling Weights for Disproportionate Sampling |
|
|
523 | (2) |
|
Communicating Use of Sampling Weights |
|
|
525 | (2) |
|
Appendix C: Brief Technical Background on Inferential Statistics |
|
|
527 | (17) |
|
Standard Error and Sample Size |
|
|
528 | (1) |
|
|
528 | (1) |
|
|
529 | (1) |
|
Criteria for Making Sense of Measures of Uncertainty |
|
|
529 | (2) |
|
|
531 | (7) |
|
Step 1: Specify Null and Alternative Hypotheses |
|
|
531 | (2) |
|
Important Aside About the "Null" Value |
|
|
533 | (1) |
|
Step 2: Calculate a Test Statistic |
|
|
533 | (1) |
|
|
534 | (1) |
|
Step 4: Assess Statistical Significance Using the p-Value |
|
|
535 | (1) |
|
Step 5: Interpret the Inferential Test Result |
|
|
536 | (2) |
|
Errors in Hypothesis Testing |
|
|
538 | (2) |
|
Plausibility Criteria for Inferential Test Statistics |
|
|
540 | (4) |
References |
|
544 | (10) |
Index |
|
554 | |