|
|
xiii | |
|
|
xv | |
Foreword |
|
xvii | |
Preface |
|
xix | |
Acknowledgements |
|
xxiii | |
|
Introduction and Measurement Contexts |
|
|
1 | (16) |
|
|
1 | (1) |
|
|
2 | (1) |
|
|
3 | (1) |
|
Importance of Falsifiable Research Questions or Hypotheses |
|
|
4 | (1) |
|
Behavior as ``Behavior'' Versus Behavior as a Sign or Indicant of a Construct |
|
|
4 | (1) |
|
Two Interpretations of Operationalism |
|
|
5 | (2) |
|
Distinction Between Context-Dependent Behavior and Generalized Tendencies to Behave |
|
|
7 | (1) |
|
Rationale for Identifying How We Are Conceptualizing Our Object of Measurement |
|
|
8 | (4) |
|
Influential Variables of a Measurement Context |
|
|
9 | (1) |
|
|
9 | (1) |
|
|
10 | (1) |
|
|
10 | (1) |
|
Tension Between Structuredness and Ecological Validity |
|
|
11 | (1) |
|
Recommendations for Measuring Generalized Characteristics From Observations |
|
|
12 | (1) |
|
Potential Disadvantages of Systematic Observational Count Measurement |
|
|
13 | (1) |
|
|
14 | (1) |
|
|
15 | (2) |
|
Improving Measurement of Generalized Characteristics Through Direct Observation and Generalizability Theory |
|
|
17 | (18) |
|
|
17 | (1) |
|
Two Concepts of Measurement |
|
|
17 | (2) |
|
Generalizability Theory as a Measurement Theory for Vaganotic Measures |
|
|
19 | (1) |
|
Example: Generalizability (G) Study With Multiple Sessions as a Single Facet |
|
|
20 | (3) |
|
Consequences of a Low G Coefficient |
|
|
23 | (1) |
|
|
24 | (1) |
|
McWilliam and Ware as an Example of a Two-Faceted Decision Study |
|
|
25 | (1) |
|
Practice Using a G Calculator on Data From a Two-Faceted G and D Study |
|
|
26 | (4) |
|
Accuracy of D Study Projections |
|
|
30 | (1) |
|
Implications of the Lessons of G and D Studies for Single-Subject Research |
|
|
31 | (1) |
|
|
32 | (1) |
|
|
33 | (1) |
|
|
33 | (2) |
|
Designing or Adapting Coding Manuals |
|
|
35 | (18) |
|
|
35 | (1) |
|
Selecting, Adapting, or Creating a Coding Manual |
|
|
36 | (12) |
|
Definition of a Coding Manual |
|
|
36 | (1) |
|
Relation of the Coding Manual to the Research Questions and Predictions |
|
|
36 | (1) |
|
Recommended Steps for Modifying or Designing Coding Manuals |
|
|
37 | (1) |
|
Conceptually Defining the Context-Dependent Behavior or the Generalized Characteristic |
|
|
37 | (1) |
|
Deciding the Level of Detail at Which the Behaviors Should Be Distinguished |
|
|
38 | (1) |
|
Physically Based Definitions, Socially Based Definitions, or Both? |
|
|
39 | (1) |
|
Defining the Lowest Level Categories |
|
|
40 | (2) |
|
Sources of Conceptual and Operational Definitions |
|
|
42 | (4) |
|
Defining Segmenting Rules |
|
|
46 | (1) |
|
Defining When to Start and Stop Coding |
|
|
47 | (1) |
|
The Potential Value of Flowcharts |
|
|
48 | (1) |
|
Do Coding Manuals Need to Be Sufficiently Short to Be Included in Methods Sections? |
|
|
49 | (1) |
|
|
49 | (2) |
|
|
51 | (2) |
|
|
53 | (20) |
|
|
53 | (1) |
|
The Elements of a Measurement System |
|
|
53 | (1) |
|
|
54 | (5) |
|
Continuous Behavior Sampling |
|
|
54 | (1) |
|
Intermittent Behavior Sampling |
|
|
55 | (1) |
|
|
56 | (2) |
|
How Does Interval Sampling Estimate Number and Duration? |
|
|
58 | (1) |
|
|
59 | (1) |
|
|
59 | (1) |
|
|
60 | (1) |
|
|
60 | (1) |
|
|
60 | (2) |
|
Live Coding Versus Recording the Observation for Later Coding |
|
|
62 | (2) |
|
Recording Coding Decisions |
|
|
64 | (2) |
|
Practice Recording Session |
|
|
66 | (3) |
|
|
69 | (1) |
|
|
70 | (3) |
|
Common Metrics of Observational Variables |
|
|
73 | (20) |
|
|
73 | (1) |
|
|
74 | (1) |
|
Quantifiable Dimensions of Behavior |
|
|
74 | (1) |
|
|
75 | (1) |
|
Proportion Metrics Change the Meaning of Observational Variables |
|
|
75 | (10) |
|
|
77 | (1) |
|
An Implicit Assumption of Proportion Metrics |
|
|
78 | (1) |
|
Testing Whether the Data Fit the Assumption of Proportion Metrics |
|
|
79 | (1) |
|
Consequences of Using a Proportion When the Data Do Not Fit the Assumption |
|
|
80 | (5) |
|
Alternative Methods to Control Nuisance Variables |
|
|
85 | (1) |
|
|
85 | (1) |
|
|
85 | (1) |
|
Transforming Metrics of Observational Variables in Group Statistical Analyses |
|
|
86 | (2) |
|
Scales of Measurement for Observational Variables |
|
|
88 | (2) |
|
Observational Variables in Parametric Analyses |
|
|
90 | (1) |
|
|
90 | (1) |
|
|
91 | (2) |
|
Introduction to Sequential Analysis |
|
|
93 | (26) |
|
|
93 | (1) |
|
Definitions of Terms Used in This Chapter |
|
|
94 | (1) |
|
Sequential Versus Nonsequential Variables |
|
|
94 | (1) |
|
Sequential Associations Are Not Sufficient Evidence for Causal Inferences |
|
|
95 | (1) |
|
Coded Units and Exhaustiveness |
|
|
96 | (2) |
|
Three Major Types of Sequential Analysis |
|
|
98 | (3) |
|
Event-Lag Sequential Analysis |
|
|
98 | (1) |
|
Time-Lag Sequential Analysis |
|
|
99 | (1) |
|
Time-Window Sequential Analysis |
|
|
100 | (1) |
|
The Need to ``Control for Chance'' |
|
|
101 | (1) |
|
How Sequential Data Are Represented Prior to Contingency Table Organization |
|
|
102 | (1) |
|
|
103 | (8) |
|
Proper 2 x 2 Contingency Table Construction of Two Streams of Data for Concurrent Analysis |
|
|
105 | (1) |
|
Proper 2 x 2 Contingency Table Construction From One Stream of Data for Event-Lag Sequential Analysis |
|
|
105 | (3) |
|
Simulation Study to Compare Results From Two Ways to Construct Contingency Tables |
|
|
108 | (1) |
|
Contingency Tables for Time-Window Lag Sequential Analysis |
|
|
109 | (2) |
|
|
111 | (5) |
|
Transitional Probabilities in Backward Sequential Analysis |
|
|
113 | (2) |
|
Summary of Transitional Probabilities |
|
|
115 | (1) |
|
|
116 | (1) |
|
|
116 | (3) |
|
Analyzing Research Questions Involving Sequential Associations |
|
|
119 | (22) |
|
|
119 | (1) |
|
Computer Software to Aid Sequential Analysis |
|
|
120 | (1) |
|
Practice Exercise Using MOOSES Software to Conduct Time-Window Analysis |
|
|
120 | (5) |
|
|
125 | (1) |
|
What Is ``Enough Data'' and How Do We Attain It? |
|
|
126 | (5) |
|
Proposed Solutions for Insufficient Data |
|
|
129 | (2) |
|
Sequential Association Indices as Dependent Variables in Group Designs |
|
|
131 | (2) |
|
Testing the Significance of a Mean Sequential Association |
|
|
131 | (1) |
|
Testing the Between-Group Difference in Mean Sequential Associations |
|
|
132 | (1) |
|
Testing the Within-Subject Difference in Sequential Associations |
|
|
132 | (1) |
|
Testing the Significance of the Summary-Level Association Between a Participant Characteristic and a Sequential Association Between Behaviors |
|
|
133 | (1) |
|
Statistical Significance Testing of Sequential Associations in Single Cases |
|
|
133 | (3) |
|
A Caveat Regarding the Use of Yule's Q |
|
|
136 | (1) |
|
|
137 | (1) |
|
|
138 | (3) |
|
Observer Training, Observer Drift Checks, and Discrepancy Discussions |
|
|
141 | (18) |
|
|
141 | (1) |
|
Three Purposes of Point-by-Point Agreement on Coding Decisions |
|
|
141 | (1) |
|
Two Definitions of Agreement |
|
|
142 | (3) |
|
|
145 | (3) |
|
|
148 | (1) |
|
Criterion Coding Standards |
|
|
149 | (2) |
|
|
151 | (2) |
|
Method of Selecting and Conducting Agreement Checks |
|
|
153 | (2) |
|
Retraining When Observer Drift Is Identified |
|
|
155 | (1) |
|
|
156 | (1) |
|
|
156 | (3) |
|
Interobserver Agreement and Reliability of Observational Variables |
|
|
159 | (24) |
|
|
159 | (1) |
|
Additional Purposes of Point-by-Point Agreement |
|
|
159 | (1) |
|
Added Principles When Agreement Checks Are Used to Estimate Interobserver ``Reliability'' of Observational Variable Scores |
|
|
160 | (4) |
|
Exhaustive Coding Spaces Revisited |
|
|
164 | (3) |
|
The Effect of Chance on Agreement |
|
|
167 | (1) |
|
Common Indices of Point-by-Point Agreement |
|
|
168 | (6) |
|
Occurrence Percentage Agreement |
|
|
168 | (1) |
|
Nonoccurrence Percentage Agreement |
|
|
168 | (1) |
|
Total Percentage Agreement |
|
|
169 | (1) |
|
|
169 | (2) |
|
Base Rate and Chance Agreement Revisited |
|
|
171 | (1) |
|
Summary of Point-by-Point Agreement Indices |
|
|
172 | (2) |
|
Intraclass Correlation Coefficient as an Index of Interobserver Reliability from the Vaganotic Concept of Measurement |
|
|
174 | (4) |
|
Options for Running ICC With SPSS |
|
|
175 | (1) |
|
Between-Participant Variance on the Variable of Interest Affects ICC |
|
|
175 | (2) |
|
Using ICC as a Measure of Interobserver Reliability for Predictors and Dependent Variables in Group Designs |
|
|
177 | (1) |
|
The Interpretation of SPSS Output for ICC |
|
|
177 | (1) |
|
The Conceptual Relation Between Interobserver Agreement and ICC |
|
|
178 | (1) |
|
Consequences of Low or Unknown Interobserver Reliability |
|
|
178 | (2) |
|
|
180 | (1) |
|
|
181 | (2) |
|
Validation of Observational Variables |
|
|
183 | (22) |
|
|
183 | (1) |
|
The Changing Concept of Validation |
|
|
184 | (1) |
|
Understanding Which Types of Validation Evidence Are Most Relevant for Different Research Designs, Objects of Measurement, and Research Purposes |
|
|
185 | (1) |
|
|
186 | (2) |
|
Definition of Content Validation |
|
|
186 | (1) |
|
Different Traditions Vary on the Levels of Importance Placed on Content Validation |
|
|
187 | (1) |
|
Weaknesses of Content Validation |
|
|
188 | (1) |
|
|
188 | (2) |
|
Definition of Sensitivity to Change |
|
|
188 | (1) |
|
Influences on Sensitivity to Change |
|
|
189 | (1) |
|
Weakness of Sensitivity to Change |
|
|
190 | (1) |
|
|
190 | (3) |
|
Definition of Treatment Utility |
|
|
190 | (2) |
|
Weaknesses of Treatment Utility |
|
|
192 | (1) |
|
Criterion-Related Validation |
|
|
193 | (1) |
|
Definition of Criterion-Related Validation |
|
|
193 | (1) |
|
Primary Appeal of Criterion-Related Validation |
|
|
193 | (1) |
|
Weaknesses of Criterion-Related Validation |
|
|
194 | (1) |
|
|
194 | (6) |
|
Definition of Construct Validation |
|
|
194 | (1) |
|
Discriminative Validation |
|
|
195 | (1) |
|
|
196 | (1) |
|
Multitrait, Multimethod Validation |
|
|
197 | (3) |
|
An Implicit ``Weakness'' of Science? |
|
|
200 | (2) |
|
|
202 | (1) |
|
|
202 | (3) |
Glossary |
|
205 | (16) |
Index |
|
221 | |