"There has been an increase in awareness (and perhaps occurrence) of individual and organized cheating on tests. Recent reports of widespread problems with state student accountability tests and teacher certification testing have raised questions about the very validity of assessment programs. While there are several books that specifically detail the issues of test security cheating on assessments, few outline the statistical procedures used for detecting various types of potential test fraud and the associated research findings. Without a significant research literature base, the new generation of researchers will have little opportunity or incentive to improve on existing methods.Enlisting a variety of experts and scholars in different fields of testing, this edited volume expands on the current literature base by including examples of detailed research findings arrived at by statistical methodology. It also provides a synthesis of the current state of the art with regard to the statistical detection of testing infidelity, particularly for large-scale assessments. By presenting methods currently used by testing organizations and research on new methods, the volume offers an important forum for expanding the literature in this area"--
There has been an increase in awareness (and perhaps occurrence) of individual and organized cheating on tests. Recent reports of widespread problems with state student accountability tests and teacher certification testing have raised questions about the very validity of assessment programs. While there are several books that specifically detail the issues of test security cheating on assessments, few outline the statistical procedures used for detecting various types of potential test fraud and the associated research findings. Without a significant research literature base, the new generation of researchers will have little opportunity or incentive to improve on existing methods.
Enlisting a variety of experts and scholars in different fields of testing, this edited volume expands on the current literature base by including examples of detailed research findings arrived at by statistical methodology. It also provides a synthesis of the current state of the art with regard to the statistical detection of testing infidelity, particularly for large-scale assessments. By presenting methods currently used by testing organizations and research on new methods, the volume offers an important forum for expanding the literature in this area.
|
|
xiii | |
|
|
xv | |
|
|
1 | (3) |
|
|
|
2 A Brief History of Research on Test Fraud Detection and Prevention |
|
|
4 | (4) |
|
|
|
3 Cheating: Some Ways to Detect it Badly |
|
|
8 | (15) |
|
|
PART I Similarities in Response Patterns |
|
|
|
4 Relationships of Examinee Pair Characteristics and Item Response Similarity |
|
|
23 | (15) |
|
|
5 A Parametric Approach to Detect a Disproportionate Number of Identical Item Responses on a Test |
|
|
38 | (15) |
|
|
|
|
6 Detection of Non-independent Test Taking by Similarity Analysis |
|
|
53 | (30) |
|
|
PART II Macro Level Cheating |
|
|
|
7 Local Outlier Detection in Data Forensics: Data Mining Approach to Flag Unusual Schools |
|
|
83 | (18) |
|
|
8 Macro Level Systems of Statistical Evidence Indicative of Cheating |
|
|
101 | (20) |
|
|
|
|
|
9 A Bayesian Hierarchical Linear Modeling Approach for Detecting Cheating and Aberrance |
|
|
121 | (16) |
|
|
|
PART III Answer Changing Behavior |
|
|
|
10 Patterns of Examinee Erasure Behavior for a Large-Scale Assessment |
|
|
137 | (12) |
|
|
|
|
|
11 AYP Consequences and Erasure Behavior |
|
|
149 | (9) |
|
|
12 An Exploration of Answer Changing Behavior on a Computer-Based High-Stakes Achievement Test |
|
|
158 | (17) |
|
|
|
PART IV Detection of Aberrant Responses |
|
|
|
13 Identifying Non-Effortful Student Behavior on Adaptive Tests: Implications for Test Fraud Detection |
|
|
175 | (11) |
|
|
|
|
14 A Method for Measuring Performance Inconsistency by Using Score Differences |
|
|
186 | (17) |
|
|
PART V Multiple Methods of Detection |
|
|
|
15 Data Forensics: A Compare-and-Contrast Analysis of Multiple Methods |
|
|
203 | (17) |
|
|
|
16 Using Multiple Methods to Detect Aberrant Data |
|
|
220 | (10) |
|
|
|
17 Test Security for Multistage Tests: A Quality Control Perspective |
|
|
230 | (9) |
|
|
|
Appendix A |
|
239 | (4) |
Appendix B |
|
243 | (4) |
Appendix C |
|
247 | (2) |
References |
|
249 | (10) |
Contributors |
|
259 | (6) |
Index |
|
265 | |
Neal Kingston is the Director of the Achievement and Assessment Institute, Co-Director of the Center for Educational Testing and Evaluation, and Professor of Educational Psychology at the University of Kansas. He has managed all aspects of the educational testing process for both general and alternate assessments, including as an Executive Director at Educational Testing Service, Associate Commissioner for Curriculum and Assessment at the Kentucky Department of Education, Senior Vice President at Measured Progress, Vice President and General Manager at CTB/McGraw-Hill, and Director of CETE at the University of Kansas. He has published and presented more than 100 articles, papers, and book chapters on assessment.
Amy Clark is a Research Associate in psychometrics at the Center for Educational Testing and Evaluation at the University of Kansas. She began her career as a classroom teacher and received both her M.S and Ph.D. in Educational Psychology from the University of Kansas specializing in Research, Evaluation, Measurement, and Statistics. Her research interests include exploring potential threats to validity, accountability issues, and diagnostic assessment.