Preface |
|
iii | |
Figures |
|
xi | |
Tables |
|
xiii | |
Summary |
|
xv | |
Acknowledgments |
|
xxv | |
Abbreviations |
|
xxvii | |
CHAPTER ONE Introduction |
|
1 | |
|
|
1 | |
|
|
1 | |
|
Phase I and Phase II Comparison |
|
|
3 | |
|
Significance of the Program |
|
|
4 | |
|
|
5 | |
|
|
6 | |
|
|
7 | |
CHAPTER TWO Implications of Phase I Program Evaluations for Phase II Evaluations |
|
9 | |
|
Previous Performance Evaluations of BK21 Phase I |
|
|
9 | |
|
Annual and Interim Evaluations |
|
|
9 | |
|
Program-Level Evaluations |
|
|
11 | |
|
Findings from Previous Evaluations |
|
|
14 | |
|
Changes Resulting from BK21 Phase I |
|
|
14 | |
|
Suggestions for Improving the Evaluation System |
|
|
16 | |
|
Limitations of Previous Evaluations |
|
|
17 | |
|
Implications for Designing the Phase II Evaluation Model |
|
|
19 | |
CHAPTER THREE |
|
|
A Logic Model for Evaluating BK21 |
|
|
21 | |
|
Literature Review and Focus Group Interviews |
|
|
21 | |
|
Developing the Logic Model |
|
|
23 | |
|
Identifying the Goals and Missions |
|
|
26 | |
|
|
27 | |
|
|
29 | |
|
|
31 | |
|
Goals and Missions Identified for the Logic Model |
|
|
33 | |
|
|
34 | |
|
|
34 | |
|
|
37 | |
|
|
40 | |
|
Recipient Selection Criteria |
|
|
41 | |
|
Performance Evaluation Criteria |
|
|
42 | |
|
Other Operational Mechanisms |
|
|
42 | |
|
|
43 | |
|
Incentives, Activities, Outputs, and Outcomes of the Program |
|
|
44 | |
|
|
44 | |
|
|
47 | |
|
|
48 | |
|
|
54 | |
|
|
55 | |
|
|
56 | |
CHAPTER FOUR Quantitative Model to Assess BK21 Program Outcomes |
|
59 | |
|
|
59 | |
|
Summary of Phase I Statistics |
|
|
61 | |
|
|
64 | |
|
Unadjusted Comparisons of BK21 Winners and Losers |
|
|
67 | |
|
Ordinary Least Squares, Matching, and Propensity Score |
|
|
69 | |
|
Fixed-Effects and Difference-in-Differences Models |
|
|
71 | |
|
|
74 | |
|
|
76 | |
|
|
77 | |
|
|
80 | |
|
The Power to Identify Impacts |
|
|
81 | |
|
Data for Use in Validating the Model |
|
|
81 | |
|
|
81 | |
|
Ministry of Education Data |
|
|
83 | |
|
|
83 | |
|
|
84 | |
|
Another Possible Comparison Group: Nonapplicants |
|
|
88 | |
|
|
90 | |
|
Variations on the Specification |
|
|
96 | |
|
|
97 | |
|
Social Science and Humanities |
|
|
100 | |
|
Other Aspects of the Logic Model |
|
|
100 | |
|
Simultaneous Equations Methods |
|
|
102 | |
|
International Competitiveness |
|
|
104 | |
|
Simulation: Ability to Precisely Estimate Effects at Phase II Funding Levels |
|
|
105 | |
|
Simulation: Outside Funding |
|
|
107 | |
|
Implications for Data Collection |
|
|
109 | |
|
|
110 | |
CHAPTER FIVE Metrics and Measurement for Program Evaluation |
|
111 | |
|
General Issues of Measurement |
|
|
112 | |
|
|
113 | |
|
|
114 | |
|
Direct and Indirect Measures |
|
|
114 | |
|
|
115 | |
|
Strategy for Framing Evaluation Measures |
|
|
116 | |
|
Matching Measures to Goals |
|
|
117 | |
|
Selecting Among Candidate Measures |
|
|
118 | |
|
Adding to Korea's Research Manpower Pool |
|
|
132 | |
|
Highly Educated Workforce Employment |
|
|
134 | |
|
High-Quality Globally Competitive Research |
|
|
135 | |
|
Measuring Quality of Research Output |
|
|
135 | |
|
Consistent Measures of Quality Within Fields |
|
|
138 | |
|
Measures of Quality Across All Fields |
|
|
144 | |
|
Relative Research Quality Measures |
|
|
146 | |
|
Changes to University/Department Infrastructure |
|
|
146 | |
|
Fostering a Competitive Environment Within Korea |
|
|
147 | |
|
Other Measures of Prestigious and Competitive Graduate Research |
|
|
150 | |
|
Measuring Enhanced University-Industry Collaboration |
|
|
154 | |
|
Measuring Enhanced Regional University Standing and Industry Linkage |
|
|
156 | |
|
Industrial Research Quantity, Quality, and National Labor Productivity |
|
|
159 | |
CHAPTER SIX Database Design to Support BK21 Program Evaluation and Measurement |
|
161 | |
|
Priority Among Proposed Measures |
|
|
161 | |
|
Database Design and Content |
|
|
166 | |
|
Format of Database Design Entries |
|
|
173 | |
CHAPTER SEVEN Future Development of BK21 and University Research in Korea |
|
178 | |
|
A Good Model for an Allocation System |
|
|
178 | |
|
Applying the Model to Korea |
|
|
179 | |
|
|
179 | |
|
Portable Student Fellowships |
|
|
181 | |
|
Facilities and Infrastructure Funding |
|
|
182 | |
|
Promotion of Specific Regions or Fields |
|
|
183 | |
|
|
183 | |
|
Requirements for University and Department Governance |
|
|
183 | |
|
Changing the Korean System |
|
|
184 | |
|
Appropriate Scale and Number of Research Universities |
|
|
185 | |
|
Analogy to the U.S. System |
|
|
185 | |
|
Applying the Proposed Measurement Tools |
|
|
189 | |
|
|
191 | |
CHAPTER EIGHT Conclusions |
|
195 | |
|
Directions for a Phase II Evaluation System |
|
|
195 | |
|
Phase II Program Goals and Characteristics Identified by the Logic Model |
|
|
196 | |
|
Quantitative Model and Data Requirements |
|
|
196 | |
|
Metrics, Measures, and Database to Assess Program Effects and to Monitor Progress |
|
|
197 | |
|
Assessing the Quality of Program Outputs and Outcomes |
|
|
199 | |
|
|
199 | |
|
Quality of the Highly Educated Workforce |
|
|
200 | |
|
Concentration Strategy and Competition Rule |
|
|
201 | |
|
Suggestions for Long-Term Changes in the Korean Scientific Research and Education System |
|
|
202 | |
|
Balancing Competition and Stability in Funding |
|
|
202 | |
|
Portable Student Fellowships |
|
|
203 | |
|
|
203 | |
|
Improving University and Department Governance |
|
|
204 | |
|
Preconditions for the Proposed Changes |
|
|
204 | |
|
Enhancing Program Evaluation and Management Capabilities |
|
|
205 | |
APPENDIX |
|
|
A. Performance Evaluation Metrics Used in Previous Evaluations of Phase I |
|
|
207 | |
|
B. Interview Guides for Representatives of Participants, Policymakers, and Users Related to the University Research and Education System |
|
|
209 | |
|
C. Selection Criteria for Applied Science Research Groups: Phase II |
|
|
217 | |
|
D. Table with Lagged Outcomes |
|
|
219 | |
|
|
221 | |
|
|
229 | |
|
|
237 | |
|
|
243 | |
Bibliography |
|
245 | |