|
|
1 | (8) |
|
1.1 Evidence-Based Personalized Medicine for Chronic Diseases |
|
|
1 | (1) |
|
1.2 Personalized Medicine and Medical Decision Making |
|
|
2 | (5) |
|
1.2.2 Single-stage Decision Problems in Personalized Medicine |
|
|
3 | (1) |
|
1.2.2 Multi-stage Decisions and Dynamic Treatment Regimes |
|
|
4 | (3) |
|
|
7 | (2) |
|
2 The Data: Observational Studies and Sequentially Randomized Trials |
|
|
9 | (22) |
|
2.1 Longitudinal Observational Studies |
|
|
9 | (6) |
|
2.1.1 The Potential Outcomes Framework |
|
|
10 | (1) |
|
2.1.1 Time-Varying Confounding and Mediation |
|
|
11 | (2) |
|
2.1.1 Necessary Assumptions |
|
|
13 | (2) |
|
2.2 Examples of Longitudinal Observational Studies |
|
|
15 | (3) |
|
2.2.2 Investigating Warfarin Dosing Using Hospital Data |
|
|
16 | (1) |
|
2.2.2 Investigating Epoetin Therapy Using the United States Renal Data System |
|
|
16 | (1) |
|
2.2.2 Estimating Optimal Breastfeeding Strategies Using Data from a Randomized Encouragement Trial |
|
|
17 | (1) |
|
2.3 Sequentially Randomized Studies |
|
|
18 | (10) |
|
2.3.3 SMART Versus a Series of Single-stage Randomized Trials |
|
|
20 | (1) |
|
|
21 | (4) |
|
2.3.3 Practical Considerations |
|
|
25 | (1) |
|
2.3.3 SMART Versus Other Designs |
|
|
26 | (2) |
|
2.4 Examples of Sequentially Randomized Studies |
|
|
28 | (2) |
|
2.4.4 Project Quit - Forever Free: A Smoking Cessation Study |
|
|
28 | (1) |
|
2.4.4 STAR*D: A Study of Depression |
|
|
29 | (1) |
|
|
30 | (1) |
|
3 Statistical Reinforcement Learning |
|
|
31 | (22) |
|
3.1 Multi-stage Decision Problems |
|
|
31 | (1) |
|
3.2 Reinforcement Learning: A Conceptual Overview |
|
|
32 | (3) |
|
3.3 A Probabilistic Framework |
|
|
35 | (3) |
|
3.4 Estimation of Optimal DTRs by Modeling Conditional Means |
|
|
38 | (7) |
|
3.4.4 Q-learning with Linear Models |
|
|
39 | (2) |
|
3.4.4 Why Move Through Stages? |
|
|
41 | (2) |
|
3.4.4 Analysis of Smoking Cessation Data: An Illustration |
|
|
43 | (2) |
|
3.5 Q-learning Using Observational Data |
|
|
45 | (6) |
|
|
51 | (2) |
|
4 Semi-parametric Estimation of Optimal DTRs by Modeling Contrasts of Conditional Mean Outcomes |
|
|
53 | (26) |
|
4.1 Structural Nested Mean Models |
|
|
53 | (4) |
|
4.1.1 Special Cases of Optimal SNMMs |
|
|
55 | (1) |
|
|
56 | (1) |
|
4.2 Model Parameterizations and Optimal Rules |
|
|
57 | (3) |
|
|
60 | (6) |
|
4.3.3 More Efficient G-estimation |
|
|
62 | (1) |
|
4.3.3 Recursive G-estimation |
|
|
63 | (1) |
|
4.3.3 G-estimation Versus OLS Regression for a One-Stage Problem |
|
|
64 | (1) |
|
4.3.3 Q-learning and G-estimation |
|
|
65 | (1) |
|
4.4 Regret-Based Methods of Estimation |
|
|
66 | (11) |
|
4.4.4 Iterative Minimization of Regrets |
|
|
67 | (4) |
|
|
71 | (1) |
|
|
71 | (2) |
|
4.4.4 Occlusion Therapy for Amblyopia: An Illustration |
|
|
73 | (3) |
|
4.4.4 Simulation of a Two-Stage Occlusion Trial for Treatment of Amblyopia |
|
|
76 | (1) |
|
|
77 | (2) |
|
5 Estimation of Optimal DTRs by Directly Modeling Regimes |
|
|
79 | (22) |
|
5.1 Estimating the Value of an Arbitrary Regime: Inverse Probability Weighting |
|
|
80 | (3) |
|
5.2 Marginal Structural Models and Weighting Methods |
|
|
83 | (11) |
|
5.2.2 MSMs for Static Treatment Regimes |
|
|
83 | (1) |
|
5.2.2 MSMs for Dynamic Treatment Regimes |
|
|
84 | (4) |
|
5.2.2 Simulation of a DTR MSM Analysis to Determine the Optimal Treatment Threshold |
|
|
88 | (2) |
|
5.2.2 Treatment for Schizophrenia: An Illustration |
|
|
90 | (4) |
|
5.3 A Classification Approach to Estimating DTRs |
|
|
94 | (3) |
|
5.3.3 Contrast-Weighted Classification |
|
|
95 | (1) |
|
5.3.3 Outcome Weighted Learning |
|
|
96 | (1) |
|
5.4 Assessing the Merit of an Estimated Regime |
|
|
97 | (2) |
|
|
99 | (2) |
|
6 G-computation: Parametric Estimation of Optimal DTRs |
|
|
101 | (12) |
|
6.1 Frequentist G-computation |
|
|
101 | (6) |
|
6.1.1 Applications and Implementation of G-computation |
|
|
103 | (1) |
|
6.1.1 Breastfeeding and Vocabulary: An Illustration |
|
|
104 | (3) |
|
6.2 Bayesian Estimation of DTRs |
|
|
107 | (5) |
|
6.2.2 Approach and Applications |
|
|
107 | (2) |
|
6.2.2 Assumptions as Viewed in a Bayesian Framework |
|
|
109 | (1) |
|
6.2.2 Breastfeeding and Vocabulary: An Illustration, Continued |
|
|
109 | (3) |
|
|
112 | (1) |
|
7 Estimation of DTRs for Alternative Outcome Types |
|
|
113 | (14) |
|
7.1 Trading Off Multiple Rewards: Multi-dimensional and Compound Outcomes |
|
|
113 | (1) |
|
7.2 Estimating DTRs for Time-to-Event Outcomes with Q-learning |
|
|
114 | (6) |
|
7.2.2 Simple Q-learning for Survival Data: IPW in Sequential AFT Models |
|
|
114 | (1) |
|
7.2.2 Q-learning with Support Vector Regression for Censored Survival Data |
|
|
115 | (5) |
|
7.3 Q-learning of DTRs for Discrete Outcomes |
|
|
120 | (2) |
|
7.4 Inverse Probability Weighted Estimation for Censored or Discrete Outcomes and Stochastic Treatment Regimes |
|
|
122 | (2) |
|
7.5 Estimating a DTR for a Binary Outcome Using a Likelihood Approach |
|
|
124 | (1) |
|
|
125 | (2) |
|
8 Inference and Non-regularity |
|
|
127 | (42) |
|
8.1 Inference for the Parameters Indexing the Optimal Regime Under Regularity |
|
|
128 | (8) |
|
8.1.1 A Brief Review of Variances for Estimating Equations |
|
|
129 | (2) |
|
8.1.1 Asymptotic Variance for Q-learning Estimators |
|
|
131 | (2) |
|
8.1.1 Asymptotic Variance for G-estimators |
|
|
133 | (2) |
|
8.1.1 Projection Confidence Intervals |
|
|
135 | (1) |
|
8.2 Exceptional Laws and Non-regularity of the Parameters Indexing the Optimal Regime |
|
|
136 | (3) |
|
8.2.2 Non-regularity in Q-learning |
|
|
138 | (1) |
|
8.2.2 Non-regularity in G-estimation |
|
|
139 | (1) |
|
8.3 Threshold Estimators with the Usual Bootstrap |
|
|
139 | (6) |
|
8.3.3 The Hard-Threshold Estimator |
|
|
140 | (1) |
|
8.3.3 The Soft-Threshold Estimator |
|
|
141 | (2) |
|
8.3.3 Analysis of Smoking Cessation Data: An Illustration, Continued |
|
|
143 | (2) |
|
|
145 | (3) |
|
8.5 Double Bootstrap Confidence Intervals |
|
|
148 | (1) |
|
8.6 Adaptive Bootstrap Confidence Intervals |
|
|
149 | (2) |
|
8.7 m-out-of-n Bootstrap Confidence Intervals |
|
|
151 | (3) |
|
|
154 | (6) |
|
8.9 Analysis of STAR*D Data: An Illustration |
|
|
160 | (4) |
|
8.9.9 Background and Study Details |
|
|
160 | (1) |
|
|
161 | (1) |
|
|
162 | (2) |
|
8.10 Inference About the Value of an Estimated DTR |
|
|
164 | (2) |
|
8.11 Bayesian Estimation in Non-regular Settings |
|
|
166 | (1) |
|
|
166 | (3) |
|
9 Additional Considerations and Final Thoughts |
|
|
169 | (12) |
|
|
169 | (5) |
|
9.1.1 Penalized Regression |
|
|
170 | (1) |
|
9.1.1 Variable Ranking by Qualitative Interactions |
|
|
171 | (2) |
|
|
173 | (1) |
|
9.2 Model Checking via Residual Diagnostics |
|
|
174 | (3) |
|
9.3 Discussion and Concluding Remarks |
|
|
177 | (4) |
Glossary |
|
181 | (4) |
References |
|
185 | (18) |
Index |
|
203 | |