Monte Carlo Experiments: Bias of Reweighting and FFE Relative to True ATE, and Efficiency of Reweighting Relative to FFE
| True ATE | Bias: | Ratio: MSE of Reweight to MSE of FE | ||
|---|---|---|---|---|
| FE | Reweight | |||
| Panel A : Constant TE; p-Score: Xig | ||||
| Switchers | 80 | −0.3 | −0.2 | 1.03 |
| Siblings | 80 | −0.3 | −0.5 | 1.19 |
| All | 80 | −0.3 | −0.5 | 1.20 |
| HS participants | 80 | −0.3 | −0.3 | 1.04 |
| Panel B: Large Family TE; p-Score: Large Family | ||||
| Switchers | 83.0 | −11.1* | −0.6 | 0.92 |
| Siblings | 49.6 | 22.2* | −0.1 | 0.70 |
| All | 40.3 | 31.6* | 0.1 | 0.54 |
| HS participants | 41.1 | 30.7* | 0.1 | 0.55 |
| Panel C: TE Linear in Xig; p-Score: Xig | ||||
| Switchers | 94.2 | −2.0* | −0.6 | 1.03 |
| Siblings | 80.1 | 12.2* | 1.6* | 0.99 |
| All | 80.0 | 12.2* | 1.7* | 1.00 |
| HS participants | 91.5 | 0.8 | −0.2 | 1.03 |
| Panel D: TE Linear in Xig; p-Score: Xig Spline | ||||
| Switchers | 94.2 | −1.5* | −0.3 | 1.04 |
| Siblings | 80.1 | 12.7* | −0.4 | 1.08 |
| All | 80.0 | 12.8* | −0.4 | 1.09 |
| HS participants | 91.5 | 1.3 | −0.2 | 1.09 |
Notes: This table shows the results from 3,000 Monte Carlo simulations. Each panel of the table shows results from a different DGP and/or different covariates used in the p-score, and each row within panel is for a different target population. The true DGP is linear and is discussed in Section IV.D. Panel A shows results where Head Start has a constant treatment effect (TE) for all individuals. Panel B shows results where Head Start (HS) has no effect on individuals from small families (three or fewer children) and a large effect for families with many children (four or more children). Panels C and D show results where treatment effects are linear in Xig. Column 1, “True Beta,” presents the true average increase in the probability of completing some college for participants in Head Start in the sample, which is a function of the DGP and sample composition. Columns 2 and 3 present the bias of various estimation strategies, defined as the difference between the estimated effects of Head Start and the true beta. The estimated effects come from a LPM, propensity-score weighted LPM, respectively. Column 4 presents the ratio of the mean squared error (MSE) of the reweighting estimators relative to LPM. Reweighted estimates are obtained using in-regression weighting, with weights adjusting for the representativeness of switchers (using the variable(s) indicated in each of the panel headings as predictors in the multinomial logit step) and the conditional variance of Head Start within families. All betas are multiplied by 1,000.
↵* p < 0.01.