State-year panel data with differences-in-differences estimation Monte Carlo Rejection Rates of True Null Hypothesis (Slope = 0) with Different Number of Clusters and Different Rejection Methods
| Nominal 5% rejection rates | |||||
|---|---|---|---|---|---|
| Numbers of Clusters | |||||
| Estimation Method | 6 | 10 | 20 | 30 | 50 |
| Wald tests | |||||
| Default standard errors, T(N – k) for critical value | 0.589 | 0.570 | 0.545 | 0.526 | 0.556 |
| Cluster on state, T(N – k) for critical value | 0.149 | 0.098 | 0.065 | 0.044 | 0.061 |
| Cluster on state, T(G – 1) for critical value | 0.075 | 0.066 | 0.052 | 0.039 | 0.058 |
| Cluster on state, T(G – 2) for critical value | 0.059 | 0.063 | 0.052 | 0.038 | 0.058 |
| Pairs cluster bootstrap for standard error, T(G – 1) for critical value | 0.056 | 0.060 | 0.050 | 0.036 | 0.057 |
| Bootstrap percentile-T methods | |||||
| Pairs cluster bootstrap | 0.005 | 0.019 | 0.051 | 0.044 | 0.069 |
| Wild cluster bootstrap, Rademacher two-point distribution | 0.050 | 0.059 | 0.050 | 0.036 | 0.055 |
| Wild cluster bootstrap, Webb 6 point distribution | 0.056 | 0.059 | 0.048 | 0.037 | 0.058 |
Notes: March 1997–2012 CPS data, from IPUMS download. Models include state and year fixed effects and a “fake policy” dummy variable that turns on in 1995 for a random subset of half of the states. For six and ten clusters, 4,000 Monte Carlo replications. For 20–50 clusters, 1,000 Monte Carlo replications. The bootstraps use 399 replications.