Cross-section individual level data Monte Carlo Rejection Rates of True Null Hypothesis (Slope = 0) with Different Number of Clusters and Different Rejection Methods
| Nominal 5 percent rejection rates | |||||
|---|---|---|---|---|---|
| Numbers of Clusters | |||||
| Wald Test Method | 6 | 10 | 20 | 30 | 50 |
| Different standard errors and critical values | |||||
| White robust, T(N – k) for critical value | 0.439 | 0.457 | 0.471 | 0.462 | 0.498 |
| Cluster on state, T(N – k) for critical value | 0.215 | 0.147 | 0.104 | 0.083 | 0.078 |
| Cluster on state, T(G – 1) for critical value | 0.125 | 0.103 | 0.082 | 0.069 | 0.075 |
| Cluster on state, T(G – 2) for critical value | 0.105 | 0.099 | 0.076 | 0.069 | 0.075 |
| Cluster on state, CR2 bias correction, T(G − 1) for critical value | 0.082 | 0.070 | 0.062 | 0.060 | 0.065 |
| Cluster on state, CR3 bias correction, T(G − 1) for critical value | 0.048 | 0.050 | 0.050 | 0.052 | 0.061 |
| Cluster on state, CR2 bias correction, T(IK degrees of freedom) | 0.052 | 0.050 | 0.047 | 0.047 | 0.054 |
| Cluster on state, T(CSS effective number of clusters) | 0.114 | 0.079 | 0.057 | 0.056 | 0.061 |
| Pairs cluster bootstrap for standard error, T(G − 1) for critical value | 0.082 | 0.072 | 0.069 | 0.067 | 0.074 |
| Bootstrap percentile-Γ methods | |||||
| Pairs cluster bootstrap | 0.009 | 0.031 | 0.046 | 0.051 | 0.061 |
| Wild cluster bootstrap, Rademacher two-point distribution, low p-value | 0.097 | 0.065 | 0.062 | 0.051 | 0.060 |
| Wild cluster bootstrap, Rademacher two-point distribution, mid p-value | 0.068 | 0.065 | 0.062 | 0.051 | 0.060 |
| Wild cluster bootstrap, Rademacher two-point distribution, high p-value | 0.041 | 0.064 | 0.062 | 0.051 | 0.060 |
| Wild cluster bootstrap, Webb 6 point distribution | 0.079 | 0.067 | 0.061 | 0.051 | 0.061 |
| Wild cluster bootstrap, Rademacher two-point, do not impose null hypothesis | 0.086 | 0.063 | 0.050 | 0.053 | 0.056 |
| IK effective DOF (mean) | 3.3 | 5.6 | 9.4 | 12.3 | 16.9 |
| IK effective DOF (5th percentile) | 2.7 | 3.7 | 4.9 | 6.3 | 9.6 |
| IK effective DOF (95th percentile) | 3.8 | 7.2 | 14.5 | 20.8 | 29.5 |
| CSS effective number of clusters (mean) | 4.7 | 6.6 | 9.9 | 12.7 | 17 |
| Average number of observations | 1,554 | 2,618 | 5,210 | 7,803 | 13,055 |
Notes: March 2012 CPS data, 20 percent sample from IPUMS download. For six and ten clusters, 4,000 Monte Carlo replications. For 20–50 clusters, 1,000 Monte Carlo replications. The bootstraps use 399 replications. “IK effective DOF” from Imbens and Kolesar (2013), and “CSS effective number of clusters” from Carter, Schnepel, and Steigerwald (2013), see Section VID. Row 11 uses lowest p-value from interval, when Wild percentile-T bootstrapped p-values are not point identified due to few clusters.
Row 12 uses midrange of interval, and Row 13 uses largest p-value of interval.