Abstract
Conditional cash transfer (CCT) programs link public transfers to human capital investment in hopes of alleviating current poverty and reducing its intergenerational transmission. However, little is known about their long-term impacts. This paper evaluates longer-run impacts on schooling and work of the best-known CCT program, Mexico’s PROGRESA/Oportunidades, using experimental and nonexperimental estimators based on groups with different program exposure. The results show positive impacts on schooling, reductions in work for younger youth (consistent with postponing labor force entry), increases in work for older girls, and shifts from agricultural to nonagricultural employment. The evidence suggests schooling effects are robust with time.
I. Introduction
Conditional cash transfer (CCT) programs were first introduced in Brazil and Mexico more than a decade ago. CCT programs aim, in addition to moderating current poverty, to alleviate future poverty by increasing human capital accumulation of children and youth from poor families and thereby increasing their income when they become adults. Their main innovation, linking cash benefits to families’ investments in human capital (particularly schooling), has been by any measure wildly popular. Well over 30 countries now have as part of their social policy CCT programs, most of which include substantial schooling conditionalities.
Some of the most important impacts of CCTs, including their longer-term effects on schooling and work, can be measured directly only after a significant number of years of program operation. Surprisingly little is known about the longer-term impacts of CCTs on schooling attainment or other outcomes, despite the rapid spread of these programs. Arguably, little is known of the long-term effects of most schooling programs, although efforts at systematically evaluating programs using controlled experiments and nonexperimental statistical methods have expanded.1 Evaluations are limited for the most part to assessing the impact of a fairly short exposure to a new program (that is, for a year or two) on outcomes over short periods. One important reason for this limitation is that most controlled experimental evaluations only last for a year or two, often because of concerns about the political feasibility or fairness of withholding treatment from controls for longer periods of time. Even if controls are incorporated into the program or if the program is stopped, it also is rare for those being evaluated to be followed for more than two or three years. Short-term estimates based on exposure to a program of a year or two have been used to extrapolate to long-run program impacts (for instance, Behrman, Sengupta and Todd 2005; Schultz 2004), but there are a number of reasons why short-term impacts might not be useful for this purpose (King and Behrman 2009). For example, at least one recent controlled experiment (Banerjee et al. 2007) reports that fairly substantial initial program effects in the first two years largely faded after the program was terminated.
In this study, we investigate two types of longer-run impacts for the well-known and influential Mexican PROGRESA/Oportunidades CCT program.2 The following matrix helps to clarify the two types. The columns give two alternative durations since program initiation: short-run (say, one to two years) and longer-run (say five to six years). The two rows give two alternative durations of differential program exposure: short (say, one to two years) and longer (say five to six years). Most of the literature on directly evaluating school programs in general, as well as CCTs in particular, falls into Quadrant A with evaluations of the short-run impact of short exposure differentials. Evaluations of longer-run impacts, as indicated in the matrix, may refer to longer-run effects of short differentials in Exposure B or to longer-run effects of longer differentials in Exposure C. In this study, we evaluate both of these types of longer-run impacts, using both experimental and nonexperimental methodologies:
Longer-run effects of short differential exposure (Quadrant B): Experimental impacts based on an initial short (18-month) differential in exposure using difference-in-difference (DID) estimates that permit assessing whether initial program impacts from short exposure differentials are robust or fade over time, and
Longer-run effects of longer differential exposure (Quadrant C): Difference-in-difference matching (DIDM) impacts based on a longer differential in exposure that provide insight into longer-run program impacts of longer differential in exposure and, in comparison with the estimates of Type 1, into whether there are increasing or diminishing returns to the duration of program exposure.
The latter estimates are obtained using a nonexperimental estimator, because the experimental design was only maintained over a short term (one and a half years in this case), as is common in many evaluation studies. The nonexperimental estimates naturally require stronger assumptions than the experimental estimates regarding the influence of time-varying unobservables on outcomes.
In addition to providing the first longer-term evidence of the impacts of both short and long differentials in program exposure to CCTs on schooling, this paper also investigates program impacts on work, with the hope of beginning to examine potential labor market effects of the program as a result of increased schooling attainment. Lastly, we examine the implications of the impact estimates for benefit-cost ratios that incorporate the real resource costs of the program.3
The original short-term evaluation results (Quadrant A) were based on the experimental program design that randomly assigned 320 communities to a treatment group and 186 to a control group. In 2000, approximately one and a half years after the start of the experiment, the control group communities also began to receive benefits. We directly estimate both types of longer-run impacts of PROGRESA/ Oportunidades using a followup 2003 dataset to analyze impacts on schooling attainment and labor force participation of youth five and a half years after the experimental treatment group first began receiving benefits. The 2003 followup round consisted of the original treatment and control families in the experiment as well as a new group of households in 152 communities that had never received benefits of Oportunidades prior to the survey. The 152 communities were selected by matching observed community-level characteristics to those of the original experimental communities. Our analysis focuses on youth who were 9–15-years-old in 1997, just prior to the program intervention, and who were 15–21-years-old in 2003. In 1997, when the program began, this group was at or near the age of the transition from primary to secondary school, a critical juncture in rural Mexico at which many children drop out of school (see Figures 1 and 2, which also illustrates the inverse labor market-schooling enrollment relation).4 By 2003, many of these youth had entered the labor market, which allows some initial evidence on how the program impacts future labor market outcomes.
School Enrollment and Labor Force Participation of Boys in Oportunidades Communities Prior to Program Implementation.
School Enrollment and Labor Force Participation of Girls in Oportunidades Communities Prior to Program Implementation.
Section II provides a brief description of the program and summarizes a simple economic model of expected impacts on schooling and work. Section III presents the data and the results for the first type of longer-run impact considered—the impact after five and a half years of an 18-month differential program exposure (Quadrant B of the matrix). Section IV presents the data and the results for the second type of longer-run impact considered—the impact after five and a half years since the initiation of treatment of five and a half years differential exposure and the impact after five and a half years since the initiation of treatment of four years differential exposure (Quadrant C). Section V presents cost-benefit analyses and Section VI concludes.
II. Program Background and a Simple Economic Analysis of Subsidy Effects
PROGRESA/Oportunidades began operating in small rural communities in 1997. It gradually expanded to urban areas and now covers five million families, or about one quarter of all families in Mexico. The program provides cash payments to families that are conditional on children regularly attending schools and on family members visiting health clinics for checkups.5 Program take up was exceedingly high when the program began, with 97 percent of families who were offered the program participating.
Table 1 shows the monthly grant levels for children between the third grade and the twelfth grade in the second semester of 2003 (the exchange rate was about 11 pesos per U.S. dollar). Originally, the program provided grants only for children between the third and ninth grades, but in 2001, the grants were extended to Grades 10–12. At Grades 7 and above, the grants are slightly higher (by about 13 percent) for girls than boys. Program rules allow students to fail each grade once, but if a student repeats a grade twice, the schooling benefits are discontinued permanently. In terms of magnitude, the school subsidies constitute the majority of program benefits. However, the program also provides some additional subsidies for school supplies and a transfer that is the same for all households linked to regular visits to health clinics. Children and youth aged 21 and younger are eligible to receive the school subsidies.
Monthly amount of schooling grants (pesos) in second semester of 2003
Broadly speaking, it is useful to think of investment in education as being determined by equating the marginal cost in terms of foregone earnings from work and foregone leisure to the marginal benefit of spending additional time in school—in particular, higher earnings as an adult. School subsidies affect the marginal costs of schooling in the same way as would a decrease in the child wage rate: The subsidy reduces the shadow wage (or relative value) of children’s time in activities other than school. The benefit to schooling in terms of earnings depends on how much time the individual spends working as an adult. Because adult males usually have higher labor market hours than females, the marginal benefit from boys’ schooling in terms of earnings is likely to be higher than that of girls’ at any given schooling level S.6 This may help explain lower average schooling level choices for girls than for boys.7 If this is the case and child wage offers are the same for boys and girls, then a higher subsidy may be required to induce girls to obtain as much schooling as boys. However, if full income rather than earnings is the relevant measure, there is less motivation for giving different subsidy rates to boys and girls.8
Although school subsidies reduce the marginal cost of schooling and therefore increase the investment in education, they have a priori ambiguous effects on work and leisure. Indeed, while the income effect tends to increase the time spent in leisure, the net effects on time spent at school and working would be respectively positive and negative if the substitution effect dominates. It is worth noting that some authors (for instance, Ravallion and Wodon 2000) view education not only as an investment good but also as a consumption good, the demand for which is complementary with the demand for leisure. Such an assumption would not change the previous conclusion regarding the effects on working as long as combined schooling and leisure is a normal good.
As children age, we might expect the substitution effect of the program to change and thus alter the overall impact of the program on work. The program subsidizes school-going, so we would expect children to initially substitute away from time spent in leisure and work and toward time spent in school. However, as they accumulate schooling, they receive higher wage offers. Assuming diminishing marginal returns to schooling, at some point, the marginal benefit of schooling (higher future wages) will no longer exceed the marginal cost (foregone wages and leisure time). These considerations lead us to expect that over the short run, the program is likely to decrease working (assuming the substitution effect continues to dominate the income effect), but over the longer run, the program might increase working.
Different mechanisms may play a role in determining how long-lasting and cumulative are the impacts of the transfers, particularly with respect to schooling. On average, we expect increasing time in the program to increase the magnitude of impacts on grades of schooling attained. However, a number of factors might make the duration of impacts at the individual level less dependent on the duration of the treatment.9 First, there may be indivisibilities in investments in schooling due to the importance of diplomas for the returns to education. For example, once students begin secondary school there is a strong incentive to complete three years required for the diploma. Second, the program may alter the process governing selection into schools and into higher grades. For example, if the program encourages children with lower scholarly ability to stay in school, we might observe an increase in school-going along with a decrease in the rates of grade progression for such children. Dynamic models of schooling decisions have been applied to study how school subsidies affect the composition of students (for instance, Cameron and Heckman 1998; Keane and Wolpin 1997). Finally, the program also may affect the time allocation of children due to other factors, such as peer effects (Bobonis and Finan 2009), changes in social norms regarding education or changes in the quality of the supply of education associated with increasing enrollment.
III. PROGRESA/Oportunidades Impacts on Schooling and Work after Five and a Half Years of an Initial 18-Month Differential Program Exposure
We first consider the longer-run (after five and a half years) impact of a short 18-month differential in program exposure (Quadrant B in the matrix in the introduction), using DID estimates based on the original experimental design.
A. Evaluation Design and Data
As noted in the introduction, the original evaluation and sample design for PROGRESA/Oportunidades involved selecting 506 communities with 320 randomly assigned to receive benefits immediately and the other 186 to receive benefits later. The eligible households in the original treatment localities (we term these T1998) began receiving program benefits in the spring of 1998; whereas the eligible households in the control group (T2000) began receiving benefits at the end of 1999. The 1997 Survey of Household Socio-Economic Conditions (ENCASEH97) serves as a baseline survey for the evaluation and is the survey that was originally used to select households in the eligible communities for participation in PROGRESA/Oportunidades. Between 1997 and 2000, evaluation surveys (ENCELs) with detailed information on demographics, schooling, health, income, and expenditures were administered every six months to all households in both the T1998 and T2000 groups. In 2003, there was a new followup round of the rural Evaluation Survey of PROGRESA/Oportunidades (ENCEL2003) that included all the households that could be located in the original 320 T1998 communities and the original 186 T2000 communities. We link the ENCASEH97 to the ENCEL2003 to have longitudinal data on individual children who were aged 9–15 years in 1997 and aged 15–21 years in 2003. We have information for both 1997 and 2003 on 8,894 youth in T1998 and on 5,591 in T2000.
B. Methodology
We first present DID treatment effect estimates that use the original treatment and control groups for children in 1997. These estimates exploit the experimental design of assignment to treatment in 1998 (T1998) versus assignment to treatment in 2000 (T2000) to evaluate whether the short differential exposure (one and a half years) to treatment between the two groups had longer-run (after five and a half years) impacts on schooling and work as of 2003.10,11
We estimate impacts of differential exposure from a linear regression of the difference in the outcome variable before and after the program on an indicator of whether each program-eligible individual resided in an original treatment or original control locality. Additional covariates, X (parental age, parental schooling attainment, indigenous status, and household characteristics including number of rooms, electricity, type of floor, and water/sewage system) are included to increase precision.12
One concern in evaluating longer-term impacts is that of sample attrition of the original evaluation ENCEL sample—that is, sample attrition of individuals who were in the baseline sample in 1997 but not in the 2003 followup sample. Behrman, Parker, and Todd (2009a), show that attrition levels of youth in the Oportunidades sample are high at around 40 percent. Nevertheless, for the variables studied here, effective attrition is only about 14 percent because information on youth outcomes has been provided by parents or other informants. Nearly all the attrition is household-level—that is, when the entire household leaves the sample. There are, however, some small but significant differences in attrition for the case of girls between the treatment and control groups. The treatment group shows a slightly greater proportion not having information than the control group (14.4 percent versus 12.8 percent) and these differences are significant at the aggregate level and for girls, although not for boys (Table 2).
Proportion attriting by 2003 from original ENCASEH: program eligible individuals 9–15 in 1997
We estimate (Appendix Table A1) the probability of not having information in 2003 for individuals 9–15-years-old in 1997 in eligible households from the T1998 and T2000 groups—and find that several of the preprogram individual, parental, and housing characteristics interacted with treatment (that is, being in the T1998 group) are significant predictors of attrition.13 To account for possible attrition biases, we employ a weighting method that is equivalent to a matching on observables approach. That is, we estimate regressions of program impact, where both the treatment and control group observations are weighted to adjust for differences in the distribution of the observable (X) characteristics arising over time because of attrition.14
C. Results
1. Impacts on Schooling
In 1997, for both boys and girls in the 9–15 age range, there was no significant difference at baseline between schooling grades completed for the T1998 versus T2000 groups (Table 3). By 2003, the estimates in Table 4 indicate that, for both boys and girls, there are significant differences of about a fifth of a grade on average (0.18 for boys and 0.20 for girls). Thus, greater exposure to the program for the T1998 group of one and a half years increased on average by 2003 the schooling grades completed by about 2.4 percent for boys and 2.7 percent for girls beyond the schooling grades completed of the T2000 group. For both girls and boys, there are significant positive impacts for almost all of those who had less than seven grades of schooling completed in 1997 (with the single exception of girls who had only up to three grades of schooling completed in 1997). The largest effects are observed for those who had completed five grades of schooling by 1997 (effects of 6.8 percent for girls, 4.4 percent for boys). The effects are most pronounced for those entering the last year of primary school at the time the program was introduced. In short, youth with 18 months greater exposure to the program accumulated significantly more schooling and this differential persisted for the longer-term five and a half-year period considered here. An F test rejects equality of coefficients at conventional significance levels for each set of subgroups (age and preprogram grades of schooling) for both males and females, suggesting different impacts of PROGRESA/Oportunidades on schooling by age and preprogram schooling grades completed.
Differential exposure to Oportunidades: Schooling and work outcomes for households receiving benefits in 1998 versus 2000
2. Impacts on Working15
The theoretical effect of PROGRESA/Oportunidades on the probability of working is ambiguous, as discussed in Section II. Over the short run, the program decreases working, but over the longer run, the program might increase working.16
In 1997 prior to the program, 0.18 of the T1998 boys and (significantly less at the 5 percent level) 0.16 of the T2000 boys were working; also, in 1997, 0.08 of the T1998 girls and (significantly less at the 1 percent level) 0.05 of the T2000 girls were employed (Table 3). Because of life-cycle work patterns, the proportions employed in 2003 were much higher: for the T2000 boys 0.65 and for the T2000 girls 0.26 with obvious important gender differentials in work (Table 4). The DID estimate of the impact of the one and a half year differential exposure to the program on working five and a half years later (in 2003) shows that greater exposure significantly decreases the proportion working by 4.1 percent for boys with no significant effects for girls (Table 4). Effects do not significantly differ by age groups or by preprogram schooling level.
Impact of Differential Exposure to Oportunidades on Schooling Grades and LFP Difference-in-difference Estimates: Adolescents 9–15 in 1997, T1998 vs. T2000
3. Do Effects Change Over Time?
The above impacts are based on beneficiaries who have received benefits about five and a half years versus nearly four years. An important policy issue is whether program impacts change over time. To analyze this, we add to our data the first round of the ENCEL evaluation survey that was carried out after the control group also began to receive benefits in 2000. Impact estimates for the year 2000 are based on comparing the treatment group which had two and a half years versus the control group at nearly a year of benefits. That is, we test whether impact estimates based on 2000 followup data are different from those based on the 2003 data used in this paper, allowing impact estimates to vary by evaluation round (2000 and 2003). Table 5 summarizes estimates for completed grades of schooling, comparing impact estimates based on the year 2000 data versus those based on year 2003 data. Although a couple of the 2003 estimates show significantly higher estimated impacts for 2003 than 2000, the large majority of the 18 reported estimates do not reject the null hypothesis of equality. The results suggest that PROGRESA/Oportunidades impacts on schooling do not diminish over time.
Do Oportunidades impacts change overtime? Experimental evidence comparing household treated in 1998 with households treated in 2000 (T1998 vs. T2000)
IV. Longer-Run PROGRESA/Oportunidades Impacts after Five and a Half Years of Differential Exposure
We now consider impacts based on a longer period of exposure to the program—that is, the five and a half year differential in program exposure, using DIDM estimates based on the new 2003 comparison group (Quadrant C) that was not exposed to the program. These estimates are useful for analyzing how longer program exposure might change impact estimates as well as studying variables such as work that are expected to have different impacts in the long run than in the short run. As an additional robustness check, we also estimate impacts comparing the original control group with the new comparison group, which provides estimates of differential exposure of about four years, impacts that should presumably be smaller for schooling than those based on five and a half years of differential exposure.
A. Data
The data used in this section are primarily from the 1997 ENCASEH and the ENCEL2003. We link the ENCASEH97 to the ENCEL2003 to have longitudinal data on individual children who were aged 9–15 years in 1997 and aged 15–21 years in 2003. For the new C2003 comparison group households, we use recall data on their 1997 characteristics to characterize their eligibility status in 1997.17 Using both the 2003 and 1997 rounds of data, we construct DIDM estimators of program impacts for the longer-run (five and a half years) impact of relatively large differences (four or five and a half years) in program exposure. For the schooling and work indicators, there are a total of 19,586 youth between the ages of 15 and 21 in 2003.
B. Methodology
To estimate the longer-run program impacts against the benchmark of no program with four or five and a half years difference in program exposure, we compare the original treatment groups (T1998 and T2000) with the new comparison group (C2003) that was drawn from rural areas that had not yet been incorporated into the program in 2003. Because the C2003 group was not selected randomly, we use matching methods to take into account differences in observed characteristics between the T2000 and C2003 samples and between the T1998 and C2003 samples.18,19
The households living in localities where the program was not yet available were unlikely to have been affected by the existence of the program. However, because they lived in different geographic areas from the treatment sample, they may have experienced different local area effects (labor market conditions, quality of schooling, prices) that also may be relevant determinants of outcomes of interest. To take such differences into account, we make use of DIDM estimators (Heckman, Ichimura, and Todd, 1997). The approach is analogous to the standard DID regression estimator, but does not impose functional form restrictions in estimating the conditional expectation of the outcome variable and reweights the observations according to the weighting functions implied by the matching estimators. We use local linear matching and bootstrapping to calculate standard errors for the main estimates presented here.
The DIDM propensity score matching estimators are estimated in two stages. In the first stage, the propensity score is estimated using a logistic model and a set X consisting of preprogram (1997) household and locality level characteristics. The second stage uses local linear regression to construct matched no-treatment outcomes for each treated individual.
The variables used for the matching include demographic characteristics of the households in 1997, grades of schooling completed of household head and spouse in 1997, whether the household head and spouse spoke an indigenous language in 1997, whether the household head and spouse were employed in 1997, a number of household characteristics and consumer and production durables in 1997, the PROGRESA/Oportunidades’ poverty index score for program eligibility in 1997, income in 1997 and state of residence in 1997. Appendix Table A2 gives the estimated propensity score model for the T1998 to C2003 comparison, for which a Chi2 test indicates that the variables are jointly significant at the 0.1 percent level and have fairly good predictive power.20 Figures 3 and 4 show the distributions of propensity scores in the original treatment group (T1998) and the distribution of propensity scores in the C2003 comparison group. Although the distributions between the two groups for each comparison (that is, T2000 versus C2003 and T1998 versus C2003) are clearly different, there is adequate support in the sense that a number of households in C2003 have propensity scores similar to those in T2000 and to those in T1998, although some comparison households with very high propensity scores are likely to be used a number of times as matches. To avoid matching children of different ages and genders, we implement the matching conditional on age and sex, as well as the household’s propensity score.
Distribution of Propensity Score: Treatment 1998
Distribution of Propensity Score: New comparison group
We implemented several variants of balancing tests as an aid in specifying an appropriate propensity score specification. These tests examine whether the distribution of the covariates included in the propensity score model is independent of program participation conditional on the estimated propensity score, as it should be if the propensity score model is correctly specified and the estimator is consistent. First, we implemented a procedure used previously by Dehejia and Wahba (2002) that stratifies treatment and control observations into strata based on the estimated propensity score (in quintiles) and then tests for significant differences between the covariates within each stratum. The vast majority of the covariates showed no significant differences by quintile; for the few with significant differences, we added higher-order terms to the propensity score model. We also carried out alternative balancing tests summarized in Todd and Smith (2005), including a test proposed in Rosenbaum and Rubin (1985) that calculates standardized differences for each covariate between the treatment and matched comparison group. Less than 5 percent of the covariates have standardized differences above the value of 20 percent, which is typically considered to be within an acceptable range.
C. Results
In this section, we present impact estimates on schooling, work and participation in agricultural work, comparing those with five and a half years of PROGRESA/Oportunidades to those never receiving benefits.21 For schooling, we also present estimates based on the T2000 versus C2003 comparison—that is, estimating impacts for those with nearly four years of benefits versus never receiving benefits. This provides a benchmark against which to judge the reasonableness of the matching-based estimates; for a cumulative variable like schooling, program estimates should be larger for those with longer time exposure to the program.
1. Impact on Schooling
Table 6 shows important effects on school grades completed for both boys and girls, particularly those who were younger when the program began. Boys aged 9–10 preprogram (15–16 in 2003) accumulate 1.0 additional grades of schooling than comparable boys without the program, while boys aged 11–12 preprogram accumulate an additional 0.9 grades. Boys 13–15 preprogram accumulate about half a year of additional schooling. Impacts also are significant, although slightly smaller, for girls. Girls aged 9–12 preprogram accumulate 0.7 to 0.8 grades of schooling, with no significant impacts for older girls. Compared with schooling grades completed of the comparison group in 2003 (about seven grades), these impacts represent significant increases of about 13–15 percent for boys and 10 percent for girls in overall schooling grades completed, relative to the comparison group never receiving benefits. Overall, the results show an important increase in schooling levels attained because of the program.
Estimated Impacts of PROGRESA/Oportunidades on Grades of Schooling Local linear longitudinal DID matching † T1998 vs. C2003 and T2000 vs. C2003
We now turn to schooling impact estimates based on comparing T2000 to the new C2003 comparison group (Table 6). These estimates compare individuals receiving four years of benefits with those never receiving benefits. Again, one would expect proportionately lower estimated impacts on schooling from this comparison than that based on five and a half years’ difference in exposure. The results indicate somewhat smaller impacts of the program, on the order of 25 to 30 percent smaller as would be expected from the lower duration of program exposure differentials. Figures 5 and 6 summarize average impact estimates on schooling. In general, the findings confirm significantly larger impacts on schooling with a longer time of exposure to the program than for those with shorter time of exposure, with impacts increasing in an approximately linear fashion with exposure.
Schooling impacts by program length: girls
Schooling impacts by program length: boys
2. Impact on Working
The DIDM results in Tables 7, for younger boys—that is, those aged 9–10 preprogram in 1997—show a negative and significant impact on the probability of employment, consistent with many boys still attending school at this age (15–16 in 2003). The magnitude of the impact is large, corresponding to a reduction of almost 30 percent in the probability of working. For the older age groups, there is no significant impact of the program on working, which may mask negative impacts from some boys continuing to study and positive impacts from those having completed their studies and entering the work force. For girls, who in these traditional communities have much lower wage labor force participation rates than boys, there is no significant impact on working for younger girls. However, for those girls aged 13–15 preprogram (19–21 post), there is an important and significant increase in the proportion working, of 6 percentage points corresponding to an increase of 20 percent (Table 7). Note however this older group of girls showed no significant increase in schooling, suggestive that the program may influence work through other channels. One possibility is that older girls substitute in the labor market for their younger siblings, who do show increases in their schooling and reductions in work (in the case of boys).
Estimated Impacts of PROGRESA/Oportunidades on Probability of Working and Participation in Agricultural Work Local linear longitudinal DID matching † T1998 vs. C2003
3. Probability of Working in Agriculture
Schooling is often claimed to have higher returns in nonagricultural than in agricultural work, so if PROGRESA/Oportunidades were effective it might induce some shift out of the agricultural sector. We look at the unconditional probability of individuals participating in agricultural work.22 Note that overall, girls have much lower participation in agricultural work than boys. For boys, the reductions in agricultural work for boys aged 9–10 preprogram are similar to the percentage reductions in work, implying similar reductions in agricultural as nonagricultural work (Table 7). For older boys—that is, aged 13–15 preprogram—however, on which there were no overall significant program effects on working, there are significant reductions in agricultural work, implying some substitution from agricultural work to nonagricultural work. For girls, there is no overall effect on participation in agricultural work for younger girls, who also showed no effect on working. For older girls, for whom there was a significant increase in the probability of working, there is no significant increase in the probability of participating in agricultural work, again suggestive of increasing nonagricultural work.
V. Benefit-Cost Estimates
The relatively large impacts on schooling attainment documented here raise the issue of whether the benefits of the PROGRESA/Oportunidades program justify the costs and in particular, how the program performs given its cost relative to other types of human capital programs. Clearly, PROGRESA/Oportunidades is not only a human capital investment program, because it also alleviates current poverty by giving money directly to the poor. Nevertheless, in this section, we offer a benefit-cost analysis of the program as if the only objective of the program were investment in human capital and the only benefits the subsequent impact on future earnings of the increased schooling attainment of students affected by PROGRESA/Oportunidades. Carrying out this analysis requires a number of additional assumptions and calculations, which are detailed below.
Here we assume the benefits from the program arise from increases in future earnings only as a result of the increase in schooling attainment from the program, ignoring other potential impacts, such as that of improved health and nutrition. We estimate the return to schooling controlling for potential endogeneity using a plausible instrument given by a change in the compulsory schooling law in 1993 from sixth to ninth grade.23 The estimates are based on data from the 2002 nationally representative Mexican Family Life Survey (MXFLS). We use the estimated returns to provide a guide for what the returns to schooling attainment for youth in rural areas are likely to be, and we also provide simulations based on varying levels of returns.
Focusing on the sample of men, a simple OLS estimate of returns to schooling attainment gives a value of 7.5 percent per year for this age group (16–24), which is similar across rural (7.2) and urban (7.6) areas. The IV estimate of wage functions increases the value to about 10 percent both for those residing in rural and in urban areas.24 We provide simulations below based on 6 percent, 8 percent, and 10 percent returns to schooling attainment.
The resource costs of the program include the administrative costs of the program (costs of transferring benefits, conditionality, and targeting) and the private costs associated with participation in the program.25 The administrative and private costs of participation in the program including the monetary and time costs of transportation associated with greater school attendance were calculated in Coady (2000). He estimates that for each 100 pesos transferred, administrative and private costs are equivalent to 11.3 pesos. We assume that an additional year spent in school results in a 0.75 year decrease in time spent in the labor force.26 We obtain estimates other than transportation of the additional household expenses incurred because of increased schooling—that is, school supplies and children’s clothing—from an earlier study by Hoddinott, Skoufias, and Washburn (2000). Finally, for each peso of governmental expenditures (whether on resources or transfers) we assume a 25 percent increase in cost (grants plus administrative) from the distortions associated with raising revenues.27 We assume that each youth in our sample receives the grants for six years. Youth in the absence of the program are assumed to begin working at age 18 and conclude at age 70.28 All costs and benefits are discounted to time zero.
We assume a starting salary equal to the average obtained by youth aged 18 in the rural ENCEL areas of 2003, which is equal to $US163 monthly. This is a conservative estimate of initial earnings opportunities as probably a number of youth will migrate to urban areas, where salaries tend to be higher. We use the schooling impacts on boys aged 15–16 (see Table 6).
Table 8 provides benefit-cost estimates of PROGRESA/Oportunidades, under the three different assumptions about the rate of return to schooling attainment (6, 8, and 10 percent) and three potential discount rates (3, 5, and 10 percent). Program benefits are several times higher than program costs under nearly all scenarios, with the exception of a very high discount rate and a low estimated return to schooling. Overall, then, even if the program were considered as only a human capital investment program working only through increasing schooling attainment, the overall benefits would seem to significantly outweigh the costs.
Costs and Benefits of the PROGRESA/Oportunidades Program in U.S. dollars
VI. Conclusions
This paper has provided a comprehensive analysis of longer-run program impacts of Oportunidades on schooling and work after five and a half years of program benefits. We have gone beyond previous studies of relatively short-term impact (two years or less) of limited differential program exposure (one and a half years at most) by investigating longer-run program effects of two types: (1) The impact of a short differential exposure on longer run outcomes (after five and a half years), estimated using treatment and control data from the large-scale randomized experiment, and (2) the impact of a longer (four or five and a half years) differential in exposure on longer-run outcomes, estimated using propensity score matching methods applied to data from the treatment group and from a nonexperimental comparison group.
In summary, the longer-term impacts of the program appear quite positive with important increases in schooling attainment, and for older youth, some higher rates of working and a shift away from agricultural work to nonagricultural work. Benefit estimates based on earnings functions from adults integrated with careful costs estimates indicate fairly high benefit-to-cost ratios unless the rate of return to schooling are low and the discount rate high. The additional years of data that we use, finally, permit much more confident inferences about longer-run effects. They suggest that (1) the initial differential exposure of one and a half years appears to have an impact on schooling attainment that is robust with the passage of time and (2) the program impacts on schooling attainment increase approximately linearly with the duration of exposure to the program.
We have combined experimental and nonexperimental data to provide what we believe are convincing and consistent estimates of longer-term impacts of PROGRESA/Oportunidades. Although experiments provide the best approach for providing unbiased estimates of program impacts, it is unusual for social experiments to last more than a year or so and PROGRESA/Oportunidades has been no exception. We have used the experimental estimates to provide guidance on the reasonableness of the nonexperimental longer-term impacts and our evidence shows consistent, robust and large program impacts of PROGRESA/Oportunidades on schooling attainment. Nevertheless, the evidence in this paper suggests it is still early to evaluate the impact of the significant increases in schooling on income and poverty. Further followup studies will be necessary as the first generation of PROGRESA/Oportunidades graduates enters adulthood.
Appendix
Probability of attriting between 1997–2003 by 1997 characteristics: T1998 vs. T2000 Girls and Boys 9–15 program eligible in 1997
Logit Model for Probability of Participating in Rural PROGRESA/Oportunidades
Footnotes
Jere R. Behrman is the W. R. Kenan Jr. Professor of Economics and Sociology and PSC research associate at the University of Pennsylvania. Susan W. Parker is a professor/researcher in the Division of Economics at the Center for Research and Teaching in Economics (CIDE) in Mexico City. Petra E. Todd is professor of economics and a research associate of PSC at the University of Pennsylvania, the National Bureau of Economic Research (NBER) and the Institute for the Study of Labor (IZA). This work received support from the Instituto Nacional de Salud Publica (INSP) and the Mellon Foundation/Population Studies Center (PSC)/University of Pennsylvania grant to Todd (P.I.) on “Long-term Impact Evaluation of the Oportunidades Program in Rural Mexico.” The authors thank three anonymous referees, Bernardo Hernández, Iliana Yaschine and seminar participants at the University of Pennsylvania, the World Bank, and the University of Goettingen for helpful comments on early versions of this paper. The authors especially acknowledge the vision of the late José Gómez de León, founding program director of PROGRESA/Oportunidades. The authors alone, and not INSP, the Mellon Foundation, the PSC, nor PROGRESA/Oportunidades, are responsible for any errors in this study. Table 4 and Figures 1 and 2 have previously been published in Chapter 7 of Poverty, Inequality and Policy in Latin America, edited by Stephan Klasen and Felicitas Nowak-Lehmann, published by The MIT Press. The authors gratefully acknowledge permission from the MIT Press to republish them here. The data used in this article can be obtained beginning August 2011 through July 2014 from Susan W. Parker; CIDE Division of Economics; Carretera Mexico Toluca 3655; Col. Lomas de Santa Fe; 01219 Mexico DF MEXICO; susan.parker{at}cide.edu.
↵1. Behrman (2009), Glewwe and Kremer (2006), and Orazem and King (2008) review many of these evaluations of schooling-related programs.
↵2. PROGRESA is an acronym for the original name of the program (Programa de Educacíon, Salud y Alimentacíon, Program for Education, Health and Nutrition) introduced in 1997 in the Zedillo government. When the Fox government came into power in 2000, the program was modified in some of its details (e.g., coverage of upper secondary schooling, extension into more urban areas) and renamed “Oportunidades.”
↵3. Much previous literature does not address costs and the studies that do tend to include only governmental budgetary costs, not real resource costs (inclusive of private costs and distortion costs, but not transfers). For example, Miguel and Kremer (2004) consider only the direct cost of administering medication to eliminate the worms for their cost estimates, but do not include the opportunity cost of the increased time students attend school, the congestion costs of expanded school enrollment, or the distortion costs of raising public revenues for the program.
↵4. Previous evaluations have, in fact, demonstrated that the largest effects of the program were precisely at this transition between primary and secondary school (see Behrman, Sengupta and Todd 2005, Schultz 2004, and Todd and Wolpin 2006).
↵5. Regular school attendance (85 percent of time) is required to continue receiving the monthly grant payments as is attendance at a health talk once a month for high school students.
↵6. Assuming the marginal opportunity cost of boy’s schooling is not higher than for girls’ schooling.
↵7. Although Behrman, Sengupta, and Todd (2005) show that actual attainment of girls in terms of grades of completed schooling in rural areas preprogram averaged more than that of boys so the policy of higher grants for girls does not seem to be justified by these gender differences on average in preprogram schooling.
↵8. Parker, Rubalcava, and Teruel (2008) have shown that if women dedicate more time to housework in the absence of subsidies, and if the marginal effect of schooling on labor productivity in housework is decreasing in the time devoted to housework, a result of school subsidies may be an increased participation of women relative to men in the labor market.
↵9. We would like to thank a referee for suggesting the points in this paragraph.
↵10. Of all those offered the program, 97 percent participated, implying that intent to treat estimates carried out here are basically equivalent to average treatment effects on the treated.
↵11. Given the evaluation design, our strategy focuses on the longer run impact of short exposure differentials, with every child receiving the program after 2000. Alternative short exposure differentials would correspond to a situation where no child receives the program after that date. These two treatments might differ if program impacts post-2000 are not identical for T1998 and T2000 because the T2000 group starts receiving benefits 1.5 years later. The alternative counterfactual might be better suited to study whether there are increasing or decreasing returns to the duration of the program. We are constrained however by the actual design, although note that in other programs coverage is commonly extended to include control groups (for instance Miguel and Kremer, 2004). To address the issue of whether program impacts are maintained over time, at the end of Section III we estimate program impacts shortly after the control group began receiving benefits (at the end of 2000) and compare these impact estimates to those estimated in 2003, the last year of our data.
↵12. Standard errors are clustered to allow for potential correlations between individuals within communities.
↵13. The treatment interactions that are significant include preprogram enrollment (positive). If pre program enrollment is positively correlated with higher impacts, then we might underestimate program impacts, in the absence of a correction for attrition bias. Note, however, we estimated impacts on completed grades of schooling dividing the sample into those enrolled preprogram in school in 1997 and those not enrolled and program impacts did not significantly differ between the two groups.
↵14. The reweighting estimator adjusts for differences in the distribution of the X characteristics between the treatment and control groups that can arise over time because of attrition. We use as the weights the ratio of the univariate densities of the propensity to attrit. Through this procedure, each individual observed post-program receives a weight equal to the ratio of the density of his/her probability of attriting with respect to the post-program distribution (of treatments or controls) divided by the density estimated with respect to the preprogram (and preattrition) distribution. Effectively, this procedure reweights the post-program observations to have the same distribution of X as they did prior to the attrition. The key assumption that justifies application of this procedure is that attrition is random conditional on X; within each of the groups. See Behrman, Parker, and Todd (2009a).
↵15. The definition of work excludes domestic work, which may underestimate the impacts of the program for girls. We do not analyze impacts on domestic work because we do not have baseline information. See Skoufias and Parker (2001) however for evidence suggesting that the program in the initial years reduced time spent in domestic work for girls.
↵16. In unreported results, we also analyze the impact on unconditional hours of work, finding similar effects as those for labor market participation.
↵17. Recall data from 1997 was only collected from the C2003 group and not from the T1998 and T2000 groups so that unfortunately we cannot judge the accuracy of the recall data compared with information collected on current characteristics in 1997. We carried out two exercises to insure our results are not biased by measurement errors in the recall data. 1) We compared average enrollment rates for youth aged 6–14 at the community level for T2000 and C2003 communities in the year 2000 and found similar patterns as to those for pre program education levels in our data in 1997 (based on recall data for C2003 and pre program surveys for T1998 and T2000). 2) We carried out alternative propensity score estimations using only current characteristics reported in 2003 and unlikely to have been affected by the program (such as parental education). The results are quite similar as to those presented.
↵18. The localities that were included in the sampling frame for C2003 were selected by matching on locality characteristics constructed using household information aggregated at the community level from the 1995 and 2000 Censuses on housing attributes, demographic structure, poverty levels, labor force participation, and ownership of durable goods. Selected localities were also constrained to come from the same states as the original 506 evaluation communities with the exception of one state where some communities from a neighbor state were used. All communities were constrained to satisfy PROGRESA/ Oportunidades eligibility characteristics with regard to distance to schools and health clinics. See Todd (2004) for further details.
↵19. Behrman, Parker, and Todd (2009b) analyze preprogram Census information on community characteristics between 1995 and 2000 for the evaluation sample, comparing T2000 and C2003. Those measuring schooling attainment show no significant differences between changes overtime in the T2000 and C2003 groups.
↵20. We carry out a separate propensity score estimation for the T2000 vs. C2003 comparison that yields similar results as that for T1998 vs. C2003.
↵21. We also carried out a longitudinal DIDM estimates on schooling varying the bandwidth for the local linear matching estimates and kernel matching estimates. The results are similar across these different matching estimators and available upon request.
↵22. We do not condition on working as this itself is an impact indicator, subject to changes with the program, as demonstrated above. Thus, the impacts on the proportion of individuals in agriculture might change under the program both as a result of working individuals shifting in or out of agriculture as well as new individuals entering/exiting work.
↵23. In 1993, lower secondary school (seventh through ninth grade) became mandatory. A main impact of this change was a large increase in the construction of lower secondary schools, the majority of which were of the telesecondary school type, which is a mode of secondary school provided only in rural areas. We use school construction interacted with whether an individual lived in rural areas at the age of 12 as a variable affecting completed schooling in 2002.
↵24. Similar increased estimated schooling effects with IV estimates are reported by Ashenfelter and Krueger (1994) for the United States, Duflo (2001) for Indonesia, and Behrman, Murphy, Quisumbing, and Young (2008) for Guatemala.
↵25. They do not include the budgetary costs of the schooling grants based on the calendar of grants (see Table 1) because these are transfers, not resource costs (see Knowles and Behrman 2004, 2005). An earlier study found no evidence of congestion costs or other spillover effects on nonbeneficiary students in the same schools, so we do not adjust for costs or benefits of spillovers either (Behrman, Sengupta, and Todd 2005).
↵26. Skoufias and Parker (2001) show that the reduction in work of PROGRESA/Oportunidades in the early years of the program was about 75 percent the size of the increase in school enrollment.
↵27. The distortionary cost of raising a dollar of tax revenue in the United States has been estimated to range from $0.17 to $0.56, depending on the type of tax used (Ballard, Shoven and Whalley 1985, Feldstein 1995). Harberger (1997) suggests using a shadow price of $1.20–1.25 for all fiscal flows on a project. We use the upper number in his suggested range.
↵28. In our sample in 1997, of all men aged 70 and older, 62 percent were working, consistent with high poverty and the lack of a pension system covering the informal sector in Mexico. Using an early retirement at age 55 reduces the estimated benefits by about 25 percent but does not significantly change the conclusions here.
- Received August 2009.
- Accepted January 2010.