Research ArticleArticles
Open Access

Does Federally Funded Job Training Work?

Nonexperimental Estimates of WIA Training Impacts Using Longitudinal Data on Workers and Firms

Fredrik Andersson, Harry J. Holzer, Julia I. Lane, David Rosenblum and Jeffrey Smith
Journal of Human Resources, July 2024, 59 (4) 1244-1283; DOI: https://doi.org/10.3368/jhr.0816-8185R1

References

  1. ↵
    1. Abowd, John
    , and Francis Kramarz. 2002. “The Analysis of Labor Markets Using Matched Employer–Employee Data.” In Handbook of Labor Economics, Volume 3, Part B, ed. Orley Ashenfelter and David Card, 2629–710. New York: Elsevier.
    OpenUrl
  2. ↵
    1. Abowd, John
    , Francis Kramarz, and David Margolis. 1999. “High Wage Workers and High Wage Firms.” Econometrica 67(2):251–333.
    OpenUrlCrossRefWeb of Science
  3. ↵
    1. Abowd, John
    , Paul Lengermann, and Kevin McKinney. 2002. “The Measurement of Human Capital in the U.S. Economy.” LEHD Technical Paper TP‐2002–09. Washington, DC: U.S. Bureau of the Census.
  4. ↵
    1. Abowd, John
    , Bryce Stephens, Lars Vilhuber, Fredrik Andersson, Kevin McKinney, Marc Roemer, and Simon Woodcock. 2009. “The LEHD Infrastructure Files and the Creation of the Quarterly Workforce Indicators.” In Producer Dynamics: New Evidence from Micro Data, ed. Timothy Dunne, J. Bradford Jensen, and Mark Roberts, 149–230. Chicago: University of Chicago Press.
  5. ↵
    1. Almandsmith, Sherry
    , Lorena Adams, and Hans Bos. 2006. “Evaluation of the Strengthening the Connections between Unemployment Insurance and the One‐Stop Delivery Systems Demonstration Project in Wisconsin: Final Report.” Oakland, CA: Berkeley Policy Associates.
  6. ↵
    1. Andersson, Fredrik
    , Harry Holzer, and Julia Lane. 2005. Moving Up or Moving On: Who Advances in the Low‐Wage Labor Market? New York: Russell Sage Foundation.
  7. ↵
    1. Ashenfelter, Orley
    . 1978. “Estimating the Effect of Training Programs on Earnings.” Review of Economics and Statistics 60(1):47–57.
    OpenUrlCrossRefWeb of Science
  8. ↵
    1. Auerbach, Alan
    , and James Hines. 2002. “Taxation and Economic Efficiency.” In Handbook of Public Finance, Volume 3, ed. Alan Auerbach and Martin Feldstein, 1347–421. New York: Elsevier.
    OpenUrl
  9. ↵
    1. Barnow, Burt
    . 2011. “Lessons from WIA Performance Measures.” In The Workforce Investment Act: Implementation Experiences and Evaluation Findings, ed. Douglas Besharov and Phoebe Cottingham, 209–32. Kalamazoo, MI: Upjohn Institute.
  10. ↵
    1. Barnow, Burt
    , and Christopher King. 2005. “The Workforce Investment Act in Eight States.” Report Prepared for U.S. Department of Labor, Employment and Training Administration.
  11. ↵
    1. Barnow, Burt
    , and Jeffrey Smith. 2004. “Performance Management of U.S. Job Training Programs.” In Job Training Policy in the United States, ed. Christopher O’Leary, Robert Straits, and Stephen Wandner, 21–56. Kalamazoo, MI: Upjohn Institute.
  12. ↵
    1. Barnow, Burt
    , and Jeffrey Smith. 2016. “Employment and Training Programs.” In Means‐Tested Transfer Programs in the United States, Volume II, ed. Robert Moffitt, 127–234. Chicago: University of Chicago Press.
    OpenUrl
  13. ↵
    1. Barron, John
    , Mark Berger, and Dan Black. 1997. On‐the‐Job Training. Kalamazoo, MI: Upjohn Institute.
  14. ↵
    1. Baum, Sandy
    , Harry Holzer, and Grace Luetmer. 2020. Should the Federal Government Fund Short‐Term Postsecondary Certificate Programs? Washington, DC: Urban Institute.
  15. ↵
    1. Bergemann, Annette
    , and Gerard van den Berg. 2008. “Active Labor Market Policy Effects for Women in Europe—A Survey.” Annals of Economics and Statistics 91/92:384–408.
    OpenUrl
  16. ↵
    1. Besharov, Douglas
    , and Phoebe Cottingham, eds. 2011. The Workforce Investment Act: Implementation Experiences and Evaluation Findings. Kalamazoo, MI: Upjohn Institute.
  17. ↵
    1. Biewen, Martin
    , Bernd Fitzenberger, Adironke Osikominu, and Marie Paul. 2014. The Effectiveness of Public‐Sponsored Training Revisited: The Importance of Data and Methodological Choices.” Journal of Labor Economics 32(4):837–97.
    OpenUrlCrossRef
  18. ↵
    1. Black, Dan
    , Kermit Daniel, and Jeffrey Smith. 2005. “College Quality and Wages in the United States.” German Economic Review 6(3):415–43.
    OpenUrl
  19. ↵
    1. Blank, Diane
    , Laura Heald, and Cynthia Fagnoni. 2011. “An Overview of WIA.” In The Workforce Investment Act: Implementation Experiences and Evaluation Findings, ed. Douglas Besharov and Phoebe Cottingham, 49–80. Kalamazoo, MI: Upjohn Institute.
  20. ↵
    1. Bloom, Howard
    , Larry Orr, Stephen Bell, George Cave, Fred Doolittle, Winston Lin, and Johannes Bos. 1997. “The Benefits and Costs of JTPA Title II‐A Programs: Findings from the National Job Training Partnership Act Study.” Journal of Human Resources 32:549–76.
    OpenUrlCrossRefWeb of Science
  21. ↵
    1. Brown, Charles
    , James Hamilton, and James Medoff. 1990. Employers Large and Small. Cambridge, MA: Harvard University Press.
  22. ↵
    1. Cahuc, Pierre
    , and Thomas Le Barbanchon. 2010. “Labor Market Policy Evaluation in Equilibrium: Some Lessons of the Job Search and Matching Model.” Labour Economics 17(1):196–205.
    OpenUrl
  23. ↵
    1. Caliendo, Marco
    , Robert Mahlstedt, and Oscar Mitnik. 2017. “Unobservable, but Unimportant? The Relevance of Usually Unobserved Variables for the Evaluation of Labor Market Policies.” Labour Economics 46:14–25.
    OpenUrl
  24. ↵
    1. Calónico, Sebastian
    , and Jeffrey Smith. 2017. “The Women of the National Supported Work Demonstration.” Journal of Labor Economics 35(S1):S65–S97.
    OpenUrl
  25. ↵
    1. Card, David
    , Jochen Kluve, and Andrea Weber. 2010. “Active Labour Market Policy Evaluations: A Meta‐Analysis.” Economic Journal 120(548):F452–F477.
    OpenUrl
  26. ↵
    1. Card, David
    , Jochen Kluve, and Andrea Weber. 2018. “What Works? A Meta‐Analysis of Recent Active Labor Market Program Evaluations.” Journal of the European Economic Association 16(3):894–931.
    OpenUrl
  27. ↵
    1. Chabé‐Ferret, Sylvain
    . 2015. “Analysis of the Bias of Matching and Difference‐in‐Difference under Alternative Earnings and Selection Processes.” Journal of Econometrics 185:110–23.
    OpenUrl
  28. ↵
    1. Courty, Pascal
    , and Gerald Marschke. 2004. “An Empirical Investigation of Dysfunctional Responses to Explicit Performance Incentives.” Journal of Labor Economics 22(1):23–56.
    OpenUrlCrossRefWeb of Science
  29. ↵
    1. Dahlby, Bev
    . 2008. The Marginal Cost of Public Funds. Cambridge, MA: MIT Press.
  30. ↵
    1. D’Amico, Ronald
    , Kate Dunham, Annelies Goger, Charles Lea, Nicole Rigg, Sheryl Ude, and Andrew Wiegand. 2009. Findings from a Study of One‐Stop Self‐Services: A Case Study Approach. Oakland, CA: Social Policy Research Associates.
  31. ↵
    1. D’Amico, Ronald
    , Kate Dunham, Jennifer Henderson‐Frakes, Deborah Kogan, Vinz Koller, Melissa Mack, Micheline Magnotta, Jeffrey Salzman, Andrew Wiegand, Gardner Carrick, and Dan Weissbein. 2004. The Workforce Investment Act after Five Years: Results from the National Evaluation of the Implementation of WIA. Oakland, CA: Social Policy Research Associates.
  32. ↵
    1. Decker, Paul
    . 2011. “Ten Years of WIA Research.” In The Workforce Investment Act: Implementation Experiences and Evaluation Findings, ed. Douglas Besharov and Phoebe Cottingham, 315–46. Kalamazoo, MI: Upjohn Institute.
  33. ↵
    1. Dehejia, Rajeev
    , and Sadek Wahba. 1999. “Causal Effects in Nonexperimental Studies: Reevaluating the Evaluations of Training Programs.” Journal of the American Statistical Association 94(448):1053–62.
    OpenUrlCrossRefWeb of Science
  34. ↵
    1. Dehejia, Rajeev
    , and Sadek Wahba. 2002. “Propensity Score Matching Methods for Nonexperimental Causal Studies.” Review of Economics and Statistics 84(1):151–61.
    OpenUrlCrossRefWeb of Science
  35. ↵
    1. Dolton, Peter
    , and Jeffrey Smith. 2011. “The Impact of the UK New Deal for Lone Parents on Benefit Receipt.” IZA Discussion Paper 5491. Bonn, Germany: IZA.
  36. ↵
    1. Fortson, Kenneth
    , Dana Rotz, Paul Burkander, Annalisa Mastri, Peter Schochet, Linda Rosenberg, Sheena McConnell, and Ronald D’Amico. 2017. Providing Public Workforce Services to Job Seekers: 30‐Month Impact Findings on the WIA Adult and Dislocated Worker Programs. Princeton, NJ: Mathematica Policy Research.
  37. ↵
    1. Friedlander, Daniel
    , David Greenberg, and Philip Robins. 1997. “Evaluating Government Training Programs for the Economically Disadvantaged.” Journal of Economic Literature 35(4):1809–55.
    OpenUrlWeb of Science
  38. ↵
    1. Friedlander, Daniel
    , and Philip Robins. 1995. “Evaluating Program Evaluations: New Evidence on Commonly Used Nonexperimental Methods.” American Economic Review 85(4):923–37.
    OpenUrlWeb of Science
  39. ↵
    1. Greenberg, David
    , Charles Michalopoulos, and Philip Robins. 2003. “A Meta‐Analysis of Government‐Sponsored Training Programs.” Industrial and Labor Relations Review 57(1):31–53.
    OpenUrlCrossRef
  40. ↵
    1. Greenberg, David
    , Charles Michalopoulos, and Philip Robins. 2004. “What Happens to the Effects of Government‐Funded Training Programs Over Time?” Journal of Human Resources 39(1):277–93.
    OpenUrlAbstract/FREE Full Text
  41. ↵
    1. Greenberg, David
    , Charles Michalopoulos, and Philip Robins. 2006. “Do Experimental and Nonexperimental Evaluations Given Different Answers about the Effectiveness of Government‐Funded Training Programs?” Journal of Policy Analysis and Management 25(3):523–52.
    OpenUrlCrossRefWeb of Science
  42. ↵
    1. Heckman, James
    , Carolyn Heinrich, Pascal Courty, Gerald Marschke, and Jeffrey Smith. 2011. The Performance of Performance Standards. Kalamazoo, MI: W.E. Upjohn Institute for Employment Research.
  43. ↵
    1. Heckman, James
    , Hidehiko Ichimura, Jeffrey Smith, and Petra Todd. 1998. “Characterizing Selection Bias Using Experimental Data.” Econometrica 66(5):1017–98.
    OpenUrlCrossRefWeb of Science
  44. ↵
    1. Heckman, James
    , Hidehiko Ichimura, and Petra Todd. 1997. “Matching as an Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme.” Review of Economic Studies 64(4):605–54.
    OpenUrlCrossRefWeb of Science
  45. ↵
    1. Heckman, James
    , Robert LaLonde, and Jeffrey Smith. 1999. “The Economics and Econometrics of Active Labor Market Programs” In Handbook of Labor Economics, Volume 3, Part A, ed. Orley Ashenfelter and David Card, 1865–2097. New York: Elsevier.
    OpenUrlCrossRef
  46. ↵
    1. Heckman, James
    , and Salvador Navarro‐Lozano. 2004. “Using Matching, Instrumental Variables and Control Functions to Estimate Economic Choice Models.” Review of Economics and Statistics 86(1):30–57.
    OpenUrlCrossRefWeb of Science
  47. ↵
    1. Heckman, James
    , and Jeffrey Smith. 1995. “Assessing the Case for Social Experiments.” Journal of Economic Perspectives 9(2):85–110.
    OpenUrlCrossRefWeb of Science
  48. ↵
    1. Heckman, James
    , and Jeffrey Smith. 1999. “The Pre‐Programme Earnings Dip and the Determinants of Participation in a Social Programme: Implications for Simple Programme Evaluation Strategies.” Economic Journal 109(457):313–48.
    OpenUrl
  49. ↵
    1. Heinberg, John
    , John Trutko, Burt Barnow, Mary Farrell, and Asaph Glosser. 2005. “Unit Costs of Intensive and Training Services for WIA Adults and Dislocated Workers: An Exploratory Study of Methodologies and Estimates in Selected States and Localities: Final Report.” Washington, DC: Capitol Research Corporation.
  50. ↵
    1. Heinrich, Carolyn
    , and Christopher King. 2010. “How Effective Are Workforce Development Programs?” Presented at the 40th Anniversary Conference of the Ray Marshall Center, University of Texas, Austin TX, October.
  51. ↵
    1. Heinrich, Carolyn
    , and Peter Mueser. 2014. “Training Program Impacts and the Onset of the Great Recession.” Unpublished. Nashville, TN: Vanderbilt University.
  52. ↵
    1. Heinrich, Carolyn
    , Peter Mueser, Kenneth Troske, Kyung‐Seong Jeon, and Daver Kahvecioglu. 2011. “A Nonexperimental Evaluation of WIA Programs.” In The Workforce Investment Act: Implementation Experiences and Evaluation Findings, ed. Douglas Besharov and Phoebe Cottingham, 371–406. Kalamazoo, MI: Upjohn Institute.
  53. ↵
    1. Heinrich, Carolyn
    , Peter Mueser, Kenneth Troske, Kyung‐Seong Jeon, and Daver Kahvecioglu. 2013. “Do Public Employment and Training Programs Work?” IZA Journal of Labor Economics 2:6.
    OpenUrl
  54. ↵
    1. Hollenbeck, Kevin
    . 2009. “Return on Investment Analysis of a Selected Set of Workforce System Programs in Indiana.” Report submitted to the Indiana Chamber of Commerce Foundation, Indianapolis, IN. http://research.upjohn.org/reports/15 (accessed January 4, 2024).
  55. ↵
    1. Hollenbeck, Kevin
    . 2012. “Return on Investment in Workforce Development Programs.” Upjohn Institute Working Paper 12‐188. Kalamazoo, MI: W.E. Upjohn Institute for Employment Research. http://research.upjohn.org/up_workingpapers/188 (accessed January 4, 2024).
  56. ↵
    1. Holzer, Harry
    . 2009. “Workforce Development Policy as an Antipoverty Tool: What Do We Know? What Can We Do?” In Changing Poverty, Changing Policy, ed. Maria Cancian and Sheldon Danziger, 301–29. New York: Russell Sage Foundation.
  57. ↵
    1. Holzer, Harry
    , Julia Lane, David Rosenblum, and Fredrik Andersson. 2011. Where Are All the Good Jobs Going? What National and Local Job Quality and Dynamics Mean for U.S. Workers. New York: Russell Sage Foundation.
  58. ↵
    1. Holzer, Harry
    , Julia Lane, and Lars Vilhuber. 2004. “Escaping Poverty for Low‐Wage Workers: The Role of Employer Characteristics and Changes.” Industrial and Labor Relations Review 57(4):560–78.
    OpenUrlCrossRef
  59. ↵
    1. Imbens, Guido
    , and Jeffrey Wooldridge. 2009. “Recent Developments in the Econometrics of Program Evaluation.” Journal of Economic Literature 47(1):5–86.
    OpenUrlCrossRefWeb of Science
  60. ↵
    1. Jacobson, Lou
    , Robert Lalonde, and Daniel Sullivan. 2005. “The Impact of Community College Retraining on Older Displaced Workers: Should We Teach Old Dogs New Tricks?” Industrial and Labor Relations Review 58(3):398–415.
    OpenUrlCrossRef
  61. ↵
    1. Katz, Lawrence
    , Jonathan Roth, Richard Hendra, and Kelsey Schaberg. 2020. “Why Do Sectoral Employment Programs Work? Lessons from WorkAdvance.” NBER Working Paper 28248. Cambridge, MA: NBER.
  62. ↵
    1. King, Christopher
    . 2004. “The Effectiveness of Publicly Financed Training in the United States.” In Job Training Policy in the United States, ed. Christopher O’Leary, Robert Straits, and Stephen Wandner, 57–100. Kalamazoo, MI: Upjohn Institute.
  63. ↵
    1. King, Christopher
    , and Burt Barnow. 2011. “The Use of Market Mechanisms.” In The Workforce Investment Act: Implementation Experiences and Evaluation Findings, ed. Douglas Besharov and Phoebe Cottingham, 81–112. Kalamazoo, MI: Upjohn Institute.
  64. ↵
    1. Lalonde, Robert
    . 1995. “The Promise of Public‐Sector Training.” Journal of Economic Perspectives 9(2):149–68.
    OpenUrlCrossRefWeb of Science
  65. ↵
    1. Lalonde, Robert
    . 1986. “Evaluating the Econometric Evaluations of Training Programs with Experimental Data.” American Economic Review 76(4):604–20.
    OpenUrlWeb of Science
  66. ↵
    1. Lechner, Michael
    . 2010. “The Estimation of Causal Effects by Difference‐in‐Difference Methods.” Foundations and Trends in Econometrics 4(3):165–224.
    OpenUrlCrossRef
  67. ↵
    1. Lechner, Michael
    , and Conny Wunsch. 2009. “Are Training Programs More Effective When Unemployment Is High?” Journal of Labor Economics 27(4):653–92.
    OpenUrlCrossRef
  68. ↵
    1. Lechner, Michael
    , and Conny Wunsch. 2013. “Sensitivity of Matching‐Based Program Evaluations to the Availability of Control Variables.” Labour Economics 21:111–21.
    OpenUrl
  69. ↵
    1. Leung, Pauline
    , and Zhuan Pei. 2020. “Further Education during Unemployment.” Industrial Relations Section Working Paper 642. Princeton, NJ: Princeton University.
  70. ↵
    1. Lise, Jeremy
    , Shannon Seitz, and Jeffrey Smith. 2004. “Equilibrium Policy Experiments and the Evaluation of Social Programs.” NBER Working Paper 10283. Cambridge, MA: NBER.
  71. ↵
    1. Mastri, Annalisa
    , and AnnaMaria McCutcheon. 2015. “Costs of Services Provided by the WIA Adult and Dislocated Worker Programs.” Issue Brief. Oakland, CA: Mathematica Policy Research and Social Policy Research Associates.
  72. ↵
    1. McCall, Brian
    , Jeffrey Smith, and Conny Wunsch. 2016. “Government‐Sponsored Vocational Training for Adults.” In Handbook of the Economics of Education, Volume 5, ed. Eric Hanushek, Stephen Machin, and Ludger Woessman, 479–652. Amsterdam: North‐Holland.
    OpenUrl
  73. ↵
    1. Morris, Pamela
    , and Charles Michalopoulos. 2003. “Findings from the Self‐Sufficiency Project: Effects on Children and Adolescents of a Program that Increased Employment and Earnings.” Journal of Applied Developmental Psychology 24(2):201–39.
    OpenUrlCrossRefWeb of Science
  74. ↵
    1. Oster, Emily
    . 2019. “Unobservable Selection and Coefficient Stability: Theory and Evidence.” Journal of Business & Economic Statistics 37(2):187–204.
    OpenUrl
  75. ↵
    1. Perez‐Johnson, Irma
    , Quinn Moore, and Robert Santilano. 2011. “Improving the Effectiveness of Individual Training Accounts: Long‐Term Findings from an Experimental Evaluation of Three Service Delivery Models: Final Report.” Princeton, NJ: Mathematica Policy Research.
  76. ↵
    1. Roder, Anne
    , and Mark Elliott. 2019. “Nine‐Year Gains: Project Quest’s Continuing Impact.” New York: Economic Mobility Corporation.
  77. ↵
    1. Rosenbaum, Paul
    . 2001. “Stability in the Absence of Treatment.” Journal of the American Statistical Association 96:210–19.
    OpenUrlCrossRefWeb of Science
  78. ↵
    1. Rosenbaum, Paul
    , and Donald Rubin. 1983. “The Central Role of the Propensity Score in Observational Studies for Causal Effects.” Biometrika 70(1):41–55.
    OpenUrlCrossRefWeb of Science
  79. ↵
    1. Rothstein, Jesse
    , and Till von Wachter. 2017. “Social Experiments in Labor Economics.” In Handbook of Field Experiments, Volume 2, ed. Abhijit Banerjee and Esther Duflo, 555–637. New York: Elsevier.
    OpenUrl
  80. ↵
    1. Schochet, Peter
    , John Burghardt, and Sheena McConnell. 2006. “National Job Corps Study and Longer‐Term Follow‐Up Study: Impact and Benefit‐Cost Findings Using Survey and Summary Earnings Records Data, Final Report.” Washington, DC: Mathematica Policy Research.
  81. ↵
    1. Schochet, Peter
    , John Burghardt, and Sheena McConnell. 2008. “Does Job Corps Work? Impact Findings from the National Job Corps Study.” American Economic Review 98(5):1864–86.
    OpenUrlCrossRefWeb of Science
  82. ↵
    1. Smith, Jeffrey
    . 2011. “Improving Impact Evaluation in Europe.” In The Workforce Investment Act: Implementation Experiences and Evaluation Findings, ed. Douglas Besharov and Phoebe Cottingham, 473–94. Kalamazoo, MI: W.E. Upjohn Institute for Employment Research.
  83. ↵
    1. Smith, Jeffrey
    , and Petra Todd. 2005. “Does Matching Overcome LaLonde’s Critique of Nonexperimental Methods?” Journal of Econometrics 125(1–2):305–53.
    OpenUrlCrossRefWeb of Science
  84. ↵
    Social Policy Research Associates. 2005. PY 2003 WIASRD Data Book. Oakland, CA: Social Policy Research Associates. https://www.dol.gov/sites/dolgov/files/ETA/Performance/pdfs/WIASRD/PY03_WIASRD_Data_Book.pdf (accessed January 19, 2024).
  85. ↵
    1. Stevens, David
    . 2007. “Employment That Is Not Covered by State Unemployment Insurance Laws.” LEHD Technical Paper TP‐2007‐04. Washington, DC: U.S. Census Bureau.
  86. ↵
    U.S. Office of Management and Budget. 1992. “Guidelines and Discount Rates for Benefit–Cost Analysis of Federal Programs.” Circular A‐94 Revised. https://obamawhitehouse.archives.gov/sites/default/files/omb/assets/a94/a094.pdf (accessed January 4, 2024).
  87. ↵
    1. Van Horn, Carl
    , and Aaron Fichtner. 2011. “Eligible Training Provider Lists and Consumer Report Cards.” In The Workforce Investment Act: Implementation Experiences and Evaluation Findings, ed. Douglas Besharov and Phoebe Cottingham, 153–76. Kalamazoo, MI: Upjohn Institute.
Back to top

In this issue

Print
Download PDF
Article Alerts
Email Article
Citation Tools
Does Federally Funded Job Training Work?
Fredrik Andersson, Harry J. Holzer, Julia I. Lane, David Rosenblum, Jeffrey Smith
Journal of Human Resources Jul 2024, 59 (4) 1244-1283; DOI: 10.3368/jhr.0816-8185R1
Twitter logo Facebook logo Mendeley logo
Bookmark this article

Jump to section

Keywords