Research ArticleArticles
Open Access

Errors in Survey Reporting and Imputation and Their Effects on Estimates of Food Stamp Program Participation

Bruce D. Meyer, Nikolas Mittag and Robert M. Goerge
Journal of Human Resources, September 2022, 57 (5) 1605-1644; DOI: https://doi.org/10.3368/jhr.58.1.0818-9704R2

References

  1. ↵
    1. Abowd, John M., and
    2. Martha H. Stinson
    . 2013. “Estimating Measurement Error in Annual Job Earnings: A Comparison of Survey and Administrative Data.” Review of Economics and Statistics 95(5):1451–67.
    OpenUrlCrossRef
  2. ↵
    1. Almada, Lorenzo,
    2. Ian McCarthy, and
    3. Rusty Tchernis
    . 2016. “What Can We Learn about the Effects of Food Stamps on Obesity in the Presence of Misreporting?” American Journal of Agricultural Economics 98(4):997–1017.
    OpenUrlCrossRef
  3. ↵
    1. Almond, Douglas,
    2. Hilary W. Hoynes, and
    3. Diane Whitmore Schanzenbach
    . 2011. “Inside the War on Poverty: The Impact of Food Stamps on Birth Outcomes.” Review of Economics and Statistics 93(2):387–403.
    OpenUrlCrossRefWeb of Science
  4. ↵
    1. Bitler, Marianne P.,
    2. Janet Currie, and
    3. John-Karl Scholz
    . 2003. “WIC Eligibility and Participation.” Journal of Human Resources 38(S):1139–79.
    OpenUrlCrossRefWeb of Science
  5. ↵
    1. Blank, Rebecca M., and
    2. Patricia Ruggles
    . 1996. “When Do Women Use AFDC & Food Stamps? The Dynamics of Eligibility vs. Participation.” Journal of Human Resources 31(1):57–89.
    OpenUrlCrossRefWeb of Science
  6. ↵
    1. Blundell, Richard, and
    2. Luigi Pistaferri
    . 2003. “Income Volatility and Household Consumption: The Impact of Food Assistance Programs.” Journal of Human Resources 38(S):1032–50.
    OpenUrlCrossRefWeb of Science
  7. ↵
    1. Bollinger, Christopher R., and
    2. Martin H. David
    . 1997. “Modeling Discrete Choice with Response Error: Food Stamp Participation.” Journal of the American Statistical Association 92(439): 827–35.
    OpenUrlCrossRefWeb of Science
  8. ↵
    1. Bollinger, Christopher R., and
    2. Martin H. David
    . 2001. “Estimation with Response Error and Nonresponse: Food-Stamp Participation in the SIPP.” Journal of Business and Economic Statistics 19(2):129–41.
    OpenUrlCrossRefWeb of Science
  9. ↵
    1. Bollinger, Christopher R., and
    2. Martin H. David
    . 2005. “I Didn’t Tell, and I Won’t Tell: Dynamic Response Error in the SIPP.” Journal of Applied Econometrics 20(4):563–69.
    OpenUrl
  10. ↵
    1. Bound, John,
    2. Charles Brown, and
    3. Nancy Mathiowetz
    . 2001. “Measurement Error in Survey Data.” In Handbook of Econometrics, Volume 5, ed. J.J Heckman and E. Leamer, 3705–843. Amsterdam: Elsevier.
    OpenUrl
  11. ↵
    1. Celhay, Pablo,
    2. Bruce D. Meyer, and
    3. Nikolas Mittag
    . 2022. “What Leads to Measurement Error? Evidence from Reports of Program Participation in Three Surveys.” NBER Working Paper 29652.
  12. ↵
    1. Celhay, Pablo,
    2. Bruce D. Meyer, and
    3. Nikolas Mittag
    . 2021. “Errors in Reporting and Imputation of Government Benefits and Their Implications.” NBER Working Paper 29184. Cambridge, MA: NBER.
  13. ↵
    1. Cerf Harris, Ben.
    2014. “Within and across County Variation in SNAP Misreporting: Evidence from Linked ACS and Administrative Records.” U.S. Census Bureau CARRA Working Paper 2014-05. Washington, DC: U.S. Census Bureau.
  14. ↵
    1. Coder, John, and
    2. Lydia Scoon-Rogers
    . 1996. “Evaluating the Quality of Income Data Collected in the Annual Supplement to the March Current Population Survey and the Survey of Income and Program Participation.” Working Paper. Washington, DC: U.S. Census Bureau Housing and Household Economic Statistics Division.
  15. ↵
    1. Courtemanche, Charles,
    2. Augustine Denteh, and
    3. Rusty Tchernis
    . 2019. “Estimating the Associations between SNAP and Food Insecurity, Obesity, and Food Purchases with Imperfect Administrative Measures of Participation.” Southern Economic Journal 86(1):202–28.
    OpenUrl
  16. ↵
    1. Currie, Janet.
    2006. “The Take-up of Social Benefits.” In Public Policy and the Income Distribution, ed. Alan J. Auterbach, David Card, and John M. Quigley, 80–148. New York: Russell Sage Foundation.
  17. ↵
    1. Currie, Janet, and
    2. Jeffrey Grogger
    . 2001. “Explaining Recent Declines in Food Stamp Program Participation.” Brookings-Wharton Papers on Urban Affairs 2001:203–29.
    OpenUrlCrossRef
  18. ↵
    1. Davern, Michael,
    2. Bruce D. Meyer, and
    3. Nikolas Mittag
    . 2019. “Creating Improved Survey Data Products Using Linked Administrative-Survey Data.” Journal of Survey Statistics and Methodology 7(3):440–63.
    OpenUrl
  19. ↵
    1. Figlio, David,
    2. Craig Gundersen, and
    3. James Ziliak
    . 2000. “The Effects of the Macro-Economy and Welfare Reform on Food Stamp Caseloads.” American Journal of Agricultural Economics 82(3):635–41.
    OpenUrlCrossRef
  20. ↵
    1. Ganong, Peter, and
    2. Jeffrey B. Liebman
    . 2018. “The Decline, Rebound, and Further Rise in SNAP Enrollment: Disentangling Business Cycle Fluctuations and Policy Changes.” American Economic Journal: Economic Policy 10(4):153–76.
    OpenUrl
  21. ↵
    1. Groves, Robert M.
    1989. Survey Errors and Survey Costs. New York: John Wiley & Sons.
  22. ↵
    1. Gundersen, Craig, and
    2. Brent Kreider
    . 2008. “Food Stamps and Food Insecurity What Can Be Learned in the Presence of Nonclassical Measurement Error?” Journal of Human Resources 43(2):352–82.
    OpenUrlAbstract/FREE Full Text
  23. ↵
    1. Gundersen, Craig, and
    2. Brent Kreider
    . 2009. “Bounding the Effects of Food Insecurity on Children’s Health Outcomes.” Journal of Health Economics 28(5):971–83.
    OpenUrlCrossRefPubMedWeb of Science
  24. ↵
    1. Gundersen, Craig,
    2. Brent Kreider, and
    3. John V. Pepper
    . 2012. “The Impact of the National School Lunch Program on Child Health: A Nonparametric Bounds Analysis.” Journal of Econometrics 166(1):79–91.
    OpenUrlCrossRefWeb of Science
  25. ↵
    1. Gundersen, Craig, and
    2. Victor Oliveira
    . 2001. “The Food Stamp Program and Food Insufficiency.” American Journal of Agricultural Economics 83(4):875–87.
    OpenUrlCrossRefWeb of Science
  26. ↵
    1. Gundersen, Craig, and
    2. James P. Ziliak
    . 2003. “The Role of Food Stamps in Consumption Stabilization.” Journal of Human Resources 38(S):1051–79.
    OpenUrlCrossRefWeb of Science
  27. ↵
    1. Gundersen, Craig, and
    2. James P. Ziliak
    . 2008. “The Age Gradient in Food Stamp Program Participation: Does Income Volatility Matter?” In Income Volatility and Food Assistance in the United States, ed. Dean Jolliffe and J.P. Ziliak, 171–214. Kalamazoo, MI: Upjohn.
  28. ↵
    1. Haider, Steven,
    2. Alison Jacknowitz, and
    3. Robert Schoeni
    . 2003. “Food Stamps and the Elderly: Why Is Participation so Low?” Journal of Human Resources 38(S):1180–220.
    OpenUrlCrossRefWeb of Science
  29. ↵
    1. Hausman, Jerry A.,
    2. Jason Abrevaya, and
    3. Fiona M. Scott-Morton
    . 1998. “Misclassification of the Dependent Variable in a Discrete-Response Setting.” Journal of Econometrics 87(2):239–69.
    OpenUrlCrossRefWeb of Science
  30. ↵
    1. Hoynes, Hilary W., and
    2. Diane Whitmore Schanzenbach
    . 2009. “Consumption Responses to In-Kind Transfers: Evidence from the Introduction of the Food Stamp Program.” American Economic Journal: Applied Economics 1(4):109–39.
    OpenUrlCrossRefWeb of Science
  31. ↵
    1. Jensen, Helen H.,
    2. Brent Kreider, and
    3. Oleksandr Zhylyevskyy
    . 2019. “Investigating Treatment Effects of Participating Jointly in SNAP and WIC When the Treatment Is Validated Only for SNAP.” Southern Economic Journal 86(1):124–55.
    OpenUrl
  32. ↵
    1. Kang, Kyung M., and
    2. Robert A. Moffitt
    . 2019. “The Effect of SNAP and School Food Programs on Food Security, Diet Quality, and Food Spending: Sensitivity to Program Reporting Error.” Southern Economic Journal 86(1):156–201.
    OpenUrl
  33. ↵
    1. Kapteyn, Arie, and
    2. Jelmer Y. Ypma
    . 2007. “Measurement Error and Misclassification: A Comparison of Survey and Administrative Data.” Journal of Labor Economics 25(3):513–51.
    OpenUrlCrossRefWeb of Science
  34. ↵
    1. Kirlin, John A., and
    2. Michael Wiseman
    . 2014. “Getting it Right, or at Least Better: Improving Identification of Food Stamp Participants in the National Health and Nutrition Examination Survey.” Unpublished.
  35. ↵
    1. Kreider, Brent.
    2010. “Identification Decay of Regression Coefficients in the Presence of Infrequent Classification Errors.” Review of Economics and Statistics 92(4):1017–23
    OpenUrlCrossRef
  36. ↵
    1. Kreider, Brent,
    2. John V. Pepper,
    3. Craig Gundersen, and
    4. Dean Jolliffe
    . 2012. “Identifying the Effects of SNAP (Food Stamps) on Child Health Outcomes When Participation Is Endogenous and Misreported.” Journal of the American Statistical Association 107(499):958–75.
    OpenUrlCrossRef
  37. ↵
    1. Marquis, Kent H.,
    2. Naihua Duan,
    3. M. Susan Marquis,
    4. J. Michael Polich,
    5. J.E. Meshkoff,
    6. Donna S. Schwarzbach, and
    7. Cathy Stasz
    . 1981. “Response Errors in Sensitive Topic Surveys: Estimates, Effects, and Correction Options.” Santa Monica, CA: RAND Corporation.
  38. ↵
    1. Marquis, Kent H., and
    2. Jeffrey C. Moore
    . 1990. “Measurement Errors in SIPP Program Reports.” SIPP Working Paper. Washington, DC: U.S. Bureau of the Census.
  39. ↵
    1. Meijer, Erik,
    2. Susann Rohwedder, and
    3. Tom Wansbeek
    . 2012. “Measurement Error in Earnings Data: Using a Mixture Model Approach to Combine Survey and Register Data.” Journal of Business & Economic Statistics 30(2):191–201.
    OpenUrl
  40. ↵
    1. Meyer, Bruce D.
    2010. “The Effects of the Earned Income Tax Credit and Recent Reforms.” In Tax Policy and the Economy, Volume 24, ed. Jeffrey Brown, 153–80. Cambridge, MA: MIT Press.
    OpenUrl
  41. ↵
    1. Meyer, Bruce D., and
    2. Nikolas Mittag
    . 2017. “Misclassification in Binary Choice Models.” Journal of Econometrics 200(2):295–311.
    OpenUrl
  42. ↵
    1. Meyer, Bruce D., and
    2. Nikolas Mittag
    . 2019a. “Using Linked Survey and Administrative Data to Better Measure Income: Implications for Poverty, Program Effectiveness and Holes in the Safety Net.” American Economic Journal: Applied Economics 11(2):176–204.
    OpenUrl
  43. ↵
    1. Meyer, Bruce D., and
    2. Nikolas Mittag
    . 2019b. “Misreporting of Government Transfers: How Important Are Survey Design and Geography?” Southern Economic Journal 86(1):230–53.
    OpenUrl
  44. ↵
    1. Meyer, Bruce D., and
    2. Nikolas Mittag
    . 2021a. “An Empirical Total Survey Error Decomposition.” Journal of Econometrics 224(2):286–305.
    OpenUrl
  45. ↵
    1. Meyer, Bruce D., and
    2. Nikolas Mittag
    . 2021b. “Combining Administrative and Survey Data to Improve Income Measurement.” In Administrative Records for Survey Methodology, ed. Asap Y. Chun, Michael D. Larsen, Gabriela Durrant, and Jerome P. Reiter, 297–322. New York: John Wiley and Sons.
  46. ↵
    1. Meyer, Bruce D.,
    2. Wallace K.C. Mok, and
    3. James X. Sullivan
    . 2015a. “The Underreporting of Transfers in Household Surveys: Its Nature and Consequences.” Working Paper. Chicago, IL: Harris School of Public Policy Studies, University of Chicago
  47. ↵
    1. Meyer, Bruce D.,
    2. Wallace K.C. Mok, and
    3. James X. Sullivan
    . 2015b. “Household Surveys in Crisis.” Journal of Economic Perspectives 28(4):199–226.
    OpenUrl
  48. ↵
    1. Millimet, Daniel L.
    2011. “The Elephant in the Corner: A Cautionary Tale about Measurement Error in Treatment Effects Models.” In Missing Data Methods: Cross-Sectional Methods and Applications, Advances in Econometrics, Volume 27, Part 1, ed. D. Drukker, 1–39. Bingley, UK: Emerald Publishing.
    OpenUrl
  49. ↵
    1. Mittag, Nikolas.
    2019. “Correcting for Misreporting of Government Benefits” American Economic Journal: Economic Policy 11(2):142–64.
    OpenUrl
  50. ↵
    1. Moore, Jeffrey C.
    2008. “Seam Bias in the 2004 SIPP Panel: Much Improved, but Much Bias Still Remains.” U.S. Census Bureau Statistical Research Division Survey Methodology Research Report Series 2008-3. Washington, DC: U.S. Census Bureau.
  51. ↵
    1. Moore, Jeffrey C.,
    2. Kent H. Marquis, and
    3. Karen Bogen
    . 1996. “The SIPP Cognitive Research Evaluation Experiment: Basic Results and Documentation.” SIPP Working Paper 212. Washington, DC: U.S. Census Bureau.
  52. ↵
    1. Moore, Jeffrey C.,
    2. Linda L. Stinson, and
    3. Edward J. Welniak, Jr.
    2000. “Income Measurement Error in Surveys: A Review.” Journal of Official Statistics 14(4):331–61.
    OpenUrl
  53. ↵
    1. Mulligan, Casey.
    2012. The Redistribution Recession: How Labor Market Distortions Contracted the Economy. Oxford, UK: Oxford University Press.
  54. ↵
    1. Nguimkeu, Pierre,
    2. Augustine Denteh, and
    3. Rusty Tchernis
    . 2019. “On the Estimation of Treatment Effects with Endogenous Misreporting.” Journal of Econometrics 208(2):487–506.
    OpenUrl
  55. ↵
    1. Nicholas, Joyce, and
    2. Michael Wiseman
    . 2009. “Elderly Poverty and Supplemental Security Income.” Social Security Bulletin 69(1):45–73.
    OpenUrlPubMed
  56. ↵
    1. Nicholas, Joyce, and
    2. Michael Wiseman
    . 2010. “Elderly Poverty and Supplemental Security Income 2002–2005.” Social Security Bulletin 70(2):1–29.
    OpenUrlPubMed
  57. ↵
    1. Niehaus, Paul, and
    2. Sandip Sukhtankar
    . 2013. “Corruption Dynamics: The Golden Goose Effect.” American Economic Journal: Economic Policy 5(4):230–69.
    OpenUrlCrossRef
  58. ↵
    NORC. 2011. “Assessment of the US Census Bureau’s Person Identification Validation System.” NORC Final Report presented to the U.S. Census Bureau.
  59. ↵
    1. Oberski, Daniel L.,
    2. Antje Kirchner,
    3. Stephanie Eckman, and
    4. Frauke Kreuter
    . 2017. “Evaluating the Quality of Survey and Administrative Data with Generalized Multitrait-Multimethod Models.” Journal of the American Statistical Association 112(520):1477–89.
    OpenUrl
  60. ↵
    1. Pierret, Charles R.
    2001. “Event History Data and Survey Recall: An Analysis of the National Longitudinal Survey of Youth 1979 Recall Experiment.” Journal of Human Resources 36(3): 439–66.
    OpenUrlCrossRefWeb of Science
  61. ↵
    1. Remler, Dahlia K., and
    2. Sherry A. Glied
    . 2003. “What Other Programs Can Teach Us: Increasing Participation in Health Insurance Programs.” American Journal of Public Health 93(1):67–74.
    OpenUrlCrossRefPubMedWeb of Science
  62. ↵
    1. Roemer, Marc I.
    2000. “Assessing the Quality of the March Current Population Survey and the Survey of Income and Program Participation Income Estimates 1990–1996.” Staff Paper. Washington, DC: U.S. Census Bureau Household Economic Statistics Division.
  63. ↵
    1. Rothbaum, Jonathan L.
    2015. “Comparing Income Aggregates: How Do the CPS and ACS Match the National Income and Product Accounts 2007–2012.” SEHSD Working Paper 2015-01. Washington, DC: U.S. Census Bureau.
  64. ↵
    1. Schmidt, Lucie,
    2. Lara Shore-Sheppard, and
    3. Tara Watson
    . 2016. “The Effect of Safety-Net Programs on Food Insecurity.” Journal of Human Resources 51(3):589–614.
    OpenUrlAbstract/FREE Full Text
  65. ↵
    1. Scholz, John K.,
    2. Robert A. Moffitt, and
    3. Benjamin Cowan
    . 2009. “Trends in Income Support.” In Changing Poverty, Changing Policies, ed. M. Cancian and S. Danziger, 1–32. Washington, DC: Russell Sage Foundation.
  66. ↵
    1. Sudman, Seymour, and
    2. Norman M. Bradburn
    . 1974. Response Effects in Surveys. Chicago: Aldine.
  67. ↵
    1. Sudman, Seymour,
    2. Norman M. Bradburn, and
    3. Norbert Schwarz
    . 1996. Thinking about Answers: The Application of Cognitive Processes to Survey Methodology. San Francisco: Josey-Bass.
  68. ↵
    1. Taeuber, Cynthia,
    2. Dean M. Resnick,
    3. Susan P. Love,
    4. Jane Stavely,
    5. Parke Wilde, and
    6. Richard Larson
    . 2004. “Differences in Estimates of Food Stamp Program Participation between Surveys and Administrative Records.” Working Paper. Washington, DC: U.S. Census Bureau.
  69. ↵
    1. Tiehen, Laura,
    2. Dean Jolliffe, and
    3. Craig Gundersen
    . 2012. “Alleviating Poverty in the United States: The Critical Role of SNAP Benefits.” ERR-132. Washington, DC: U.S. Department of Agriculture, Economic Research Service.
  70. ↵
    U.S. General Accounting Office. 2004. “Food Stamp Program: Steps Have Been Taken to Increase Participation of Working Families, but Better Tracking of Efforts Is Needed.” GAO-04-346. Washington, DC: GAO.
  71. ↵
    1. Wagner, Deborah, and
    2. Mary Layne
    . 2014. “The Person Identification Validation System (PVS): Applying the Center for Administrative Records Research and Applications’ (CARRA) Record Linkage Software.” Washington, DC: U.S. Census Bureau.
  72. ↵
    1. Wheaton, Laura.
    2007. “Underreporting of Means-Tested Transfer Programs in the CPS and SIPP.” In 2007 Proceedings of the American Statistical Association, Social Statistics Section. Alexandria, VA: American Statistical Association
  73. ↵
    1. Wooldridge, Jeffrey M.
    2007. “Inverse Probability Weighted Estimation for General Missing Data Problems.” Journal of Econometrics 141(2):1281–301.
    OpenUrlCrossRefWeb of Science
  74. ↵
    1. Wu, Yanyuan.
    2010. “Essays on the Economic Well-Being of the Elderly and Public Policy.” Ph.D. Dissertation. Chicago: University of Chicago.
  75. ↵
    1. Ziliak, James,
    2. Craig Gundersen, and
    3. David Figlio
    . 2003. “Food Stamp Caseloads over the Business Cycle.” Southern Economic Journal 69(4):903–19.
    OpenUrlCrossRefWeb of Science
Back to top

In this issue

Print
Download PDF
Article Alerts
Email Article
Citation Tools
Errors in Survey Reporting and Imputation and Their Effects on Estimates of Food Stamp Program Participation
Bruce D. Meyer, Nikolas Mittag, Robert M. Goerge
Journal of Human Resources Sep 2022, 57 (5) 1605-1644; DOI: 10.3368/jhr.58.1.0818-9704R2
Twitter logo Facebook logo Mendeley logo
Bookmark this article

Jump to section

Keywords