Abstract
We study how the societal disruptions of the COVID-19 pandemic impacted diagnosis of a prevalent childhood mental health condition, attention deficit hyperactivity disorder (ADHD). Using both nationwide private health insurance claims and a single state’s comprehensive electronic health records, we compare children exposed to the pandemic to same-aged children prior to the pandemic. We find the pandemic reduced new ADHD diagnoses by 8.6 percent among boys and 11.0 percent among girls nationwide through February 2021. We further show that higher levels of in-person schooling in fall 2020 dampened the decline for girls but had no moderating effect for boys.
I. Introduction
Identifying factors that impact the development and subsequent diagnosis and treatment of child mental and behavioral health conditions is crucial for understanding child well-being. The implications of such factors are important because they affect not only child development, but also other outcomes related to education, risky behaviors, crime, and long-run economic variables (see Currie 2020 for a recent review and discussion). In practice, diagnosing children with mental or behavioral health conditions involves interactions between a diverse and decentralized collection of people, including parents, teachers, and physicians. Understanding if and how the COVID-19 pandemic disrupted these interactions and/or influenced underlying mental and behavioral health needs among children is essential for efforts to improve child well-being in a post-pandemic world.
We examine how child mental health diagnosis changed during the COVID-19 pandemic, with a focus on diagnosis of initial cases of a common child mental health condition, attention deficit hyperactivity disorder (ADHD). We use a nationwide private health insurance claims database and a state-specific electronic health records database to analyze how the flow of new ADHD diagnoses was impacted over the first year of the COVID-19 pandemic. Our nationwide analysis uses Optum’s de-identified Clinformatics® Data Mart Database of comprehensive commercial claims with nationwide coverage. Our state-specific analysis uses the Indiana Network for Patient Care (INPC) research database. These two data sets are complementary, with one providing nationwide coverage from a large private insurer and the other comprising all payers in a single state with finer geographic detail.
In both data sets, our research design compares two cohorts of elementary school children: one from February 2019–February 2021 that was affected by the pandemic beginning in March 2020 and a previous cohort from February 2018–February 2020 that was not affected by the pandemic. We compare changes in new diagnosis rates between these two cohorts, where members of the pre-pandemic cohort serve as a control group for members of the pandemic cohort at the same age. In other words, we compare how diagnosis rates evolved over a 19-month period for children of the same age but at different points in calendar time. We estimate both event study and difference-in-differences models, and we conduct each analysis separately by gender given evidence that ADHD presents differently in boys and girls.
We find a substantial decline in new ADHD diagnosis starting in March 2020 and extending into early 2021. Overall, our estimates imply that new ADHD diagnosis fell by 8.6 percent among boys and by 11.0 percent among girls in our nationwide analysis. The fall in diagnosis was concentrated among white and Hispanic children, with a negligible decrease among Black children. In Indiana, we find a higher effect among boys (18 percent decrease) and a not statistically significant 7 percent decrease among girls. We provide a detailed discussion of potential mechanisms that likely contribute to this decline in initial ADHD diagnosis rates, and we address how the school disruptions that occurred in the beginning months of the COVID-19 pandemic could have had counteracting effects on both the development and diagnosis of ADHD.
We explore heterogeneity by in-person school activity to examine how the sudden contraction and subsequent reopening of in-person schooling affected patterns of initial ADHD diagnosis during the COVID-19 pandemic. We compare the size of the ADHD diagnosis drop across areas with higher versus lower levels of in-person schooling, using SafeGraph Mobility data to measure physical activity near schools as a proxy for the level of in-person activity. In our nationwide Optum analysis, we can only compare outcomes across states with different opening levels. In our Indiana analysis, we can measure school activity at a much finer geographic level, in that we can identify the student’s likely zoned school. We find evidence that the pandemic depressed ADHD diagnosis more in states with lower levels of in-person school activity, suggesting that schools are an important element in the child mental health system. The moderating effect of in-person schooling is strongest for girls in our nationwide analysis. In particular, we find that for girls the cumulative ADHD diagnosis returned to pre-pandemic rates in states that went back to in-person instruction earlier. Within Indiana, which itself was a relatively open state in fall 2020, we find similar patterns comparing counties with different in-person schooling levels.
Finally, we analyze the potential welfare effects of these declines in diagnosis by plotting the observed ADHD diagnosis rate and the counterfactual values based on predictions from our estimated model. We then compare these diagnosis paths to estimates of pre-pandemic ADHD prevalence for boys and girls. While the proportional decline for boys and girls is similar, our counterfactual analysis suggests the pandemic may have worsened the underdiagnosis problem for girls and potentially mitigated some of the overdiagnosis problem for boys. However, we note that these welfare effects are conditional on pre-pandemic ADHD prevalence estimates and will therefore depend on if/how the pandemic and associated changes in school learning environment also impacted underlying ADHD prevalence directly. Future research will be necessary to better understand the implications of the drop in cumulative ADHD diagnosis that we document here.
These results on how societal disruption affects child mental health contribute to the recent literature on the importance of schools as a determinant of child well-being during the COVID-19 pandemic. Baron, Goldstein, and Wallace (2020) and Bullinger et al. (2021) show that reports of child maltreatment fell at the beginning of the COVID-19 pandemic, partly driven by school closures and the associated decline in child maltreatment referrals from school personnel, which have been shown to be a crucial source of early maltreatment identification in pre-pandemic times (Benson, Fitzpatrick, and Bondurant 2022). Hopkins et al. (2023) document a drop in special education classifications for elementary school children in Michigan, likely driven by constraints on in-person assessments and other school resources.
Previous research shows that the pandemic adversely affected educational outcomes, leading to lower academic test scores and pass rates due to schooling disruptions early in the pandemic (Engzell, Frey, and Verhagen 2021; Agostinelli et al. 2022; Jack et al. 2023). It is somewhat less clear what happened to underlying rates of mental illness in children. Surveys that rely on parental reports indicate that school closures and disruptions are associated with worsening child behavior and increased mental health concerns (Hawrilenko et al. 2021; Gassman-Pines et al. 2022). A meta-analysis of related surveys suggests that this is true for ADHD-specific symptoms as well (Rogers and MacLean 2023). However, these results could reflect parents spending more time with their children under stressful circumstances. Straub et al. (2023) present trends in mental health care utilization among adolescents, finding that visits for eating disorders increased for females during the later waves of the pandemic. Hansen, Sabia, and Schaller (2022) document a drop in teen suicides with school closures that reversed when teens returned to in-person schooling. They show that exposure to bullying may be a driving factor. We contribute to this literature by focusing on pandemic-related trends in initial visits for ADHD, one of the most common child mental health diagnoses.
By measuring in-person school activity with SafeGraph mobility data, we also contribute to the related and overlapping area of research that examines pandemic effects by using detailed cellphone location data to capture mobility patterns at various geographic levels (for example, Parolin and Lee 2021; Andersen 2020; Gupta, Simon, and Wing 2020; Buckee et al. 2020). We discuss the benefits of these data as measures of societal disruption that may not be captured in administrative or survey data.
Our work also relates to the literature on accuracy, costs, and benefits of ADHD diagnosis more broadly. ADHD is defined by symptoms of inattention, hyperactivity, and impulsivity, which can negatively impact child well-being and human capital development. This condition is associated with lower educational attainment (Currie and Stabile 2006), risky behaviors (Chorniy and Kitashima 2016) and longer-run labor market outcomes (Knapp et al. 2011; Fletcher 2014). While accurate diagnosis and subsequent treatment can help manage ADHD symptoms and reduce negative consequences, overdiagnosis and overprescribing are also a cause of concern. Many studies show that a child’s birthdate in relation to school entry cutoff date is a strong predictor of ADHD diagnosis. This suggests that teachers may be subjectively comparing younger and older students in the same class and mistaking immaturity for ADHD symptoms (Elder 2010; Evans, Morrill, and Parente 2010; Schwandt and Wuppermann 2016; Layton et al. 2018; Persson, Qiu, and Rossin-Slater 2021). Overdiagnosis of marginal children can have detrimental effects as well. Stimulant medications used to manage ADHD symptoms can have significant side effects and potential long-run negative consequences, especially if not used according to clinical guidelines (Currie, Stabile, and Jones 2014). While data limitations prevent us from studying prescriptions and/or rates of ADHD diagnostic errors, our paper relates to this literature by highlighting the net change in diagnosis rates during the pandemic, along with the potential role of schools in influencing ADHD diagnosis.
Finally, we also add to the literature on disparities and bias in child mental health identification by analyzing heterogeneity in ADHD diagnoses by child gender and race/ethnicity. While boys are more likely than girls to have ADHD, the literature suggests that the diagnostic gap is larger than the true prevalence gap due to an overdiagnosis of boys and underdiagnosis of girls (Marquardt 2022). There is less consensus on true prevalence differences based on individual race and ethnicity. Figure 1 plots the national trends in ADHD diagnosis rates since 2000. While white children are more likely to be diagnosed with ADHD on average, there are significant changes in the race-based diagnostic gaps. Possible explanations for these changing diagnostic disparities include barriers to care, stigma, cultural differences, and implicit biases in the clinical setting.1 Besides parents and physicians, research shows that school personnel also differ in their perceptions or awareness of behaviors when comparing boys to girls or minority students to nonminority peers (Sciutto, Nolfi, and Bluhm 2004; Elder et al. 2021). Further, existing research documents differences in the level of societal disruptions and social support during the pandemic across demographic groups, suggesting there may also be differential mental health responses by race/ethnicity (Montenovo et al. 2022). Our paper extends these research lines by exploring gender and racial heterogeneity in diagnosis changes due to pandemic-related disruptions.
National ADHD Diagnosis Trends
Notes: This figure presents the percent of children aged 5–17 ever told they had ADHD by a medical professional, as identified within the National Health Interview Survey (NHIS). Gender and race/ethnicity group averages are weighted by NHIS person sample weights. Trends are smoothed using five-year moving averages.
The remainder of this paper is structured as follows. In Section II, we describe the ADHD diagnosis process and provide a detailed discussion of possible mechanisms contributing to the change in diagnosis rates observed during the COVID-19 pandemic. Section III outlines our general empirical strategy. Section IV describes the three sources of data: the Optum national sample, the Indiana state sample, and SafeGraph mobility data. This section also introduces our definition of new ADHD diagnosis and how we measure fall 2020 school activity. In Section V, we present both our nationwide and within-state results. We discuss the potential implications of these results with a welfare exercise in Section VI. Finally, Section VII concludes with motivation for future research.
II. Background and Mechanisms
A. Attention Deficit Hyperactivity Disorder
According to the 2019 National Health Interview Survey, approximately 10 percent of school-aged children have attention deficit hyperactivity disorder, making it the most diagnosed mental health condition in the United States. The American Academy of Pediatrics (2019) presents the best-practice guidelines for ADHD treatment and strongly recommends FDA-approved medication for ADHD to be used along with behavioral therapy and/or educational interventions. However, in order to obtain these treatments, children must first receive a behavioral assessment in which a provider will determine whether symptoms meet the criteria for clinical diagnosis as defined within The Diagnostic and Statistical Manual of Mental Disorders (DSM-V, American Psychiatric Association 2013).
Initial diagnosis of ADHD in childhood is a complex process that involves a chain of events starting with symptom development/prevalence, behavior recognition, and clinical diagnosis (described further in Figure 2). Symptoms of ADHD often develop at ages 3–12, with a median diagnosis age of 7 (Wolraich et al. 2019). The medical literature documents a strong genetic component of ADHD development; however, the exact cause of the condition is both complex and unknown. In addition to genetic predisposition, many researchers argue that manifestation of ADHD may be triggered by prenatal or early childhood environmental exposures, including psychosocial factors, such as stressful life events and societal disruptions (Thapar et al. 2013).
There are three subtypes of ADHD: inattentive, hyperactive/impulsive, and combined (which includes symptoms of both other types). Males are more likely than females to have ADHD and, conditional on having ADHD, are more likely to experience the hyperactive or combined subtype relative to females (Hinshaw 2018; Hinshaw et al. 2022). That said, estimates of true underlying ADHD prevalence rates vary significantly, and existing studies find evidence of overdiagnosis among boys and underdiagnosis among girls (Bruchmüller, Margraf, and Schneider 2012; Marquardt 2022).
Because screening for ADHD is not universal, any ADHD symptoms expressed by children must be recognized and addressed via assessment. These behavioral concerns are typically first recognized by family members or teachers/school staff. While school psychologists may diagnose students with ADHD, they are not allowed to prescribe medication, and thus many are referred to a pediatrician for clinical assessment and treatment.2 The national shortage of school psychologists further pushes children to receive diagnosis and treatment outside of the school setting or may even limit care entirely (National Association of School Psychologists 2021).
During assessment, physicians should both consult the official diagnostic criteria for ADHD (the DSM-V) and request input from parents, teachers, or other school support staff. The DSM-V criteria requires at least six symptoms be present and that these symptoms occur in at least two environments over at least a six-month period. Typically, these two environments are at home and at school. However, clinical research shows significant heterogeneity in adherence to these formal guidelines.3
B. Potential Mechanisms
The COVID-19 pandemic has likely impacted the pathways to initial diagnosis in a variety of ways. We describe potential impacts at each point in the diagnosis process, starting first with the effect on ADHD symptom development and expression. The heightened general stress and environmental disruptions that were caused by the COVID-19 pandemic could have exacerbated ADHD symptoms or triggered gene expression, as suggested by the recent meta-analysis in Rogers and MacLean (2023). Changes to caregiver employment and work-from-home status may also have influenced child symptoms. The impact of school shutdown and subsequent reopening on symptoms is ambiguous and probably heterogeneous. As noted above, the added stress due to lack of school structure and in-person instruction could have exacerbated ADHD symptoms. On the other hand, it is also possible that online learning reduced external distractions and/or provided a more conducive learning environment for some children to thrive, which could lead to reductions in ADHD symptom prevalence (Dvorsky, Breaux, and Becker 2021).4
In addition to symptom expression itself, the recognition of symptoms by teachers, parents, and caregivers was impacted by the pandemic. Given the important role of schools and teachers/staff in the ADHD diagnosis process, it is likely that school closures and subsequent schooling disruptions due to COVID-19 had effects on symptom recognition, though again the direction is ambiguous. If students were attending school from home (virtually), parents and caregivers may have had a new view on child behavior during the school day, leading to increases in recognition. At the same time, behavior recognition and subsequent referrals from schools likely declined early in the pandemic, due to limited face-to-face time between students and teachers and restrictions in supply of school-based mental health services.5
Finally, the pandemic may have also directly impacted the clinical diagnosis stage. The most immediate result is a reduction in clinical diagnosis due to canceled or delayed behavioral assessments. In the initial months of the pandemic, local healthcare systems followed state guidelines in suspending elective and nonurgent medical care to prevent the spread of COVID-19. This made it difficult to provide the types of health services required to diagnose and treat child mental health conditions, as it did for other conditions (Helsper et al. 2020). And while physicians were allowed to conduct behavioral assessments via telehealth, these may not have been as informative about child symptoms when compared to those traditionally done in person.6
Even if assessment did occur (either telehealth or in person), physicians may have changed how they perceive symptoms and diagnose ADHD. For example, they may have seen symptoms as a short-term response to temporary changes in child environment rather than underlying symptoms of ADHD. School disruptions could have also indirectly impacted the clinical diagnosis stage, as the instability in school attendance or instruction mode constrained teachers/school staff attention, further limiting their ability to respond to requests for input from physicians. As a result, physicians may have deviated from the “two-environment” requirement specified within the DSM-V medical guidelines, or they may have chosen to delay diagnosis until symptoms could be documented and confirmed by those within a school setting.
Taken together, there are many factors that could impact initial diagnosis of ADHD due to the COVID-19 pandemic. In this paper, we document overall trends in cumulative rates of initial ADHD diagnosis, separately for elementary school–aged boys and girls. We focus on the early stage of the pandemic and examine heterogeneity in effects by school-openness status. Our estimates reflect the net effect of many potential mechanisms, likely through all three stages of the diagnosis process (see Figure 2). While heterogeneity by school-openness status does not necessarily imply specific school-based mechanisms, it provides suggestive evidence of the roles of schools in mitigating or exacerbating the pandemic-related disruptions.
III. Empirical Strategy
Our study is organized around cohort-based event study and difference-in-differences designs. We focus on two cohorts: the pandemic-exposed cohort, and the pre-pandemic unexposed cohort. The pandemic-exposed cohort is followed over a 19-month study window that runs from August 2019 to February 2021. The pre-pandemic unexposed cohort is followed over the 19-month period from August 2018 to February 2020. Throughout, we use t = 1. . .19 to index months measured in event time. Event time is assigned so that calendar months are always aligned in the two cohorts: period t = 1 refers to August 2019 in the exposed group and August 2018 in the unexposed cohort. The key idea is that the societal disruptions brought on by the COVID-19 pandemic began in event time period t = 8 for the exposed cohort (in March 2020), but not for the unexposed cohort (March 2019).
We construct the exposed and unexposed cohorts by building balanced panel data sets of children who are expected to be in Grades K–5 at the beginning of our study window, based on their age in August of 2018 (unexposed) and August 2019 (exposed).7 We describe the cohort inclusion criteria for each of our data sets in more detail in Section IV. We restrict to those children with no record of an ADHD diagnosis during a look-back period covering the six months immediately before the start of the cohort-specific study window. This means that the exposed cohort consists of elementary school–aged children who were ADHD-naive during the six-month period preceding August 2019. The unexposed cohort is defined the same way, except that they are required to be ADHD-naive during the six-month period preceding August 2018.
For each member of the two cohorts, we observe the child’s age at the beginning of the panel, geographic location of residence, and race/ethnicity.8 Let aict be an indicator variable set to one if child i from cohort c has ever been diagnosed with ADHD as of event time period t. Since both cohorts are ADHD-naive at baseline, aict is a cumulative measure that turns on if and when a child is “identified” as having ADHD. We use the age information to assign each child to a likely elementary school grade at baseline and then collapse the individual panel data into cohort × event time × grade × race × geography cells.9 After collapsing, Agrzct represents the cumulative fraction of the children in expected grade g with race/ethnicity r living in geography z in cohort c who have been diagnosed with ADHD by event time period t. This is simply the mean of aict in the grade–race–geography–cohort–period cell. Agrzct starts out at zero in the period just prior to the initial study period in each cell and then rises over the 19 months of the study period as ADHD cases are identified over time.
We study the effects of the pandemic shock on the rate of new ADHD cases in ADHD-naive cohorts using event study and difference-in-differences regressions that allow for an exponential conditional mean function, which we estimate with fixed-effect Poisson regression models.10 The goal of these models is to use the growth rate of cumulative new diagnoses in the unexposed cohort as a counterfactual for the growth rate of cumulative new diagnoses in the exposed cohort. Importantly, we also control for expected grade fixed effects. Therefore, all of our comparisons are within expected grade level. In other words, we compare first grade–aged children in the exposed cohort to first grade–aged children in the unexposed cohort. We note that our cohort construction allows some children in the unexposed cohort, particularly those not diagnosed with ADHD through the end of August 2019 and meeting our other inclusion criteria, to also appear in the exposed cohort. However, any students who do appear in both cohorts will have mechanically aged into a different expected grade between the years. Therefore, they will be part of the control group for one expected grade level comparison and the treatment group for a different expected grade level comparison. We do not believe this materially impacts our identification strategy, though in Section V we do discuss results where we restrict the sample to each expected grade level separately.
Because we are tracking cumulative new diagnoses, the trend in our dependent variable will be weakly monotonically increasing over event time. We therefore identify whether or not new diagnoses grow more slowly in the post-period for the exposed cohort relative to the unexposed cohort. This implies that the key identifying assumption is that the exposed and unexposed cohorts experience a common growth rate, which we test by checking if they were experiencing common growth rates over the pre-period—the first seven months of event time.
The workhorse specification in our analysis is:
1
In the model, Exposedc is a dummy variable set to one if the cell belongs to the pandemic exposed cohort and set to zero if it belongs to the control cohort. μg is an expected grade fixed effect, μr is a race/ethnicity fixed effect, μz is a geography fixed effect, μc is a cohort fixed effect, and μt is an event time fixed effect. Xgrzct is a covariate vector that—in most of our analyses—includes the cumulative rate of well-child visits in the cell and adult evaluation and management visits in the cell’s geographic area to proxy for healthcare supply constraints and avoidance behavior during the pandemic. In our Indiana analysis, where we do not use race-based cells, we omit μr and control for the race composition of the cell. The event study covers a 19-month window. Again, the first seven months (August–February) represent the pre-period of the event study, and the final 11 months (March–February) represent the post period. The coefficients of interest are the set of αs that trace out the relative difference in the growth rates of cumulative diagnoses between the cohorts relative to the reference period, February of the cohort base year (t = 7). In other words,
2
In addition to this event study model, to summarize the effects, we estimate the following difference-in-differences model:
3
PostMarcht is equal to one beginning in t = 8, which is March 2020 for the exposed cohort and March 2019 for the unexposed cohort. The difference-in-differences model is similarly interpreted as in Equation 2, but comparing the pre- and post-period rather than two specific event months.
We also estimate heterogeneous effects by race/ethnicity in our nationwide analysis. Here, we interact the event study variables or Exposedc × PostMarcht with indicators for whether the cell represents Asian, non-Hispanic Black, or Hispanic children, with non-Hispanic white cells as the reference group. In these specifications, we also interact the race indicators with all design-based fixed effects, which include the cohort and event time fixed effects.
Finally, to examine heterogeneity by in-person school availability, we estimate similar interaction models where we interact the event study variables or Exposedc × PostMarcht with measures of school mobility in geographic area z. As described in more detail in Section IV, we divide geographic areas into those with low, medium, and high in-person school activity. We treat low school activity areas (that is, those that stayed closed) as the reference group, and we include cohort-by-openness level fixed effects and event month-by-openness level fixed effects. These event month-by-openness fixed effects allow states with different levels of openness to have different time profiles of ADHD diagnosis in event time. Given common trends in the pre-period within openness categories, we interpret these interaction estimates as the differential effect of the pandemic by school openness level.11
In all of these regressions, we weight by the number of children in the cell and cluster the standard errors at the geographic area (state or county) by cohort level to allow arbitrary correlation of the error term both over time and across groups within a state cohort.
IV. Data
Our analysis leverages three different data sets. To measure changes in ADHD diagnosis nationwide, we use de-identified medical claims from Optum’s Clinformatics® Data Mart Database (Optum) for 2020 and 2021. We also use electronic health records data from the Indiana Network for Patient Care (INPC). These two data sets are complementary. Optum data are nationwide in coverage, but only among those with a single private insurer, whereas INPC covers only one state, but all payment types. In addition, the INPC data allow local-level—rather than state—variation in school activity. Finally, we use SafeGraph mobility data to measure in-person schooling activity at the state and school level during the COVID-19 pandemic.12
A. Health Data Sources
1. Optum
Our nationwide analysis uses medical claims from Optum’s de-identified Clinformatics® Data Mart Database (2020–2021). This database captures approximately 20 percent of the commercially insured population nationwide. Demographic data indicate that individuals represented are comparable to the U.S. commercially insured population (Lee et al. 2021), and we show this to be true among children in our sample.
In Table 1, we compare basic demographics of the pre-pandemic Optum sample to nationally representative statistics from the National Health Interview Survey (NHIS), restricted to children with private health insurance. The Optum sample has a similar gender, income, and race/ethnicity composition to the NHIS, although Optum has more white and fewer Hispanic children. In addition, ADHD diagnosis rates are very similar across NHIS and Optum.
Comparing Study Data Sets and the NHIS
2. INPC
Our Indiana-specific analysis uses data from the Indiana Network for Patient Care research database (INPC). INPC includes data from electronic health records from providers across the state collected through the Indiana Health Information Exchange. This information exchange is one of the longest standing in the country and includes most major healthcare providers in the state. About two-thirds of the state population have at least one encounter within the INPC (RDS, Regenstrief Institute 2022).13 It is difficult to know how many patients receive all or most of their healthcare from INPC-participating providers. However, Indiana is dominated by a small handful of large healthcare systems covered by the network.
Our INPC pre-pandemic cohort comprises all payers and, therefore, includes Medicaid-covered children not captured in the Optum data. Compared to Optum, the ADHD diagnosis rates are very similar in INPC, but lower than the national average for all payers. Table 1 provides additional sample composition comparisons to Optum and NHIS.
B. Defining Cohorts and New ADHD Diagnoses
In both the Optum national sample and the INPC sample, we define two cohorts of elementary school–aged children. In this subsection, we describe how we construct the exposed and unexposed cohorts and identify new ADHD diagnoses. In the next subsection, we describe how we identify which students are likely elementary school aged.
In our Optum analysis, we restrict the “exposed” cohort to those children continuously enrolled with the insurance provider from February 2019 through February 2021, who thus experienced the pandemic beginning in March 2020. The “unexposed” cohort is continuously enrolled from February 2018 through February of 2020 and did not experience the pandemic. In INPC, we can only observe when patients receive healthcare from INPC providers. We cannot directly observe when they “enter” or “exit” the data. We therefore define the exposed cohort as children with at least one INPC encounter during the look-back period, represented by the first six months of each cohort, between February 2019 and July 2019, and then follow these children through February 2021. The “unexposed” cohort includes children with at least one INPC encounter between February 2018 and July 2018 and then followed through February 2020.
One concern with comparing these two continuously enrolled cohorts is that pandemic-related job loss will lead to losses in private insurance. This could alter the composition of the continuously enrolled exposed cohort relative to the unexposed cohort. However, this appears not to be a major limitation in our data. Online Appendix Figure B1 plots the fraction of elementary school–aged children enrolled in an Optum-covered plan in February of 2018 for the unexposed cohort and February of 2019 for the exposed cohort who remain enrolled in an Optum-covered plan in each subsequent month through our study period. As expected, the rate of children who remain enrolled falls over time for both cohorts. However, there is no observable drop in enrollment starting in March 2020 due to COVID-19. Instead, we see that 2020 enrollment drops at the start of the year in January for both cohorts (reflecting usual changes at new plan year).14
In addition to the fact that continuous enrollment stayed on trend during the pandemic, Online Appendix Table B1 shows that our two continuously enrolled cohorts are very similar in gender, race/ethnicity, and income demographics. The difference in average age is statistically significant but not meaningful in magnitude. While our identification strategy hinges on similar trends in ADHD diagnosis between cohorts, which we verify below, it is also important to see that the two cohorts are quite comparable.
To determine new diagnoses, we first separate the panel for each cohort into a six-month look-back period (February–July 2019 for the exposed cohort and February–July 2018 for the unexposed cohort) and the remaining study period (beginning in August 2019 for the exposed cohort and August 2018 for the unexposed cohort).15 We label a child as receiving a new ADHD diagnosis in month t of the study period if that child has a medical visit with a recorded ADHD diagnosis code in month t but no visits with ADHD diagnosis codes during the look-back period.16 ICD-10 diagnosis codes that indicate ADHD are those with the first three digits F90. For the main analysis, we group all ADHD subtypes into a single diagnosis category, and we discuss heterogeneity by subtype in Section VI.
To analyze the effect of the pandemic on cumulative new ADHD diagnoses, we essentially calculate how the cumulative rate of new ADHD diagnoses for the exposed cohort between August 2019 and February 2021 deviates from its pre-pandemic trend in the unexposed cohort between August 2018 and February 2020. Figure 3 plots our main outcome variable over time for each cohort, with months lined up in event time. From August of 2019 to March of 2020, the growth of new ADHD diagnoses was following a similar trend as the year before; however, the trend starts to deviate for both boys and girls once the pandemic begins and schools begin to shut down in March and April of 2020, with the exposed cohort accumulating fewer new diagnoses.17 This is true in both the nationwide Optum sample (Panel A) and the Indiana INPC sample (Panel B).
Cumulative New Diagnoses by Cohort
Notes: In Panel A, exposed cohort is children continuously enrolled between February 2019 and February 2021. Unexposed cohort is children continuously enrolled between February 2018 and February 2020. In Panel B, exposed cohort is children with at least one INPC encounter between February 2019 and July 2019. Unexposed cohort is children with at least one INPC encounter between February 2018 and July 2018. In both panels, sample includes children without an ADHD diagnosis during the six-month look-back period (February 2019–July 2019 and February 2018–July 2018 for the exposed and unexposed cohorts, respectively).
C. Assigning Children to Expected Grade in School
We assign each child in our sample to an expected grade in school at the beginning of our study window (August 2018 for the unexposed cohort and August 2019 for the exposed cohort). We use these expected grades to construct the grade-based cells discussed in Section III and to restrict our sample to likely elementary school–aged children: those who we expect would have been entering kindergarten through fifth grade in the fall of 2018 (unexposed) and fall of 2019 (exposed).
The Optum data only include year of birth. As a result, we assign expected grade as the grade that the majority of children in a year of birth would have been entering in their respective fall semesters. For example, in the exposed cohort, a large majority of the 2014 birth year group would have turned 5 by the fall of 2019, and therefore we assign this group to kindergarten. In the INPC data, we have exact birth dates, so we can be more precise. In this analysis, we assign expected grade based on actual age on August 1, the school cutoff date in Indiana. For example, in Indiana for the exposed cohort, kindergarten would be assigned as the expected grade to anyone born between August 2, 2013 and August 1, 2014.
We note that we are assigning children to their “expected” grade based on their age. It may be that some differences between actual and expected grades are related to ADHD symptoms. This is particularly a concern if the pandemic impacts decisions parents make about starting kindergarten or advancing grades. Given this concern, there are two advantages of our approach. First, it is better that we label grades based on age only and not based on the actual grade in which a student is enrolled, which is endogenous. Second, we assign expected grades based on ages in August of 2019 for the exposed cohort—prior to the pandemic. As a result, the cohort of students whose families were making decisions about entering kindergarten in the fall of 2020 are not included in our sample.
D. Additional Control Variables
In most specifications, we control for the cumulative rate of well-child visits and adult evaluation and management visits to proxy for healthcare supply constraints and avoidance behavior during the pandemic. In the nationwide Optum analysis, we identify these visits based on CPT codes for new and established patient well-child visits (99383, 99384, 99385, 99393, 99394, 99395) and new and established adult evaluation and management visits (99202, 99203, 99204, 99205, 99212, 99213, 99214, 99215). In the Indiana INPC analysis, we do not observe CPT codes, so we use ICD-10 Z-codes to identify child and adult general healthcare utilization. Online Appendix Figure B6 compares the flow of well-child visits and adult evaluation and management visits in both exposed and unexposed cohorts and the similar measures in Indiana. These types of visits fell during March of 2020 and returned to pre-pandemic levels by summer 2020.
In the Indiana analysis, we do not have a sufficient number of observations to stratify by race, but we do include controls for cell-based racial composition: percent non-Hispanic white, non-Hispanic Black, Hispanic, and Asian. We also control for cell-level health insurance composition, as INPC includes all payers, whereas the nationwide Optum analysis includes only one private health insurer: percent Medicaid, percent privately insured, and percent other insurance.
E. Measuring Fall 2020 School-Openness Activity
To measure in-person schooling at the state and school level over the course of 2020 and 2021, we follow Parolin and Lee (2021) by using data from “smart” devices that record the level of activity by location and date. Unlike administratively reported data or data scraped from school district websites, cell-device mobility measures cover all schools and can measure both intensive and extensive margins.
There was a clear and uniform closing of schools for all states right after the start of the pandemic, but we see substantial geographic variation in the availability of in-person schooling at the start of the next school year in fall 2020. For our nationwide analysis, we examine heterogeneity in ADHD diagnosis by state fall 2020 school-opening level. To do so, we first aggregate total cell-device visits to all elementary schools within a state, by month. We define state–month relative opening as the total monthly visits in 2020 divided by total visits in 2019 within the same state and calendar month. Finally, for each state, we take the average relative opening across fall months (September, October, November) and partition on this measure to create “high,” “medium,” and “low” opening groups, each with 17 states.18 Low opening states are those with fall 2020 visit levels less than 54.4 percent, medium opening states range from 54.4 percent to 70.4 percent, and high opening states are above 70.4 percent relative to their 2019 levels. Figure 4 displays this geographic variation in school-openness levels in fall 2020.
Fall 2020 School-Openness Groupings
Notes: Figure displays fall 2020 school-openness groupings derived from SafeGraph mobility data as described in Section IV. In Panel A, low opening states are those with an opening level less than 54.4 percent, medium opening states range from 54.4 percent to 70.4 percent, and high opening states are above 70.4 percent relative to their 2019 levels. In Panel B, low opening counties are those with an opening level less than 76 percent, medium opening counties range from 76 percent to 93 percent, and high opening counties are above 93 percent relative to their 2019 activity levels.
In Online Appendix A we compare our SafeGraph-derived measure of school activity to reported learning mode policies collected by the COVID-19 School Data Hub (CSDH, COVID-19 School Data Hub 2022). We show that once aggregated to the state level, there is a strong correlation between in-person school policies and actual in-person school visits. However, the CSDH only reports for a subset of states, whereas SafeGraph mobility data cover all states.
Because in-person school availability was typically made at the school or district level and not the state level, the average state-level analysis may potentially miss important variation in school availability within a state. For example, the average aggregate school-openness in Indiana for fall 2020 is 84 percent, making it a high opening state for the nationwide analysis. However, Figure 4 shows there is significant variation in in-person school availability for elementary schools across the state during the fall of 2020.
When we conduct the within-state analysis in Indiana, we proceed in a slightly different way for partitioning individuals into different school-openness groupings, because in INPC data we have access to very detailed geographical locations (zip code and census tract) of the patient. Specifically, we construct a measure of the opening level of schools serving the location where a child lives using the zip code–census tract combination. We first start by obtaining school-specific measures of fall relative opening, defined as the average of relative visits to the school across weeks in the fall semester.19 School educational mode decisions are often at the school level, but school catchment areas do not necessarily coincide with census tract borders or zip code borders. Using GIS software, we overlay the elementary school catchment areas obtained from the National Center for Education Statistics (NCES) on top of zip code and census tract borders. We then calculate the fraction of a zip code–census tract combination pairing that is covered by each school catchment area and construct the weighted average of relative school opening levels for the zip code–census tract pair, where weights correspond to the fraction of the zip code–census tract covered by the school catchment area. We attach school-openness measures to each individual based on their zip code–census tract reported in INPC, then collapse to the county level and divide into three equal-sized groups, so that there is an equal number of elementary school–aged children in each group. Low opening counties are those with an opening level less than 76 percent, medium opening counties range from 76 percent to 93 percent, and high opening counties are above 93 percent relative to their 2019 activity levels.
Similar to the state-level comparison, we also compare our zip code–census tract SafeGraph-derived school-openness measures to CSDH in-person learning mode at the school and/or district level in Indiana. We show that at these finer geographic regions, administrative policies and realized activity differ. This is likely driven by both local demand for in-person schooling and somewhat arbitrary definitions of in-person learning during the fall of 2020. In Indiana, learning mode is determined by whether the majority of students receive at least three-quarters of their instruction in person, which leaves room for a lot of variation in actual attendance even in schools coded as fully “in person” by the CSDH data set. See Online Appendix A for further discussion and implications of these differences.
V. Results
A. Nationwide Results (Optum)
Figure 5 plots the event study estimates for boys and girls. We translate the event time Poisson coefficients 
from Equation 1 into percent changes by exponentiating and subtracting one. Before the pandemic started, diagnosis rates for the exposed and unexposed cohorts trended similarly. However, for the group of children who were exposed to the pandemic, we see a sharp decline in new diagnoses that starts in March 2020 and continues to the end of our sample period. Though slightly smaller, the decline in diagnosis persists even after controlling for well-child visits and adult evaluation and management visits, suggesting that the drop in general healthcare utilization that occurred throughout the beginning of the pandemic is not driving our results. We note here that the decline is of similar proportional magnitude for boys and girls and that the gap persisted even through the winter of 2020–2021, when many had returned to in-person schooling. In other words, after an initial decrease in new diagnosis, the exposed cohort did not “catch up” by receiving more new diagnoses later in the year.
Event Study Estimates, Nationwide (Optum)
Notes: This figure presents percent changes derived from event study estimates of changes in cumulative new diagnosis rate between the exposed and unexposed cohort. Exposed cohort is children continuously enrolled between February 2019 and February 2021. Unexposed cohort is children continuously enrolled between February 2018 and February 2020. Sample includes children without an ADHD diagnosis during the six-month look-back period (February 2019–July 2019 and February 2018–July 2018 for the exposed and unexposed cohorts, respectively). February 2019/2020 is the reference period for the unexposed/exposed cohorts. Plotted percent changes are the exponentiated event study coefficients minus one. The 95 percent confidence intervals are derived using the delta method. Standard errors are clustered at the state by cohort level.
Table 2 presents the difference-in-differences specification results in Columns 1 and 3. For both boys and girls, we see a statistically significant overall decline in diagnoses in the exposed cohort relative to the unexposed cohort (labeled “Pandemic” in the table). Consistent with our event studies, Panel B translates the Poisson coefficients into overall percent changes. Cumulative ADHD diagnosis fell by 8.58 percent for boys and 11.0 percent for girls. Online Appendix Table B3 presents difference-in-differences estimates and transformed percent changes separately for each expected grade (K–5) to explore heterogeneity across different age groups of children. We find negative effects for both boys and girls across all age groups. While overlapping confidence intervals prevent us from making strong conclusions about differential effects, we do note that the point estimates are largest among kindergarten-aged boys and third grade–aged girls. These results imply no systematic patterns of pandemic effects on ADHD diagnosis response by age groups.20
Difference-in-Differences Estimates by Race and Ethnicity, Nationwide (Optum)
Columns 2 and 4 of Table 2, Panel A present heterogeneity by race/ethnicity. Here, the reference group is white children, so the table presents differential declines in diagnosis among other race and ethnicity groups compared to the decline for white children. While results are noisy, we see a positive interaction effect that implies the decrease in ADHD diagnosis is smaller for Black children than for white children.
To better summarize the heterogeneity, we translate the Poisson coefficient estimates into percent changes by group; see Panel B of Table 2.21 The overall decline in ADHD diagnosis rates seems to be driven by both white and Hispanic children. Though not statistically different from each other, both white children and Hispanic children experienced significantly large declines in new ADHD diagnoses during the pandemic. The point estimates are negative for Asian children, though only marginally statistically significant for boys. For Black children, on the other hand, point estimates are positive in magnitude, though again not statistically different from zero. Online Appendix Figure B7 presents the associated event study estimates.
Next, we analyze the importance of school opening status on cumulative ADHD diagnosis. Figure 6 plots event study estimates based on a model with school opening interactions. Panel A of Table 3 summarizes these results numerically, with low opening states as the reference group. Compared to states that stayed closed the most, children in high opening states experienced smaller declines in cumulative ADHD incidence, though the difference is only statistically significant for girls. Translated into percent changes (Panel B), we see that cumulative ADHD diagnosis for girls fell significantly in states with low and medium opening levels but not for states with schools that were largely open. For boys, the cumulative decline is largest in medium opening states, but statistically different from zero in all opening categories. In the event study figure, we see that girls in high opening states did experience a decline in new diagnoses during the spring and summer of 2020, but began to catch back up to the unexposed cohort during the fall and winter of 2020, when they were more likely in in-person schooling.
Event Study Estimates by State School Opening Level, Nationwide (Optum)
Notes: This figure presents percent changes derived from event study estimates of changes in cumulative new diagnosis rate between the exposed and unexposed cohort, interacted with state school opening level. Exposed cohort is children continuously enrolled between February 2019 and February 2021. Unexposed cohort is children continuously enrolled between February 2018 and February 2020. Sample includes children without an ADHD diagnosis during the six-month look-back period (February 2019–July 2019 and February 2018–July 2018 for the exposed and unexposed cohorts, respectively). February 2019/2020 is the reference period for the unexposed/exposed cohorts. Plotted percent changes are the exponentiated event study coefficients for each state school opening group minus one. The 95 percent confidence intervals are derived using the delta method. Standard errors are clustered at the state by cohort level.
Difference-in-Differences Estimates by State School Opening Level, Nationwide (Optum)
It is worth noting that our state measure of school openness is based on average relative visits across the fall 2020 semester and therefore does not fully capture the dynamics of in-person school activity that occurred during this early pandemic period. For example, suppose one state is remote for the first half of the semester and returns to fully in-person schooling for the second half. While this state will have the same openness level as a state that alters between remote and in-person schooling every other week, it is clear that the former allows for less disruptions and more stability in learning mode, which may impact the rate of new ADHD diagnoses.
We examine this possibility at the national level by analyzing heterogeneity by state “stability” measures, according to the variability in elementary school relative opening during the fall semester. Similar to before, we group states into “high,” “medium,” or “low” stability based on the standard deviation in their relative cell phone visit levels across weeks in the fall semester, ensuring equal number of states in each group. Low stability corresponds to states with the largest standard deviation in their weekly relative visits, implying frequent changes in in-person visits to the schools during the fall semester. Conversely, high stability corresponds to states with the smallest standard deviation in their weekly relative visits, meaning that the visits to schools are almost always high (consistently in-person learning) or almost always low (consistently remote learning). While most of the high stability states are also those with low openness, we show in Online Appendix Table B4 that there are multiple states in each stability–openness group pair. In this heterogeneity analysis, we interact Exposedc × PostMarcht from Equation 3 with the state stability grouping. We note that since the regression also includes state fixed effects, it implicitly controls for the state openness grouping as well.
Table 4 presents the results from this exercise. Panel A displays the fixed-effect Poisson coefficients with low stability states as the reference group, corresponding to the “Pandemic” coefficient in the table. Panel B displays the overall percent changes in ADHD diagnosis implied by these coefficients. These results suggest that stability of school visits is indeed a factor of ADHD diagnosis, particularly for boys. Compared to those in low stability states (high variation in visits to schools during the fall semester), we see that boys in states with high stability experience lower declines in new ADHD diagnosis rates. For girls on the other hand, we do not find any statistically significant differences in ADHD diagnosis rate response based on stability of schooling mode. Further, the overall percent changes for girls go in the opposite direction than for boys, with the largest decline in girl ADHD diagnosis rates among the high stability states. Because these high stability states are also more likely to be the low openness states (see Online Appendix Table B4), these results are consistent with our openness group heterogeneity results that show the largest declines in ADHD diagnosis for girls in low opening states.
Difference-in-Differences Estimates by State School Stability Level, Nationwide (Optum)
Taken together, these findings suggest that ADHD diagnosis for boys is dependent on consistency in schooling mode, whereas higher absolute levels of in-person schooling may matter more for ADHD diagnosis for girls.
B. Indiana Results (INPC)
Figure 7 presents the within-state INPC event study estimates. Besides two months of higher-than-average diagnosis rates for boys well before the start of the pandemic, we see that new monthly diagnosis rates trended similarly for “exposed” and “unexposed” cohorts prior to March 2020. Given the Poisson specification, the event studies are comparing ratios of diagnosis rates. Because the base rate of new diagnoses is very low in early months by construction, small absolute differences in new diagnosis changes between the two groups can be exaggerated when looking at relative changes in these early months. While the event study estimates appear to show differential relative trends over the first two months, the raw data plotted in Figure 3B show that the absolute differences were very small and therefore unlikely to impact our difference-in-differences estimates, which aggregate the pre- and post-period. Similar to the nationwide analysis, we see a decline in new ADHD diagnoses that starts in March 2020 and continues throughout the sample period. Within Indiana, however, the magnitude and strength of the decline is larger for boys than for girls.
Event Study Estimates, Indiana (INPC)
Notes: This figure presents percent changes derived from event study estimates of changes in cumulative new diagnosis rate between the exposed and unexposed cohort. Exposed cohort is children with at least one INPC encounter between February 2019 and July 2019. Exposed cohort is children with at least one INPC encounter between February 2018 and July 2018. Sample includes children without an ADHD diagnosis during the six-month look-back period (February 2019–July 2019 and February 2018–July 2018 for the exposed and unexposed cohorts, respectively). February 2019/2020 is the reference period for the unexposed/exposed cohorts. Plotted percent changes are the exponentiated event study coefficients minus one. The 95 percent confidence intervals are derived using the delta method. Standard errors are clustered at the county by cohort level.
Table 5 presents the difference-in-differences specification results. Columns 1 and 3 summarize the main effect for boys and girls, respectively. The overall decline is statistically significant for boys but not for girls. The transformed percent changes (in Panel B) show there was a statistically significant 18 percent decline in new ADHD diagnoses for boys and a nonsignificant 7 percent decline for girls.22 Online Appendix Table B6 presents difference-in-differences estimates and transformed percent changes by expected grade. As with the nationwide results, we do not find any systematic pattern in effects by expected grade.
Difference-in-Differences Estimates by Local School Opening Level (INPC)
Columns 2 and 4 of Table 5 presents the heterogeneous effects by school-openness level within the state. The associated event study estimates are shown in Figure 8. Noise and pre-trend differences in some school-openness level groups limit our interpretation of differential effects. However, in both the event study and difference-in-differences regressions, we find no evidence of substantial heterogeneity by school openness for boys, though the estimates suggest that ADHD diagnosis rates did not fall as much in medium opening areas.
Event Study Estimates by School Opening, Indiana (INPC)
Notes: This figure presents percent changes derived from event study estimates of changes in cumulative new diagnosis rate between the exposed and unexposed cohort, interacted with county school opening level. Exposed cohort is children with at least one INPC encounter between February 2019 and July 2019. Exposed cohort is children with at least one INPC encounter between February 2018 and July 2018. Sample includes children without an ADHD diagnosis during the six-month look-back period (February 2019–July 2019 and February 2018–July 2018 for the exposed and unexposed cohorts, respectively). February 2019/2020 is the reference period for the unexposed/exposed cohorts. Plotted percent changes are the exponentiated event study coefficients minus one. The 95 percent confidence intervals are derived using the delta method. Standard errors are clustered at the county by cohort level.
VI. Discussion and Implications
In this section, we discuss the potential implications of our main findings. We include results by ADHD subtype, a discussion of potential welfare effects, and present suggestive evidence of changes to other behavioral health outcomes, including various treatment methods for ADHD.
A. ADHD Subtypes
The results thus far show in nationwide and single-state data that the impacts of school opening levels as well as stability in opening levels differ for boys versus girls.
One possible explanation for these various school openness and stability results is that ADHD presents differently in boys and girls. As discussed in Section II, relative to boys, girls with ADHD are more likely to have internalizing symptoms (for example, inattentive subtype), which are less salient to observers, whereas boys are more likely to have externalizing symptoms (for example, hyperactive subtype).23 In Online Appendix Tables B7–B10, we present the difference-in-differences estimates by ADHD subtype categorized as inattentive only, any hyperactive (hyperactive or combined subtypes), and other/no category specified. While we do not place too much emphasis on these results given some inconsistencies in how ADHD subtypes are reported in the data (especially within INPC), we do note some overall trends in the nationwide results. Diagnoses fall by similar relative magnitudes for boys across all subtypes. For girls, the decrease in hyperactive diagnoses is smaller, and the 95 percent confidence interval of this estimate excludes the estimated effect on the other two types. When estimating separate effects by school opening, the most notable finding is within the hyperactive subtype for girls; compared to low opening states, girls in high opening states were less likely to experience declines in new hyperactive diagnoses. These results suggest that in-person schooling is an important factor for determining hyperactivity symptoms in girls in particular. The results in Indiana are less clear. We again see similar declines across types for boys. For girls, we actually find an increase in inattentive diagnoses, though we note that these are very rare at baseline in the INPC data. We also find no clear patterns by school openness within Indiana.
B. Welfare Exercise
If we assume that all children would have been accurately diagnosed in the absence of the COVID-19 pandemic and that underlying ADHD symptom prevalence stayed constant, our finding of a substantial decline in new cumulative ADHD diagnoses indicates vast underidentification of ADHD in both boys and girls. These “missed diagnoses” can be extremely costly both at the individual level and to society as a whole. Such costs arise from lack of symptom management, which can explain lower test scores and educational attainment in childhood (Currie and Stabile 2006) and can influence labor market outcomes in the long run (Fletcher 2014). Missed diagnosis may also have spillover costs, especially if unmanaged symptoms cause disruptions in the home or classroom (Aizer 2009). At the other extreme, if all children would have been overdiagnosed in the absence of the COVID-19 pandemic, again holding underlying ADHD symptom prevalence constant, the cumulative decline may be welfare increasing, as misdiagnosis is also costly due to excess medical and education spending, exacerbated by the added side effects and the lack of evidence supporting long-run benefits of stimulant treatment (Currie, Stabile, and Jones 2014).
However, it may be the case that underlying ADHD symptom prevalence changed during this period, and as we note in Section II, the societal disruptions associated with the pandemic could have ambiguous and/or heterogeneous effects on childhood ADHD symptoms. If ADHD symptoms worsened, then the cumulative decline noted in this paper could be a concerning indication of underdiagnosis. On the other hand, if ADHD-related symptoms fell during this time (perhaps due to lack of classroom distraction or flexibility in daily schedules), then the decline in cumulative new ADHD diagnosis may reflect an accurate diagnostic response to falling prevalence, with net welfare gains. However, the persistence of the effect, especially for boys, even when children returned to school suggests this channel is not likely to dominate.
Given heterogeneity in individual child behavioral responses to the pandemic disruptions, along with the subjective nature of ADHD diagnosis in general, it is unlikely that the decline in ADHD diagnoses summarized here represents either all “missed” diagnoses or all children who would have been misdiagnosed. Further, the welfare effects may differ by gender given the literature suggesting boys are more often overdiagnosed with ADHD and girls underdiagnosed (Bruchmüller, Margraf, and Schneider 2012; Marquardt 2022).
To visualize this, Figure 9 uses our nationwide estimates to plot observed ADHD diagnosis rates and counterfactual diagnosis rates, with the latter summarizing what diagnosis rates would have been had the COVID-19 pandemic not occurred. For each month, we use our event study model estimates to calculate the counterfactual diagnosis rate by predicting monthly outcomes with exposed cohort–time interactions set to zero. In other words, we estimate what the diagnosis rate would have been in each month had the pandemic never happened. We also calculate the predicted monthly total diagnosis rate from our model,24 with the difference between the counterfactual and predicted rates being comparable to our event study estimates. We focus on total diagnosis rates and therefore incorporate the number of children who already had an ADHD diagnosis during our look-back period.25
Counterfactual Total Diagnoses
Notes: This figure plots predicted and counterfactual diagnosis rates. We first calculate predicted and counterfactual (by setting the exposed cohort indicator to zero) cumulative new diagnosis rates for the exposed cohort based on the event study estimates. To translate these into total diagnosis rates we incorporate the number of children that did have an ADHD diagnosis during the look-back period. Total diagnosis rates are equal to (#PreviouslyDiagnosed + NewDiagnosisRate × #NotPreviouslyDiagnosed)/(#PreviouslyDiagnosed + #NotPreviouslyDiagnosed), where the new diagnosis rate is replaced by either the predicted or counterfactual rate for each month. The black dotted line corresponds to the estimated true ADHD prevalence by gender from Marquardt (2021, 2022). The shaded area corresponds to the range of true ADHD prevalence found in the epidemiological and/or psychological literature.
For reference, we plot the range of true ADHD prevalence estimates for boys and girls from the medical literature shaded in gray.26 We also include the gender-specific prevalence estimate from Marquardt (2022), the horizontal black dashed line, which takes a more quantitative/modeling approach to finding ADHD true prevalence based on text analysis of doctor note text and selection adjustments into mental health care. We note that these underlying prevalence estimates come from pre-pandemic samples and thus would not reflect any changes in underlying ADHD symptom prevalence that may have occurred during the early pandemic period.
Compared to the Marquardt (2021, 2022) true prevalence estimates, Figure 9A shows that while boys are on net overdiagnosed, the final observed rate is closer to the true prevalence estimate than the counterfactual. This would imply that the decline in cumulative ADHD diagnosis for boys may be welfare-increasing if it reduced the number of boys misdiagnosed and therefore limited the costs associated with misdiagnosis. However, it is difficult to quantify this given the large range in ADHD prevalence among boys, which again may have changed during the pandemic period. For girls, on the other hand, Figure 9B shows that the pandemic may have made the underdiagnosis problem even worse, with predicted diagnosis rates further from estimated prevalence than the counterfactual rate, though again, this is difficult to quantify given varying true prevalence estimates.
C. Other Behavioral Health Outcomes
The welfare effects of this decline in ADHD diagnosis may also depend on how treatment of ADHD has changed and/or if children are receiving alternative mental health care. In Figure 10, we plot the flows of these varying outcomes over time, for both the exposed and unexposed cohort, separately by gender. The first panels on the left show the flow of new ADHD diagnosis, with cumulative additions by month corresponding to our main cumulative new diagnosis rate in Figure 3A. As expected, we see a drop beginning in March for the exposed cohort relative to the unexposed cohort, with evidence of “catch-up” for girls but not for boys.27
Flows of New ADHD Diagnoses, Prescriptions, and Behavioral Therapy among ADHD-Naive, Nationwide (Optum)
Notes: Exposed cohort is children continuously enrolled between February 2019 and February 2021. Unexposed cohort is children continuously enrolled between February 2018 and February 2020. Sample includes children without an ADHD diagnosis during the six-month look-back period (February 2019–July 2019 and February 2018–July 2018 for the exposed and unexposed cohorts, respectively). Prescriptions are any ADHD-related prescription, ADHD behavioral therapy includes any behavioral therapy visits with an ADHD diagnosis recorded, and behavioral therapy is any behavioral therapy visit.
In the next two panels, we plot the monthly rate of ADHD prescriptions and monthly rate of ADHD behavioral therapy,28 respectively. These rates increase over time mechanically as more ADHD-naive children become eventually diagnosed (and treated) for ADHD. Interestingly, the prescription rate for the exposed cohort is lower than that of the unexposed cohort even before the pandemic began. This is not the case for ADHD behavioral therapy, which appears to follow the same pre-pandemic trend for both boys and girls. This suggests that before the start of the pandemic, there may have been a transition away from prescription drugs for ADHD treatment, while rates of ADHD-specific behavioral therapy stayed the same. The prescription gap between the exposed and unexposed cohort seems to widen after the pandemic starts, and it remains larger for the rest of sample period. ADHD-related behavioral therapy also drops after the start of the pandemic and returns to pre-pandemic levels only for girls.
Finally, to assess whether the fall in ADHD diagnosis may have resulted from a substitution towards general behavioral therapy, rather than ADHD-specific treatment, we also plot trends for all behavioral therapy visits in the final panel. We see that even before the pandemic began, the exposed cohort had higher behavioral therapy rates than the unexposed cohort, especially for girls. For boys, these rates dropped significantly after the pandemic began and remained lower throughout our study period. For girls, there was a slight drop during the first few months of the pandemic, and while rates returned to similar unexposed cohort levels, they remain below their pre-pandemic trend. We note that the vast majority of behavioral therapy visits (regardless of ADHD status) were conducted in person before the pandemic began, and we approximate from our data that about 30 percent of these monthly visits were conducted via telehealth after the onset of the pandemic.
Taken together, these figures show that both prescriptions and behavioral therapy for ADHD fell during the pandemic and remained below that of the unexposed cohort for the sample period, with the exception of ADHD behavioral therapy rates for girls, which return to pre-pandemic levels by the end of 2020. We do not find evidence of substitution to other mental health related treatments as rates of behavioral therapy more broadly also fell during the start of the pandemic; they remain below unexposed cohort rates for boys and return to similar unexposed cohort rates for girls, albeit still below their pre-pandemic trend.
The decline in cumulative new ADHD diagnosis rates that we identify is most likely a combination of reduced overdiagnosis, underdiagnosis, and changes to underlying ADHD symptom prevalence. Further, we do not find visual evidence of substitution to different treatments and/or different mental health diagnoses. While a longer-run study of additional outcomes will be necessary to identify the mechanisms and potential consequences of this decline, the evidence in this paper suggests that school closures have potential impacts on child mental health diagnoses.
VII. Conclusion
Despite reports that child mental health worsened during the COVID-19 pandemic (Gassman-Pines et al. 2022), we find diagnosis rates fell for attention deficit hyperactivity disorder, a very common child mental health condition. While the pandemic conditions may have reduced the underlying incidence of ADHD in children overall, recent studies in the medical literature suggest that the pandemic instead exacerbated ADHD-related symptoms (Rogers and MacLean 2023). Therefore, we interpret the decline in new ADHD diagnosis rates documented here as driven mostly by changes to the inputs of diagnosis rather than overall reductions in the number of children with ADHD-specific symptoms.
We document a statistically significant and large decline in cumulative new ADHD diagnosis starting in March of 2020 and extending into early 2021. We use two different data sets to show this decline appears for both individuals within a specific state, captured by electronic health records, and across states nationwide, captured by private health insurance claims. Taking into account the gender differences in baseline ADHD diagnosis rates, we show the proportional decline in cumulative new diagnoses is similar for boys and girls nationwide, but larger for boys within Indiana. In our nationwide analysis, we also estimate heterogeneous effects by child race/ethnicity and find that Black children were significantly less likely than white children to experience drops in ADHD diagnosis rates. Overall percent declines were largest among Hispanic and non-Hispanic white children. Asian children also experienced similar point estimate declines in ADHD diagnosis rates, though those declines were only statistically different from zero for boys. Point estimates go in the other direction for Black children, and while not statistically significant, suggest that Black children experienced a potential increase in ADHD diagnosis rates during the COVID-19 pandemic.
Because schools and teachers are important inputs of the ADHD diagnosis process, we examine how the lack of in-person schooling during the early parts of the COVID-19 pandemic may have contributed to the fall in cumulative new diagnosis rates. We explore this idea using SafeGraph mobility data at elementary schools and analyze heterogeneity in the ADHD diagnosis decline based on school openness groupings in the fall of 2020.
When analyzing differences at the state level, we find that girls in states with low and medium school-openness levels had larger declines in diagnoses than girls in high-openness states, suggesting that in-person schooling is a particularly relevant input of ADHD diagnosis in girls. This does not appear to be the case for boys, however, where diagnosis rates fell similarly regardless of school openness grouping. Within Indiana, which is a state with a high level of school-openness overall, we find similar patterns, though the results are noisier and less conclusive.
At the nationwide level, we also test for heterogeneity by stability of school instruction mode, defined as the standard deviation in relative school opening across the fall semester. We show that compared to boys in states with low stability (high variation in school visits), boys in high stability states experienced lower declines in ADHD diagnosis rates. There was no statistical difference in ADHD response for girls based on school stability grouping.
We discuss the potential welfare implications of the decline in new ADHD diagnosis with a counterfactual analysis that shows the pandemic may have reduced the overdiagnosis problem for boys but compounded the underdiagnosis problem for girls. With this analysis, we emphasize that longer-run studies will be essential to further our understanding of pandemic effects on child well-being. Specific to this study in particular, we identify some important future questions. Do cumulative diagnosis rates rebound to pre-pandemic levels following our study period? How has the decline in new ADHD diagnoses affected test scores and other academic outcomes? Are there long-run impacts of this decline, such as compromised adult outcomes from a delay in initiation of treatment and/or benefits from reducing the potential to overprescribe ADHD medication? And, how do these associated outcomes differ across states or demographic groups?
Our work also contributes to other research outside of ADHD, documenting the trade-offs made when cutting back in-person schooling for the sake of public health protections. There are difficult choices that policymakers needed to make, based on little evidence about the implications for the short run, let alone for the longer run. The literature has been producing evidence along many lines—from test scores to mental health to physical fitness to cyberbullying—of non-COVID consequences of school closures and disruptions from shifting instruction to online formats. This evidence base will be important for future management of national emergency decision-making, as well as for economic studies tracing the roles of parents, providers, and schools in producing child health.
Acknowledgments
Sponsored by the NOMIS Foundation and the Center for Health and Wellbeing at Princeton University.
The authors thank Lauren Jones, Martin Anderson, Tony LoSasso, Bill Brinkman, and the participants of the American Society of Health Economists conference, Association for Public Policy Analysis and Management conference, Mid-Midwest Applied Microeconomics Workshop, and the Causes and Consequences of Child Mental Health conference hosted by the Center for Health and Wellbeing at Princeton University for valuable comments and suggestions. They also thank the Regenstrief Institute for facilitating access to data from the Indiana Network for Patient Care. The views here do not represent those of the Federal Reserve Bank of Chicago or the Federal Reserve System. All errors are those of the authors. Simon acknowledges receiving financial support from NIH summer salary through the NBER (not for this research) and from Elsevier for journal editing. Other authors have nothing else to disclose. Optum reviewed the research for accurate description of the database, but does not engage in the science of the research. The analysis was deemed exempt by the Indiana University IRB. This paper primarily uses confidential data from Optum’s de-identified Clinformatics® Data Mart Database, the Indiana Network for Patient Care (INPC), and SafeGraph. Optum data were used under license and are not publicly available. Interested researchers should contact Optum. INPC data can be requested through the Regenstrief Institute at https://www.regenstrief.org/rds/data/. SafeGraph data can be requested at https://www.deweydata.io/data-partners/safegraph. The authors are willing to assist with introductions. Replication code is provided in an Online Appendix of Replication Materials.
Footnotes
↵1. For a further discussion, see Nigg (2022) and citations therein.
↵2. Only five states have psychologist prescriptive authority laws, but these are only allowed under limited circumstances that, at the current time, do not extend to the school setting (Curtis, Hoffmann, and O’Leary Sloan 2023).
↵3. Chan et al. (2005) finds that only 28 percent of physicians strictly follow the DSM guidelines in diagnosing ADHD. Via chart reviews, Epstein et al. (2014) find that teacher and parent rating scales were used in just over half of all ADHD assessments, though they do find higher rates of DSM documentation than previous studies.
↵4. See https://www.cdc.gov/ncbddd/adhd/features/adhd-and-school-changes.html (accessed October 4, 2023) for additional discussion on ADHD and school changes.
↵5. During the COVID-19 pandemic, 67 percent of schools increased mental health services to students to address rising mental health concerns; however, less than half of these reported hiring more mental health staff. Recent government policies and funding have been aimed at helping school-based mental health support, though the majority of these were not used or implemented until the 2021–2022 school year, which is outside the scope of our analysis (Panchal, Cox, and Rudowitz 2022).
↵6. We estimate that about 17 percent of the new ADHD diagnosis visits that took place during the pandemic period were done virtually via telehealth.
↵7. Each panel is balanced as we first identify children who are within the specified age range at the start of the sample, then follow those same continuously enrolled children throughout the sample period. Children will not “age in” or “age out” of the sample, though they will most likely advance one grade in the fall of 2019 (unexposed) and fall of 2020 (exposed), given our study period covers 19 calendar months.
↵8. In the nationwide Optum data, we observe the child’s state of residence. In the Indiana electronic health records, all of the children reside in one state, and we observe the zip code, census tract, and county of residence. For our main analysis, we use county as the main geography.
↵9. In our Indiana analysis, we do not collapse by race due to small cell sizes.
↵10. We use the fixed-effect Poisson only to model the conditional expectation function. We relax the assumption that the cumulative diagnosis rates are actually distributed Poisson by allowing for a heteroskedasticity and cluster robust variance matrix. Sometimes this approach is called a pseudo-Poisson model or a generalized linear model with a log link function (Wooldridge 2010).
↵11. We note the possibility that other factors, such as policy responses and attitudes towards COVID, could have been correlated with school openness while deferentially impacting ADHD diagnosis in the exposed and unexposed cohorts. To the extent that these factors are also correlated with healthcare access, we are able to control for these through well-child visits and adult evaluation and management visits. We therefore expect the first-order driver of any differences by school openness to be school related, though we cannot disentangle this mechanism entirely.
↵12. We use the Core Place of Interest (POI) and the Patterns files from SafeGraph, a data company that aggregates anonymized location data from numerous applications in order to provide insights about physical places, via the SafeGraph Community. To enhance privacy, SafeGraph excludes census block group information if fewer than two devices visited an establishment in a month from a given census block group.
↵14. We consider continuous enrollment in any Optum-covered plan. Enrollees may change plans, change employers, or switch to the ACA marketplace, and as long as their new plan is from the same insurer, we will still capture them.
↵15. We believe the six-month look-back period is sufficient, as the majority of children with an ADHD diagnosis will be observed with an ADHD-related claim every one to three months for “medication management.” This is due to federal policies surrounding the prescribing and dispensing of stimulant medication, which is a schedule II controlled substance.
↵16. We use visit diagnosis codes rather than prescriptions because the latter may not be fully captured in our data sets. In Optum, prescription coupons (for example, GoodRx discounts) or alternative prescription insurance plans mean that we may not observe all ADHD-specific prescription fills in the data. In fact, we find that prescription rates in Optum are much lower than in NHIS, despite the similarity in diagnosis rates. In INPC, collection of prescription-specific data changed significantly during our sample period, which results in undercounting in later sample months. In Online Appendix Figures B2 and B3 and Table B2, we show that a nationwide analysis using new ADHD prescriptions are similar but noisier than our main results in Section V.
↵17. The analogous nationwide cumulative ADHD diagnosis rates by race/ethnicity and by school-openness grouping are shown in Online Appendix Figures B4 and B5, respectively.
↵18. The analysis includes data from all 50 states and the District of Columbia.
↵19. Here, relative opening is the 2020 school visits divided by the 2019 school visits to the same school in the same calendar week.
↵20. This similarity also suggests that our results are not biased by the presence of children appearing in both the unexposed cohort at one age and the exposed cohort at a later age, since this does not occur in these age-group specific subsamples.
↵21. Mathematically, we first estimate the race-interacted main effect coefficients in Equation 3 with non-Hispanic white children as reference group. We then calculate the decline for non-Hispanic white children as
, and the decline for each other race group as .↵22. These results are larger in magnitude than the estimate for high-opening level states in the nationwide analysis. Online Appendix Table B5 and Figure B8 show that this is not driven by the fact that the INPC data is all-payer and Optum is only privately insured, as within Indiana results are similar across Medicaid and privately insured patients.
↵23. In fact, in our nationwide data, girls are equally likely to have an inattentive diagnosis or a hyperactive diagnoses, whereas boys are far more likely to have a hyperactive diagnosis. We find similar patterns in Indiana, though inattentive diagnoses are very rare in the INPC data.
↵24. Average observed and predicted diagnoses are almost identical.
↵25. Total diagnosis rate is equal to (#PreviouslyDiagnosed + NewDiagnosisRate × #NotPreviouslyDiagnosed)/(#PreviouslyDiagnosed + #NotPreviouslyDiagnosed), where we replace the new diagnosis rate with either the predicted or counterfactual versions.
↵26. These prevalence estimates vary significantly based on sample size, setting (for example, community vs. clinical), time frame, and review criteria. We pull these estimates from meta-analyses on ADHD prevalence that differentiate by gender, including Froehlich et al. (2007); Polanczyk et al. (2007); Willcutt (2012); Kessler et al. (2012); and Cordova et al. (2022).
↵27. While plotting “flows” of new ADHD diagnosis rather than cumulative rate changes, this subfigure is consistent with our main nationwide event study estimates in Figure 5, which shows that cumulative diagnoses continued to fall relative to the unexposed cohort throughout our study period for boys, whereas the divergence for girls occurred during the spring and then began to level out during the summer and fall.
↵28. ADHD behavioral therapy is identified as a visit with a behavioral therapy CPT code and an ADHD diagnosis code.
- Received December 2022.
- Accepted July 2023.
This open access article is distributed under the terms of the CC-BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0) and is freely available online at: https://jhr.uwpress.org.
