Abstract
Rising reports of suicidal behaviors in children and adolescents have led to the recognition of a youth mental health crisis. However, reported rates can be influenced by access to screening and changes in reporting conventions, as well as by changes in social stigma. Using data on all hospital visits in New Jersey for 2008–2019, we investigate two inflection points in adolescent suicide-related visits and show that a rise in 2012 followed changes in screening recommendations, while a sharp rise in 2016–2017 followed changes in the coding of suicidal ideation. Rates of other suicidal behaviors, including self-harm, attempted suicides, and completed suicides were essentially flat over this period. These results suggest that underlying suicide-related behaviors among children, while alarmingly high, may not have risen as sharply as reported rates suggest. Hence, researchers should approach reported trends cautiously.
I. Introduction
The COVID-19 pandemic focused renewed attention on child and adolescent mental health, leading the American Academy of Pediatrics to join with the American Academy of Child and Adolescent Psychiatrists and the Children’s Hospital Association in declaring a state of national emergency (American Academy of Pediatrics 2021). The Surgeon General also issued an urgent public health advisory about youth mental health (Murthy 2021). While the pandemic no doubt made a bad situation worse, diagnoses of serious mental health disorders, and especially suicidal behaviors in children and teens, had been rising since the Great Recession. For example, Curtin and Heron (2019) report that after falling from 1.5 to 0.9 per 100,000 between 2000 and 2007, suicide rates for children 10–14 rose to 2.5 in 2017. Over the same time intervals, suicides for children 15–19 first fell from 8 to 6.7 per 100,000 between 2000 and 2007 and then rose to 11.8 per 100,000 by 2017. However, it is possible that at least some of these recent increases, even in reported suicides, reflect changes in the reporting of events rather than changes in actual underlying behaviors.
As we seek to understand the causes of the youth mental health crisis, it is tremendously important to understand whether the time-series patterns reported reflect underlying behavioral changes, greater societal attention to youth mental health, or both. As discussed further below, reporting changes could be driven by expansions in health insurance coverage for mental illness, by expansions in screening, by changes in the way that mental health problems are diagnosed and coded in medical records, as well as by declining social stigma associated with mental illness, increased willingness to discuss suicide and suicidal behaviors, and a concurrent increased willingness to seek care.
We examine a startling pre-pandemic increase in recorded suicidal behaviors among New Jersey children 10–18 years old presenting at hospital emergency departments (ED) between 2008 and 2019. We show that there was relatively little increase in teen self-injury, intentional self-harm, suicide attempts, or in completed suicides in New Jersey over this period.1 Instead, the rise is mainly accounted for by a sharp increase in diagnoses of suicidal ideation (SI). Suicidal ideation is a serious condition that is grouped with self-harm in commonly used mental health disorder classification systems, such as the Clinical Classification Software or the Child and Adolescent Mental Health Disorders Classification System (Zima et al. 2020; Children’s Hospital Association 2019).
We show that the increase in reports of suicidal ideation over this period corresponded with two sets of changes in medical practice. The first was new guidance from the U.S. Preventive Services Task Force (USPSTF) and other professional organizations recommending screening of adolescent girls for depression, which was announced in 2011. The Affordable Care Act also made it mandatory for insurance companies to cover such recommended services beginning in 2011. The second change was a revision of the instructional notes included in the new version of the International Classification of Disease (ICD-10) in October 2016. Hospitals were required to start using the new version of the ICD in October 2015. The revised instructions marked the first time that clinicians were specifically directed to record suicidal ideation as a secondary diagnosis in cases with a primary diagnosis of a mental health condition.
Our results suggest that there may always have been large numbers of children and adolescents with serious mental health conditions, but that the changes in screening and coding practices helped to bring the problem to light.
The rest of this paper proceeds as follows. Section II provides some background discussion of several factors that could drive changes in the reporting of suicidal behaviors. Section III gives an overview of the data, and Section IV discusses methods. Section V presents results, and Section IV provides a discussion and offers conclusions.
II. Background: Factors Potentially Driving Increases in Reports of Suicidal Behaviors
A. Changes in Screening
Changes in screening have the potential to impact the reported incidence of mental health conditions by uncovering cases that were previously missed. In particular, more screening for depression could have increased referrals to emergency rooms for diagnoses of suicidal ideation because most screens for depression include specific questions about SI (Patra and Kumar 2022).
Online Appendix Table A1 summarizes changes in clinical guidelines and federal regulations that may have affected the probability that adolescents, especially adolescent girls, were screened for depression. In 2009, the USPSTF recommended that providers screen adolescents 12–18 for depression. However, insurers were not required to pay for these services until 2011, when the Affordable Care Act (ACA, passed in 2010) mandated that insurance companies pay for these recommended services without patient cost-sharing.
In addition, the ACA required the Department of Health and Human Services (HHS) to develop new guidelines for preventive services for women. As a result, in August 2011, the Women’s Preventive Services Guidelines (WPSI) were published. These guidelines included a recommendation that women and girls aged 12 and older be screened annually for depression. The HHS further required insurance plans to cover services in the WPSI without patient cost-sharing beginning in plan years starting on or after August 1, 2012. The combination of these new guidelines with the mandate that insurers cover the screenings was likely to have had a particularly large impact on girls, a hypothesis that we test below.
As shown in Online Appendix Table A1, there were further changes in screening recommendations after 2012, though these likely had less force. For example, in February 2016, the USPSTF updated their 2009 recommendation about screening adolescents 12–18 years old for major depressive disorder. The original guidance specified that screening should be implemented only when adequate provision was in place for either cognitive-behavioral or interpersonal psychotherapy. The new guidance was less specific about the type of follow-up required, which might, in principle, have resulted in more screening. However, the HHS recommendation that women and girls 12–18 be screened annually for depression was binding both before and after this update, so that if the update had been effective, it would have applied mainly to boys.
The American Academy of Pediatrics also updated their guidelines in March 2018 by recommending that all pediatricians screen for depression and include suicide-related questions in children’s annual wellness visits (Zuckerbrot et al. 2018). This change could have resulted in more screening for boys after 2018 (since screening of girls was already recommended by HHS), though insurers were not mandated to cover such screenings.
B. Changes in Diagnosis and Coding of Mental Health Conditions
A second possibility is that at least some of the increase in reported mental health problems has been driven by changes in the way these conditions are being diagnosed and recorded in surveillance data. There have been several significant changes that are likely to impact diagnosis rates.
First, there have been broad changes in diagnostic criteria for some mental health conditions with the publication of new versions of the Diagnostic and Statistical Manual of Mental Disorders (DSM) in 1994 and 2013 (American Psychiatric Association 1994, 2013). The DSM is the manual used by clinicians and researchers to diagnose and classify mental disorders. A prominent example is the creation of the new “autism spectrum disorder” (ASD) category in the DSM-5, which combined the DSM-4 categories of autism, Asperger’s syndrome, pervasive developmental disorder, and childhood disintegrative disorder. This change was associated with an increase in the incidence of autism from about one in 150 children in 2000 to one in 54 children diagnosed with ASD in 2016 (Maenner et al. 2021).
The DSM-5 also updated the definitions of depression and anxiety, two of the diagnoses most commonly associated with suicidal behaviors. For example, bereavement had been excluded from the definition of depression but is now included. Other changes include merging chronic major depressive disorder and dysthymic disorder (that is, mild but chronic depression) into a new category of persistent depressive disorder and adding premenstrual dysphoric disorder and disruptive mood dysregulation disorder to the list of depressive disorders (Grohol and Wright 2022). Diagnostic thresholds for anxiety disorders were lowered (Park and Kim 2020). The overall impact of these changes may have tended to increase the number of people diagnosed with depression and anxiety disorders after the changes went into effect in 2013.
A second reason for changes in coding was the mandatory switch from the ninth to the tenth edition of the International Classification of Diseases (ICD) in U.S. hospitals in October 2015. This switch involved sweeping changes in the coding of mental illness and self-harming behaviors. In the ICD-9, a clinician had to enter two codes to indicate self-harm, one recording the type of injury and the second indicating its “external cause.” Notably, the external cause codes were frequently missing, making it impossible to determine whether a harm was intentionally inflicted or not.
In the ICD-10, the clinician only enters one code that has the external cause embedded. For example, the clinician must choose whether to code “intentional drug overdose” or “unintentional drug overdose.” The adoption of the ICD-10 in October 2015 has been the focus of previous research that showed it increased the number of injury and poisoning visits coded as self-harm and decreased the number of visits with “undetermined intent.” The net result was that the total number of visits involving intentional self-harm remained largely unchanged around the October 2015 adoption point (Stewart, Crawford, and Simon et al. 2017; Stewart et al. 2019; Zima et al. 2020).
However, in addition to the new codes, the ICD-10 also introduced a new system of notes intended to make diagnostic coding more precise (Centers for Disease Control and Prevention 2015).2 There are two types of these new notes. An “Exclude 1” note says that certain codes cannot be entered together—for example, a suicide attempt code cannot be used with the suicidal ideation code because the latter is considered redundant—anyone who actually attempted suicide must have been considering it. There are also (confusingly named) “Exclude 2” notes indicating that certain codes should both be included on the claim if present.
The initial version of ICD-10 adopted in October 2015 had an Exclude 1 note on “symptoms and signs involving cognition, perception, emotional state and behavior” (diagnosis codes R40–R46, which include suicidal ideation). This rule prevented coders from listing any of these codes if the person had a primary diagnosis of a mental health disorder (diagnosis codes F01–F99). This meant that a suicidal ideation code could not be used with a primary diagnosis code for a mood disorder code like depression. Hospital claims coders have a clear incentive to obey these coding rules because a claim that has combinations of codes that have been disallowed will be rejected by both public and private insurers.
The Cooperating Parties in charge of implementing the ICD-10 in the United States soon recognized that their original instruction was a mistake and changed the note about “symptoms and signs involving cognition, perception, emotional state and behavior” (diagnosis codes R40–R46, which include suicidal ideation) to an “Exclude 2” effective October 2016 (Centers for Disease Control and Prevention 2015).
This change in coding instructions recognized that symptoms such as suicidal ideation do not always occur with primary mental health diagnoses such as depression, so that, in a 180-degree about-face, clinicians were now actively encouraged to include symptoms such as suicidal ideation as secondary diagnoses when present with a primary diagnosis of mental illness. To avoid confusion, in what follows, we will refer to the October 2016 instruction to include suicidal ideation as a secondary diagnosis when present as a coding directive to “include SI.”
An important question is whether hospitals had any incentive to train clinicians quickly to adopt this new more detailed coding system. Adoption of the new coding notes was in their interest, as cases with more secondary diagnoses are more complex and often result in higher reimbursements.3
C. Changes in Insurance Coverage
A third possibility is that expansions of health insurance coverage are likely to increase the frequency with which people seek care. People who never see a clinician will not have diagnoses for mental or physical health conditions. Hence, researchers using claims data are usually dealing with a selected sample. In a study of university student use of mental health services, for example, Cowan and Hao (2021) found that the level of diagnosis and treatment was lower in students from disadvantaged families, even though they were more likely to report mental health symptoms in surveys.
Expansions of public health insurance under the ACA have greatly increased access to mental health care among adults (Winkelman and Chang 2018), as well as other forms of care. Using pre-ACA data, Anderson, Dobkin, and Gross (2012) found that when young adults “aged out” of their parents’ health insurance, ED visits fell by 40 percent, with the drop being concentrated in privately owned, for-profit hospitals. This situation was mitigated by the ACA. Using state-level data, Maclean et al. (2019) find that in states that expanded their Medicaid programs, Medicaid-reimbursed prescriptions for antidepressants, anti-anxiety drugs, and stimulant drugs used for ADHD increased by 34 percent, 23 percent, and 16 percent, respectively. Even in nonexpansion states, the use of mental health services improved, likely reflecting the ACA’s strengthening of rules requiring insurance policies to cover these services and forbidding the exclusion of coverage for pre-existing conditions (Thomas et al. 2018).
These expansions in the coverage of both mental health and other services may have simultaneously both increased the number of adults being diagnosed with mental health conditions and decreased the proportion of adults seeking care who presented with mental health conditions.
More subtle changes in public health insurance coverage could also play a role in increasing screening for mental health disorders. Chorniy, Currie, and Sonchak (2018) study children in South Carolina who were enrolled in Medicaid. The state mandated that Medicaid children be moved from fee-for-service coverage to managed care in the mid-2000s. Different counties made the change at different times. Their study follows about 210,000 children younger than age 17 who appeared in the Medicaid records before and after the switch. They show that the change to managed care increased well-child visits and developmental screenings and led to significant increases in diagnoses of depression and ADHD, as well as in physical health conditions such as asthma. These changes in screenings were incentivized by higher capitated payments for children with chronic conditions, as well as by penalties for low screening rates.
These very large changes in insurance coverage and access to mental health care for adults make it useful to focus on children in New Jersey, a group that had widespread access to public health insurance coverage prior to the ACA and who did not see large changes in the type of insurance coverage available over our sample period. Since 2008, at least 90 percent of children in New Jersey have had either public or private health insurance. In contrast, 20 percent of adults were uninsured during our sample period, and many gained insurance coverage with the ACA expansion. If those who gained health insurance and starting using hospital services were disproportionately adults without mental health problems, then the rates of mental health problems observed among adults who accessed EDs could even decline.
D. Mental Health Stigma
A fourth possible reason for an increase in reports of suicidal behaviors and even in suicide itself is a decline in the stigma associated with mental illness. In the past, some conditions (such as personality disorders, which often lead to suicidal behaviors) have been so stigmatized that many providers rejected patients with these diagnoses, leading other providers to be reluctant to diagnose patients younger than 18 years old with these conditions at all (Aviram, Brodsky, and Stanley 2006). It is common to observe people being treated for mental illness (that is, with therapy or psychotropic drugs) even though there is no mental health diagnosis associated with the treatment (Druss et al. 2007). This practice may also reflect an attempt on the part of providers to shield patients from mental health stigma.
On the patient side, Bharadwaj, Pai, and Suziedelyte (2017) compare patient self-reports to administrative records of diagnosis and prescription drug use for mental illness. They find that survey respondents underreported mental health diagnoses 36 percent of the time and underreported prescription drug use for mental health 20 percent of the time. In contrast, there was little underreporting of diabetes or hypertension. This contrast is notable because both diabetes and hypertension are often linked to lifestyle choices, and one might therefore also expect them to be subject to stigma. However, apparently any stigma associated with these diagnoses is less than the stigma associated with mental health problems.
However, Pescosolido et al. (2021) argue that in the United States, the stigma associated with mental illness has declined over time. Angermeyer et al. (2017) confirm that knowledge about mental health, attitudes towards mental health treatment, and the propensity to seek such treatment have improved greatly in the past 25 years. Greater awareness is likely to lead to more diagnoses. In a study using autism registry data from California that included all known cases, King and Bearman (2011) found that children living in wealthier communities were at significantly higher risk of an autism diagnosis. They hypothesize that this pattern is due to community characteristics, such as the resources available for diagnosis and community awareness of the disorder.
Reductions in stigma could lead to more openness about the reporting of suicidal behaviors. In turn, such openness might be reflected in a change in the coding of cases in which the deceased’s intentions were unclear, as is often the case in drug overdose deaths (O’Carroll 1989; Breiding and Wiersema 2006). For example, Cutler, Glaeser, and Norberg (2001) show that “accidental” youth gun deaths fell as reported suicides rose, although the number of youth gun deaths was too small in the time period they study for it to account for the full increase in suicides.
It is also possible that better treatments for mental health conditions could cause more people to enter treatment, which in turn could lessen stigma. Conti, Busch, and Cutler (2011) argue that the availability of SSRIs as a treatment for depression resulted in many more new diagnoses. Likewise, Biasi, Dahl, and Moser (2021) find that diagnoses of depression in Denmark increased over time, even although diagnoses of bipolar disorder and schizophrenia were stable.
We expect that reductions in the stigma associated with mental health conditions may have both increased the propensity to seek treatment and reduced doctors’ reluctance to make mental health diagnoses. However, both of these changes are expected to have taken place gradually. Hence, while they can explain a general upward trend in diagnoses, they probably cannot explain sharp jumps.
E. Summary
This section has discussed several factors that may have tended to increase the reported incidence of mental health conditions both generally and among teens specifically. In what follows, we focus on two discrete changes in the screening and coding of mental illnesses in adolescents and show that these were associated with inflections in reported diagnoses of suicidal behaviors after 2011 and again after 2016.
III. Data
The main source of data for this study is the 2008–2019 data on all New Jersey hospitalizations and Emergency Department (ED) visits. While much mental health care takes place outside a hospital setting, hospital and ED records are a rich source of information about mental health patients in crisis. Psychiatric patients are always encouraged to go to the nearest ED if they feel suicidal, and any self-harm that results in serious injury or potential injury is also likely to be evaluated in the ED (Zeller 2018).4 Data from hospital visits are also less subject to reporting biases due to stigma (as in Bharadwaj, Pai, and Suziedelyte 2017) or due to parents reporting on behalf of children.
We analyze all New Jersey hospital discharge records and ED visit records for children aged 10–18 between 2008 and 2019, with the exception of records related to pregnancy and childbirth. The American Academy of Pediatrics recommends including preteens (aged 10–11) in analyses of teen mental health because they are often developmentally adolescent (Patra and Kumar 2022). We do not include children younger than 10 years old because suicidal incidents are much rarer for them (New Jersey Department of Children and Families 2016).
These data include patient visit dates, discharge dates, hospital identifiers, diagnosis codes, insurance type, and patient demographics, such as race, gender, and the zip code of residence. We use all of the listed diagnostic codes in each record to define suicide-related encounters. Specifically, we follow the Healthcare Cost and Utilization Project (HCUP) (2021) and Ridout et al. (2021) in order to identify diagnosis codes related to suicidal behavior.5 To differentiate mental-health-related visits and hospitalizations from non-mental-health visits, we use the ICD-9 Mental Disorders category codes (290–319) and the ICD-10 Mental, Behavioral, and Neurodevelopmental disorders category codes (F01–F99). We also use the HCUP Clinical Classification Software to identify mood-disorder encounters within the broader group of mental-health-related visits.6
IV. Methods
In order to conduct a graphical analysis of trends in suicide-related hospital visits, we aggregate the individual-level data to the annual level and compute rates per 10,000 children aged 10–19 using population data from the American Community Survey’s five-year data set for 2013–2017.7 For some subdivisions of the data (such as children going to hospitals with psychiatric units), we do not have good estimates of the relevant population denominator, so we instead plot the number of visits.
We next turn to regression analyses to ask whether the inflection points that we see after 2011 and after 2016 appear to be related to the changes in screening and coding that we identified above. We first use data at the individual visit level from 2008 to 2015 to analyze changes that may be related to the implementation of the Women’s Preventive Services Guidelines (WPSG) announced in August 2011. Since the guidelines only mandate annual screenings for women and girls, we estimate the effect of the change in screening guidelines on girls relative to boys. Specifically, we estimate the following regression model:
1
where Yijt is an indicator variable for a hospital visit with a mood disorder, anxiety disorder, suicidal ideation or any other mental health diagnosis code; FEMALEi is an indicator equal to one if the child is female and zero otherwise; AGEit is a series of indicators for each single year of age; RACEi is a vector of indicators for whether the child is Black, white, or other race; and INSit represents a vector of two indicators equal to one if the person has Medicaid or private health insurance, respectively.
The ZIPCHARSi vector refers to the child’s zip code of residence and includes demographic characteristics from the American Community Survey five-year data sets for 2013–2017. This vector includes median income; percent poor; percent renters; percent college educated; percent non-Hispanic Black, non-Hispanic white, Hispanic, and percent Asian; and indicators for the percent younger than 19 years of age and the percent older than 65. YEAR is a vector of year indicators, and MONTH is a vector of indicators for each month that is included to capture seasonal effects. These indicators capture trends in visits to the hospital over time, as well as seasonal effects in these visits. Most models also include HOSPj, a vector of fixed effects for each of the 75 hospitals in our sample, since different hospitals may have different baseline coding practices. In practice, however, we do not find large differences between specifications with and without hospital fixed effects.
In these regressions, the coefficients β1 to β4 represent the size of the jump in diagnoses of mental health disorders in each of the indicated years for girls relative to boys. If the new screening guidelines caused a jump in the screening of girls, and if in turn this increase in screening led to more secondary diagnoses of suicidal ideation, then we should see a significant coefficient on β1 to β4 indicating that girls started being diagnosed at significantly higher rates after 2011. Standard errors are clustered at the level of the hospital–year to allow for correlations in unobservable variables at that level.
In order to further investigate the impact of the changes in coding instructions in 2016, we use individual visit-level data to investigate trends in the use of suicidal ideation as a secondary diagnosis code in teen hospital visits with a primary diagnosis of mood disorder or other mental health disorder. The idea is to see whether the jump in the use of suicidal ideation as a secondary diagnosis is statistically significant in a model that controls for all of the observable characteristics of visits and to ask whether the jump in visits coincided with the introduction of ICD-10 in October 2015 or with the change in coding instructions a year later.
These regression models take the following form:
2
where Sijt is an indicator equal to one if a visit for child i, in hospital j, and year t is suicide-related and is zero otherwise. Alternatively, in some specifications, the independent variable Sijt is an indicator equal to one if there is a secondary diagnosis of suicidal ideation or if there is self-injury, intentional self-harm, or a suicide attempt.
Indicators for the years 2016–2019 are equal to one if the visit occurred within the indicated calendar year and zero otherwise. MDijt is an indicator equal to one if the primary diagnosis is a mood disorder and zero otherwise, and OMHijt is an indicator equal to one if the primary diagnosis is another mental health disorder and zero otherwise. The other variables are defined as in Equation 1.
In these regressions, the coefficients β1 to β4 (β5 to β8) represent the size of the jump in the outcome in each of the indicated years for teens with mood disorders (or other mental health disorders) relative to the rest of the population. If the switch to ICD-10 in October 2015 caused the coding of suicidal ideation diagnoses to jump, then we should see a significant coefficient on β1 (β5). If, on the other hand, it is the instruction to “include SI” that caused the increase, then we should see significant coefficients starting in 2017, that is, for β2 to β4 (β6 to β8) but not for β1 (β5). Standard errors are clustered at the level of the hospital–year to allow for correlations in unobservable variables at that level.
V. Results
A. Graphical Analysis
One potential problem with analyzing visits to hospital is that if place of service changed over time, then trends in hospital visits could present a misleading picture of trends in the general population. Online Appendix Figure A1 shows that the number of visits for mental health disorders per 10,000 teens in New Jersey rose relatively slowly and smoothly over the sample period. This suggests that any breaks in suicidal behaviors cannot be explained by sudden changes in the number or composition of children presenting at hospitals and EDs with mental health problems in New Jersey but must have to do with the way that the children presenting at the hospital are being diagnosed.
Figure 1A compares the trend in suicide-related visits to the trend in completed suicides among teens. These two time series have very different scales—there were 2.9 suicides per 100,000 children aged 10–18 in New Jersey in 2014 (New Jersey Department of Children and Families 2016) compared to approximately 431 suicide-related hospital visits per 100,000 children. Hence, we normalize each series to start at one in 2008, so that it is possible to compare the time-series trends. Completed suicides among teens in New Jersey do not show a clear trend over this period. Those in the northeast region as a whole show some upward movement between 2015 and 2017, and suicides actually declined slightly in New Jersey after 2017.
Trends in Suicide-Related Visits and Suicides in New Jersey, 2008–2019
Notes: These figures plot trends in suicide-related hospital visits, suicide deaths, and suicidal ideation hospital visits in New Jersey. The vertical lines at 2011 and 2016 help to visualize the changes related to the implementation of the Women’s Preventive Services Guidelines in 2012, as well the difference between 2015 and 2016 (implementation of ICD-10) and between 2016 and 2017 (implementation of the “include SI” guidance).
a Includes ICD-9 codes for self-injury and ICD-10 codes for intentional self-harm and suicide attempts.
In contrast, suicide-related visits clearly tick up after 2011 and again after 2016. The figure has vertical lines at these two points in order to show these inflections in the data more clearly. The increase in suicide-related visits after 2016 is particularly large and is more sustained that any changes in suicides, suggesting that there was a divergence between reports of suicidal behaviors and actual completed suicides.
Figure 1B divides teen suicide-related visits into three categories: visits for self-injury or intentional self-harm, visits where suicidal ideation is the primary diagnosis, and visits where suicidal ideation is the secondary diagnosis. Suicidal ideation is considered a symptom and should not be listed as primary if there is another confirmed diagnosis. However, half of these SI primary visits had another diagnosis code listed, indicating that they were coded as primary in error. This coding error is discussed further below; Figure 1B takes the data as given.
One can see in Figure 1B that the trend in self-injury and intentional self-harm, which includes attempted suicide, is essentially flat. Visits with suicidal ideation as the primary diagnosis trend upwards gently and smoothly throughout the period. Only visits with suicidal ideation as the secondary diagnosis show a break in trend. These visits had been trending up slowly since 2008, increased after 2011, but then leveled off again by 2016. They then jumped between 2016 and 2017, continuing to climb between 2017 and 2018 before leveling off in 2019.
Thus, it appears that the trend breaks in aggregate suicidal behaviors shown in Figure 1A are mainly accounted for by increases in the use of suicidal ideation as a secondary diagnosis code. These increases, in turn, are in line with the 2011 directive to screen girls and with the October 2016 recommendation to include SI where present with other primary mental health diagnoses. This finding is significant because it suggests that much of the rise in reported suicidal behaviors after 2011, and especially after 2016, is in fact due to changes in screening for and coding of suicidal ideation rather than to an underlying change in actual suicidal behaviors.
Figure 1C confirms that the increased use of suicidal ideation as a secondary code occurs primarily when the primary diagnosis is a mental health condition. The increases in the use of the SI code as a secondary diagnosis when mental health conditions are primary are quite clear and occur mainly between 2011 and 2013 and after 2016. The use of suicidal ideation as a secondary diagnosis when other nonmental health conditions are primary appears to be quite flat in the figure, though this is partly due to the fact that the code is rarely used when non-mental-health visits are primary.
Table 1 shows some of the numbers underlying these figures. The first row indicates that while the overall number of ED visits declined over our sample period, the number of visits with any suicide-related code increased between 2009–2010 and 2014–2015, and again between 2014–2015 and 2018–202019. For example, between 2009–2010 and 2014–2015, the number of suicide-related visits increased by 14 per 10,000 teens, with all of that increase accounted for by an increase in suicidal ideation. Between 2014–2015 and 2018–2019 the number of visits with a suicide-related code grew by 50 percent, from 50.3 per 10,000 teens to 75.6 per 10,000 teens. Again, the bulk of this increase, 24.9 out of 25.3 visits, was due to an increase in diagnoses of suicidal ideation, and 18.5 of these were secondary diagnoses. Panel A confirms that most of the secondary diagnoses of SI were for teens with primary diagnoses of mood disorders or other mental health disorders.
Teens’ Suicide-Related and Total Hospital Visits by Reported Primary Diagnosis
The remaining increase in visits came mainly from a rise in visits with a primary diagnosis of suicidal ideation. As discussed above, a significant fraction of these diagnoses seem to have been coded in error because SI is not supposed to be coded as primary if there is another diagnosis on record. Panel B of Table 1 corrects for these apparent coding errors by reassigning cases with SI primary to SI secondary if there is another diagnosis. For example, if the visit had SI primary and a mood disorder secondary, we treat the visit as if the mood disorder were primary and SI secondary, since this is arguably the correct coding. With this corrected coding, it is even clearer that almost all of the increase in suicide-related visits came from increases in secondary diagnoses of SI in cases with primary mental health disorders. For example, between 2014–2015 and 2018–2019, 22.7 out of a total increase of 25.3 visits were for SI as a secondary diagnosis.
Since the increases in suicidal ideation are mainly in teen visits with primary diagnoses of mood disorders or other mental health disorders, Figure 2 focuses on these visits and looks at which groups were most affected by the increases in the number of secondary diagnoses of SI. We look at visits by gender, race, insurance status, and by whether the hospital had a psychiatric unit or not. Because we do not have accurate population denominators for each of these groups (for example, by insurance status), this figure focuses on trends in the number of visits.
Teens with Primary Diagnoses of Mental Health Disorders and Secondary Diagnosis of Suicidal Ideation by Gender, Race, Insurance Status, and Hospital Type
Notes: These figures show mood-disorder visits with a suicidal ideation diagnosis by gender, race, insurance status, and hospital type in New Jersey. The vertical lines at 2011 and 2016 help to visualize the changes related to the implementation of the Women’s Preventive Services Guidelines in 2012, as well as the difference between 2015 and 2016 (implementation of ICD-10) and between 2016 and 2017 (implementation of the “include SI” guidance).
The counts underlying the figures appear in Online Appendix Table A2. Online Appendix Table A3 presents a regression analogue of the figures in which the various demographic, insurance, and hospital-level factors are interacted with year dummies. For example, the first panel of Online Appendix Table A3 presents estimates from a model similar to Equation 1, except that we use the full sample of teen hospital visits for mental health from 2008 to 2019 and include interactions of an indicator for “female” with indicators for each year from 2012 to 2019. Subsequent panels interact the year dummies with indicators for race, insurance status, and whether the hospital has a psychiatric unit.
The motivation for focusing on these groups is as follows. First, it is natural to examine differences by gender. Girls are known to have higher rates of mood disorders, while boys are more likely to have a fatal suicide attempt. Also, as discussed above, there were differences in screening recommendations that affected girls and boys differentially.
Dividing the sample by race is motivated by several considerations. First, there is a large literature showing the Black patients are likely to use different facilities than white patients and are often treated differently than white patients even within facilities. These disparities extend to mental health care (McGuire and Miranda 2008). Second, Black patients might exhibit different care-seeking behavior, if for example, people in the Black community feel more stigma around mental health problems. Both of these issues might cause Black patients to be less likely to be coded as having suicidal ideation.
There is also good reason to suppose that coding patterns might vary with insurance coverage. In particular, given that hospitals often end up writing off charges to self-pay patients, they may have little incentive to code complex diagnoses accurately in these patients. Hence, one might expect additional coding of suicidal ideation only in patients with insurance.
Finally, hospitals with psychiatric units have psychiatrists on staff and have personnel who are trained to do psychological assessments. Other hospitals may lack these specialized staff and so may be less equipped to assess conditions such as suicidal ideation.
Figure 2A shows that secondary diagnoses of SI in teens with primary diagnoses of mental health disorders rose among both girls and boys but showed much greater increases among girls. This finding is consistent with the fact that girls became more likely to be screened for depression after the new guidelines went into effect. Receiving a primary diagnosis of a mood disorder such as anxiety and depression puts them at higher risk of getting a secondary diagnosis of SI.
The first panel of Online Appendix Table A3 shows that the interactions between the indicators for female and the year dummies are positive and statistically significant for all years after 2012. Relative to the pre-2012 mean for girls (equal to the mean for boys plus the main effect on the female indicator), diagnoses for suicidal ideation as a secondary diagnosis rose by an average of 25 percent in 2012–2016. In 2017–2019, they were 41.6 percent higher than the baseline.
Looking by race, Figure 2B shows that both Black and white children, as well as children of other race, saw increases in secondary diagnoses of SI, though the increases were greatest for white teens, followed by teens of other race, with the smallest increases among Black teens. Panel B of Online Appendix Table A3 confirms that Black teens experienced a smaller increase in SI diagnosis after 2012, with the interactions between the indicator for Black race and the year dummies being negative and jointly statistically significant (p-value of 0.025) in the model that includes hospital indicators. The size of the disparity is numerically and statistically insignificant prior to 2012, but grows in magnitude over the sample period, reaching about 41.3 percent by 2018. Unfortunately, New Jersey hospital discharge data do not include a reliable indicator for Hispanic/Latino ethnicity, so we were unable to look at this important group separately.
Figure 2C shows that there were similar initial increases in diagnoses of suicidal ideation as a secondary diagnosis among both privately insured children and children with Medicaid coverage, although the raw numbers for children with Medicaid coverage declined after 2018, while those for children with private health insurance continued to rise. Although the “self-pay” group is small, as 90 percent of New Jersey children have health insurance, it is interesting to note that there were no increases in secondary diagnoses of SI in this group.
Panel C of Online Appendix Table A3 confirms that self-pay teens experience a smaller increase in SI diagnosis, with the interaction coefficients for self-pay being negative and jointly marginally statistically significant (p-value 0.093) in models with hospital fixed effects. In contrast, interactions between Medicaid and the year indicators are not jointly statistically significant.
Finally, Figure 2D shows that the hospitals with psychiatric units were more likely to make diagnoses of suicidal ideation as a secondary diagnosis over the entire sample period. The gap gradually widens up to 2016 and then increases sharply between 2016 and 2017.8 This pattern suggests that hospitals with on-staff psychiatric expertise increased their use of the secondary SI diagnosis code faster after 2016 than other hospitals.
Online Appendix Table A3 shows that the interaction terms between the indicator for having a psychiatric unit and the year dummies only become statistically significant after 2016. This finding confirms that hospitals with psychiatric units started using suicidal ideation as a secondary diagnosis code much faster than other hospitals after 2016. However, the joint F-test for coefficients from 2012 to 2019 is not statistically significant.
While there is some possibility that teens experiencing a mental health crisis would tend to select hospitals with psychiatric units, patients are generally advised to go to the nearest hospital, and all hospitals are able to triage mental health patients who present at the ED. We have also verified that there was no significant change in the distance between a teen’s home address and the hospital they visited between 2016 and 2017.
Together, these figures suggest that among patients with mental health disorders, one of the main demographic factors that predict a secondary diagnosis of SI is being female. A possible explanation is that female children are both at higher baseline risk of mood disorders and are more likely to be screened. Black children are consistently less likely to receive these diagnoses than white children treated in the same hospitals. Estimates for self-pay patients suggest that they are less likely to receive a diagnosis of SI, but given the small numbers of children in this category, our results are imprecise. In addition, patients who presented at psychiatric hospitals had consistently higher rates of secondary diagnosis for suicidal ideation. This gap widened dramatically after 2016, when hospitals with psychiatric units were much more likely to increase the use of SI as a secondary diagnosis.
As discussed above, it is possible that the U.S. Department of Health and Human Services, Health Resources and Services Administration (HRSA) 2011 requirement that insurance plans cover the costs of recommended screenings combined with new recommendations to screen adolescent girls and women for depression in 2012 had an impact on diagnoses of mood disorders, anxiety, and suicidal ideation, leading, in particular, to increased screening for girls relative to boys.
Figure 3 shows that before 2012, the trend in the number of mental health visits was quite similar for boys and girls. This is true for total visits (Panel A) and for visits for depression (Panel B), anxiety (Panel C), and suicidal ideation (Panel D). However, after 2012, visits for girls jump relative to visits for boys and stay higher, which is consistent with greater screening of girls after 2011. Interestingly, Panel D shows that the gender gap in suicidal ideation visits narrowed after 2016, suggesting that the new guidance to code SI as a secondary diagnosis may have had an even larger percentage impact on boys than on girls.
Trends in Teen’s Mental Health Visits by Sex
Notes: These figures plot trends in mental-health-related visits in New Jersey by sex at birth. The vertical lines at 2011 and 2016 help to visualize the changes related to the implementation of the Women’s Preventive Services Guidelines in 2012, as well as the difference between 2015 and 2016 (implementation of ICD-10) and between 2016 and 2017 (implementation of the “include SI” guidance).
The next section provides a regression analysis of trends in the diagnoses of teen mental health conditions.
B. Regression Analysis
Table 2 focuses on documenting the opening of the gender gap in visits for mental health disorders after the new Women’s Preventive Services Guidelines were published in 2011. This analysis includes the subsample of all teen hospital visits from 2008 to 2015. The models include interactions between an indicator for female and an indicator for each year after 2011, as in Equation 1. Table 2 shows that there was a marginally statistically significant increase in mood disorder diagnoses in girls relative to boys in 2012, as well as a significant increase in the coding of suicidal ideation. The effect of being female was even larger in 2013. For example, in the base period of 2008–2011, 2 percent of boys visiting the hospital were diagnosed with a mood disorder compared to 3.5 percent of girls. By 2013, 4.7 percent of girl’s visits carried a mood disorder diagnosis, approximately doubling the gap between boys and girls. Similarly, the gender gap in diagnoses of suicidal ideation rose from 0.22 percent in the baseline period to 0.47 percent by 2013.
Impact of the Women’s Preventive Screening Guidelines for Depression on Mental Health Hospital Visits for Teens
So far, we have not adjusted the data for the fact that there might be repeated visits to the hospital by the same person, especially if they have suicidal ideation. Table 3 repeats the analysis from Table 2 taking only the first visit in a calendar year for each person. This restriction results in the loss of the 32 percent of visits that represent repeated trips to the hospital. The estimated gap in suicidal ideation between males and females after 2012 remains statistically significant, though it is approximately 60 percent of the size of the estimated gap in Table 2. Hence, the results suggest that teens who repeatedly visit the hospital are more likely to be diagnosed with suicidal ideation than those who make a single visit, perhaps because they are more likely to be screened. It is also possible that a teen who was previously coded as having suicidal ideation is more likely to be screened for SI on their next visit.
Impact of the Women’s Preventive Screening Guidelines for Depression on Mental Health Hospital Visits for Teens, Including First Visit in the Year Only
Table 4 provides a further exploration of the timing of the increase in the use of secondary diagnoses of SI for teens with primary diagnoses of mood disorders and other mental health disorders. These models are estimated using a sample that includes all teen hospital visits from 2013 to 2019. The regressions are in the form of Equation 2 and include indicators for each year and month, which captures underlying trends and seasonality in diagnoses of suicidal ideation. As discussed above, the year indicators are interacted with indicators for whether the primary diagnosis is a mood disorder or other mental health disorder.
ICD-10 Coding Changes and Suicide-Related Hospital Visits for Teens
The graphical analysis in Figure 1 showed that most of the increase in the use of the secondary SI code was in patients with mental health disorders. If the implementation of ICD-10 itself in October 2015 caused an increase in diagnoses of suicidal ideation, then there should be a significant interaction between mental health disorders and the indicator for 2016. If, on the other hand, it is the guidance to “include SI” when other mental health diagnoses are primary that matters, then there will be a significant interaction between the indicators for primary mental health diagnoses and the indicators for 2017, 2018, and 2019.
Columns 1 and 2 of Table 4 show the estimated coefficients on the interaction terms from regression models where each visit is an observation and the dependent variable is an indicator equal to one if the visit had a suicide-related diagnosis code. Column 1 includes all of the control variables, except for indicators for each hospital, while Column 2 also includes these hospital fixed effects. The estimates are very similar, suggesting that changes in coding were occurring primarily within hospitals. Both sets of estimates indicate that there was no significant rise in the use of suicide-related diagnoses between 2015 and 2016, but that there was a large and statistically significant increase between 2016 and 2017, which continued to grow until 2018 when it leveled off. Column 3 confirms that the estimates are very similar in magnitude when the dependent variable is an indicator for suicidal ideation as a secondary diagnosis. This result shows that most of the increase in reported suicidal behaviors is due to increases in secondary diagnoses of suicidal ideation.
Column 4 shows that, in contrast, there was no significant increase in self-injury, intentional self-harm, or suicide attempts among teens with primary diagnoses of mental health disorders relative to other teens after 2015 or in the following years. The increase in suicidal behaviors is entirely accounted for by increases in suicidal ideation as a secondary diagnosis in teens with other mental health disorders.
In terms of magnitudes, the coefficient of 0.1175 on the interaction between mood disorder and the indicator for 2018 in Column 3 indicates that secondary diagnoses of SI for teens with primary diagnoses of mood disorders rose by 57.5 percent relative to the estimated baseline effect prior to 2016 (0.2041). The comparable figure for teens with other primary mental health diagnoses implies an increase of 100 percent in the probability of having SI as a secondary diagnosis.
Table 5 shows results similar to Table 4 for suicidal ideation as a secondary diagnosis but excluding children with multiple visits. Column 1 repeats Column 3 of Table 4 for reference. Column 2 takes each child’s first visit within a calendar year. The estimates are actually quite similar to those using all visits. Column 3 uses a much smaller sample that includes only each child’s first observed hospital visit over the entire sample period (2008–2019). Again, the results are very similar. Hence, in terms of using the new coding standards, it appears that repeat visits were coded very similarly to new visits after 2016.
ICD-10 Coding Changes and Suicidal Ideation Diagnosis, First Visits Only
As Online Appendix Table A1 shows, there were additional changes to screening guidelines after 2012, which could theoretically have led to additional screening of adolescents and more diagnoses of SI. In particular, the American Academic of Pediatrics recommended screening all adolescents for depression in 2015 and again in 2018. These recommendations applied mainly to boys, since the HHS had already recommended the screening of girls. However, we see little evidence that these later AAP recommendations for screening increased diagnoses of suicidal behaviors. For example, the specific 2018 recommendation that all pediatricians screen for mood disorders was not accompanied by any increase in diagnoses of SI between 2018 and 2019. It is possible that the lack of a strong mandate for insurers to fund these screenings for boys prevented implementation of these guidelines.9
VI. Discussion
This study offers a careful examination of trends in suicide-related hospital and ED visits among children 10–18 in New Jersey between 2008 and 2019, just prior to the COVID epidemic. A limitation of the analysis is that it only includes hospital visits in New Jersey. However, the evidence suggests that these patterns may also hold in other U.S. jurisdictions. For example, Owens et al. (2020)21 report national data that groups suicidal ideation and suicide attempts together. As we have seen, this time series is likely to be dominated by reports of SI, given the vastly different baseline levels of these behaviors. The national data for children aged 15–19 indicate that these incidents increased by 6.6 percent per year between 2014 and 2016 but rose 28.7 percent between 2016 and 2017, consistent with the timing of the new coding instructions discussed here. Reports for children aged 10–14 show a similar pattern, increasing by 7.0 percent per year between 2014 and 2016 and by 21.4 percent in a single year between 2016 and 2017.
In New Jersey, much of the increase in suicide-related diagnoses of teens in the ED or hospital is shown to be driven by sharp increases in secondary diagnoses of suicidal ideation among children who had a primary diagnoses of a mental health disorder. One uptick in these diagnoses occurred after 2011, and a second much larger increase occurred after 2016.
Importantly, the large increases in reported suicidal behaviors after 2011 have been widely interpreted as increases in actual underlying suicidal behaviors. This analysis suggests that they may in fact be due to changes in screening as well as changes in the diagnostic coding of mental health disorders that more accurately capture rates of suicidal behaviors that have been gently trending upwards. This conclusion is consistent with the observation by Richter et al. (2019) that the actual prevalence of adult mental health disorders (many of which start in childhood) has shown only small increases over time.
There are several factors that could have been implicated in the increasing number of suicide-related visits after 2016. Reductions in stigma over time may be an important contributor to the rise in mental health diagnoses generally, but this would be expected to have gradual rather than sudden effects. Expansions of health insurance under the ACA should primarily have impacted adults in New Jersey, since children already had relatively generous public health insurance coverage.
The ACA’s requirement that insurers cover recommended services, combined with new recommendations to screen adolescent girls for depression, is associated with an upward tick in diagnoses of suicidal ideation that is much more pronounced among girls than among boys. However, subsequent screening recommendations that would have applied mainly to boys (since it was already recommended that girls be screened) appear to have had little effect on rates of diagnosis. A possible explanation is that boys do not suffer from SI, but the large increase in diagnoses of suicidal ideation for both boys and girls after 2016 suggests that this is not the case.
These large increases in diagnoses of suicidal ideation after 2016 were associated with changes in coding instructions that focused on the coding of suicidal ideation in patients with primary diagnoses of other mental health disorders. There was, however, no parallel change in self-injury, intentional self-harm, or suicide attempts, nor were there changes in the incidence of visits with underlying primary mental health disorders. Hence, the sharp increase in suicidal ideation is likely to reflect the change in coding rather than a change in underlying suicidal behaviors. Previous work examining the role of the ICD-10 in the increase in recorded mental health disorders focused on the introduction of the ICD-10 in October 2015 and not on the changes in instructions that occurred one year later. As far as we can determine, the relationship between the change in coding instructions and the huge jump in reported suicidal behaviors has been previously overlooked.
These results should not be interpreted in a way that casts doubt on the seriousness of the youth mental health crisis. Rather, they suggest that there have always been large numbers of youths with serious mental health issues and that recent changes in the way that children are diagnosed may be bringing more cases to light. Ironically then, the sharply higher rate of suicide-related diagnoses after 2016 may be a positive rather than a negative development if it means that more children in crisis are being diagnosed and treated.
It is possible that greater openness about the prevalence and seriousness of the youth mental health crisis could be one positive legacy of the social changes wrought by the COVID-19 pandemic. Given evidence that some interventions can reduce suicide (Ludwig, Marcotte, and Norberg 2009; Garraza et al. 2019; Rees, Sabia, and Kumpas 2022), it is to be hoped that increases in diagnosis and treatment of suicide-related disorders such as suicidal ideation will be accompanied by reductions in youth suicides and self-harm.
Acknowledgments
Sponsored by the NOMIS Foundation and the Center for Health and Wellbeing at Princeton University.
The authors thank Barbara Bolden, Darrin Goldman, and Yong Sung Lee from the New Jersey Department of Health for assistance in accessing the data and the NOMIS Foundation for funding this research. All interpretations and any errors are solely the responsibility of the authors. The authors have nothing to disclose. Data for this project are confidential but can be accessed through a process outlined at https://iphd.rutgers.edu/application-process. The replication code is available as an Online Appendix of Replication Materials.
Footnotes
↵1. In other words, youth suicides rose nationally but not in New Jersey. One possible reason is that New Jersey has relatively strict gun control laws and ranks 48th in terms of gun suicides and gun suicide attempts (Everytown for Gun Safety 2021). According to 2018–2020 data from CDC Wonder, the five states with the lowest teen suicide rate were Massachusetts, New Jersey, New York, Connecticut, and California, with rates between 5 and 6.6 per 100,000, while the states with the highest teen suicide rate were Utah, Wyoming, South Dakota, Montana, and Alaska, with rates between 22 and 40.4 per 100,000. These statistics suggest that regionally specific factors are driving much of the overall increase in teen suicides. However, we argue below that the increases in reporting of suicidal behaviors are of a different order of magnitude and are actually similar nationally.
↵2. Diagnosis of mental illness is objectively difficult, and doctors make many mistakes. For example, it often takes more than eight years for patients with bipolar disorder to be diagnosed correctly and treated (Fritz et al. 2017).
↵3. Some codes, when added to a claim, result in the event being a “Complication or Comorbidity,” and this in turn increases reimbursement. Moreover, it is perfectly legal for providers to code in a way that maximizes their reimbursements, as long as the patient actually does have all the conditions listed (Boggs 2019).
↵4. As Zeller (2018) states: “‘If you’re having a psychiatric emergency, hang up and dial 911, or go to your nearest emergency room.’ Most psychiatric patients are all too familiar with this voicemail message. And those who follow this advice, even calling 911, will still likely find themselves at the nearest emergency department (ED).”
↵5. From 2008 through 2015, we use the following ICD-9 codes: V62.84 (suicidal ideation) and E950–E959 (the self-inflicted external cause of injury codes). From 2016 to 2019, we use the following ICD-10 codes: R45.851 (suicidal ideation), T14.91 (suicide attempt), and T36.xx2–T50.xx2, T51.xx2–T65.xx2, and X71–X83 (intentional self-harm).
↵6. In particular, we use the ICD-9 codes 293.83, 296.xx, 300.4, and 311 for 2009–2015. For 2016–2019, we use the ICD-10 codes F30–F34, F39, and F0630.
↵7. For comparison, we also compute similar rates for other age groups as shown in Online Appendix Figure A2. The figure shows the trend in suicide-related visits per 10,000 teens aged 10–18 from 2008 to 2019 in comparison with people 19–64 and those 65 and older. The figure illustrates that such visits had been slowly trending up since 2008 and were fairly similar for teens and adults. However, for teens, there was a large jump in the rate between 2016 and 2017, coinciding with the directive to code SI as a secondary diagnosis. There also appears to be a rise in the rate between 2016 and 2017 among adults and seniors, though it is less pronounced. As discussed above, there may be compositional changes in the adult sample after 2014 due to the ACA, and the rise in reported mental health problems among teens is of particular interest, so in this study we focus on teens.
↵8. We chose to measure the indicator for whether a hospital had a psychiatric unit at a fixed point in time just prior to the changes in ICD-10, so that hospitals were not switching in and out of the category. In 2014, 28 hospitals had psychiatric units, and 22 of the 28 had them continuously from 2014 to 2019. The estimates are similar if we an indicator for the contemporaneous presence of a psychiatric unit instead.
↵9. In an attempt to further examine the role of screening, we also examined visits with a code of V70.1 in ICD-9 or Z04.6 in ICD-10. This billing code, “General psychiatric examination, requested by authority” is used when a psychiatric examination has been recommended and performed, but no evidence in support of a psychiatric diagnosis is found. Although it is relatively rarely used, one might expect this diagnosis to have increased if more teens were being screened because only some of those screened would have been found to have psychiatric diagnoses. However, we did not find any statistically significant increases in the use of this code.
- Received April 2023.
- Accepted July 2023.
This open access article is distributed under the terms of the CC-BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0) and is freely available online at: https://jhr.uwpress.org.
