Adverse Impacts of Mental Health Needs Assessment on Jail Outcomes

A. Mental Illness of Transition Age Youth

Transition age youth, individuals aged 17–25 years old, sit at the confluence of two sets of risks—both a period of emergence or worsening of mental health issues and a period of increasing interaction with the criminal justice system. This period represents a delicate moment in the lives of these individuals, as the worsening symptoms may lead to an arrest prior to reduction in symptoms.

Regarding the first set of risks, a recent global meta-analysis found one-third of individuals have their first mental disorder before age 14, with most being neurodevelopmental. The modal age of onset for substance use, personality, mood, and schizophrenia-spectrum and other psychotic disorders is around 20 years old (Solmi et al. 2022). Almost two-thirds of individuals developed their first mental disorder before the age of 25.

The second risk is that this period corresponds to the peak years of criminal activity. Peak age–crime distributions are below 25 years old for all crimes except gambling in the FBI UCR database (Ulmer and Steffensmeier 2014). Further, the age–crime distributions have been trending towards younger ages. The peak age–crime distribution and trends appear to be driven by a range of biological, sociological, economic, and demographic factors.

As a result of these two sets of risks, significant numbers of youth with mental illness end up in jail. The prevalence estimates for individuals with mental illnesses within criminal justice settings are dramatically higher than the general community (Prins 2014). Consequently, jails have become the de facto mental hospital of last resort.

B. Large Urban County Jail

The large urban county jail is a 3,050-bed capacity facility that services the county’s detention needs, has been in operation for several decades, and is spread over two sites. It averages about 2,150 inmates per day across these sites.

Nearly everyone who is arrested and charged for a misdemeanor or felony in the county will arrive at the jail for “booking” and bail setting.⁸ Booking is the process where information about the individual is entered into police records upon arrival. Once an individual has been arrested and booked, the arresting officer will file a probable cause affidavit with the court. The affidavit is then reviewed by a judge, who also investigates the defendant’s criminal history and sets the bail. There is no set bail schedule, and the amount could be more or less depending on the judge who initially reviews the case.

Once the judge has set the bail amount, a defendant may be released from jail by personal bonds, cash bonds, lawyer-assisted bonds, or surety or bail bonds. A personal bond is a sworn agreement by the defendant that they will return to court as ordered and will comply with the conditions placed on their release. No money is required at the time of release, but the defendant is required to pay an administrative fee of the bond amount to the jail within a certain number of days of release. The county’s pretrial services or an attorney may submit a request for release on personal bond to a judge, but defendants cannot request a personal bond themselves. Factors influencing the availability of a personal bond include residence in the community, employment, and criminal history. Only a judge can approve release on a personal bond. In cases where a personal bond is not available, bond must be posted for the individual to be released.

Inmate mental health is evaluated within 36 hours in compliance both with the jail’s own historic administrative policies and with statewide regulations. The formal process of booking starts with a topical mental health screening by the officer receiving the individual into residency and then followed by an interview with a randomly assigned trained clinician. Clinicians employed at the jail all have a minimum of a masters of social work.⁹ Most of the interviews are done by clinicians who have substantial experience with these intake interviews. In a robustness check, we dropped clinicians with fewer than 500 interviews in our sample. In doing so, we lost 5 percent of the sample, indicating that 95 percent of the interviews are done by clinicians with substantial experience.

Clinicians are alphabetized daily by last name and then assigned one-to-one to a newly booked inmate as they arrive at the facility, creating a quasi-randomization of intake clinician to inmates at booking. The purpose of the intake clinician is to screen inmates and collect information, not to treat mental health problems. These clinicians use a structured survey questionnaire developed by the facility to assess the mental health needs of each inmate. The questionnaire also includes the Columbia Suicide Severity Rating Scale (Posner et al. 2008) to provide the facility with information about suicide risk. A short description of the survey is contained in the Online Appendix. Completion of the interview can take anywhere from 15 to 30 minutes, depending on the inmate and their responses.

Based on the survey and their own subjective judgment, clinicians produce a mental health needs score. Subjective judgement can come both in the scoring of individual items and in the aggregation of these items into a final score. Clinicians classify the inmate into one of four groups: 0 (“high daily functioning”), 1 (“mild functioning problems”), 2 (“moderate functioning problems”), or 3 (“severe functioning problems”). For the empirical analysis, we collapse the mental health needs score into a dummy variable equal to zero if none/mild and one if moderate/severe. Inmates with moderate/severe scores and charged with misdemeanors are routed to the county’s mental health docket.¹⁰ As a result, this is a natural comparison and is salient to officials at the jail.

C. National Data on Mental Illness and Comorbidities

Mental illnesses include many different conditions that vary in degree of severity and may also be comorbid with alcohol or substance abuse. Two broad categories describe such conditions: any mental illness and serious mental illness. Any mental illness includes all recognized mental illnesses, while serious mental illness is a smaller and more severe subset of any mental illness. More specifically, serious mental illness is defined as a mental, behavioral, or emotional disorders resulting in serious functional impairment, which substantially interferes with or limits one or more major life activities.

In 2020, young adults (18–25) had higher incidence of mental illness with and without substance use disorder than adults (ages 26–49) in the past year (SAMHSA 2021). The incidence of any mental illness without substance use disorder was 18.4 percent for young adults and 16.1 percent for adults. The incidence for having both mental illness and substance use disorder was 12.2 percent for young adults and 9.2 percent for adults. The differences also hold for serious mental illness without substance use disorder, where the incidence was 5.1 percent for young adults and 3.7 percent for adults, and for serious mental illness with substance use disorder, where the incidence was 4.6 percent for young adults and 3.2 percent for adults. Young adults are more likely than adults to have received inpatient care in the past year, 1.4 percent versus 1 percent. Finally, young adults and adults are equally likely to have taken prescription medication for mental illness in the past year, 14.5 percent versus 14.4 percent.

In 2020, young adults were more likely than adults to have had suicidal thoughts, a suicide plan, and attempted suicide in the past year (SAMHSA 2021). Among young adults, 11.3 percent had serious thoughts of suicide, 4.0 percent made a suicide plan, and 1.9 percent attempted suicide. Among adults, 5.3 percent had serious thoughts of suicide, 1.3 percent made a suicide plan, and 0.4 percent attempted suicide.

Deaths from suicide, and from alcohol and drug intoxication, in the overall population have been trending up over time and are lower for youth than adults (CDC 2021). The mortality rate from suicide was 10.7 per 100,000 in 2001 and 13.5 per 100,000 in 2020. Alcohol-induced deaths rose from 7.1 per 100,000 in 2000 to 10.4 per 100,000 in 2019 and further to 13.1 per 100,000 in 2020 (Spencer et al. 2022a). The mortality rate from drug intoxication rose from 6.1 per 100,000 in 2001 to 21.6 per 100,000 in 2019 to 28.3 per 100,000 in 2020 (Spencer et al. 2022b). It is difficult to make age-based comparisons directly with the previous statistics because mortality statistics are often reported for different age ranges. That said, the rates of death are lower for those 15–24 years old than those 25–39 years old. In 2020, the suicide rate was 14.2 per 100,000 for ages 15–24 and 18.4 for ages 25–34. For both alcohol-induced deaths and drug intoxication, the rates were also lower for youth than adults.

In jail, young adults (18–24) also have lower rates than adults (25–34) of death from suicide and death from alcohol and drug intoxication per 100,000 inmates (Carson 2021). For young adults, average mortality rates over 2000–2016 were 28 per 100,000 for suicide and five per 100,000 for alcohol and drug intoxication. For adults, the rates were 41 per 100,000 for suicide and ten per 100,000 for alcohol and drug intoxication. Suicide rates in jail for all ages were relatively stable over time—they were 48 per 100,000 in 2000 and 46 per 100,000 in 2016. In contrast, deaths from alcohol and drug intoxication for all ages rose from six per 100,000 in 2000 to 15 per 100,000 in 2016. It is worth noting that the patterns of death for the two causes were quite different. The median time in jail for deaths from alcohol and drug intoxication was one day, while it was nine days for deaths from suicide.

A. Data

We use de-identified administrative data from a large urban county correctional complex (“jail”), which encompasses the universe of inmates booked in the county between 2016 and 2019. These data were collected during routine mental and physical health assessments on inmates and include information on each inmate’s offense type (felony, misdemeanor), demographics, results from the clinician interview (mental health needs score), and outcomes, including length of stay, suicide attempts, and recidivism.¹¹ A unique inmate ID and unique booking ID are jointly used to identify a unique inmate booking event, as well as to link inmates over time. In our analyses, we focus on inmates who received a mental health assessment.¹²

Our main sample spans January 2016 to November 2019. It includes 79,571 inmate–booking observations; 20,430 are for youth and the remaining 59,141 are for adults. For some of the analysis, we limit the sample period. Richer data on mental health of the inmates including self-reported data on psychiatric medications and hospitalization became available in July 2018. This sample includes 31,608 observations; 7,597 observations are for youth and 24,011 are for adults.¹³ While the data we have are quite rich in some dimensions, it is worth noting the data we do not have. For inmate privacy reasons, we were unable to obtain data on medical, psychiatric, and psychological treatment in jail; housing assignments within the jail; specifics of charges beyond misdemeanor levels (A, B, C) and felony levels (1, 2, 3); and outcomes beyond jail, such as whether they were convicted. These are fundamental limitations on our analysis.

B. Youth and Adult Characteristics

Table 1 contains summary statistics for the full sample for 2016–2019 and shows that youth and adults in jail generally have similar demographic characteristics. Most inmates are male, white, and many have prior offenses within 365 days. Although the majority of inmates are white, Black inmates are significantly overrepresented relative to the general population. The county as a whole is 9 percent Black and 34 percent Hispanic. Across the four columns, inmates are 25–32 percent Black and 24–36 percent Hispanic. Compared to adults in jail, youth are more likely to be Hispanic. The Hispanic population is growing rapidly and is on average quite young. Youth who are charged with a misdemeanor are less likely to have been charged with a prior offense.

The distributions of crimes are also similar for youth and adults; 55 percent of youth and 53 percent of adults are charged with misdemeanors, and the remainder are charged with felonies. The charges are concentrated in more serious misdemeanors and lower-level felonies.¹⁴ Online Appendix Figure A.1 shows the aggregate distribution of case types for all ages.¹⁵ The county does not separately report misdemeanor thefts and felony thefts.

Furthermore, Online Appendix Tables A.2 and A.3 present descriptive statistics by the initial assessment of mental health needs (none/mild vs. moderate/severe) for misdemeanors and felonies, respectively. Interestingly, within both age groups (youth and adults), individuals assessed with moderate/severe mental health needs tend to be slightly older and have a higher number of prior convictions. These trends are consistent for both misdemeanors and felonies, indicating that more severe mental health needs and more frequent contact with the criminal justice system are interconnected.

C. Evaluation of Mental Health

Initial evaluations provide three measures of mental health: (i) the clinician evaluation of mental health needs; (ii) Columbia Suicide Severity Rating Scale, which measures suicide risk; and (iii) self-reported prior psychiatric hospitalization and use of psychiatric medication.¹⁶ Online Appendix Table A.1 shows that youth score lower than adults for mental health needs and self-reported hospitalization and medication but were similar on suicide risk. For mental health needs, 9–11 percent of youth score moderate or severe, as compared to 14–20 percent of adults. For psychiatric hospitalization, 13–14 percent of youth and 17–20 percent of adults report prior hospitalization. For psychiatric medication, 23–24 percent of youth and 32 percent of adults report taking medication. For suicide risk, 13–16 percent of youth score moderate or severe, as compared to 13–16 percent of adults.

In Tables 2 and 3, we include descriptive regressions for mental health needs ratings for both the sample we use in our main IV analysis (Table 2) and for the last five quarters of our data, where additional suicide measurements became available (Table 3). For individual i, booking b, and month–year time t, consider the following model: 1

where the inmate’s mental health needs score, Score_ibt is a 0/1 dummy variable, indicating that the clinician’s classification was none/mild (0) or moderate/severe (1). Score_ibt is related to an inmate’s booking and defendant-level characteristics, X_ibt, as well as month–year fixed effects τ_t, and ɛ_ibt, an error term. The baseline inmate characteristics are age at booking, sex, race, ethnicity, and the number of prior offenses.

Table 2 indicates that the mental health needs score is generally higher for older inmates, female inmates, non-Hispanic inmates, and inmates with prior offenses. Table 3 shows that the mental health needs score captures important features of inmate mental health. For example, suicide risk ratings have large positive and statistically significant coefficients. Prior psychiatric hospitalization also has a positive and statistically significant coefficient in most specifications.

D. Description of Outcomes

Table 1 indicates that, during their time at the jail, youth and adults have similar length of stay and recidivism rates. For misdemeanors, the average length of stay is eight days for youth and 11 days for adults. One-year recidivism rates are 37 percent for youth and 43 percent for adults. For felonies, average length of stay is 59 days for both youth and adults. One-year recidivism rates are 39 percent for youth and 35 percent for adults.¹⁷

Furthermore, Online Appendix Tables A.2 and A.3 present descriptive statistics by the initial assessment of mental health needs (none/mild vs. moderate/severe) for misdemeanors and felonies, respectively. Individuals initially assessed as moderate/severe typically have longer lengths of stay, more suicide attempts, and higher recidivism levels compared to those assessed as none/mild. These trends are consistent for both misdemeanors and felonies, suggesting that the severity of initial assessment (moderate/severe) is associated with more negative outcomes overall.

Table 1 reports that 0.57–0.61 percent of youth and 0.64–0.69 percent of adults attempted suicide while in jail. Our large urban county jail defines a suicide attempt as self-harm, expressed ideation, or other actions, regardless of how effective they might be.¹⁸ Note, since inmates stay 8–11 days for misdemeanors and 59 days for felonies, these attempts are happening over a fairly short time frame.

From January 2018 through November 2019, we have detailed information on what jail officials considered the most serious suicide attempts. The information from this subsample is summarized in Online Appendix Table A.8. During this time, 85 attempts occurred, of which 30 required medical intervention to address physical harm, psychiatric needs, or both at the jail, and 19 required hospitalizations. Similar shares of youth and adults appear in the data set, and they needed medical intervention at the same rate. The most common methods by which inmates attempted suicide were strangulation, banging their head, hanging, and cutting. At the end of the period covered by these data, the recording of serious suicide attempts shifted to electronic medical records. These data were not available to our research team.

Online Appendix Figure A.2 shows that the most frequent causes of death in this corrections facility are cardiac arrest, followed by suicide. Over our sample period, there were nine cardiac deaths, two suicides, and one death each from substance toxicity, hematoma, and hypertension.

A. Overview

To better understand the research design, let us consider a model in which an inmate i at booking b remains in jail longer, attempts suicide, or recidivates based on interventions given in response to their mental health needs score, Score_ibt.

2

where Y_ibt is the outcome of interest for individual i in booking b in month–year t: (1) length of stay, (2) suicide attempt, or (3) recidivism within a year of release. The matrix X_ibt contains baseline inmate and booking characteristics, τ_t is a vector of month–year dummies, and ɛ_ibt is an error term.

Ordinary least squares (OLS) estimates of Equation 2 cannot identify the causal effect of classification scores on outcomes if scores are systematically assigned to people already at an elevated risk level. Identifying the causal effect of the score requires exogenous variation. We use the inmate’s quasi-randomly assigned clinician’s residualized average mental health needs score as an instrument for an inmate’s own classification score. Under the standard IV assumptions, we identify the local average treatment effect associated with having moderate to severe mental health needs.

B. Instrumental Variable Calculation

We construct our instrument of clinician tendency to assign moderate to severe mental health needs classification scores. Our instrument is the residualized, leave-one-out mean intake clinician tendency measure conditional on month-of-year fixed effects. This method is similar to the judge leniency designs used in other criminal justice studies (Aizer and Doyle 2015; Arnold, Dobbie, and Yang 2018).

To isolate the effect of clinician scoring on the mental health needs score of inmate i at booking b in time t, we first estimate a linear model that includes a vector of month-of-year fixed effects. These fixed effects account for any countywide trends that might affect mental health needs scores: 3

where ɛ is an idiosyncratic error. We then residualize the inmate’s mental health needs score using the fitted coefficients from Equation 3: 4

This residualized score isolates the part of the inmate’s mental health needs score that is due to the clinician’s scoring, independent of the month-of-year fixed effects.

Next, we construct a measure of each clinician’s propensity to assign a high residualized score. This measure is the average residualized score assigned by the clinician, excluding the score for the inmate in question. We construct a measure of residualized clinician c propensity to assign a high score Z_btc as: 5

where n_tc is the number of cases seen by clinician c in month–year t, and n_itc is the number of bookings of inmate i seen by clinician c in month–year t.¹⁹ is the residualized mental health needs score that clinicians give at time t, and is the residualized score by inmate i. In other words, we remove the residualized mental health needs score assignment of all of an inmate’s bookings seen by clinician c in each month.²⁰

C. Clinician Leniency

The screening process itself is structured. Clinicians meet one-on-one with individuals within 36 hours after booking. During this 15–30-minute encounter, a specific set of questions are asked, and the clinician records the inmate’s self-reported answers.²¹ Despite this structured process, variation can still occur. Clinicians may vary in the recording of specific answers and in how they view the overall set of answers. They are not given specific guidelines for aggregating responses, such as summing the responses and comparing the number to a threshold. This variation can be influenced by a range of factors, including experience, workloads, and personal biases, such as implicit biases towards certain demographic groups or mental health conditions.

Figure 1 presents four distributions of normalized intake clinician residualized leniency measure for high scores stratified by each of our four subpopulations: youth misdemeanor, adult misdemeanor, youth felony, and adult felony. Moving from left to right on each of the subfigures shows the association between the share of the sample assigned a moderate to severe mental health needs score and intake clinicians whose average tendency is to assign worse classification scores. The spread contained in each of the histograms represents the spread of the intake clinician leave-one-out-mean for each of the subpopulations. If clinicians are randomly assigned to inmates and there is no scope for clinician subjectivity in assessing inmate functioning, there would be no variation in our instrument, as all clinicians would agree. Moreover, if clinician assignment is indeed random, then the positive slope in each of the subfigures represents the causal effect of the residualized leave-one-out-mean on the inmate’s own classification score.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Distribution of Clinician Leniency and First Stage

D. First Stage

We explore this positive association between the residualized leave-one-out-mean clinician scores and an inmate’s own score by estimating the following linear probability model: 6

where Score_ibt is the binary treatment variable (“classification score”), indicating whether an inmate received a mental health needs score of “Moderate” or “Severe,” and Z_btc is a vector of the residualized leave-one-out-mean clinician score, X_ibt is an array of pre-booking inmate characteristics, including race, sex, age at booking, whether they had a prior offense in the last year, the number of offenses per booking, τ_t are month-of-year fixed effects, and ɛ_ibt is the inmate specific error term. Standard errors are two-way clustered by clinician and inmate. Since intake clinicians are quasi-randomly assigned, the first stage has a causal interpretation. Thus, Equation 6 is a linear estimate of the average treatment effect of being assigned a clinician with a higher average mental health needs score on one’s own mental health needs score.

The first-stage relationship is very strong for youth and adults across both misdemeanor and felony offenses. Table 4 shows the first-stage results for misdemeanor populations. Our instrument has a standard deviation of 0.101 in the misdemeanor sample, and we multiply that by the first-stage coefficient to better interpret the magnitudes associated with the leniency instrument.

Shifting clinician leniency one standard deviation would cause a 7.7–7.9 percentage point increase in youth with misdemeanors being classified as moderate to severe mental health needs, and an 8.7–9.9 percentage point increase in adults with misdemeanors. The Kleibergen–Paap first-stage F-statistics for misdemeanors are 575–715 for youth and 981–861 for adults, which are significantly higher than commonly employed benchmarks.²² Table 5 shows the same statistics for the felony cases. A one standard deviation (0.098) increase in clinician leniency caused an 8.4–8.6 percentage point increase in receiving a high score among youth with felonies and a 9.2–9.6 percentage point increase among adults with felonies. The effective F-statistics for these two subpopulations are also very high and range from 370 to 455.

E. Instrument Validity

Imbens and Angrist (1994) delineates three additional assumptions for the IV approach to identify the local average treatment effect (LATE). The first assumption posits that the instrument is as good as randomly assigned. This random assignment, or independence, allows both the first stage and the reduced form to possess causal interpretations. We offer indirect evidence supporting this assumption through randomization tests and balance tables.

Tables 6 and 7 provide support that the assignment of clinicians is random for youth and adults. In Table 6, we evaluate the correlation between an inmate’s classification score and baseline inmate covariates, specifically among youth and misdemeanors. As expected, individuals with a more severe mental health needs score exhibit differences along observable dimensions compared to those with less severe scores. In the second column, we regress the residualized leave-one-out-mean intake clinician score onto the same inmate characteristics. The correlation weakens, with many coefficients becoming insignificant or zero, and the joint significance of all covariates in the model yields an F-test value of 2. In the third and fourth columns, we conduct randomization tests for adults with misdemeanors. The F-test value for the excludability of covariates drops from 33 to 4, and any discernible differences are relatively small compared to a covariate measurement, such as age in years. Table 7 further provides evidence supporting randomization for both youth and adults with felonies.

In Tables 8 and 9, we follow Aizer and Doyle (2015) and use an alternative test for randomization by showing that covariate means are balanced across the bottom, middle, and top terciles of the instrument’s distribution. The mean values of covariates are roughly equivalent across the instrument distribution. In certain instances, such as age for misdemeanors and felonies, and race for felonies, the values exhibit statistical differences, albeit small. However, the balance is significantly less for felony priors.

While the principle of independence lends a causal interpretation to the reduced form equation, the second assumption, exclusion restriction, confines the sole causal pathway to the inmate’s mental health needs score. This is a challenging assumption to uphold, as there are no straightforward tests, such as balance tables, which can help validate it.

One such violation pertains to the “cohort” of other inmates used to generate the clinician’s leave-one-out mean. Since these inmates will all receive systematic scores, any interaction between the inmate and that cohort could introduce bias through peer effect channels once the inmate is placed in a specific type of housing within the jail. However, it is important to note that housing assignments are based on the inmate’s exogenous criteria, not the clinician’s decision, so this problem of mixing is more likely to occur with other inmates in the same housing unit than it is with other inmates from the “clinical scoring cohort.” In other words, the clinician does not influence these other downstream factors, nor does the inmate’s “scoring cohort” necessarily follow them.²³

The final condition needed to interpret our estimates as a LATE parameter associated with the effect of higher mental health needs scores on outcomes is monotonicity. Monotonicity is a shape restriction without which the IV estimates do not have causal interpretations. But it also has a behavioral interpretation. In this context, monotonicity means that if there are two clinicians at intake, and one tends to give more moderate to severe mental health needs scores than the other, then that clinician would weakly always do so if we could rotate the inmates to be seen by the other clinician. This is not something that can be directly tested because it involves counterfactual inmate scoring by different therapists than the ones they had seen. A commonly employed indirect test is to evaluate whether the first-stage estimates are nonnegative for all subsamples. We present that evidence in Table 10 using the full sample of inmates to calculate our measure of clinician leniency. All first-stage coefficients are nonnegative and highly significant for all subsamples of youth misdemeanors and adult misdemeanors. Differences in magnitudes represent stronger roles for clinician discretion, and Black inmates are the ones with the largest magnitudes among misdemeanors. Table 11 shows that the first-stage relationship is also nonnegative for all felony subsamples.

F. Instrumental Variable Modeling

We present two IV models—two-stage least squares (2SLS) and instrumental variables LASSO (IVLASSO). In a just identified 2SLS model, the residualized leave-one-out-mean is similar to a propensity score that absorbs information from all clinician fixed effects into a single scalar. If any of these clinician fixed effects are weak predictors of mental health needs scores, then the bias of 2SLS will be centered on the bias of OLS.²⁴ IVLASSO, on the other hand, is robust to many instruments, as well as many controls (Chernozhukov, Hansen, and Spindler 2015). When using IVLASSO, our instrumental variables are the full vector of intake clinician fixed effects.

One of the criticisms of the LATE as a policy parameter is that it describes the causal effects of an unknown and invisible subpopulation, the compliers. As we do not observe a person’s potential treatment status under different instrument values, we cannot directly identify who is and is not a complier. However, we can learn information about their mean characteristics by regressing X_iD_i on D_i using 2SLS (Angrist, Hull, and Walters 2023). Online Appendix Tables A.4 and A.5 lists those estimates. In general, the compliers do not look vastly different than the overall population.