The MacArthur Violence Risk Assessment Study Revisited: Two Views Ten Years After Its Initial Publication

The MacArthur Violence Risk Assessment Study, which was conducted between 1992 and 1995, followed 951 psychiatric patients ages 18 to 40 after discharge from state and university psychiatric inpatient units in Kansas City, Pittsburgh, and Worcester, Massachusetts. Participants were interviewed before discharge and recontacted at their community residence every ten weeks for one year. The primary diagnoses of the participants were schizophrenia, 17%; bipolar disorder, 13%; depression, 40%; abuse of or dependence on alcohol or drugs or both, 24%; and other diagnoses, 6%.

The purpose of the study was to ascertain the prevalence of community violence in a sample of people discharged from acute psychiatric facilities. Violence was ascertained by interviewing participants and collateral informants (most often a family member) and through arrest and rehospitalization records. To compare the prevalence of violence attributable to the discharged patients, a random community sample was obtained in Pittsburgh, one of the study sites, of persons living in the same sections of the city as the discharged patients. The community sample was interviewed regarding violent acts; collateral informants were also interviewed.

In 1998 the initial findings of the study were published in the Archives of General Psychiatry ( 1 ). The study has been widely cited by advocates as proof that discharged psychiatric patients are not more dangerous than the general population. Since the original publication of the study, additional papers, book chapters, and a book about the study have been published. These publications were examined to ascertain whether the study's original conclusion needs to be modified and whether additional conclusions can be drawn from the expanded data.

Six points of interest

Was the prevalence of violent behavior high?

At issue. The prevalence of violent behavior among the discharged patients in the original MacArthur Study was, in fact, high. The details of the violent acts committed by the discharged patients were made available in publications after the initial report. Among the 951 discharged patients followed up after discharge from the hospital, 262 (27.5%) "committed at least one act of violence while in the community" during an average follow-up period of 41 weeks ( 2 ). The 262 individuals were responsible for 608 separate violent acts; 21 individuals each committed five or more acts, including individuals credited with 21, 25, and 41 separate violent acts. In 52% of incidents, the acts consisted of "kicking, biting, choking, hitting, or beating up a target." In 26% of incidents, the acts involved the use of a weapon or threat with a weapon in hand; 3% of incidents involved forced sex, and "death to the target" was the outcome in 1% of incidents. The authors acknowledged that the number of violent incidents might have been even higher if those who committed the acts had not been rehospitalized or jailed during the study period, thus yielding "less time at risk for committing violence in the community" ( 1 ).

It is difficult to compare the prevalence of violent acts in the MacArthur Study with rates reported in similar studies in the United States because definitions of violence and diagnostic groups vary across studies. However, compared with rates in seven other major studies reported since 1992 ( 3 , 4 , 5 , 6 , 7 , 8 , 9 , 10 , 11 , 12 , 13 , 14 ), the incidence found in the MacArthur Study appears to be higher. The study that is probably most comparable is the Triangle Mental Health Survey, in which 169 individuals discharged from community and state psychiatric hospitals were followed for 18 months ( 3 , 4 , 5 ). Acts of violence were ascertained by both self-report and an interview with a "significant other." The 18-month prevalence of serious violence in that study was 14%, compared with the 12-month prevalence of 27.5% in the MacArthur Study.

Specifically regarding homicides in the MacArthur Study, six individuals were killed by three of the 951 discharged patients ( 2 ). This contrasts with the national homicide rate, which in 2005 was 5.6 homicides per 100,000 population. One perpetrator in the MacArthur Study, who had a diagnosis of schizophrenia, "had apparently been talking to the television," "had exhibited increasingly paranoid behavior," and "may have been delusional" at the time he killed his girlfriend. Another individual, who had a diagnosis of "depression with severe psychoactive substance dependence," had for two years been hearing multiple voices, had persecutory ideation, and "may have been delusional" when he shot two men, killing one. The third perpetrator, who had a diagnosis of psychoactive substance abuse, had shot his father at age nine and had been admitted to a psychiatric unit at age 21 for having stabbed his roommate. While being followed up during the MacArthur Study, he killed four strangers ( 2 ).

Response. While any killing is tragic, deciding when to refer to the prevalence of violence among any group in the population as "high" is a matter of judgment. It is worth noting that more than half of the general public (61 percent) believe that people with schizophrenia are likely to be violent to others ( 15 ). Compared with this widespread public perception, the prevalence of actual violent behavior carried out by the discharged patients in the MacArthur study was not, in fact, high. Rather than selecting a few individual cases out of the sample of 951 discharged patients to make the point, a more systematic way to assess the relationship between psychosis and violence would be to look at the proportion of discharged patients who were either delusional or hallucinating at the time of a violent incident. Those proportions—7.4% delusional and 5.2% hearing voices ( 16 )—are actually quite low.

It is also important to note the limitations of two of the comparisons cited by Torrey and Stanley. The official arrest figures cited provide no corrections for factors apart from mental illness, such as socioeconomic status, that may differentiate the sample of patients we followed from the population as a whole and that may account for some or all of the apparent difference in homicide rates. Also, the Triangle Mental Health Survey cited by Torrey and Stanley for a comparison did not include violence reported in official criminal justice records and mental hospital records as did the MacArthur data, which may account for the higher rates we reported. Finally, we underscore that most violence we assessed did not result in serious injury to the victim; severe injury or death was an uncommon outcome in this sample.

Was the comparison group appropriate?

At issue. In the MacArthur Study the prevalence of violent behavior among the discharged patients was compared with the prevalence among individuals living in very violent neighborhoods. In publishing the initial results, the authors acknowledged that the prevalence rates of violent acts "may appear high" but "are meaningful only in comparison to the prevalence of violence by other residents in the same community" ( 1 ). In Pittsburgh a matched community sample was selected of 519 people who were living in the same census tracts as the patients placed in that community. These 519 individuals and their collateral informants were interviewed once regarding violent acts. A comparison of the discharged patients (interviewed five times) and the community sample (interviewed once) showed a significantly higher prevalence of violent acts among patients who also had substance abuse, but not among those without substance abuse. This was the basis for the original study's conclusion that discharged psychiatric patients who do not have substance abuse are not more violent than residents of the same community.

The authors of the MacArthur Study acknowledged that "many of the neighborhoods" in which the patients and community sample resided "were disproportionately impoverished and had higher crime rates than the city as a whole" ( 1 ). We conducted areanalysis of the Pittsburgh data that showed that the patients and community sample were indeed living disproportionately in very high-crime neighborhoods. For example, 23% of the combined patient and community sample in Pittsburgh were living in census tracts 305, 314, 501, 502, 506, and 508–511 in the Hill District, in which less than 4% of all Pittsburgh residents reside. The Hill District's crime rate is among the highest in the city, as measured by aggravated assaults, drive-by shootings, and homicides ( 17 ). The fact that many of the patients and members of the community sample were living in such high-crime neighborhoods led Silver and colleagues ( 18 , 19 ) to conclude that "living in a socially disorganized neighborhood increased the probability of violence among the sample."

Response. As we stated in the original article ( 1 ), "Care should be taken in making patient-community comparisons. We sampled from the census tracts in which the patients resided after discharge…. We sampled in this manner to control for exposure to environmental opportunities for violence between the patient and comparison groups. The comparison group was not intended to be an epidemiologically representative sample of the general population of Pittsburgh." It would have made little sense to compare discharged patients living in poor neighborhoods with nonpatients living in wealthy neighborhoods, because any observed differences could be attributable to situational factors rather than to patient status.

In a 1998 article in the National Review ( 20 ) individuals affiliated with the Treatment Advocacy Center have previously taken issue with our conclusion that discharged psychiatric patients who do not have substance abuse are not more violent than residents of the same community. At the time, we responded in a letter to the editor ( 21 ), and our views have not changed over the succeeding decade: "Patients who were not substance abusers were no more likely to be violent than were their neighbors. Does this mean that we claim that mental disorder has nothing to do with violence? No. Mental disorder has a significant effect on violence by increasing people's susceptibility to substance abuse. When first discharged, patients were twice as likely as their neighbors to be abusing substances, and alcohol and drugs raised the risk of violence for patients abusing them even more than for others."

Was violence greater for diagnoses other than schizophrenia?

At issue. In the original report of the MacArthur Study, individuals who had schizophrenia were said to have a low prevalence of violent behavior compared with patients who had bipolar disorder or depression ( 1 ). The authors of the report noted that individuals with schizophrenia made up 17% of the study patients and that patients who consented to participate were "less likely to have a medical record diagnosis of schizophrenia." In a publication one year later, the MacArthur authors reported that the refusal rate among individuals with schizophrenia was 43.7%, significantly higher than the refusal rates for other diagnoses ( 22 ). In a subsequent publication, the MacArthur authors reported that "a diagnosis of schizophrenia was associated with lower rates of violence than was a diagnosis of depression or bipolar disorder" and that the difference was "statistically significant" ( 16 ). This has led subsequent researchers to also claim that "patients with schizophrenia had the lowest occurrence of violence" compared with patients with bipolar disorder or major depression ( 23 ).

In fact, the comparatively low rate of violent behavior among patients with schizophrenia was almost certainly a result of selection bias. The psychiatric patients who were included in the study had to sign consent forms and agree to an initial interview lasting approximately two hours and to follow-up interviews every ten weeks for a year; all interviews were to be tape recorded ( 16 ). Participants also had to give permission for the researchers to interview a family member or friend regarding their behavior every ten weeks. Most individuals with schizophrenia who have limited awareness of their illness (anosognosia) or paranoid symptoms—or both—are very unlikely to agree to such conditions, yet such individuals are known to have an above-average prevalence of violent behavior ( 24 , 25 , 26 ).

Response. It is certainly true that research ethics require that patients sign consent forms and agree to be interviewed. We trust that no one would have it otherwise. As we stated in the original article ( 1 ), "Some of the biases we observed are in the direction of patients in our sample being more likely to be violent than other eligible patients… and other biases are in the direction of patients in our sample being less likely to be violent than other eligible patients…. It is impossible to estimate the precise effect of these countervailing biases on the results."

Hence, the conclusion that "the comparatively low rate of violent behavior among patients with schizophrenia was almost certainly a result of selection bias" represents pure speculation. It would have been more helpful for Dr. Torrey and Mr. Stanley to have conducted sensitivity analyses to determine what the rate of violence would have to be within the nonconsenting group in order to reduce the observed effect size to zero.

We found that discharged patients diagnosed as having schizophrenia were more likely to be violent than people in the nonpatient comparison group but less likely to be violent than patients with other diagnoses. Several other major studies published at about the same time as the MacArthur Study reported the same finding ( 27 , 28 ). More recent research has reached different conclusions ( 12 ). The relationship between diagnosis and violence, we believe, is still an open question, but the MacArthur data can legitimately be cited as supporting the conclusion stated in our original paper.

Are public fears justified?

At issue. The MacArthur Study's conclusion that "public fears of violence on the streets by discharged patients who are strangers is misdirected" ( 1 ) is not justified. The authors reported that family members were the most frequent targets of violent behavior, accounting for 51% of the violent acts committed by the patient group, followed by friends and acquaintances (35%) and strangers (14%). However, the fact that four of the homicide victims—and possibly five (the information is not clear)—were strangers is disquieting.

Response. Again, selecting four or five high-profile, very serious violent acts out of the 608 we observed is less useful than systematically analyzing the data. Our view is congruent with that expressed in a 2005 article by Dr. Torrey ( 29 ) published in the Catalyst, the newsletter of the Treatment Advocacy Center: "Families … face the very real risk of violence. A 1997 study focusing on the prevalence of abuse faced by families of individuals with a mental illness found that 32% of relatives had been struck on at least one or two occasions. Verbal abuse, threats, and temper outbursts were reported by more than 50% of the relatives. The American Psychiatric Association notes that 'Family members are most at risk of a violent act committed by a mentally ill person.' … According to the MacArthur Violence Risk Assessment Study, 'The people at highest risk are family members and friends who are in their homes or in the patient's home.' An important reason why I founded the Treatment Advocacy Center was to address the issue of violence against family members and others."

Are the results generalizable?

At issue. The results of the MacArthur Study are not applicable to mentally ill patients in general. In their original report ( 1 ) the authors said that they believed "that our results are fairly representative of the community behavior of patients between 18 and 40 years old who are discharged from acute psychiatric facilities in the United States." This was accurate. However, press coverage of the study at the time of its release ( 30 ) and continuing professional citations that the study showed a "lack of an association between psychosis and crime" ( 31 ) have implied that the results were applicable to all psychiatric patients. The authors of the study cannot, of course, be blamed for the misuse of their study's findings, but it should be clearly stated that the study did not include patients with psychiatric illness who were in forensic hospitals, patients in jails and prisons, or those living on the street—all of whom would be expected to have a comparatively high prevalence of violent behavior. In fact, the organizers of the MacArthur Study originally did plan to include "mentally disordered offenders," but on the basis of pilot studies, they "determined that our design and instrumentation would need to be substantially modified to adapt to the criminal patients, and we chose to limit our study to civil patients" ( 16 ).

Response. Dr. Torrey and Mr. Stanley believe that "it should be clearly stated that the study did not include patients with psychiatric illness who were in forensic hospitals, patients in jails and prisons, or those living on the street." We do not know how we could have stated this any more clearly than we did. The title of the original article was "Violence by People Discharged From Acute Psychiatric Inpatient Facilities and by Others in the Same Neighborhoods." Had the MacArthur Study claimed to be an epidemiological study of violence by all people with mental disorders, similar to the seminal Epidemiologic Catchment Area survey ( 8 ), this point would be valid. But our study of violence risk assessment had no such epidemiological aspirations and made no such epidemiological claims. It should be noted, however, that many of the patients in the sample were in jail or on the street in the course of their lives (and in the course of the study). They were just not recruited from these places.

Does treatment reduce violence?

At issue. The most important finding of the MacArthur Study was that treating individuals with psychiatric disorders reduces violent behavior. In the original publication ( 1 ) the authors stated: "The effects of hospitalization and treatment on these rates [of violence] are unknown." However, three years later, they reported data on the effectiveness of psychiatric treatment in reducing violence after discharge from the hospital ( 16 ). They compared individuals who attended treatment sessions with those who did not; participants were grouped into those who in any ten-week period attended six or fewer treatment sessions and those who attended seven or more. The results are presented in Table 1 .

Enlarge table

From these data, the authors concluded that "treatment in the community may significantly reduce the likelihood of subsequent patient violence and thus be one important risk management strategy." This finding is consistent with results of other studies showing that the treatment of individuals with severe psychiatric disorders reduces the prevalence of violent behavior ( 32 , 33 , 34 ).

Response. We concur with Dr. Torrey and Mr. Stanley's point in regard to the probable importance of community treatment in reducing violence by persons being released to the community from acute psychiatric inpatient facilities. However, the limitations of the MacArthur Study data in supporting this conclusion should be underscored. It may be that persons who are unlikely to commit violence are more likely to adhere to treatment, which would account for the association between attending treatment sessions and avoiding violence. Without randomized assignment to treatment or no-treatment conditions, one cannot rule out that possibility. In our subsequent analyses of data from the the MacArthur Study, we used a statistical technique—propensity scores—to attempt to control for this possible relationship. But such post facto controls are not perfect. As we stated in Rethinking Risk Assessment ( 16 ), "While we proceed with caution, given that we have not conducted a randomized clinical trial, we believe that our findings give rise to guarded optimism about treatment as one way to manage violence risk." Subsequent research using the MacArthur data set ( 35 ) increases our optimism.

However, to call the analysis presented above "the most important finding of the MacArthur Study" is hyperbolic and misleading. A final caveat about the implications of the table: the analysis addressed attendance at treatment sessions not adherence to medications; conclusions about the latter cannot be drawn on the basis of the data in the table.

Conclusions

At issue In retrospect, therefore, what are the most important lessons to be learned from the MacArthur Study? Given the findings, how should you feel when a discharged psychiatric patient moves in next door? In such circumstances, the critical questions to be asked are: Who is monitoring the patient's continuing treatment? How will they know if the patient stops taking his or her medication? And does the individual abuse alcohol or drugs? If the discharged patient is being carefully monitored for treatment and does not abuse alcohol or drugs, chances are very good that you will have a good neighbor. If the discharged patient is not being monitored and is abusing alcohol or drugs, chances are very good that you will have problems.

Response. The lessons to be learned from the MacArthur Study are those that we have identified in our various publications. Among the more salient findings, violence risk attributed to people with mental disorders vastly exceeds the actual risk presented. Indeed, for people who do not abuse alcohol and drugs, there is no reason to anticipate that they present greater risk than their neighbors. The predictors of violence by people with mental disorders are more similar to than different from the predictors for the population as a whole, including alcohol and drug abuse. Violence in this population only rarely results in serious injury or death and generally does not involve the use of weapons. People with mental disorders are less likely than people without such disorders to assault strangers and to commit assaults in public places. Although there is suggestive evidence that remaining in treatment may reduce rates of violence among some persons with mental disorders, better data are needed; it is unlikely that treatment alone will eliminate violence risk.

As it happens, the direction in which the MacArthur Foundation's research efforts have turned since the publication of the Violence Risk Assessment Study is precisely toward patient monitoring and the use of "leverage" to ensure treatment adherence in the community. People with mental disorders are often dependent upon goods and services provided by social welfare agencies, including disability benefits and housing. Their access to these goods and services is often tied to treatment participation. Similarly, when people with mental disorders are arrested for criminal offenses, dispositions with alternatives to incarceration may be tied to treatment participation. Approximately half of all people receiving public-sector outpatient treatment have had one or more forms of leverage applied to them ( 36 ). The MacArthur Research Network on Mandated Community Treatment is now engaged in studies of the measurable outcomes of each of these attempts to improve treatment adherence. As with the past studies of violence risk discussed here, we continue to promote evidence-based decision making and advocacy that accurately reflect relevant research.

Acknowledgments and disclosures

Dr. Torrey and Mr. Stanley's work on this article was supported by the Stanley Medical Research Institute. They thank Edward Mulvey, Ph.D., for sharing the Pittsburgh community data.

The authors report no competing interests.