The Utility of Patients’ Self-Perceptions of Violence Risk: Consider Asking the Person Who May Know Best

Assessing an individual’s risk of violence is an inescapable aspect of practice for mental health clinicians. In outpatient settings, clinicians are often required to monitor a patient’s risk of violence to help fulfill their duty to protect third parties from harm (1). In inpatient settings, clinicians are often required to assess dangerousness to inform decisions about involuntary admission (2,3). Given changes in behavioral health care, the demand to repeatedly assess patients’ risk in both clinical settings has increased. Now, even patients at “high risk of violence to others may be discharged in a few weeks or, increasingly, in a few days, assuming that they are ever hospitalized in the first place” (4). Especially for these patients, an accurate method of monitoring risk is needed.

Assessment tools that structure clinical judgment or replace it with statistically derived decision rules typically outperform unaided clinical judgment in predicting violence (5–7). Nevertheless, there is no evidence that the average clinician uses these tools. In a survey of 93 general psychological practitioners in the United States, Tolman and Mullendore (8) found that less than 10% relied upon validated risk assessment tools. The current standard of practice appears to be reliance upon unaided—and often inaccurate—clinical judgment.

In part, clinical judgment may be relied upon because leading risk assessment tools are infeasible for use in routine clinical settings—they can require that a specially trained clinician spend several hours gathering and integrating information (9). This potential barrier to technology transfer could be addressed by shifting focus to brief risk assessment tools that focus on a chart review, such as the Clinically Feasible Iterative Classification Tree (ICT-CF) (4), or that “screen out” low-risk patients from further assessment, such as the Modified Screening Tool (MST) (7,10).

Given that the validated derivative of the ICT-CF (11) has seemed to gain little traction in clinical settings, focusing on brief tools may not bridge the gap between science and practice. Dramatic reductions in the use of psychological testing over recent decades (12,13) suggest that the method of structured assessment itself may be a barrier to technology transfer.

For this reason, we went beyond brief tools in this study to also examine a novel method of risk assessment—one that shifts focus away from formal testing and clinical prediction toward patients’ self-perception of risk. The principle underlying this approach is simple: “If you want information from someone, the best way to get it is to ask them” (14). With a lifetime of experience, patients have more information about their feelings, thoughts, and behavior across situations than any external evaluator (15). Indeed, patients can be valuable colleagues—“if we don’t stop them by asking the wrong questions and if we provide the appropriate structure, they often can tell us much about themselves” (16).

To date, calls for attention to self-prediction largely have been ignored, given a dominant view that people are unwilling or unable to provide accurate information about themselves (17). This view may be strongly activated when one considers using self-prediction to help assess a patient’s risk of violence. Nevertheless, evidence suggests that despite noise introduced by limited ability or bias, patients’ reports are likely to contain a valuable signal that is useful for monitoring risk.

Across a number of domains, there is evidence that people are able to predict their behavior better than psychological tests and external evaluators, such as therapists (18–21). For example, Peterson and colleagues (22) found that psychiatric inpatients’ perceptions of the likelihood of harming themselves during the months after discharge significantly predicted that outcome (area under the curve [AUC]=.72–.75). In fact, patients’ predictions of self-harm added incremental utility to a measure of depression (indicating that they improve on the predictive validity of this other method) and achieved a level of predictive utility that rivaled that of suicide risk assessment tools found in other studies (22).

It is possible that patients would be less forthcoming about self-perceived risk of violence toward others than toward themselves. However, several studies indicated that people often provide socially undesirable but valid information about themselves, even when there is incentive to deceive (23,24). For example, Loza and colleagues (25) found no significant difference in endorsement of past criminality between prisoners who were told that their responses would be used only for research and prisoners who were told that their responses would be used to inform release decisions.

In this study, we assessed the predictive utility of psychiatric patients’ self-perceptions of violence risk. These self-perceptions are both clinically feasible and consistent with routine practice, which emphasizes discussion with patients rather than testing. The secondary aim was to compare the predictive utility of self-perceptions with that of two brief risk assessment tools—the ICT-CF and the MST. Given our focus on improving practice in routine settings, these brief tools are better benchmarks for self-perceptions than unaided clinical judgment—which is relatively inaccurate—and leading risk assessment tools—which are seldom used.

Methods

We conducted an interview with patients at high risk of violence at a large psychiatric hospital. Two months after the patients’ discharge (a relatively high-risk period [26]), we located them in the community to assess their involvement in violence. Data were collected in 2002–2003.

Participants

We selected a relatively homogeneous sample of psychiatric patients at high risk of violence, both to maximize the study’s statistical power and to represent a policy-relevant population for risk monitoring and reduction. Eligibility criteria included young age (18–40 years old) (27), current diagnoses of co-occurring mental and substance use disorders (26), no current diagnosis of schizophrenia or schizoaffective disorder, and no current delusions. With respect to the last eligibility criterion, although active thought disorder is a risk factor for violence in the general population, it relates weakly or even inversely to violence in clinical populations, where the comparison groups are other psychiatric patients or offenders (7,28–31). For example, in one rigorous study, only 11% of 608 violent incidents detected occurred while patients were delusional or hallucinating (28). Our analyses indicate that among the subset of patients in that study who had a co-occurring major mental disorder and substance abuse disorder (N=468), those with a diagnosis of schizophrenia were significantly less likely than those without schizophrenia to be violent during the first two follow-up periods (19.5% versus 34.0%, χ²=6.06, df=1, p<.01).

Of patients who were eligible to participate in the present study, 86 (94%) agreed to participate. Their written informed consent was obtained, consistent with a protocol approved by the University of Pittsburgh Institutional Review Board. The sample included 44 (51%) African Americans and 42 (49%) Caucasians with a mean±SD age of 31.3±6.5, and the sample was divided equally between men and women. A majority (N=73, 85%) had sought treatment voluntarily. The most common chart diagnoses were affective disorders, including depression (N=60, 70%), bipolar disorder (N=19, 22%), and others (N=7, 8%). The co-occurring substance use disorders most commonly referenced alcohol (N=47, 55%) or cocaine (N=75, 87%). A total of 59 (69%) patients had a prior hospitalization, and 69 (80%) had a history of arrest (the latter rates were slightly higher than observed in previous studies [32,33]). The mean length of hospitalization was 8.3±4.7days.

Procedures

Patients completed a private, semistructured interview that chiefly focused on risk assessment. Two months after discharge, 63 (73%) patients completed a follow-up interview in the community that assessed substance use, symptoms, services, and violence since discharge. Participants were paid for their participation. Interviewers reviewed treatment records at both time points as a collateral source of information.

Measures

Self-perceptions.

Before being asked about their self-perceived potential for violence, patients discussed their recent living circumstances, relationships, involvement in violent incidents, proneness to anger and impulsivity, and treatment experiences. This discussion was meant to encourage patients to construe violence at less symbolic, more specific, and more contextualized levels (34). Near the end of the interview, self-perceptions were assessed with the following two sentences: “We define violence as any act that causes physical harm to another or is intended to do so. Given a scale of 0 to 5, where 0 is ‘no concern’ and 5 is ‘greatly concerned,’ how concerned should your therapist be that you might be violent in the next two months?” Patients were asked to express their risk of violence in terms of a “clinical concern” partly to convey risk in a form that clinicians can readily understand. In this study, both continuous (mean=1.4±1.7) and dichotomous (scores of 0–2, N=63, 73%, versus 3–5, N=22, 26%) scores were examined.

Clinically feasible tools.

The ICT-CF (4) and the MST (7,10) were developed with psychiatric patients and can be completed in less than 15 minutes. Although one other brief risk assessment tool shows promise, it requires more training to reliably administer than those we chose (35).

The ICT-CF was developed by applying an iterative statistical approach to data for 900 psychiatric patients and 106 risk factors generally available in records or routine assessments. It consists of decision trees that efficiently estimate risk and that manifested good predictive utility in the derivation sample (AUC=.80) (4). Individuals are classified into 11 risk categories, and three categories are considered high risk. Patients were also assigned an ordinal score of 1 to 11 to indicate their risk group.

The MST was developed on the basis of data from over 700 psychiatric patients. The tool consists of two stages: a chart review that identifies patients who are young, have no current thought disorder, and have a history of violence; and a brief interview and symptom inventory to identify recent substance abuse, violence, and high anger. In a cross-validation of the MST with approximately 130 patients, 89% of persons classified as high risk became violent over the next six months, with an average of seven incidents (10). In this study, we raised the MST’s age criterion from 30 to 40 years old, given that supplemental analyses suggested that this did not affect predictive utility. We used both MST classifications and scores (a simple count of risk factors, from 0 to 5).

Violence at follow-up.

At the follow-up, patients were asked whether they had been victimized or been violent since discharge for eight categories of aggressive behavior (36). Treatment records served as a collateral source of information about violence. In keeping with past research (28), responses were coded to reflect serious violence—battery resulting in injury, sexual assault, threat with a weapon in hand, or an assaultive act with a weapon—and any violence, which included both serious violence and other aggressive acts, for example, battery that did not result in injury. Of the 63 patients who completed a follow-up, 10 (16%) were involved in serious violence, and 23 (37%) were involved in any violence, in keeping with earlier research (26).

Results

All analyses were performed by using SPSS v.19. As shown in Table 1, self-perceptions seemed to outperform the MST and ICT-CF in forecasting serious violence and, to a lesser extent, any violence. AUC values (ranging from .5, chance accuracy, to 1.0, perfect accuracy) indicate the probability that the predictor will rank a randomly chosen violent patient higher than a randomly chosen nonviolent patient. The AUC for self-perceptions of serious violence and any violence translated into a large (d=.92) and medium (d=.71) effect size, respectively (37). Although statistical power for this test was limited, there was a trend toward significance (p=.13) for the differences in AUCs for prediction of serious violence by self-perceptions and the ICT-CF (38).

Table 1 also depicts each measure’s sensitivity in identifying violent patients, specificity in omitting nonviolent patients, positive predictive value (proportion of patients identified as potentially violent who actually were violent), negative predictive value (proportion of patients identified as potentially nonviolent who actually were not violent), and predictive efficiency (proportion of patients correctly classified). Even when the scores were dichotomized, the predictive efficiency of self-perceptions equaled or surpassed that of the tools—and were relatively sensitive to serious violence.

To assess the incremental utility of self-perceptions in predicting violence after controlling for effect of the tools, we conducted sequential logistic regression analyses. After the tools were entered in block 1, the addition of self-perceptions in block 2 significantly improved violence prediction (χ²=7.91, df=1_, p<.01, Nagelkerke R²=.21, △Nagelkerke R²=.19). When the analyses were completed in reverse order, adding the risk assessment tools in block 2 did not improve self-perceptions’ prediction of violence. This suggests that self-perceptions added unique predictive utility to the tools but that the reciprocal was not true.

Discussion

Although there have been considerable advances in risk assessment technology over recent decades, they seldom reach clinical practice. As a step toward reducing sole reliance upon unaided clinical judgment, we tested the accuracy of two clinically feasible methods in this study. Our findings may be organized into two points. First, patients’ self-perceptions of risk—assessed via one global rating—manifested relatively good accuracy in predicting violence. Second, self-perceptions added incremental utility to brief risk assessment tools in predicting violence, but the reverse was not true.

By design, these findings pertain specifically to a relatively homogeneous, policy-relevant group of high-risk psychiatric patients. Generalizability to patients who are older, have no substance abuse problems, or have active thought disorder is unknown. It is also important to note study limitations. Our sample was relatively small, and we were unable to locate 27% of the sample at follow-up. Although no significant differences were detected in the demographic variables and self-perceived risk of patients who were retained or lost, there might have been unobserved differences. In addition, we assessed the criterion measure of violence on the basis of patients’ self-reports and, when available, treatment records. This limitation seems unlikely to explain our main findings. First, given that collateral informants, such as friends and family, identify only a small proportion of violence not identified by patients—4% according to the most rigorous research (26)—we probably captured most incidents. Second, the accuracy of self-perceptions is unlikely to be a simple function of reporting bias, given evidence that predicting events does not influence reports of their subsequent occurrence (17). Nevertheless, as we consider the study’s main findings, it is important to bear in mind that our results pertain to high-risk patients and must be replicated in a larger study that adds collateral reports of violence.

Promise of clinically feasible methods

In our view, it is remarkable that patients’ responses to one question about their own risk of violence were moderately to strongly predictive of their later involvement in violence. The AUC values indicated a 69% (any violence) and 74% (serious violence) chance that a randomly selected violent patient assessed himself or herself as higher risk than a randomly selected nonviolent patient. Although one might assume that patients would be inclined to minimize or hide their risk, there is little evidence that they did so—self-perceptions manifested greater sensitivity than the two brief tools.

This finding that self-perceptions possess predictive utility raises a question—compared with what? Because this study was designed to address barriers to applying risk assessment technology, the most appropriate benchmark for the predictive utility of self-perceptions is tools that are feasible for use in routine practice. We found that those brief tools were weakly to moderately predictive of violence (AUCs=.58–.66) and added no incremental utility to self-perceptions in predicting violence. In our view, this finding underscores the promise of self-perceptions as a clinically feasible method of risk assessment.

The predictive utility of both clinically feasible methods examined here can be compared with that of clinical judgment and of leading risk assessment tools examined in other studies. First, the predictive utility of self-perceptions and (to a lesser extent) of the brief tools compares favorably with the utility of unaided clinical judgment observed in other studies (AUC=.61) (39). If replicated, this suggests that adding either method would be preferable to the current practice of relying upon judgment alone. Second, the predictive utility of self-perceptions is comparable to that of leading risk assessment tools that are more resource intensive. In an elegant meta-analysis (40), the predictive efficiencies of nine risk assessment instruments were found to be essentially “interchangeable,” with accuracy estimates falling within a narrow band (AUC=.65–.71) that easily encompass the accuracy estimates observed in this study.

In summary, this study suggests that patients’ self-perceptions of risk perform as well as brief tools in predicting violence. When we examine the present results in light of past research, it seems that both clinically feasible approaches perform at least as well as unaided clinical judgment and that self-perceptions perform within range of leading risk assessment tools. To the extent that resource limitations are a barrier to technology transfer, future research should focus on validating brief tools. However, to the extent that testing itself is a barrier, exploring the limits of self-perceptions may be a more fruitful avenue.

Understanding self-perceptions

Although it is an empirical question, we doubt that merely having a stranger ask patients to predict their own violence would elicit accurate self-perceptions. In future research, it will be important to test the extent to which three clinically relevant conditions affect accuracy. First, to what extent is “cognitive scaffolding” necessary? In this study, well-trained interviewers led open-ended discussions with patients about violence-relevant topics before eliciting self-perceived risk. Patients were encouraged to construe violence not as an abstract representation of a bad event they would avoid, but instead as a concrete representation of a specific event they could experience.

Second, to what extent is strong rapport necessary for eliciting accurate self-prediction? A patients’ willingness to confide in an interviewer may hinge upon strong rapport (41,42).

Third, to what extent will accurate self-prediction of violence generalize from research to a clinical context? Will patients be less willing to disclose a perception of high risk of violence if they believe that doing so would translate into a longer hospital stay? This possibility must be tested, despite evidence that people disclose risk in similar contexts.

Conclusions

Assuming that future studies fruitfully answer the questions above, we believe that self-perceptions of violence risk hold substantial promise as a method for improving risk assessment in routine clinical settings. Self-prediction is as feasible for use as direct questions about suicidality and “no-suicide contracts” (43). Of course, using self-perceptions will eradicate patient violence no more than using suicide contracts has made patients “suicide proof.” A given patient may not understand her risk or may choose not to accurately report it. For these reasons, self-perceptions should be viewed as a component of—not a replacement for—risk assessment.

Still, the process of eliciting patients’ self-perceived risk of violence may reveal useful information and casts patients as partners in a collaborative relationship—partners who hold expert information essential to the risk-management enterprise. Beyond helping potentially violent patients and their clinicians, the availability of a method for monitoring risk that is both feasible and accurate can only enhance public safety.