Letter

Helpful and Unhelpful Risk Assessment Practices: Reply

In Reply: The letter from Large and colleagues describes issues that have appeared in the literature many times. The interested reader should start with Harris and Rice ( 1 ). Here we have room only for a précis.

First, our study was predictive in the sense that this word is used in risk assessment. We assessed at time 1 and assigned a probability of violence based on group membership and then followed the patient for three months.

Second, Large and colleagues suggest that risk assessment has nothing to say about an individual. The COVR classifies people into groups (low risk to high risk and so forth) and on the basis of previous research assigns a "risk rating" according to how many people in that group go on to commit the target behavior. So, for example, we might find that among people in the highest-risk group, 75 of 100 go on to commit the specified behavior, and among those in a lower-risk group, only 25 commit this behavior. Thus would it not be sensible to use this information to allocate resources and plan accordingly? Let us take another example. Perhaps I want to take a trip somewhere to find some sun. How should I choose where to go? I could look at the previous number of days of sunshine for different regions. I might find that New South Wales has 75 days of sunshine per 100, whereas South Wales has only 25 per 100. Which should I choose? I presume Large and colleagues would say it doesn't matter because these are only group data and do not apply to any individual day.

The third issue is in how to present the efficacy of a risk-assessment tool. Most scientific studies tend to use ROC analysis because it is independent of the base rate and therefore we can assess the efficacy across different contexts, changes in the definition of violence, and so forth ( 2 ). Indeed the AUC statistic (also known as C) is well established in medicine as the preferred index of test accuracy ( 3 , 4 ). Testing accuracy was the aim of our study, and hence we used this statistic. However, risk assessment and management are not merely a matter of accuracy. Clinicians and managers must deal with such matters as policies and limited resources. These factors have an impact on decision making and on selecting one or more thresholds at which actions are taken. Hence statistics such as the positive predictive value and negative predictive value are very important for the real-work situation (with the COVR or without any formal aid), but they are context dependent and thus not a good way of comparing and contrasting risk assessment techniques.

The independent validation of such instruments is a crucial step toward evidence-based practice that can aid the clinician in making very difficult judgments related to risk. It is hard to see how such science could be regarded as a "worrying trend toward the promotion of unhelpful risk assessment practices in mental health services." Do the authors suggest that we should abandon such aids and simply guess about risk?

Acknowledgment

The authors thank Grant Harris for helpful comments.