0
Get Alert
Please Wait... Processing your request... Please Wait.
You must sign in to sign-up for alerts.

Please confirm that your email address is correct, so you can successfully receive this alert.

1
Letters   |    
Computer-Administered Versus Paper-and-Pencil Mental Health Surveys
Susan V. Eisen, Ph.D.; Linda L. Toche-Manley, Ph.D.; Grant R. Grissom, Ph.D.
Psychiatric Services 2004; doi: 10.1176/appi.ps.55.11.1316-a

To the Editor: Computers are increasingly used to assess mental health and substance abuse treatment outcomes (1). Computerized assessment has the potential to reduce burden on clinical staff, increase the efficiency of data collection and management, reduce the rate of missing data, increase interrater reliability of clinician-administered assessments, eliminate inapplicable items by using branching logic, provide immediate feedback of results, and help identify individuals at risk of poor outcomes in a timely manner that could allow for clinical intervention (2,3). However, it is important to determine whether responses to computerized assessments are comparable to those obtained by paper-and-pencil methods (4).

We conducted a small study to compare feasibility, respondent acceptance, reliability, and mean scores for computerized assessment versus a paper-and-pencil survey. We used a repeated-measures design in which psychiatric outpatients completed two self-report mental health questionnaires by using paper and pencil and by computer. The two survey instruments used were the Polaris Strength Scales, a new 29-item scale focused on goal setting, resiliency, development of emotional and relational skills, and social supports, and the revised Behavior and Symptom Identification Scale (BASIS-24), a 24-item scale that assesses depression and functioning, interpersonal relationships, psychotic symptoms, substance abuse, self-harm, and emotional lability (5). Both instruments are designed for use among persons with serious and persistent mental illness.

The study was conducted over a two-month period in 2002. The sample consisted of 52 English-speaking adults, evenly divided by gender. The age of the participants ranged from 19 to 64 years. A total of 41 participants (84 percent) were Caucasian, 33 (67 percent) were unemployed, and 28 (57 percent) had been in treatment for more than ten years. Two respondents refused to complete the computerized survey. All those who agreed to complete it were able to use the computer after receiving brief (less than five minutes) instructions. More respondents needed help completing the survey by computer than by paper and pencil (eight respondents, or 17 percent, compared with one respondent, or 2 percent; χ2=4.28, df=1, p<.05), although respondents preferred the computer by a ratio of 3:1. No significant differences were noted in respondents' evaluation of the time to complete the instruments, understanding of items, clarity of instructions, or any other concerns.

The reliability of ratings made by computer versus by paper and pencil was high, with intraclass correlation coefficients ranging from .83 to .9 for the Polaris Strength Scales and from .82 to .95 for the BASIS-24 domains. Differences in mean scores on the domains assessed were not statistically significant, with one exception: participants reported significantly more frequent psychotic symptoms by paper and pencil than by computer (F=7.19, df=1,44, p=.01).

Limitations of this study include its small sample and possible memory or order effects (that is, whether respondents used the computer first or paper and pencil first), which were not assessed. Self-report of mental health status by computer may enhance quality improvement efforts by improving the capacity to monitor treatment outcomes with greater efficiency than can be achieved with paper-and-pencil methods. Further research is needed to test computerized administration of standardized instruments in larger samples from multiple sites.

Dr. Eisen is affiliated with the health services department of the Boston University School of Public Health and the Center for Health Quality, Outcomes, and Economics Research at the E. N. Rogers Memorial Hospital in Bedford, Massachusetts. Dr. Toche-Manley and Dr. Grissom are with Polaris Health Directions in Fairless Hills, Pennsylvania.

Schmitz N, Hartkamp N, Brinschwitz C, et al: Comparison of the standard and the computerized versions of the Symptom Check List (SCL-90-R): a randomized trial. Acta Psychiatrica Scandinavica 102:147—152,  2000
[PubMed]
[CrossRef]
 
Brown GS, Burlingame GM, Lambert MJ, et al: Pushing the quality envelope: a new outcomes management system. Psychiatric Services 52:925—934,  2001
[PubMed]
[CrossRef]
 
Kobak KA, Schaettle SC, Greist JH, et al: Computer-administered rating scales for social anxiety in a clinical drug trial. Depression and Anxiety 7:97—104,  1998
[PubMed]
[CrossRef]
 
Guidelines for Computer Based Tests and Interpretation. Washington, DC, American Psychological Association, 1986
 
Eisen SV, Normand SLT, Belanger AJ, et al: BASIS-32 and the Revised Behavior and Symptom Identification Scale (BASIS-R), in The Use of Psychological Testing for Treatment Planning and Outcome Assessment, 3rd ed, vol 3. Edited by Maruish M. Mahway, NJ, Lawrence Erlbaum, 2004
 
+

References

Schmitz N, Hartkamp N, Brinschwitz C, et al: Comparison of the standard and the computerized versions of the Symptom Check List (SCL-90-R): a randomized trial. Acta Psychiatrica Scandinavica 102:147—152,  2000
[PubMed]
[CrossRef]
 
Brown GS, Burlingame GM, Lambert MJ, et al: Pushing the quality envelope: a new outcomes management system. Psychiatric Services 52:925—934,  2001
[PubMed]
[CrossRef]
 
Kobak KA, Schaettle SC, Greist JH, et al: Computer-administered rating scales for social anxiety in a clinical drug trial. Depression and Anxiety 7:97—104,  1998
[PubMed]
[CrossRef]
 
Guidelines for Computer Based Tests and Interpretation. Washington, DC, American Psychological Association, 1986
 
Eisen SV, Normand SLT, Belanger AJ, et al: BASIS-32 and the Revised Behavior and Symptom Identification Scale (BASIS-R), in The Use of Psychological Testing for Treatment Planning and Outcome Assessment, 3rd ed, vol 3. Edited by Maruish M. Mahway, NJ, Lawrence Erlbaum, 2004
 
+
+

CME Activity

There is currently no quiz available for this resource. Please click here to go to the CME page to find another.
Submit a Comments
Please read the other comments before you post yours. Contributors must reveal any conflict of interest.
Comments are moderated and will appear on the site at the discertion of APA editorial staff.

* = Required Field
(if multiple authors, separate names by comma)
Example: John Doe



Related Content
Books
Dulcan's Textbook of Child and Adolescent Psychiatry > Chapter 34.  >
Dulcan's Textbook of Child and Adolescent Psychiatry > Chapter 7.  >
Dulcan's Textbook of Child and Adolescent Psychiatry > Chapter 65.  >
Dulcan's Textbook of Child and Adolescent Psychiatry > Chapter 65.  >
The American Psychiatric Publishing Textbook of Geriatric Psychiatry, 4th Edition > Chapter 34.  >
Topic Collections
Psychiatric News
PubMed Articles