Effectiveness of a Low-Cost Curriculum Among Entry-Level Psychiatric Rehabilitation Workers

Nationally, among the 577,669 mental health staff who provide direct care to people with major mental illnesses, an estimated 40% in private organizations and 60% in public institutions do not possess a graduate-level degree ( 1 ). Many have had no prior mental health training and receive sparse in-service training that has diminished over time ( 2 , 3 ). Turnover rates for these staff approach 50% over a two-year period. A recent report that examined 100,000 persons in the psychiatric rehabilitation workforce estimated that 73% did not have graduate degrees ( 4 ). Earlier studies found similar conditions ( 5 , 6 ). There is also growing concern over the capacity of existing educational programs to prepare credentialed professionals for a rapidly changing mental health system ( 7 , 8 ).

Core competencies for psychiatric rehabilitation staff ( 9 , 10 ) and best-practice teaching methods ( 7 ) have been identified. That is, it is known what to teach entry-level staff and how to teach them. However, reaching this rotating, dispersed workforce remains an issue. Several universities and community colleges offer degrees or certificate programs in psychiatric rehabilitation, but these are few and costly. Young, inexperienced staff who may be only exploring a career in mental health and are likely to have frequent turnover in these positions may not be willing to devote their time and limited finances toward an advanced degree.

The National Alliance on Mental Illness offers a practitioner education program. However, the effectiveness of these types of programs has not been evaluated in peer-reviewed journals. Although New York's in-service education curriculum has been evaluated ( 11 ), it is devoted only to staff working on inpatient wards. However, most psychiatric rehabilitation staff work in community-based agencies. The Annapolis Coalition on the Behavioral Health Workforce claims that there is a national "training gap" and has recently called for the creation of portable, low-cost, and effective curricula for entry-level direct-care staff ( 7 ).

The Pennsylvania Office of Mental Health and Substance Abuse Services (OMHSAS) contracted the Behavioral Healthcare Education Department of Drexel University College of Medicine to develop and disseminate a continuing education curriculum in psychiatric rehabilitation. It consists of an orientation course and eight skill-building courses. The entire curriculum was first offered during 2001 at various locations. Practitioners anywhere in Pennsylvania could complete the curriculum within the year without cost or extensive travel. Staff turnover necessitated repeated yearly offerings of this curriculum. The Drexel faculty anecdotally reported that compared with previous years, in 2004 and 2005, there seemed to be more course participants who did not have graduate degrees or past training and experience in mental health. Although the course was developed as a continuing education curriculum, it had become a basic education curriculum devoted to entry-level staff who resembled the workforce in the cited reports.

This study evaluated the curriculum of Drexel's orientation course by comparing the participants' pre- and postcourse scores on an empirically validated scale that measures practitioners' knowledge of psychiatric rehabilitation beliefs, goals, and practices. The orientation was evaluated because it was the initial, prerequisite course and it was expected to provide a basic but comprehensive knowledge of psychiatric rehabilitation principles and approaches.

A majority of the orientation's participants were expected to be entry-level staff (less than one year of experience) without graduate degrees or past training in mental health. Therefore, all participants' postcourse scores were also compared with those from a national sample of more experienced, educated practitioners. This benchmark comparison was conducted to determine whether persons who completed the orientation achieved a level of information commensurate with these more experienced, educated practitioners.

Methods

Mental health staff throughout Pennsylvania were the target population for the orientation course and the eight skill-building courses. Drexel and OMHSAS advertised in print and online a description and schedule of its psychiatric rehabilitation curriculum.

The orientation course is a two-day, 12-hour introductory course that presents the paradigm, mission, values, goals, models, and some practices of recovery-focused psychiatric rehabilitation. This information is conveyed to the participants via a standardized format that includes lectures, interactive exercises, role playing, videos, and Web site links. Trainers of the orientation are given PowerPoint slides that detail the course materials and a trainer's guide, which includes directions and objectives for exercises and videos. Trainees receive a workbook containing printouts of the PowerPoint slides, exercises, and references for printed and Web-based materials. The orientation is approved for continuing education credits by several national and state professional associations representing physicians, psychologists, social workers, nurses, and rehabilitation counselors. It is the prerequisite for 11 skill-building, applied practicum courses.

The Psychiatric Rehabilitation Beliefs, Goals, and Practices Scale (PRBGP) was the dependent measure in this study. It is a 26-item scale that measures practitioners' knowledge of the beliefs, goals, and practices that define psychiatric rehabilitation as a recovery-focused subspecialty in psychiatry. Possible scores range from 26 to 130, with higher scores indicating greater knowledge. Studies have established its reliability (intraclass correlation coefficient=.92, N=90), convergent-discriminant validity (N=279), and its association with consumer outcomes (N=191) ( 12 ). It has been shown to be sensitive to changes resulting from educational interventions (N=228) ( 13 ). A national survey of 478 members of the United States Psychiatric Rehabilitation Association from 39 states reconfirmed its validity ( 14 ), and a factor analytic study identified the scale's three major dimensions ( 15 ). The mean±SD total PRBGP score (mean score of 98.5±10.8) for this national sample served as the benchmark in this study.

Trainees provided information about their professional characteristics, including the year that they began working in behavioral health and in psychiatric rehabilitation and their highest educational degree, professional affiliation, and role in their agency. They also answered three dichotomously scored questions (yes or no): Did your degree program include a specialized course in mental health? Are you currently enrolled in an advanced degree program in mental health? And if you are not currently enrolled in an advanced degree program, are you planning to enroll in the next two years?

The orientation course was offered ten times between September 2005 and April 2006 throughout Pennsylvania. On each occasion the trainees completed the trainee characteristics survey and the PRBGP scale before the course. At the course's conclusion they completed the PRBGP scale again. The trainees' pre- and postcourse mean total PRBGP scores were compared by using paired t tests, and both were compared with the benchmark national sample's mean total PRBGP score by using t tests.

Results

Of 379 people registered for the orientation course, 367 (97%) attended and completed the survey and the pre- and postcourse PRBGP scales. These staff represented 84 separate agencies from 34 of Pennsylvania's 67 counties. Trainee characteristics for the total sample and three subsamples are reported in Table 1 . The three subsamples were established on the basis of the trainees' years working in psychiatric rehabilitation—that is, less than one year, one to three years, and four or more years.

Enlarge table

As expected half the trainees were entry level (less than one year of experience), and 73% had less than four years of experience working in psychiatric rehabilitation. Among the trainees with less than four years of experience, 81% did not have graduate degrees and 73% did not have prior specialized training in mental health. The subsample with the most years of experience (four or more years) had significantly more years working in the field and a larger proportion of individuals with graduate degrees than the two less experienced subsamples. Yet, among the subsample with the most experience, only 36% had graduate degrees and 37% had taken specialized courses. The precourse mean total PRBGP scores for the three subsamples were not significantly different. This finding is not unusual because the PRBGP measures specialized knowledge and is only minimally correlated with work experience and formal education in mental health.

As expected, among the 280 trainees who did not have a graduate degree, only 17 (6%) were currently enrolled in an advanced degree program. Among those not enrolled (N=263), only 60 (23%) planned to enroll in the next two years. The three subsamples did not differ significantly on these variables. For 203 trainees (55%) the Drexel curriculum may be the only formal training that will guide their practice for several years.

The trainees' postcourse mean total PRBGP score (mean of 103.1±10.6) was significantly and substantially higher than their precourse mean score (mean of 92.7; paired t =-21.72, df=366, p<.001; d=.99). A similar training effect was found for the three subsamples and for the 203 trainees who were not planning to enroll in an advanced degree program in the next two years (mean score of 91.7±9.4 for the pretest, compared with 103.1±10.0 for the posttest; paired t=-18.83, df=200, p<.001).

These trainees had a significantly lower precourse mean total PRBGP score than the benchmark national sample of members of the United States Psychiatric Rehabilitation Association (mean score of 98.5±10.8, compared with 92.7; t=7.82, df=809, p<.001, d=.53). However, the trainees' postcourse mean total PRBGP score (mean score of 103.1) was significantly higher than the mean for the benchmark national sample (t=-6.28, df=805, p<.001, d=.43). The orientation's immediacy and intensity for the trainees may have accounted for this finding. The PRBGP and the orientation were independently created and their contents do not entirely overlap. Seventy-five percent of the variance in the trainees' pre-post difference in their total PRBGP scores was accounted for by changes in responses for only 11 of the 26 items on the PRBGP.

Discussion and conclusions

Drexel's orientation to psychiatric rehabilitation was effective in increasing the knowledge of recovery-focused psychiatric rehabilitation principles and approaches among staff with minimal experience and education in mental health. These trainees achieved an information level commensurate with the more experienced and educated membership of the field's professional organization. A majority of these trainees may rely solely on this knowledge to guide their practice for several years to come.

The Annapolis Coalition called for the creation of a portable, low-cost, and effective curriculum in order to close the "training gap" for entry-level mental health workers. This standardized orientation course's portability and low-cost contributed to its overall effectiveness in narrowing the training gap in Pennsylvania. The state-academic partnership created the foundation for the development of this curriculum.

This study was limited to knowledge acquisition alone. Knowledge is not always applied, nor is it equivalent to practice skills. However, these trainees can participate in the curriculum's remaining skill-building courses over the next year. These additional courses can build on the demonstrated knowledge that these trainees acquired in the orientation course. The curriculum's portable and cost-free features are likely to encourage these trainees to continue their education as psychiatric rehabilitation practitioners. A continuing education format can provide portable, low-cost, and effective basic education to entry-level psychiatric rehabilitation workers.

Acknowledgments and disclosures

The authors report no competing interests.