0
Get Alert
Please Wait... Processing your request... Please Wait.
You must sign in to sign-up for alerts.

Please confirm that your email address is correct, so you can successfully receive this alert.

1
Columns   |    
Best Practices: New York's Outpatient Mental Health Clinic Licensing Reform: Using Tracer Methodology to Improve Service Quality
Jennifer P. Wisdom, Ph.D., M.P.H.; Susan Knapik, M.S.; Michael W. Holley, L.C.S.W.-R.; Jayne Van Bramer, M.A.; Lloyd I. Sederer, M.D.; Susan M. Essock, Ph.D.
Psychiatric Services 2012; doi: 10.1176/appi.ps.20120p418
View Author and Article Information

Dr. Wisdom and Dr. Essock are affiliated with the Department of Psychiatry, Columbia University, and New York State Psychiatric Institute, 1051 Riverside Dr., Box 100, New York, NY 10032 (e-mail: jpw2129@columbia.edu).
Ms. Knapik, Mr. Holley, Ms. Van Bramer, and Dr. Sederer are with the New York State Office of Mental Health, Albany, where Dr. Essock is also affiliated. William M. Glazer, M.D., is editor of this column.

Copyright © 2012 by the American Psychiatric Association.

Abstract

This column describes how the New York State Office of Mental Health (OMH) redesigned licensing procedures for outpatient clinics by applying a person-centered focus and clinically relevant standards of care. OMH incorporated a tracer methodology to assess services; a licensing instrument that reflected OMH goals; and a systematic piloting, feedback, and implementation process. Clinic staff reported that the new procedures facilitated communication between OMH and clinics, accurately identified clinics' successes, and provided actionable and programmatic feedback. This initiative represents the first step by a state mental health authority to create a system of accountability that is clinically relevant and supported by stakeholders. (Psychiatric Services 63:418–420, 2012; doi: 10.1176/appi.ps.20120p418)

Abstract Teaser
Figures in this Article

Professional accountability requires a profession to set and maintain credible and useful standards for its members (1) and to implement best practices (2). New York State oversees mental health services provided by state-licensed treatment providers. In response to several incidents of violence that involved people with serious mental illness, New York State and New York City sponsored a Mental Health-Criminal Justice Panel to improve coordination, oversight, and accountability in mental health treatment services. The panel's report was published in June 2008 (3). Although prompted by the incidents of violence, the New York State Office of Mental Health (OMH) took advantage of the call for better assessment of violence to revise the licensing process to underscore recovery-oriented values, such as treatment responsive to individuals' stated needs and family involvement in treatment, in redesigned clinic licensing procedures.

Mental health care has been moving toward a recovery-focused, person-centered, strengths-based system. The President's New Freedom Commission on Mental Health (4) and others (57) have called for increasing consumer- and family-driven services, improving quality and accountability in behavioral health services, and incorporating the perspectives of the people who receive services. Licensing processes have not always kept pace with these efforts to improve service delivery.

Outpatient clinics are the “face of the public mental health system” and are the only level of service used by most people. In New York State, for example, more than 500 licensed clinics offer outpatient mental health services at more than 1,200 sites serving more than 400,000 individuals each year. Though the panel's mandate to apply new standards of care was the initial impetus for change, OMH saw this as an opportunity to promote high-quality outpatient services.

The licensing system as it existed at the time of the panel's report in 2008 was a tiered process focused largely on policies and documentation. This licensing process usually included a discussion with the clinic administrator and a review of medical records, policies, meeting minutes, and safety-related procedures. The panel report included guidelines for standards of care at mental health clinics (3). The guidelines focused on new areas of integrated, coordinated care through “clinical homes” (8); risk assessment; information sharing across providers; services based on individual and family needs; and engagement and clinical supervision. The recommendations challenged OMH to identify licensing procedures that would capture these changing responsibilities. This column describes the first steps in redesigning and implementing the clinic licensing system in New York State.

OMH proposed a redesign with three aims: to base the licensing process on the guidelines for standards of care, to focus the process on improving care and emphasizing family involvement, and to model strengths-based interactions to reinforce how OMH would like clinics to interact with people with serious mental illness. To accomplish these goals, OMH created a work group that solicited input from central and field office staff; experts on clinical service, psychometrics, engagement, wellness, and co-occurring disorders; people with serious mental illness and family members; family advocacy groups; service providers and provider associations; and special interest groups. In collaboration with these stakeholders, OMH proposed a tracer methodology (9) to create an interactive process to assess current clinic functioning, evaluate systems for providing care, evaluate the experience of services for people with serious mental illness and their families, and provide clinically relevant and actionable feedback to clinics (10).

Tracer methodology, as promoted by the Joint Commission (9), is a tool to assess a facility's compliance with selected standards and to evaluate systems for providing care and services. To use the tracer methodology, individuals with high service needs are selected and, using the medical record as a “road map,” each service contact is traced to evaluate whether program components were compliant with standards of care. Discussion of findings with organizations can also provide relevant feedback to guide mental health service improvements (11).

Throughout the development process, OMH openly shared presentations and data on the OMH Web site, sought feedback from stakeholders, and pilot-tested survey instruments and procedures.

OMH developed three tools to support the redesign process.

+

Site visit protocols

Protocols to select tracer cases directed OMH survey teams to choose individuals with high service needs, such as individuals needing complex care (for example, those with co-occurring medical or substance abuse problems) or with recent use of intensive services, such as hospitalization. To pilot the protocols, OMH survey teams included two central office staff and two or three field office staff from regional offices who conducted site visits and surveys; currently survey teams include two field office staff. OMH developed spreadsheets that listed clients who had used a particular clinic in the past year and who met criteria for having high needs. On the morning of the visit, clinic staff were asked to recommend the most high-need individuals on the appointment list for that day, and the OMH team then compared the clinic's list with names on their spreadsheets and, when possible, selected the matching individuals to trace. Using the medical record as a “road map,” surveyors interviewed people receiving services, their significant others, program clinical and administrative staff, and other service providers to evaluate whether service contacts were compliant with standards of care. After the initial interview, clinic staff and state surveyors reviewed cases together, compared information, compared scoring, and discussed each item to determine areas of consensus, clarity, and ambiguity.

+

Licensing instrument

The team developed care standards and integrated them into a licensing instrument that would maintain a strengths-based approach, promote person and family centeredness, reflect the current evidence base involving treatment engagement, and include a quality improvement perspective that would continuously push clinics to increase their quality of care. OMH worked with clinicians and administrators to translate standards of care into 31 measurable criteria and examples and to assign them to three levels of competency: exemplary, adequate, and needs improvement. OMH incorporated evidence-based practice in the exemplary and adequate levels. [The criteria are available in an online data supplement to this article.]

Licensing criteria directly reflected standards of care. Criteria were classified into three categories: exemplary, adequate, and needs improvement. The criteria enumerated in the exemplary category enabled OMH to communicate a clear vision to clinics and to provide an aspirational goal. The final licensing instrument included a decision matrix for determining the length of the operating license (which ranges from 0 to 36 months). [A table in the online data supplement presents an example of how a standard of care (violence assessment) was classified into the three categories.]

+

Implementation toolkit

OMH developed a toolkit to aid providers (12). The toolkit includes documents associated with the licensing process; video presentations on topics related to the standards of care and on implementing best practices in clinics; a questions-and-answers page addressing topics raised by providers and other stakeholders; and a dedicated e-mail address for questions, suggestions, issues, and comments.

Central office and field office staff field-tested the licensing instrument and then pilot tested the redesigned process with 11 clinics, soliciting feedback from programs. These activities identified strengths and potential shortcomings of the instrument as a licensing tool, determined the viability of tracer methodology, assessed the validity of the scoring protocol, and provided opportunity for field office and central office staff to work together to ensure consistency in implementation.

Two or three clinics from each region of the state took part in the pilot testing of the new licensing standards, scoring protocol, and tracer methodology. These 11 clinics were promised a minimum of an 18-month operating license to encourage participation. The clinics were a mix of not-for-profit (N=4), county (N=3), hospital-based (N=3), and state-operated (N=1) clinics. The clinics served a variety of populations, including adults (N=4), children and adolescents (N=1), and both adults and children (N=6). Clinics also were a mix of those with satellite offices (N=4) and those without (N=7). Licensing staff reviewed materials and methods with each provider before the site visit and throughout the process.

The OMH survey team conducted pilot visits, spending two days at each of the 11 clinics. At each site, staff examined at least four open records (range 4–8) and the records of two individuals no longer engaged in services (range 2–6) to identify individuals to trace. Interviews were generally conducted by one field office and one central office staff member; some interviews, such as those with administrators, were conducted by the entire OMH survey team. During the visit, staff interviewed two or more individuals receiving services at each site (N=26 interviews). Staff also conducted in-person and telephone interviews with 12 family members or significant others, ten collateral service providers, and 86 clinic staff, including clinicians, administrators, nursing staff, and quality assurance staff. The OMH survey team concluded visits with an exit conference to discuss survey findings and invited clinics to complete an online survey about the visit.

Under the new process, five of the 11 clinics (45%) received an operating license for fewer months of operation than in the previous licensing process, and six clinics (55%) received a license for more months of operation. The current process was comparable to the previous process, in which most clinics received a license for the same period of operation as previously. A common challenge identified by the tracer methodology, which might not have been identified by the former licensing procedures, was that providers were reluctant to share information about service users with other providers. The new licensing procedures prompted discussion between clinic and OMH staff about confidentiality and privacy laws that allow communication for treatment purposes.

Clinic staff members' responses to the online survey indicated that the new process was more focused and clinically relevant, that the revised standards were reasonable and appropriate, and that the survey team's contact with people receiving services and their family members increased OMH's ability to assess the quality and effectiveness of the treatment offered. (The survey results are available from the authors.) Clinic staff reported that they felt more involved in the process and that the process was more collaborative, affirming, and clinically relevant. They also said that there was more consensus and clarity on clinic functioning than in the previous licensing processes and that the new process was more likely than the previous system to identify clinics' exemplary activities, which was more affirming. One provider wrote, “In my 20+ years of experience, our 2010 recertification process was by far the most patient-centered, clinic-enhancing state process I have overseen.” Implementation of the redesigned licensing procedures began statewide in April 2010.

We are not aware of other state mental health authorities' efforts to radically redesign licensing procedures, aimed at increasing the person-centeredness of licensing processes and evaluating clinically relevant standards of care. The previous process emphasized policies and procedures, minutes of meetings, and adherence to medical record documentation, whereas the tracer methodology allows for more accurate assessment and follow-up of clinical issues. Key collaborations contributed to successful development of the tracer procedures, with all stakeholders in agreement that they would like every clinic to achieve exemplary status. Both the OMH survey team and clinic staff reported a more positive licensing process focused on quality enhancement and clinical issues, rather than the previous focus on paperwork.

There are still challenges in continuously incorporating feedback from clients and providers, ensuring consistency among the OMH survey team, and maintaining consensus. OMH staff anticipate that as research-informed best practices, stakeholder perspectives, OMH priorities, and clinic performance change, the licensing processes will require continued updating. Future efforts will include applying the redesigned procedures to other types of programs and assisting other states that want to adopt similar procedures.

Benson  JA:  Certification and recertification: one approach to professional accountability.  Annals of Internal Medicine 114:238–252, 1991
 
Drake  RE;  Goldman  HH;  Leff  HS  et al:  Implementing evidence-based practices in routine mental health service settings.  Psychiatric Services 52:179–182, 2001
[CrossRef]
 
Hogan  MF;  Gibbs  LI;  O'Donnell  DE  et al:  New York State/New York City Mental Health-Criminal Justice Panel Report and Recommendations .  June2008. Available at www.nyc.gov/html/om/pdf/2008/pr221-08_report.pdf
 
 Achieving the Promise: Transforming Mental Health Care in America . Pub no SMA-03-3832.  Rockville, Md,  Department of Health and Human Services, President's New Freedom Commission on Mental Health, 2003
 
 The Breakthrough Series: IHI's Collaborative Model for Achieving Breakthrough Improvement.  IHI Innovation Series white paper .  Boston,  Institute for Healthcare Improvement, 2003. Available at www.IHI.org
 
 Institute of Medicine: Improving the Quality of Health Care for Mental and Substance-Use Conditions: Quality Chasm Series .  Washington, DC,  National Academies Press, 2005
 
 Infusing Recovery-Based Principles Into Mental Health Services: A White Paper by People Who Are New York State Consumers, Survivors, Patients, and Ex-Patients .  Albany,  New York State Office of Mental Health,  Sept2004
 
Smith  TE;  Sederer  LI:  A new kind of homelessness for individuals with serious mental illness? The need for a “mental health home.” Psychiatric Services 60:528–533, 2009
 
 Mock Tracer Workbook .  Washington, DC,  Joint Commission International, 2009
 
Greenfield  D;  Travaglia  J;  Braithwaite  J  et al:  Unannounced Surveys and Tracer Methodology: Literature Review .  New South Wales, Australia,  Centre for Clinical Governance Research in Health, University of New South Wales, 2007
 
Smith  GR;  Fisher  EP;  Nordquist  CR  et al:  Implementing outcomes management systems in mental health settings; in Outcomes Assessment in Mental Health Treatment: A Compendium of Articles .  Washington, DC,  American Psychiatric Association, 1998
 
 Clinic Licensing Toolkit .  Albany,  New York State Office of Mental Health, Bureau of Inspection and Certification, 2011. Available at www.omh.ny.gov/omhweb/clinic_standards/toolkit.html
 
References Container
+

References

Benson  JA:  Certification and recertification: one approach to professional accountability.  Annals of Internal Medicine 114:238–252, 1991
 
Drake  RE;  Goldman  HH;  Leff  HS  et al:  Implementing evidence-based practices in routine mental health service settings.  Psychiatric Services 52:179–182, 2001
[CrossRef]
 
Hogan  MF;  Gibbs  LI;  O'Donnell  DE  et al:  New York State/New York City Mental Health-Criminal Justice Panel Report and Recommendations .  June2008. Available at www.nyc.gov/html/om/pdf/2008/pr221-08_report.pdf
 
 Achieving the Promise: Transforming Mental Health Care in America . Pub no SMA-03-3832.  Rockville, Md,  Department of Health and Human Services, President's New Freedom Commission on Mental Health, 2003
 
 The Breakthrough Series: IHI's Collaborative Model for Achieving Breakthrough Improvement.  IHI Innovation Series white paper .  Boston,  Institute for Healthcare Improvement, 2003. Available at www.IHI.org
 
 Institute of Medicine: Improving the Quality of Health Care for Mental and Substance-Use Conditions: Quality Chasm Series .  Washington, DC,  National Academies Press, 2005
 
 Infusing Recovery-Based Principles Into Mental Health Services: A White Paper by People Who Are New York State Consumers, Survivors, Patients, and Ex-Patients .  Albany,  New York State Office of Mental Health,  Sept2004
 
Smith  TE;  Sederer  LI:  A new kind of homelessness for individuals with serious mental illness? The need for a “mental health home.” Psychiatric Services 60:528–533, 2009
 
 Mock Tracer Workbook .  Washington, DC,  Joint Commission International, 2009
 
Greenfield  D;  Travaglia  J;  Braithwaite  J  et al:  Unannounced Surveys and Tracer Methodology: Literature Review .  New South Wales, Australia,  Centre for Clinical Governance Research in Health, University of New South Wales, 2007
 
Smith  GR;  Fisher  EP;  Nordquist  CR  et al:  Implementing outcomes management systems in mental health settings; in Outcomes Assessment in Mental Health Treatment: A Compendium of Articles .  Washington, DC,  American Psychiatric Association, 1998
 
 Clinic Licensing Toolkit .  Albany,  New York State Office of Mental Health, Bureau of Inspection and Certification, 2011. Available at www.omh.ny.gov/omhweb/clinic_standards/toolkit.html
 
References Container
+
+

CME Activity

There is currently no quiz available for this resource. Please click here to go to the CME page to find another.
Submit a Comments
Please read the other comments before you post yours. Contributors must reveal any conflict of interest.
Comments are moderated and will appear on the site at the discertion of APA editorial staff.

* = Required Field
(if multiple authors, separate names by comma)
Example: John Doe



Web of Science® Times Cited: 1

Related Content
Articles
Books
The American Psychiatric Publishing Textbook of Geriatric Psychiatry, 4th Edition > Chapter 33.  >
Dulcan's Textbook of Child and Adolescent Psychiatry > Chapter 62.  >
Dulcan's Textbook of Child and Adolescent Psychiatry > Chapter 62.  >
The American Psychiatric Publishing Textbook of Geriatric Psychiatry, 4th Edition > Chapter 33.  >
The American Psychiatric Publishing Textbook of Geriatric Psychiatry, 4th Edition > Chapter 33.  >
Topic Collections
Psychiatric News
APA Guidelines