Clinical Computing: A Performance Indicator Spreadsheet for Physicians in Community Mental Health Centers

Legislators and other stakeholders to whom our community mental health center is accountable are increasingly insistent on critical indicators to demonstrate progress in meeting objectives and implementing strategies for mental health care. The proper use of physician time can play a critical role in our center's performance.

Physician time significantly affects our cost-efficiency because of its expense compared with that of other clinical staff. Our physicians also have a significant impact on cost-effectiveness through their decisions about resource utilization, such as their decision making about the use of acute care.

Few studies have evaluated the role of physicians in the cost-efficiency and cost-effectiveness of community mental health centers. Methods have been offered to evaluate the relationship between cost and outcome for clinicians, programs, and entire centers. McCrone and Weich (1) recently reviewed the literature on measures of cost of mental health services and found few or inappropriate measures. Butler and Docherty (2) described a system to collect data on the cost of clinical and administrative services and then link these to clinical effectiveness. Studies of cost-efficiency of specific programs include Rosenheck and colleagues' (3) recent analysis of different approaches to caring for homeless mentally ill veterans. Marks (4) described a method for collecting and analyzing cost and outcome data that included evaluation of individual patients and clinicians.

Although a precedent exists in the literature for cost and outcome indicators for individual clinicians, our center was interested in ensuring broader physician accountability. Therefore, we developed our own indicators of performance for individual physicians and for our medical staff. This column describes the indicators and how they are calculated and presented in a spreadsheet for use in quality monitoring.

Setting

The indicators were developed in late 1995 for the psychiatrists serving adult consumers in our large community mental health center in Houston, Texas. Most of the center's adult mental health services are provided through eight outpatient community mental health clinics spread throughout the county. Each clinic has from three to six full-time psychiatrists, each of whom provides leadership to a multidisciplinary treatment team. The members of each team together care for about 300 patients.

Diagnoses tend to be consistent across treatment teams. Virtually all patients are diagnosed as having schizophrenia, major depression, or bipolar disorder. Broad measures of acuteness, such as patients' level of need, also tend to be consistent across teams. (Patients served by the center's assertive community treatment teams have more acute illness and needs, and physicians on these teams were not included in the comparative measures for this study.)

Each clinic is assigned a medical director who devotes no more than 50 percent of his or her time to administrative responsibilities, with the remaining time used for clinical work such as patient triage and coverage for absent physicians. The medical director is ultimately responsible for the performance of the clinic's physicians. Therefore, chief among the medical director's responsibilities is the collection and monitoring of performance data for individual physicians in the clinic.

Performance data

Two sets of data on performance are collected, one for individual physicians and the other for overall performance of each clinic. The data are collected at the end of each calendar month. All of the data required for the indicators are available from reports generated by the center's information systems department. The various indicators are described below.

Individual physician indicators

End-of-month treatment team caseload.

The center's database includes the name of the attending physician assigned to each patient. The active caseload is measured on the last day of the month and includes a count of all active patients on the caseload, even if they are not seen by the physician during the month.

Value of services provided.

A monetary value is assigned to each service provided by physicians, based on its value if billed to Medicaid. These assignments are made regardless of the patient's Medicaid eligibility. For example, even services that are provided to medically indigent patients are assigned a value. The total value of all services provided by each physician is reported monthly.

Percentage of time spent in direct patient service.

A calculation is made of the total number of hours that each physician spends in face-to-face services with patients (direct service). That number is divided by the total number of hours that each physician was paid during the month, excluding leave time.

Percentage of treatment team patients with a current treatment plan.

Funding sources require patients to have a new treatment plan every 90 days for the center to qualify for reimbursement for clinical services. The center's database reports a count of patients for whom the treatment plan has expired. Because of the potential impact on revenue and quality of care, the percentage of the total caseload with expired treatment plans is monitored on the last day of the month.

Year-to-date referrals to acute care.

Each attending physician keeps a count of patients who are referred to acute care, whether by the treatment team, by another physician, or by self-referral. An annualized figure (total referrals for the preceding 12 months) is reported as a percentage of the end-of-month caseload.

Year-to-date case closures.

Because of the chronic nature of most of the disorders treated in our clinics, most case closures represent patients who have been lost to follow-up. Losing patients to follow-up is a quality issue, but it can also become a cost-efficiency matter if lost patients return by way of the hospital. Therefore, a count is maintained of cases closed during the month. The count is annualized and reported as a percentage of the end-of-month caseload for the physician.

Year-to-date scores on quality review instruments.

Random monthly chart reviews by physician peers are used to monitor 20 parameters related to the quality of the treatment teams' patient care. Two of the parameters— "Does the chart documentation support the diagnosis?" and "Is the treatment plan appropriate?"—are included among the physician performance indicators. Scores are annualized and reported as the percentage scored "yes."

Clinicwide indicators

Mean value of physician services for the entire clinic.

To calculate the value of physician services, the total value of direct services for all clinic physicians is divided by the number of full-time-equivalent (FTE) physicians assigned to the clinic

Mean number of new psychiatric assessments performed.

Accessibility of services is one measure of the center's efficacy in its mission to the community. Therefore, the total number of new assessments performed is divided by the number of FTE physicians assigned to the clinic.

No-show rates for psychiatric assessments and medication maintenance appointments.

The rate of no-shows affects quality of care and physician efficiency. It is monitored as the rate of missed appointments for each FTE physician.

Mean utilization of sick leave by physicians.

The unexpected absences of physicians can have a significant impact on efficiency and quality. Therefore, the total number of hours of physicians' sick leave are divided by the total number of FTE physicians assigned to the clinic.

Mean physician caseload for the clinic.

Proper distribution of physicians among clinics can enhance quality and productivity. The total caseload for each clinic is divided by the total number of FTE physicians assigned to the clinic

Physician turnover.

The quality of care in our center, as measured by the monthly peer reviews, appears to deteriorate with physician turnover. Therefore, an annualized count of the number of FTE physicians who voluntarily leave each clinic is kept.

Rate of case quality review.

The total annualized number of medical records reviewed by the clinic's quality review process is divided by the total caseload for the clinic.

Rate of patients lost to follow-up for more than 90 days.

As a means of addressing poor follow-up, the number of patients who received no services during the previous 90-day period is divided by the total number of FTE physicians assigned to the clinic.

Discussion

At each clinic, an administrative assistant to the medical director enters the data into two computer spreadsheets. The spreadsheet for physician performance indicators lists the physicians' names in the left-hand column and performance indicators across the top row of the worksheet. For each performance indicator, a centerwide mean and standard deviation are calculated by the spreadsheet program, allowing comparison of physicians' performance. A similar spreadsheet is developed for the clinicwide performance indicators.

Each physician receives the performance indicator spreadsheet for the entire division on a monthly basis, with all physicians listed. Individual physicians are asked to comment on their indicators that fall more than one standard deviation outside the mean for the group as a whole, whether the performance is relatively high or relatively low. Responses may reflect individual, team, or systemic successes or problems. Based on the response obtained, low performers may be asked to generate a plan of improvement for subsequent months.

A marked improvement in physician performance at the clinics was noted after the monitoring was initiated, although the monitoring may not have caused the improvement. For example, a 30 percent increase in the mean number of hours of direct service occurred over the first 12 months of monitoring. In our center, this increase added about $4,000 to each physician's value of services. Similar but less dramatic improvements occurred in many other areas.

The indicators have also been used to explore other issues about performance. A correlation of the value of services to the amount of direct service time provided information about variability in the use of billing codes. A study of the relationship of caseload size to direct service time shed light on the impact of a large caseload on the time spent with individual patients. Comparisons of caseload size and quality review measures (the diagnosis monitor and treatment plan monitor) gave information about the effect of high caseloads on the quality of care.

Conclusions

Despite its usefulness, the method presented here has significant limitations. For example, because our center measures improvements in patients' symptoms by instruments completed by physicians, our desire for objectivity left us without any indicators that were based on specific symptom measures.

Nonetheless, our method has proven to be an easy yet meaningful way to collect and use data on physicians' performance. The method is flexible enough to allow us to adjust the indicators based on stakeholder requests and discussions with medical staff. But in doing so, we remain mindful of the importance of objective and accessible data. We were initially concerned that our physician staff might feel threatened by the process. Instead, it generally has been experienced as reassuring, because the method compares them to peers rather than to an arbitrary standard. They also generally recognize the role of the indicators in demonstrating the value of psychiatric staff within the center. Stakeholders, too, seem reassured by the objectivity of our indicators, and by our determination to demonstrate both efficient and effective utilization of scarce resources.

Dr. Baker is medical director for adult mental health with the Mental Health- Mental Retardation Authority of Harris County, 2850 Fannin, Suite 200, Houston, Texas 77002 (e-mail, [email protected]). He is also clinical assistant professor of psychiatry at Baylor College of Medicine in Houston. John H. Greist, M.D., is editor of this column.