Veterans’ Perceptions of Behavioral Health Care in the Veterans Health Administration: A National Survey

The Institute of Medicine (IOM) (1) outlines six key dimensions of health care quality, including safety, timeliness, effectiveness, efficiency, equity, and patient centeredness. Patients are increasingly asked to report their perceptions of care across a range of dimensions via standardized surveys (2). For some domains, such as patient centeredness, self-reported perceptions may be the primary data source. Because patient centeredness reflects whether care is respectful or responsive to individuals’ needs, patients may be the best source for that information. However, patients can also report on numerous domains of quality, including interpersonal quality, technical quality, timeliness of care, and satisfaction with care. Therefore, patients’ self-reported perceptions may supplement medical record review and administrative data for other IOM domains, such as effectiveness.

Although positive perceptions of care among patients is a worthy objective on its own, prior research has demonstrated that patient perceptions are related to other key health care processes and outcomes, such as patients’ provider choice (3), adherence to treatment regimens (4), complaints (5), and health outcomes (6–11). These associations suggest that efforts to improve patient perceptions may also improve treatment outcomes.

Given the importance of patient perceptions of care, there is a need to investigate factors that contribute to variations in these perceptions. Identifying variations in these perceptions among different patient populations may contribute to strategies to improve care or address disparities. Studies suggest that patient perceptions, such as satisfaction and perceived barriers to care, may vary by psychiatric diagnosis (12–16). To extend this work, we examined perceptions of behavioral health care for a broad range of domains in a national sample of patients in the Veterans Health Administration (VHA) and compared perceptions by patients across selected diagnoses.

The VHA is the largest integrated health care organization in the country. Over 900,000 veterans received care for mental or substance use disorders from the VHA in fiscal year (FY) 2008 (17). These veterans accounted for a disproportionate share of VHA health care utilization and costs (17). The number of veterans receiving behavioral health services has been increasing for over a decade (17,18). The size, growth, and treatment needs of this population suggest that it is important to understand the perceptions of care among veterans receiving behavioral health services.

This article has two objectives. First, it provides national estimates for multiple domains of perceptions of care. Second, it presents an evaluation of whether patients’ perceptions differ by mental health diagnosis (bipolar disorder, depression, posttraumatic stress disorder [PTSD], and schizophrenia) or by whether the patient has a substance use disorder.

Methods

These data were collected as part of a national evaluation of VHA mental health services (17,19), which focused on patients who received care for disorders in one or more of the following five diagnostic groups: schizophrenia, bipolar I disorder, PTSD, major depressive disorder, and substance use disorders.

Participants and recruitment

Study population.

The study population was identified by using existing VHA administrative databases. The population consisted of veterans who received at least one inpatient episode or two outpatient encounters in FY 2007 from, or care paid for by, the VHA for one of the four mental health diagnoses listed above. Patients were assigned to one of four mental health diagnostic cohorts on the basis of the modal frequency of episodes of care. Ties were resolved by using the following rank order: schizophrenia, bipolar disorder, PTSD, and depression. In addition, patients with substance use disorder diagnoses were identified as a separate cohort. Patients with co-occurring substance use and mental disorders could be assigned to the substance use disorder cohort plus one mental health diagnostic cohort.

Given the concern that asking patients to recall the care they received in FY 2007 could result in recall bias, survey questions focused on mental health care during the 12 months prior to the interview date. Therefore, we restricted the study population to patients with any VHA health care utilization in FY 2008 to minimize the number of survey respondents who would report having received no care from the VHA in the 12 months before being contacted for the survey.

Sampling plan.

We stratified the study population by geographic region, mental health diagnostic cohort, and substance use disorder cohort. We randomly sampled 9,619 patients across these strata; 320 deceased patients were subsequently removed from the sample, leaving an eligible sample of 9,299.

Survey procedures

The survey was fielded between November 2008 and August 2009. Prospective participants received a letter and a study brochure one week prior to being contacted by telephone. An additional copy of these materials was sent to the mailing address on file after three weeks of attempting to contact the respondent. Interviewers made an average of four call attempts for each completed interview. After receiving a complete description of the study, the participants gave verbal informed consent. Completed interviews lasted approximately 30 minutes. Participants received a $10 check. The RAND Human Subjects Protection Committee approved all procedures.

Measures

Using an item adapted from the Experience of Care and Health Outcomes (ECHO) survey (20), we asked participants if they received “counseling or treatment” for mental health or substance use problems in the prior 12 months. For patients who endorsed receiving behavioral health care, subsequent survey items assessed patient perceptions across a variety of domains.

Demographics characteristics.

Demographic information was drawn from administrative data and survey data.

Functioning.

We assessed functional status by using the Veterans RAND 12-Item Health Survey (21). The survey has been used extensively with veteran populations and has been shown to effectively identify the incremental impact of several chronic diseases (21). Using this measure, we derived two composite scores reflecting mental and physical health functioning. Possible scores range from 0 to 100, with higher scores indicating better functioning. Both scores are norm-based and can be interpreted in relation to the distribution of scores in the 1990 adult U.S. population (mean±SD=50±10).

Perceptions of VHA behavioral health care.

We assessed patients’ perceptions of timeliness of care, the recovery orientation of care, perceptions of psychosocial services, and overall satisfaction. We assessed timeliness of care by using four items from the ECHO (20). Patients were asked if they needed counseling or treatment in the past 12 months. If so, they were asked to rate how often they received appointments as soon as they wanted on a 4-point scale, from 1, never, to 4, always. Patients were asked separately about routine and urgent care. Timeliness is reported as the percentage who endorsed “always.”

We assessed perceptions of the recovery orientation of VHA staff by using seven items from the Recovery Self-Assessment (22,23). The selected items assess whether staff listened to and respected the patient’s decisions, conveyed high expectations for the patient’s recovery, believed the patient could make his or her own life choices, asked about the patient’s interests, helped develop life goals beyond symptom management, helped to include other important people in treatment planning, and introduced the patient to role models or mentors. Possible responses range from strongly disagree to strongly agree. Because responses on these items are typically highly skewed, we report the percentage who endorsed “strongly agree.”

We assessed perceptions of two aspects of psychosocial services. We assessed need for and receipt of housing services in the past 12 months by using two adapted items from the HIV Cost and Services Utilization Study (24). Patients were asked if they needed help finding a place to live. If they answered yes, they were asked whether they had received help from the VHA. The same items were repeated to assess perceived need for help with employment.

We assessed patients’ overall perception of their VHA counseling or treatment by using one item from the ECHO. Patients were asked to rate their behavioral health care from 0 to 10, with 0 indicating the “worst counseling or treatment possible” and 10 indicating the “best counseling or treatment possible.” Patients who reported a rating of 9 or 10 were considered highly satisfied.

Perceived improvement.

We assessed patients’ perceived improvement by using five items from the ECHO (20). Four items ask about improvement in ability to deal with daily problems and social situations and to accomplish what they would like to do and improvements in symptoms in the past 12 months. There are five possible responses, ranging from much worse to much better. We report the percentage who endorsed “much better” or “a little better.” An additional item asks how much patients were helped by their counseling or treatment overall. Response options include not at all, a little, somewhat, and a lot. We report the percentage who endorsed “a lot” or “somewhat.”

Statistical analyses

A sampling weight was derived to adjust for the stratified sampling design (25). A nonresponse weight reflecting the inverse probability of each sampled veteran completing the survey was derived by using logistic regression analysis of the relationship of survey completion and veteran characteristics (26). To reflect the study population, all estimates were weighted by a final analysis weight that was the product of the sampling and nonresponse weights.

We present demographic and other descriptive characteristics of the sample and national estimates of patient perceptions. We evaluated whether perceptions varied by mental health diagnostic cohort or by substance use disorder cohort. Chi square tests were used to measure significant differences across diagnostic cohorts for the categorical outcomes. Subsequently, logistic regressions were conducted for each diagnostic cohort to test whether diagnosis remained a significant predictor after adjustment for age, race, sex, service-connected disability, mental and physical health functioning, utilization (outpatient visits and inpatient nights), and co-occurring mental or substance use disorder. Given that multiple statistical tests were conducted, p values were compared against an alpha of .01 to determine significance. For these analyses, this threshold corresponds to controlling the false discovery rate (the proportion of rejected null hypotheses that are actually true) to an alpha level of .05 (27).

Results

Sample characteristics

A total of 6,190 patients participated in the telephone interviews (67% response rate). These analyses focused on the 5,185 (84%) participants who reported receiving behavioral health care from the VHA in the past 12 months. Administrative data for the 1,005 patients who denied receiving VHA behavioral health care indicated that 534 (53%) had received such care within the prior 12 months, suggesting some error in recall or understanding of the question.

Demographic and other characteristics of the sample, in weighted percentages, are provided in Table 1. The average age was 58, over 90% were men, and about 63% were non-Hispanic Caucasian. A majority of patients at least had completed some college (59.6%), were not in the labor force (74.3%), and reported annual incomes of $30,000 or less (57.1%). More than half of patients had a service-connected disability for any condition, and approximately 40% of patients received VHA disability compensation for a diagnosis that was the same as the diagnostic cohort to which they were assigned. Approximately 6% of patients served in Operation Enduring Freedom or Operation Iraqi Freedom.

Respondents’ mental and physical health functioning scores were approximately 1.5 and 1.9 standard deviations below the general population, respectively. Among patients with at least one outpatient visit or one inpatient night in the past year, patients reported a median of 6.0 outpatient visits and 13.0 inpatient nights, respectively.

Perception of VHA behavioral health care

Table 2 summarizes patient perceptions of timeliness of care, staff’s recovery orientation, psychosocial services, and overall satisfaction. Among patients (61.1%) who reported making routine appointments for counseling or treatment in the past 12 months, nearly half reported always receiving appointments as soon as they wanted. Among the 32.9% of patients who reported making urgent appointments, 42.8% reported always receiving appointments as soon as they wanted. Perceptions of timeliness did not differ by mental health diagnostic cohort. Patients who were treated for a substance use disorder were significantly less likely than patients who were not treated for a substance use disorder to endorse always receiving routine and urgent appointments as soon as they wanted.

For the seven aspects of recovery orientation, the percentage of patients expressing strong agreement was highest for whether staff listened to and respected decisions about treatment (43.0%) and lowest for whether staff introduced patients to role models (13.9%). Patients with bipolar disorder were most likely to strongly agree that VHA staff believed that patients could make their own life choices, and patients with schizophrenia were most likely to strongly agree that VHA staff introduced them to role models or mentors. Significant differences were found in perceptions of patients with and without a substance use disorder for three of the recovery-orientation measures.

Overall, 13.7% of patients reported needing help finding a place to live, and 10.5% of patients reported needing help finding a job. Among patients who reported a housing need, 27.2% reported receiving help from the VHA. Among patients who reported an employment need, 28.4% reported receiving help from the VHA.

There was significant variation by mental health diagnostic cohort in the proportion of patients reporting a need for housing and employment help and receipt of housing help. Patients in the PTSD cohort were the least likely to report both needs and were also the least likely to report receiving housing help. Patients in the substance use disorder cohort were significantly more likely than patients who were not in the cohort to report needs for housing and employment services and were more likely to receive housing, but not employment, services.

Overall, 42.3% of patients reported being highly satisfied with VHA behavioral health care. Among mental health diagnostic cohorts, patients with schizophrenia were most likely to be highly satisfied with their care (p=.003). Patients with a substance use disorder were significantly less likely than patients without a substance use disorder to report being highly satisfied with their care.

Perceived improvement

Nearly three-quarters (73.9%) of patients reported being helped by their counseling or treatment (Table 3). Fewer patients reported improvement in their ability to deal with daily problems (41.4%), deal with social situations (34.3%), and accomplish goals (30.8%) or improvement of their symptoms (31.7%). The modal response was “about the same” for all four of these items. Patients with schizophrenia were most likely to report improvement in all four areas, whereas patients in the PTSD cohort were least likely to report improvement. Patients in the substance use disorder cohort were more likely than patients who were not in the cohort to report improvements in all four areas.

Multivariate model results

Tables 4 and 5 provide the results of logistic regression models that evaluated differences in perceptions of VHA behavioral care and perceived improvement by mental health diagnostic cohort and substance use disorder cohort after the analyses controlled for demographic characteristics, mental and physical health functioning, service utilization, and comorbid substance use or mental disorder.

Compared with patients with bipolar disorder, patients in the PTSD and schizophrenia cohorts had the highest odds of any mental health diagnostic cohort of expressing strong agreement that staff introduced patients to role models or mentors (p<.01) (Table 4). There were no other significant differences across mental health cohorts for timeliness of care, other aspects of recovery orientation, psychosocial services, and satisfaction with care.

Patients with a substance use disorder were significantly less likely than patients without a substance use disorder to endorse two of the recovery-orientation items; remained more likely to report a perceived need for housing and employment services and to report receipt of housing help; and remained significantly less likely to report being highly satisfied with the care they received.

Patients with a substance use disorder remained significantly less likely than those without a substance use disorder to report being helped by the care that they received (Table 5). All other differences in perceived improvement across mental health diagnostic and substance use disorder cohorts were reduced to nonsignificance after the analyses controlled for the covariates.

Discussion

Overall, patient perceptions of behavioral health care delivered in the VHA were favorable, but there was room for significant improvement across all domains. The percentages of patients reporting timeliness of care (49.6%, routine care; 42.8%, urgent care) compared favorably with existing data on the percentages of patients receiving mental health care through both public (40.0% and 38.0%, respectively) and private (42.9% and 42.3%, respectively) insurance plans (28). Overall satisfaction with care (42.3%) was slightly lower than satisfaction with private plans (45.7%) and similar to satisfaction with public plans (43.0%) (28). Although most patients reported that treatment was helpful (73.9%), rates of satisfaction in four specific areas were far lower (30.8%−41.4%). These estimates were lower than the percentages of patients in public and private plans who described their treatment as helpful (83.8% and 87.6%, respectively) and who reported improvement (55.2%−69.7% and 58.9%−81.1%, respectively) (28).

Direct comparison of the data from this sample and data from the literature is challenging because comparison data for matched samples were not available. The comparison samples included a higher proportion of patients who were female, younger, and Caucasian than did our sample of veterans. Although results are mixed across the patient perceptions literature, women and younger patients tend to report less positive ratings, and Caucasians tend to report higher ratings (29–31). Nonetheless, the national estimates of patient perceptions of care presented here could be used as benchmarks with which to compare future health system performance.

Our findings suggested that, in this population, there are approximately 70,000 veterans who may need housing assistance and 54,000 veterans who may need employment assistance nationwide. The multivariate analyses suggested that patients with a substance use disorder were three times more likely to report needing help finding a place to live and twice as likely to report needing help with employment compared with patients without a substance use disorder.

Patients with a substance use disorder reported lower satisfaction with care and perceived their treatment to be less helpful than patients without a substance use disorder. What contributed to these differences is unknown, but one possibility is that the diagnosis was associated with variation in expectations for care, types of care received, or interactions between providers and patients. Patients with a substance use disorder may have less favorable perceptions of care if providers interact less warmly or provide lower quality of care, particularly if providers have negative biases toward patients with these diagnoses (32,33). Understanding more about factors that contribute to variations in patient perceptions of care, particularly for patients with substance use disorders, could inform quality improvement efforts.

The differences in recovery-orientation ratings by persons with and without a substance use disorder suggest areas for quality improvement and improvements in overall satisfaction and perceived helpfulness of treatment. Specifically, patients with substance use disorders were significantly less likely than patients without substance use disorders to report that staff listened to and respected their decisions and helped to include other important people in treatment planning. This finding suggests that staff training in a recovery-orientation model or in motivational interviewing (a patient-centered approach that emphasizes active listening) (34) could be helpful. The VHA has recently increased its focus on patient-centered care (35), which could lead to improvements in perceptions of care. To reduce the differences in the extent to which staff introduce patients in different mental health diagnostic cohorts to role models, it may be helpful to increase availability of mutual support groups (36). Unfortunately, reports of patients’ perceptions of care have been criticized for not providing clear direction on how these variables can be improved (37), suggesting that this is an area for future work.

Our study had limitations. Although patient perceptions of care are valuable, they are subject to limitations similar to those of other data gathered through patient self-report (38). For example, patients may have difficulty recalling the care they received, differentially weight the value of positive and negative experiences, differentially weight recent versus distal experiences, or base their perceptions on varying expectations for their care. In addition, because diagnostic cohorts were identified through administrative data, they may not accurately represent diagnoses assigned through standardized clinical assessments.

In this study, we focused on a broad range of patient perception domains. Although this approach allowed us to characterize patient perceptions of VHA behavioral health care, it limited our ability to thoroughly assess each domain. Further, most of the variables we examined do not have benchmarks of acceptable performance, and matched sample comparison data were not available; however, prior work suggests that quality improvement efforts can lead to improvements in patient perceptions of care (39,40). Nonetheless, this study extended prior work by focusing on a large, national sample of veterans who received behavioral health care and examining a broad range of patient perceptions of their behavioral health care.

Conclusions

Patient perceptions of care represent a potentially important assessment of health care quality. This study provided national estimates of patient perceptions of care for mental and substance use disorders among veterans. Further, it highlights the potential importance of diagnosis in assessing patient perceptions of care.