Consumer-Directed Care for Beneficiaries With Mental Illness: Lessons From New Jersey's Cash and Counseling Program

Mental disorders are common in the United States and internationally. An estimated 26% of Americans ages 18 and older—approximately 57.7 million people—experience diagnosable mental disorders each year ( 1 , 2 ). Moreover, mental disorders are the leading cause of disability among Americans aged 15 to 44 years ( 3 ).

Publicly funded mental health services account for almost three of every five dollars spent on mental health services, and the Medicaid program is the single largest source of public funding ( 4 ). Medicaid beneficiaries with physical or cognitive disabilities who qualify for home- and community-based assistance with personal care typically rely on Medicaid-certified home care agencies. However, the agencies' assistance often fails to reflect the beneficiaries' needs and preferences ( 5 ). These limitations may adversely affect the quality of life for beneficiaries and their unpaid caregivers. The detrimental consequence for both groups may, in turn, force beneficiaries into nursing homes ( 6 ).

One of the most innovative and flexible consumer-directed care options currently undergoing evaluation is the Cash and Counseling Demonstration and Evaluation (CCDE) program. The CCDE program is an expanded model of consumer-directed care. It provides consumers a monthly budget to hire their preferred workers, including family members, and to purchase care-related services and goods permitted by states. The CCDE program also allows consumers to designate representatives, such as relatives or friends, to help make decisions regarding their care. The CCDE program offers counseling on hiring and managing caregivers and fiscal management services to help participants handle their program responsibilities. These tenets seek to provide a viable option for consumers of all ages and abilities, including those with a diagnosis of mental illness.

The CCDE program was originally implemented in three states: Arkansas, New Jersey, and Florida. Previous research has demonstrated large, positive effects on the well-being of Medicaid beneficiaries from all three states ( 6 , 7 , 8 , 9 ).

However, critics of consumer-directed care and of the CCDE program raise several concerns regarding consumers' welfare. Some object that consumer-directed care may be inappropriate for beneficiaries unable to manage their own care effectively and safely, such as consumers with a mental illness diagnosis. Others worry that the workers hired by consumers with a mental illness diagnosis may not receive the formal training or supervision available to agency workers. This could lead to inadequate or substandard care.

In response, a study using secondary data analysis of the effects of the CCDE program on personal care and well-being of elderly Medicaid beneficiaries in Arkansas with a diagnosis of mental illness was conducted and yielded very positive results ( 10 ).

Using the same outcome measures as the general evaluation ( 5 , 6 ), the study presented here examined the effects of the CCDE program for nonelderly adult participants in New Jersey who had a mental illness diagnosis on any of their Medicaid payment claims. We examined how consumer direction affected the use and quality of consumers' personal care assistance, as measured by their satisfaction with care, the frequency of unmet needs, and the incidence of adverse health events arising from inadequate care.

Methods

Program description

The demonstration enrollment period for New Jersey's CCDE program, Personal Preference, was November 1999 to July 2002. A feature of the CCDE program was random assignment of participants into treatment and control groups, allowing estimation of the program effects through differences between the treatment and control groups ( 5 , 6 ). In this study we assessed program outcomes for those with a diagnosed mental illness in the following areas: the use of personal care services (for example, the likelihood of receiving paid assistance), clients' perceptions of caregivers' reliability and caregivers' attitudes, satisfaction with overall care arrangements and clients' life satisfaction, perceived unmet needs, and adverse events.

Data

As in the original evaluation, this study used two data sources: the CCDE baseline survey and the nine-month follow-up survey. The Medicaid claims data set provides the data on consumers' mental illness status. The diagnosis was captured only if there was a Medicaid claim related to mental illness in the year before the demonstration; Medicaid claims were based on the Chronic Illness and Disability Payment System (CDPS). Those without Medicaid coverage and Medicaid beneficiaries who could have had a psychiatric condition but had not received treatment for this condition in the pre-enrollment year were therefore excluded.

Mental illness was divided into three subcategories—high cost, medium cost, and low cost—according to the level of medical expenditures that would be expected for a particular diagnosis. We created a dichotomous measure of mental illness status: 1 for those with and 0 for those without a diagnosis. By using this definition, the rate of mental illnesses among the nonelderly participants (aged 18 to 64) at follow-up was 34% (228 of 680 consumers). Among those with a diagnosis of mental illness, 80% (N=183) belonged to the low-cost subcategory, 3% (N=6) belonged to the medium-cost subcategory, and 17% (N=39) belonged to the high-cost subcategory. Medicaid claims data do not provide any information about the severity or treatment status of the participants.

Consumers with a diagnosis of mental illness were roughly evenly distributed across treatment statuses. For outcome measures of consumers' satisfaction and unmet needs, we excluded sample members with a proxy respondent who also was a paid caregiver (about 28% at baseline and at follow-up surveys) because they may not have been able to give objective answers. Therefore, for some outcome variables in this study, the sample size is less than 228.

Outcome measures

Outcome measures included both objective and subjective measures. Objective measures included whether consumers received paid care at the ninth month after enrollment, whether consumers received care services on weekday mornings, weekday evenings, or weekends (as opposed to business hours during the week), and what health-related adverse events and health problems they experienced in the past month. In order to measure other components of the quality of personal care services, beneficiaries offered perceptions and opinions regarding the following: satisfaction with their paid caregivers' reliability and schedule; perceptions of their paid caregivers' attitudes and performance; satisfaction with their overall care arrangement and perceptions of quality of life in general; and their unmet needs, adverse events, health problems, and health status in general. Many of these outcome measures are clients' perceived satisfaction with various aspects of caregivers' services. Although using client's self-perceived satisfaction as a measure of success of an intervention has its limitations, perceived satisfaction measures an important dimension of quality of care and is often used to assess services in a number of settings ( 11 , 12 ).

Analysis methods

Most outcome variables were categorical, and our major independent variable was binary: treatment status (treatment group=1, control group=0). Therefore, we performed a series of chi square analyses to test the impact the CCDE program had on various outcome measures, comparing the expected and observed frequencies. We next estimated the impact of the CCDE program on consumers with a mental illness diagnosis through logistic regression models. To yield robust and conservative results in the logistic models, we included baseline covariates of demographic characteristics, health and functioning, satisfaction with care arrangement, and receipt of publicly funded home care. Several outcome measures were derived from survey questions with 4-point scales (for example, degree of satisfaction). Following earlier practice ( 9 , 13 ), we converted each 4-point scale into two binary measures: 1 for the most favorable rating (for example, very satisfied) and 0 for other ratings (for example, somewhat satisfied or dissatisfied). We then estimated the impact of the program on each outcome measure.

Because this study includes a key binary predictor variable (treatment status), we report both logistic coefficients and odds ratios for each of the outcome measures. For each outcome measure, we used a sequential procedure. First we included baseline characteristics as covariates and then added the binary independent variable (treatment status) in the model.

Results

Major characteristics of the sample

Table 1 presents major characteristics of the sample by treatment status: the treatment group was made up of CCDE participants, and the control group was made up of participants who received services provided by an agency. As shown, the sample of nonelderly persons with a diagnosis of mental illness was disproportionately female and older (over 40 years old); about half of them perceived their health as poor, and over 60% needed help to get in and out of bed. This is expected because the prevalence of mental disorders is higher among females. Because the baseline results have more females in the treatment group and because mental illness prevalence is correlated with being female, it is important to control for baseline characteristics. No statistically significant difference existed between the treatment and control groups except for gender.

Enlarge table

Chi square analysis

For the use of personal care services, chi square analysis identified significant differences between the treatment and control groups. An objective outcome measure showed a statistically significant difference between the two groups: 102 clients (94%) in the treatment group versus 92 (77%) in the control group received paid care at nine months ( χ² =11.86, df=1, p<.001). In addition, we performed the same chi square analyses for 25 outcome measures, covering satisfaction with paid caregivers' reliability and schedules, satisfaction with the relationship with the paid caregiver, perception of caregivers' attitudes, satisfaction with paid caregivers' performance and transportation assistance, and satisfaction with overall care arrangements, as well as client life satisfaction, unmet needs, adverse events, health problems, and general health status.

Out of the 25 outcome measures, 11 belong to the area encompassing adverse events, health problems, and general health status. Examples are "fell in past month," "saw a doctor because of fall, cut, burn, scald, etc." In this area, our chi square analyses yielded the same findings as did the earlier overall program evaluation by Mathematica Policy Research (MPR) ( 8 , 9 ): there were no statistically significant differences between consumers in the treatment and control groups, indicating that under the CCDE program, care was at least as safe as agency-directed care. In terms of other outcome measures in other areas, chi square analysis demonstrated positive, although not always statistically significant, effects of the CCDE program.

To save space, we do not report the chi square analysis results here. To yield conservative results, we needed to control for the effect of baseline characteristics through logistic regression analysis.

Multivariate logistic regression analysis results

In multivariate logistic regression analyses, we used hierarchical analysis strategy. As the first step, we included the clients' baseline demographic characteristics, health and functioning, use of personal assistance, and satisfaction with care. Then we added treatment status to the model to estimate the "pure" effect of the CCDE program on use of personal care services and quality of service after removing the effects of all the variables that were controlled. As shown in Table 2 to Table 5 , the covariates that were controlled include all the baseline characteristics reported in Table 1 except for "proxy completed the survey." Our initial logistic regression analyses included all of the 25 outcome measures used in chi square analyses. To save space, Tables 2 to Table 5 present the results for 16 selected outcomes from the multivariate logistic models. The full results for other outcome measures and chi square analysis results are available upon request.

Enlarge table

Enlarge table

Table 2 presents logistic regression coefficients and odds ratio for effects of the CCDE program on four outcome measures reflecting client satisfaction with caregivers' schedule and reliability.

As mentioned earlier, all the outcome variables are dichotomous. The first outcome measure is "always completed task." Always is coded as 1 and otherwise is coded as 0. As shown in Table 2 , 166 participants were involved and had complete data. Out of the ten baseline characteristics used as control variables (covariates), one has a significant and positive effect—very satisfied with overall care arrangement. With the covariates in the model, consumers in the treatment group had a 47% (odds ratio=1.47) greater likelihood of saying that the paid caregivers always completed the task, compared with those in the control group. However, after the analyses controlled for the effects of baseline covariates, the difference was no longer significant.

The second outcome measure is never arrived late or left early. As shown, with the ten covariates in the model, consumers in the treatment group had more than twice a greater likelihood of indicating that the caregiver never arrived late or left early, compared with those in the control group (p<.05).

The third outcome measure is whether the consumer was satisfied with the caregiver's schedule for providing service. After controlling for the effects of the ten covariates, consumers in the treatment group had a significantly higher likelihood of claiming that they were very satisfied, compared with those in the control group (p<.01).

The fourth outcome is an objective outcome measure: whether consumers received their caregivers' service on a weekday morning, weekday evening, or weekend. Again, there was a statistically significant difference between the groups, with the treatment group being more than twice as likely to receive services during these times (p<.05).

Table 3 details outcome measures reflecting consumers' satisfaction with their relationship with the caregiver, perception of their caregivers' attitudes, and two measures of their perception of their caregivers' service performance. As shown in Table 3 , all of the four outcomes demonstrate CCDE's positive effect after the analyses controlled for the effects of the baseline characteristics. However, only satisfaction with caregiver's help around the house and community was statistically significant (p<.05).

Enlarge table

Table 4 addresses two aspects of the effects of the CCDE program—consumers' satisfaction with their overall care arrangements and satisfaction with the way they are spending life—and two measures of unmet needs. As shown in the table, after the analyses controlled for the effects of the ten baseline covariates, including "very satisfied with overall care arrangement" at baseline, the first two outcome variables demonstrate the CCDE program's significant positive effects (very satisfied with overall care arrangement [p<.001] and very satisfied with the way life is being spent [p<.05]). For the third and fourth outcome measures (unmet needs), both odds ratios are less than 1, indicating that consumers in the treatment group were less likely to claim unmet needs for help with household activities (such as meal preparation, laundry, housework, and yard work) and help with routine health care (such as help taking medications, monitoring blood pressure, and performing exercises). However, the category unmet needs for help with household activities was not statistically significant. The odds ratio of .43 for need help with routine health care indicates that consumers in the treatment group had about half the likelihood of claiming such needs, compared with those in the control group (p<.01).

Enlarge table

Table 5 presents CCDE's effects on issues reflecting adverse events, health problems, and general health status. MPR's original study included several measures of adverse events in the past month including "fell," "saw a doctor because of fall," and "saw a doctor due to cut, burn, or scald," and "was injured while receiving paid help." We analyzed all of these measures and found that the CCDE program had no significant effect in any model. To save space, we included just one of these adverse event measures—"fell in past month"—in Table 5 . The odds ratio of .61 indicates that consumers in the treatment group were less likely to fall, although the difference was not statistically significant.

MPR's measures of heath problems in the past month included five variables. We report two of them in Table 5 : "shortness of breath developed or worsened" and "had a respiratory infection." As shown, consumers in the treatment group experienced significantly less likelihood of reporting respiratory infection problems than those in the control group (p<.05). The other three measures of health problems examined but not reported here are "had a urinary tract infection," "bedsores developed or worsened," and "spent night in hospital or nursing home in past month." We found no significant differences between those in the treatment and the control groups, indicating that people in the treatment group were at least not worse off. The last outcome measure in Table 5 is perceived general health status: "current health poor relative to peers." As shown, no significant difference between the two groups existed in our data. It is noted that those who claimed to be in poor health at baseline were more likely to make the same claim at ninth months after enrollment. Again, multivariate logistic regression analyses revealed that in the CCDE program, care was at least as safe as usual treatment.

In summary, both chi square analyses and logistic analyses suggest that overall the CCDE program increased the likelihood that consumers with a diagnosis of mental illness received paid care service, better met the needs of service delivery time and reliability, increased clients' satisfaction with the overall care arrangement, increased their satisfaction with the caregivers' performance, and increased their perceived quality of life. Also, the CCDE program somewhat reduced unmet needs, and in no case did the program increase adverse events and health problems. In short, our findings are consistent with earlier findings of the CCDE program: the CCDE program worked just as well for clients with mental illnesses as for those without mental illnesses ( 9 ).

Discussion

Our results demonstrate that the CCDE program had positive effects on personal care for and well-being of Medicaid beneficiaries with a diagnosis of mental illness without any increased risk of adverse events. If clients have mental illnesses and want to manage their own care, the CCDE program may be an appropriate option. These results hold up across a variety of outcomes, including receiving paid care at nine months, consumers' satisfaction with caregivers' reliability and schedule, caregivers' care performance, care arrangements, and caregivers' relationship and attitudes, as well as satisfaction with quality of life and the reduction of some unmet needs. These results for nonelderly adult consumers with a diagnosis of mental illnesses are consistent with earlier analyses of New Jersey Medicaid recipients in general ( 9 ).

Policymakers have expressed doubts about the appropriateness of consumer-directed care, in general, and in the CCDE program specifically, for consumers with mental illnesses. The findings presented here indicate that CCDE consumers with mental illnesses in New Jersey were able to successfully manage the cash option. Our findings are consistent with the findings from our earlier analysis for elderly Medicaid beneficiaries with mental illness in Arkansas ( 10 ).

Three features of the CCDE program are crucial for beneficiaries with a diagnosis of mental illness: consumers in the treatment group could appoint a representative (a family member or friend) to help them manage all cash option responsibilities; they were also offered the assistance of consultants and a bookkeeping service for managing their fiscal responsibilities. These features could alleviate many policy makers' concerns regarding the ability of beneficiaries with a diagnosis of mental illness to navigate the system.

With the growing need for long-term care services and limited available resources, a consumer-directed cash option may help avoid institutionalization and other high-cost options for consumers with mental illnesses ( 14 ). This study's findings have implications for both consumers and their caregivers. For consumers, the findings presented here indicate an improved quality of care in the CCDE program, reflected by greater satisfaction with several program aspects. For caregivers, the flexibility and choices allowed in the CCDE program are likely to allow consumers and their families to choose dedicated workers who are able to work with consumers with mental illnesses. This process may create a better match between caregivers and consumers. This process may lead to improved caregiver recruitment and retention—important to consider in the context of the looming nationwide long-term caregiver shortage.

Study limitations

Because our findings are based on one consumer-directed program for nonelderly consumers in New Jersey, they may not generalize to other age groups or other states. In addition, all outcome measures were from the survey completed nine months after enrollment. To ensure the positive effects of the program, longitudinal data over a longer period of time are needed.

The psychiatric diagnosis was captured only if there was a Medicaid claim related to it in the year before the demonstration. Clients could have had a psychiatric condition that was not captured in the claims data if they had not received treatment in the pre-enrollment year. In addition, we do not have detailed information regarding the severity of their mental illness. We also cannot distinguish between clients with one claim and multiple claims.

The outcome measures in this study include clients' perceived satisfaction with various aspects of the service they received. Although satisfaction measures assess an important dimension of quality of care, they are subjective measures, susceptible to cultural norms and bias. Another limitation is the exclusion of consumers with a diagnosis of mental illness who used a proxy to complete the survey. These consumers might be those with the most severe levels of mental illness.

Research recommendations and future analyses

This study provides important evidence that adult consumers with mental illness faired well in the New Jersey CCDE program. However, additional analyses of data from all three CCDE states and consumer populations would expand our knowledge of this topic. Analyses of survey data from paid and informal caregivers for consumers with mental illness would also add to our understanding of the impact of the CCDE program on those key individuals. Finally, analysis of ethnographic stories about consumers with mental illness would also add to our understanding of the impact of the CCDE program on those key individuals. Finally, analysis of ethnographic stories about consumers with mental illness, bringing in the points of view of key informal caregivers, paid workers, and consultants, would provide a richer understanding of the qualitative aspects that made the program successful.

Medicaid programs and political environments differ considerably across states. The three states in the original CCDE program adhered to the basic CCDE tenets, but they implemented their programs in different ways. To achieve a more complete understanding of the conditions for CCDE program success for those with mental illnesses, future research should extend the current analysis to several other age and state groups: nonelderly Arkansas consumers, elderly consumers in New Jersey, nonelderly and elderly adult consumers in Florida, and children in Florida.

Conclusions

By examining outcome measures—including satisfaction with care arrangements, consumers' perceptions of paid caregivers' attitudes, unmet needs, adverse events, and satisfaction with life—this study offers evidence that, from the perspective of consumers, the CCDE program is appropriate for participants with a diagnosis of mental illness. For most outcome measures the CCDE program demonstrated a positive effect after baseline characteristics were controlled for. The analysis of measures of adverse events, health problems, and general health status did not yield statistically significant differences between the control group and the treatment group, indicating that the CCDE care was at least as safe as agency-directed care. Considering the growing need for long-term care services and the limited resources available, a consumer-directed option can be a valuable alternative for persons with a diagnosis of mental illness.

Acknowledgments and disclosures

This study was funded by the Department of Health and Human Services, Substance Abuse and Mental Health Services Administration, Center for Mental Health Services, under contract 06M000133.

The authors report no competing interests.