Responses of Children With Different Intake Profiles to Mental Health Treatment

Many children with serious emotional disturbances are being served by agencies whose primary responsibilities do not include the provision of mental health services (1). To meet the needs of these children, multidisciplinary, multiagency, child-centered, family-focused, community-based, culturally competent systems of care have been developed. As defined by Stroul and Friedman (2), a system of care is "a comprehensive spectrum of mental health and other necessary services which are organized into a coordinated network to meet the multiple and changing needs of children and adolescents with severe emotional disturbances and their families."

Many communities in the United States have developed extensive collaborations for children with serious emotional disturbances and their families that link mental health services, such as prevention, outpatient treatment, home-based care, emergency services, therapeutic foster care, and inpatient care, with services and support from other agencies and systems, such as education, juvenile justice, social welfare, health care, and substance abuse treatment (3).

In 1993 the child, adolescent, and family branch of the Center for Mental Health Services of the Substance and Mental Health Services Administration began a demonstration program aimed at the development of local systems of care. To date 65 sites have been funded to develop and implement systems of care for children with serious emotional disturbances and their families (4). The East Baltimore mental health partnership, one of the originally funded sites, is a joint community, city, and state effort to develop and implement a system of care (5). In Baltimore, agencies involved in the provision of services to children collaborated to provide a continuum of care to redress the fractured and inconsistent manner in which many children and their families were being served. These agencies incorporated the key elements of systems of care and introduced more intensive services than those traditionally provided, including wraparound services, respite care, and parental support services (2,5,6,7).

Since 1994 the East Baltimore mental health partnership has coordinated a comprehensive multiagency program of services to children and families. The target population includes children and adolescents experiencing a substantial inability to function in their families, schools, or community who are 18 years old or younger, have a DSM-IV diagnosis, are being served by at least two community agencies, and whose disorder has either persisted or can be expected to persist for at least one year.

These children are referred to the partnership from a variety of public service agencies: schools, mental health centers, the departments of juvenile justice and social services, and the Child Advocacy Network. Children may also be referred when services are planned for or provided to their siblings. Children are referred from public service agencies because of behavioral or emotional problems and because they have needs that the agency cannot meet. As a result, compared with more traditional outpatient services, the multiagency coordination of services provided by the partnership is a longer-term, broader-based service plan.

Even though many of the children referred for services are similar with regard to neighborhood, race, and socioeconomic status, variation in the profiles of these children, as well as children referred into other systems of care, has been well documented (8,9,10,11). Specifically, some factors have been shown to vary considerably within the referred population, such as the referring agency, diagnosis, age, gender, history of substance use, history of physical abuse of the child, previous psychiatric hospitalization of the child or parent, history of family mental illness or violence, problem behaviors, and psychosocial functioning.

Little if any research has been published on whether children who enter these collaborative networks with different profiles show equivalent changes in their functioning after referral or whether changes in functioning vary as a function of these profiles. Previous research points out that if we are to determine the clinical effectiveness of systems of care and for whom systems of care work best, this type of subgroup analysis is required (12).

This study determined and compared changes in functioning for children with different presenting problems, referral sources, risk factors, and demographic profiles. We hypothesized that children receiving services from a system of care will show improved functioning over time regardless of variation within these categories. A single-group pretest-posttest design was used to examine this hypothesis.

Methods

Community description

The East Baltimore community is a densely populated, relatively small urban area with approximately 20,000 residents under age 18. Ninety-one percent of these children are African American, and more than half live in households with incomes below the poverty level.

System-of-care services

The components of the East Baltimore mental health partnership system of care include case management, therapeutic foster care, crisis outreach, assessment and evaluation, intensive home-based services, outpatient care, family support services, recreation services, and school-based services. On referral to the partnership, each child and family is assigned a family resource specialist. Specialists are master's-level mental health clinicians who oversee and coordinate the development and implementation of service plans. In addition, they provide direct service to their assigned children and families in the form of individual and family counseling. Children and families also receive recreational instruction, which includes instruction in the arts, music, and culture, and support services, which include family support, respite care, tutoring, and mentoring, among other services. Assessment and crisis services are also available for all referred children.

Participants

Between August 1994 and November 1997, 478 children were referred for intensive community-based mental health services. Of the 478 children, 311 (65 percent) were boys. The mean±SD age at referral for this group was 11.1±2.3 years, and ages ranged from three to 17. A total of 459 (96 percent) were African American. A total of 129 families (27 percent) reported a gross annual household income below $5,000, 186 (39 percent) reported an income between $5,000 and $10,000, and 76 (16 percent) reported an income between $10,000 and $15,000.

Of the 478 children, 124 (26 percent) were referred from Baltimore City public schools, 96 (20 percent) from the Baltimore City Department of Social Services, 81 (17 percent) from the Baltimore City Department of Juvenile Justice, 67 (14 percent) from the Child Advocacy Network, and 57 (12 percent) from the Johns Hopkins Children's Mental Health Center. The remaining 53 children (11 percent) were siblings of referred children.

All 478 children and their families were eligible to participate in an ongoing evaluation. A total of 406 eligible children and their families (85 percent) agreed to participate in the evaluation. Race, gender, and age of the child did not differ between those who participated and those who did not. In addition, no differences were found between the groups in gross household income, the number of child and family risk factors at the time of referral, or the number of child and family services received before referral. In both groups the most common primary presenting problem of the child was physical aggression.

The sample for this study was composed of the 203 children who agreed to participate, who had been receiving services from the partnership for at least six months, and who had functional assessment ratings both at intake and at six months.

Measures

The Child and Adolescent Functional Assessment Scale (CAFAS) (13) is included in the ongoing evaluation protocol of the East Baltimore mental health partnership to provide an assessment of functional impairment at six-month intervals. The CAFAS is a clinician rating scale used to assess child functioning across five life domains: role performance at home, in the community, and at school; behavior toward others; moods and emotion, including self-harmful behavior; substance use and abuse; and thinking. The scale yields five subscale scores as well as a total functional impairment score. The total score ranges from 0 to 150, with higher scores indicating greater functional impairment. This investigation used only the total score. Interrater reliability and validity have been demonstrated in previous studies (15). In addition, for the study reported here, a structured self-training protocol was used, and all raters had to achieve correspondence between their ratings and the established criterion before rating children (16).

The CAFAS is less vulnerable to respondent bias than other commonly used instruments such as axis IV of DSM-IV and the Children's Global Assessment of Functioning (CGAF), because the rater generates scores for life domains rather than global scores. In addition, regression analyses have indicated that the CAFAS total score at intake is a stronger predictor of the cost and number of services received than other measures such as the CGAF and the Child Behavior Checklist (17).

Statistical analyses

For these analyses, we divided subjects into two school-anchored age categories—prekindergarten through elementary and middle school through high school—and five referral sources —Department of Juvenile Justice, Baltimore City public schools, the community mental health center, Department of Social Services, and other, which primarily included children referred from the Child Advocacy Network and siblings of previously referred children. We used two collapsed diagnostic categories: internalizing, which included mood, adjustment, and anxiety disorders, and externalizing, which included attention-deficit hyperactivity disorder, conduct disorder, and oppositional defiant disorder.

We assessed the presence of risk by taking the sum of 14 possible self-reported risk factors that are assessed at intake. The clinician coded these risk factors as present, 1, or absent, 0. This procedure has been used to assess risk in evaluations of other systems of care (8). The risk factors are listed in Table 1, with the number of children and the percentage of families reporting each. We then used a median split to dichotomize the scale into low risk, two or fewer risk factors, and high risk, more than two.

We conducted five independent general linear model repeated-measures analyses of variance (GLM-RM ANOVAs). These analyses assessed the interaction between change in the CAFAS score over time and gender, age, diagnosis, referral source, and level of risk, respectively. Because of small subgroup sizes and empty cells, we chose to avoid the combined analyses of gender, age, diagnosis, and referral source and their relationship with change in the CAFAS score over time.

A priori power analyses indicated sufficient power to detect a 10-point difference in CAFAS scores between groups for each of the five analyses. Power estimates ranged from .58 for the diagnosis and referral source analyses to .87 for the risk analysis.

Results

Sample description

Table 2 presents data on the referral source. The referral pattern for the sample of 203 children was similar to that for the population served by the East Baltimore mental health partnership. The Baltimore City public school system was the single largest referral source (38 percent), followed by the Department of Social Services (22 percent), the community mental health center (16 percent), and the Department of Juvenile Justice (12 percent). Sixty-six percent of the children were boys, and 67 percent were of prekindergarten to elementary school age.

As Table 2 shows, gender and average age of the referred children varied by the referring agency. For example, most referrals from juvenile justice were boys, and they tended to be older than those referred from other sources. Children referred from social services were mostly girls. Those referred from the public school system tended to be younger than those referred from other sources. The average number of risk factors was relatively stable across referral groups, with the exception of children referred from juvenile justice, who tended to report fewer risk factors.

Changes in functioning

Changes in functioning varied considerably from baseline to six months. Changes in the CAFAS score ranged from an 80-point decline to an 80-point improvement over the study period. At six months 31 children (15 percent) had the same CAFAS score as at baseline and 121 children (60 percent) had a lower score. Fifty-one children (25 percent) had a higher score at six months than at baseline.

Table 3 presents the baseline CAFAS score for each group and the mean±SD change in the CAFAS score over six months. On entry into the system of care, boys were more impaired on average than girls. However, the mean change in functioning was similar for both. Older children were more impaired on average than younger children, but the change in functioning was similar for the two groups. At intake children referred from juvenile justice were more impaired on average than children from other referral sources, and children in the "other" category were least impaired. Children referred from both the juvenile justice system and community mental health centers demonstrated the greatest mean change in the CAFAS score.

Children in the high-risk category were more functionally impaired on average than those in the low-risk group, although the mean change in the CAFAS score from baseline to six months was similar for the two groups.

As expected, the within-subjects effect for the change in the CAFAS score from baseline to six months was significant (F=34.61, df=1, 201, p< .001), indicating improvement in functioning. In addition, between-group differences were found for gender (F= 11.07, df=1, 201, p<.001), grade (F= 14.09, df=1, 201, p<.001), referral source (F=4.10, df=4, 198, p<.003), and risk level (F=11.06, 1, 201, p< .001) but not for diagnostic category.

We evaluated the interaction terms of each model to assess the hypothesis of differential improvement as a function of referring agency, gender, age, diagnostic category, and risk level (Table 3). The changes in the interaction between the CAFAS score and the referral source and between the CAFAS score and gender were not significant, indicating that groups from all referral sources as well as boys and girls had statistically equivalent rates of improved functioning over the six months.

The change in the interaction between the CAFAS score and age also was not significant, indicating equivalent rates of improvement for the prekindergarten-elementary group and the middle-high school group. The change in the interaction between the CAFAS score and diagnostic category was not significant, indicating equivalent rates of improved functioning for all diagnostic categories. Finally, the evaluation of the CAFAS score and risk level was not significant, indicating that both risk groups had equivalent rates of improvement.

Discussion and conclusions

Evidence from previous research has suggested that the children referred to and served in systems of care are heterogeneous with regard to their referring agency and their psychosocial and demographic profiles (8,9,10,11,8,18,19). The results of this investigation substantiate those differences at intake and demonstrate that children who were served in a local system of care showed equivalent improvement in functioning during their first six months of receiving services, regardless of referral agency, risk level, diagnosis, age, or gender.

Limitations

Although these findings have important implications, some limitations must be considered. Because of the limited sample size, we chose not to investigate potential higher-order interactions between the psychosocial and demographic variables and change in functioning or to assess the relationship between the reason for referral and change in functioning. The range of a priori power estimates is reasonable; however, some caution may be warranted in interpreting the absence of significant differences for children from various referral sources and in different diagnostic groups. The risk factor scale depended on self-reports, which may have affected its validity and reliability. The only measure of functioning used was a clinician rating, and multiple perspectives of improvement were not available, such as those of teachers, parents, and children. In addition, other contextual factors were not controlled for, such as the treating mental health professional.

Implications

Despite these limitations, the results have important implications for the delivery of behavioral health services. Although it is widely agreed that systems of care are the preferred method of delivering services to children with serious emotional disturbances and their families (2,3), the relative extent to which these systems benefit children referred from mental health, juvenile justice, social service, and education agencies has not been examined. This study provides initial evidence that children with various demographic profiles who are referred from a wide variety of settings demonstrate comparable improvement in functioning over time. We must now focus attention on more detailed assessments of children who do not respond to these services to understand why they do not and to develop alternatives that might be more effective.

Recent outcome and effectiveness studies have required us to investigate who can, does, and will benefit from the comprehensive service delivery model (12,20,21). It is clear that not all children meeting criteria for a psychiatric disorder require a multiagency planning team and an extensive coordination of services to improve. The children included in this study were referred because their referral agencies saw them as being inadequately served within those agencies because of their emotional or behavioral problems. Given the large number of children who meet diagnostic criteria (22,23) and the costs associated with coordinating multiagency planning and service programs, additional research is required to determine the cost-effective thresholds for activating a multiagency service team. Although it is clear that characteristic differences at the start of services have implications for planning and delivery, this preliminary investigation suggests consistent improvement despite subgroup differences.

Future research

This study examined functional outcomes based on gender, age, risk factors, diagnosis, and referral source. On average, children and adolescents in each group showed reduced impairment after six months. However, functional outcome varied considerably, and 25 percent of the children showed a functional decline at six months. It is crucial that we begin to investigate and understand this variability. For example, although the presence of risk factors was assessed in this study, it will also be important to examine risk factors independently and assess their relationship to changes in functioning. Risk factors related to parental functioning such as parental history of mental illness or substance abuse may affect children differently than child-specific risk factors such as the child's previous psychiatric hospitalization or substance use.

Finally, the East Baltimore mental health partnership operates in an impoverished urban minority community that precluded the assessment of differential improvement as a function of race, ethnicity, or income level. As greater attention is paid to the overrepresentation of minorities in the public service sector, this issue must be investigated within and across system of care sites.

Acknowledgments

This research was supported by grant MH-19545-07 from the National Institute of Mental Health and by grant CMHS-5-44SM51392 from the Center for Mental Health Services.

Dr. Walrath is a research associate at the Johns Hopkins School of Hygiene and Public Health, where Dr. Leaf is a professor in the department of mental hygiene. Dr. Mandell is a postdoctoral fellow at the Center for Mental Health Policy and Services Research at the University of Pennsylvania in Philadelphia. Send correspondence to Dr. Walrath at the Johns Hopkins School of Hygiene and Public Health, 624 North Broadway, Eighth Floor, Baltimore, Maryland 21205 (e-mail, [email protected]).

Enlarge table

Enlarge table

Enlarge table