0
Get Alert
Please Wait... Processing your request... Please Wait.
You must sign in to sign-up for alerts.

Please confirm that your email address is correct, so you can successfully receive this alert.

1
Article   |    
Mental Health Care for Children With Disruptive Behavior Problems: A View Inside Therapists' Offices
Ann F. Garland, Ph.D.; Lauren Brookman-Frazee, Ph.D.; Michael S. Hurlburt, Ph.D.; Erin C. Accurso, M.S.; Rachel J. Zoffness, M.A., M.S.; Rachel Haine-Schlagel, Ph.D.; William Ganger, M.A.
Psychiatric Services 2010; doi: 10.1176/appi.ps.61.8.788
View Author and Article Information

Dr. Garland, Dr. Brookman-Frazee, Ms. Accurso, and Ms. Zoffness are affiliated with the Department of Psychiatry, University of California, San Diego, 3020 Children's Way, MC 5033, San Diego, CA 92123 (e-mail: agarland@ucsd.edu). Dr. Hurlburt is with the University of Southern California School of Social Work, Los Angeles. Dr. Haine-Schlagel is with the Department of Psychology and Mr. Ganger is with the Research Foundation, both at San Diego State University. All authors are also with the Child and Adolescent Services Research Center, Rady Children's Hospital, San Diego.

Abstract

Objectives: In the United States, more money is spent on treatment for children's mental health problems than for any other childhood medical condition, yet little is known about usual care treatment for children. Objectives of this study were to characterize usual care outpatient psychotherapy for children with disruptive behavior problems and to identify consistencies and inconsistencies between usual care and common elements of evidence-based practices in order to inform efforts to implement evidence-based practices in usual care. Methods: Participants included 96 psychotherapists and 191 children aged four to 13 who were presenting for treatment for disruptive behavior to one of six usual care clinics. An adapted version of the Therapy Process Observational Coding System for Child Psychotherapy—Strategies scale (TPOCS-S) was used to assess psychotherapy processes in 1,215 randomly selected (out of 3,241 collected) videotaped treatment sessions; treatment sessions were recorded for up to 16 months. Results: Most children received a large amount of treatment (mean number of sessions=22, plus children received other auxiliary services), and there was great variability in the amount and type of care received. Therapists employed a wide array of treatment strategies directed toward children and parents within and across sessions, but on average all strategies were delivered at a low intensity. Several strategies that were conceptually consistent with evidence-based practices were observed frequently (for example, affect education and using positive reinforcement); however, others were observed rarely (for example, assigning or reviewing homework and role-playing). Conclusions: Usual care treatment for these youths reflected great breadth but not depth. The results highlight specific discrepancies between evidence-based care and usual care, thus identifying potentially potent targets for improving the effectiveness of usual care. (Psychiatric Services 61:788–795, 2010)

Abstract Teaser
Figures in this Article

More money is spent on treatment for mental illness among children in the United States than for any other childhood medical condition (1). Unfortunately, outcome data on the effectiveness of community-based psychotherapeutic care are discouraging (2,3,4), and virtually nothing is known about what types of psychotherapeutic care are delivered in usual care settings, making it difficult to know how to target care improvement (5,6,7). National research and policy initiatives call for dissemination and implementation of evidence-based practices in usual care (8). These efforts could be more efficient and sustainable if informed by a better understanding of the current care context (9,10)—that is, "it is difficult and perhaps foolhardy to try to improve what you do not understand" (6).

Studies of children's psychotherapeutic usual care have focused primarily on examining outcomes as opposed to treatment processes. On average, findings regarding the effectiveness of usual care youth psychotherapy reflect minimal impact on children's symptom severity or functional status (2,3,4). However, none of these studies included detailed descriptive data about the nature of the usual care treatment; therefore, attributions regarding the links between usual care treatment processes and outcomes are largely speculative.

In one of the only studies of the nature of usual care for children receiving publicly funded mental health services, Zima and colleagues (11) used a chart review method to assess quality of care. Approximately half of the 813 cases studied met broad quality-of-care indicators for psychosocial treatment. However, the authors acknowledged that chart review was not a good data source for details about psychotherapy practice. Characterizing psychotherapeutic treatment processes is methodologically complex (12). Treatment can be characterized across a continuum ranging from broad molar classifications (for example, theoretical orientations) to molecular-level detailed verbal and nonverbal behaviors (13). Assessment of clinical strategies represents an intermediate level of analysis and includes assessment of therapeutic techniques (for example, modeling skills and interpreting the meaning of behavior) and content (for example, problem-solving skills and family member roles) (14). The most objective method to assess clinical strategies in psychotherapy is direct observation, but it is labor intensive (12).

This study utilized observational assessment to examine clinical strategies delivered in publicly funded usual care for children with disruptive behavior problems and their parents. This patient population was selected because the vast majority of youths in publicly funded care are referred for disruptive problems, including oppositional, defiant, aggressive, and delinquent behavior (15). Effective treatment is essential because children with disruptive behavior problems are at significantly elevated risk of multiple maladaptive outcomes in adolescence and adulthood, including criminal behavior and psychopathology (16,17). Further, families served in the public sector are at particularly high risk because of multiple risk factors and life stressors (11).

Many psychotherapy treatment models have demonstrated efficacy for children with disruptive behavior problems (18,19,20,21), and several common core elements across individual evidence-based treatment models have been identified (22,23). Common elements of evidence-based treatment for this patient population include clinical strategies directed toward children (for example, affect or anger management) and toward parents or caregivers (hereafter referred to as parents) (for example, principles of positive reinforcement and limit setting). It is assumed that delivery of complete evidence-based treatment models is rare in usual care (21); however, no studies have assessed the extent to which usual care includes treatment strategies generally consistent with common elements of evidence-based practice. Empirical data that identify specific convergence and divergence between elements of evidence-based practice and usual care are needed to provide a road map for targeted efforts to improve care.

The goals of this study were to provide a detailed description of usual care psychotherapy for a representative sample of children with disruptive behavior problems treated in publicly funded outpatient clinics and to examine the extent to which therapists employ strategies conceptually consistent with common elements of evidence-based practices in order to inform efforts to implement evidence-based practices in usual care.

+

Participants

Clinics. The six participating clinics were selected because they represented the largest contractors for publicly funded, clinic-based outpatient care for children in one of the largest counties in the United States. These clinics serve a racially, ethnically, and diagnostically diverse group of children and their families. None of the clinics specialized or provided therapist training in delivery or incorporation of evidence-based treatments before or during the study period.

Therapists. In late 2003, all therapists from participating clinics were assigned random numbers and recruited sequentially until a sample was gathered that reflected the county's therapist distribution by discipline, training, and licensure status. From 2004 to 2006, all new therapists working at least half-time were recruited. The sample included a large representation of trainees and early career therapists because of greater turnover rates for these groups. Of the 163 therapists recruited, 131 (80%) agreed to participate; only 96 had a patient in the study who met criteria, agreed to participate, and attended sessions for video data collection. Therapists who declined to participate did not differ significantly from those who participated on age, gender, or racial or ethnic distribution; licensed staff had a slightly lower participation rate (N=17 of 24, 72%) compared with unlicensed staff or trainees (N=114 of 139, 86%).

Children and parents. Children were eligible to participate if they presented with a disruptive behavior problem (aggression, defiance, delinquency, or oppositional behavior by parent report), if they were aged four to 13 years, if the primary language for them and their parent was English or Spanish, and if they were entering a new episode of psychotherapy (defined as no therapy for the previous three months) with a participating therapist. Clinic administrative staff asked parents who contacted the clinic seeking an appointment for permission to share contact information with the researchers for recruitment; 10% declined to be contacted. Of the 550 who agreed to be contacted and met inclusion criteria, 292 (53%) did not attend a clinic appointment, leaving 258 potential participants; 218 (84%) agreed to participate. Videotaped sessions were collected for 191 children (27 participants did not continue treatment after the baseline research interview). Health Insurance Portability and Accountability Act restrictions prohibited a comparison of participants and nonparticipants.

Informed written consent was provided by therapists and parents, and assent was provided by children aged eight to 13 years. Participants received a modest cash honorarium for research participation, and all protocols were approved by affiliated human subjects review committees.

+

Procedures

Descriptive data on children, parents, and therapists were collected during baseline interviews. Number of treatment sessions attended and clinician-assigned child psychiatric diagnoses were collected from billing and administrative records. Data on other treatment services received were obtained from telephone interviews with parents conducted every four months for 16 months.

All psychotherapy sessions occurring between the baseline research interview and the 16-month follow-up were videotaped; 3,241 videotaped sessions were collected. A random sample of up to ten sessions per child was selected for coding (four tapes from baseline to four months, three tapes from five to eight months, two tapes from nine to 12 months, and one tape from 13 to 16 months; this pattern was chosen in order to provide the most data during the intervals in which most patients were attending), resulting in 1,215 total coded sessions.

+

Measures

Adaptation of TPOCS-S. The Therapeutic Process Observational Coding System for Child Psychotherapy—Strategies scale (TPOCS-S) (24,25) assesses for a wide array of intervention strategies that are theoretically and nontheoretically, or cross-theoretically, derived. The content was adapted from the Therapy Procedures Checklist (TPC) (26), and the format is based on the Therapist Behavior Rating Scale (TBRS) (27). The TPOCS-S was adapted for the current study (known locally as Practice and Research: Advancing Collaboration [PRAC]) in collaboration with the PRAC Therapist Advisory Group (TAG) (12,28) to maximize relevance to community practice. The PRAC TAG included one therapist from each participating clinic and is described in the study by Garland and colleagues (28). The final revised PRAC-TPOCS-S includes 27 clinical strategies, divided into 15 therapeutic techniques (for example, modeling and addressing the client-therapist relationship), and 12 therapeutic content areas (for example, affect management and principles of positive reinforcement) (29). Coders looked at the occurrence and intensity of each strategy directed to children and directed to adults separately. Occurrence indicates whether the strategy was observed at all. Intensity reflects both the time spent on the strategy and the thoroughness with which it was pursued (27). The combined occurrence and intensity scale was measured on a Likert scale ranging from 0 to 6 (0, clinical strategy was not observed; 1 or 2, low intensity; 3 or 4, medium intensity; and 5 or 6, high intensity). For example, a low-intensity rating on the therapeutic content strategy of problem-solving skills would reflect addressing one aspect of problem-solving skills, such as generating alternative solutions, but only for one particular experience the child or parent faced and in a somewhat fleeting or cursory manner. A high-intensity rating would reflect a thorough approach addressing multiple steps in problem solving and generalization to other problems.

Among the 27 clinical strategies of PRAC-TPOCS-S, a subset has been identified that comprises common elements of evidence-based treatment for children with disruptive behavior problems (23). This subset was identified through an iterative process of culling out elements of established empirically supported youth treatment models, parent-training models, and treatment models targeting youths and parents, identifying elements common across treatments, and validating these elements through expert consensus.

Coders and coder training. Seventeen research assistants served as coders (three were fluent in Spanish and coded sessions were conducted in Spanish). Coder training was conducted by three of the authors and included didactics, manual review, practice sessions, and training to an established criterion (defined as reaching 80% agreement on training tapes) (12).

Interrater reliability for the PRAC-TPOCS-S. Of the 1,215 total coded sessions, a random selection of 379 (31%) were coded by two coders to test interrater reliability. The intraclass correlation (ICC) assessing reliability on the combined occurrence and intensity scale across all codes at the session level was .78, representing strong reliability. ICCs were also calculated at the individual code level and ranged from .21 to .91, with a mean of .61, which is within an acceptable range (30). The eight codes with very low occurrence (observed <10% of sessions) had the lowest reliability.

+

Data analyses

Descriptive data on frequencies of observed clinical strategies across all 1,215 coded sessions were calculated. Therapists varied in the number of cases (mean±SD=2.5±1.8, range=1–7) and sessions (mean=12.7±11.6, range=1–44) coded; however, the observed frequencies did not vary significantly (all were within 3 points, and the majority were within 2 points) when just one case per therapist was randomly selected. Thus analyses presented rely on the full sample of 1,215 sessions.

+

Participants

Characteristics of the 96 therapists are presented in Table 1, and characteristics of the child and parent participants are presented in Table 2.

+

Treatment received

Patients attended an average of 22.4±15.6 sessions (range=0–70) during the 16-month study period, and 54 (28%) were still attending sessions at 16 months. Across the 1,215 coded sessions, children participated in 1,184 sessions (97%) and parents participated in at least part of 851 sessions (70%). By parent report, children received a variety of services in addition to outpatient psychotherapy. Specifically, during the 16-month study period, 134 (63%) of the children received some type of psychoactive medication, 30 (15%) presented to emergency rooms for mental health or behavioral reasons, 19 (9%) were placed in a psychiatric hospital or residential treatment center, and 177 (88%) received school-based mental health services.

+

Psychotherapeutic clinical strategies observed

Table 3 lists the 27 clinical strategies assessed in the PRAC-TPOCS-S. The average number of clinical strategies observed per session directed to children was 10.9±3.7 (range=1–20), and the average number directed to parents was 8.1±3.9 (range=1–22 of 27). Table 3 presents data on the observed occurrence and intensity for each of the assessed strategies in rank order of occurrence frequency. Any occurrence represents the percentage of sessions in which the strategy was observed relative to the total number of sessions in which each target (child or parent) participated (for example, affect education was observed in 964 (81%) of the 1,184 sessions in which a child participated). High-intensity occurrence represents the percentage of sessions in which the strategy was observed at an intensity level of 5 or 6 on the 6-point scale. Mean intensity is the average intensity of the strategy when it was observed. Mean intensity when observed was 2.3±.3 across strategies directed to children and 2.4±.5 for parent strategies. Of all observed sessions, 724 (60%) included at least one strategy delivered to either the child or parent at high intensity.

Clinical strategies that have previously been identified as common to evidence-based treatment for children with behavior problems and their parents (23) are noted in Table 3. Some of these evidence-based strategies were observed frequently (for example, affect education and use of positive reinforcement), whereas others were observed relatively infrequently (for example, assigning or reviewing homework and role-playing with parents). On average, all elements were observed at a low intensity.

This study provides the first detailed data on the type and variability of psychotherapeutic treatment strategies observed in mental health care delivered to children with disruptive behavior problems, the most common presenting problems in child mental health services. Results reflect heterogeneity in amount and type of treatment, including treatment duration (0–70 sessions in 16 months), additional service use, and within-session psychotherapeutic strategies observed.

Although treatment duration varied widely, the mean number of sessions attended (22.4) is somewhat consistent with the limited data available on duration of community-based care, with reported total treatment session averages ranging from 17 sessions (31) to 23 sessions (32). Most evidence-based treatment models for children with disruptive behavior problems require a minimum of 12 weekly visits (23). In addition to having outpatient sessions, a majority of patients reportedly received other types of mental health services. Almost two-thirds of these four- to 13-year-olds received some psychoactive medication, which is relatively consistent with 58% of a somewhat similar sample of youths with disruptive behavior problems who received medication as part of their community-based care in the Netherlands (33). A majority of children in our sample also received some type of school-based mental health services (88%), and a minority received more intensive services, including hospitalization for psychiatric reasons (9%), during the 16-month study period. Combined with the fact that 28% of children were still in treatment after 16 months, these data reflect a substantial volume of mental health services for families who engaged in usual care. The extent to which these services were coordinated across treatment modalities is not known.

Within and across treatment sessions, therapists were observed delivering a wide array of clinical strategies, which is consistent with the limited body of research indicating that usual care therapists prefer an eclectic approach to psychotherapy (10,14). Although only 25% of the therapists in this study endorsed "eclectic" as their primary theoretical orientation, the observational data indicate that eclecticism is the norm rather than the exception. The findings that almost half of sessions did not include a single therapeutic strategy delivered at high intensity and that the mean intensity rating across all observed strategies was relatively low reflect a cursory or incomplete application of the treatment strategy with limited follow-through. Explanations for low intensity are largely speculative, but the low intensity may reflect variability in therapist training, interference of patient crises and case management challenges, assumptions or perceptions of patients' responsiveness, or pragmatic constraints (for example, not enough time). Overall, the usual care psychotherapy we observed could be characterized as reflecting great breadth but not depth in therapeutic approaches.

Parents participated in at least part of most treatment sessions, and similar therapeutic techniques were observed frequently being delivered to both children and parents (for example, information gathering, psychoeducation, and establishing or reviewing goals). However, therapists were observed addressing different therapeutic content areas with parents than they did with children. Specifically, two of the most common content areas addressed with parents were "child's external care" (that is, case management or coordination of services received outside of the therapy session) and parent's own issues (for example, psychosocial issues and care), but these topics were not addressed as often with children. These findings are consistent with anecdotal reports from usual care therapists who indicate that the complex, multidetermined needs of families in public-sector care require significant case management, which can interfere with delivery of evidence-based psychotherapeutic approaches such as child skill building or parent management training. Although our data confirm that therapists are spending a great deal of time with parents on case management, the data cannot address whether this use of time is necessary or whether it has an impact on child outcomes.

The second aim of this study was to examine the extent to which usual care therapists employed strategies conceptually consistent with predetermined common elements of evidence-based treatment for this patient population. Results indicate that although several strategies common to evidence-based practices were observed in a majority of sessions (for example, affect education, problem-solving skills, use of positive reinforcement, and psychoeducation), other strategies common in evidence-based treatment were observed relatively infrequently. Even when observed, several evidence-based strategies were usually rated at a low-intensity level, thus not consistent with expectations in evidence-based treatment models. Similar observations were reported in other work comparing observational and therapist self-report methods for characterizing psychotherapy practice (34).

Many of the infrequently observed evidence-based treatment strategies, such as assigning or reviewing homework, role-play or behavioral rehearsal, and modeling, are characterized as more directive psychotherapeutic techniques (35), and these strategies are at the core of virtually all evidence-based treatment models for children with disruptive behavior problems. We have found that therapists in usual care clinics generally have positive attitudes about many of these psychotherapeutic techniques (36), yet they do not employ them often. Research on adult psychotherapy similarly finds that directive therapeutic approaches are not observed as frequently in usual care as they are in evidence-based treatment models (35). Because more directive treatment approaches have been associated with greater improvement in specific behavioral outcomes (37), more attention to this discrepancy is warranted.

Our findings raise natural questions about factors associated with therapist variability in practice. In related research, we have tested for therapist characteristics that may be associated with delivery of treatment relatively more or less consistent with elements of evidence-based practice; few significant effects were found (38). For example, therapist experience (months practiced), discipline, and staff versus trainee status were not significantly associated with observed intensity of delivery of evidence-based practice elements in our sample of therapists, which includes many therapists with few years of accumulated experience (38). Thus explanations for variability in practice likely require more complex investigation of training and supervision experiences. Accordingly, our results have implications for therapist training (as discussed below).

The study findings need to be interpreted in the context of some additional strengths and limitations of the study. Representativeness of the therapist and patient sample is critical for the generalizability of these findings. Our patient sample was comparable to several other clinical samples of children in publicly funded care. Specifically, the 2:1 male-to-female ratio is consistent across many studies (11,20), as is the overrepresentation of youths in some racial or ethnic minority groups relative to general population estimates (11,39) and the distribution of the most common diagnoses (conduct disorder or oppositional defiant disorder, attention-deficit hyperactivity disorder, and mood disorders) (39,40). Our therapist sample was also highly similar to a recent national sample of 1,200 providers in children's mental health care in terms of distribution by education level, gender, and race-ethnicity (41). Trainees with limited experience were somewhat overrepresented in our study, but other studies of community-based usual care also report high representation of trainees (42). Our sample also includes a large representation of therapists trained in the marriage and family therapy discipline, which represents a rapidly growing segment of the workforce across the United States but is overrepresented in California (43), where this study was conducted. Of course, the extent to which the participating therapists represent therapists in other types of service sectors (for example, private practice) or geographic regions is unknown.

The strengths of the study methods reflect a balance of relevance and rigor. We achieved adequate interrater reliability on our observational measure (PRAC-TPOCS-S), which also benefited from collaborative input from practicing therapists to ensure ecological validity and comprehensiveness (28). Use of such qualitative methodology to support the validity of practice measures has been strongly reinforced in previous studies (10). Despite these strengths, the resulting measure did not capture all possible therapeutic interventions. The data reflect only the observable behavior of the therapist, and they do not capture therapists' intentions, goals, or decision-making processes; patients' responses to different intervention strategies; or additional therapeutic contacts, such as communication outside the office. Finally, we do not know how observing practice may have affected practice itself, although we attempted to minimize this effect by establishing videotaping as routine in the clinics and by using small, unobtrusive cameras.

In the context of recent reports of the extraordinarily high cost of mental health treatment for children and discouraging data on effectiveness, this study offers timely detail regarding the types of care being delivered in one large public system. The study provides the first glimpse into usual care psychotherapy practice offices, and it thus provides essential contextual data for the development of tailored efforts to improve care, as well as providing baseline data for change efforts. Our results highlight some areas of relative convergence between evidence-based practice and usual care, as well as significant discrepancies between evidence-based treatment elements and usual care. Areas of convergence may represent a common ground upon which to build, and discrepancies represent potentially potent targets for improving care. For example, training efforts designed to increase the delivery of infrequently observed directive evidence-based treatment strategies for children and parents, such as role-playing, modeling, and assigning or reviewing homework, are needed. These efforts must also address needed improvements in intensity of treatment. Mental health clinician training is most effective when training interventions are tailored to address the existing service context (44). Further, it may be important to infuse evidence-based training into graduate programs preparing master's-level therapists because they represent a significant proportion of the providers of community-based mental health services.

Critical next steps in this practice-based research program include examination of child and family outcome trajectories to determine how specific practice patterns may be associated with different clinical outcomes. More detailed analyses can examine potential moderating effects of patient characteristics (demographic or clinical factors), as well as therapist characteristics. Analyses of the mediating role of perceived therapeutic alliance are also needed because previous research supports the role of alliance in outcomes. Richer information about usual care practice will provide essential contextual data for ongoing efforts to improve the translation of evidence-based interventions into practice, balancing the emphasis on evidence-based practice with attention to practice-based evidence (45).

This work was supported by grants R01-MH66070 (AFG), K23-MH077584 (LBF), and K01-MH064079 (MSH) from the National Institutes of Health. The authors thank Scott Roesch, Ph.D., for data analysis assistance, as well as Deb Dupuis, M.P.H., and Robin Taylor for project management. In addition, this project could not have been completed without significant contributions from the Therapist Advisory Group and all participating therapists and families.

The authors report no competing interests.

Soni A: The Five Most Costly Children's Conditions, 2006: Estimates for the US Civilian Noninstitutionalized Children, Ages 0–17: Statistical Brief #242. Rockville, Md, Agency for Healthcare Research and Quality, Apr 2009. Available at www.meps.ahrq.gov/mepsweb/data_files/publications/st242/stat242.pdf
 
Bickman L, Noser K, Summerfelt WT: Long-term effects of a system of care on children and adolescents. Journal of Behavioral Health Services and Research 26:185–202, 1999
 
Weiss B, Catron T, Harris V, et al: The effectiveness of traditional child psychotherapy. Journal of Consulting and Clinical Psychology 67:82–94, 1999
 
Weisz JR, Weiss B, Han SS, et al: Effects of psychotherapy with children and adolescents revisited: a meta-analysis of treatment outcome studies. Psychological Bulletin 117:450–468, 1995
 
Bickman, L: The most dangerous and difficult question in mental health services research. Mental Health Services Research 2:71–72, 2000
 
Hoagwood K, Kolko DJ: Introduction to the special section on practice contexts: a glimpse into the nether world of public mental health services for children and families. Administration and Policy in Mental Health and Mental Health Services Research 36:35–36, 2009
 
Weisz JR, Jensen-Doss A, Hawley KM: Evidence-based youth psychotherapies versus usual clinical care: a meta-analysis of direct comparisons. American Psychologist 61:671–689, 2006
 
Translating Behavioral Science Into Action: A Report by the National Advisory Mental Health Council's Behavioral Science Workgroup. Bethesda, Md, National Institute of Mental Health, 2000
 
Westfall JM, Mold J, Fagnan L: Practice-based research— "Blue Highways" on the NIH roadmap. JAMA 297:403–406, 2007
 
Baumann BL, Kolko DJ, Collins K, et al: Understanding practitioners' characteristics and perspectives prior to the dissemination of an evidence-based intervention. Child Abuse and Neglect 30:771–787, 2006
 
Zima BT, Hurlburt MS, Knapp P, et al: Quality of publicly-funded outpatient specialty mental health care for common childhood psychiatric disorders in California. Journal of the American Academy of Child and Adolescent Psychiatry 44:130–144, 2005
 
Garland AF, Hurlburt M, Brookman-Frazee L, et al: Methodological challenges of characterizing usual care psychotherapeutic practice. Administration and Policy in Mental Health and Mental Health Services Research 37:208–220, 2010
 
Heaton KJ, Hill CE, Edwards LA: Comparing molecular and molar methods of judging therapist techniques. Psychotherapy Research 5:141–153, 1995
 
Goldfried MR: Toward the delineation of therapeutic change principles. American Psychologist 35:991–999, 1980
 
Garland AF, Hough R, McCabe K, et al: Prevalence of psychiatric disorders for youths in public sectors of care. Journal of the American Academy of Child and Adolescent Psychiatry 40:409–418, 2001
 
Copeland WE, Miller-Johnson S, Keeler G, et al: Childhood psychiatric disorders and young adult crime: a prospective, population-based study. American Journal of Psychiatry 164:1668–1675, 2007
 
Earls FJ: Violence and today's youth. Future Child 4:4–23, 1994
 
Burns BJ, Hoagwood K, Mrazek PJ: Effective treatment for mental disorders in children and adolescents. Clinical Child and Family Psychology Review 2:199–254, 1999
 
Carr A: What Works With Children and Adolescents: A Critical Review of Psychological Interventions With Children, Adolescents and Their Families. Florence, Ky, Taylor and Francis, 2000
 
Eyberg SM, Nelson MM, Boggs SR: Evidence-based psychosocial treatments for children and adolescents with disruptive behavior. Journal of Clinical Child and Adolescent Psychology 37:215–237, 2008
 
Kazdin AE, Weisz JR (eds): Evidence-Based Psychotherapies for Children and Adolescents. New York, Guilford, 2003
 
Chorpita BF, Becker KD, Daleiden EL: Understanding the common elements of evidence-based practice: misconceptions and clinical examples. Journal of the American Academy of Child and Adolescent Psychiatry 46:647–652, 2007
 
Garland AF, Hawley KM, Brookman-Frazee L, et al: Identifying common elements of evidence-based psychosocial treatments for children's disruptive behavior problems. Journal of the American Academy of Child and Adolescent Psychiatry 47:505–514, 2008
 
McLeod BD: The Therapy Process Observational Coding System for Child Psychotherapy. Los Angeles, University of California, Department of Psychology, 2001
 
McLeod BD, Weisz JR: The Therapy Process Observational Coding System for Child Psychotherapy Strategies Scale. Journal of Clinical Child and Adolescent Psychology 39:436–443, 2010
 
Weersing VR, Weisz JR, Donenberg GR: Development of the Therapy Procedures Checklist: a therapist-report measure of technique use in child and adolescent treatment. Journal of Clinical Child and Adolescent Psychology 31:168–180, 2002
 
Hogue A, Liddle HA, Rowe C: Treatment adherence process research in family therapy: a rationale and some practical guidelines. Psychotherapy 33:332–345, 1996
 
Garland AF, Plemmons D, Koontz L: Research-practice partnerships in mental health: lessons from participants. Administration and Policy in Mental Health and Mental Health Services Research 33:517–528, 2006
 
Garland AF, Brookman-Frazee L, McLeod BD: Scoring Manual for the PRAC Study Therapy Process Observational Coding System for Child Psychotherapy: Strategies Scale. Department of Psychiatry, University of California, San Diego, 2008
 
Cichetti DV: Guidelines, criteria, and rule of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment 6:284–290, 1994
 
Brannan AM, Heflinger CA: Child behavioral health service use and caregiver strain: comparison of managed care and fee-for-service Medicaid systems. Mental Health Services Research 7:197–211, 2005
 
Hawley KM, Weisz JR: Youth versus parent working alliance in usual clinical care: distinctive associations with retention, satisfaction, and treatment outcome. Journal of Clinical Child and Adolescent Psychology 34:117–128, 2005
 
Van de Wiel NM, Matthys W, Cohen-Kettenis PT, et al: The effectiveness of an experimental treatment when compared to care as usual depends on the type of care as usual. Behavior Modification 31:298–312, 2007
 
Hurlburt MS, Garland AF, Nguyen K, et al: Child and family therapy process: concordance of therapist and observational perspectives. Administration and Policy in Mental Health and Mental Health Services Research, Nov 10, 2009 [Epub ahead of print]
 
Malik ML, Beutler LE, Alimohamed S, et al: Are all cognitive therapies alike? A comparison of cognitive and noncognitive therapy process and implications for the application of empirically supported treatments. Journal of Consulting and Clinical Psychology 71:150–158, 2003
 
Brookman-Frazee L, Garland AF, Taylor R, et al: Therapists' attitudes towards psychotherapeutic strategies in community-based psychotherapy with children with disruptive behavior problems. Administration and Policy in Mental Health and Mental Health Services Research 36:1–12, 2009
 
Schoenwald SK, Henggeler SW, Brondino MJ, et al: Multisystemic therapy: monitoring treatment fidelity. Family Process 39:83–102, 2000
 
Brookman-Frazee L, Haine RA, Baker-Ericzen M, et al: Factors associated with use of evidence-based practice strategies in usual care youth psychotherapy. Administration and Policy in Mental Health and Mental Health Services Research 37:254–269, 2010
 
Foster EM, Kelsch CC, Kamradt B, et al: Expenditures and sustainability in systems of care. Journal of Emotional and Behavioral Disorders 9:53–70, 2001
 
Rosenblatt A, Rosenblatt J: Demographic, clinical, and functional characteristics of youth enrolled in six California systems of care. Journal of Child and Family Studies 9:51–66, 2000
 
Glisson C, Landsverk J, Schoenwald S, et al: Assessing the organizational social context (OSC) of mental health services: implications for research and practice. Administration and Policy in Mental Health and Mental Health Services Research 35:98–113, 2008
 
Hawley KM, Weisz JR: Youth versus parent working alliance in usual clinical care: distinctive associations with retention, satisfaction, and treatment outcome. Journal of Clinical Child and Adolescent Psychology 34:117–128, 2005
 
Northey WF: Characteristics and clinical practices of marriage and family therapists: a national survey. Journal of Marital and Family Therapy 28:487–494, 2002
 
Casper ES: The theory of planned behavior applied to continuing education for mental health professionals. Psychiatric Services 58:1324–1329, 2007
 
Garland AF, Bickman L, Chorpita BF: Change what? Identifying quality improvement targets by investigating usual mental health care. Administration and Policy in Mental Health and Mental Health Services Research 37:15–26, 2010
 
Table 1  Characteristics of 96 participating therapists who treated children with disruptive behavior
Table 2  Characteristics of 191 children aged four to 13 who received treatment for disruptive behavior and their parents
Table 3  Occurrence and intensity of clinical strategies observed in 1,215 therapy sessions
Table 1  Characteristics of 96 participating therapists who treated children with disruptive behavior
Table 2  Characteristics of 191 children aged four to 13 who received treatment for disruptive behavior and their parents
Table 3  Occurrence and intensity of clinical strategies observed in 1,215 therapy sessions
+

References

Soni A: The Five Most Costly Children's Conditions, 2006: Estimates for the US Civilian Noninstitutionalized Children, Ages 0–17: Statistical Brief #242. Rockville, Md, Agency for Healthcare Research and Quality, Apr 2009. Available at www.meps.ahrq.gov/mepsweb/data_files/publications/st242/stat242.pdf
 
Bickman L, Noser K, Summerfelt WT: Long-term effects of a system of care on children and adolescents. Journal of Behavioral Health Services and Research 26:185–202, 1999
 
Weiss B, Catron T, Harris V, et al: The effectiveness of traditional child psychotherapy. Journal of Consulting and Clinical Psychology 67:82–94, 1999
 
Weisz JR, Weiss B, Han SS, et al: Effects of psychotherapy with children and adolescents revisited: a meta-analysis of treatment outcome studies. Psychological Bulletin 117:450–468, 1995
 
Bickman, L: The most dangerous and difficult question in mental health services research. Mental Health Services Research 2:71–72, 2000
 
Hoagwood K, Kolko DJ: Introduction to the special section on practice contexts: a glimpse into the nether world of public mental health services for children and families. Administration and Policy in Mental Health and Mental Health Services Research 36:35–36, 2009
 
Weisz JR, Jensen-Doss A, Hawley KM: Evidence-based youth psychotherapies versus usual clinical care: a meta-analysis of direct comparisons. American Psychologist 61:671–689, 2006
 
Translating Behavioral Science Into Action: A Report by the National Advisory Mental Health Council's Behavioral Science Workgroup. Bethesda, Md, National Institute of Mental Health, 2000
 
Westfall JM, Mold J, Fagnan L: Practice-based research— "Blue Highways" on the NIH roadmap. JAMA 297:403–406, 2007
 
Baumann BL, Kolko DJ, Collins K, et al: Understanding practitioners' characteristics and perspectives prior to the dissemination of an evidence-based intervention. Child Abuse and Neglect 30:771–787, 2006
 
Zima BT, Hurlburt MS, Knapp P, et al: Quality of publicly-funded outpatient specialty mental health care for common childhood psychiatric disorders in California. Journal of the American Academy of Child and Adolescent Psychiatry 44:130–144, 2005
 
Garland AF, Hurlburt M, Brookman-Frazee L, et al: Methodological challenges of characterizing usual care psychotherapeutic practice. Administration and Policy in Mental Health and Mental Health Services Research 37:208–220, 2010
 
Heaton KJ, Hill CE, Edwards LA: Comparing molecular and molar methods of judging therapist techniques. Psychotherapy Research 5:141–153, 1995
 
Goldfried MR: Toward the delineation of therapeutic change principles. American Psychologist 35:991–999, 1980
 
Garland AF, Hough R, McCabe K, et al: Prevalence of psychiatric disorders for youths in public sectors of care. Journal of the American Academy of Child and Adolescent Psychiatry 40:409–418, 2001
 
Copeland WE, Miller-Johnson S, Keeler G, et al: Childhood psychiatric disorders and young adult crime: a prospective, population-based study. American Journal of Psychiatry 164:1668–1675, 2007
 
Earls FJ: Violence and today's youth. Future Child 4:4–23, 1994
 
Burns BJ, Hoagwood K, Mrazek PJ: Effective treatment for mental disorders in children and adolescents. Clinical Child and Family Psychology Review 2:199–254, 1999
 
Carr A: What Works With Children and Adolescents: A Critical Review of Psychological Interventions With Children, Adolescents and Their Families. Florence, Ky, Taylor and Francis, 2000
 
Eyberg SM, Nelson MM, Boggs SR: Evidence-based psychosocial treatments for children and adolescents with disruptive behavior. Journal of Clinical Child and Adolescent Psychology 37:215–237, 2008
 
Kazdin AE, Weisz JR (eds): Evidence-Based Psychotherapies for Children and Adolescents. New York, Guilford, 2003
 
Chorpita BF, Becker KD, Daleiden EL: Understanding the common elements of evidence-based practice: misconceptions and clinical examples. Journal of the American Academy of Child and Adolescent Psychiatry 46:647–652, 2007
 
Garland AF, Hawley KM, Brookman-Frazee L, et al: Identifying common elements of evidence-based psychosocial treatments for children's disruptive behavior problems. Journal of the American Academy of Child and Adolescent Psychiatry 47:505–514, 2008
 
McLeod BD: The Therapy Process Observational Coding System for Child Psychotherapy. Los Angeles, University of California, Department of Psychology, 2001
 
McLeod BD, Weisz JR: The Therapy Process Observational Coding System for Child Psychotherapy Strategies Scale. Journal of Clinical Child and Adolescent Psychology 39:436–443, 2010
 
Weersing VR, Weisz JR, Donenberg GR: Development of the Therapy Procedures Checklist: a therapist-report measure of technique use in child and adolescent treatment. Journal of Clinical Child and Adolescent Psychology 31:168–180, 2002
 
Hogue A, Liddle HA, Rowe C: Treatment adherence process research in family therapy: a rationale and some practical guidelines. Psychotherapy 33:332–345, 1996
 
Garland AF, Plemmons D, Koontz L: Research-practice partnerships in mental health: lessons from participants. Administration and Policy in Mental Health and Mental Health Services Research 33:517–528, 2006
 
Garland AF, Brookman-Frazee L, McLeod BD: Scoring Manual for the PRAC Study Therapy Process Observational Coding System for Child Psychotherapy: Strategies Scale. Department of Psychiatry, University of California, San Diego, 2008
 
Cichetti DV: Guidelines, criteria, and rule of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment 6:284–290, 1994
 
Brannan AM, Heflinger CA: Child behavioral health service use and caregiver strain: comparison of managed care and fee-for-service Medicaid systems. Mental Health Services Research 7:197–211, 2005
 
Hawley KM, Weisz JR: Youth versus parent working alliance in usual clinical care: distinctive associations with retention, satisfaction, and treatment outcome. Journal of Clinical Child and Adolescent Psychology 34:117–128, 2005
 
Van de Wiel NM, Matthys W, Cohen-Kettenis PT, et al: The effectiveness of an experimental treatment when compared to care as usual depends on the type of care as usual. Behavior Modification 31:298–312, 2007
 
Hurlburt MS, Garland AF, Nguyen K, et al: Child and family therapy process: concordance of therapist and observational perspectives. Administration and Policy in Mental Health and Mental Health Services Research, Nov 10, 2009 [Epub ahead of print]
 
Malik ML, Beutler LE, Alimohamed S, et al: Are all cognitive therapies alike? A comparison of cognitive and noncognitive therapy process and implications for the application of empirically supported treatments. Journal of Consulting and Clinical Psychology 71:150–158, 2003
 
Brookman-Frazee L, Garland AF, Taylor R, et al: Therapists' attitudes towards psychotherapeutic strategies in community-based psychotherapy with children with disruptive behavior problems. Administration and Policy in Mental Health and Mental Health Services Research 36:1–12, 2009
 
Schoenwald SK, Henggeler SW, Brondino MJ, et al: Multisystemic therapy: monitoring treatment fidelity. Family Process 39:83–102, 2000
 
Brookman-Frazee L, Haine RA, Baker-Ericzen M, et al: Factors associated with use of evidence-based practice strategies in usual care youth psychotherapy. Administration and Policy in Mental Health and Mental Health Services Research 37:254–269, 2010
 
Foster EM, Kelsch CC, Kamradt B, et al: Expenditures and sustainability in systems of care. Journal of Emotional and Behavioral Disorders 9:53–70, 2001
 
Rosenblatt A, Rosenblatt J: Demographic, clinical, and functional characteristics of youth enrolled in six California systems of care. Journal of Child and Family Studies 9:51–66, 2000
 
Glisson C, Landsverk J, Schoenwald S, et al: Assessing the organizational social context (OSC) of mental health services: implications for research and practice. Administration and Policy in Mental Health and Mental Health Services Research 35:98–113, 2008
 
Hawley KM, Weisz JR: Youth versus parent working alliance in usual clinical care: distinctive associations with retention, satisfaction, and treatment outcome. Journal of Clinical Child and Adolescent Psychology 34:117–128, 2005
 
Northey WF: Characteristics and clinical practices of marriage and family therapists: a national survey. Journal of Marital and Family Therapy 28:487–494, 2002
 
Casper ES: The theory of planned behavior applied to continuing education for mental health professionals. Psychiatric Services 58:1324–1329, 2007
 
Garland AF, Bickman L, Chorpita BF: Change what? Identifying quality improvement targets by investigating usual mental health care. Administration and Policy in Mental Health and Mental Health Services Research 37:15–26, 2010
 
+
+

CME Activity

There is currently no quiz available for this resource. Please click here to go to the CME page to find another.
Submit a Comments
Please read the other comments before you post yours. Contributors must reveal any conflict of interest.
Comments are moderated and will appear on the site at the discertion of APA editorial staff.

* = Required Field
(if multiple authors, separate names by comma)
Example: John Doe



Web of Science® Times Cited: 34

Related Content
Articles
Books
Dulcan's Textbook of Child and Adolescent Psychiatry > Chapter 63.  >
Dulcan's Textbook of Child and Adolescent Psychiatry > Chapter 63.  >
Dulcan's Textbook of Child and Adolescent Psychiatry > Chapter 63.  >
Dulcan's Textbook of Child and Adolescent Psychiatry > Chapter 16.  >
Dulcan's Textbook of Child and Adolescent Psychiatry > Chapter 63.  >
Topic Collections
Psychiatric News
APA Guidelines
PubMed Articles