A Best-Worst Scaling Experiment to Prioritize Caregiver Concerns About ADHD Medication for Children
Abstract
Objective:
The objective of this feasibility study was to develop and pilot an instrument to elicit caregivers’ priorities when initiating attention-deficit hyperactivity disorder (ADHD) medication for their child.
Methods:
A best-worst scaling experiment was used to rank competing priorities when initiating ADHD medicine. Forty-six participants were recruited for a two-phase study involving survey development (phase 1, N=21) and the survey pilot (phase 2, N=25). Best-worst scores and 95% confidence intervals indicating the relative importance of 16 concerns were determined, and t tests were used to determine the scores’ significance.
Results:
The significance of best-worst scores for most concerns indicated that the choices were purposeful. Concerns about helping the child become a successful adult, having a doctor who addresses caregivers’ concerns, and improving school behavior were ranked highest.
Conclusions:
The best-worst scaling method can elicit priorities for children’s mental health treatment. Future work using this method will guide family-centered care.
Attention-deficit hyperactivity disorder (ADHD) now affects 11% of U.S. children ages 17 or younger (1,2), and 3.5 million are prescribed a stimulant medication (2). Children often need medication—yet among caregivers, the acceptability of medication is low, and there is much uncertainty about using medication for their child (3–7). Even when medication is initiated, many caregivers discontinue use within two years (3,4,8).
Several studies have focused on caregivers’ perceptions of treatment for ADHD, mostly among low-income families from racial-ethnic minority groups. Caregivers initially do not use medication, reluctantly turn to medication only after exhausting all other options, and do not always view ADHD medication as appropriate for children (3–5). However, prior research has not elicited how caregivers’ priorities may influence decisions to initiate medication for their child (7,9). Therefore, this feasibility study aimed to develop and pilot a best-worst scaling instrument to assess caregivers’ priorities when initiating ADHD medicine for their child. The University of Maryland Institutional Review Board approved the study and granted a waiver of informed consent.
Methods
Mixed methods were used to develop and test a best-worst scaling instrument to elicit caregivers’ priority concerns when deciding whether to use ADHD medication for their child. Best-worst scaling was preferred to a conjoint discrete-choice experiment, often used in health care research (10,11), for several reasons. Grounded in random utility theory, best-worst scaling evokes tradeoffs by asking individuals to select one best and one worst attribute among competing alternatives within a profile. By comparison, conjoint experiments force selections among two or more different profiles. With best-worst scaling, individuals select attributes that are of greatest value to them relative to other shown attributes; as a result, information about what matters most to individuals is gained (12–14). This provides more enriched information on heterogeneity of specific priority concerns than can be obtained from selecting one profile containing multiple priorities (12–14). In addition, best-worst scaling makes it possible to estimate and compare the average utility of a profile’s attributes, whereas in a conjoint discrete choice experiment, the reference group is the whole scenario (14).
Two separate convenience samples were recruited from two support organizations in metropolitan Baltimore for caregivers of children with mental health needs. First, a sample of 21 caregivers participated in focus groups as part of the development of the best-worst scaling instrument. A second sample of 25 caregivers of children ages four to 14 with an ADHD diagnosis participated in a pilot study of the best-worst scaling instrument. The demographic characteristics of the two samples were very similar. A majority (>75%) was African American, and most (>85%) were the children’s biological mothers.
Attribute statements for the best-worst scaling instrument were identified by using data from a previous qualitative study of caregivers’ experiences. The study examined the experiences of caregivers as they came to terms with the ADHD diagnosis and medication treatment (3,4). This prior work generated a model of caregivers’ priorities in initiating medication that was grounded in their views of treatment in term of appropriateness (for example, whether the child was too young), anticipated effects (whether the medication would harm the child), and symbolic representation (whether using medicine meant being a bad parent) (3). This model was cross-referenced with the published literature (5,6) to develop a list of attribute statements.
In October 2012, a family support group leader from one of the family organizations recruited caregivers for the first sample. Caregivers were asked to participate in focus groups to assess attribute statement relevance. Fifteen caregivers participating in the first of two focus groups were presented with 26 attribute statements reflecting the potential priorities of caregivers when considering whether to initiate ADHD medication for their child. They were asked to classify the statements into four categories (short-term concern, long-term impact, societal views, and supportive network) or, if needed, to suggest a new category. On the basis of this feedback, attribute statements were revised and presented to a second focus group of six members of the same support organization for verification and relevance. No further amendments were suggested.
Sixteen attribute statements for the best-worst scaling instrument were retained. The statements were divided evenly by category, with each category containing two positively and two negatively phrased statements. Two child psychiatrists reviewed the clinical and practical relevance of the attribute statements.
A balanced, incomplete block design was used to construct the choice task profiles so that each attribute statement was seen the same number of times and any two attribute statements appeared together the same number of times. This design ensured equal probability of selection for each attribute statement. The survey had 16 choice task profiles, each displaying six of the 16 attribute statements. [An example of a best-worse choice task profile is available online as a data supplement to this article.]
In each choice task profile, participants were asked to think back to when they first learned of their child’s ADHD diagnosis and some of the situations that influenced their decision to initiate ADHD medication. They were instructed to select one attribute statement from among the six choices that reflected the most important concern (best choice) and then select one attribute statement that reflected the least important concern (worst choice) that influenced their decision to initiate ADHD medication for their child.
The family support group leader from a different organization helped to recruit caregivers for the pilot from five support groups for families in the Baltimore metropolitan area. During the pilot, conducted from November 2012 to January 2013, 25 caregivers used paper and pencil to complete the best-worst scaling instrument. All of the participants had children between the ages of 4 and 14 who had been diagnosed as having ADHD, and all used medication for their child. Most of the children also were currently using psychotherapy or had an individualized education plan.
The principal investigator and a graduate research assistant attended the support group meetings, explained the purpose of the survey, and provided instructions for completing the choice tasks. The pilot survey was completed, on average, in 15 minutes. At the conclusion of the meeting, participants were asked to provide feedback regarding the clarity and relevance of the choice task profiles. No further modifications were recommended.
Survey responses for each choice task profile were coded into two binary variables. The statements chosen as best and worst each received a score of 1, and the statements that were not chosen as best or worst received a score of 0. Best-worst scores for each attribute statement were calculated as the sum of the best selections minus the sum of the worst selections across all respondents divided by 150 (the number of times each attribute statement was displayed [N=6] multiplied by 25 participants) (15). A t test assessed if scores differed significantly from 0 (α=.05), which would imply that selections were not made at random but reflected stated priorities.
Results
A positive best-worst score indicated that the attribute statement was selected as most important more frequently than it was selected as least important, and negative best-worst scores indicated the opposite. The number of times each attribute statement was chosen as best and worse is shown in Table 1, along with each attribute’s mean best-worst score and 95% confidence intervals. All attribute statements except “ADHD medicine is not needed to control my child’s home behavior” were significant (p<.05).
Category and attribute statements | Best-worse score (M)a | 95% CI | Bestb | Worstc | p |
---|---|---|---|---|---|
Short-term concerns | |||||
ADHD medicine is needed to control my child’s school behavior | .39 | .35 to .43 | 58 | 0 | <.001 |
ADHD medicine side effects outweigh its benefits | .23 | .19 to .26 | 37 | 3 | .038 |
ADHD medicine will help my child get better grades | .07 | .03 to .10 | 21 | 11 | <.001 |
ADHD medicine is not needed to control my child’s home behavior | –.05 | –.02 to –.09 | 9 | 17 | .057 |
Long-term impact | |||||
ADHD medicine will help my child be a successful adult | .41 | .36 to .45 | 63 | 2 | <.001 |
ADHD medicine has risks that will affect my child's future health | .28 | .24 to .32 | 43 | 1 | <.001 |
ADHD medicine will help my child finish high school | .15 | .12 to .19 | 28 | 5 | <.001 |
ADHD medicine will limit my child's career options | –.09 | –.06 to –.13 | 8 | 22 | .005 |
Supportive network | |||||
The doctor addresses my concerns about ADHD medicine | .29 | .25 to .33 | 46 | 2 | <.001 |
The school has pressured me to use ADHD medicine for my child | –.30 | –.29 to –.37 | 5 | 50 | <.001 |
My family does not see why my child needs ADHD medicine | –.33 | –.29 to –.37 | 2 | 52 | <.001 |
My friends agree with using ADHD medicine for my child | –.43 | –.38 to –.47 | 4 | 68 | <.001 |
Societal views | |||||
ADHD medicine will help my child get along with others | .25 | .21 to .30 | 45 | 7 | <.001 |
ADHD medicine will hurt my child's self-esteem | –.05 | –.02 to –.07 | 4 | 11 | .034 |
Giving my child ADHD medicine does not mean I am a bad parent | –.30 | –.25 to –.35 | 18 | 63 | <.001 |
Others will think badly of my child if he or she uses ADHD medicine | –.53 | –.48 to –.57 | 5 | 84 | <.001 |
Best-worst score for 16 attribute statements related to medication of children with ADHD, ranked by importance within each category of concerns
Ranking of mean best-worst scores from largest to smallest was used to determine relative attribute importance. Overall, using medication to help their child become a successful adult (.41) was ranked highest, and concern that others would think badly of the child if he or she used ADHD medicine (–.53) was ranked lowest. [A list of attribute statements ranked by highest to lowest best-worse score is available online as a data supplementto this article.] Table 1 lists the attribute statements in each category by order of importance. Control of school behavior (.39) was the highest-ranked short-term concern (p<.001). The role of medicine in helping the child be a successful adult was a key long-term concern (p<.001). The only positive score in the supportive-network category was having a doctor who addressed the caregivers’ concerns about ADHD medicine (p<.001). Scores for the other attributes in the category indicated that family, friends, and school personnel were less important influences compared with doctors. With the exception of the child’s peer relations, all other attributes in the societal views category were negative.
Discussion
This study demonstrates the feasibility of best-worst scaling for eliciting caregivers’ priorities in initiating medication for their child’s ADHD. Significant best-worst scores indicated that choices were not random selections. Caregivers completed the instrument with relative ease.
The caregiver-centered instrument holds great promise for advancing family-centered research and clinical practice. Children’s mental health services research has been limited by a lack of rigorous methods for eliciting caregiver priorities. Eliciting caregivers’ priorities early in the clinical encounter can guide family-centered treatment planning.
There were several limitations. The sample was limited in diversity, size, and geographic locale and may not generalize to all caregivers of children with ADHD. Although recruitment from different advocacy organizations can result in potentially different samples, convenience sampling was used to recruit caregivers from homogeneous sources. The perspectives reflected the priorities of one parent—the mother. Although we sought continuous caregiver feedback, this list of relevant attribute statements may not be exhaustive. However, attribute development was an iterative feedback process in which statements were confirmed separately by several individuals. Finally, stated priorities were not correlated with treatment adherence, but that was not the goal of this feasibility study.
The purpose of this feasibility study was to test a best-worst scaling instrument prior to use in a larger comprehensive survey. The instrument is currently being used in a study that is designed to capture clinical diagnoses and receipt of mental health care services in order to assess the association between priorities and treatment adherence.
Conclusions
Caregivers’ priorities are nuanced and have an impact on decisions about their child’s or adolescent’s mental health care. The methods described here may help to better define, recognize, and understand caregivers’ priorities so that clinicians may engage caregivers in shared decision making about treatment for their child. This could accelerate caregiver-centered outcomes research in children’s mental health.
1 : Prevalence and treatment of mental disorders among US children in the 2001–2004 NHANES. Pediatrics 125:75–81, 2010Crossref, Medline, Google Scholar
2 : Trends in the parent-report of health care provider–diagnosed and medicated attention-deficit/hyperactivity disorder: United States, 2003–2011. Journal of the American Academy of Child and Adolescent Psychiatry 53:34–46 e2, 2014Crossref, Medline, Google Scholar
3 : The meaning of ADHD medication and parents’ initiation and continuity of treatment for their child. Journal of Child and Adolescent Psychopharmacology 19:377–383, 2009Crossref, Medline, Google Scholar
4 : Coming to terms with ADHD: how urban African-American families come to seek care for their children. Psychiatric Services 58:636–641, 2007Link, Google Scholar
5 : Investigating ADHD treatment trajectories: listening to families’ stories about medication use. Journal of Developmental and Behavioral Pediatrics 28:179–188, 2007Crossref, Medline, Google Scholar
6 : Barriers to detection, help-seeking, and service use for children with ADHD symptoms. Journal of Behavioral Health Services and Research 30:176–189, 2003Crossref, Medline, Google Scholar
7 : Attitudes about stimulant medication for attention-deficit/hyperactivity disorder among African American families in an inner city community. Journal of Behavioral Health Services and Research 33:423–430, 2006Crossref, Medline, Google Scholar
8 : Stimulant treatment over five years: adherence, effectiveness, and adverse effects. Journal of the American Academy of Child and Adolescent Psychiatry 43:559–567, 2004Crossref, Medline, Google Scholar
9 : Parental preferences and goals regarding ADHD treatment. Pediatrics 132:692–702, 2013Crossref, Medline, Google Scholar
10 : Providing information to parents of children with mental health problems: a discrete choice conjoint analysis of professional preferences. Journal of Abnormal Child Psychology 37:1089–1102, 2009Crossref, Medline, Google Scholar
11 : Things are looking up since we started listening to patients: recent trends in the application of conjoint analysis in health, 1982–2007. The Patient 1:273–282, 2008Crossref, Medline, Google Scholar
12 : Estimating preferences for a dermatology consultation using best-worst scaling: comparison of various methods of analysis. BMC Medical Research Methodology 8:76, 2008Crossref, Medline, Google Scholar
13 : Using best-worst scaling choice experiments to measure public perceptions and preferences for healthcare reform in Australia. The Patient 3:275–283, 2010Crossref, Medline, Google Scholar
14 : Best-worst scaling: what it can do for health care research and how to do it. Journal of Health Economics 26:171–189, 2007Crossref, Medline, Google Scholar
15 : Using best-worst scaling in horizon scanning for hepatocellular carcinoma technologies. International Journal of Technology Assessment in Health Care 28:339–346, 2012Crossref, Medline, Google Scholar