Practice Change After Training Psychiatry Residents in Tobacco Use Disorder

Tobacco use among individuals with a mental illness or substance use disorder remains alarmingly high, accounting for half of all deaths in this population. Education for health care professionals is an important aspect of changing behavioral health systems to treat tobacco use disorder, yet educational opportunities for psychiatrists and residents remain limited (1–3).

Web-based training on treatment of tobacco use disorder can be provided during residency education. Advantages to this format include flexible scheduling, reduced need for skilled instructors, and the ability to deliver a uniform curriculum. Williams et al. previously described an online training curriculum on treatment of this disorder that was effective in increasing relevant knowledge among psychiatry residents (3). At baseline, knowledge of tobacco treatment practices was low and did not differ by residency year, suggesting no gain in knowledge during routine residency training. Many residents reported that they did not feel prepared to treat tobacco use disorder from prior education, and the vast majority (98%, N=164) felt the online curriculum enhanced their professional experience.

A new curriculum’s impact on residents’ behavior change exceeds simple knowledge acquisition; it demonstrates the incorporation of knowledge and skills into practices that can improve patients’ health. The purpose of this study was to examine changes in clinical practices among psychiatry residents 3 months after completing an online training course on tobacco use disorder. We hypothesized that residents would apply tobacco treatment practices at higher rates than baseline after completing the course.

Methods

Program directors from U.S. psychiatry residency programs provided e-mail addresses for current residents. No other identifying information was collected, and residents were not required to participate. Two hundred seven residents completed the pretest that measured baseline knowledge, and 199 completed surveys that assessed their attitudes toward tobacco treatment practices and estimated their current rates of applying those practices. Residents then viewed three 1-hour webinars on tobacco treatment. The online curriculum, which has been described in detail elsewhere (3), covered the prevalence and consequences of tobacco use disorder among individuals with behavioral health conditions, assessment and overview of counseling strategies, and pharmacological treatments.

Residents who completed the entire course were invited to complete follow-up assessments 3 months later. These included the same survey assessing attitudes toward and application of treatment practices for tobacco use disorder and a posttest to assess knowledge retention. The pretest and posttest included questions on tobacco, tobacco withdrawal symptoms, evidence-based counseling, and medication treatments. Participants were offered a $40 Amazon gift card for completing the 3-month follow-up. The study was approved by the Rutgers-Robert Wood Johnson Medical School Institutional Review Board.

The survey included 12 statements about the frequency with which the residents offered various tobacco use disorder treatment practices to tobacco-using patients. Possible responses were 0, never; 1, occasionally; 2, usually; and 3, always. We examined scores for each item on the survey at baseline and at 3 months. Because the data were not normally distributed and measured on an ordinal scale, nonparametric tests (Kruskal-Wallis and Wilcoxon tests) were used to examine pre-post differences for each item. Using t tests and chi-square tests, we compared baseline scores of the follow-up sample with baseline scores of those who did not complete the follow-up. For samples with fewer than five participants, we used Fisher’s exact test. We calculated regression analyses to determine predictors of both change and completion of follow-up. Statistical analyses were performed with SAS version 9.4.

Results

Participants.

A total of 152 participants completed the online tobacco treatment course, making them eligible to participate in the 3-month follow-up. Of these residents, 89 (59% of those eligible) completed the 3-month follow-up survey, and 86 completed the posttest. The posttest scores at 3 months, which have previously been reported (3), indicated knowledge retention. Baseline demographic characteristics were compared between the follow-up participants and those in the original sample who did not complete the follow-up. The two samples were not significantly different regarding gender, baseline pretest score, or posttest score taken after the course completion (all p>0.05). The distribution of the postgraduate training year was different in that a considerably lower proportion of fourth- and fifth-year residents participated in the 3-month follow-up survey (2% [N=2] of follow-up sample were fourth- and fifth-year residents versus 21% [N=11] of the baseline group). Participants in the follow-up group achieved greater knowledge acquisition, as measured by the change from pretest to posttest score at baseline, compared with the non-follow-up group (3.86 versus 3.05; p<0.05).

Tobacco treatment practices at baseline.

At baseline, residents showed considerable variability in their ratings of specific tobacco treatment practices. Residents reported that they usually (44%, N=40) or always (47%, N=43) ask patients about smoking status; this was the most highly reported tobacco treatment practice among participants. Most other items, such as advising patients to stop smoking and recommending pharmacotherapy, were reported less frequently (Table 1). Referring smokers to a telephone cessation service like Quitline or taking carbon monoxide readings were often rated as “never.” Tobacco treatment practices at baseline were not significantly different between the non-follow-up and the 3-month follow-up groups for any of the 12 items.

Tobacco treatment practices at 3-month follow-up.

We compared participants’ frequency of applying tobacco treatment practices between baseline and 3-month follow-up in the group that completed the follow-up assessments (N=89). Table 1 lists the items in the tobacco practices survey and the frequency with which participants reported engaging in that activity at baseline and at 3-month follow-up. These results are presented descriptively, although we analyzed comparisons with the Kruskal-Wallis test. Residents reported a significant increase in their use of nine of the 12 tobacco treatment practices. They reported being more likely to assess patient willingness to quit smoking (H=78.4, df=3, p<0.01), advise patients to stop smoking (H=64.8, df=3, p=0.03), refer patients who smoke to others for appropriate cessation treatment (H=54.3, df=3, p<0.01), recommend or prescribe nicotine replacement therapy (H=54.7, df=3, p=0.04), recommend or prescribe other tobacco treatment medications (H=80.1, df=3, p<0.01), and refer smokers to a telephone cessation service such as Quitline (H=138.5, df=3, p=0.01). They also reported being more likely to monitor patient progress in attempting to quit (H=26.4, df=3, p=0.02) and to help patients set a quit date (H=68.9, df=3, p<0.01). The increase in likeliness to arrange follow-up visits with patients to address smoking did not meet statistical significance. Three areas that did not increase between baseline and 3-month follow-up were asking about smoking status (H=134.5, df=3, p=0.45), providing brochures and/or self-help materials (H=90.6, df=3, p=0.23), and taking carbon monoxide readings with a handheld monitor (H=394.4, df=3, p=0.25).

Predictors of change.

As an exploratory analysis, we examined predictors of changes to residents’ tobacco treatment practices. We defined a “practice changer” as anyone who exhibited an increase in the frequency of applying one or more of the 12 tobacco practices (e.g., changing from “occasionally” to “usually” in a given behavior). Fifty-one individuals (57%) met this criterion. We also defined participants as being “biased” if they endorsed any statement suggesting they viewed smoking negatively or not as an addiction (e.g., agreed that “most smokers are not interested in quitting”). Fifty-six residents (63%) endorsed at least one “bias statement.” Finally, we defined residents as “role averse” if they endorsed items indicating that tobacco treatment was inconsistent with their role as a psychiatrist (e.g., disagreed that “part of my role as a psychiatrist is to help patients who are motivated to stop using tobacco”). Fifty-seven residents (64%) met this criterion.

A logistic regression model was fitted with stepwise selection. The independent variables included female gender, postgraduate training year (years 1–2 versus years 3–5), baseline pretest score, baseline posttest score, change in score, presence of bias, and role aversion. After stepwise selection, only being a first- or second-year resident was selected into the model as a significant predictor of change.

We also looked at predictors of completing the follow-up survey by using the same baseline variables. After stepwise selection, being a woman and having a higher baseline pretest score were selected into the model as significant predictors of completing the 3-month follow-up assessment.

Discussion

This study indicated that not only is a Web-based course on tobacco use disorder for psychiatry residents a viable option for addressing knowledge deficits, but, more importantly, that this course can influence tobacco treatment practices. Specifically, residents reported an increase in the frequency of applying nine of 12 practices, indicating an overall improvement in providing tobacco treatment to their patients. Importantly, the findings included increases in prescribing of tobacco treatment medications, counseling, and referral to other resources.

The endorsement of tobacco treatment practices varied considerably among items on the survey. It is not surprising that asking about smoking status was the most highly reported activity because this practice may be a requirement at a clinical site or a required field to be completed in the electronic medical record. Most other items, such as advising patients to stop smoking or recommending pharmacotherapy, were reported less frequently. Residents reported infrequently taking carbon monoxide readings, suggesting that most do not have access to a handheld carbon monoxide monitor in their setting. It is interesting that first- and second-year residents were more likely to change their tobacco treatment practice than were senior (third- through fifth-year) residents. This finding may imply that educational efforts are more effective before residents have solidified their practice patterns, even though junior residents have less control over their own practice. The lower follow-up rates among fourth-year residents, which could have been due to fellowship, graduation, or practice-site change, limit better understanding of these differences.

Limitations of the study included lack of randomization or a control group. In addition, the tobacco treatment practices survey is a self-report assessment, and we did not verify use of these practices by chart review or other documentation. We also had limited demographic information about the residents, so we could not assess other factors that may have contributed to changes in tobacco treatment practices. In addition, follow-up rates were somewhat low; however, because the baseline practices were not different between the follow-up and non-follow-up groups, the low rates were not likely due to differences in interest in treating tobacco use disorder between the groups. The study was also not designed to evaluate state-level trends in response rates or other results.

Future research should seek to replicate these data in a parallel groups design with random assignment to the current online training or a control intervention. These data, however, suggest that an online training opportunity, especially early in residency, may increase knowledge about tobacco use disorder and, more importantly, increase the frequency of behaviors likely to result in reduced tobacco use in the vulnerable populations these residents will treat.