The Humble Leader: Association of Discrepancies in Leader and Follower Ratings of Implementation Leadership With Organizational Climate in Mental Health

There is increasing demand for the use of public health interventions supported by rigorous scientific research, but frequently the promise of such evidence-based practices (EBPs) fails to translate into their effective implementation, sustained use, or intended public health benefits. To bridge this gap between research and effective delivery in practice, researchers increasingly recognize the importance of studying the process of EBP implementation and sustainment (1–4). Although individual provider factors contribute to successful EBP implementation (5), organizational factors are likely to have an equal or greater influence on EBP implementation (6,7). Leadership is one factor that has been suggested to play an important role in the organizational context and implementation of health innovations (8–10).

Organizational climate that supports EBP implementation and sustainment can facilitate implementation, and leadership is an antecedent of organizational culture and climate (11–17). For example, more positive leadership is associated with a climate of involvement, in which followers feel involved in problem solving and organizational decision making (18). Leaders who emphasize the importance of learning and who establish trust with their followers foster development of a positive feedback climate, which encourages receipt of formal and informal performance feedback (19). Leader “credibility” has also been identified as an important facet of feedback climate, because leaders should be knowledgeable about their followers’ assigned tasks, in order to accurately judge performance on those tasks (20).

Early research on leadership and implementation focused on general leadership constructs, such as transformational leadership (21,22). Leaders enact transformational leadership through behaviors that embody inspirational motivation, individualized consideration of followers, ability to engender buy-in and intellectual stimulation, and idealized influence or serving as a role model (23). However, research on developing specific types of climates, such as safety climate (24,25) and service climate (26), has increasingly considered leadership focused on the achievement of a specific strategic outcome—for example, reducing accidents and improving customer service, respectively. Such a strategic leadership approach can also be applied to EBP implementation in the form of implementation leadership (27).

Implementing EBPs can be incredibly challenging and requires specific leader attributes, such as being knowledgeable about EBPs, engaging in proactive problem solving, perseverance in the face of implementation challenges, and supporting service providers in the implementation process. The Implementation Leadership Scale (ILS) was developed as a pragmatic, brief, and efficient (3,28,29) measure to assess these leadership behaviors that are thought to promote a strategic climate for implementing and sustaining EBPs (27). The construct of implementation leadership is complementary to general leadership and is the focus of this study, which involved “first-level leaders” and their followers. Although the “follower” label may have a negative or narrow connotation in some instances, the relatively nascent work on “followership” has begun to shift from this negative connotation to one that views followers as proactively involved in the leadership process. Therefore, we argue that without the concept of followers or followership, it is difficult to fully understand the leadership process, and thus the use of the terms “leader” and “follower” are appropriate for this study (30,31). First-level leaders (that is, those who supervise others who provide direct services) may be particularly influential in supporting new practices because they are on the front line directly supervising clinicians and bridging organizational imperatives and clinical service provision as EBPs are integrated into daily work routines (32). However, leaders and followers do not always agree about the leader’s behavior.

Research comparing leader and follower leadership ratings has focused on agreement and outcomes related to agreement. For example, Atwater and Yammarino’s (33) model of leader-follower agreement posits that congruence in positive leadership ratings are more likely linked to positive outcomes, and conversely, leader-follower agreement in negative leadership ratings are linked to negative outcomes. For leaders who under- or overestimate their own leadership abilities and skills, findings are equivocal. For example, one set of studies found that leaders who rated themselves lower in relation to others’ ratings of them were considered to be more effective as leaders (34,35). Other studies have shown that leaders who overestimate their leadership abilities tend to use hard persuasion tactics, such as pressure, to influence followers (36). Followers of such leaders are likely to think unfavorably of such tactics and recognize the leaders’ erroneous evaluation of their own strengths. Moreover, leaders who overestimate their leadership behaviors tend to misdiagnose their strengths, adversely affecting their effectiveness as a leader (33). Although these studies have added to an understanding of the different types of disagreement, there has been limited research specifically focusing on perceptual distance, or discrepancy in ratings of leadership and its effect on outcomes, such as organizational climate. This is an important area of inquiry, because recent work has shown that mental health leader-follower discrepancies in transformational leadership ratings can negatively affect organizational culture (37).

This study, conducted in public mental health organizations, addressed the extent to which leader-follower discrepancies in leadership ratings are related to the organizational climate of the leaders’ units, particularly with regard to organizational climate for involvement and performance feedback. Climate for involvement is important because EBP implementation requires participation and buy-in across organizational levels, especially for clinicians and service providers. Indeed, congruence of leadership across multiple levels may also be important during implementation (10). Climate for performance feedback is also critically important for EBP implementation, in that feedback and coaching regarding intervention fidelity is a critical part of implementation of many EBPs. For example, in previous work in home-based services, a key implementation strategy was providing feedback through in-vivo coach observation and real-time feedback (38,39). Thus it is important to understand how implementation leadership affects the organizational climate of involvement and feedback.

The purpose of this study was to examine the association of discrepancy between leaders’ (that is, clinic supervisors) self-ratings and their followers’ (that is, clinical service providers) ratings on the ILS and the associations of discrepancy with involvement and performance feedback climate in the leaders’ teams. On the basis of past research showing that leaders who underestimate their leadership may be more effective (34,35), we hypothesized that discrepancies—when leaders rated themselves lower than their follower ratings of them—would be associated with a more positive climate for involvement and performance feedback.

Methods

Participants

Participants were 753 public mental health team leaders (leaders) and the service providers whom they supervised (followers) from 31 mental health service organizations in California. Of the 753 eligible participants, 593 (80 leaders and 513 providers) completed the measures that were used in these analyses (79% response rate).

Data Collection Procedures

This study was conducted from approximately January 2013 to December 2014. The research team first obtained permission from agency executive directors or their designees to recruit leaders and their followers for participation in the study. Eligible leaders were identified as those who directly supervise staff in mental health treatment teams or work groups. Data collection was completed by using an online survey or in person with a paper-and-pencil survey. For online surveys, each participant received a link to the Web survey and a unique password via e-mail. For in-person surveys, paper forms were provided and completed at team meetings. In previous research, we found no differences in ILS scores by method of survey administration (40). The survey took approximately 20–40 minutes to compete, and participants received incentives by e-mail following survey completion. The Institutional Review Board of San Diego State University approved this study. Participation was voluntary, and informed consent was obtained from all participants.

Measures

ILS.

The ILS includes 12 items scored on a scale of 0, not at all, to 4, to a very great extent (27). The ILS includes four subscales, proactive leadership (Cronbach’s α=.93), knowledgeable leadership (α=.95), supportive leadership (α=.90), and perseverant leadership (α=.93). The total ILS score (α=.95) was created by computing and averaging the mean of the four subscales. The complete ILS measure and scoring instructions can be found in the “additional files” associated with the original scale development study (27). Leaders completed self-ratings of implementation leadership, and followers completed ratings of their leader’s implementation leadership.

Organizational Climate Measure.

The Organizational Climate Measure (OCM) consists of 17 subscales, with items scored on a scale of 0, definitely false, to 3, definitely true, and capturing a number of organizational climate dimensions (41). In this study, we used the involvement (α=.87; six items) and performance feedback (α=.79; five items) subscales that measure potentially important aspects of organizational climate for implementation. Clinicians completed these OCM subscales.

Statistical Analyses

Follower ratings were aggregated to create a team-level rating of implementation leadership for each leader. Intraclass correlation coefficients and average within-group agreement statistics (42) supported the aggregation of team ratings to the unit level (average within-group agreement >.70). As in the study by Fleenor and colleagues (43) and as recommended by Shanock and colleagues (44), scores were standardized, and scores that differed by ≥.5 standard deviations were considered discrepant values.

To explore the relationship between discrepancies in leadership ratings and organizational climate (that is, involvement climate and performance feedback climate), we conducted polynomial regressions and response surface analyses (44–46). As in past research that used this technique, we focused on the slope and curvature along the y=x and y=−x axes of the response surface, because they correspond directly to the substantive research questions of interest. The y=x axis is the axis along which follower and leader ratings are congruent, whereas the y=−x axis is the axis along which follower and leader ratings are incongruent. The relationship between organizational climate and either congruence or incongruence of ILS ratings was then explored by examining the response surfaces of the alignment between leader and follower ratings of implementation leadership and associations with organizational climate.

Results

Table 1 provides demographic information about the leaders and providers. Means, standard deviations, and correlations among the study variables included in the discrepancy analyses are presented in Table 2. Before polynomial regression and response surface analyses for examining discrepancies were conducted, ILS data were analyzed to ensure that discrepancies existed in the data (44). Three groups were identified: 31% (N=33) of leaders rated themselves higher than their followers rated them; for 33% (N=35) of leaders, ratings were in agreement with their followers’ ratings of them; and 36% (N=38) of leaders rated themselves lower than their followers rated them. Thus over 65% (N=71) of the sample showed discrepancies.

Results for the polynomial regression for associations between discrepancy on the ILS and the OCM involvement subscale are provided in a detailed table in the online supplement. The response surface is depicted in Figure 1. The line of incongruence (the dashed line in Figure 1) had a significant slope (a₃=−.30, t=−3.15, df=77, p<.01) and curvature (a₄=.42, t=3.47, df=77, p<.01). The significant slope indicates that involvement climate scores were higher when leader ILS ratings were low and follower ILS ratings were high, and this is in contrast to where leader ILS ratings were high and follower ILS ratings were low. Thus involvement climate was affected by discrepancy differently, depending on whose ILS rating was more favorable (that is, direction of discrepancy matters). The significant positive curvature (that is, convex surface) shown in Figure 1 shows that involvement climate scores were higher as levels of discrepancy increased.

With regard to the line of congruence (the solid line), the slope was nonsignificant (a₁=.12, p=.22), indicating that involvement climate scores were not different when leaders and followers agreed that ILS levels were high compared with when they agreed that ILS levels were low. However, the curvature of the line of congruence was significant, indicating that the lowest levels of involvement occurred when there was agreement between leaders and followers regarding ILS ratings of the leader. As a follow-up analysis to clarify the nature of the findings, we compared the four corner points of the response surface, in line with recommendations of Lee and Antonakis (47). This analysis revealed that involvement was highest at the left corner of the response surface (labeled A in Figure 1), where leaders rated themselves low and followers rated the leader high on the ILS. As summarized in the table in the online supplement, point A was significantly higher than all other corners of the surface (points B, C, and D), and the other three points were not different from each other.

The table in the online supplement provides the detailed polynomial regression results for the associations between discrepancy on the ILS and the OCM performance feedback climate subscale, and the corresponding response surface is provided in Figure 2. The line of incongruence (the dashed line in Figure 2) had a significant slope (a₃=−.37, t=−5.15, df=77, p<.001) and curvature (a₄=.47, t=6.29, df=77, p<.001). The significant slope indicates that performance feedback scores were higher when leader ILS ratings were low and follower ILS ratings were high compared with when leader ILS ratings were high and follower ILS ratings were low. Thus performance feedback climate was affected by discrepancy differently, depending on whose ILS rating was more favorable (that is, direction of discrepancy matters). The significant, positive curvature (that is, convex surface) indicates that performance feedback climate scores were higher as levels of discrepancy increased.

With regard to the line of congruence (the solid line), the slope was also significant (a₁=.16, t=2.25, df=76, p<.05), meaning that performance feedback scores were different when leaders and followers agreed that ILS scores were high versus when they agreed that ILS scores were low. Likewise, the curvature of the line of congruence was significant, indicating that the lowest levels of feedback climate occurred when there was agreement regarding ILS scores. As in the follow-up analysis conducted for involvement, we compared the four corner points of the response surface (47). This analysis revealed that performance feedback climate was highest at the left corner of the response surface (labeled A in Figure 2), where followers ILS ratings were high and leaders rated themselves low. As summarized in Table 3, point A was significantly higher than all other corners of the surface (points B, C, and D), point B was significantly higher than point D, and none of the other comparisons were significant.

Discussion

We found three almost equally distributed discrepancy-agreement groups: leaders and followers who agreed, leaders who rated themselves higher than did their followers, and leaders who rated themselves lower than did their followers. We refer to the latter as “humble leaders.” Organizational climates for involvement and feedback were most positive for humble leaders. These findings are consistent with research examining general leadership in other settings (34,35) and support the effectiveness of humble leaders (48,49). Moreover, discrepancies were associated with two aspects of organizational climate likely to be important for EBP implementation and sustainment.

Humble leadership was associated with significantly higher involvement and performance feedback climates in contrast to leadership characterized by high self-ratings and low follower ratings. This finding suggests that this leader-follower dynamic, in which leaders rate themselves lower than do their followers, creates a more positive climate that supports the leader’s capacity to implement EBPs. For example, leader humility has been found to be associated with increased humble behaviors of followers and the development of a shared team process that supports team goal achievement (50). However, the presence of humble leadership does not necessarily mean that EBPs will be implemented effectively. It is likely that effective leadership is a necessary but not sufficient condition for effective implementation and that leadership is one component of organizational capacity for implementation (51). Further research is needed to better understand the nuances of how leader-follower discrepancies develop and influence follower experiences of their workplace and to examine additional factors that may have an impact on effective implementation for both leaders and followers. Qualitative or mixed-methods might be used to better understand leader and follower perceptions of leadership and their relationships to implementation climate (52) and to advance leadership and climate improvement strategies.

There are promising interventions for improving leadership and organizational context for implementation. The Leadership and Organizational Change for Implementation intervention (53) combines principles of transformational leadership with implementation leadership to train first-level leaders to develop a more positive EBP implementation climate in their teams while working with organizations to ensure the availability of organizational processes and supports (for example, fidelity feedback, educational materials, and coaching) for effective implementation. Another example is the ARC (availability, responsiveness, continuity) implementation strategy, which works across organizational levels to improve molar organizational culture and climate (54). In another approach, Zohar and Polacheck (55) demonstrated that providing feedback to leaders about their followers’ perceptions of the leader’s team’s safety climate affected leaders’ verbalizations and behaviors, organizational safety climate, and safety outcomes. Thus there may be multiple strategies (some extremely low cost and low burden) that can be employed to influence leader cognition and behavior and ultimately improve organizational context and strategic outcomes.

There is a need for brief and pragmatic measures to guide leader development, with the goal of changing strategic climate and improving implementation (56). Leader self-ratings can be compared with provider ratings of the leader to provide insight to leaders about the degree to which their own perspective is aligned with that of their followers. Thus the ILS can be used by health care and allied health care organizations so that leadership for EBP implementation can be assessed at any stage of the implementation process as outlined in the exploration, preparation, implementation, or sustainment implementation framework (1). In the early implementation phases (for example, exploration and preparation), leaders might be provided training in effective leadership to support EBP implementation. Such an implementation strategy could contribute to facilitating the implementation process.

Some limitations of this study should be noted. First, this study focused on organizational climate supportive of implementation context as the distal outcome. Future studies of implementation leadership should examine additional outcomes, such as implementation effectiveness, innovation effectiveness, and patient outcomes (57,58). Second, this study was conducted in mental health organizations. Generalizability of these findings should be examined through replication in other health and allied health service sectors. Third, in this study there were apparent differences in race-ethnicity distribution for the samples of leaders and followers. There have been calls for leadership research to examine the degree to which such differences affect perceptions, relative power, and causality (59). Although it was beyond the purview of this study, we recommend more detailed examination of these issues. Finally, the data were cross-sectional; future research should examine these relationships prospectively, in addition to examining whether leader interventions may affect leader-follower discrepancies.

Conclusions

Effective EBP implementation and sustainment is critical to improve the impact of effective interventions. Sadly, many implementation efforts fail or do not deliver interventions with the needed rigor or fidelity. It is critical to understand how health care organization leaders and providers interact to create an organizational climate conducive to effective implementation and sustainment. This study demonstrated that discrepancy, or perceptual distance, in regard to organizational leadership has an impact on organizational climate relevant for EBP implementation. Leadership and organizational interventions to improve implementation and sustainment should be further developed and tested in order to advance implementation science and improve the public health impact of investments in clinical intervention development and implementation.