Introduction

Ratings of physician-patient communication (PPC) provide guidance for many important decisions in health research, health care delivery and health care professional trainings. In the field of health research, these ratings serve to promote evidence-based medicine. In doing so, ratings of PPC are used to investigate the effects of PPC on relevant clinical outcomes (e.g., patient quality of life, satisfaction with care and health status) [1–3]. The ratings are applied to assess physicians’ communication skills, to reveal needs for improvement and to derive appropriate approaches for interventions [1, 2]. Finally, they are used to evaluate such interventions and to decide whether they are effective and worthy of further implementation [4, 5]. In regard to health care delivery, these ratings serve as quality criteria for health facilities [6, 7]. In terms of health care professional training programs, ratings of PPC are incorporated to evaluate the communication competence of students, to assess learning needs and to justify modifications of curricula [8]. Accordingly, a complete understanding of PPC ratings is vital to progress and to defensible decision making in research, education and health care delivery.

However, there is no gold standard, accepted measure across settings which is used to assess ratings of PPC. For example, the different measures of PPC vary in the dimensions of PPC they assess (e.g. relationship building, promoting information exchange, facilitating patient involvement or satisfaction with the medical encounter) [9, 10]. In addition, measures of PPC also differ in the perspectives from which the ratings are made (e.g., physicians, patients, neutral observers) [11]. Thus, depending on the measure used, the ratings reflect PCC under specific measurement conditions. However, usually the aim is not to evaluate the quality of PPC under specific measurement conditions but to draw generalizable conclusions across different situations and conditions. For example, a neutral observer may assess a physicians’ communication competence using the OPTION scale [12] to evaluate an intervention program aiming to improve PPC. Then, one may not be interested in physicians’ communication competence how it is perceived by this specific rater on the specific dimension the OPTION scale assesses. Instead, one may be interested in the physicians’ overall communication competence across all possible measurement conditions (e.g. across all communication dimensions and all possible raters). Thus, it is important to reveal to what degree the assessed ratings reflect the construct of interest itself rather than its specific operationalized variant. This can be done by investigating the amount of score variance of the assessed ratings attributable to the specific measurement conditions. The less score variance is attributable to a specific aspect of the measurement situation, the more generalizable are the assessed ratings over this specific aspect.

Due to the possible variations in measurement conditions when investigation PPC, there are multiple sources of variance, which may contribute to the total score variance of PPC ratings. First, score variance may be due to the observer perspective [13, 14]. Physicians and patients may have different consistent tendencies to rate PPC quality as either high or low, which is also known as leniency or severity bias [15]. This bias may be specific to particular communication dimensions or particular physicians, leading to interaction effects of rater perspective and communication dimensions or rater perspective and physicians [14]. For example, in the case of the dimension satisfaction with the medical encounter, there is evidence, that patients tend to be more satisfied than their physicians when individual encounters have been rated from both perspectives [16, 17]. Furthermore, in a meta-analysis of six studies investigating shared decision making (SDM) physician and patient ratings proved to be weakly associated, while physicians rated SDM to be higher [18] (r_polyc = .15; 95% CI [0.08–0.21]). Two additional studies, consulting neutral observers and patients also indicate that there may be score variance in PPC ratings due to the observed perspective: observers and patients did not agree in their ratings of the same consultation. Thus, the ratings may depend more on the perspective of the assessing person than on the quality of PPC itself [19, 20].

Second, there may be score variance of PPC ratings due to an interaction effect of the analyzed perspective and the assessed communication dimensions. Particularly, physicians’ or patients’ impressions created in one area may influence their judgements in another area (halo effect). This phenomenon will lead to correlations on the different communication dimensions among ratings from a particular perspective [13]. For example, patient satisfaction with the medical encounter may also lead to higher ratings of information exchange and SDM but may not increase physicians’ ratings. Third, the assessed communication dimensions as well as an interaction effect of communication dimensions and investigated physicians may explain the score variance of PPC ratings. For example, ratings on satisfaction exhibit ceiling effects [15, 16]. Hence, observing the communication dimension of satisfaction may indicate a higher quality of PPC than observing other dimensions of communication, such as information exchange or SDM.

Previous studies did not reveal which of these specific characteristics of the measurement situation contribute to the variance of PPC ratings. When investigating the accuracy of PPC ratings, these studies typically focus on one global measurement error [20–22]. In doing so, all variance components accountable to the investigated type of error are incorporated into one single error term. None of these studies examines variability due to physician characteristics, rater perspective, or investigated dimensions of PPC and potential interaction effects between these factors simultaneously.

Generalizability theory (g-theory) allows simultaneous examination of multiple sources of variance [23]. By applying a generalizability study (g-study), it can be revealed how much of the score variance of PPC ratings is due to physicians’ communication behavior, raters’ perspective, communication dimensions, and interactions. Thus, in the context of PPC, g-theory outperforms previous used methods to investigate generalizability of ratings as it allows to separate the variability of ratings scores in different variance components instead of incorporating them into one global, unresolved error term. This process allows decision makers to enhance their understanding of the information included in the ratings and to draw more valid conclusions concerning the generalizability of PPC ratings across different measurement conditions. Two previous studies applying g-theory in the context of PPC investigated students’ communication competence in the field of health care professional training programs [24, 25]. Both studies investigated PPC ratings assessed from observers. G-theory proved to be effective as it allowed to decompose variance components differentially. However, to our knowledge, there is no study applying g-theory to investigate PPC ratings assessed from self-report measures.

In the presented study we investigated variance components of PPC ratings gained from physicians’ and patients’ perspectives. To represent the multidimensional characteristic of PPC, variance components of PPC ratings were analyzed on three communication dimensions representing two major aspects of PPC and one global indicator of the quality of PPC using g-theory. The observed communication dimensions are SDM, effective and open communication and satisfaction with the overall communication.

Methods

Results

A total of 252 patient questionnaires and 32 physician questionnaires from 25 family practices could be included in the analysis. Thirty-nine patient questionnaires and 16 physician questionnaires were excluded because there was no questionnaire from the second perspective. Another 11 patient questionnaires were excluded due to missing data. Table 1 shows the sociodemographic data of the included physicians and patients.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Characteristics of patients and physicians included in the study.

https://doi.org/10.1371/journal.pone.0252968.t001

The range of patients nested within physicians was from 1 to 38 (M = 7.88; SD = 7.83). The ICCs of patients’ ratings were not significant for SDM and were only small for satisfaction and effective and open communication (ICC = .08). Thus, the same variability is noted in the communication scores from one patient to another patient of the same physician as from one patient of one physician to another patient of another physician.

Mean scores, standard errors and Cronbach’s α for the ratings from physicians and patients’ perspectives of the three communication dimensions are shown in Fig 1. T-test results indicate that physicians’ and patients’ rating levels differ significantly for all three dimensions, t(31) = 3.11, p = .004, d = 0.55 (SDM); t(31) = -2.65, p = .012, d = 0.47 (effective and open communication); t(31) = -4.10, p < .001, d = 0.72 (satisfaction). While physicians rate SDM to be higher, patients judge effective and open communication as well as satisfaction to be higher. All scales assessed revealed high internal consistency for both perspectives (α > .83).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Mean scores, standard errors and Cronbach’s α for physician and patient rating scales.

https://doi.org/10.1371/journal.pone.0252968.g001

Table 2 provides variance components and G-coefficients for the ratings of PPC. The variance component attributable to physician effects indicates to what extent individual physicians differ in overall PPC. This variance component accounts for 1% of the total score variance.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. G-study estimates of variance components and G-coefficients.

https://doi.org/10.1371/journal.pone.0252968.t002

The variance component for physician and perspective interaction (P x R) reflects variance attributable to rater perspectives and a particular physician. Depending on the particular physician, the congruence of rater perspectives varies systematically. This interaction accounts for approximately 16% of the total score variance. This interaction effect is shown in Fig 2. The difference in the overall PPC ratings between the two perspectives is for 12 physicians within the 95% CI with a mean difference of 0 indicating concordant ratings (diagonal). However, 13 physicians provided lower ratings for their own overall communication quality than their patients, whereas 7 physicians gave higher ratings.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Interaction effect of perspective facet and physician facet.

https://doi.org/10.1371/journal.pone.0252968.g002

The interaction effect of perspective and communication (R x C) becomes evident in Fig 1: leniency and severity biases of rater perspectives are communication dimension specific. The g-study further reveals that this interaction effect explains 33% of the total variance in scores. Variance due to communication dimension indicates to what extent ratings on a specific dimension of PPC differ from ratings of other dimensions of PPC and contributes 4% to score variance. The residual variance component (P x R x C) accounts for approximately 46% of the score variance. This component reflects variance attributable to the three-way interaction and is confounded with error variance. The low g-coefficients indicate that physician and patient ratings have limitations in reliably assessing the overall communication of individual physicians.

Discussion

PPC ratings were minimally able to distinguish among the communication competence of individual physicians. Instead, the ratings more substantially reflect differences among physicians and patients in their tendency to perceive SDM, effective and open communication and to be satisfied with the communication. Thus, patients evaluate communication more effectively and openly and are more satisfied with communication in the consultation than their counterparts. In contrast, physicians perceive that they do better at SDM than their patients perceive. Furthermore, PPC ratings reflect physicians’ and patients’ leniency and severity bias unique to specific physicians. Thus, some physicians are rated in their communication more positively by their patients than indicated by their self-ratings. However, other physicians are rated more negative by their patients than indicated by their self-ratings. Finally, a high amount of score variance cannot be explained despite investigating the different variance facets P, R and C as well as their interactions.

The findings of this study are consistent with previous studies applying g-theory to examine ratings from neutral observers [24, 25] or in other contexts [14, 41] and are emphasized by studies investigating PPC ratings using other methods [18, 42]. Two studies examined ratings of PCC assessed from neutral observers to evaluate students communication competence in the context of health professional training programs [24, 25]. It was shown that students’ communication competence could only be explained to a small extent (3% [24]; 11% [25]). This was also shown by a study examining the cultural competency of attending physicians using g-theory [41]. Physicians’ cultural competency could only be explained to a small extent (3%). Thus, the physician effect was also very low. Instead, the ratings also reflected the observer’s unique perceptions of specific physicians (47%; interaction effect of observer and physician) [41]. A meta-analysis investigating a broad variety of ratings in the context of psychological research showed similar results [14]. Approximately 37% of the score variance was attributable to raters’ unique perceptions of particular targets and to raters’ different tendencies to interpret the investigated rating scales. The latter is further emphasized by studies examining interrater agreement of physician and patient ratings [18, 42]. They indicate that physicians and patients tend to perceive single communication dimensions differently, whereas physicians rated the presence of SDM higher [43].

The different tendencies of the rater perspectives to rate the single communication dimensions of PPC may be due to several reasons. Different attributions of the meanings of the communication dimensions could provide different ratings from the perspectives [14]. Since patients are less familiar with the language of the items and instructions, their ratings may reflect broader aspects of communication than those from physicians [44]. Especially in the case of SDM ratings, physicians and patients may refer to distinctive content meanings given that patients’ ratings do not only reflect the decision-making process. Instead, they may also reflect other aspects inherent in the patient’s perspective, i.e., the alternative they focus on, their assessment of constraints on choices, and their evaluation of how good the best possible solution is [45]. However, physicians may focus on their strengths and overlook their weaknesses when evaluating their own competence to involve patients in decision making. In doing so, they enhance their self-esteem and produce a self-serving bias. Furthermore, the importance of SDM is forced in health professionals’ training and in evaluations of health care delivery quality. This phenomenon may lead physicians to reveal higher SDM ratings due to social desirability [46].

However, other sources of disagreement may be relevant regarding ratings of satisfaction with communication in the consultation. As subjective perception may be more important than the evaluation of physician performance, patients may be more satisfied with communication in consultations than physicians due to different expectations concerning medical encounters [47, 48]. Although physicians are educated about what aspects constitute good PPC, they may have expectations concerning medical encounters other than their patients, who may have formed their expectations due to former experiences. Thus, physician satisfaction ratings may more reflect on the degree to which medical standards are met. Patient ratings, however, may reflect to what extent their expectations were met based on former experiences. Another reason for the higher satisfaction ratings of patients could be a social desirability response bias. They may feel that positive comments are more likely to be accepted by survey administrators and therefore report greater satisfaction than they actually feel. Furthermore, being at the family practice when completing the questionnaires may have evoked the desire of patients to integrate themselves with medical staff, leading to more favorable ratings [47]. Finally, a halo effect may produce higher satisfaction ratings from patients, resulting in the contribution of unique impressions to the ratings [14].

The consequences of the high amount of variance in PPC ratings due to rater-perspective interaction effects should be considered separately for every single communication dimension as they underlie different conceptual frameworks of communication [11]. For example, satisfaction ratings are considered to be subjective judgments of an interaction between two individuals. Therefore, the individual perception and experience of the two parties is of interest. Thus, both variance due to rater perspective and variance due to the interaction of rater perspective and physician may reflect valid information. However, regarding ratings of effective and open communication behavior of the physician, communication is considered an observational behavior of one individual. It is not of interest how the behavior was interpreted by the perspectives but how the physicians acted. Hence, variance due to rater perspectives would be a source of error. In the case of SDM, both the observational behavior of the interaction and the perceived involvement could be of interest [49]. Thus, whether rater-perspective variance is a problem in SDM ratings depends on the research question.

To further facilitate the interpretation of PPC ratings, further research is needed. The development and evaluation of methods aiming to improve the reflection of study participants when responding to items of PPC may reduce rater biases, such as self-serving bias and halo effects. Additionally, investigating the decision-making process underlying when giving ratings of PPC would be valuable. Further examination of which characteristics lead to different ratings of the two perspectives for particular physicians will be performed. Furthermore, investigating further variance components that could not be explained in this study may provide a deeper understanding of the ratings. This information may also reveal additional indicators for interventions aiming to improve PPC. Additionally, conducting a g-study may be instructive taking into account not only self-assessments by physicians and patients, but also assessments by neutral observers. Finally, further research is necessary that goes beyond samples in the primary care setting with the indication of back pain.

Supporting information

Acknowledgments

We would like to thank all participants for their cooperation. In addition, we would like to thank the local study coordinator in Erlangen, Piet van der Keylen and Kathrin Schnitzius for recruiting participating physicians and coordinating emails and paper-based surveys.