Original Research

How well do patients and providers agree on the severity of dyspnea?

Abstract

BACKGROUND

Understanding the severity of patients' dyspnea is critical to avoid under‐ or overtreatment of patients with acute cardiopulmonary conditions.

OBJECTIVE

To evaluate the agreement between dyspnea assessment by patients and healthcare providers and to explore which factors contribute to discordance in assessment.

DESIGN, SETTINGS AND PARTICIPANTS

Prospective study of patients hospitalized for acute cardiopulmonary diseases at an urban teaching hospital.

INTERVENTION AND MEASUREMENTS

A numerical rating scale (0–10) was used to assess dyspnea severity as perceived by patients and assessed by providers. Agreement was defined as a score within ±1 between patient and healthcare provider; differences of ≥2 points were considered over‐ or underestimations. The relationship between patient self‐perceived dyspnea severity and provider rating was assessed using a weighted kappa coefficient.

RESULTS

Of the 138 patients enrolled, 33% had a diagnosis of heart failure, 30% chronic obstructive pulmonary disease, and 13% pneumonia; median age was 72 years, and 57% were women. In all, 96 patient‐physician and 138 patient‐nurses pairs were included in the study. The kappa coefficient for agreement was 0.11 (95% confidence interval [CI]: 0.01 to 0.21) between patients and physicians and 0.18 (95% CI: 0.12 to 0.24) between patients and nurses. Physicians underestimated patients' dyspnea 37.9% of the time and overestimated it 25.8% of the time, whereas nurses underestimated it 43.5% of the time and overestimated it 12.4% of the time. Admitting diagnosis was the only patient factor associated with discordance.

CONCLUSIONS

Agreement between patient perception of dyspnea and healthcare providers' assessment is low. Future studies should prospectively test whether routine assessment of dyspnea results in better patient outcomes. Journal of Hospital Medicine 2016;11:701–707. © 2016 Society of Hospital Medicine

© 2016 Society of Hospital Medicine

Breathlessness, or dyspnea, is defined as a subjective experience of breathing discomfort that is comprised of qualitatively distinct sensations that vary in intensity.[1] Dyspnea is a leading reason for patients presenting for emergency care,[2] and it is an important predictor for hospitalization and mortality in patients with cardiopulmonary disease.[3, 4, 5]

Several professional societies' guidelines recommend that patients should be asked to quantify the intensity of their breathlessness using a standardized scale, and that these ratings should be documented in medical records to guide dyspnea awareness and management.[1, 6, 7] During the evaluation and treatment of patients with acute cardiopulmonary conditions, the clinician estimates the severity of the illness and response to therapy based on multiple objective measures as well as the patient's perception of dyspnea. A patient‐centered care approach depends upon the physicians having a shared understanding of what the patient is experiencing. Without this appreciation, the healthcare provider cannot make appropriate treatment decisions to ensure alleviation of presenting symptoms. Understanding the severity of patients' dyspnea is critical to avoid under‐ or overtreatment of patients with acute cardiopulmonary conditions, but only a few studies have compared patient and provider perceptions of dyspnea intensity.[8, 9] Discordance between physician's impression of severity of dyspnea and patient's perception may result in suboptimal management and patient dissatisfaction with care. Furthermore, several studies have shown that, when physicians and patients agree with the assessment of well‐being, treatment adherence and outcomes improve.[10, 11]

Therefore, we evaluated the extent and directionality of agreement between patients' perception and healthcare providers' impression of dyspnea and explored which factors contribute to discordance. Additionally, we examined how healthcare providers document dyspnea severity.

METHODS

Study Setting and Population

The study was conducted between June 2012 and August 2012 at Baystate Medical Center (BMC), a 740‐bed tertiary care hospital in western Massachusetts. In 2012, the BMC hospitalist group had 48 attending physicians, of whom 47% were female, 48% had 0 to 3 years of attending experience, and 16% had 10 years of experience.

We enrolled consecutive admissions of English‐speaking adult patients, with a working diagnosis of heart failure (HF), chronic obstructive pulmonary disease (COPD), asthma, pneumonia, or a generic diagnosis of shortness of breath. Because we surveyed only hospitalists, we did not include patients admitted to an intensive care unit.

All participants gave informed consent to be part of the study. The research protocol was approved by the Baystate Health Institutional Review Board, Springfield, Massachusetts.

Dyspnea Assessment

Dyspnea intensity was assessed on an 11‐point (010) numerical rating scale (NRS).[12, 13] A trained research assistant interviewed patients on day 1 and 2 after admission, and on the day of discharge between 8 am and 12 pm on weekdays. The patient was asked: On a scale from 0 to 10, how bad is your shortness of breath at rest now, with 0 being no shortness of breath and 10 the worst shortness of breath you could ever imagine? The hospitalist or the senior resident and day‐shift nurse taking care of the patients were asked by the research assistant to rate the patient's dyspnea using the same scoring instrument shortly after they saw the patient. The physicians and nurses based their determination of dyspnea on their usual interview/examination of the patient. The patient, physician and the nurse were not aware of each other's rating. The research assistant scheduled the interviews to minimize the time intervals between the patient assessment and provider's rating. For this reason, the number of assessments per patient varied. Nurses were more readily available than physicians, which resulted in a larger number of patient‐nurse response pairs than patient‐physician pairs. All assessments were done in the morning, between 9 am and 12 pm, with a range of 3 hours between provider's assessment of the patient and the interview.

Dyspnea Agreement

Agreement was defined as a score within 1 between patient and healthcare provider; differences of 2 points were considered over‐ or underestimations. The decision to use this cutoff was based on prior studies, which found that a difference in the range of 1.6 to 2.2 cm was meaningful for the patient when assessment was done on the visual analog scale.[8, 14, 15] We also evaluated the direction of discordance. If the patient's rating of dyspnea severity was higher than the provider's rating, we defined this as underestimation by the provider; in the instance where a provider's score of dyspnea severity was higher than the patient's score, we defined this as overestimation. In a sensitivity analysis, agreement was defined as a score within 2 between patient and healthcare provider, and any difference 3 was considered disagreement.

Other Variables

We obtained information from the medical records about patient demographics, body mass index (BMI), smoking status, and vital signs. We calculated the oxygen saturation index as the ratio between the oxygen saturation and the fractional inspired oxygen (SpO2/FIO2). Comorbidities were assessed based on the International Classification od Diseases, Ninth Revision, Clinical Modification codes from the hospital financial decision support system. We calculated an overall combined comorbidity score based on the method described by Gagne, which is based on elements from the Charlson Comorbidity Index and from the Elixhauser comorbidities.[16]

Charts of the patients included in the study were retrospectively reviewed for physicians' and nurses' documentation of dyspnea at admission and at discharge. We recorded if dyspnea was mentioned and how it was assessed: whether it was described as present/absent; graded as mild, moderate, or severe; used a quantitative scale (010); used descriptors (eg, dyspnea when climbing stairs); and whether it was defined as improved or worsened without other qualifiers.

Statistical Analysis

Descriptive statistics of dyspnea scores, patient characteristics, comorbidities and vital signs were calculated and presented as medians with interquartile range (IQR) for continuous variables, and counts with percentages for categorical factors. Every patient‐provider concurrent scoring was included in the analysis as 1 dyad, which resulted in patients being included multiple times in the analysis. Patient‐physician and patient‐nurse dyads of dyspnea assessment were examined separately. Analyses included all dyads that were within same assessment period (same day and same time window).

The relationship between patient self‐perceived dyspnea severity and provider rating of was assessed in several ways. First, a weighted kappa coefficient was used as a measure of agreement between patient and nurse or physician scores. A weighted kappa analysis was chosen because it penalizes disagreements that are further apart from each other.

Second, we defined an indicator of discordance and constructed multivariable generalized estimating equation models that account for clustering of multiple dyads per patient, to assess the relationship of patient characteristics with discordance. Finally, we developed additional models to predict underestimation when compared to agreement or overestimation of dyspnea by the healthcare provider relative to the patient. Using the same definitions for agreement, we also compared the dyspnea assessment estimation between physicians and nurses.

We present the differences in dyspnea assessment between patient and healthcare provider and between nurses and physicians by Bland‐Altman plots.

All analyses were performed using SAS (version 9.3; SAS institute, Inc., Cary, NC), Stata (Stata statistical software release 13; StataCorp, College Station, TX), and RStudio version 0.99.892 (Bland‐Altman plots, R package version 0.3.1; The R Foundation for Statistical Computing, Vienna, Austria).[8, 9, 17]

RESULTS

Patient Characteristics

Among the 219 patients who met the screening criteria, 81 were not enrolled (Figure 1). Data from 138 patients, with both patient information and provider data on dyspnea assessment, were included. The median age of the patients was 72 years (IQR, 5880 years), 56.5% were women, 75.4% were white, and 28.3% were current smokers. Approximately 30% had a diagnosis of HF, 30% of COPD, and 13.0% of pneumonia. The median comorbidity score was 4 (IQR, 26), and 37.0% of the patients had a BMI 30. At admission, the median oxygen saturation index was 346 (IQR, 287.5460) indicating mild to moderate levels of hypoxia. (Table 1).

Patient Characteristics (N = 138)
Value
  • NOTE: Abbreviations: BMI, body mass index; COPD, chronic obstructive pulmonary disease; IQR, interquartile range; NRS, numerical rating scale.

Age, median (IQR), y 72 (5880)
Gender
Female 78 (56.5)
Male 60 (43.5)
Race
White 104 (75.4)
Black 16 (11.6)
Hispanic 17 (12.3)
Other 1 (0.7)
Body mass index, median (IQR) 28 (23.334.6)
Obese (BMI 30) 51 (37.0)
Smoker, current 39 (28.3)
Admitting diagnosis
Heart failure 46 (33.3)
COPD/asthma 41 (29.7)
Pneumonia 18 (13.0)
Other 33 (23.9)
Depression 32 (23.2)
Comorbidity score, median (IQR) 4 (26)
Respiratory rate at admission, median (IQR) 20 (1924)
Oxygen saturation index at admission, median (IQR) 346.4 (287.5460)
Patient NRS, median (IQR)
At admission 9 (710)
At discharge 2 (14)
Discharged on home oxygen 45 (32.6)
Respiratory rate at discharge, median (IQR) 20 (1820)
Oxygen saturation index at discharge, median (IQR) 475 (350485)
Figure 1

Creation of the study cohort by application of inclusion and exclusion criteria. The final analytic sample included 96 patient‐physician pairs which generated 124 assessments and 138 patient‐nurse pairs which generated 336 assessments.

Agreement Between Patients' Self‐Assessment and Providers' Assessment of Dyspnea Severity

Not all patients had complete data points, and more nurses were interviewed than physicians. Overall, 96 patient‐physician and 138 patient‐nurse pairs participated in the study. A total of 336 patient‐nurse rating dyad assessments of dyspnea and 124 patient‐physician rating dyads assessments were collected (Figure 1). The mean difference between patient and physicians and patient and nurses assessments of dyspnea was 1.23 (IQR, 3 to 0) and 0.21 (IQR, 2 to 2) respectively (a negative score means underestimation by the provider, a positive score means overestimation).

The unadjusted agreement on the severity of dyspnea was 36.3% for the patient‐physician dyads and 44.1% for the patient‐nurse dyads. Physicians underestimated their patients' dyspnea 37.9% of the time and overestimated it 25.8% of the time; nurses underestimated it 43.5% of the time and overestimated it in 12.4% of the study patients (Table 2). In 28.2% of the time, physicians were discordant more than 4 points of the patient assessment. Bland‐Altman plots show that there is greater variation in differences of dyspnea assessments with increase in shortness of breath scores (Figure 2). Nurses underestimated more when the dyspnea score was on the lower end. Physicians also tended to estimate either lower or higher when compared to patients when the dyspnea scores were <2 (Figure 2A,B).

Underestimation and Overestimation and Concordance of Dyspnea
Underestimation Concordance Overestimation
3 2 %* 0 1 % 2 3 %
  • NOTE: NRS scores by nurses and physicians as compared with patients *Percent underestimation out of all dyads. Percent concordance out of all dyads. Percent overestimation out of all dyads.

Patient‐nurse dyads 110 48 43.5 82 78 44.1 17 28 12.4
Patient‐physician dyads 33 14 37.9 21 24 36.3 12 20 25.8
Figure 2

Bland‐Altman plots comparing differences in assessment of dyspnea between patients and healthcare providers. (A) Nurse‐patient assessment. (B) Physician‐patient assessment. (C) Physician‐nurse assessment. For each data point, the mean value (patient healthcare provider)/2) figures are on the x‐axis, and the difference value (healthcare provider score‐patient score) are on the y‐axis. The size of the markers reflects the number of observations at that locus. The mean differences and limits of agreement between patients and healthcare providers are represented by dashed lines.

The weighted kappa coefficient for agreement was 0.11 (95% confidence interval [CI]: 0.01 to 0.21) for patient‐physician assessment, 0.18 (95% CI: 0.12 to 0.24) for patient‐nurse, and 0.09 (0.02 to 0.20) for physician‐nurse indicating poor agreement. In a sensitivity analysis in which we used a higher threshold for defining discordance (difference of more than 2 points), the kappa coefficient increased to 0.21 (95% CI: 0.06 to 0.36) for patient‐physician assessments, to 0.24 (95% CI: 0.15 to 0.33) for patient‐nurse, and to 0.24 (95% CI: 0.09 to 0.39) for nurse‐physician assessments.

Predictors of Discordance and Underestimation of Dyspnea Severity Assessment

Principal diagnosis was the only factor associated with the physicians' discordant assessment of patients' dyspnea. Patients with admission diagnoses other than HF, COPD, or pneumonia (eg, pulmonary embolism) were more likely to have an accurate assessment of their dyspnea by providers (Table 3). Similar results were obtained in the sensitivity analysis by using a higher cutoff for defining discordance and when assessing predictors for underestimation (results not shown).

Predictors of Discordant Assessment of Dyspnea Between Patient and ProviderUnivariate and Multivariable Analysis
Modeling Probability of Discordance
Physician‐Patient Dyads, OR (95% CI), N = 124 Nurse‐Patient Dyads, OR (95% CI), N = 363
  • NOTE: Abbreviations: BMI, body mass index; CI, confidence interval; COPD, chronic obstructive pulmonary disease; OR, odds ratio. *P < 0.10.

Univariate Analysis
Body mass index 1.00 (0.991.01) 1.00 (0.991.00)
Comorbidity score 1.01 (0.981.05) 0.99 (0.961.01)
Respiratory rate at admission 1.00 (0.991.02) 0.99 (0.981.00)
Oxygen saturation at admission 1.00 (1.001.00) 1.00 (1.001.00)
Age (binary)
65 years Referent Referent
>65 years 1.21 (0.572.55) 0.96 (0.571.64)
Gender
Female Referent Referent
Male 1.10 (0.522.32) 0.81 (0.481.37)
Race
White Referent Referent
Nonwhite 1.02 (0.442.37) 1.06 (0.581.95)
Obese (BMI >30) 1.43 (0.663.11) 0.76 (0.441.30)
Smoker 1.36 (0.613.05) 1.04 (0.591.85)
Admitting diagnosis
Heart failure Referent Referent
COPD/asthma 0.68 (0.251.83) 1.91 (0.983.73)*
Pneumonia 0.38 (0.101.40) 1.07 (0.462.45)
Other 0.30 (0.110.82)* 1.54 (0.763.11)
Depression 1.21 (0.572.55) 1.01 (0.541.86)
Multivariable analysis
Admitting diagnosis
Congestive heart failure Referent Referent
COPD 0.68 (0.251.83) 1.91 (0.983.73)*
Pneumonia 0.38 (0.101.40) 1.07 (0.462.45)
Other 0.30 (0.110.82)* 1.54 (0.763.11)

In the multivariable analysis that assessed patient‐nurse dyads, the diagnosis of COPD was associated with a marginally significant likelihood of discordance (OR: 1.91; 95% CI: 0.98 to 3.73) (Table 3). Similarly, multivariable analysis identified principal diagnosis to be the only predictor of underestimation, and COPD diagnosis was associated with increased odds of dyspnea underestimation by nurses. When we used a higher cutoff to define discordance, the principal diagnosis of COPD (OR: 3.43, 95% CI: 1.76 to 6.69) was associated with an increased risk of discordance, and smoking (OR: 0.54, 95% CI: 0.29 to 0.99) was associated with a decreased risk of discordance. Overall, 45 patients (32.6%) were discharged on oxygen. The odds of discrepancy (under‐ or overestimation) in dyspnea scores between patient and nurse were 1.7 times higher compared to patients who were not discharged on oxygen, but this association did not reach statistical significance; the odds of discrepancy between patient and physician were 3.88 (95% CI: 1.07 to 14.13).

Documentation of Dyspnea

We found that dyspnea was mentioned in the admission notes in 96% of the charts reviewed; physicians used a qualitative rating (mild, moderate, or severe) to indicate the severity of dyspnea in only 16% of cases, and in 53% a descriptor was added (eg, dyspnea with climbing stairs, gradually increased in the prior week). Nurses were more likely than physicians to use qualitative ratings of dyspnea (26% of cases), and they used a more uniform description of the patient's dyspnea (eg, at rest, at rest and on exertion, or on exertion) than physicians. At discharge, 83% of physicians noted in their discharge summary that dyspnea improved compared with admission but did not refer to the patient's baseline level of dyspnea.

DISCUSSION

In this prospective study of 138 patients hospitalized with cardiopulmonary disease, we found that the agreement between patient's experience of dyspnea and providers' assessment was poor, and the discordance was higher for physicians than for nurses. In more than half of the cases, differences between patient and healthcare providers' assessment of dyspnea were present. One‐third of the time, both physicians and nurses underestimated patients' reported levels of dyspnea. Admitting diagnosis was the only patient factor predicting lack of agreement, and patients with COPD were more likely to have their dyspnea underestimated by nurses. Healthcare providers predominantly documented the presence or absence of dyspnea and rarely used a more nuanced scale.

Discrepancies between patient and provider assessments for pain, depression, and overall health have been reported.[8, 18, 19, 20, 21] One explanation is that patients and healthcare providers measure different factors despite using the same terminology. Furthermore, patient's assessment may be confounded by other symptoms such as anxiety, fatigue, or pain. Physicians and nurses may underevaluate and underestimate the level of breathlessness; however, from the physician perspective, dyspnea is only 1 data point, and providers rely on other measures, such as oxygen saturation, heart rate, respiratory rate, evidence of increasing breathing effort, and arterial blood gas to drive decision making. In a recent study that evaluated the attitudes and beliefs of hospitalists regarding the assessment and management of dyspnea, we found that most hospitalists indicated that awareness of dyspnea severity influences their decision for treatment, diagnostic testing, and timing of the discharge. Moreover, whereas less than half of the respondents reported experience with standardized assessment of dyspnea severity, most stated that such data would be very useful in their practice.[22]

What is the clinical significance of having discordance between patients self‐assessment and providers impression of the patient's severity of dyspnea? First, inaccurate assessment of dyspnea by providers can lead to inadequate treatment and workup. For example, a physician who underestimates the severity of dyspnea may fail to recognize when a complication of the underlying disease develops or may underutilize symptomatic methods for relief of dyspnea. In contrast, a physician who overestimates dyspnea may continue with aggressive treatment when this is not necessary. Second, lack of awareness of dyspnea severity experienced by the patient may result in premature discharge and patient's frustration with the provider, as was shown in several studies evaluating physician‐patient agreement for pain perception.[21, 23] We found that discrepancy between patients and healthcare providers was more pronounced for patients with COPD. In another study using the same patient cohort, we reported that compared to patients with congestive heart failure, those with COPD had more residual dyspnea at discharge; 1 in 4 patients was discharged with a dyspnea score of 5 of greater, and almost half reported symptoms above their baseline.[24] The results from the current study may explain in part why patients with COPD are discharged with higher levels of dyspnea and should alert healthcare providers on the importance of patient‐reported breathlessness. Third, the high level of discordance between healthcare providers and patients may explain the undertreatment of dyspnea in patients with advanced disease. This is supported by our findings that the discordance between patients and physicians was higher if the patient was discharged on oxygen.

One key role of the provider during a clinical encounter is to elicit the patient's symptoms and achieve a shared understanding of what the patient is experiencing. From the patient's perspective, their self‐assessment of dyspnea is more important that the physician's assessment. Fortunately, there is a growing recognition and emphasis on using outcomes that matter to patients, such as dyspnea, to inform judgment about patient care and for clinical research. Numerical measures for assessment of dyspnea exist, are easy to use, and are sensitive to change in patients' dyspnea.[6, 25, 26, 27] Still, it is not standard practice for healthcare providers to ask patients to provide a rating of their dyspnea. When we examined the documentation of dyspnea in the medical record, we found that the description was vague, and providers did not use a standardized validated assessment. Although the dyspnea score decreased during hospitalization, the respiratory rate did not significantly change, indicating that this objective measure may not be reliable in patient assessment. The providers' knowledge of the intensity of the symptom expressed by patients will enable them to track improvement in symptoms over time or in response to therapy. In addition, in this era of multiple handoffs within a hospitalization or from primary care to the hospital, a more uniform assessment could allow providers to follow the severity and time course of dyspnea. The low level of agreement we found between patients and the providers lends support to recommendations regarding a structured dyspnea assessment into routine hospital practice.

Study Strengths and Limitations

This study has several strengths. This is 1 of the very few studies to report on the level of agreement between patients' and providers' assessments of dyspnea. We used a validated, simple dyspnea scale that provides consistency in rating.[28, 29] We enrolled patients with a broad set of diagnoses and complaints, which increases the generalizability of our results, and we surveyed both physicians and nurses. Last, our findings were robust across different cutoff points utilized to characterize discordance and across 3 frequent diagnoses.

The study has several limitations. First, we included only English‐speaking patients, and the results cannot be generalized to patients from other cultures who do not speak English. Second, this is a single‐center study, and practices may be different in other centers; for example, some hospitals may have already implemented a dyspnea assessment tool. Third, we did not collect information on the physician and nurse characteristics such as years in practice. However, a recent study that describes the agreement of breathlessness assessment between nurses, physicians, and mechanically ventilated patients found that underestimation of breathlessness by providers was not associated with professional competencies, previous patient care, or years of working in an intensive care unit.[9] In addition, a systematic review found that length of professional experience is often unrelated to performance measures and outcomes.[30] Finally, although we asked for physicians and nurses assessment close to their visit to the patient, assessment was done from memory, not at the bedside observing the patient.

CONCLUSION

We found that the extent of agreement between a structured patient self‐assessment of dyspnea and healthcare providers' assessment was low. Future studies should prospectively test whether routine assessment of dyspnea results in better acute and long‐term patient outcomes.

Acknowledgements

The authors acknowledge Ms. Anu Joshi for her help with formatting the manuscript and assisting with table preparations. The authors also acknowledge Ms. Katherine Dempsey, Jahnavi Sagi, Sashi Ariyaratne, and Mr. Pradeep Kumbaham for their help with collecting the data.

Disclosures: M.S.S. is the guarantor for this article and had full access to all of the data in the study, and takes responsibility for the integrity and accuracy of the data analysis. M.S.S., P.K.L., E.N., and M.B.R. conceived of the study. M.S.S. and B.M. acquired the data. M.S.S., A.P., P.S.P., R.J.G., and P.K.L. analyzed and interpreted the data. M.S.S. drafted the manuscript. All authors critically reviewed the manuscript for intellectual content. M.S.S. is supported by grant 1K01HL114631‐01A1 from the National Heart, Lung, and Blood Institute of the National Institutes of Health and by the National Center for Research Resources and the National Center for Advancing Translational Sciences, National Institutes of Health, through grant UL1RR025752. The authors report no conflicts of interest.

References

Online-Only Materials

   Comments ()