Brief Reports

Can social media be used as a hospital quality improvement tool?

Abstract

Many hospitals wish to improve their patients’ experience of care. To learn whether social media could be used as a tool to engage patients and to identify opportunities for hospital quality improvement (QI), we solicited patients’ narrative feedback on the Baystate Medical Center Facebook page during a 3‐week period in 2014. Two investigators used directed qualitative content analysis to code comments and descriptive statistics to assess the frequency of selected codes and themes. We identified common themes, including: (1) comments about staff (17/37 respondents, 45.9%); (2) comments about specific departments (22/37, 59.5%); (3) comments on technical aspects of care, including perceived errors and inattention to pain control (9/37, 24.3%); and (4) comments describing the hospital physical plant, parking, and amenities (9/37, 24.3%). A small number (n = 3) of patients repeatedly responded, accounting for 30% (45/148) of narratives. Although patient feedback on social media could help to drive hospital QI efforts, any potential benefits must be weighed against the reputational risks, the lack of representativeness among respondents, and the volume of responses needed to identify areas of improvement. Journal of Hospital Medicine 2016;11:52–55. © 2015 Society of Hospital Medicine

© 2015 Society of Hospital Medicine

Patient experience has become a major component of the Center for Medicare and Medicaid Services Value‐Based Purchasing initiative.[1] Hospitals have therefore focused quality improvement (QI) efforts on this area.[2] Hospital performance in the realm of patient experience is generally determined using systematic surveys with closed‐ended questions, but patient‐generated narrative feedback can help hospitals identify the components of care that contribute to patient satisfaction and or are in need of improvement.[3] Online narrative responses posted by patients on rating websites or social media have been criticized because they may not be representative of the population,[4] but they also have some advantages.[5] Any patient may leave a comment, not just those who are selected for a survey. Patients may also experience benefits through the act of sharing their story with others. Moreover, most US hospitals use some form of social media,[6] which they can theoretically use to self‐collect narrative data online. To realize the full potential of patient‐generated online narratives, we need a clearer understanding of the best practices for collecting and using these narratives. We therefore solicited patient feedback on the Facebook page of a large tertiary academic medical center to determine whether it is feasible to use social media platforms for learning about and improving hospital quality.

METHODS

Baystate Medical Center (BMC) is a tertiary care medical center in western Massachusetts. We identified key BMC stakeholders in the areas of QI and public affairs. Noting that patients have expressed interest in leaving comments via social media,[7] the group opted to perform a pilot study to obtain patient narratives via a Facebook prompt (Facebook is a social media site used by an estimated 58% of US adults[8]). The BMC public affairs department delivered a press release to the local media describing a 3‐week period during which patients were invited to leave narrative feedback on the BMC Facebook wall. The BMC Institutional Review Board deemed that this study did not constitute human subjects research.

During March 2014 (March 10, 2014March 24, 2014), we posted (once a week) an open‐ended prompt on BMC's Facebook wall. The prompt was designed to elicit novel descriptions of patient experience that could help to drive QI. It read: We want to hear about your experiences. In the comment section below, please tell us what we do well and how we can improve your care. Because of concerns about the potential reputational risks of allowing open feedback on a public social media page, the prompt also reminded patients of the social media ground rules: there should be no mention of specific physicians, nurses, or other caregivers by name (for liability reasons); and patients should not include details about their medical history (for privacy reasons).

We collected all posts to preserve comments and used directed qualitative content analysis to examine them.[9] Two research team members[3, 10, 11] independently coded the responses. Starting with an a priori codebook that was developed during a previous study,[3] they amended the codebook through an iterative process to incorporate new concepts. After independently coding all blocks of text, the coders reviewed their coding selections and resolved discrepancies through discussion. We then performed second‐level coding, in which codes were organized into major pertinent themes. We reviewed the coded text after applying secondary codes in order to check for accuracy of coding and theme assignment as well as completeness of second‐level coding. We calculated percent agreement, defined as both raters scoring a block of text with the same code divided by total number of codes. We also calculated the Spearman correlation between the 2 reviewers. We used descriptive statistics to assess the frequency of select codes and themes (see Supporting Information, Appendix 1 and Appendix 2, in the online version of this article).[9, 12, 13]

RESULTS

Over a 3‐week study period, 47 comments were submitted by 37 respondents. This yielded 148 codable statements (Table 1). Despite limited information on respondents, we ascertained from Facebook that 32 (86%) were women and 5 (14%) were men.

Number of Total, Positive, and Negative Comments and Representative Quotations for Themes
Theme Total Respondents, N (%) % Positive Positive Quotation % Negative Negative Quotation
  • NOTE: Abbreviations: ER, emergency room; IV, intravenous; NICU, neonatal intensive care unit.

Staff 17 (46) 45% The nurses in the pediatric unit, as well as the doctors in radiology and x‐ray department were AMAZING! 55% My 24‐year‐old daughter had to go for 5 days of IV treatmentwhile getting her infusion there was a fire alarm. She has a video showing the flashing of the light and the sound of the alarm and the closing of doors and NOT A SINGLE staff member to be found. Her infusions take about 2 hours. They set it and forget it. Luckily there wasn't a fire and someone did finally come to disconnect her.
Had a fabulous experience with Wesson women's this week! Had a C section and 3‐day admission. All staff from preoperative to inpatient were so helpful and really anticipated my needs before I could even ask for things. My mother was hospitalized for at least 3 weeks right after the cardiovascular center openedwhen she went into cardiac arrest and in acute care and the step unit the care was great, very attentive nurses and doctors. When she was starting to recover and moved upstairs, downhill it went. She'd ring for assistance because she wanted to walk to the bathrooms and more times she was left to her own devices because no one would respond.
Facility 9 (24) 25% New buildings are beautiful and the new signs are way better. 75% The parking situation was disappointing and the waiting room was also very dirty.
I really like the individual pods in the ER. I could have used a single room as my roommate was very annoying and demanding.
Departments 22 (60) 44% The NICU was great when my son was in there. The children's unit was great with my daughter and respected my needs. 56% Revamp maternity; it needs it desperately.
Labor and delivery was a great place. Love Baystate but hate the ER.
Technical aspects of care (eg, errors) 9 (24) 0 100% Day 2 of my 24 year old getting her 2‐hour IV infusion....she was set up with her IV. When checked 2 hours later, the staff member was very upset to find that only the saline had run. She never opened the medication clamp. So now they gave her the medication in 1 hour instead of 2.
If I had 1 suggestion it would be to re‐evaluate patient comfort when patients are waiting to be admitted.

From coded text, several broad themes were identified (see Table 1 for representative quotes): (1) comments about staff (17/37 respondents, 45.9%). These included positive descriptions of efficiency, caring behavior, good training, and good communication, whereas negative comments included descriptions of unfriendliness, apparent lack of caring, inattentiveness, poor training, unprofessional behavior, and poor communication; (2) comments about specific departments (22/37, 59.5%); (3) comments on technical aspects of care, including perceived errors, incorrect diagnoses, and inattention to pain control (9/37, 24.3%); and (4) comments describing the hospital physical plant, parking, and amenities (9/37, 24.3%). There were a few miscellaneous comments that did not fit into these broad themes, such as expressions of gratitude for our solicitation of narratives. Percent agreement between coders was 80% and Spearman's Rho was 0.82 (p<0.001).

A small number (n=3) of respondents repeatedly made comments over the 3‐week period, accounting for 30% (45/148) of codes. These repetitive commenters tended to dominate the Facebook conversation, at times describing the same experience more than once.

DISCUSSION

In this study evaluating the potential utility of social media as a hospital QI tool, several broad themes emerged. From these themes, we identified several areas that could be deemed as QI targets, including: training staff to be more responsive and sensitive to patients needs and concerns, improving patient and visitor parking, and reducing emergency department waiting times. However, the insight gained from solicited Facebook comments was similar to feedback gained from more traditional approaches of soliciting patient perspectives on care, such as patient experience surveys.[14]

Our findings should be viewed in the context of prior work focused on patient narratives in healthcare. Greaves et al. used sentiment analysis to describe the content of nearly 200,000 tweets (comments posted on the social networking website Twitter) sent to National Health Service (NHS) hospitals.[15] Themes were similar to those found in our study: (1) interaction with staff, (2) environment and facilities, and (3) issues of access and timeliness of service. Notably, these themes mirrored prior work examining narratives at NHS hospitals[3] and were similar to domains of commonly used surveys of patient experience.[14] The authors noted that there were issues with the signal to noise ratio (only about 10% of tweets were about quality) and the enforced brevity of Twitter (tweets must be 140 characters or less). These limitations suggest that using Twitter to identify QI targets would be difficult.

In contrast to Greaves et al., we chose to solicit feedback on our hospital's Facebook page. Facebook does not have Twitter's enforced brevity, allowing for more detailed narratives. In addition, we did not encounter the signal‐to‐noise problem, because our prompt was designed to request feedback that was relevant to recent experiences of care. However, a few respondents dominated the conversation, supporting the hypothesis that those most likely to comment may be the patients or families who have had the best or worst experiences. In the future, we will attempt to address this limitation and reduce the influence of repeat commenters by changing our prompt (eg, Please tell us about your experience, but please do not post descriptions of the same experience more than once.).

This pilot demonstrated some of the previously described benefits of online narratives.[5] First, there appears to be value in allowing patients to share their experiences and to read the experiences of others (as indicated in a few grateful patients comments). Second, soliciting online narratives offers a way for hospitals to demonstrate a commitment to transparency. Third, in contrast to closed‐ended survey questions, narrative comments help to identify why patients were satisfied or unsatisfied with their care. Although some surveys with closed‐ended questions also allow for narratives, these comments may or may not be carefully reviewed by the hospital. Using social media to solicit and respond to comments enhances existing methods for evaluating patient experience by engaging patients in a public space, which increases the likelihood that hospitals will attempt to improve care in response.

Notably, none of the identified areas for improvement could be considered novel QI targets for BMC. For example, our hospital has been very focused on training staff around patient experience, and emergency department wait times are the focus of a system‐wide improvement effort called Patient Progress.

This study has other limitations. We conducted this study over a 3‐week time period in a single center and on a single social media site whose members may not be representative of the overall patient population at BMC. Although we do not know how generalizable our findings are (in terms of identifying QI targets), we feel that we have demonstrated how using social media to collect data on patient experience is feasible and could be informative for other hospitals in other locations. It is possible that we did not allow the experiment to run long enough; a longer time or broader outreach (eg, a handout given to every discharged patient over a longer period) may be needed to allow patients adequate opportunity to comment. Of note, we did not specifically examine responses by time period, but it does seem, in post hoc analysis, that after 2 weeks of data collection we reached theoretical saturation with no new themes emerging in the third week (eg, third‐week comments included I heart your nurses. and Love Baystate but hate the ER.). More work is also needed that includes a broader range of social media platforms and more participating hospitals.

In conclusion, the opportunity to provide feedback on Facebook has the potential to engage and empower patients, and hospitals can use these online narratives to help to drive improvement efforts. Yet potential benefits must be weighed against reputational risks, a lack of representative respondents, and the paucity of novel QI targets obtained in this study.

Disclosures: Dr. Lagu is supported by the National Heart, Lung, and Blood Institute of the National Institutes of Health under award number K01HL114745. The authors report no conflicts of interest.

References

Online-Only Materials

   Comments ()