Often, we find ourselves relying solely on quantitative data when making decisions. However, this could be one of the biggest mistakes when assessing measures of patient experience as the full story is not always captured with numbers. Hospitals have been analyzing HCAHPS scores to gather insight into patient experience for years but often do not know how to best understand qualitative responses outside of domain scores.
When a “top box” score is received (the highest rating on every item in a particular survey category), frequently the qualitative responses are overlooked. Only when a negative survey score is received are comments more likely to be reviewed. This suggests that hospitals are more accustom to reacting to negative feedback instead of proactively identifying missed opportunities present in patient comments. If these negative comments are not proactively acted upon patients that are positive promoters could shift into overly critical detractors, consequentially negatively impacting the organizations net promoter score.
In a recent study “Perfect rating with negative comments: Learning from contradictory patient survey responses,” perfect top boxes scores were often accompanied by several negative comments. How could a patient rate their experience perfect with their submitted quantitative scores, but then go on to offer disapproval within their comments? Andrew Gallan and Marina Girju from DePaul University and Roxana Girju from University of Illinois Urbana-Champaign offer possible explanations for this phenomenon.
The research is based on vendor data from eleven different hospitals of a major U.S. health care system from January 2015 through October 2016. The dataset included survey scores and comments from 56,900 patients and of the patient responses, 30,485 (54%) included at least one patient comment. Over 91,281 total comments were reviewed. The research focused on likelihood to recommend the hospital to others. Ten domains of interest were explored: meals, tests/treatment, admission, discharge, visitor/family, personal issues, nurses, hospital room, physicians and overall hospital experience. In each domain, patients answered anywhere between two and six questions (five-point continuous survey from “Very Poor” to “Very Good”). The four key domain areas that held the most influence based on natural language processing of patient comments showed that despite perfect top box scores, patients wrote negative comments most often about personal issues, nurses, hospital room, and physicians.
Patients have a lot to share with their health care organizations about their hospital experience through their comments. The analysis illustrates that not only do doctors and nurses influence patient experience, but also allied health professionals (i.e. technicians, social works, nurse aides, etc.). Patient comments further revealed that lack of communication was a significant source of errors in patient care and added to patient stress (e.g. nurses do not communicate with other nurses, physicians do not talk to nurses). Only qualitative responses could identify these opportunities since none of the inpatient surveys captured these concerns.
The disparity between scores and comments supports the insight that survey data inflates overall patient experience scores and could lead to further issues if hospitals only focus on addressing issues present in domain scores. Both qualitative and quantitate data leveraged together can truly depict patient experience. Now it’s up to each health care organization to start actively listening to their patients through their comments and drive desired change through innovative solutions that proactively address patient concerns.