Serious Choices: A Systematic Environmental Scan of Decision Aids and Their Use for Seriously Ill People Near Death


Seriously ill people near death face difficult decisions about life-sustaining treatments such as cardiopulmonary resuscitation and mechanical ventilation. Patient decision aids may improve alignment between patients’ preferences and the care they receive, but the quantity, quality, and routine use of these tools are unknown. We conducted a systematic environmental scan to identify all decision aids for seriously ill people at high risk of death facing choices about life-sustaining treatments, assess their quality, and explore their use in clinical settings. We searched MEDLINE, Google, and mobile application stores and surveyed experts. We included 27 decision aids in our scan. Concerning content, 14 of 27 decision aids for seriously ill people near death were for people with specific diseases and conditions (ie, advanced cancer or kidney disease); 11 concerned individual life-sustaining treatment decisions (ie, cardiopulmonary resuscitation or mechanical ventilation). Only two focused on more general care pathways (ie, life-sustaining intervention, palliative care, and hospice). Twenty-four of 27 decision aids presented options in a balanced way; 23 identified funding sources, and 19 of 27 reported their publication date. Just 11 used plain language. A minority, 11 of 27, listed evidence sources, five documented rigorous evidence-synthesis methods, six disclosed competing interests, and three offered update policies. Preliminary results suggest that few health systems use decision aids in routine patient care. Although many decision aids exist for life-sustaining treatment decisions during serious illness, the tools are deficient in some key quality areas.

© 2019 Society of Hospital Medicine

People often do not receive the kind of care they want at the end of their lives.1,2 Although most people say they do not wish to have aggressive interventions if they are dying,3-5 nearly one in five dies in the hospital and one in seven dies in the intensive care unit (ICU), where aggressive care is usually provided.6 Coming demographic shifts will put this phenomenon in relief. The US Census Bureau estimates the number of people over age 85 will balloon to 20 million by 2050.7

A proposed strategy for reducing this mismatch is to expand shared decision making for people facing life-sustaining treatment decisions.8-10 Patient decision aids are tools that help people make informed healthcare decisions in light of their values and preferences, facilitating shared decision making.8,11 Decision aids can take many forms: paper-based, audio/video-based, or online. They can be intended for the clinical encounter (used in partnership with a physician, nurse, or other clinician), independent patient use, or peer-to-peer use.8 In a 2017 review, Stacey and colleagues found that patient decision aids improve knowledge, clarify values, encourage more active decision making, and improve risk perception, across a variety of treatment and screening decisions.12 They also concluded that decision aids might help people make decisions that are more aligned with their values, without affecting health outcomes negatively. 12

The number of available patient decision aids for people making life-sustaining treatment choices during serious illness near death is currently unknown. A 2014 review of all advanced care planning decision aids, including those for people who are healthy and people who are seriously ill, found 16 published studies in the peer-reviewed literature that tested patient decision aids for advanced care planning, but they did not systematically search the Internet and query key informants.13

Given the frequency of serious illness and death in hospital settings, awareness of potentially useful tools, their quality, and their use may be of interest to practicing hospitalists. This awareness may inform their decision making around whether or not to use decision aids in their own practice.


Study Aims and Design

With our systematic environmental scan, we aimed to identify all decision aids available to seriously ill people near death facing choices about life-sustaining treatments, developed by both academic researchers and private organizations. We set out to articulate their quality and the degree to which they are used.


We developed four research questions to address our study objectives. Our questions were as follows: (1) What English-language patient decision aids are available? (2) What are the characteristics of these patient decision aids? (3) What is the quality of these patient decision aids, including readability? (4) What organizations use these patient decision aids in routine care (exploratory)? 14-16 See protocol: doi: 10.1007/s40271-017-0268-2.17

Decision Aid Search Strategy

We searched for patient decision aids among published systematic reviews, Internet search results (, and app stores (Google Play and Apple App Store). To identify previously published systematic reviews, we searched MEDLINE via PubMed, with the date range from inception to 2017. We chose not to include other academic databases because the unit of observation for this environmental scan was the decision aids themselves, not the published articles. Additionally, we were aware of systematic reviews concerning this issue and felt that adding additional databases would not appreciably improve our likelihood of identifying eligible decision aids. We conducted searches using on November 30, 2016, and January 26, 2017, and included the first 100 search results. We also contacted shared decision-making and palliative care experts using a previously established list, via an online survey and one-on-one interviews between April 17, 2017, and August 30, 2017.

Published Reviews

Using a search strategy developed with a librarian, we identified reviews of decision aids that met our inclusion criteria using the MEDLINE database.17 The primary reviewer (CHS) examined the results of the search, identifying reviews appropriate for further investigation and the secondary reviewer (KP) extracted patient decision aids potentially eligible for our study. See Appendix Table 1 and our published protocol.17 Notably, given that the decision aids themselves, not published articles, were the unit of observation for our environmental scan, we did not perform dual coding on the MEDLINE extraction.

Google and App Stores

Two reviewers (CHS and MAD) performed the Google and application screening, including both the Apple App Store and Google Play.17 Using Google Advanced Search, we ran the queries detailed in Appendix Table 2. We disabled cookies and limited our search to English.

The primary reviewer ran each Google search and app store search, archiving the first 100 results of Google searches and first 50 results of app store searches.18 Then, the primary reviewer opened each page and scanned for patient decision aids or references to patient decision aids, marking those that met our inclusion criteria, those that might meet our inclusion criteria with further research, and those that were not appropriate. We documented specific reasons for exclusion. The secondary reviewer assessed a randomly-selected, 10% subsample. We calculated interrater reliability using a Cohen’s Kappa statistic.

Key Informants

To identify decision aids that did not appear in our online search, we surveyed 187 key informants who work in or study issues related to aging, death and dying and shared decision making.19 We developed a questionnaire for these informants and deployed it using the online survey software Qualtrics (see Appendix 1. Key Informant Survey). We used a snowball approach, asking participants for other individuals they thought we should speak with about other relevant decision aids. We corresponded with individuals who suggested decision aids that were not already in our decision aid database.

Decision Aid Selection Criteria

We included patient decision aids designed to help seriously ill people near death or their caregivers make decisions about life-sustaining treatments. See Appendix Table 1 for an explanation of terms. We saved decision aids that met our inclusion criteria in an online database, organizing them by target user or index decision(s). When identified decision aids were unavailable online, we e-mailed developers three times to ask for access to the decision aid. If after three queries, we did not receive access to the decision aid, we excluded the tool from our review. Similarly, if developers explicitly refused to participate in the study, we excluded them.

Once we banked and organized the decision aids, one reviewer (KP) systematically collected information about decision aid characteristics using a data collection form (see Appendix 2. Table 3). The data we collected for decision aids from all sources included (1) the index decision, (2) secondary decision(s), (3) the disease/condition, (4) availability (whether the decision aids are available publicly or proprietorially), and (5) use, ie, whether we learned anything about routine use in clinical environments.

Decision Aid Quality Grading Methods

At least two or three reviewers (C.H.S., K.P., M.A.D.), independently assessed the quality of each included patient decision aid, using the NQF standards. Before assessing the quality of each decision aid, we tested an NQF quality assessment form on five decision aids. We subsequently added specificity to the NQF quality criteria for this review. At least two of three reviewers (CHS, KP, MAD) assessed the quality of all included patient decision aids. We calculated interrater reliability using both Cohen’s Kappa statistic for individual quality categories and Spearman’s correlations for overall scores.

Notably, one of the NQF items concerns plain language. We assessed plain language using average readability scores, generated via If readability scores were below seventh-grade level, we considered them plain language. When we could not assess readability using an average score, ie, in the case of video decision aids, the researchers made a qualitative judgment about the plain language criteria.

Statistical Analysis

Our primary outcome was the number and variety of decision aids available for seriously ill individuals near death facing choices about life-sustaining treatments. Secondary outcomes included the quality, actual availability, and use of the available decision aids. We used Stata 13 to synthesize our results. We also reported overall quality and use. We conducted subgroup analyses, including quality, availability, and use of decision aids by category.


Decision Aid Selection Process

We identified 608 links with information about potential decision aids from our Google search. The two raters had substantial interrater reliability according to Cohen’s Kappa statistic (K = 0.64).20 We did not detect any possible decision aids with our app store searches. We identified 31 studies from our MEDLINE search with information about potential decision aids eligible for inclusion. We received 60 responses to our expert survey from the 187 administered (a 32% response rate).

Altogether, we identified 105 potential decision aids from these sources. We excluded 22/105 potential decision aids from our analysis because they were not publicly accessible, and we could not successfully obtain them from the developers. It remains unknown whether these tools would have qualified for inclusion in our review. We excluded 55/105 tools for not meeting one of the following criteria: 1) not being decision aids according to the NQF criteria 2) not concerning life-sustaining treatments 3) not being targeted at people with serious illness near death. A majority of decision aids for life-sustaining treatment decisions are intended for people who do not yet have an advanced serious illness or are not near death. There were 27 decision aids in our final review (Figure 1).

Characteristics of Included Decision Aids

Of the 27 decision aids we included in our review, 14 (52%) were tailored to seriously ill individuals with specific conditions. Eleven decision aids (41%) concerned specific life-sustaining treatments. Two decision aids concerned general treatment approaches, such as life-sustaining care versus palliative care (Table 1).

The decision aids were of variable length and approach. Some were text only, while others were image heavy. The mean length of decision aids was 19 pages, while the median length was 10 pages. Included decision aids offered interventions meant to return patients to health, as well as palliative interventions and comfort care.

Notably, most of the decision aids we included in our review (25 decision aids; 93%) were freely available online. Three (11%) were not. Seventeen (63%) decision aids were developed in the U.S., eight (30%) in Canada, two (7%) in Australia, and one (4%) in the Netherlands (in Dutch, translated using Google Translate). Additionally, there were 22 potentially eligible decision aids that we could not access to review and therefore could not include.

Quality of Included Decision Aids

The overall correlation of scores between the two reviewers was high (0.85). Agreement was high for both reviewers for all categories (balanced 90%, K = 0.0; outcome probabilities 86%, K = 0.7; publication date 93%, K = 0.8; update policy 93%, K = 0.7; funding sources 96%, K = 0.8), except the category concerning the rigor of the decision aid development process (66%, K = 0.2) and the evidence sources used (79%, K = 0.6) categories.

The quality of the decision aids was high in some categories. Of 27 decision aids, most presented options in a balanced way (24, 89%) and identified funding sources (23, 85%). They also reported publication dates most of the time (19, 70%). Readability of the included decision aids was mixed. The average readability grade level was 7.5, with a low score of 4.1 and a high score of 10.7. Eleven decision aids (41%) had readability levels less than seventh grade (Table 2). Thirteen had plain language, including video decision aids that we agreed used plain language.

The decision aids also had consistently low scores in some categories. Of 27, only 11 listed their evidence sources (41%), 11 reported a rigorous evidence-synthesis method (41%), six stated their competing interests (22%), and three offered an update policy (11%). There were no notable differences in the quality of the decision aids in each of the three category types (condition-specific, treatment-specific, general).

Use of Included and Excluded Decision Aids (exploratory)

We received 60 of 187 responses to our key informant survey. We asked every respondent if they were aware of any relevant decision aids. Of the 60 respondents, 45 (75%) said they were aware of decision aids, but only 38 (63%) offered the names of potential tools. Twenty-six respondents (43%) said they were aware of institutions that used the decision aids in routine and sustained care. Twenty-four respondents (40%) offered names of organizations, but most of the suggestions concerned decision aids that did not qualify for inclusion in our review or care that was not routine or sustained. In this preliminary use estimation, we found evidence for the use of only three decision aids or similar tools in routine care, two of which we included in our review.


We found many decision aids of varying quality for people with serious illnesses facing decisions about life-sustaining treatments. Most available decision aids are customized for people with particular diseases or conditions, like cancer or heart failure, with few generalized tools. This may make it difficult for practicing clinicians to find tools that are appropriate for their patients. It could also contribute to the gap between their availability and use in routine care, which is an essential but exploratory finding of this systematic environmental scan. Even if seriously ill people or those who cared for them wanted to obtain and use a decision aid independently, a large proportion of them are not publicly accessible.

Concerning the quality of decision aids, they were usually balanced and listed their funding sources, but other quality areas we often missing concerning their development, content, and disclosures. These deficiencies may affect the trustworthiness of decision aids, which may make practicing clinicians less likely to use them in hospital settings. Reporting of outcome probabilities was particularly weak. Reporting outcome probabilities in ways that people who are ill and their relatives can understand, especially during times of heightened emotion, is critically important. Therefore, it is a cause for concern that the available decision aids often neglect to use evidence-based techniques for conveying outcome information.

Our work built on Butler and colleagues’ “state of the science” review in 2014.13 Focusing specifically on proximal life-sustaining treatment decisions, we found many more decision aids by expanding our search beyond the peer-reviewed literature to include the Internet and experts.13 We also identified an important gap worthy of further exploration between the decision aids available and their usage in real-world clinical environments.

Our review confirms that implementation of decision aids in routine care is a continued challenge, especially for seriously ill people facing life-sustaining treatment decisions.53 Why tools that are efficacious in controlled trial environments have failed to gain acceptance in real-world settings remains unanswered for this population.54 For decision aids in general, researchers have reported barriers concerning clinician awareness, perception, and comfort, as well as usability issues.55,56 Additionally, systems-level barriers exist, like culture and priorities, difficulty incorporating decision aids into the workflow, resistance from parties who favor other interventions, and the costs associated with implementation.56 There may also be particular barriers related to the topics of death and dying.A strength of this work is thatwe applied the rigor of the systematic review method to the environmental scan, a newer method that answers different questions, such as “How many?”, “How much?”, and “How often?” We hope our use of the word systematic will reinforce perception among the scientific community that the environmental scan method is thorough, valid and worthwhile. We believe this method unearthed more decision aids than a traditional systematic review limited to the academic literature would have revealed. Another strength of our review was the rigor of screening and assessment.

A limitation of our work is the challenge of defining serious illness. We worked with palliative care physicians to make these judgments as grounded in clinical practice as possible. The preliminary nature and selection of experts for our sustained—use survey are limitations as well. Despite our efforts to conduct a comprehensive review of a vast environment of tools, we may have missed some decision aids that met our inclusion criteria. An additional limitation of our work is that due to the exploratory nature of our sustained-use survey, we cannot determine with accuracy how often these tools are used, although we have provided the first preliminary assessment of use, to our knowledge.

The gap between prolific patient decision aid development and real-world usage is puzzling. It is possible that using a tool at all is inappropriate for the complex, emotionally-laden decision-making process associated with death and dying. Alternatively, the tools may be inappropriate for serious illness, due to their design, their content, or some other characteristics. Perhaps the existing tools are too tailored for specific conditions and interventions―less appropriate for generalized use. Indeed, only two decision aids included in our final review addressed general care pathways, like life-sustaining care, palliative care, and hospice care. The others were highly specific, concerning particular diseases like kidney disease and particular interventions, like CPR. We know that most people die with comultimorbidities, meaning such specificity may paradoxically make it more difficult for individuals and their families to identify with the content in the materials.57,58 Without having data from real-world use, we cannot know whether any particular tool is suited or helpful for hospital practice.

It is essential for practicing hospitalists to know whether patient decision aids are appropriate for use in routine care. We hope that our review will help clinicians and health systems find appropriate tools to use with their patients. We also believe there should be mechanisms for providing feedback on whether decision aids are feasible and acceptable to hospitalized people and their caregivers and to practicing hospitalists and what leads to their sustained implementation.55,56 This can be explored with on-the-ground observational research or through health system quality improvement efforts.


Pamela J. Bagley provided search strategy support. Meredith MacMartin provided clinical counsel. Amber Barnato provided comments and insight as an advisor and a new member of Catherine’s Ph.D. committee.

Author contributions

Catherine H. Saunders designed the study, with support from Marie-Anne Durand, Glyn Elwyn, and Kathryn Kirkland. Catherine H. Saunders conducted all screening, with support from Marie-Anne Durand. Khusbu Patel managed the inventory of decision aids. Catherine H. Saunders designed and distributed the key informant survey, with support from Marie-Anne Durand. Hyunkyung Kang and Catherine H. Saunders managed follow-up with key informants. Khusbu Patel and Catherine H. Saunders conducted the decision aid quality review. Catherine H. Saunders, Marie-Anne Durand, and Kathryn Kirkland screened decision aids to determine appropriateness for people with serious illness. Catherine H. Saunders drafted the manuscript, and all authors reviewed and approved it.

Ethical approval

The Dartmouth College Committee for the Protection of Human Subjects designated this project as exempt from further review. All survey participants confirmed their consent via an online form.


Ms. Saunders, Ms. Patel, Ms. Kang, and Dr. Kirkland have nothing to disclose. Dr. Elwyn reports personal fees from ACCESS Community Health Network, personal fees from EBSCO Health, personal fees from Chicago (Federally Qualified Medical Centers), outside the submitted work, and as Director of &think LLC, which owns the registered trademark for OptionGrids(TM) patient decision aids. He owns copyright in measures of shared decision making and care integration, namely collaboRATE, integRATE, Observer OPTION-5, and Observer OPTION-12, which are freely available for use. He is codeveloper of the OptionGrid patient decision aids, which are licensed to EBSCO Health. He has received reimbursement for travel, accommodations, and expenses from EBSCO Health, ACCESS Community Health Network, and Chicago (Federally Qualified Medical Centers). Dr. Durand reports personal fees from ACCESS Community Health Network, personal fees from EBSCO Health, outside the submitted work, and as codeveloper of the OptionGrid patient decision aids, which are licensed to EBSCO Health. She has received reimbursement for travel, accommodations, and expenses from EBSCO Health and ACCESS Community Health Network.

Financial conflicts of interest

Glyn Elwyn (GE) and Marie-Anne Durand (M-A D) have developed the Option Grid patient decision aids, which are licensed to EBSCO Health. They receive consulting income from EBSCO Health and may receive royalties in the future. M-A D is a consultant for ACCESS Community Health Network. No other competing interests declared.


The authors did not receive funding for this research.

Published protocol linked here:


Online-Only Materials

File appendix_jhm.docx23.53 KB
   Comments ()

Related Articles from PubMed

JAMA Intern Med. 2015 Jul;175(7):1213-21. doi: 10.1001/jamainternmed.2015.1679.