Original Research

Factors associated with medication warning acceptance for hospitalized adults



Computerized provider order entry (CPOE) systems can warn clinicians ordering medications about potential allergic or adverse reactions, duplicate therapy, and interactions with other medications. Clinicians frequently override these warnings. Understanding the factors associated with warning acceptance should guide revisions to these systems.


Increase understanding of the factors associated with medication warning acceptance.


Retrospective study of all single‐medication warnings generated in a CPOE system from October 2009 through April 2010.


Academic medical center.


All adult non‐intensive care unit patients hospitalized during the study period.


A total of 40,391 medication orders generated a single‐medication warning during the 7‐month study period. Of these warnings, 47% were duplicate warnings, 47% interaction warnings, 6% allergy warnings, 0.1% adverse reaction warnings, and 9.8% were repeated for the same patient, medication, and provider. Only 4% of warnings were accepted. In multivariate analysis, warning acceptance was positively associated with male patient gender, admission to a service other than internal medicine, caregiver status other than resident, parenteral medications, lower numbers of warnings, and allergy or adverse reaction warning types. Older patient age, longer length of stay, inclusion on the Institute for Safe Medication Practice's List of High Alert Medications, and interaction warning type were all negatively associated with warning acceptance.


Medication warnings are rarely accepted. Acceptance is more likely when the warning is infrequently encountered, and least likely when it is potentially most important. Warning systems should be redesigned to increase their effectiveness for the sickest patients, the least experienced physicians, and the medications with the greatest potential to cause harm. Journal of Hospital Medicine 2015;10:19–25. © 2015 Society of Hospital Medicine

© 2015 Society of Hospital Medicine

Many computerized provider order entry (CPOE) systems suffer from having too much of a good thing. Few would question the beneficial effect of CPOE on medication order clarity, completeness, and transmission.[1, 2] When mechanisms for basic decision support have been added, however, such as allergy, interaction, and duplicate warnings, reductions in medication errors and adverse events have not been consistently achieved.[3, 4, 5, 6, 7] This is likely due in part to the fact that ordering providers override medication warnings at staggeringly high rates.[8, 9] Clinicians acknowledge that they are ignoring potentially valuable warnings,[10, 11] but suffer from alert fatigue due to the sheer number of messages, many of them judged by clinicians to be of low‐value.[11, 12]

Redesign of medication alert systems to increase their signal‐to‐noise ratio is badly needed,[13, 14, 15, 16] and will need to consider the clinical significance of alerts, their presentation, and context‐specific factors that potentially contribute to warning effectiveness.[17, 18, 19] Relatively few studies, however, have objectively looked at context factors such as the characteristics of providers, patients, medications, and warnings that are associated with provider responses to warnings,[9, 20, 21, 22, 23, 24, 25] and only 2 have studied how warning acceptance is associated with medication risk.[18, 26] We wished to explore these factors further. Warning acceptance has been shown to be higher, at least in the outpatient setting, when orders are entered by low‐volume prescribers for infrequently encountered warnings,[24] and there is some evidence that patients receive higher‐quality care during the day.[27] Significant attention has been placed in recent years on inappropriate prescribing in older patients,[28] and on creating a culture of safety in healthcare.[29] We therefore hypothesized that our providers would be more cautious, and medication warning acceptance rates would be higher, when orders were entered for patients who were older or with more complex medical problems, when they were entered during the day by caregivers who entered few orders, when the medications ordered were potentially associated with greater risk, and when the warnings themselves were infrequently encountered.


Setting and Caregivers

Johns Hopkins Bayview Medical Center (JHBMC) is a 400‐bed academic medical center serving southeastern Baltimore, Maryland. Prescribing caregivers include residents and fellows who rotate to both JHBMC and Johns Hopkins Hospital, internal medicine hospitalists, other attending physicians (including teaching attendings for all departments, and hospitalists and clinical associates for departments other than internal medicine), and nurse practitioners and physician assistants from most JHBMC departments. Nearly 100% of patients on the surgery, obstetrics/gynecology, neurology, psychiatry, and chemical dependence services are hospitalized on units dedicated to their respective specialty, and the same is true for approximately 95% of medicine patients.

Order Entry

JHBMC began using a client‐server order entry system by MEDITECH (Westwood, MA) in July 2003. Provider order entry was phased in beginning in October 2003 and completed by the end of 2004. MEDITECH version 5.64 was being used during the study period. Medications may generate duplicate, interaction, allergy, adverse reaction, and dose warnings during a patient ordering session each time they are ordered. Duplicate warnings are generated when the same medication (no matter what route) is ordered that is either on their active medication list, was on the list in the preceding 24 hours, or that is being ordered simultaneously. A drug‐interaction database licensed from First DataBank (South San Francisco, CA) is utilized, and updated monthly, which classifies potential drug‐drug interactions as contraindicated, severe, intermediate, and mild. Those classified as contraindicated by First DataBank are included in the severe category in MEDITECH 5.64. During the study period, JHBMC's version of MEDITECH was configured so that providers were warned of potential severe and intermediate drug‐drug interactions, but not mild. No other customizations had been made. Patients' histories of allergies and other adverse responses to medications can be entered by any credentialed staff member. They are maintained together in an allergies section of the electronic medical record, but are identified as either allergy or adverse reactions at the time they are entered, and each generates its own warnings.

When more than 1 duplicate, interaction, allergy, or adverse reaction warning is generated for a particular medication, all appear listed on a single screen in identical fonts. No visual distinction is made between severe and intermediate drug‐drug interactions; for these, the category of medication ordered is followed by the category of the medication for which there is a potential interaction. A details button can be selected to learn specifically which medications are involved and the severity and nature of the potential interactions identified. In response to the warnings, providers can choose to either override them, erase the order, or replace the order by clicking 1 of 3 buttons at the bottom of the screen. Warnings are not repeated unless the medication is reordered for that patient. Dose warnings appear on a subsequent screen and are not addressed in this article.

Nurses are discouraged from entering verbal orders but do have the capacity to do so, at which time they encounter and must respond to the standard medication warnings, if any. Medical students are able to enter orders, at which time they also encounter and must respond to the standard medication warnings; their orders must then be cosigned by a licensed provider before they can be processed. Warnings encountered by nurses and medical students are not repeated at the time of cosignature by a licensed provider.

Data Collection

We collected data regarding all medication orders placed in our CPOE system from October 1, 2009 to April 20, 2010 for all adult patients. Intensive care unit (ICU) patients were excluded, in anticipation of a separate analysis. Hospitalizations under observation were also excluded. We then ran a report showing all medications that generated any number of warnings of any type (duplicate, interaction, allergy, or adverse reaction) for the same population. Warnings generated during readmissions that occurred at any point during the study period (ranging from 1 to 21 times) were excluded, because these patients likely had many, if not all, of the same medications ordered during their readmissions as during their initial hospitalization, which would unduly influence the analysis if retained.

There was wide variation in the number of warnings generated per medication and in the number of each warning type per medication that generated multiple warnings. Therefore, for ease of analysis and to ensure that we could accurately determine varying response to each individual warning type, we thereafter focused on the medications that generated single warnings during the study period. For each single warning we obtained patient name, account number, event date and time, hospital unit at the time of the event, ordered medication, ordering staff member, warning type, and staff member response to the warning (eg, override warning or erase order [accept the warning]). The response replace was used very infrequently, and therefore warnings that resulted in this response were excluded. Medications available in more than 1 form included the route of administration in their name, and from this they were categorized as parenteral or nonparenteral. All nonparenteral or parenteral forms of a given medication were grouped together as 1 medication (eg, morphine sustained release and morphine elixir were classified as a single‐medication, nonparenteral morphine). Medications were further categorized according to whether or not they were on the Institute for Safe Medication Practice (ISMP) List of High‐Alert Medications.[30]

The study was approved by the Johns Hopkins Institutional Review Board.


We collected descriptive data about patients and providers. Age and length of stay (LOS) at the time of the event were determined based on the patients' admit date and date of birth, and grouped into quartiles. Hospital units were grouped according to which service or services they primarily served. Medications were grouped into quartiles according to the total number of warnings they generated during the study period. Warnings were dichotomously categorized according to whether they were overridden or accepted. Unpaired t tests were used to compare continuous variables for the 2 groups, and [2] tests were used to compare categorical variables. A multivariate logistic regression was then performed, using variables with a P value of <0.10 in the univariate analysis, to control for confounders and identify independent predictors of medication warning acceptance. All analyses were performed using Intercooled Stata 12 (StataCorp, College Station, TX).


A total of 259,656 medication orders were placed for adult non‐ICU patients during the 7‐month study period. Of those orders, 45,835 generated some number of medication warnings.[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20] The median number of warnings per patient was 4 (interquartile range [IQR]=28; mean=5.9, standard deviation [SD]=6.2), with a range from 1 to 84. The median number of warnings generated per provider during the study period was 36 (IQR=6106, mean=87.4, SD=133.7), with a range of 1 to 1096.

There were 40,391 orders placed for 454 medications for adult non‐ICU patients, which generated a single‐medication warning (excluding those with the response replace, which was used 20 times) during the 7‐month study period. Data regarding the patients and providers associated with the orders generating single warnings are shown in Table 1. Most patients were on medicine units, and most orders were entered by residents. Patients' LOS at the time the orders were placed ranged from 0 to 118 days (median=1, IQR=04; mean=4.0, SD=7.2). The median number of single warnings per patient was 4 (IQR=28; mean=6.1, SD=6.5), with a range from 1 to 84. The median number of single warnings generated per provider during the study period was 15 (IQR=373; mean=61.7, SD=109.6), with a range of 1 to 1057.

Patient and Provider Features
No. (%)
  • NOTE: Abbreviations: GYN, gynecology; IM, internal medicine; Neuro/psych/chem dep, neurology/psychiatry/chemical dependence; NP, nurse practitioner; OB, obstetrics; PA, physician assistant.

  • Hospital unit at the time of order entry.

  • Total is >100% due to rounding.

Patients (N=6,646)
1545 years2,048 (31%)
4657 years1,610 (24%)
5872 years1,520 (23%)
73104 years1,468 (22%)
Male2,934 (44%)
Hospital unita
Medicine2,992 (45%)
Surgery1,836 (28%)
Neuro/psych/chem dep1,337 (20%)
OB/GYN481 (7%)
Caregivers (N=655)
Resident248 (38%)b
Nurse154 (24%)
Attending or other97 (15%)
NP/PA69 (11%)
IM hospitalist31 (5%)
Fellow27 (4%)
Medical student23 (4%)
Pharmacist6 (1%)

Patient and caregiver characteristics for the medication orders that generated single warnings are shown in Table 2. The majority of medications were nonparenteral and not on the ISMP list (Table 3). Most warnings generated were either duplicate (47%) or interaction warnings (47%). Warnings of a particular type were repeated 14.5% of the time for a particular medication and patient (from 2 to 24 times, median=2, IQR=22, mean=2.7, SD=1.4), and 9.8% of the time for a particular caregiver, medication, and patient (from 2 to 18 times, median=2, IQR=22, mean=2.4, SD=1.1).

Characteristics of Patients, Caregivers, Orders, Medications, and Warnings for Medication Orders Generating Single Warnings, and Association With Warning Acceptance
VariableNo. of Warnings (%)aNo. of Warnings Accepted (%)aP
  • NOTE: Abbreviations: GYN, gynecology; IM, internal medicine; ISMP, Institute for Safe Medication Practices; Neuro/psych/chem dep, neurology/psychiatry/chemical dependence; NP, nurse practitioner; OB, obstetrics; PA, physician assistant.

  • Totals may not equal 100% due to rounding.

  • Total number of medications is >454 because many medications generated more than 1 warning type.

Patient age
1545 years10,881 (27)602 (5.5%)<0.001
4657 years9,733 (24)382 (3.9%)
5872 years10,000 (25)308 (3.1%)
73104 years9,777 (24)262 (2.7%)
Patient gender
Female23,395 (58)866 (3.7%)0.074
Male16,996 (42)688 (4.1%)
Patient length of stay
<1 day10,721 (27)660 (6.2%)<0.001
1 day10,854 (27)385 (3.5%)
24 days10,424 (26)277 (2.7%)
5118 days8,392 (21)232 (2.8%)
Patient hospital unit
Medicine20,057 (50)519 (2.6%)<0.001
Surgery10,274 (25)477 (4.6%)
Neuro/psych/chem dep8,279 (21)417 (5.0%)
OB/GYN1,781 (4)141 (7.9%)
Ordering caregiver
Resident22,523 (56)700 (3.1%)<0.001
NP/PA7,534 (19)369 (4.9%)
IM hospitalist5,048 (13)155 (3.1%)
Attending3225 (8)219 (6.8%)
Fellow910 (2)34 (3.7%)
Nurse865 (2)58 (6.7%)
Medical student265 (<1)17 (6.4%)
Pharmacist21 (<1)2 (9.5%)
Day ordered
Weekday31,499 (78%)1276 (4.1%)<0.001
Weekend8,892 (22%)278 (3.1%)
Time ordered
000005594,231 (11%)117 (2.8%)<0.001
0600115911,696 (29%)348 (3.0%)
1200175915,879 (39%)722 (4.6%)
180023598,585 (21%)367 (4.3%)
Administration route (no. of meds)
Nonparenteral (339)27,086 (67%)956 (3.5%)<0.001
Parenteral (115)13,305 (33%)598 (4.5%)
ISMP List of High‐Alert Medications status (no. of meds)[30]
Not on ISMP list (394)27,503 (68%)1251 (4.5%)<0.001
On ISMP list (60)12,888 (32%)303 (2.4%)
No. of warnings per med (no. of meds)
11062133 (7)9,869 (24%)191 (1.9%)<0.001
4681034 (13)10,014 (25%)331 (3.3%)
170444 (40)10,182 (25%)314 (3.1%)
1169 (394)10,326 (26%)718 (7.0%)
Warning type (no. of meds)b
Duplicate (369)19,083 (47%)1041 (5.5%)<0.001
Interaction (315)18,894 (47%)254 (1.3%)
Allergy (138)2,371 (6%)243 (10.0%)
Adverse reaction (14)43 (0.1%)16 (37%)
Multivariate Analysis of Factors Associated With Acceptance of Medication Warnings
VariableAdjusted OR95% CI
  • NOTE: Abbreviations: CI, confidence interval; GYN, gynecology; IM, internal medicine; ISMP, Institute for Safe Medication Practices; Neuro/psych/chem dep, neurology/psychiatry/chemical dependence; NP, nurse practitioner; OB, obstetrics; OR, odds ratio; PA, physician assistant.

  • Day ordered and time of order entry were included but were not significant in the multivariate model.

Patient age
1545 years1.00Reference
4657 years0.890.771.02
5872 years0.850.730.99
73104 years0.910.771.08
Patient gender
Patient length of stay
<1 day1.00Reference
1 day0.650.550.76
24 days0.490.420.58
5118 days0.490.410.58
Patient hospital unit
Neuro/psych/chem dep1.351.151.58
Ordering caregiver
IM hospitalist1.241.021.50
Medical student1.170.701.95
Medication factors
HighAlert Medication status (no. of meds)[30]
Not on ISMP list1.00Reference
On ISMP list0.370.320.43
No. of warnings per medication
Warning type
Adverse reaction9.244.5218.90

One thousand five hundred fifty‐four warnings were erased (ie, accepted by clinicians [4%]). In univariate analysis, only patient gender was not associated with warning acceptance. Patient age, LOS, hospital unit at the time of order entry, ordering caregiver type, day and time the medication was ordered, administration route, presence on the ISMP list, warning frequency, and warning type were all significantly associated with warning acceptance (Table 2).

Older patient age, longer LOS, presence of the medication on the ISMP list, and interaction warning type were all negatively associated with warning acceptance in multivariable analysis. Warning acceptance was positively associated with male patient gender, being on a service other than medicine, being a caregiver other than a resident, parenteral medications, lower warning frequency, and allergy or adverse reaction warning types (Table 3).

The 20 medications that generated the most single warnings are shown in Table 4. Medications on the ISMP list accounted for 8 of these top 20 medications. For most of them, duplicate and interaction warnings accounted for most of the warnings generated, except for parenteral hydromorphone, oral oxycodone, parenteral morphine, and oral hydromorphone, which each had more allergy than interaction warnings.

Top 20 Medications Generating Single Warnings and Warning Type Distribution for Each
MedicationISMP ListbNo. of WarningsDuplicate, No. (%)cInteraction, No. (%)cAllergy, No. (%)cAdverse Reaction, No. (%)c
  • NOTE: Abbreviations: ISMP, Institute for Safe Medication Practices.

  • Medications not noted as injectable should be presumed not parenteral.

  • SMP List of High‐Alert Medications.[30]

  • Total may not add up to 100% due to rounding.

Hydromorphone injectableYes2,1331,584 (74.3)127 (6.0)422 (19.8)
Metoprolol 1,432550 (38.4)870 (60.8)12 (0.8)
Aspirin 1,375212 (15.4)1,096 (79.7)67 (4.9)
OxycodoneYes1,360987 (72.6) 364 (26.8)9 (0.7)
Potassium chloride 1,296379 (29.2)917 (70.8)
Ondansetron injectable 1,1671,013 (86.8)153 (13.1)1 (0.1)
Aspart insulin injectableYes1,106643 (58.1)463 (41.9)
WarfarinYes1,034298 (28.8)736 (71.2)
Heparin injectableYes1,030205 (19.9)816 (79.2)9 (0.3)
Furosemide injectable 980438 (45.0)542 (55.3)
Lisinopril 926225 (24.3)698 (75.4)3 (0.3)
Acetaminophen 860686 (79.8)118 (13.7)54 (6.3)2 (0.2)
Morphine injectableYes804467 (58.1)100 (12.4)233 (29.0)4 (0.5)
Diazepam 786731 (93.0)41 (5.2)14 (1.8)
Glargine insulin injectableYes746268 (35.9)478 (64.1)
Ibuprofen 713125 (17.5)529 (74.2)54 (7.6)5 (0.7)
HydromorphoneYes594372 (62.6)31 (5.2)187 (31.5)4 (0.7)
Furosemide 586273 (46.6)312 (53.2)1 (0.2)
Ketorolac injectable 48739 (8.0)423 (86.9)23 (4.7)2 (0.4)
Prednisone 468166 (35.5)297 (63.5)5 (1.1)


Medication warnings in our study were frequently overridden, particularly when encountered by residents, for patients with a long LOS and on the internal medicine service, and for medications generating the most warnings and on the ISMP list. Disturbingly, this means that potentially important warnings for medications with the highest potential for causing harm, for possibly the sickest and most complex patients, were those that were most often ignored by young physicians in training who should have had the most to gain from them. Of course, this is not entirely surprising. Despite our hope that a culture of safety would influence young physicians' actions when caring for these patients and prescribing these medications, these patients and medications are those for whom the most warnings are generated, and these physicians are the ones entering the most orders. Only 13% of the medications studied were on the ISMP list, but they generated 32% of the warnings. We controlled for number of warnings and ISMP list status, but not for warning validity. Most likely, high‐risk medications have been set up with more warnings, many of them of lower quality, in an errant but well‐intentioned effort to make them safer. If developers of CPOE systems want to gain serious traction in using decision support to promote prescribing safe medications, they must take substantial action to increase attention to important warnings and decrease the number of clinically insignificant, low‐value warnings encountered by active caregivers on a daily basis.

Only 2 prior studies, both by Seidling et al., have specifically looked at provider response to warnings for high risk medications. Interaction warnings were rarely accepted in 1,[18] as in our study; however, in contrast to our findings, warning acceptance in both studies was higher for drugs with dose‐dependent toxicity.[18, 26] The effect of physician experience on warning acceptance has been addressed in 2 prior studies. In Weingart et al., residents were more likely than staff physicians to erase medication orders when presented with allergy and interaction warnings in a primary care setting.[20] Long et al. found that physicians younger than 40 years were less likely than older physicians to accept duplicate warnings, but those who had been at the study hospital for a longer period of time were more likely to accept them.[23] The influence of patient LOS and service on warning acceptance has not previously been described. Further study is needed looking at each of these factors.

Individual hospitals tend to avoid making modifications to order entry warning systems, because monitoring and maintaining these changes is labor intensive. Some institutions may make the decision to turn off certain categories of alerts, such as intermediate interaction warnings, to minimize the noise their providers encounter. There are even tools for disabling individual alerts or groups of alerts, such as that available for purchase from our interaction database vendor.[31] However, institutions may fear litigation should an adverse event be attributed to a disabled warning.[15, 16] Clearly, a comprehensive, health system‐wide approach is warranted.[13, 15] To date, published efforts describing ways to improve the effectiveness of medication warning systems have focused on either heightening the clinical significance of alerts[14, 21, 22, 32, 33, 34, 35, 36] or altering their presentation and how providers experience them.[21, 36, 37, 38, 39, 40, 41, 42, 43] The single medication warnings our providers receive are all presented in an identical font, and presumably response to each would be different if they were better distinguished from each other. We also found that a small but significant number of warnings were repeated for a given patient and even a given provider. If the providers knew they would only be presented with warnings the first time they occurred for a given patient and medication, they might be more attuned to the remaining warnings. Previous studies describe context‐specific decision support for medication ordering[44, 45, 46]; however, only 1 has described the use of patient context factors to modify when or how warnings are presented to providers.[47] None have described tailoring allergy, duplicate, and interaction warnings according to medication or provider types. If further study confirms our findings, modulating basic warning systems according to severity of illness, provider experience, and medication risk could powerfully increase their effectiveness. Of course, this would be extremely challenging to achieve, and is likely outside the capabilities of most, if not all, CPOE systems, at least for now.

Our study has some limitations. First, it was limited to medications that generated a single warning. We did this for ease of analysis and so that we could ensure understanding of provider response to each warning type without bias from simultaneously occurring warnings; however, caregiver response to multiple warnings appearing simultaneously for a particular medication order might be quite different. Second, we did not include any assessment of the number of medications ordered by each provider type or for each patient, either of which could significantly affect provider response to warnings. Third, as previously noted, we did not include any assessment of the validity of the warnings, beyond the 4 main categories described, which could also significantly affect provider response. However, it should be noted that although the validity of interaction warnings varies significantly from 1 medication to another, the validity of duplicate, allergy, and adverse reaction warnings in the described system are essentially the same for all medications. Fourth, it is possible that providers did modify or even erase their orders even after selecting override in response to the warning; it is also possible that providers reentered the same order after choosing erase. Unfortunately auditing for actions such as these would be extremely laborious. Finally, the study was conducted at a single medical center using a single order‐entry system. The system in use at our medical center is in use at one‐third of the 6000 hospitals in the United States, though certainly not all are using our version. Even if a hospital was using the same CPOE version and interaction database as our institution, variations in patient population and local decisions modifying how the database interacts with the warning presentation system might affect reproducibility at that institution.

Commonly encountered medication warnings are overridden at extremely high rates, and in our study this was particularly so for medications on the ISMP list, when ordered by physicians in training. Warnings of little clinical significance must be identified and eliminated, the most important warnings need to be visually distinct to increase user attention, and further research should be done into the patient, provider, setting, and medication factors that affect user responses to warnings, so that they may be customized accordingly and their significance increased. Doing so will enable us to reap the maximum possible potential from our CPOE systems, and increase the CPOE's power to protect our most vulnerable patients from our most dangerous medications, particularly when cared for by our most inexperienced physicians.


The authors thank, in particular, Scott Carey, Research Informatics Manager, for assistance with data collection. Additional thanks go to Olga Sherman and Kathleen Ancinich for assistance with data collection and management.

Disclosures: This research was supported in part by the Johns Hopkins Institute for Clinical and Translational Research. All listed authors contributed substantially to the study conception and design, analysis and interpretation of data, drafting the article or revising it critically for important intellectual content, and final approval of the version to be published. No one who fulfills these criteria has been excluded from authorship. This research received no specific grant from any funding agency in the public, commercial, or not‐for‐profit sectors. The authors have no competing interests to declare.


Online-Only Materials

PDF icon Supplementary Information (1)812.96 KB
   Comments ()