Related Links:

University of Pennsylvania Health System

University of Pennsylvania's School of Medicine

Clinical Effectiveness & Quality Improvement (Intranet Access Only)

Center for Evidence-based Practice
(Intranet Access Only)



The mission of CEP is to support healthcare quality and safety at the University of Pennsylvania Health System (UPHS) through the practice of evidence-based medicine. To that end, the Center summarizes scientific evidence for UPHS decision making about high impact drugs, devices and processes of care, and is charged with building evidence-based collaborative enterprises with outside organizations.



Penn Study Finds Important Gaps in Evidence for Best Methods for Cleaning Hospital Rooms to Prevent Healthcare-associated Infections

Additional research needed to help lower risk for patients

August 10, 2015 – Tray tables, bed rails, light switches, and toilets: All are common vectors for swapping germs between patients and health care workers. While a new systematic overview in this week’s Annals of Internal Medicine points to several promising cleaning tactics of these “high-touch surfaces,” there’s a lack of evidence as to which is the most effective at reducing healthcare-associated infections (HAIs). Few studies measured patient outcomes or focused on newer technologies, and even less compared cleaning tactics against one another — important gaps to fill as the U.S. health care system works to reduce the 75,000 HAI-related deaths that occur annually.

The systematic overview was led by Craig A. Umscheid, MD, MSCE, an assistant professor of Medicine and Epidemiology in the Perelman School of Medicine at the University of Pennsylvania, and Senior Associate Director at the ECRI Institute-Penn Medicine Agency for Healthcare Research and Quality (AHRQ)-funded Evidence-Based Practice Center (EPC), Jennifer Han, MD, MSCE, an assistant professor of Medicine and Epidemiology, and Brian Leas, MS, MA, and Nancy Sullivan, research analysts in the ECRI-Penn AHRQ EPC, and revealed major gaps in existing evidence for the best practices for cleaning hospital room surfaces to prevent HAIs, including Clostridium difficile, MRSA, andVRE.  

“The cleaning of hard surfaces in hospital rooms is critical for reducing healthcare-associated infections,” said Han, the study’s lead author. “We found that the research to date does provide a good overall picture of the before and after results of particular cleaning agents and approaches to monitoring cleanliness. Researchers now need to take the next step and compare the various ways of cleaning these surfaces and monitoring their cleanliness in order to determine which are the most effective in driving down the rate of hospital-acquired infections.”

While studies examining HAIs have increased over the last 15 years, infections acquired in the hospital remain a leading cause of death and morbidity. In 2011, there were over 721,000 HAIs in the U.S., according to the most recently available data from the U.S. Centers for Disease Control and Prevention (CDC). What’s more, many experts believe that only 50 percent of surfaces are typically disinfected during cleaning of a patient’s room. 

Examining 80 studies published between 1998 and 2014, the research team found that comparative effectiveness studies were uncommon. Such studies would have directly compared different ways of cleaning, disinfecting, and monitoring the cleanliness of hard surfaces in order to determine which were most effective. There were also relatively few studies that focused on measuring outcomes of most interest to patients, such as changes in HAI rates or the presence of pathogens on patients. Only five of the studies were randomized controlled trials.

Instead, the existing studies were largely before and after experiments, comparing the magnitude of surface contamination after cleaning with a particular agent to the magnitude of contamination before cleaning.  Over 65 percent of the studies assessed surface contamination, such as bacterial burden and colony counts, as the primary outcome.  Less than 35 percent reported on patient-centered outcomes, such as HAI rates or acquisition of a specific organism in the body, known as colonization.

The ECRI-Penn EPC team looked at three broad categories of evidence: 1) which agents and methods were used to clean hard surfaces; 2) what approaches were available to monitor the effectiveness of cleaning; and 3) what systems-level factors are needed for cleaning and monitoring to be successful. In addition to its literature review, the researchers interviewed a number of national experts.

“Our goal was to provide a comprehensive review of evidence in all three domains,” said Umscheid, the study’s senior author. “While there is a clear need for more patient-centered and comparative effectiveness research, the findings that do exist provide a good place to start in terms of a hospital or health care entity seeking information on ways to mitigate healthcare-associated infections.”

Among its findings, the EPC team identified several studies showing that rates of C. difficile (or “C. diff”),the most common cause of hospital-acquired gastrointestinal infections, fell with the use of bleach-based disinfectants but that a chlorine dioxide-based product was ineffective in reducing C. diff contamination and infection rates. Patients taking antibiotics are at special risk of becoming infected with C. diff because antibiotics can disrupt the normal bacteria of the bowel. According to the CDC, C. diff caused almost half a million infections in the United States in 2011.

In addition, six studies integrating various wipes moistened with hydrogen peroxide and other chemicals into preventive strategies reported positive outcomes, including sustained reductions in HAIs. Seventeen studies implementing “no-touch” modalities to clean hard surfaces – such as devices that emit ultraviolet light or hydrogen peroxide vapor – reported positive findings, with three specifically demonstrating reductions in infection rates. Seven of eight studies evaluating enhanced coatings on hospital room surfaces, such as copper-coated bed rails, reported positive findings. Surfaces made of solid, copper-based metals or alloys continuously kill bacteria that cause infections.

The EPC team also highlighted several priority areas for future research, based on their review of the evidence and interviews with leading experts. Questions to emphasize in future studies include: what surfaces present the greatest infection risk to patients, what benchmarks should be established for measuring cleanliness, and what factors affect the quality of routine disinfection practices? Further research is also needed on recently emerging disinfection strategies.

“In addition to expanding the use of comparative effectiveness research and placing greater emphasis on patient-centered outcomes, future research should investigate the effectiveness of a number of promising new technologies and approaches,” said Han. “These include self-disinfecting coatings and increasingly used surface markers for monitoring the presence of pathogens. Other challenges include identifying high-touch surfaces that confer the greatest risk of pathogen transmission and developing standard thresholds for defining cleanliness.”

Other co-authors include David A. Pegues, MD, from Penn, and Janice L. Kaczmarek, MS from ECRI Institute.


Penn Study Shows Computer-Assisted Diagnosis Tool Helps Physicians Assess Skin Conditions Without Aid from Dermatologists: Tool Harnesses Photo Database to Improve Diagnostic Accuracy

April 22, 2015 – In the first major study to examine the use of a computer-assisted, photo-driven differential diagnosis generator for skin conditions, researchers at the Perelman School of Medicine at the University of Pennsylvania found physicians routinely used the tool, without an increase in calling for inpatient dermatology consultations.  The software diagnostic tool, VisualDx, aids in diagnosing  dermatologic conditions by allowing physicians to enter information such as the type and location of a rash, and associated symptoms such as pain or itching, and then generating a range of possible diagnoses accompanied by photographs. This enables users to rapidly compare the rash of an individual patient to a database of more than 1,300 pediatric and adult skin conditions represented by nearly 30,000 images, with the aim of improving diagnostic accuracy reducing misdiagnosis-related harm.

The study, published early online in Diagnosis (, found that the rate of dermatologic consultations at the Hospital of the University of Pennsylvania, the flagship hospital of Penn Medicine, did not change for the 18 months after VisualDx was introduced compared to the 12 months before it was introduced.

“This is important because these tools by design suggest numerous potential diagnoses, which could result in an increase in unnecessary testing and specialty consultation, and associated costs and harms, particularly in the hands of less experienced clinicians,” says the study’s senior author, Craig A. Umscheid, MD, MSCE, assistant professor of Medicine and Epidemiology and director of the Penn Medicine Center for Evidence-based Practice. “Conversely, if there were a significant reduction in dermatologic consultations, it would have suggested that general internists, emergency room physicians, family doctors and pediatricians, all of whom by definition are not specialists in dermatology, may have relied on the tool to make dermatologic diagnoses, rather than consulting a dermatologist for help.”

The Penn team examined the use of VisualDx throughout the University of Pennsylvania Health System by month for the 18 months following its introduction in September 2012. Researchers found a median of 474 unique monthly VisualDx sessions by users, a rate that continued throughout the time period under study. The researchers found that, overall, VisualDx was accessed through mobile devices (35 percent); inpatient (34 percent), outpatient (11 percent), and emergency department (one percent) electronic health records; and via searches in UpToDate, a point-of-care evidence summary resource (19 percent).

“Technology like this has great promise, but it can’t help patients unless it’s actually used,” says Umscheid. “Previous studies have been primarily simulations, where researchers take variables from case studies and input them into the software to retroactively assess the diagnostic accuracy of the software. However, our purpose was to determine if a differential diagnosis generator like VisualDx would actually be used by providers if implemented in a hospital, and we found that it is — most often on mobile devices and by inpatient providers.”

Differential diagnosis generators have been suggested as a potential way of reducing misdiagnosis, which is estimated to result in 40,000 to 80,000 deaths in U.S. hospitals annually. “The technology can help users overcome cognitive shortcomings such as availability bias, in which providers diagnose patients with conditions they’ve recently seen or can easily recall, rather than those conditions that are most likely to occur. It can also reduce fund of knowledge deficiencies by directing users to diagnoses they might not have otherwise considered,” commented the study’s lead author John Barbieri, MD, MBA, a 2014 graduate of the Perelman School of Medicine and The Wharton School.

The other Penn co-author was Benjamin French, PhD. The study was supported in part by the National Institutes of Health (UL1RR024134 and UL1TR000003).The ECRI Institute-Penn Medicine Evidence-based Practice Center (EPC) was recently redesignated by the Agency for Healthcare Research and Quality (AHRQ) to serve as an EPC through 2019. ECRI Institute has maintained this designation since the inception of the EPC program in 1997. The ECRI Institute-Penn Medicine EPC was established in 2012.


AHRQ Redesignates ECRI Institute-Penn Medicine as Evidence-based Practice Center
New 5-year contract supports the Agency for Healthcare Research and Quality in its efforts to disseminate scientific research of important healthcare topics

January 30, 2015  — The ECRI Institute-Penn Medicine Evidence-based Practice Center (EPC) was recently redesignated by the Agency for Healthcare Research and Quality (AHRQ) to serve as an EPC through 2019. ECRI Institute has maintained this designation since the inception of the EPC program in 1997. The ECRI Institute-Penn Medicine EPC was established in 2012.

Under the new contract, known as EPC V, the ECRI Institute-Penn Medicine EPC will work with clinicians, policymakers, patients, consumers, and other scientific partners to develop topics and questions for systematic review of the literature, and to promote the synthesis, translation, and dissemination of research evidence. The EPC has reported on a variety of topics, many in the area of cancer, genetic testing, and imaging tests (access free reports).

As part of the new contract, all 13 Evidence-based Practice Centers will be eligible to bid on work to support the U.S. Preventive Services Task Force (USPSTF), an independent, volunteer panel of experts in prevention and evidence-based medicine dedicated to improving the health of Americans.

“Since we began working together, we’ve been able to utilize ECRI Institute’s independent, evidence-based research and Penn Medicine’s clinical and academic leadership to advance research to improve patient care,” says Karen Schoelles, MD, SM, director of the ECRI Institute-Penn Medicine EPC. “This new contract inspires us to build on our previous collaborations and break into new research territory.

”“We look forward to continuing our efforts to transform patient care delivery as we work alongside ECRI Institute and AHRQ,” says Craig A. Umscheid, MD, MSCE, director of the Penn Medicine Center for Evidence-based Practice and senior associate director of the ECRI Institute-Penn Medicine EPC.

The EPC will work with Insight Policy Research, a small woman-owned disadvantaged business.

To learn more about ECRI Institute’s work in evaluating health technologies through the Evidence-based Practice Center, membership in the Health Technology Assessment Information Service, and customized consultation and support for clinical practice guideline development, e-mail, or call (610) 825-6000, ext. 5519.


Penn Medicine Experts Urge Expansion of Federal Incentives to Improve Patient Care Quality and Safety
Focusing on structural changes in addition to individual clinical outcomes would bring greater gains

January 13, 2015 — In an invited commentary published online by JAMA Internal Medicine, Patrick J. Brennan, MD, chief medical officer of the University of Pennsylvania Health System, and Craig A. Umscheid, MD, MSCE, director of Penn’s Center for Evidence-based Practice, recommend that the Centers for Medicare and Medicaid Services shift its use of financial incentives from an exclusive focus on preventing specific clinical outcomes toward encouraging the creation of hospital-wide structures that promote the translation of research evidence into clinical practice to improve the quality and safety of patient care.

In 2008, Medicare implemented the Hospital-Acquired Conditions Initiative, a policy denying incremental payment for eight complications of hospital care, also known as “never events.” Another study published in JAMA Internal Medicine led by Teresa Waters, PhD, of the University of Tennessee Health Science Center, found that after the new federal policy was introduced, central line-associated bloodstream infections fell by 11 percent and catheter-associated urinary tract infections fell by ten percent. But there was no significant reduction in hospital-acquired pressure ulcers and serious inpatient falls.

The study authors suggested that the difference in the policy’s effects on infections as compared to falls and pressure ulcers were the result of the evidence available at the time to guide hospitals to improve these outcomes. 

But the Penn authors disagreed.  Instead, they suggested that the difference in the policy’s effect had more to do with the challenge of improving complex quality outcomes such as pressure ulcers and inpatient falls, when compared to the easier-to-improve outcome of infections.  Unlike infection prevention efforts, which often target care providers and involve changes to a limited set of procedures such as catheter insertion and occur in closely monitored settings such as intensive care units, efforts at reducing falls and pressure ulcers are much more complex.  They require multidisciplinary, patient-centered collaboration across the entire spectrum of care, not just critical care. 

“Incentives are indeed important,” says Brennan. “But they must be the right incentives. There was good-quality evidence available to underpin fall and pressure ulcer prevention efforts when the CMS initiative was launched. Yet neither of these measures has significantly improved. This may be because it’s simply not possible to reduce falls and pressure ulcers by taking a few clear-cut steps, as in the case of reducing infections. Instead, broader and more comprehensive structural changes within hospitals should be considered. Such efforts would reduce an even larger number of never events than by simply tackling them one at a time.”

Hospitals should aim to bridge what’s called the “knowing-doing gap,” Umscheid says. “That’s the disconnect between what we know works based on the best available evidence and what we actually practice. To bridge the gap, we are not advocating abandoning incentives that focus on individual clinical outcomes. Instead we are urging federal regulators to also encourage institution-wide structural changes. We believe these changes can create the necessary conditions for solving difficult problems such as falls and pressure ulcers -- and others as well.”

The structural changes suggested by Brennan and Umscheid are the fabric of Penn Medicine’s successful quality improvement program. They include:

  • establishing unit-based leadership, where physicians, nurses, and quality administrators work together on their local wards or clinics to ensure that quality standards are met  
  • creating data stores that can provide local evidence for  which quality initiatives are effective
  • investing in staff education on evidence-based quality improvement, including in-house learning academies
  • opening local evidence-based practice centers, where experts can identify and adapt national evidence and best practices to address local problems identified by staff
“We believe that incentivizing these structures, in concert with more focused incentives on specific clinical outcomes, together offer the best chance of improving patient care quality and safety in a variety of situations,” says Brennan. 


Penn Medicine's "Sepsis Sniffer" Generates Faster Sepsis Care and Suggests Reduced Mortality: Early identification and intervention lead to better results for hospital inpatients

October 9, 2014  — An automated early warning and response system for sepsis developed by Penn Medicine experts has resulted in a marked increase in sepsis identification and care, transfer to the ICU, and an indication of fewer deaths due to sepsis. A study assessing the tool is published online in the Journal of Hospital Medicine.

Sepsis is a potentially life-threatening complication of an infection; it can severely impair the body’s organs, causing them to fail. There are as many as three million cases of severe sepsis and 750,000 resulting deaths in the United States annually. Early detection and treatment, typically with antibiotics and intravenous fluids, is critical for survival.

The Penn prediction tool, dubbed the “sepsis sniffer,” uses laboratory and vital-sign data (such as body temperature, heart rate, and blood pressure) in the electronic health record of hospital inpatients to identify those at risk for sepsis. When certain data thresholds are detected, the system automatically sends an electronic communication to physicians, nurses, and other members of a rapid response team who quickly perform a bedside evaluation and take action to stabilize or transfer the patient to the intensive care unit if warranted.

The study developed the prediction tool using 4,575 patients admitted to the University of Pennsylvania Health System (UPHS) in October 2011.  The study then validated the tool during a pre-implementation period from June to September 2012, when data on admitted patients was evaluated and alerts triggered in a database, but no notifications were sent to providers on the ground.  Outcomes in that control period were then compared to a post-implementation period from June to September 2013.  The total number of patients included in the pre and post periods was 31,093.

In both the pre- and post-implementation periods, four percent of patient visits triggered the alert. Analysis revealed 90 percent of those patients received bedside evaluations by the care team within 30 minutes of the alert being issued. In addition, the researchers found that the tool resulted in:

• A two to three-fold increase in orders for tests that could help identify the presence of sepsis

• A 1.5 to two-fold increase in the administration of antibiotics and intravenous fluids

• An increase of more than 50 percent in the proportion of patients quickly transferred to the ICU

• A 50 percent increase in documentation of sepsis in the patients’ electronic health record

The study found a lower death rate from sepsis and an increase in the number of patients successfully discharged home, although these findings did not reach statistical significance.

“Our study is the first we’re aware of that was implemented throughout a multihospital health system,” said lead author Craig A. Umscheid, MD, MSCE, director of Penn’s Center for Evidence-based Practice. “Previous studies that have examined the impact of sepsis prediction tools at other institutions have only taken place on a limited number of inpatient wards. The varied patient populations, clinical staffing, practice models, and practice cultures across our health system increases the generalizability of our findings to other health care settings.”

Umscheid also noted that the tool could help triage patients for suitability of ICU transfer. “By better identifying those with sepsis requiring advanced care, the tool can help screen out patients not needing the inevitably limited number of ICU beds.”

In addition to Umscheid, the other Penn co-authors are Joel Betesh, MD; Christine Vanzandbergen, PA, MPH; Asaf Hanish, MPH; Gordon Tait, BS; Mark E. Mikkelsen, MD, MSCE; Benjamin French, PhD; and Barry D. Fuchs, MD, MS.

Dr. Umscheid’s contribution to this project was supported in part by the National Institutes of Health, National Center for Advancing Translational Sciences, grant # UL1TR000003.



About Penn Medicine   Contact Us   Site Map   Privacy Statement   Legal Disclaimer   Terms of Use

Penn Medicine, Philadelphia, PA 800-789-PENN © 2015, The Trustees of the University of Pennsylvania