The mission of CEP is to support healthcare quality and safety at the University of Pennsylvania Health System (UPHS) through the practice of evidence-based medicine. To that end, the Center summarizes scientific evidence for UPHS decision making about high impact drugs, devices and processes of care, and is charged with building evidence-based collaborative enterprises with outside organizations.
Evidence-based Practice Model Can Help Hospitals Crunch Data to Deliver Best Care, Cut Costs, and Reduce Hospital Error
Penn Medicine Researchers Produce First Comprehensive Assessment of the Role of Evidence-Based Practice Centers in Decision-Making within a United States Hospital
November 4, 2015 – Amid continued pressures to minimize errors and cut costs, hospitals are continuing to scramble to find solutions to problems plaguing health systems nationwide.
A possible solution to many of those issues can be found in hospital evidence-based practice centers (EPCs), says a new Penn Medicine study suggesting EPCs can effectively inform decision-making in medical settings. In the first comprehensive assessment of the role of EPCS in decision-making in a United States hospital, study authors say EPCs can help clinical and administrative leaders understand problems and evaluate possible solutions by reviewing the available scientific literature when national guidelines are not available, thus filling a critical knowledge gap. By putting this published evidence into practice, these centers can improve quality, safety and value of medical care offered, and improve relationships between clinicians and administrators, all fostering a culture of evidence-based practice.
“Hospital EPCs are also uniquely equipped to identify and adapt national evidence-based guidelines and systematic reviews to benefit their local institutions,” says Craig A. Umscheid, MD, MSCE, an assistant professor of Medicine, director of Penn Medicine’s Center for Evidence-Based Practice (CEP), and senior author of the study, published online ahead of print last week in the Journal of Hospital Medicine. “By synthesizing this valuable data, EPCs can inform health care leaders on how to make decisions that improve care, cut costs, and achieve other positive changes throughout their operations.”
Established in 2006, Penn Medicine’s CEP is staffed by a hospitalist director, three research analysts, six physician and nurse liaisons, a health economist, a biostatistician, administrator, and librarians. To accomplish its mission of strengthening quality, safety, and value of care delivered at the Health System, the CEP performs rapid systematic reviews of published research, translates evidence into practice using computer-based interventions in electronic health records, and educates trainees, staff, and faculty on evidence-based decision making.
While the number of hospital EPCs has grown worldwide, there are few known EPCs doing this work in the U.S. Also, the limited research available on EPCs focuses on their evaluations of technology, neglecting to take a comprehensive look at the role hospital EPCs play in integrating evidence into clinical practice.
In this study, the researchers analyzed an internal database of evidence reviews performed by the University of Pennsylvania Health System’s CEP for internal requestors between July 2006 (when the CEP was established) and June 2014. The team then conducted an anonymous web survey of all of those who requested a report during the last four of the Center’s eight fiscal years.
The study found that nearly 250 reports were produced in the Center’s first eight years. Clinical departments, chief medical officers, and purchasing committees were the most common requestors of reports. Drugs and devices were the two most commonly reviewed types of technologies, but topics ranging from care processes, organizational systems and policies, and medical and surgical procedures were also commonly reviewed. Examples of reviews include evaluations of the effectiveness and safety of:
Drugs (for example, celecoxib for pain in joint replacement)
Devices (for example, automated systems to monitor hand hygiene of hospital staff)
Care processes (for example, how to best manage patients experiencing gastrointestinal bleeding)
Reports were completed within two to three months on average, and more than 10 percent of reports were used to develop computerized interventions in the electronic health record.
The survey data of 46 respondents found that 98 percent of report requestors said the scope of the review and level of detail was “about right,” and 77 percent said reports confirmed their tentative decision.
When asked whether the report informed their decision, 79 percent of requestors “agreed” or “strongly agreed,” underlining the vital role of the evidence reviews. Respondents also found the reports easy to request, easy to use, timely, and relevant, resulting in high requestor satisfaction.
The most common reasons cited for requesting a report was the CEP’s skills in identifying and synthesizing the available evidence, and the Center’s objectivity. Authors add that this credibility underscores the value of a neutral center in an environment where clinical departments and hospital committees may have competing interests, and where politics and external influences (including industry) may otherwise negatively influence institutional decision making.
The study’s other co-authors were Kishore L. Jayakumar, Julia Lavenberg, Matthew D. Mitchell, Jalpa A. Doshi, Brian Leas, David R. Goldman, Kendal Williams, and Patrick J. Brennan, all of Penn Medicine.
Funding for this research was provided by the National Center for Research Resources (UL1RR024134), which is now at the National Center for Advancing Translational Sciences (UL1TR000003).
Penn Study Finds Important Gaps in Evidence for Best Methods for Cleaning Hospital Rooms to Prevent Healthcare-associated Infections
Additional research needed to help lower risk for patients
August 10, 2015 – Tray tables, bed rails, light switches, and toilets: All are common vectors for swapping germs between patients and health care workers. While a new systematic overview in this week’s Annals of Internal Medicine points to several promising cleaning tactics of these “high-touch surfaces,” there’s a lack of evidence as to which is the most effective at reducing healthcare-associated infections (HAIs). Few studies measured patient outcomes or focused on newer technologies, and even less compared cleaning tactics against one another — important gaps to fill as the U.S. health care system works to reduce the 75,000 HAI-related deaths that occur annually.
The systematic overview was led by Craig A. Umscheid, MD, MSCE, an assistant professor of Medicine and Epidemiology in the Perelman School of Medicine at the University of Pennsylvania, and Senior Associate Director at the ECRI Institute-Penn Medicine Agency for Healthcare Research and Quality (AHRQ)-funded Evidence-Based Practice Center (EPC), Jennifer Han, MD, MSCE, an assistant professor of Medicine and Epidemiology, and Brian Leas, MS, MA, and Nancy Sullivan, research analysts in the ECRI-Penn AHRQ EPC, and revealed major gaps in existing evidence for the best practices for cleaning hospital room surfaces to prevent HAIs, including Clostridium difficile, MRSA, andVRE.
“The cleaning of hard surfaces in hospital rooms is critical for reducing healthcare-associated infections,” said Han, the study’s lead author. “We found that the research to date does provide a good overall picture of the before and after results of particular cleaning agents and approaches to monitoring cleanliness. Researchers now need to take the next step and compare the various ways of cleaning these surfaces and monitoring their cleanliness in order to determine which are the most effective in driving down the rate of hospital-acquired infections.”
While studies examining HAIs have increased over the last 15 years, infections acquired in the hospital remain a leading cause of death and morbidity. In 2011, there were over 721,000 HAIs in the U.S., according to the most recently available data from the U.S. Centers for Disease Control and Prevention (CDC). What’s more, many experts believe that only 50 percent of surfaces are typically disinfected during cleaning of a patient’s room.
Examining 80 studies published between 1998 and 2014, the research team found that comparative effectiveness studies were uncommon. Such studies would have directly compared different ways of cleaning, disinfecting, and monitoring the cleanliness of hard surfaces in order to determine which were most effective. There were also relatively few studies that focused on measuring outcomes of most interest to patients, such as changes in HAI rates or the presence of pathogens on patients. Only five of the studies were randomized controlled trials.
Instead, the existing studies were largely before and after experiments, comparing the magnitude of surface contamination after cleaning with a particular agent to the magnitude of contamination before cleaning. Over 65 percent of the studies assessed surface contamination, such as bacterial burden and colony counts, as the primary outcome. Less than 35 percent reported on patient-centered outcomes, such as HAI rates or acquisition of a specific organism in the body, known as colonization.
The ECRI-Penn EPC team looked at three broad categories of evidence: 1) which agents and methods were used to clean hard surfaces; 2) what approaches were available to monitor the effectiveness of cleaning; and 3) what systems-level factors are needed for cleaning and monitoring to be successful. In addition to its literature review, the researchers interviewed a number of national experts.
“Our goal was to provide a comprehensive review of evidence in all three domains,” said Umscheid, the study’s senior author. “While there is a clear need for more patient-centered and comparative effectiveness research, the findings that do exist provide a good place to start in terms of a hospital or health care entity seeking information on ways to mitigate healthcare-associated infections.”
Among its findings, the EPC team identified several studies showing that rates of C. difficile (or “C. diff”),the most common cause of hospital-acquired gastrointestinal infections, fell with the use of bleach-based disinfectants but that a chlorine dioxide-based product was ineffective in reducing C. diff contamination and infection rates. Patients taking antibiotics are at special risk of becoming infected with C. diff because antibiotics can disrupt the normal bacteria of the bowel. According to the CDC, C. diff caused almost half a million infections in the United States in 2011.
In addition, six studies integrating various wipes moistened with hydrogen peroxide and other chemicals into preventive strategies reported positive outcomes, including sustained reductions in HAIs. Seventeen studies implementing “no-touch” modalities to clean hard surfaces – such as devices that emit ultraviolet light or hydrogen peroxide vapor – reported positive findings, with three specifically demonstrating reductions in infection rates. Seven of eight studies evaluating enhanced coatings on hospital room surfaces, such as copper-coated bed rails, reported positive findings. Surfaces made of solid, copper-based metals or alloys continuously kill bacteria that cause infections.
The EPC team also highlighted several priority areas for future research, based on their review of the evidence and interviews with leading experts. Questions to emphasize in future studies include: what surfaces present the greatest infection risk to patients, what benchmarks should be established for measuring cleanliness, and what factors affect the quality of routine disinfection practices? Further research is also needed on recently emerging disinfection strategies.
“In addition to expanding the use of comparative effectiveness research and placing greater emphasis on patient-centered outcomes, future research should investigate the effectiveness of a number of promising new technologies and approaches,” said Han. “These include self-disinfecting coatings and increasingly used surface markers for monitoring the presence of pathogens. Other challenges include identifying high-touch surfaces that confer the greatest risk of pathogen transmission and developing standard thresholds for defining cleanliness.”
Other co-authors include David A. Pegues, MD, from Penn, and Janice L. Kaczmarek, MS from ECRI Institute.
Penn Study Shows Computer-Assisted Diagnosis Tool Helps Physicians Assess Skin Conditions Without Aid from Dermatologists: Tool Harnesses Photo Database to Improve Diagnostic Accuracy
April 22, 2015 – In the first major study to examine the use of a computer-assisted, photo-driven differential diagnosis generator for skin conditions, researchers at the Perelman School of Medicine at the University of Pennsylvania found physicians routinely used the tool, without an increase in calling for inpatient dermatology consultations. The software diagnostic tool, VisualDx, aids in diagnosing dermatologic conditions by allowing physicians to enter information such as the type and location of a rash, and associated symptoms such as pain or itching, and then generating a range of possible diagnoses accompanied by photographs. This enables users to rapidly compare the rash of an individual patient to a database of more than 1,300 pediatric and adult skin conditions represented by nearly 30,000 images, with the aim of improving diagnostic accuracy reducing misdiagnosis-related harm.
The study, published early online in Diagnosis (http://www.degruyter.com/view/j/dx.ahead-of-print/dx-2014-0058/dx-2014-0058.xml), found that the rate of dermatologic consultations at the Hospital of the University of Pennsylvania, the flagship hospital of Penn Medicine, did not change for the 18 months after VisualDx was introduced compared to the 12 months before it was introduced.
“This is important because these tools by design suggest numerous potential diagnoses, which could result in an increase in unnecessary testing and specialty consultation, and associated costs and harms, particularly in the hands of less experienced clinicians,” says the study’s senior author, Craig A. Umscheid, MD, MSCE, assistant professor of Medicine and Epidemiology and director of the Penn Medicine Center for Evidence-based Practice. “Conversely, if there were a significant reduction in dermatologic consultations, it would have suggested that general internists, emergency room physicians, family doctors and pediatricians, all of whom by definition are not specialists in dermatology, may have relied on the tool to make dermatologic diagnoses, rather than consulting a dermatologist for help.”
The Penn team examined the use of VisualDx throughout the University of Pennsylvania Health System by month for the 18 months following its introduction in September 2012. Researchers found a median of 474 unique monthly VisualDx sessions by users, a rate that continued throughout the time period under study. The researchers found that, overall, VisualDx was accessed through mobile devices (35 percent); inpatient (34 percent), outpatient (11 percent), and emergency department (one percent) electronic health records; and via searches in UpToDate, a point-of-care evidence summary resource (19 percent).
“Technology like this has great promise, but it can’t help patients unless it’s actually used,” says Umscheid. “Previous studies have been primarily simulations, where researchers take variables from case studies and input them into the software to retroactively assess the diagnostic accuracy of the software. However, our purpose was to determine if a differential diagnosis generator like VisualDx would actually be used by providers if implemented in a hospital, and we found that it is — most often on mobile devices and by inpatient providers.”
Differential diagnosis generators have been suggested as a potential way of reducing misdiagnosis, which is estimated to result in 40,000 to 80,000 deaths in U.S. hospitals annually. “The technology can help users overcome cognitive shortcomings such as availability bias, in which providers diagnose patients with conditions they’ve recently seen or can easily recall, rather than those conditions that are most likely to occur. It can also reduce fund of knowledge deficiencies by directing users to diagnoses they might not have otherwise considered,” commented the study’s lead author John Barbieri, MD, MBA, a 2014 graduate of the Perelman School of Medicine and The Wharton School.
The other Penn co-author was Benjamin French, PhD. The study was supported in part by the National Institutes of Health (UL1RR024134 and UL1TR000003).The ECRI Institute-Penn Medicine Evidence-based Practice Center (EPC) was recently redesignated by the Agency for Healthcare Research and Quality (AHRQ) to serve as an EPC through 2019. ECRI Institute has maintained this designation since the inception of the EPC program in 1997. The ECRI Institute-Penn Medicine EPC was established in 2012.
AHRQ Redesignates ECRI Institute-Penn Medicine as Evidence-based Practice Center
New 5-year contract supports the Agency for Healthcare Research and Quality in its efforts to disseminate scientific research of important healthcare topics
January 30, 2015 — The ECRI Institute-Penn Medicine Evidence-based Practice Center (EPC) was recently redesignated by the Agency for Healthcare Research and Quality (AHRQ) to serve as an EPC through 2019. ECRI Institute has maintained this designation since the inception of the EPC program in 1997. The ECRI Institute-Penn Medicine EPC was established in 2012.
Under the new contract, known as EPC V, the ECRI Institute-Penn Medicine EPC will work with clinicians, policymakers, patients, consumers, and other scientific partners to develop topics and questions for systematic review of the literature, and to promote the synthesis, translation, and dissemination of research evidence. The EPC has reported on a variety of topics, many in the area of cancer, genetic testing, and imaging tests (access free reports).
As part of the new contract, all 13 Evidence-based Practice Centers will be eligible to bid on work to support the U.S. Preventive Services Task Force (USPSTF), an independent, volunteer panel of experts in prevention and evidence-based medicine dedicated to improving the health of Americans.
“Since we began working together, we’ve been able to utilize ECRI Institute’s independent, evidence-based research and Penn Medicine’s clinical and academic leadership to advance research to improve patient care,” says Karen Schoelles, MD, SM, director of the ECRI Institute-Penn Medicine EPC. “This new contract inspires us to build on our previous collaborations and break into new research territory.
”“We look forward to continuing our efforts to transform patient care delivery as we work alongside ECRI Institute and AHRQ,” says Craig A. Umscheid, MD, MSCE, director of the Penn Medicine Center for Evidence-based Practice and senior associate director of the ECRI Institute-Penn Medicine EPC.
The EPC will work with Insight Policy Research, a small woman-owned disadvantaged business.
To learn more about ECRI Institute’s work in evaluating health technologies through the Evidence-based Practice Center, membership in the Health Technology Assessment Information Service, and customized consultation and support for clinical practice guideline development, e-mail email@example.com, or call (610) 825-6000, ext. 5519.
Penn Medicine Experts Urge Expansion of Federal Incentives to Improve Patient Care Quality and Safety
Focusing on structural changes in addition to individual clinical outcomes would bring greater gains
January 13, 2015 — In an invited commentary published online by JAMA Internal Medicine, Patrick J. Brennan, MD, chief medical officer of the University of Pennsylvania Health System, and Craig A. Umscheid, MD, MSCE, director of Penn’s Center for Evidence-based Practice, recommend that the Centers for Medicare and Medicaid Services shift its use of financial incentives from an exclusive focus on preventing specific clinical outcomes toward encouraging the creation of hospital-wide structures that promote the translation of research evidence into clinical practice to improve the quality and safety of patient care.
In 2008, Medicare implemented the Hospital-Acquired Conditions Initiative, a policy denying incremental payment for eight complications of hospital care, also known as “never events.” Another study published in JAMA Internal Medicine led by Teresa Waters, PhD, of the University of Tennessee Health Science Center, found that after the new federal policy was introduced, central line-associated bloodstream infections fell by 11 percent and catheter-associated urinary tract infections fell by ten percent. But there was no significant reduction in hospital-acquired pressure ulcers and serious inpatient falls.
The study authors suggested that the difference in the policy’s effects on infections as compared to falls and pressure ulcers were the result of the evidence available at the time to guide hospitals to improve these outcomes.
But the Penn authors disagreed. Instead, they suggested that the difference in the policy’s effect had more to do with the challenge of improving complex quality outcomes such as pressure ulcers and inpatient falls, when compared to the easier-to-improve outcome of infections. Unlike infection prevention efforts, which often target care providers and involve changes to a limited set of procedures such as catheter insertion and occur in closely monitored settings such as intensive care units, efforts at reducing falls and pressure ulcers are much more complex. They require multidisciplinary, patient-centered collaboration across the entire spectrum of care, not just critical care.
“Incentives are indeed important,” says Brennan. “But they must be the right incentives. There was good-quality evidence available to underpin fall and pressure ulcer prevention efforts when the CMS initiative was launched. Yet neither of these measures has significantly improved. This may be because it’s simply not possible to reduce falls and pressure ulcers by taking a few clear-cut steps, as in the case of reducing infections. Instead, broader and more comprehensive structural changes within hospitals should be considered. Such efforts would reduce an even larger number of never events than by simply tackling them one at a time.”
Hospitals should aim to bridge what’s called the “knowing-doing gap,” Umscheid says. “That’s the disconnect between what we know works based on the best available evidence and what we actually practice. To bridge the gap, we are not advocating abandoning incentives that focus on individual clinical outcomes. Instead we are urging federal regulators to also encourage institution-wide structural changes. We believe these changes can create the necessary conditions for solving difficult problems such as falls and pressure ulcers -- and others as well.”
The structural changes suggested by Brennan and Umscheid are the fabric of Penn Medicine’s successful quality improvement program. They include:
“We believe that incentivizing these structures, in concert with more focused incentives on specific clinical outcomes, together offer the best chance of improving patient care quality and safety in a variety of situations,” says Brennan.
- establishing unit-based leadership, where physicians, nurses, and quality administrators work together on their local wards or clinics to ensure that quality standards are met
- creating data stores that can provide local evidence for which quality initiatives are effective
- investing in staff education on evidence-based quality improvement, including in-house learning academies
- opening local evidence-based practice centers, where experts can identify and adapt national evidence and best practices to address local problems identified by staff
Penn Medicine's "Sepsis Sniffer" Generates Faster Sepsis Care and Suggests Reduced Mortality: Early identification and intervention lead to better results for hospital inpatients
October 9, 2014 — An automated early warning and response system for sepsis developed by Penn Medicine experts has resulted in a marked increase in sepsis identification and care, transfer to the ICU, and an indication of fewer deaths due to sepsis. A study assessing the tool is published online in the Journal of Hospital Medicine.
Sepsis is a potentially life-threatening complication of an infection; it can severely impair the body’s organs, causing them to fail. There are as many as three million cases of severe sepsis and 750,000 resulting deaths in the United States annually. Early detection and treatment, typically with antibiotics and intravenous fluids, is critical for survival.
The Penn prediction tool, dubbed the “sepsis sniffer,” uses laboratory and vital-sign data (such as body temperature, heart rate, and blood pressure) in the electronic health record of hospital inpatients to identify those at risk for sepsis. When certain data thresholds are detected, the system automatically sends an electronic communication to physicians, nurses, and other members of a rapid response team who quickly perform a bedside evaluation and take action to stabilize or transfer the patient to the intensive care unit if warranted.
The study developed the prediction tool using 4,575 patients admitted to the University of Pennsylvania Health System (UPHS) in October 2011. The study then validated the tool during a pre-implementation period from June to September 2012, when data on admitted patients was evaluated and alerts triggered in a database, but no notifications were sent to providers on the ground. Outcomes in that control period were then compared to a post-implementation period from June to September 2013. The total number of patients included in the pre and post periods was 31,093.
In both the pre- and post-implementation periods, four percent of patient visits triggered the alert. Analysis revealed 90 percent of those patients received bedside evaluations by the care team within 30 minutes of the alert being issued. In addition, the researchers found that the tool resulted in:
• A two to three-fold increase in orders for tests that could help identify the presence of sepsis
• A 1.5 to two-fold increase in the administration of antibiotics and intravenous fluids
• An increase of more than 50 percent in the proportion of patients quickly transferred to the ICU
• A 50 percent increase in documentation of sepsis in the patients’ electronic health record
The study found a lower death rate from sepsis and an increase in the number of patients successfully discharged home, although these findings did not reach statistical significance.
“Our study is the first we’re aware of that was implemented throughout a multihospital health system,” said lead author Craig A. Umscheid, MD, MSCE, director of Penn’s Center for Evidence-based Practice. “Previous studies that have examined the impact of sepsis prediction tools at other institutions have only taken place on a limited number of inpatient wards. The varied patient populations, clinical staffing, practice models, and practice cultures across our health system increases the generalizability of our findings to other health care settings.”
Umscheid also noted that the tool could help triage patients for suitability of ICU transfer. “By better identifying those with sepsis requiring advanced care, the tool can help screen out patients not needing the inevitably limited number of ICU beds.”
In addition to Umscheid, the other Penn co-authors are Joel Betesh, MD; Christine Vanzandbergen, PA, MPH; Asaf Hanish, MPH; Gordon Tait, BS; Mark E. Mikkelsen, MD, MSCE; Benjamin French, PhD; and Barry D. Fuchs, MD, MS.
Dr. Umscheid’s contribution to this project was supported in part by the National Institutes of Health, National Center for Advancing Translational Sciences, grant # UL1TR000003.