Evaluating and Disseminating the Impact of an Evidence-Based Intervention

Evaluating and Disseminating the Impact of an Evidence-Based Intervention: Show and Tell
After the data are gathered and analyzed, it’s time to share what you’ve learned.
This is the 11th article in a series from the Arizona State University College of Nursing and Health Innovation’s Center for the Advancement of Evidence-Based Practice. Evidence-based practice (EBP) is a problem-solving approach to the delivery of health care that integrates the best evidence from studies and patient care data with clinician expertise and patient pref- erences and values. When delivered in a context of caring and in a supportive organizational culture, the highest quality of care and best patient outcomes can be achieved.
The purpose of this series has been to give nurses the knowledge and skills they need to implement EBP consistently, one step at a time. The final article in the series will be published in the September issue.
In the previous article in this series, Carlos A., Rebecca R., and Chen M. completed the unit-based pilot phase of the rapid response team (RRT) roll- out. They found that the RRT worked well, and they are now ready to evaluate its impact on their chosen outcomes. The hos- pital leadership as well as the staff had agreed upon the following outcomes: code rates outside the ICU (CRO), unplanned ICU ad- missions (UICUA), and hospital- wide mortality rates (excluding do-not-resuscitate situations) (HMR). Karen H., the nurse from the Clinical Informatics Depart- ment, and the pilot unit’s quality council representative devised a mechanism to successfully ex- port the RRT data from the elec- tronic medical record (EMR) to a database that would serve as a repository until the data could be analyzed. The other depart- ments collecting RRT outcomes data have been forwarding their information to Rebecca and Chen, who’ve asked Karen for help in getting this additional data onto the hospital’s quality dash- board. Karen suggests that she
and the EBP team meet to discuss ways to upload all of the data to one place and create a single com- prehensive and regularly available summary of the RRT outcomes.
At that meeting, Karen suggests that the EBP team work out a plan with the Quality/Performance Improvement Department to analyze the data before they’re posted on the dashboard, where they’ll be available to everyone on the hospital intranet. The EBP team members share their excite- ment about taking the next step in the EBP implementation pro- cess. But when Carlos contacts the director of the department, the director informs him that it may be impossible for quality/ performance improvement to take on this project at this time, as their analysts are already over- loaded with work. Chen mentions that she’s heard that university researchers may be interested in these kinds of projects, and that collaboration with a university might lead to further projects, which could keep the kind of ex- citement generated by the RRT initiative going. Carlos says that he has some connections at the
local university and offers to dis- cuss this opportunity with them.
GATHERING AND EVALUATING THE RESULTS Carlos calls the dean of research at the hospital’s academic partner to inquire about interest in collabo- rating on the RRT project, particu- larly from a research perspective. The dean says there’s a researcher who is very interested in the pro- cesses of codes and may want to get on board with their project. Carlos asks about data analysis and interpretation as part of that collaboration, and the dean replies that the university has resources they can use to accomplish that part of the evaluation process. Carlos lets Rebecca and Chen know of this opportunity and sends an e-mail to Debra P., the faculty researcher, outlining the RRT project and asking if she’s interested in participating. Debra responds the next day, indicating her delight to be involved. The EBP team is excited that they’ll have this opportunity to partner with the local university and ac- complish their goal of performing data analysis.
56 AJN ▼ July 2011 ▼ Vol. 111, No. 7 ajnonline.com
 
 
By Ellen Fineout-Overholt, PhD, RN, FNAP, FAAN, Lynn Gallagher-Ford, MSN, RN, NE-BC, Bernadette Mazurek Melnyk, PhD, RN, CPNP/PMHNP, FNAP,
FAAN, and Susan B. Stillwell, DNP, RN, CNE
Carlos discusses the initial RRT data with Debra, and they ana- lyze it together. First, they look at the mean outcomes of CRO, HMR, and UICUA that were ob- tained from the real-time RRT re- ports. When they compare these outcomes over time, they see that the mean CRO was reduced, but that the mean HMR and UICUA hadn’t changed from baseline. Debra asks whether there was any variation in the occupancy rate over the period of the pilot rollout; if there was, then the proportion of patients experienc- ing codes before and during the rollout might not be comparable. When Carlos replies that the oc- cupancy rate remained consistent, Debra recommends that they conduct an independent t test to see if there’s a statistically signifi- cant difference between CRO be- fore and after the pilot phase. They find that the decrease in CRO is statistically significant, which means that the RRT had a posi- tive effect on this important out- come that most likely wasn’t a chance finding. The EBP team can’t wait to share this great news with the unit. The team reviews with Debra the code records and RRT comments to determine if
there were any RRT processes that might have had an impact on UICUA and HMR, and thereby explain the lack of a change from baseline. The team also provides Debra with questions about how the pilot went (who called the RRT and why? what challenges did the RRT face?) that they be- lieve would be important to ask the stakeholders during the de- briefing after the pilot. Debra says that these questions will be very helpful as she looks over the RRT processes. Having them in mind, she can see if the answers exist in the current data, if more data need to be gathered, or if further questions need to be asked.
After taking time to reflect on these processes, the EBP team works with Debra to revise them. Debra explains that it’s impor- tant to plan the hospital-wide rollout so that all unit managers and staff are confident they un- derstand the protocol, processes, and desired outcomes. They ask Pat M., the manager of the pilot unit, and two of her EBP champi- ons to relate their experiences with the RRT to the executive leadership team, the unit manag- ers’ meeting, and the unit council
leadership meeting. The unit man- agers were especially glad to hear Pat’s story and her answers to their questions.
As the EBP team continues to discuss plans for a hospital-wide RRT, Debra’s suggestions for how to improve the RRT processes in the larger rollout are easily inte- grated into the plan. For example, she proposes a simple way to ex- amine the outcomes of HMR and UICUA: since ICU deaths were included in the HMR data, she suggests that they ask the Health Information Management Sys- tems/Medical Records (HIMS) Department to compare the ICU deaths that occurred despite the presence of an RRT with those that occurred without an RRT present. Debra explains to the team that these data may help them to have a better picture of the impact of the RRT on HMR. She applies the same approach to UICUA, comparing the ICU admissions of those who’d been treated by the RRT with those who hadn’t. She further explains how the team can continue to observe the changes in these two outcomes over time. The EBP team is glad to hear that Debra will continue to help as they col- lect and analyze these data.
In preparation for the hospital- wide rollout, the EBP council confirms that EBP champions on each unit will be responsible for working with the educators to conduct education sessions about the RRT. Each unit par- ticipating in the rollout has al- ready had three in-services on all shifts, posters put up in the bath- room and staff lounge, and an algorithm posted at the unit hub explaining how to call the RRT. Finally, nurses and secretaries from all units are invited to a meeting at which Debra and the EBP team answer all questions
ajn@wolterskluwer.com AJN ▼ July 2011 ▼ Vol. 111, No. 7 57
Dissemination Workshop Agenda Joint session (one hour)
Dissemination: Purposes and Passions • What outcome do you want to achieve by disseminating
your results? • Discussion
Methods of Dissemination • Determine which method of dissemination is the best
match for your message or outcome or both. • Determine which method capitalizes on your strengths. • Discussion and demonstration or case study
Breakout sessions (one hour) Publishing: Who, What, When, Where, and How of Publishing Presentations: Effective, Fun Presentations People Will Remember
 
 
however, says that there’s no way she can support anyone from her unit presenting at a conference. The EBP team informs her that several manuscripts about the RRT will be submitted for publi- cation, which creates the perfect opportunity for those who wish to contribute, but who may not have the budget this year, to sup- port the presentations.
The EBP team decides to hold a continuing education workshop on dissemination. They invite the EBP council members to come and bring anyone from their units who has been involved in the RRT proj- ect and is interested in contributing to presentations or publications about it. In preparing to conduct this class, the team makes a list of the aspects of the RRT project that would be important to in- clude in a presentation or publi- cation or both. They work out an agenda for the workshop (see Dis- semination Workshop Agenda). Rebecca, Chen, and Carlos are excited about sharing the out- comes of first the pilot and then the rollout to the whole hospital. They are thrilled that they’ve made such a difference in their hospital’s culture, as well as in patient outcomes.
MAKING DISSEMINATION PLANS The EBP council, the educators, the RRT, and the EBP team, along with Debra, meet to discuss how to plan for dissemination of the project and its results. They dis- cuss first putting the results of
PREPARING TO DISSEMINATE THE RESULTS As the EBP team discusses how to disseminate the results of their project, they reiterate their com- mitment to involve the EBP coun- cil members, who have made such a major contribution to the proj- ect’s success. Debra suggests that they hold a special meeting with unit managers to answer their
questions, and to give them an overview of the dissemination plan, including the impact it may have on each unit’s budget. The meeting with the managers turns out to be a lively discussion about the value of dissemination and its related costs. The managers are concerned that presenting the re- sults of the RRT intervention at conferences is not a budgeted item for this year; they’re also concerned about the challenges these opportunities will present, such as being able to support the scholarship of those clinicians whose work is accepted.
The EBP team helps the unit managers to understand that each time a clinician presents an aspect of the RRT process or outcome, the unit and hospital get positive exposure. Eventually most man- agers agree that dissemination is a worthwhile investment and com- mit to be as creative and flexible with their budgets as possible as they plan for the next fiscal year. They discuss how important it is to support these new learning and development opportunities for their staff. One unit manager,
concerning the procedure for calling an RRT.
After the hospital-wide project begins, the EBP team asks HIMS if all is well with the baseline data and how the outcomes data are being collected. HIMS informs them that indeed the staff is doing a terrific job of entering the data into the EMR. The initial RRT reports indicate that the hospital- wide rollout is going well and that the RRT protocol is being used appropriately. When the EBP team informally interviews EBP council members, they find that everyone is seeing the difference the RRT is making—and not only in the outcomes. Clinicians, for example, are experiencing a dif- ference in how they’re helping patients avoid those outcomes. This pleases the EBP team and they look forward to sharing this serendipitous finding.
Presentation Tips • Keep the outcome that you want for your pre-
sentation in mind from the beginning: what do you want the audience to take away?
• Take care with the background and color schemes for your PowerPoint slides. Simple is best.
• Keep your presentation simple, innovative, and interesting. Don’t overuse animation or sound.
• Use pictures to enhance, not dominate, the presentation.
• Keep your time frame in mind: usually one slide per minute works well.
• Use no smaller than a 20-point font on a slide if the presentation is for a smaller audience or room, no smaller than a 28-point font for larger rooms or audiences.
• Use text on a slide for sharing highlights and important points, not for everything.
• Revise your presentation at least three to five times before submission.
• Keep backups of the presentation on a jump- drive (or two)
• Have fun as your create YOUR presentation— be unique.
The EBP team reflects on what a
difference just asking and
answering the right question has
made in their hospital.
58 AJN ▼ July 2011 ▼ Vol. 111, No. 7 ajnonline.com
 
 
little longer to prepare their man- uscripts, while their team leaders call or write the journals they’ve selected to see if there’s any inter- est in articles on various aspects of the RRT. The EBP team reflects on their initial PICOT question and on what a difference just asking the right question and an- swering it appropriately has made in their hospital.
Join the EBP team next time as they complete the hospital-wide rollout and make the RRT a hos- pital policy. In so doing, they will learn how to create system-wide sustainable change. ▼
Ellen Fineout-Overholt is clinical pro fessor and director of the Center for the Advance- ment of Evidence-Based Practice at Ari- zona State University in Phoenix, where Susan B. Stillwell is clinical professor and associate director, Lynn Gallagher-Ford is clinical assistant professor and assistant director, and Bernadette Mazurek Melnyk is dean and distinguished foundation pro- fessor of nursing at the College of Nurs- ing and Health Innovation. Contact author: Ellen Fineout-Overholt, ellen. fineout-overholt@asu.edu.
may be a good venue for this proj- ect. They readily discuss sharing how their transdisciplinary team worked together to improve out- comes and other issues from the project that would interest IHI participants. They all agree to en- gage in this discussion further as the project continues.
Amid all this activity, Rebecca and Chen remind Carlos that there are clinicians who would rather publish than present. Carlos and Debra meet with those who are interested in publishing to pro- vide an overview of the publish- ing process (see Publishing Tips). They assure those individuals who feel they don’t write well enough to publish in a journal that they’ll do fine as part of a team.
With plans in hand, the teams of clinicians begin to prepare their abstracts or manuscripts. The presenting teams submit their ab- stracts to their respective confer- ences. The writing teams take a
the pilot and then of the hospital- wide RRT rollout on the hospi- tal’s intranet. Carlos invites Karen from clinical informatics to join them to discuss the possibility of having an “EBP Corner” on the intranet, where updates can be provided for the latest EBP events. Karen says this is very doable and that she’ll get back to them in a couple of days on how to set this up and how they’ll be able to con- tribute to it. Carlos agrees to take the lead for this aspect of the dis- semination project.
The EBP council, with mentor- ship from Rebecca and Chen, ex- presses the desire to present the RRT project at a professional meeting. The group decides that one of the annual EBP confer- ences across the country would be the best place to share this proj- ect. Debra offers to help council members review the variety of EBP conferences and discuss which would be the best match. She asks them to consider which audience would like to hear about their project and where it could have a meaningful impact. She offers to join them when they start to write and then submit an abstract, and, if it’s accepted, to help them put together the pre- sentation. She also shares tips she’s used that have served her well (see Presentation Tips).
To the EBP team’s great delight, the chief nursing officer pops into the council meeting and tells ev- eryone that she wants to submit this project to the American Or- ganization of Nurse Executives (AONE) annual meeting. She’s so excited about the synergy be- tween leadership and staff that she believes this is just what par- ticipants at AONE need to hear. Carlos asks the members of the RRT if they’d like to discuss the possibility of presenting their ex- perience at the annual Institute for Healthcare Improvement (IHI) meeting, which he tells the group
Publishing Tips • Know the purpose of your manuscript. • Determine the audience for your manuscript. • Determine the journal that best matches the purpose of your
manuscript. • Obtain the author guidelines for this journal. • Review several journal articles from this journal; noting the struc-
ture of these articles can help with structuring your manuscript. • Send a query letter to the editor. • Develop an outline for your manuscript; be as descriptive and
detailed as possible. • Divide writing the outline among the authors; all authors should
contribute to the manuscript. • Write, read, rewrite, reread, rewrite, reread, and rewrite your
manuscript. Have others read the manuscript and provide feed- back; now is the time to get critical feedback to assist in the suc- cessful submission to a journal.
• Decide on a relevant title that would compel you to read the manuscript.
• Reread and revise one last time. • SUBMIT—although rewriting has moved your manuscript toward
perfection, don’t wait for it to be entirely perfect. Expect journal reviewers to have suggestions and criticism.
• Believe in your message and its benefit to the reader.
ajn@wolterskluwer.com AJN ▼ July 2011 ▼ Vol. 111, No. 7 59