Journal Entry 1

Workflow Continuity—Moving Beyond Business Continuity in a Multisite 24–7 Healthcare Organization Brian J. Kolowitz &Gonzalo Romero Lauro & Charles Barkey &Harry Black &Karen Light & Christopher Deible Published online: 6 July 2012 # Society for Imaging Informatics in Medicine 2012 AbstractAs hospitals move towards providing in-house 24 × 7 services, there is an increasing need for information systems to be available around the clock. This study inves- tigates one organization’s need for a workflow continuity solution that provides around the clock availability for in- formation systems that do not provide highly available services. The organization investigated is a large multifacil- ity healthcare organization that consists of 20 hospitals and more than 30 imaging centers. A case analysis approach was used to investigate the organization’s efforts. The results show an overall reduction in downtimes where radiologists could not continue their normal workflow on the integrated Picture Archiving and Communications System (PACS) solution by 94 % from 2008 to 2011. The impact of un- planned downtimes was reduced by 72 % while the impact of planned downtimes was reduced by 99.66 % over the same period. Additionally more than 98 h of radiologist impact due to a PACS upgrade in 2008 was entirely elimi- nated in 2011 utilizing the system created by the workflow continuity approach. Workflow continuity differs from high availability and business continuity in its design process and available services. Workflow continuity only ensures that critical workflows are available when the production systemis unavailable due to scheduled or unscheduled downtimes.

Workflow continuity works in conjunction with business continuity and highly available system designs. The results of this investigation revealed that this approach can add significant value to organizations because impact on users is minimized if not eliminated entirely.

KeywordsWorkflow continuity.

Business continuity.

PACS planning.

PACS integration.

PACS downtime procedures.

PACS administration.

PACS.

PACS service.

Software design.

Systems integration.

Workflow.

Productivity.

Management information systems.

Information system.

Image retrieval.

Health level 7 (HL7).

Efficiency Background Recently, the US government mandated the use of health information technology for healthcare providers [1]. The legislation outlines financialpenalties for providers that choose not to adopt technologies as well as benefits for those that do adopt the technologies. As the adoption of health information technology increases, so will the need for information systems that allow critical organizational work- flows to continue when those systems are unavailable due to either scheduled or unscheduled system downtimes.

This paper is a case analysis of one organization’s solu- tion to a need for a system that provides workflow continu- ity around the clock. Workflow continuity moves beyond business continuity because the focus is placed on the con- tinuity of critical clinical workflows rather than creating a fully featured redundant system. A review of the organiza- tion’s existing business continuity solutions for the Radiol- ogy Information System (RIS) and Picture Archiving and Communications System (PACS) was performed. The B. J. Kolowitz (*) :C. Barkey :H. Black :C. Deible UPMC, Enterprise Imaging Informatics, 450 Melwood Avenue, Pittsburg, PA 15213, USA e-mail: [email protected] G. R. Lauro UPMC, Clinical Department Systems, 450 Melwood Avenue, Pittsburg, PA 15213, USA K. Light UPMC, Radiology Informatics, 450 Melwood Avenue, Pittsburg, PA 15213, USA J Digit Imaging (2012) 25:744–750 DOI 10.1007/s10278-012-9504-4 system requirements were reevaluated using the workflow continuity approach.

The organization studied consists of 20 hospitals and more than 30 imaging centers. The hospitals investigated are distributed across geographical area contained within a 125 mile radius. More than 2.2 million radiology procedures per year are performed across these hospitals. This case spans 4 years of work analyzing and improving the organ- ization’s business continuity solution. The business continu- ity solution was a key component in the design of the workflow continuity system. Throughout the process, nearly every aspect of information technology and workflow was evaluated for inefficiencies and workflow impact. Findings are presented along with a discussion of the results, practical implications for the organization, delimitations, and recom- mendations for further research.

The makeup of the organization’s radiology Information System (IS) infrastructure consists of 12 independent PACS silos, one consolidated RIS and one consolidated PACS integrated voice recognition application. A teleradiology solution allows radiologists to view and read imaging stud- ies for any location within the organization, regardless of the physical site of the radiologist. This subspecialist radiology coverage provides subspecialist services 24 × 7 seven days a week.

The centralization and availability of the subspecialist 24 × 7 service required an IS infrastructure that allowed the users to continue to provide patient care around the clock.

Planned maintenance and unplanned downtimes inhibit the user’s ability to perform their jobs and provide patient care efficiently and may affect their ability to provide care effec- tively. The Workflow Continuity approach identified a sub- set of the Information System features required to support the most critical user workflows.

Business Continuity Business continuity plans typically revolve around identify- ing risks and creating redundant systems. A business conti- nuity plan may identify risks and provide operational and information technology solutions to mitigate or eliminate those risks. One plan in particular identifies an eight step cycle used to protect the business needs of the organization including the following: (1) initiate the business continuity plan (BCP) project, (2) identify business threats, (3) conduct risk analysis, (4) establish business continuity plan, (5) design business continuity plan, (6) define business conti- nuity process, (7) test business continuity plan, and (8) review business continuity plan [2]. Each step identifies substeps, objectives, and goals. The purpose of such a detailed procedure is to protect the business against all of the threats to the organization, or at least the ones that would have the most critical impact on the organization. Threatsmay be technological, operational, or anything else such as hackers’attacks, earthquakes, or other disasters. Once threats are identified and system requirements are scoped, the BCP is at the mercy of the information systems ability to meet those requirements. Cost and information systems flexibility become considerations when attempting to exe- cute the business continuity plan.

The business continuity plans offered by the organiza- tion’s vendors did not meet the changing 24 × 7 service needs of the organization. Offsite backups are available to rebuild the system in the event of a disaster but this solution requires a minimal turnaround of 96 h per server. The PACS infrastructure alone consists of over 300 servers. The tech- nology and architecture of the PACS platform does not allow for a highly available solution to be created. High availability does not imply constant availability, and there may be times when the highly available system is unavail- able. In addition, the PACS Failover system did not meet functionality needs of the users.

Methods This case study is built upon data collected over a 4-year period. The investigators are also members of the informa- tion system team responsible for developing and implement- ing the solution. Informal interviews with the information system designers were performed, information system doc- umentation was reviewed, and quantitative data in the form of system down times was collected. Descriptive statistics were used to analyze the data, findings presented, and a discussion concludes the investigation.

Results This article evaluated the existing RIS and PACS failover strategies and compared the system downtimes utilizing those strategies versus the system downtimes after the im- plementation of the workflow continuity system. The fail- over strategies were designed around the limitations of the information systems when taking a business continuity plan approach. The design of these systems was reevaluated using the workflow continuity approach and results of the approach taken are presented.

RIS Failover Strategy The organization’s RIS failover strategy consisted of manual processes by which radiology technologists entered order information directly into the modalities whenever the pro- duction RIS was unavailable. The radiology technologists would then record the order information for the exams J Digit Imaging (2012) 25:744–750745 performed though a pen and paper process. Radiology tech- nologists would then key the order information from the paper record into the production RIS whenever the system became available. This translates into approximately 251 duplicate order entries per hour of downtime [2.2 million exams/(24 h per day × 365 days per year)]. This calculation does not account for the actual distribution of exams across hospitals and time of day, which can make the number of duplicate order entries per hour significantly higher. More procedures are performed during an hour of downtime at a core hospital than a community based hospital during and during peak hours than off-peak hours.

Radiologists or other physicians that required viewing access to the performed exams were only able to view exams that had orders prescheduled in the RIS prior to the downtime. The PACS system requires both DICOM images and health level 7 (HL7) messages for the user to view the exams within their normal workflows. Man- ual procedures were implemented for physicians to call the radiology reading rooms for STAT exams. The radi- ologist would then have to search through a list of exceptions in order to locate the appropriate exam for review. Relevant clinical information for exams that were not pre-ordered prior to the RIS downtime was verbally relayed over the phone.

PACS Failover Strategy The organization’s existing PACS failover strategy con- sisted of a single failover PACS for each of the hospitals with a limited set of prior images. Each failover PACS only contained images for the particular hospital the PACS serves. The effects of being on the PACS Failover system are felt by the entire enterprise because physicians typically cover patients by divisions that span hospital boundaries.

The independent PACS failover servers were not HL7 enabled or integrated with the production PACS. The ab- sence of HL7 limited the trust in the information contained within the system because the pairing process between DICOM image and HL7 order did not exist. Fat-finger errors performed at the modality could not be easily caught by the users. This might result in incorrect patient demo- graphics being associated with a series of images. Therefore, users were not allowed to perform final interpretations and dictate findings.

A business process was enacted where physicians would call the radiologists in the reading room for the most critical exams. The radiologists would then perform the interpreta- tion on-demand and relay the findings over the phone to the physician requesting an interpretation. Once the systems became available, the radiologists would then interpret the exams in the production system and dictate the exams within the PACS integrated voice recognition system.The absence of HL7 also eliminated the effectiveness of fine-grained user filters. Academic zone hospitals that pro- vide subspecialist coverage may have specialized work lists such as display only MR and CT neurology exams for the emergency department. Whereas the radiologists at the com- munity zone hospitals cover all exams for the hospital regardless of modality or location. Some of the fields need- ed to create these filters including patient location and exam code reside in HL7 messages. Since there was no HL7 interface on the PACS failover system, many of these filters were unable to be provided requiring the radiologist to search though larger lists of patients to find the set of exams he or she was required to interpret. As a result, the users had to user more general filters which resulted in a longer list of exams to navigate.

The longer list of exams on PACS failover is only related to the most recent hospital exams. The PACS failover servers only contain 1.6 TB of image cache storage. The oldest and least accessed exams are purged from cache when space is needed. The largest hospital contains approximately 4 weeks of recent patient history on the image cache whereas the smallest hospital con- tains nearly 2 years. A complete breakdown of available PACS workflow continuity systems (WCS) storage and exam history is shown in Table1. The inability to access relevant prior images may limit the radiologists’ ability to provide detailed subspecialist interpretations.

RIS/PACS Failover Impact on Workflow The users’workflows were negatively impacted when users were required to use either the RIS or the PACS Failover systems. The primary workflow impact on the technologists resulted from duplicate entry in both the paper based and Table 1Available WCS storage and exam history Hospital Storage (terabytes) History on WCS Production WCS % Months 1 144.0 1.6 1.1 1 2 17.6 1.6 9.1 6 3 49.6 1.6 3.2 2 4 46.4 1.6 3.5 2 5 46.4 1.6 3.5 2 6 9.6 1.6 16.7 8 7 33.6 1.6 4.8 2 8 35.2 1.6 4.6 3 9 4.8 1.6 33.3 48 10 4.8 1.6 33.3 5 11 19.2 1.6 8.3 4 12 14.4 1.6 11.1 5 746J Digit Imaging (2012) 25:744–750 then electronic records. The primary workflow impact on the radiologists was their inability to provide final interpre- tations within the integrated voice recognition system. As a side effect, the radiologists were also distracted by the large number of phone calls requesting STAT interpretation. The fully integrated production system provides status indicators that identify STAT exams whereas the PACS Failover sys- tem does not.

Radiology Workflow Continuity Workflow continuity is a term created by the organization to describe the information system architecture, flow of infor- mation, contingency procedures, and workflow that allows physicians to continue an uninterrupted workflow while the production information systems are unavailable due to both scheduled and unscheduled outages. Workflow continuity includes information system concepts such as highly avail- able system design and disaster recovery but the design focus is different. Highly available information systems provide services on demand quickly [3]. Redundancy is a key component so that parts of the system can continue working as other parts are unavailable [3]. A system is considered to provide Workflow Continuity if the core set of features users require are available for use when the primary system is unavailable due to a scheduled or un- scheduled downtime. Workflow continuity systems do not require redundant systems, but redundant system designs may provide workflow continuity.

RIS/PACS Workflow Continuity Design A comparison of features available between the Production system, the former PACS Failover System, and the new WCS is shown in Table2. The core set of features from the radiologists’perspectives is the ability to view and dictate an exam. Other cursory features may not be available but are not missed by the radiologists’during these down- times. The system diagram for the RIS/PACS Workflow Continuity System implemented is shown in Fig.1.

The production RIS has HL7 orders and results interfaces with both the production and WCS PACS. In addition, the WCS RIS has orders and results interfaces with the produc- tion and WCS PACS. A DICOM interface forwards images from the production PACS to the WCS PACS so that the twosystems are in sync. A special HL7 interface exists between the WCS RIS and production RIS. This interface allows the migration of orders placed on the WCS RIS to be registered into the production RIS.

The production and WCS PACS remain in sync with regards to orders and results. In order for the production RIS and WCS RIS to remain in sync, a series of manual procedures needs to occur when moving between the two systems. One-way transaction log shipping is set up from the production RIS to the WCS RIS. Other key identifiers such as accession number are set by the Database Administrator.

Other Aspects of Workflow Continuity The focus of the workflow continuity approach should not only include information technology solutions, but also preventative maintenance and vendor relations. Regular preventative maintenance such as operating system patch- ing, firmware upgrades, and hardware refresh are key com- ponents in minimizing the risk of impact on workflow.

Sometimes business process redesign for the organization and its vendors is also needed.

Workflow continuity will be difficult to achieve if the organization does not understand its own workflows as well as how the information systems affect those workflows. The organization studied invested a significant amount of effort to understand the technical architecture of the vendor pro- vided solutions. In addition, the organization educated the vendor on how the vendor product was used within the organization. This educational process is bidirectional Table 2Radiologist capabilities by system aCritical workflow as defined by the usersProduction RIS Failover RIS WCS RIS Production PACS Full workflow View only Critical workflow a Failover PACS View only View only View only WCS PACS Critical workflow a View only Critical workflow a RIS (Production) PACS(Production) PACS(WCS) RIS (WCS) HL7 InterfaceDICOM Interface Fig. 1RIS/PACS workflow continuity system J Digit Imaging (2012) 25:744–750747 where the vendor learns specific ways in which the organi- zation differs from other customers as well as the organiza- tion learning ways in which they are similar to other customers.

Once a shared understanding of the way the vendor solution integrates into the workflow was met, the parties involved could reevaluate maintenance and up- grade processes. One of the hospital PACS systems consists of two application servers, two database serv- ers, and over 60 storage modules where DICOM images reside. Prior to the workflow continuity approach, the entire system was unavailable until all 64 servers were upgraded. The organization was able to propose a new upgrade process because they understood the architec- ture as well as the impact on workflow. The organiza- tion’s suggestion that the vendor implemented as a result of the workflow continuity approach drastically minimized the time the system was unavailable. Now priority is given to the application and database servers as well as the storage modules with the most recent images. Working from most recent to oldest allows the entire system to come online quicker because the only impact is that the oldest exams are unavailable for a period of time. The users accepted this limitation be- cause many cases do not require prior images for com- parison, at least for a preliminary interpretation.

PACS Workflow Continuity Impact As previously stated, patient care was still required even though the RIS and PACS were operating on their failover systems. Exams that were interpreted and relayed to physi- cians over the phone were also dictated into the PACS integrated voice recognition system. This required the phy- sician to interpret and dictate the exam twice. Total down- time (scheduled and unscheduled) experienced by each of the 12 different PACS implementations across the organiza- tion is outlined in Fig.2.

Moving from the existing failover model to a sys- tem based on the concept of workflow continuity resulted in significant reductions in downtime over the last 4 years. Downtime data prior to 2008 was unavailable for analysis. By focusing on proper planned maintenance, the organization was able to reduce their unplanned downtime by 72.06 %. Conse- quently, proper maintenance and the workflow conti- nuity approach reduced the planned downtime by 99.66 % (Table3).

The case study covers two major PACS upgrades.

The first upgrade occurred in 2008 and the second upgrade occurred in 2011. Software on all application and database servers was upgraded. The amount of time spent on the PACS failover system in 2008 is equivalentto the amount of time radiology workflow was interrup- ted. The amount of time spent on the PACS WCS solution in 2011 is equivalent to a savings of radiologist interruption, or workflow continuity (Table4). The 2011 PACS upgrade utilizing the workflow continuity ap- proach saved the organization in 98 h and 36 min in terms of interruptions to radiologist workflow.

Discussion Continuity of workflow and core functionality are very valuable to radiologists [4] and the solution imple- mented at the organization has virtually eliminated radi- ology workflow interruptions during downtimes. The unconventional system development approach and oper- ational workflow procedures implemented at the organi- zation under the workflow continuity guidelines almost entirely eliminated the radiologist workflow interrup- tions due to planned or unplanned system downtimes.

Workflow continuity does not exclude business continu- ity but builds on top of it. Advantages of business continuity solutions available offer a level of redundan- cy that can fully restore functionality when part of the system fails [5]. Many companies develop disaster con- tingency recovery plans that include alternative proce- dures and workflows [6,7] which inevitably disrupt the normal flow of operations. Business continuity solutions require a system architecture that can support the 2008 2009 2010 2011 Planned Downtime 10440 3623 2467 36 Unplanned Downtime 2448 2080 1275 684 0 2000 4000 6000 8000 10000 12000 14000 Minutes Fig. 2Total planned and unplanned downtimes by year Table 3Percent im- provement from 2008 to 2011Downtime type % improvement (2008–2011) Unplanned 72.06 % Planned 99.66 % Total 94.41 % 748J Digit Imaging (2012) 25:744–750 functionality. Workflow continuity is not bound by this constraint because the focus is not on a fully redundant system, but a system that allows the critical workflows within the organization to continue. Critical workflows are those workflows that are essential to the delivery of patient care. In this instance, there were three critical workflows defined including the following: the ability for radiology orders to flow from the RIS to the PACS, the ability for radiologists to interpret exams and dictate the PACS without disruption to their normal workflow, and the ability for results to file in the RIS and PACS from the voice transcription service. In addition, an emphasis was put on the reduction of manual effort by radiology technologists and PACS administrators when resolving PACS exceptions. These exceptions occur when there is a discrepancy between DICOM images and HL7 order messages as a result of system unavail- ability or human error.

Workflow continuity should not be thought of as a system design but a mindset for designing systems.

Highly available systems that allow workflows to con- tinue without significant interruption provide work- flow continuity. Other workflow continuity system configurations might include two systems, A and B, that have bidirectional synchronization, one-way syn- chronization, or no synchronization at all (Fig.3). The key concept in workflow continuity is not the techni- cal architecture of how the information systems pro- vide continuity, but only that continuity of critical workflows exists.Delimitations of Workflow Continuity Workflow continuity should not be thought of as the only solution needed for an organization. Highly avail- able systems, business continuity, and disaster recovery are all critical components to the organization’s overall information system enterprise architecture. Workflow con- tinuity systems are bound by the same architectural restrictions as the fully functional production systems.

In the case studied, the workflow continuity system does not address 1.6 TB image cache issues that existed with the PACS failover system. During the workflow continuity de- sign sessions, it was determined that the prior history restric- tions were acceptable to the users. Extended periods of downtime might change the users’perceptions of what level of acceptable history is required, and those situations are where disaster recovery fit in. A 96-h disaster recovery turnaround becomes more acceptable when the users are able to continue their normal workflows during that period of time.

Conclusion The organization has implemented a comprehensive workflow continuity solution that significantly improved uptime and enabled the success of a centralized 24 × 7 subspecialty radiol- ogy coverage across multiple hospitals. While workflow con- tinuity does not provide 100 % of the functionality available during normal operations, it delivers the most critical function- ality which allows radiologists to continue working uninter- rupted. The solution is relatively low cost, highly effective, and requires only limited changes in the architecture of existing systems. The organization is applying this methodology to other departments and information systems.

Table 4Failover interruption vs. WCS continuity Hospital 2008 PACS failover duration and radiologist workflow interruption2011 PACS WCS duration and radiologist workflow continuity (min) 1 278 min 554 2 575 min 307 3 424 min 535 4 1,112 min 494 5 360 min 720 6 527 min 299 7 555 min 327 8 386 min 458 9 555 min 397 10 569 min 600 11 575 min 500 12 a 0 min 424 Total workflow interruption98 h, 36 min 0 aHospital 12 was not part of the organization in 2008 Workflow Continuity Highly Available System Workflow Continuity System A System B Workflow Continuity System A System B Workflow Continuity System A System B Fig. 3Possible workflow continuity system configurations J Digit Imaging (2012) 25:744–750749 References 1. American Recovery and Reinvestment Act of 2009, HR 1, 111th Cong., 1st sess, 2009 2. Lam W: Ensuring business continuity. IT Prof 4(3):19–25, 2002 3. Lampson B: How to build a highly available system using consen- sus. Distributed Algorithms, 1151:1–17, 19964. Joshi V, Lee K, Melson D, Narra V: Empirical investigation of radiologists’priorities for PACS selection: an analytical hierarchy process approach. J Digit Imaging 24:700–708, 2011 5. Khorasani R: Business continuity and disaster recovery: PACS as a case example. J Am Coll Radiol 5(2):144–145, 2008 6. Cerullo V, Cerullo MJ: Business continuity planning: a comprehen- sive approach. Inf Syst Manag 21(3):70–78, 2004 7. Keene S, Auger L: Continuity of business: does your hospital have a plan.Internet J World Health Societal Polit 4(2), 2007 750J Digit Imaging (2012) 25:744–750 Copyright of Journal of Digital Imaging is the property of Springer Science & Business Media B.V. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.