Abstract
The absence of user-friendly systems for reporting complications is a major barrier to improving quality assurance (QA) programs in interventional radiology (IR) services. We describe the implementation of a QA application that is completely integrated with the radiology dictation system. We implemented an IR QA process as a module within the electronic medical record and radiologist dictation system applications used at our institution. After a radiologist completes a dictation, he or she must select from a drop-down list of complications before proceeding to the next case. Delayed QA events can be entered using the same applications. All complication entries are sent to a database, which is queried to run reports. During the study period, all the 20,034 interventional procedures were entered in the QA database, 1,144 complications were reported, 110 (9.6%) of which were classified as major. Although majority of the complications (996) were entered at the time of dictation, 148 complications (12.9%) were entered afterwards. All major complications were referred to the IR peer review committee, and 30 of these were discussed in the morbidity and mortality meetings. We studied post-lung-biopsy pneumothorax and chest tube rates and initiated a quality improvement process based on the results.The integration of the IR QA reporting system into the workflow process and the mandatory requirements for completion has the potential to minimize the work effort required to enter complication data, and improve participation in the QA process.
Key words: Interventional radiology, quality assurance, complications, data collection, workflow
Introduction
The Joint Commission on Accreditation of Healthcare Organizations requires all hospital organizations to assess the performance of practicing health care providers for the purpose of renewing privileges during reappointments. According to the guidelines, practitioner-specific data must be accumulated and compared with aggregate data.1–5
Evaluating radiology faculty’s performance is especially challenging because outcome data are limited. Major efforts are being made to define performance metrics in the field of diagnostic radiology.3,6–10 In most radiology departments, the most common parameter used to evaluate radiologists’ performance is a discrepancy in the formal interpretation of an imaging study, with little or no emphasis on interventional radiology procedures.3,6–12 However, since interventional radiologists perform invasive procedures and do not interpret imaging studies, a review of formal interpretation is not a good measure of assessing interventional radiologists’ competence. According to Society of Interventional Radiology (SIR) guidelines, the safety- or procedure-related complication rate is one of the most important indicators for evaluating the success of any invasive procedure13. However, a major barrier to reporting complications or participating in quality assurance (QA) activities is the fact that most reporting systems are stand-alone systems that are not integrated into the routine workflow and require the physician/radiologist to interrupt his or her routine clinical or academic activities, log on to the system, find the case, and enter the event.
Here, we describe the implementation of a QA program that was integrated with a digital dictation system to facilitate the mandatory reporting of procedure-related complications with minimal impact on the radiologist’s workflow efficiency. The entries populate a secure database that can be used for identifying and managing trends.
Materials and methods
Our institution uses an internally developed electronic medical record (EMR) system called ClinicStation that provides physicians a complete and current view of patients’ data. The Radiologist Dictation Application (RadStation) integrates Philip’s iSite Radiology Application and ClinicStation. RadStation gives the radiologist a dedicated application for dictating interventional radiology reports; it also gives the radiologist immediate access to each patient’s EMR. We collaborated with our institution’s EMR department to develop an Interventional Radiology (IR) QA component to be used in conjunction with the ClinicStation and RadStation applications to provide efficient QA workflow for interventional radiologists. Our institutional review board (IRB) reviewed this protocol and deemed that this was not human subjects research, but rather a QA project, and hence did not need IRB approval.
The QA data entry program was designed to be compliant with Health Insurance Portability and Accountability Act (HIPAA) regulations. Only physicians and other staff members who have successfully completed the HIPAA compliance course are allowed data entry privileges.
Data Entry
When a user logs in to RadStation, the application recognizes the person as an interventional radiologist based on the user’s profile. The administrative task of designating a user as an interventional radiologist is performed via an administrative tool that can be used to grant or revoke access to users. In addition to all IR faculty, IR fellows and radiology residents can access the IR QA system during their IR rotation and enter a complication, if appropriate, to an IR procedure. By departmental policy, access to fellows and residents is revoked when they finish their rotations.
The radiologist can select an undictated interventional procedure from one of the worklists available in RadStation. Alternatively, the radiologist can access the procedure number using the barcode reader function of the handheld Dictaphone. Once a procedure is selected for dictation, RadStation displays the patient’s name and medical record number, procedure name, and the name of the radiologist who performed the procedure. The radiologist can view the patient’s clinical and laboratory records on the primary workstation monitor and the diagnostic images on the two secondary monitors.
A “complication panel” featuring several buttons (Fig. 1) is displayed on the primary monitor for all IR dictations. After dictation is completed, data entry can be completed either by using the mouse to click on the “end-dictation” tab on the computer panel or by pressing the top left-hand button on the Dictaphone. However, radiologists participating in the IR QA program are required to enter complication data before ending dictation. The IR QA system does not allow the interventional radiologist to end dictation without performing this QA step. Attempts to end dictation without having entered complication data results in the display of an alert (Fig. 2), that prompts the radiologist to add the required information.
Fig 1.
Screen capture showing the “complication panel” that is displayed on the monitor for all interventional radiology procedure dictations.
Fig 2.
Screen shot of the alert that pops up on the computer monitor if the dictating radiologist attempts to end dictation without having entered complication data.
If no complications occurred, the radiologist makes a “none” entry in the complication box by pressing the lower of the two left-hand buttons on the Dictaphone or by selecting the “none (all)” tab on the complication panel (Fig. 1). If, however, a complication did occur, the radiologist can enter information about the complication by selecting from a drop-down list in the complication panel (Fig. 2). The system them prompts the radiologist to classify the complication as major or minor (using the standard SIR guidelines). Generally, entering “no complication” or “none” takes less than a second, and entering a specific complication takes less than 5 seconds.
In addition to the above-described mandatory entry of complications during IR procedure dictation using RadStation, IR radiologists can also perform QA on a previous IR procedure using ClinicStation. In ClinicStation, the radiologist can select any previously dictated IR procedure and add a complication to the procedure using the same complication panel buttons described for RadStation. If the complication occurs within 30 days of the procedure, the radiologist selects the “<30 days” tab from the complication panel (Fig. 1). However, if the complication occurs more than 30 days after the procedure, the radiologist selects the “>30 days” tab, and this entry appears as a delayed complication in the database. This allows entry of complications that do not happen at the time of the procedure but happen or are discovered later on. The radiologists can enter multiple complications for each procedure. Because ClinicStation also serves as the institution’s EMR system, IR staff can enter complication events from any institutional computer, even while on rounds in the hospital.
We also designed another method to promote and encourage radiologists to enter information about delayed complications in patients who have returned for repeat or new IR procedures. When an examination is selected for dictation, the system will identify any previous IR procedures performed on that patient. In patients who have undergone prior IR procedures, selecting the “end current dictation” command activates an alert (Fig. 3) that prompts the radiologist to add QA data for the prior procedure(s). Once the radiologist acknowledges the alert, the system allows the “end current dictation” process to continue. The alert compels the interventional radiologist to check prior IR dictations on the selected patient and add delayed complications if indicated and appropriate.
Fig 3.
Screen shot showing the alert that pops up on selecting the “end current dictation” command in patients who have undergone prior IR procedures. It prompts the radiologist to add QA data for the prior procedure(s), if appropriate.
Database, Data Review, Query Process
All complications entered through either ClinicStation or RadStation are transmitted via a services architecture using standard web services that connects the data entry applications with database servers (Fig. 4). A three tier application infrastructure is employed: local client, business logic servers and database. The web services data transmission populates an IR QA database (SQL 2005; Microsoft Corporation, Redmond, WA, USA) that records the following data: the patient’s medical record number, the IR procedure performed, the procedure date, the accession number, the IR faculty who performed the procedure, the person submitting the data, the IR trainee involved in the procedure (if applicable), the date and time of the entry, and the complication type (Fig. 5). Other than the complication entered by the physician at the time of dictation, all other data are extracted from the existing information in ClinicStation. The database is stored on a secure server in the institutional network. Access to the database server is granted via a secure network login that complies with HIPAA regulations. Only the QA director and other staff members who have successfully completed the HIPAA compliance course may access the database.
Fig 4.
Flow chart showing the architecture that integrates the IR QA application with the ClinicStation and RadStation applications.
Fig 5.
Screen shot demonstrating the IR query tool that is used to identify and manage trends in events collected in the QA database.
We implemented a separate application, “IR Query,” to identify and manage trends in events collected in the QA database. This application’s functions include search capabilities, predefined reports, and graphic data reports, which provide visual representations of data. This application can also be used to edit the data if necessary; all changes are tracked by SQL database events. Currently, the IR QA director generates monthly reports for all complications and classifies them as major or minor according to SIR criteria.
Results
From the time of the IR QA application’s implementation on November 1, 2005, until May 8, 2008, 20,034 IR procedures were entered in the QA database (100% compliance). During that interval, 1,144 complications were reported; 110 (9.6%) of these events were classified as major using the SIR guidelines, and the remaining complications (1,034; 90.4%) were classified as minor. Most procedures (n = 1116) had one complication each; however, two complications each were reported for 14 procedures. The vast majority of complications (996; 87.1%) were entered at the time of dictation by the faculty member (780; 78.3%) or the trainee (217; 21.7%) dictating the procedure; the remaining 148 events (12.9%) were entered afterwards by the same faculty member who did the initial dictation or a different faculty member using either ClinicStation or RadStation. More than three-fourths (123; 83.1%) of these later entries represented events that occurred within 30 days of the procedure, while the rest (25; 16.9%) represented events that happened after the 30-day period. The faculty members who performed the procedures reported 51 (34.5%) of these late entries, while the remaining later events (97; 65.5%) were entered by a different faculty member.
All the major complications were referred to the IR peer review committee, which met every month. During this time period, 30 of the major complication events were also discussed in the monthly morbidity and mortality meetings of the Section of Interventional Radiology. We compared the overall complication rates for individual interventional radiologists (Fig. 6). We also evaluated the pneumothorax and chest tube insertion rates from lung biopsy procedures for the entire group and for individual radiologists, and based on the results, initiated a new QI project aimed at reducing the post-lung biopsy chest tube insertion rates. The query tool also allowed evaluation of trends in complication rates for individual radiologist over defined time periods (Fig. 7)
Fig 6.
Bar diagram showing the complication rates for different IR physicians during the study period.
Fig 7.
Graph showing the change in complication rates over consecutive 6-month periods for 4 IR physicians.
Discussion
Most recent articles on online radiology QA reporting systems have focused on diagnostic radiology and are primarily related to errors in interpreting images, with little emphasis on IR procedures.6,9,13–16 The system discussed in this paper focuses on entering complications related to IR procedures and can therefore be used for QA programs in interventional radiology departments.
Some of the previously reported methods of tracking IR-related complications require physicians, technologists, secretaries, nurses and system operators to manually enter patient- and procedure-related information.17,18 One of these systems requires the physician to enter all demographic and procedure-related details in on hard-copy data sheets, which are stored in binders and subsequently entered into an electronic database by a technologist, taking 2 minutes of technologist time per procedure.18
HI-IQ is a commercially available free-standing electronic database that allows for online entry and tracking of procedure-related complications; however, the stand-alone QA program requires the operator to specifically log on to the database, thereby interrupting routine workflow9 This interruption, in our opinion, could be perceived as an obstacle by the interventional radiologists, potentially resulting in an underreporting of complications.
The IR QA program described here avoids the expense and complexity of maintaining myriad interfaces to support a stand-alone QA program by being specifically integrated into pre-existing systems containing patients’ EMRs. One limitation of non-integrated information systems is the difficulty of sharing of data between the systems. Our system differs from the previous systems in that it is completely integrated with the online dictation system and thereby results in minimal, if any, workflow interruption for the interventional radiologist, as QA event recording becomes an extension of the existing dictation process.19–21 More important, our system makes data entry mandatory and prevents the radiologist from ending the dictation and proceeding to the next dictation without completing the QA component. This step allowed all of the interventional procedures performed during the study period to be entered into the system. Since this database is linked to the institutional EMR system (ClinicStation), other patient- and procedure-related data are automatically acquired from existing information in ClinicStation, thus obviating the need for duplicate entries, as is required by some other database systems.19–21
Our system has the potential to be used in many types of QA and peer review programs within our department, and even for IR departments outside our own. Using the database, we can calculate the rate of major complications (overall and for different kinds of procedures) for the entire group and that for individual faculty members over a given time period. These rates can be compared with the thresholds described in SIR guidelines, and if indicated and appropriate, can lead to Quality Improvement initiatives, potentially resulting in better and safer patient care. The complication rates for individual faculty members can provide criteria to be used for peer review. We can also follow the trends in complications rates over a period of time to assess the usefulness of quality improvement initiatives.
Institution of the IR QA also allows radiologists to easily identify and retrieve cases for discussion in morbidity and mortality sessions. Before instituting the current IR QA program, we had to ask individual faculty members to voluntarily contribute such cases for analysis. With the current system, however, a quick query of the IR QA program can identify all complications during any given time period and thereby allow the peer review committee to select appropriate cases to be discussed.
This system’s design has its limitations. However, we have implemented or are in the process of implementing modifications to address those limitations to maximize the system’s usefulness. In our opinion, the entry of complications discovered after the initial dictation was not reported at 100% compliance, as evidenced by the fact that only 13% of all entries were made after the initial dictation. Although, we do not have evidence to validate this statement, the low proportion of late entries does suggest that many of the complications discovered after the initial dictation are not being captured. We feel that the most important reason for not entering data was that it required a voluntary effort by the radiologist to access a computer, log on to ClinicStation, find the appropriate IR procedure, and enter the complication. Although the IR QA is popular among radiologists, any process that involves interrupting routine clinical or academic activities is likely to achieve less than 100% participation. After the initial months, we implemented the automatic prompt of “please QA previous report. We have not yet evaluated whether this option has increased the rate of reporting of delayed events. Although this prompt reminds the physician to QA the previous procedure, it does not make it a mandatory step, as acknowledging the prompt allows the radiologist to end the current dictation. We plan to modify the current system so acknowledging the prompt will automatically lead the radiologist to the previous procedure. The radiologist will then be forced to make an entry into the dictation (either “none” or a complication, if indicated) before starting the next dictation.
In addition, we have recently modified the system to allow IR physician assistants to enter delayed complications through ClinicStation. As an extension of this plan, physician assistants will be prompted to enter complications events to previous IR procedures if indicated, whenever they dictate a follow-up note regarding IR patients. Our physician assistants are involved in routine post-procedure evaluation of inpatients and outpatients and are aware of delayed complications. We hope that this requirement will increase our capture rate of information about delayed complications.
Since we did not have any formal method of collection of complication events related to interventional procedures prior to the implementation of this program, we are not able to conclusively show that our system results in improved participation in the QA program. Another limitation is the fact that since this program is internally developed at our institution and not commercially available, this could limit broad applicability of the program by other departments. However, the goal of the study is to introduce the concept of integration of the QA step into the routine workflow of the interventional radiologist and use of force function that makes the data entry step a mandatory step. Similar integration concept could easily be applied to other commercially available QA programs (e.g., Hi-IQ) that will integrate the QA entry step to the routine workflow of the interventional radiologists and will force the individual to record the event.
In conclusion, the integration of IR QA reporting system into the workflow process and making completion mandatory has the potential to minimize the work required for data entry and to improve participation in the QA process compared to data entry requirements for stand-alone software programs. Further experience is necessary to identify the limitations of the system, and implement modifications to improve the usefulness of the QA program.
References
- 1.Barraclough BH, Birch J. Health care safety and quality: where have we been and where are we going? Med J Aust. 2006;184:S48–50. doi: 10.5694/j.1326-5377.2006.tb00362.x. [DOI] [PubMed] [Google Scholar]
- 2.Becher EC, Chassin MR. Improving the quality of health care: who will lead? Health Aff (Millwood) 2001;20:164–179. doi: 10.1377/hlthaff.20.5.164. [DOI] [PubMed] [Google Scholar]
- 3.Donnelly LF, Strife JL. Performance-based assessment of radiology faculty: a practical plan to promote improvement and meet JCAHO standards. AJR Am J Roentgenol. 2005;184:1398–1401. doi: 10.2214/ajr.184.5.01841398. [DOI] [PubMed] [Google Scholar]
- 4.Thrall JH. Quality and safety revolution in health care. Radiology. 2004;233:3–6. doi: 10.1148/radiol.2331041059. [DOI] [PubMed] [Google Scholar]
- 5.Walsh K, Antony J. Improving patient safety and quality: what are the challenges and gaps in introducing an integrated electronic adverse incident and recording system within health care industry? Int J Health Care Qual Assur. 2007;20:107–115. doi: 10.1108/09526860710731807. [DOI] [PubMed] [Google Scholar]
- 6.Deitch CH, Chan WC, Sunshine JH, Zinninger MD, Cascade PN, Cochran ST. Quality assessment and improvement: what radiologists do and think. AJR Am J Roentgenol. 1994;163:1245–1254. doi: 10.2214/ajr.163.5.7976910. [DOI] [PubMed] [Google Scholar]
- 7.Frank MS, Mann FA, Gillespy T. Quality assurance: a system that integrates a digital dictation system with a computer data base. AJR Am J Roentgenol. 1993;161:1101–1103. doi: 10.2214/ajr.161.5.8273618. [DOI] [PubMed] [Google Scholar]
- 8.Johnson CD, Krecke KN, Roberts RM, Denham C. Quality initiatives: developing a radiology quality and safety program: a primer. Radiographics. 2009;29:951–959. doi: 10.1148/rg.294095006. [DOI] [PubMed] [Google Scholar]
- 9.Kruskal JB, Yam CS, Sosna J, Hallett DT, Milliman YJ, Kressel HY. Implementation of online radiology quality assurance reporting system for performance improvement: initial evaluation. Radiology. 2006;241:518–527. doi: 10.1148/radiol.2412051400. [DOI] [PubMed] [Google Scholar]
- 10.Mahgerefteh S, Kruskal JB, Yam CS, Blachar A, Sosna J. Peer review in diagnostic radiology: current state and a vision for the future. Radiographics. 2009;29:1221–1231. doi: 10.1148/rg.295095086. [DOI] [PubMed] [Google Scholar]
- 11.Kruskal JB, Anderson S, Yam CS, Sosna J. Strategies for establishing a comprehensive quality and performance improvement program in a radiology department. Radiographics. 2009;29:315–329. doi: 10.1148/rg.292085090. [DOI] [PubMed] [Google Scholar]
- 12.Kruskal JB, Siewert B, Anderson SW, Eisenberg RL, Sosna J. Managing an acute adverse event in a radiology department. Radiographics. 2008;28:1237–1250. doi: 10.1148/rg.285085064. [DOI] [PubMed] [Google Scholar]
- 13.Guidelines for establishing a quality assurance program in vascular and interventional radiology. J Vasc Interv Radiol 14:S203-207, 2003 [PubMed]
- 14.FitzGerald R. Radiological error: analysis, standard setting, targeted instruction and teamworking. Eur Radiol. 2005;15:1760–1767. doi: 10.1007/s00330-005-2662-8. [DOI] [PubMed] [Google Scholar]
- 15.Renfrew DL, Franken EA, Jr, Berbaum KS, Weigelt FH, Abu-Yousef MM. Error in radiology: classification and lessons in 182 cases presented at a problem case conference. Radiology. 1992;183:145–150. doi: 10.1148/radiology.183.1.1549661. [DOI] [PubMed] [Google Scholar]
- 16.Soffa DJ, Lewis RS, Sunshine JH, Bhargavan M. Disagreement in interpretation: a method for the development of benchmarks for quality assurance in imaging. J Am Coll Radiol. 2004;1:212–217. doi: 10.1016/j.jacr.2003.12.017. [DOI] [PubMed] [Google Scholar]
- 17.Hahn PF, Lee MJ, Gazelle GS, Forman BH, Mueller PR. A simplified HyperCard data base for patient management in an interventional practice: experience with more than 4000 cases. AJR Am J Roentgenol. 1994;162:1443–1446. doi: 10.2214/ajr.162.6.8192015. [DOI] [PubMed] [Google Scholar]
- 18.Mayo-Smith WW, Jayaraman MV, Han RS, Dupuy DE, Movson JS. Multiinstitutional computer database for recording nonvascular imaging-guided interventions. AJR Am J Roentgenol. 2003;181:1491–1493. doi: 10.2214/ajr.181.6.1811491. [DOI] [PubMed] [Google Scholar]
- 19.McEnery KW, Suitor CT, Hildebrand S, Downs R. RadStation: client-based digital dictation system and integrated clinical information display with an embedded Web-browser. Proc AMIA Symp :561-564, 2000 [PMC free article] [PubMed]
- 20.McEnery KW, Suitor CT, Hildebrand S, Downs RL. Integration of radiologist peer review into clinical review workstation. J Digit Imaging. 2000;13:101–104. doi: 10.1007/BF03167636. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.McEnery KW, Suitor CT, Hildebrand S, Downs RL. Radiologist's clinical information review workstation interfaced with digital dictation system. J Digit Imaging. 2000;13:45–48. doi: 10.1007/BF03167623. [DOI] [PMC free article] [PubMed] [Google Scholar]