Panel 3: Particular Systems Issues

Testimony of Mark E. Bruley, ECRI

Introduction

I am Mark Bruley, Vice President for Accident and Forensic Investigation with ECRI, an independent, nonprofit health services research organization devoted to the assessment, planning, procurement, and management of health care technology. We have a special emphasis on medical devices and maintain laboratories where this technology is evaluated in a fashion similar to Consumer Reports. ECRI does not accept funding from the medical device or pharmaceutical industries, and functions at arm's length from them. The results of our research are used in the overwhelming majority of the nation's larger- and medium-sized hospitals, and in many healthcare institutions abroad. We are designated as an Evidence-based Practice Center by AHRQ and as a Collaborating Center of the World Health Organization. We have 250 employees, worldwide, and a large variety of programs to ensure patient safety. My particular expertise derives from investigating accidents and problems with medical devices in both the hospital and laboratory settings over the past 25 years.

ECRI's free Medical Device Safety Reports web site (http://www.mdsr.ecri.org) is a repository of medical device incident and hazard information independently investigated by ECRI. In regard to medical errors involving medical devices, ECRI's research draws on its expertise in evaluating medical devices in bench-scale laboratory tests, from formal, in-depth investigations of more than 2,000 adverse device events in the field at the incident institution and/or at ECRI's laboratories, and in investigating more than 25,000 medical device problem reports submitted to ECRI since 1971 by healthcare institutions through our international medical device problem reporting network.

As a biomedical engineer at ECRI, I have learned a great deal about the causes of accidents and problems and have been able to analyze many of the causes of these adverse events. However, I stress at the outset that the firm research base that would meet the standards that are expected from health services research does not exist related to medical errors involving healthcare technology. Therefore, one of my key messages in discussing a research agenda today is that we must develop that research base. Specifically, we do not have a "denominator" or baseline from which to launch truly meaningful studies. I will describe my approach to how we might begin to construct a denominator in the course of this presentation.

The lack of good quantitative data and research has not, and should not, stall the effort to improve equipment-related safety. The experiences I have had and the programs my organization has developed, have made an extremely strong contribution over the decades to improving patient safety. Qualitative studies allow us to understand how many of these accidents and adverse occurrences take place and to design corrective measures. I am proud of my organization's work, but unfortunately, we are likely to have a great deal more work to do in future. Let me give you an example of the qualitative thinking that I think is helpful. The 1999 Institute of Medicine (IOM) report on medical error cites an example of a free-flow problem with intravenous infusion pumps in the operating room (39). The example is highlighted throughout the text of the IOM report and is specifically used to show corrective actions that could have been taken by personnel and systems that could have been erected to avoid error. But what is missing from that example's analysis, is something more fundamental. The hospital should never have purchased an infusion pumps that permitted free flow. Recommendations have long been available from ECRI that would have guided an equipment acquisition process that would never have permitted this type of accident to occur.

These omissions in the IOM report example point to the complex nature of healthcare technology accident analysis and remediation and to the need for medical device accidents to be addressed in a research agenda on medical errors and patient safety.

Defining a Research Agenda

Medical device related errors may account for a minority of errors in medicine, but they are, nonetheless, a critical component in the development of a research agenda for improved patient safety. Because their analysis for causation reveals much about the genesis of medical error, user limitations, human factors, and system design, it is essential to increase the attention given to the safe design, use, and management of medical devices. The healthcare technology research agenda must obviously extend beyond the medical error discussions and debates that, so far, have generally limited their focus to the ubiquitous example of "medication errors with infusion devices."

More research is needed to provide baseline or denominator data for studying medical device-related errors as has been put done to great positive effect already in, for example, anesthesia management (13, 14, 15, 28, 33, 34, 51, 57), hospital pharmacy and medication delivery (3, 5, 12, 31, 32, 38, 40, 59), and cardiopulmonary bypass perfusion (4, 18, 36, 43, 44). Although equipment failures, per se, are rare in these areas, most of the related medical errors occurring in these settings do involve the use, or mis-use, of a medical device. The human-machine interface in the surgical setting, and in particular in the practice of anesthesia, has been analogized to that of pilot and airplane (45, 51). As in the aviation industry, it is important to study these events to identify and examine correlations between contributing factors before hypotheses about causation can be formulated and tested. This is the first step in preventing medical errors involving adverse device events and the patient injuries associated with them.

Virtually every aspect of medical care, diagnosis, treatment, and therapy involves the use of multiple technologies, ranging from simple needles and syringes to complex computer-based monitoring or diagnostic systems. The healthcare setting is replete with medical devices, many of which have caused or contributed to numerous patient deaths and injuries, and countless potential injuries or "near misses." (See, in general, the hundreds of thousands of Medical Device Reports collected by the U.S. Food and Drug Administration, Rockville, MD; see also Health Devices Alerts published by ECRI, Plymouth Meeting, PA.) Indeed, the public safety concerns on medical devices have spurred a plethora of U.S. legislative and regulatory requirements, including a mandatory reporting scheme for medical device deaths and injuries (Sections 519(a), (b) of the Federal Food, Drug, and Cosmetic Act, as amended, 21 U.S.C. 360i.(54)).

These issues are especially acute in technology intensive medical specialty settings where one would expect accidents or adverse events and the device-related research agenda should concentrate, naturally enough, on these specialty areas. These areas include intensive care medicine, emergency medicine, obstetrics and gynecology, general surgery, cardiac catheterization, clinical laboratory, and respiratory therapy. A research agenda focused on these technology-intensive medical specialties will provide specialty-specific data of accident and error causes, i.e., denominator data, that is necessary for undertaking meaningful studies of patient safety. Broad-based research approaches are not likely to be effective.

While most medical errors have multiple causes, there are three "constants" in any such accident: the medical device or technology, the procedure being performed, and the injury that results. It is in the areas where these constants intersect (i.e., in the correlations between them) that the causes of accidents are discovered and from which the recommendations to prevent future incidents are derived. Within the context of the high risk medical specialty settings, ECRI believes that research, constructed on the narrower of the two 1999 Institute of Medicine (IOM) report definitions related to medical errors, i.e., "accidents" and "adverse events" (39), should be targeted on investigating these relationships by addressing the following key questions related to healthcare technology:

1. To what extent do medical devices and related information systems contribute to medical errors (including medication errors), especially within the technology intensive medical specialties?
2. What engineering controls, including those based on continuing human factors research, can be employed to minimize the likelihood of medical errors?
3. Does the frequency and severity of device-related error justify developing broader reporting and prevention interventions?

Constructing a research agenda on "medical error" or "patient safety" related specifically to medical devices and these posed questions will depend on the scope given to these terms. The 1999 IOM report does raise some medical device related issues. It concentrates on preventing "adverse events" and accidents, while at the same time it defines medical error and patient safety more broadly to include overuse, underuse, and misuse of technology. The report also focuses heavily on medication errors, but with attention only to medical devices that may play a role in these types of adverse events—it does not devote attention to the non- medication related device arena. We propose here that it is necessary to broaden that perspective. Following is a discussion of these research agenda questions along with related issues.

Research to Date

There has been little in the way of formal, published investigation of adverse device events. As noted above, there have been critical incident studies looking at anesthesia management failures and errors (13, 14, 15, 21, 28, 33, 34, 51, 57), some of which deal with adverse device events. Beyond this, there have been several studies on adverse events in general. Retrospective medical record review has been used to identify the incidence of adverse events, the epidemiology of adverse events, and potential quality improvement or risk reduction methods and techniques (5, 7, 8, 47, 48, 53, 61). Certain hospital characteristics have been associated with the increased risk of adverse events or adverse events due to negligence, e.g., ownership, discharges, minority population, payer mix, operating costs (8, 11). Prospective observational studies have also been conducted (2) in this area.

Studies of closed malpractice claims provide data on adverse events in general, but little on adverse device events. The American Society of Anesthesiologists (ASA), for example, has analyzed closed malpractice claims since 1985 (35, 42). Other studies have examined closed claims in other specialties (41). Several professional liability carriers conduct proprietary closed claims studies with their own data. The methodology for closed claims studies has some common elements. For example, reviewers examine preexisting files related to a malpractice case, which vary in scope from a few pages to several hundred. They complete data collection forms that include objective factors (i.e., patient demographics, type of anesthetic agent) and subjective factors (i.e., was the appropriate standard of care met, would better monitoring have prevented the event). The abstracted data are examined for accuracy and then analyzed. However, the purpose of closed claims studies is to identify areas of liability for a particular specialty and, in some studies, to determine the extent to which substandard care contributes to injury (35). Future studies on medical error with healthcare technology should focus instead on causes and errors, not allegations of liability.

Research Agenda Question 1

To what extent do medical devices and related information systems contribute to medical errors (including medication errors), especially within the technology intensive medical specialties?

Despite widespread concern and the implementation of government- mandated reporting systems, there has been little in the way of formal published investigation of adverse device events. Although FDA occasionally issues safety alerts, it does not provide the user community with any feedback on device evaluations. Nor does it provide reports of trends or patterns from its database (GAO. Medical Device Reporting. Improvements Needed in FDA's System for Monitoring Problems With Approved Devices. January 1997.(50)) An understanding of the causes of medical device accidents and the errors and failures underlying them, is the key to prevention—learning a good lesson from a bad event.

The number of medical devices is highest in the areas of medical specialty mentioned above, in which a single procedure can involve dozens or hundreds of devices, components, and accessories—both disposable and reusable devices. In these special healthcare areas, there is a complex interplay among person and machine, the behaviors of individuals and specialty care teams, and organizational structures and processes. Research on medical error should establish an evidence base, similar to that done in anesthesia, for medical technology used in these areas of clinical specialty, and other areas where one would expect to find accidents or adverse events involving the use of medical devices, equipment, and systems. Doing so will further help establish denominator data.

Most adverse device events occur from multiple causes and/or devices. For example, there is likely a strong positive correlation between: 1) the use of electrosurgical units during head and neck surgery, and 2) surgical fires (17, 23, 26, 27). In such cases, it is unlikely that the electrosurgical unit or the anesthesia equipment malfunctioned, but it is likely that the interaction of the electrosurgical spark, flammable surgical materials, and patient hair in the presence of the oxygen delivered from the anesthesia machines is a primary cause of many such accidents. [There are other third-order reasons why the spark resulted in ignition of patient hair or surgical drapes, such as the inadequate understanding or dissemination of fire prevention principles.] Knowledge of multiple causes is likely to induce more lasting change (46).

The results of research into this question will also allow development of recommendations that would reduce the likelihood of "design" deficiencies in the broad sense—for both the user community and the vendor community. Users, for example, may need to improve procurement or other management processes for specific devices. Manufacturers, for example, may need to improve human factors or labeling. Medical and nursing schools will likely need to develop curricula that address the fundamental physics and mechanics of medical technology—a process we have pursued for three decades and which medical educators still ignore.

Question 1 and the Denominator Problem

Denominator data is generally lacking related to medical errors with medical devices, with the exception of a few specific areas that have been well studied over the past two decades, such as anesthesia and cardiopulmonary bypass perfusion. Medical device accidents are uniquely complex to investigate. General estimates suggest that up to 90% of all errors in medicine are caused by human error (6), with human error in medical device accidents accounting for 50-70% of all device related accidents (52). However, at this time researchers cannot readily go to the published medical literature to determine denominator data related to medical errors involving medical devices. Lacking are data relating to the incidence of injuries per device type, the number of mishaps per the number of patient's treated with a particular device type, or the number of injuries related to the number of uses of a device.

Lacking good denominator data, it is difficult to estimate the percentage of medical errors that involve healthcare technology. At ECRI, giving consideration to our databases and our detailed knowledge of the FDA medical device problem reporting databases (MDR and MAUDE), our guarded impression is that approximately 5% of all medical errors involve a medical device, system, or technology as a contributing factor to the error. Clinicians, regulatory personnel, and other researchers with whom we have discussed this issue are similarly cautious in ascribing percentages, but, even lacking data, believe that the percentage is significant enough to warrant detailed study.

Within the studies of anesthesia mishaps, human error, equipment failure, and disconnections were specifically studied (14, 15). The percentages of accidents involving actual equipment failure has been consistently low, from 4-14% in these studies. However, review of these study's data reveals that the percentage of user error mishaps involving equipment is much higher, ranging from 59-62%. These figures exclude the reported accidents from device disconnections (e.g., breathing circuits and intravenous medication lines), which, if included, would raise the percentage of medical errors in anesthesia that involve medical devices even higher. Perspectives that dismiss research into medical error based on the observation that patient injuries related to equipment failures in certain specialties are rare (such as anesthesia) miss the point that valid research of medical errors related to devices should not focus on device failures, per se, but on medical errors related to device use and attendant user errors. A dismissive perspective also, unfortunately, suggests that such patient injuries and deaths are not worthy of detailed prospective or retrospective study.

Although denominator data is very scant, research into medical errors involving medical devices is warranted, but on a focused basis, either by medical specialty or, in some cases, by technology type, recognizing that the majority of medical device accidents can be attributed to user error (37, 52). Such focused research in this area will help establish denominator data for the development appropriate preventive recommendations and for the assessment of the assessment of their effectiveness.

Analysis of device related mishaps in the demanding environment of the intensive care setting has yielded some limited data (1, 18, 20) suggesting that 66% of equipment related medical errors in the ICU were caused by user error. When considering the clinical relevance of these findings, it is clear that the perception that "engineers perceive theoretical hazards," or that the serious device related accidents reported to ECRI or FDA are "unique, rare, exceptional events," are claims to be rejected. The reality is quite different, but requires research to better quantify it.

Question 1 and Human Factors Research

The study of system failures and human factors has, over the past decade, been performed with greater frequency in the medical context (6, 9, 22, 34, 37, 46, 49, 55, 56, 58). Considerable work has already been done in this arena, much to the benefit of patient safety, and the study of the user- machine interface continues. However, medical error with medical devices involves much more than the study of human factors engineering criteria for the device design. Unfortunately, the focus on healthcare technology has, as seen in the IOM report, generally been limited to medication errors with infusion technologies. Future research into medical errors that occur with the use of medical devices must be extend beyond the bounds of human factors research, as suggested in the research questions posed herein.

In regard to the medication error issues contained within this question, the human factors issues related to infusion pump design are very important, but the extent to which these pumps, or other related devices and medication order systems, contribute to medication errors is not known. There is no denominator data because no one has researched the topic. With denominator data, the healthcare community could perform safety audits with a medication error reduction focus. From those audits, the development of recommended policies and procedure related solutions could proceed, followed by education to hospitals and, ultimately, the drafting of best practice guidelines. However, this approach hinges on being able to derive sound data.

The answers to the research questions posed here will facilitate many patient safety initiatives on the part of users, regulators, vendors, and other stakeholders. Manufacturers of medical devices would better anticipate, during the product design process, the errors and injuries related to medical devices and, thus, incorporate appropriate engineering, design, and safety controls to minimize the risk of common errors. For example, a number of studies have shown a high correlation between user error and mishaps with anesthesia technology (13, 14, 15, 28, 33, 34, 51, 57). Consequently, over the past 15 years, manufacturers of anesthesia machines and related perioperative physiologic monitors used by anesthesia personnel have incorporated significant human factors considerations into the design of devices that protect against user error. For the user community, such studies have led to the development of improved administrative guidelines and enhanced organizational structures within the anesthesia work environment (16).

Research Agenda Question 2

What engineering controls, including those based on continuing human factors research, can be employed to minimize the likelihood of medical errors?

Innovation in safety features and engineering controls in medical device technology is driven in part by the feedback that device manufacturers receive from clinicians, not only during the product development testing phase but also after FDA marketing approval. Unfortunately, many hazards that could be mitigated by safety features or engineering controls are not identified until the post-market surveillance phase, subsequent to FDA clearance, i.e., when adverse events have occurred and patients have suffered injury. The answers to this question will serve as additional input for medical device manufacturers seeking to prospectively identify device- specific classes of hazards and to "design them out" before their devices are marketed.

An example of a rapidly developing technology that holds promise in this area in the near future is the computerized physician order entry system (CPOES). In the past 5-15 years, error reducing technologies placed into service over the past 5-15 years have played a large role in the reduction of errors in a few areas of medical specialty such as anesthesia, pharmacy, surgery, cardiopulmonary bypass perfusion, and radiation oncology. Some of the devices and technologies that have been developed and placed into clinical service in this regard are:

• Anesthesia agent monitors
• Capnographs
• Pulse oximeters
• Computerized Physician Order Entry systems (CPOES)
• Drug compounders used for automated mixing of IV solutions and parenteral solutions in the hospital pharmacy
• Electrosurgical unit return electrode monitors (REM)
• Membrane-based cardiopulmonary oxygenators
• Extracorporeal blood flow, blood level, and foam detectors
• Radiologic quality control devices: real-time monitors of delivered dose from linear accelerators.

The development of each of these technologies addressed a specific patient safety need within that specialty. It is indicated that research into errors with technology be targeted within the specialties mentioned above order to be effective. Broad-based research approaches are not likely to be effective.

Recommendations for patient safety developed from the research in the area anesthesia management have not been the only success. Systems- based approaches to the management of healthcare technology have been developed and implemented (28, 29, 30). These technology and risk management related strategies have helped ensure hospital selection of equipment appropriate for the use environment, maintained equipment in proper working order, and rapidly alerted clinicians and administrators to devices subject to recall or hazard notifications.