Panel 2: Broad-based Systems Approaches

Testimony of David Woods, Past President, Human Factors and Ergonomics Society (Continued)

Select for Part 1 of Testimony

One of the most productive areas of work on human error in the last 10 years has been about reactions to failure (Woods et al., 1994; Rochlin, 1999). In this work we have come to understand more about what characterizes high reliability organizations (Rochlin et al., 1987; Weick and Roberts, 1993; Westrum, 1993; Grabowski and Roberts, 1997) and about the common oversimplifications or fallacies about error that block forward progress (Cook et al., 1998).

High reliability organizations create safety by anticipating and planning for unexpected events and future surprises. These organizations continue to invest in anticipating the changing potential for failure, regardless of past success, because they appreciate that their knowledge is imperfect and that their environment continues to change. The heart of this process is learning activities which depend on open flow of information about the changing threats and about the changing effectiveness of their failure sensitive strategies. For these organizations, safety is a value, not a commodity.

Escape from Hindsight Bias

There are a variety of factors that block or inhibit the learning processes central to a high reliability culture. One is the hindsight bias (Fischhoff, 1975; Woods et al., 1994; Woods and Cook, 1999). The hindsight bias is one of the most reproduced research findings relevant to accident analysis and reactions to failure. Knowledge of outcome biases our judgment about the processes that led up to that outcome.

In the typical study, two groups of judges are asked to evaluate the performance of an individual or team. Both groups are shown the same behavior; the only difference is that one group of judges are told the episode ended in a poor outcome; while other groups of judges are told that the outcome was successful or neutral. Judges in the group told of the negative outcome consistently assess the performance of humans in the story as being flawed in contrast with the group told that the outcome was successful. Surprisingly, this hindsight bias is present even if the judges are told beforehand that the outcome knowledge may influence their judgement.

Hindsight is not foresight. After an accident, we know all of the critical information and knowledge needed to understand what happened. But that knowledge is not available to the participants before the fact. In looking back we tend to oversimplify the situation the actual practitioners faced, and this tends to block our ability to see the deeper story behind the label human error.

Researchers use methods designed remove hindsight bias to see the multiple factors and contributors to incidents, to see how people usually make safety in the face of hazard, and to see systemic vulnerabilities before they contribute to failures. Research has developed a variety of techniques to reduce hindsight bias. These are available to used and modified as necessary in health care research on safety.

Despite widespread communication about this bias, it continues to plague the literature on medical error. The studies of injury or death rates as a result of error and virtually all incident review procedures used in health care today fail to control for hindsight bias. This should not be considered acceptable by any one interested in improving safety. It is time to stop repeating this "error" in the study of error.

Innovate New Models of Accountability

As the new messages about a systems approach to safety circulated and became more visible in health care, they collided with the common belief that practitioners and managers should be "accountable" to patients and to other stakeholders. But accountability in this common belief is operationalized in terms of pursuit of culprits, threats of disciplinary actions, and threats of stigmatization.

These two trends collide in a basic double bind for the patient safety movement: blame, even if disguised as accountability, drives out information about systemic vulnerabilities, stops learning, and undermines the potential for improvement (Billings, 2000). The challenge for research is to find a way out of the double bind—How do we create a safe environment for learning about the potential for failure in a publicly accountable system of health care delivery?

Accountability is emphasized in the debate on patient safety because how decision makers are held accountable is presumed to influence how they make decisions and the quality of those decisions. Social links such as accountability can indeed be powerful forces influencing human decision making, and these relationships have been studied in organizational dynamics, social cognition, and human-machine interaction (e.g., Hirschhorn, 1993; Lerner and Tetlock, 1999; Tetlock, 2000) This information can be integrated with ethical and legal scholarship to stimulate the innovation of new systems for managing accountability relationships (Sharpe, 2000; Palmer et al., 2000).

Practitioner decision making always occurs in a context of expectations that one may be called to give accounts for those decisions to different parties. How and to whom people expect to be called to account affects their performance in implicit and explicit ways. The expectations for what are considered adequate accounts and the consequences for people when their accounts are judged inadequate are a critical part of the cycle of giving accounts and being called to account.

Interestingly, different factors in this reciprocating cycle can support or undermine practitioner performance and systems learning in predictable ways. Note that from a behavioral science point of view, accountability is a neutral term that only points to the processes in this cycle of giving and being called to give accounts or reasons for a decision.

First, past research shows that there are a complex set of factors, relationships and effects at work in the reciprocating cycle of calling on and giving of accounts. Second, the empirical regularities and relationships are not consistent with motivational accounts, i.e., that accountability creates general improvements by increasing task motivation. Third, and most startling, the research demonstrates that some factors in the reciprocating cycles of accountability may degrade decisions, performance, cooperation and learning, while other relationships in the cycle may enhance these cognitive processes.

For example, under some conditions the need to give an account for a decision to others can increase critical thinking and attenuate commitment (presumably ways to enhance the decision), while other conditions can increase self justification, bolster an initial attitude and commitment (presumably ways that reduce the quality of a decision). Some forms of accountability can increase defensive behavior, create adversarial relationships between parties who need to cooperate, or lead people to prefer options that are easier to justify given knowledge of the standards others impose for giving of suitable accounts.

These results allow us to see the label "culture of blame" in a new way. It is a kind of system of accountability, but only one way to design and manage such systems. If one analyzes a "culture of blame" in terms the dynamics of cycles of accountability, we find many of the factors that have been implicated in degrading performance, cooperation, and learning.

The social cognition research on cycles of accountability clearly captures the complexity of the effects and demonstrates the naïveté of the belief that improving safety only requires holding others accountable. The slogan of "moving beyond a culture of blame" is a call to abandon poor systems of accountability, not an absence of accountability. The systems approach does not mean abandoning accountability. It is a necessity part of our life as social creatures that we explain our actions to others. The systems approach examines the reciprocating cycle of giving accounts and calling to accounts between the sharp end of practice and the blunt end of organizational context to determine the lawful effects of different systems for accountability.

New work is needed to model and describe systems of accountability and their effects in health care. The results from this work should be used to engage all stakeholders in a process to explore new designs of systems of accountability that will produce the desired effects and advance our common goals.

Create and Share Learning Tools

The research on high reliability organizations emphasizes continued learning about risks and mitigation strategies. But in observations of health care, direct learning and improvement from experience with accidents and incidents has proven to be very limited and narrow. This appears to be partly because of the fear of blame and litigation. In addition, there are few organizational structures that promote learning about paths to failure.

An important area for new work is creating learning tools that function throughout health organizations. Some believe that expanded requirements for notification of adverse events and near misses will accomplish this. But lessons from aviation indicate that much more is needed in the analysis of cases and in feedback mechanisms to health care practitioners to complete a cycle of learning based on incident reporting (Billings, 1999). New incident reporting systems will need to encompass much more than new forms to fill out and new notification requirements to be effective. Serious questions remain about whether these programs will be effective including: the independence of the review teams, the need to move beyond looking at cases one at a time to analyzing sets of cases, how to generate meaningful sustained improvements, and how to provide feedback to practitioners about how what was learned is relevant to them in their portion of the world of health care.

One mechanism to enhance learning about risks of adverse events is the creation of regional centers like the Veterans Health Administration’s Centers for Inquiry to collaborate with multiple health care systems. These centers can be incubators to develop new techniques for learning and sharing the lessons from cases including role play simulations, walkthroughs of cases, and other techniques. These centers can provide the infrastructure to build partnerships between human performance and health care experts to promote the information flow and learning processes central to a culture of safety.

Interdisciplinary Partnerships

The success of NASA programs in aviation safety were built on interdisciplinary partnerships. The success of safety initiatives in health care are very likely to depend to building the same kinds of partnerships across quite different technical disciplines. Considering patient safety inevitably leads to technical issues about human performance, human-computer cooperation, organizational dynamics, software engineering and other fields normally considered outside health care. The new research efforts will need to be structured to build up these partnerships between technical areas concerned with different aspects of human performance and different medical and practitioner specialties. One important activity will be to develop a new cadre of experts who are skilled at these interdisciplinary projects (for example, programs that invest in supporting dissertation research).

The success of NASA programs was based on a portfolio of research and development activities that included more basic work on human performance (e.g., mental workload), innovation of new directions for aiding human performance (e.g., cockpit resource management), advanced development, and technology transfer projects. In this process different kinds of work were carried out and sponsored including field research, simulator studies, concept development, evaluation studies. The infrastructure at NASA helped cross-stimulate all of these activities around the goal of developing new design directions that would be useful in improving aviation safety. The structure at NASA and NASA’s role in the aviation industry provided another essential ingredient for success—independence. The issues underlying safety are potentially controversial. A technically grounded, independent organization whose only purpose is advancing safety can develop a reservoir of technical results and organizational confidence to ride through these controversies with their substantive efforts for safety intact.

Health care would do well to study the formal and informal organizational basis of the successes of the NASA R&D program and to model their own research efforts on patient safety on NASA’s programs.

We have a window of opportunity for improving safety for patients, but there are many false trails that will consume the energy and resources available. To take advantage of this window we must be prepared to question conventional wisdom and assumptions by building a partnership between different health care specialties and different human performance specialties that intersect at the label human error. These human performance specialties are substantive, deep, and unfamiliar to health care. I have not been able to cover all of them in this piece. But they are the wellspring for techniques, concepts, and systems that will improve human performance in health care as has been the case in other high risk domains.

From the past work in Human Factors a simple standard emerges for judging success in research on error and safety. Research is successful to the degree that it helps recognize, anticipate, and defend against paths to failure that arise as organizations and technology change, before any patient is injured.

Barley S and Orr J, Eds. (1997). Between Craft and Science: Technical Work in U.S. Settings.

Billings CE (1996). Aviation automation: The search for a human-centered approach. Mahwah, NJ: Erlbaum.

Billings CE (1999). The NASA Aviation Safety Reporting System: Lessons Learned from Voluntary Incident Reporting. In Proceedings of Enhancing Patient Safety and Reducing Errors in Health Care. National Patient Safety Foundation, Chicago IL (held at Annenberg Center for Health Sciences, Rancho Mirage CA Nov. 8-10, 1998).

Billings CE (2000). Human Error in Perspective.

Bogner MS, Ed. (1994). Human Error in Medicine, Mahwah, NJ: Erlbaum.

Carroll JM and Rosson MB (1992). Getting around the task-artifact cycle: How to make claims and design by scenario. ACM Transactions on Information Systems 10, 181-212.

Clark HH and Brennan SE (1991). Grounding in communication. In L. Resnick, J. Levine, and S. Teasley. (Eds.) Socially Shared Cognition. Washington DC: American Psychological Association.

Cook RI, McDonald JS, and Smalhout R (1989). Human error in the operating room: Identifying cognitive lock up. Cognitive Systems Engineering Laboratory Technical Report 89-TR-07, Columbus, OH: Department of Industrial and Systems Engineering, The Ohio State University.

Cook RI, Woods DD, and Howie MB (1992). Unintentional delivery of vasoactive drugs with an electromechanical infusion device. Journal of Cardiothoracic and Vascular Anesthesia 6:1-7, 1992.

Cook RI and Woods DD (1996). Implications of automation surprises in aviation for the future of total intravenous anesthesia (TIVA). Journal of Clinical Anesthesia 8:29s-37s.

Cook RI, Woods DD, Miller C (1998) A Tale of Two Stories: Contrasting Views on Patient Safety. National Patient Safety Foundation, Chicago IL, April 1998 (www.npsf.org/exec/report.html).

Cook RI, Render ML, and Woods DD (2000). Gaps in the continuity of care and progress on patient safety. British Medical Journal 320, 791-94, March 18, 2000.

Dekker S and Woods DD (1999). Extracting Data from the Future: Assessment and Certification of Envisioned Systems. In S. Dekker and E. Hollnagel (Eds.), Coping with Computers in the Cockpit. Ashgate, pp. 7-27.

Dekker S (in press). The Field Guide to Human Error. Cranfield University Press.

De Keyser V (1992). Why field studies? In M.G. Helander & M. Nagamachi (eds.) Design for Manufacturability, London: Taylor & Francis.

De Keyser V and Woods DD (1990). Fixation errors: Failures to revise situation assessment in dynamic and risky systems. In A.G. Colombo and A. Saiz de Bustamante, editors, Systems Reliability Assessment, Kluwer Academic Publishers, Dordrechts, The Netherlands, p.231-52.

De Keyser V and Samercay R (1998). Activity theory, situated action and simulators. Le Travail Humain 61(4), 305-12.

Dominguez C, Flach J, Lake P, McKellar D, & Dunn M (in press). The conversion decision in laparoscopic surgery: Knowing your limits and limiting your risks. In J. Shanteau, K. Smith, & P. Johnson (Eds.), Psychological explorations of competent decision making. New York: Cambridge University Press.

Ericsson KA and Simon HA. Protocol Analysis: Verbal Reports as Data. MIT Press, Cambridge MA, 1984.

Feltovich P, Ford K, and Hoffman R, Eds. (1997). Expertise in Context. Cambridge MA: MIT Press.

Fischhoff B (1975). Hindsight ≠ foresight: The effect of outcome knowledge on judgment under uncertainty. Journal of Experimental Psychology: Human Perception and Performance 1(3), 288-99.

Flach JM and Dominguez CO (1995). Use-Centered Design: Integrating the User, Instrument, and Goal. Ergonomics in Design July.

Flanagan JC (1954). The critical incident technique. Psychological Bulletin 51(4), 327-58.

Gaba DM, Maxwell MS, and DeAnda A (1987). Anesthetic mishaps: Breaking the chain of accident evolution. Anesthesiology 66, 670-76.

Gaba DM, Howard SK, and Jump B (1994). Production pressure in the work environment: California anesthesiologists’ attitudes and experiences. Anesthesiology 81, 488-500.

Galegher J, Kraut R, and Egido C, Eds. (1990). Intellectual teamwork: Social and technical bases of cooperative work, Hillsdale, NJ: Erlbaum.

Grabowski M and Roberts KH (1997). Risk mitigation in large-scale systems: Lessons from high reliability organizations. California Management Review 39(4), Summer.

Greenbaum J and Kyng M, Eds. (1991), Design at Work.: Cooperative Design of Computer Systems. Hillsdale, NJ: Lawrence Erlbaum Associates.

Grudin J (1994). Computer-supported cooperative work: History and focus. IEEE Computer 27(5), 19-27.

Guerlain S, Smith PJ, Obradovich JH, Rudmann S, Strohm P, Smith J, and Svirbely J (1996). Dealing with brittleness in the design of expert systems for immunohematology. Immunohematology 12, 101-7.

Guerlain S, Smith PJ, Obradovich J, Heintz, Rudmann S, Strohm P, Smith JW, Svirbely J, and Sachs L (1999). Interactive Critiquing as a Form of Decision Support: An Empirical Evaluation. Human Factors 41(1), 72-89.

Heath C and Luff P (1992). Collaboration and Control: Crisis Management and Multimedia Technology in London Underground Line Control Rooms. Computer-Supported Cognitive Work 1, 69-94.

Hendee W, Ed. (1999). Proceedings of Enhancing Patient Safety and Reducing Errors in Health Care. National Patient Safety Foundation, Chicago IL (held at Annenberg Center for Health Sciences, Rancho Mirage CA Nov. 8–10, 1998).

Hirschhorn L (1993). Hierarchy vs. bureaucracy: The case of a nuclear reactor. In KH Roberts (Ed.), New challenges to understanding organizations. New York: McMillan.

Hoffman R and Woods DD (2000). Special Section on Studying Cognitive Systems in Context. Human Factors 42(1).

Hollnagel E (1993). Human Reliability Analysis: Context and Control. London: Academic Press.

Howard SK, Gaba DM, Fish KJ, Yang GS, and Sarnquist FH (1992). Anesthesia crisis resource management training: teaching anesthesiologists to handle critical incidents. Aviation, Space, and Environmental Medicine 63, 763-70.

Howard SK, Smith BE, Gaba DM, and Rosekind MR (1997). Performance of well-rested vs. highly-fatigued residents: A simulator study. Anesthesiology A-981.

Hutchins E (1995). How a cockpit remembers its speeds. Cognitive Science 19, 265-88.

Jones PM (1995). Cooperative work in mission operations: Analysis and implications for computer support. Computer Supported Cooperative Work 3, 103-45.

Jordan B and Henderson A (1995) Interaction Analysis: Foundations and Practice. The Journal for the Learning Sciences 4(1), 39-103.

Klein G (1998). Sources of Power: How People Make Decisions. MIT Press, Cambridge MA.

Klein G, Orasanu J, Calderwood R, and Zsambok C, Eds. (1995). Decision Making in Action: Models and Methods. Norwood, NJ: Ablex.

LaBerge D (1995). Attentional Processing: The Brain’s Art of Mindfulness. Cambridge, MA: Harvard University Press.

Layton C, Smith PJ, and McCoy CE (1994). Design of a Cooperative Problem-Solving System for En-Route Flight Planning: An Empirical Evaluation. Human Factors 36:94-119.

Leape LL, Woods DD, Hatlie MJ, Kizer KW, Schroeder SA, and Lundberg GA (1998). Promoting Patient Safety by Reducing Medical Errors, JAMA 280, 1444-47, October 28, 1998 (editorial)

Lerner JS and Tetlock PE (1999). Accounting for the Effects of Accountability. Psychological Bulletin 125, 255-75.

Lin L, Isla R, Doniz K, Harkness H, Vicente K, Doyle D (1998). Applying Human Factors to the design of medical equipment: Patient controlled analgesia. Journal of Clinical Monitoring 14, 253-63.

Malin JT, Schreckenghost DL, Woods DD, Potter SS, Johannesen L, Holloway M, and Forbus KD (1991). Making Intelligent Systems Team Players (NASA Technical Memorandum 104738). NASA Johnson Space Center: Houston, TX.

Nielsen J (1993). Usability Engineering. Boston Academic Press.

Norman DA and Draper SW, Eds. (1986). User Centered System Design: New Perspectives on Human-Computer Interaction. Hillsdale, N.J.: Lawrence Erlbaum Associates.

Norman DA (1988). The Design of Everyday Things. New York, Basic Books.

Norman DA (1990). The ‘problem’ of automation: Inappropriate feedback and interaction, not ‘over-automation.' Philosophical Transactions of the Royal Society of London B 327, 585-93.

Norman DA (1993). Things that Make us Smart. Addison-Wesley: Reading, MA.

Nyssen AS, and De Keyser V (1998). Improving training in problem solving skills: Analysis of anesthetists’ performance in simulated problem situations. Le Travail Humain 61(4), 387-402.

Nyssen AS and Javaux D (1996). Analysis of synchronization constraints and associated errors in collective work environments, Ergonomics 39, 1249-64.

Obradovich JH and Woods DD (1996). Users as designers: How people cope with poor HCI design in computer-based medical devices. Human Factors 38(4), 574-92.

Palmer L, Emanuael L, and Woods DD (2000). Managing Systems of Accountability for Patient Safety. National Patient Safety Foundation, in progress.

Rasmussen J (1986). Information Processing and Human-Machine Interaction. New York: North Holland.

Rasmussen J (1990). The role of error in organizing behavior. Ergonomics 33: 1185-99.

Rasmussen J (1999). The concept of human error: Is it useful for the design of safe systems in health care? In Vincent C, deMoll B. (Eds.) Risk and Safety in Medicine. London: Elsevier.

Rasmussen J, Brehmer B, and Lepat J, Eds. (1991). Distributed Decision Making: Cognitive Models for Cooperative Work. Chichester, England: Wiley.

Rensink RA, O’Regan JK, and Clark JJ (1997). To see or not to see: The need for attention to perceive changes in scenes. Psychological Science 8(5), 368-73.

Rochlin GI (1999). Safe operation as a social construct. Ergonomics 42(11), 1549-60.

Rochlin GI, La Porte TR, and Roberts KH (1987). The self-designing high-reliability organization: Aircraft carrier flight operations at sea. Naval War College Review Autumn, 76-90.

Roth EM, Malin JT, and Schreckenghost DL (1997). Paradigms for intelligent interface design. In M. Helander, T. K. Landauer, and P. Prabhu (Eds.), Handbook of Human-Computer Interaction (2nd ed.), North-Holland, pp. 1177-1201.

Sanders EB-N (2000). Generative Tools for Co-Designing. In Scrivener, Ball and Woodcock (eds.) Collaborative Design. London: Springer-Verlag.

Sarter NB, Woods DD, and Billings CE (1997). Automation Surprises. In G. Salvendy (Ed.), Handbook of Human Factors/Ergonomics (2nd ed.). New York, NY: Wiley.

Senders JW On patient injury and death stemming from the design of medical devices. Report, 12 September, 1993

Sharpe VA (2000). Behind Closed Doors: Accountability and Responsibility in Patient Care. Journal of Medicine and Philosophy 25, 28-47.

Smith P, Woods D, McCoy E, Billings C, Sarter N, Denning R, and Dekker S (1998). Using forecasts of future incidents to evaluate future ATM system designs. Air Traffic Control Quarterly 6(1), 71-85.

Stanton N (1994). Human Factors of Alarm Design, Taylor & Francis, London.

Tetlock PE (2000). Cognitive biases and organizational correctives: Do both disease and cure depend on the politics of the beholder? Administrative Science Quarterly 45, 293-326.

Vicente K (1999). Cognitive Work Analysis: Toward, Safe, Productive, and Healthy Computer-Based Work. Mahwah, NJ: Erlbaum.

Weick KE and Roberts KH (1993). Collective mind and organizational reliability: The case of flight operations on an aircraft carrier deck. Administration Science Quarterly 38, 357-81.

Westrum R (1993). Cultures with requisite imagination. In J. A. Wise, V. D. Hopkin and P. Stager (Eds.), Verification and validation of complex systems: Human factors issues. Springer-Verlag: Berlin. NATO ASI Series.

Winograd T and Woods DD (1997). Challenges for Human-Centered Design. In Human-Centered Systems: Information, Interactivity, and Intelligence. J. Flanagan, T. Huang, P. Jones, S. Kasif, (Eds.), National Science Foundation, Washington DC, July 1997.

Woods DD, Johannesen L, Cook RI, and Sarter NB (1994). Behind Human Error: Cognitive Systems, Computers and Hindsight. Crew Systems Ergonomic Information and Analysis Center, Wright Patterson Air Force Base, Dayton OH.

Woods DD, Cook RI, and Billings CE (1995). The impact of technology on physician cognition and performance. Journal of Clinical Monitoring 11, 92-95.

Woods DD and Cook RI (1999). Perspectives on Human Error: Hindsight Bias and Local Rationality. In F. Durso (Eds.), Handbook of Applied Cognitive Psychology. New York, Wiley, pp. 141-71.

Zhang J (1997). The nature of external representations in problem solving. Cognitive Science 21(2): 179-217.

Current as of September 2000

Internet Citation:

Testimony of David Woods, Panel 2: Broad-based Systems Approaches. Written Statement. National Summit on Medical Errors and Patient Safety Research. September 2000. http://www.quic.gov/summit/wwoods.htm

Return to Testimony Directory
National Summit on Medical Errors and Patient Safety
QuIC Home Page
Department of Health and Human Services