Patient Safety at the Organizational Level

Additional Statement, Submitted by Robert L. Wears, M.D., M.S., Professor, University of Florida College of Medicine, Department of Emergency Medicine, and Kathleen M. Sutcliffe, Ph.D., School of Business, University of Michigan

Executive Summary

In his keynote address to the 2^nd Annenberg Conference, Berwick pointed out that, "the design of the work system is only half the battle. There is a system above the work system that must be equally carefully designed—the management system—and compared to the work system, it may need even more overhauling in American health care if we are to reach the safety goals we ought to." This testimony will argue that research efforts should be aimed at identifying and measuring those characteristics of healthcare organizations (HCOs) that are associated with patient safety.

There is abundant evidence from other fields that organizational characteristics affect safety. The linkage is so strong that there is even a specific term, "organizational accident," for mishaps resulting from complex organizational characteristics. The ability to identify HCOs at greater risk for patient safety problems would assist the public in choosing providers, and would provide strong incentives for organizations to address safety issues.

In addition, better knowledge of organizational characteristics related to safety would be valuable within HCOs. One common problem all organizations face in balancing safety and production is the paucity of information about safety performance, especially compared to information about production. This information asymmetry leads organizations to drift ever more closely toward the margins of unsafe performance in the interests of enhancing production.

Important research questions to be answered would include whether it is possible to distinguish between less safe and more safe HCOs, what characteristics are useful in this regard, how they can be best measured, and how to distinguish between less and more safe conditions in the same HCO. A broad-based interdisciplinary initiative could ultimately lead to the development of useful "safety barometers" to guide HCOs as they adapt to changing circumstances.

The Problem

The "New Look" approach to errors in healthcare^1-4 identifies a hierarchy in healthcare organizations (HCOs) where errors or error-spawning conditions occur. Moving progressively "upstream" from an untoward event, we have: the individual (e.g., the cognitive processes that create slips, lapses, and mistakes); the workplace (e.g., poor teamwork, awkward automation, bad work environments, and poor equipment design); and ultimately, the organization (where policies, procedures, and resource allocation decisions are made).

The cultural, organizational and management characteristics of organizations affect the probability of error in global ways⁵, but most of the work on errors in healthcare has been addressed at "downstream" issues at the level of the workplace, work team, or individual caregiver. These could be referred to as the "micro" level (individual) or the "mezzo" level (work group). Higher level organizational contributors to error (the "macro" level) have been less commonly examined in health care, although they have received considerable attention in industry⁶.

Berwick¹ has called attention to this omission: ". . . design of the work system is only half the battle. There is a system above the work system that must be equally carefully designed—the management system—and compared to the work system, it may need even more overhauling in American health care if we are to reach the safety goals we ought to."

In health care, no systematic and standardized research assessing organizational factors contributing to safety exists, although instruments for assessing resistance to catastrophic error (e.g., Tripod-Delta, MESH, HEART, et al) have been used in other industries⁶. Such a research would be useful in health care, for at least seven reasons.

First, corrective measures applied at the "blunt end" of the system are likely to be have wider and more long-lasting effects than those applied nearer to the "sharp end." In Reason’s words: "Only in the upper levels of the system can we begin to get to grips with the ‘parent’ failure types—the processes that create the downstream ‘problem children.’ If these remain unchanged, then efforts to improve things at the workplace and worker level will be largely in vain. The damaging effects of certain kinds of unsafe act may be reduced and specific conditions within the workplace improved, but the continuing existence of the ‘parent’ failures in the upper echelons of the organization will ensure their rapid replacement by other kinds of human and workplace problems. Clearly, then, organizational factors represent the priority area for process measurement^{6, pg. 121}." In essence, the ability to identify preconditions, latent failures, and failed defenses prior to the occurrence of an untoward event should make it possible for HCOs to take proactive steps to correct them. Errors arise from combinations of these "resident pathogens" and local triggering effects. But local triggers are invisible, unknowable in advance and hence uncontrollable. Tools resulting from the research effort proposed here could help make these latent conditions visible and thus potentially available for control.

Second, there is considerable evidence that there are vast differences between the mental models shared by high level managers in complex organizations and those shared by the front line workers whom they lead^{7, 8}. This was dramatically exemplified when caregivers studied at the Veterans’ Health Administration, Kaiser Permanente, and CareGroup pointed out the distinction between "official priorities" and "real priorities"⁹. A tool that would allow repeated assessment of organizational resistance would make it more difficult for upper level managers to believe that they have an effective safety system in place, when conditions at the sharp end belie it.

Third, by creating a means of openly reporting the organization’s safety status to top leadership, such a tool would provide visible evidence of the leadership’s commitment to patient safety, and would enhance the creation of a safety culture within the organization.

Fourth, the ability to conduct repeated assessments over time within a single HCO should help in the sustainment of patient safety efforts in an HCO. Safety and reliability is not bankable; it must be continually recreated. Regular reports of an organization’s resistance to failure would make it more difficult for its leadership to feel that they had resolved all their safety problems and could turn their attention to something else. The idea that patient safety is a long guerilla war with no dramatic victories is not a comfortable one for managers to maintain. A periodic reminder of "slippage" would be useful in developing the type of systemic mindfulness characteristic of high reliability organizations¹⁰. It would reinforce the idea that a long period of inactivity without serious accidents means not that problems have been solved, but rather that danger is building.

Fifth, it would enhance an increasingly neglected aspect of leadership in healthcare organizations, namely that of preserving a balance of values in the face of a myriad of competing values. Paul Schulman¹¹ has described academic leadership as having adopted the role of change agent, and pointed out that leaders in high reliability organizations play a different role, that of the "partisan of the neglected perspective." Similar forces are at work in modern HCOs and a similar rebalancing of forces and priorities would be valuable.

Sixth, the ability to measure organizational safety characteristics over time could begin to rebalance an asymmetry of information within organizations, that leads them to drift ever more closely toward the margins of unsafe performance in the interests of enhancing production. The problem is that information on the organization’s performance with respect to production is abundant and easily acquired, but information on its safety performance is not. This is in large part due to the fact that safety is a non-event—the absence of things that should never happen anyway. Perrin has called this "the fallacy of the unrocked boat"^{6, pg. 6}, where a period of time without accidents disguises a drift towards the margins of unsafe practice. What is needed is, in effect, a barometer, so that a "non-rocking boat but a falling barometer" could warn organizational leaders in time to take corrective action. In other industries, evaluation instruments have been successfully developed to redress this information asymmetry and maintain an appropriate balance between safety and production.

Finally, the current turmoil in healthcare in the United States increases the need for such knowledge. It seems clear from the foregoing that the ability to assess an HCO’s resistance to error would be useful, even if the health care were stable. But, it is not stable—it is currently undergoing a huge organizational transformation, accompanied by wrenching dislocations and financial stresses, all with unpredictable effects on patient safety. Now more than ever the ability to assess organizational resistance to error is of great importance.

The effect of macro-level organizational factors on safety is dramatically illustrated by the ValuJet crash of May 1996, where organizational factors were identified as prominent antecedents in the chain of errors leading to the tragedy^12-13. The parallels between today’s HCOs and ValuJet are striking: severe cost pressures, frequent mergers and acquisitions, increasing use of contract and temporary employees, heavy reliance on outsourcing, etc. The current financial dislocations in health care organizations suggests that there is a great need for an estimate of this dimension of an organization’s resistance to error, and the patient safety consequences of organizational changes. It is chilling to juxtapose Wildavsky’s comment on organizational safety—"Richer is safer"—with recent reports of a growing chasm between good and poor financial performance in U.S. HCOs¹⁴.

What Is Known

In contrast to other industries, there has been little effort to develop this knowledge in health care. However, there are three related lines of inquiry in other fields that are informative.

The first is exemplified by Reason’s work on what he has called "organizational accidents," i.e., the relatively rare, occasionally catastrophic events that occur within complex technological systems such as nuclear power plants, commercial aviation, petrochemical refining, and so on. The influence of organizational factors on such events had been well known for some time and several error management tools have been used in industry to assess workplace and organizational. A review of some of these tools gives a picture of how similar research results might be useful in health care.

Tripod Delta¹⁶, discussed online in ^{6, pp. 132-38}, was developed for the petroleum industry, beginning in 1989. It has gone through several iterations, and has spawned some related tools such as MESH and HEART, discussed below. Tripod-Delta consists of three main components: a coherent safety philosophy, an integrated way of thinking about processes that disrupt safe operations, and a set of instruments for measuring the disruptive processes. Its development began with the identification of what were called ‘General Failure Types,’ which included factors such as hardware, work design, maintenance, work procedures, incompatible goals, communications, organizational deficiencies, training, and so on. Measures for each of the general failure types were derived from indicators provided by task specialists, i.e., those involved in day-to-day operations. This gives them a degree of ownership of the results and ensures the relevance of the items. Items are designed to be unambiguously answerable by yes/no responses, e.g., "Are there locations on this platform where the deck and the walkway differ in height?" The instrument is applied at regular intervals, typically quarterly, and the accumulated results displayed as a general "Failure State Profile" which is used by upper management as an indicator of the organization’s "safety health."

MESH and Review, both discussed online in ^{6, pp. 138-42}, are descendants of Tripod-Delta, developed for aircraft maintenance and railway operations, respectively. They use rating scales rather than binary indicators to measure the presence of latent factors. Their general development proceeded in much the same manner, with the identification of classes of general problems, derivation of specific factors related to those problem classes, and then development of measurement instruments for those factors, culminating in periodic, global summaries of organizational resistance to failure.

HEART, online in ^{6, pp. 142-46}, was originally developed in the power generation industry but was intended to be more generally applicable. HEART begins with a set of generic task types, in a general hierarchy of difficulty or error-proneness. A given task and type is then associated with lists of potential error-producing conditions and violation-producing conditions likely to impact successful performance. HEART differs from the preceding tools in that it attempts to provide a probabilistic risk of error based on factors and multipliers associated with the condition lists.

A second line of work concerns the interaction of professional, organizational, and even national cultures on the way in which people learn and behave in organizations. For example, Helmreich¹⁷, Klein¹⁸, Hutchins¹⁹, Vaughn²⁰, and Westrum²¹ have explored aspects of the rich and complex way in which professional, workplace, organizational and national cultures affect behavior and decision-making. The instruments mentioned previously have not addressed cultural factors directly, although they all acknowledge culture as pervading all of them.

Finally, there is a body of work describing the characteristics so-called "high reliability organizations"^{22, 23}. These organizations, such as aircraft carrier flight deck crews, nuclear power plant operators, etc, work in inherently hazardous operations for long periods of time, but have achieved a remarkably low rate of accidents (orders of magnitude below that in HCOs). Several characteristics of high reliability organizations have been advanced to explain their success¹². These organizations are preoccupied with the risk of failure, and are unlikely to become complacent in quiet periods. They are reluctant to simplify interpretations, and tend to cultivate variety in opinions, and train their members to assume less but notice more. High reliability organizations show resilience when faced with unusual circumstances. They tend to shift to more flexible decision structures, subordinating rank to experience and expertise in crisis situations²⁴. Interestingly, to date this information has not been applied in organizational management in healthcare. It would be reasonable to believe that a health care organization that possesses more of the characteristics of a high reliability organization and fewer of the characteristics of the ‘normal accident theory’ organization would be more conducive to patient safety²⁵.

What Is Needed

The existence of a body of knowledge and successful develop of tools for assessing organizational resistance to failure in other industries suggests such work could be successfully performed in HCOs, and would be useful in enhancing patient safety. One approach might take the following course.

First, the theoretical foundations of macro level organizational factors relating to safety in health care should be developed, drawing on existing work in other domains, such as research on high reliability organizations, normal accident theory, and others. This work would concentrate on identifying issues that would be relevant to HCOs and would also likely be observable or measurable, and would provide a tentative but not immutable framework for what follows.

Second, an effort could be made to identify a set of characteristics of high reliability organizations that are applicable to and measurable in health care organizations. These would include the following some of the areas previously outlined such as resilience, shifting decision structures, sensitivity to operations, etc. A simultaneous effort would identify characteristics of poor reliability, stemming from normal accident theory and the "garbage can" model of organizations²⁶.

Third, basic research on classes of organizational factors contributing to error and to safety could be developed by identifying a set of general failure classes relevant to health care organizations from previous research on medical error and organizational failure. Six broad areas should be addressed: culture, training, management, safety-specific issues, design of procedures, and technical factors.

Fourth, once the domains of interest have been well identified, the focus could shift toward development and evaluation of solutions. For example, specific test instruments could be developed and validated for each of the relevant dimensions, using information gathered from both sharp and blunt end workers in the organization, because dual collection will help identify perceptual gaps between upper level and front line workers, e.g., safety as an "official" vs a "real" priority of the organization. Measurement instruments should be designed and validated for administration repeatedly over time, and should involve both objective and attitudinal measures.

Fifth, pilot assessments should be done. Because different parts of a complex HCO will have different safety profiles and subcultures, it will be likely be necessary to learn how to measure and interpret characteristics relevant to different care areas. In addition, it is known that adverse events are more common in some areas (such as the Operating Room [OR]) than in others, and that errors are more often preventable in some areas (such as the Emergency Room [ER]) than in others²⁷. These areas could be over-represented in an organization’s sampling plan.

Sixth, factorial validity assessment could be undertaken, to assist in the association of unsafe conditions with specific constructs. This factorial validity will give organizational leaders useful information for proactively eliminating error prone conditions by identifying specific problems areas ("OR nursing staff is overworked") rather than simply providing global assessments ("Something’s going sour somewhere").

Seventh, prototype systems for administration, storage, analysis, and reporting would be developed, followed full-scale field testing.

Eighth, the information and implementations could be broadly disseminated to HCOs. Research in this area could have broad and rapid impact on patient safety. It is important because there is a great deal of concern in the minds of the public, policy-makers, and health professionals about the impact of organizational changes, largely motivated by significant and real financial concerns, on patient welfare. This research would provide the knowledge and tools with which to address those concerns, a way to avoid the hindsight bias that impedes effective analysis of performance after bad outcomes², and would allow motivated organizations to assess themselves, follow their progress (or decline) over time, and could facilitate comparisons between organizations. If coupled with a comprehensive and reliable error reporting system, the tools could be externally validated and would improve organizational learning by ensuring that the impact of organizational changes on patient safety could be assessed in a proactive, objective and quantitative way.

References

1. Berwick DM. Taking action to improve safety: How to increase the odds of success. In: Proceedings of the Second Annenberg Conference on Enhancing Patient Safety and Reducing Errors in Health Care. Rancho Mirage, CA: National Patient Safety Foundation at the AMA; 1998:1-10.

2. Cook RI, Woods DD. Operating at the sharp end: The complexity of human error. In: Bogner MS, ed. Human Error in Medicine. Hillsdale, NJ: Lawrence Erlbaum Associates; 1994:255-310.

3. Cook RI. Two years before the mast: Learning how to learn about patient safety. In: Second Annenberg Conference on Enhancing Patient Safety and Reducing Errors in Healthcare. Rancho Mirage, CA; 1998.

4. Cook RI, Woods DD, Miller C. A Tale of Two Stories: Contrasting Views of Patient Safety.

5. Moray N. Error reduction as a systems problem. In: Bogner MS, ed. Human Error in Medicine. Hillsdale, NJ: Lawrence Erlbaum Associates; 1994:67-91.

6. Reason J. Managing the Risks of Organizational Accidents. Aldershot, UK: Ashgate Publishing Co; 1997.

7. Sutcliffe KM. Organizational environments and organizational information processing. In: Jablin F, Putnam L, eds. Handbook of Organizational Communication. 2nd ed. Beverly Hills, CA: Sage Press; 1999.

8. Sutcliffe KM. What executives notice: Accurate perceptions in top management teams. Acad Manage J 1994;7:1360-78.

9. Adelson R, Bieber J, Blum L, et al. Patient Safety: Listening to Healthcare Employees. In. Chicago, IL: National Patient Safety Foundation at the AMA; 1998.

10. Weick KE, Sutcliffe KM, Obstfeld D. Organizing for high reliability: Processes of collective mindfulness. In: Proceedings of the Second Annenberg Conference on Enhancing Patient Safety and Reducing Errors in Health Care. Rancho Mirage, CA: National Patient Safety Foundation; 1998:147-50.

11. Schulman P. The leadership crisis of the American academy: Lessons from "high reliability" organizations. In.: Mills College, Oakland, CA; 1999.

12. Weick KE, Sutcliffe KM, Obstfeld D. Organizing for high reliability: processes of collective mindfulness. In: Staw B, Sutton R, eds. Research in Organizational Behavior: JAI Press; 1999.

13. Langewiesche W. The lessons of ValuJet 592. Atlantic Monthly 1998;281:81-98.

14. Moore JD. Chasm grows between rich and poor. Mod Healthc 1999;29:34-38.

15. Perrow C. Normal Accidents: Living With High-Risk Technologies. Princeton, NJ: Princeton University Press; 1999.

16. Hudson P, Reason J, Wagenaar W, et al. Tripod-Delta: Proactive approach to enhanced safety. Journal of Petroleum Technology 1994;40:58-62.

17. Helmreich RL, Merritt AC. Culture at Work in Aviation and Medicine. Aldershot, UK: Ashgate; 1998.

18. Klein G. Sources of Power. Cambridge, MA: MIT Press; 1998.

19. Hosmer DW, Taber S, Lemeshow S. The importance of assessing the fit of logistic regression models: A case study. Am J Public Health 1991;81:1630-35.

20. Vaughn D. The Challenger Launch Decision: Risky Technology, Culture and Deviance at NASA. Chicago, IL: University of Chicago Press; 1996.

21. Westrum R. Cultures with requisite imagination. In: Wise JA, Hopkins D, Stager P, eds. Verification and Validation in Complex Systems: Human Factors Issues. New York: Spring-Verlag; 1992:401-16.

22. Roberts K. Research in nearly failure-free, high reliability organizations: Having the bubble. IEEE Trans Engineering Management 1989;36:132-39.

23. Roberts KH. Some characteristics of high reliability organizations. Organization Science 1990;1:160-77.

24. Roberts KH, Stout SK, Halpern JJ. Decision dynamics in two high reliability military organizations. Management Science 1994;40:614-24.

25. Roberts KH, Libuser C. From Bhopal to banking: Organizational design can mitigate risk. Organ Dyn 1993;21.

26. Cohen MD, March JG, Olsen JP. A garbage can model of organizational choice. In: March JG, ed. Decisions and Organizations. New York, NY: Blackwell; 1988.

27. Brennan TA, Leape LL, Laird NM, et al. Incidence of adverse events and negligence in hospitalized patients. Results of the Harvard Medical Practice Study I. N Engl J Med 1991;324:370-6.

Current as of September 2000

Internet Citation:

Patient Safety at the Organizational Level. Additional Statement by Robert L. Wears and Kathleen M. Sutcliffe. National Summit on Medical Errors and Patient Safety Research. September 2000. http://www.quic.gov/summit/aswearsut.htm

Return to Additional Statement Directory
National Summit on Medical Errors and Patient Safety Research
QuIC Home Page
Department of Health and Human Services