NUREG-1624, Revision 1, "Technical Basis and Implementation Guidelines for a Technique for Human Event Analysis (Atheana)", 12/15/99

December 15, 1999

Dr. William D. Travers
Executive Director for Operations
U.S. Nuclear Regulatory Commission
Washington DC 20555-0001

Dear Dr. Travers:

SUBJECT:

NUREG-1624, REVISION 1, "TECHNICAL BASIS AND IMPLEMENTATION GUIDELINES FOR A TECHNIQUE FOR HUMAN EVENT ANALYSIS (ATHEANA)"

During the 468^th meeting of the Advisory Committee on Reactor Safeguards, December 2-4, 1999, we reviewed Revision 1 of NUREG-1624, "Technical Basis and Implementation Guidelines for A Technique for Human Event Analysis (ATHEANA)." Our Subcommittee on Human Factors also reviewed this document on November 19, 1999. During our review, we had the benefit of discussions with representatives of the NRC staff and of the documents referenced.

Conclusions and Recommendations

1.	The objective of ATHEANA is to develop a methodology that: (a) allows a realistic, qualitative analysis of potential accident sequences and past incidents involving human actions and (b) allows a realistic evaluation of the probabilities of unsafe human actions for inclusion in probabilistic risk assessments (PRAs). The qualitative evaluation in NUREG-1624, Revision 1, is at an advanced stage of development and is achieving its purpose. The quantitative portion still needs significant development.
2.	ATHEANA's focus on the context within which the operators must act as well as on the error mechanisms is an appropriate paradigm shift away from a focus on "human error."
3.	ATHEANA deals with operator actions that take place after an abnormal event has occurred, e.g., a fire or an initiating event, as defined in PRAs. Its scope should be extended to include normal activities that may cause a plant event.
4.	The term "error-forcing context" is not used consistently and is misleading in some situations. An alternative, more descriptive term must be defined.
5.	The process of searching for error-forcing contexts is complex. Not all human actions require such a detailed treatment, and a screening process should be developed to identify the level of analysis that a given situation requires. The development of the screening process should be given priority.
6.	In developing symptom-based procedures, the industry considered many deviations from expected plant behavior. The ATHEANA search process for deviations should take advantage of this experience.
7.	The elements of a plant's safety culture that influence the operators when they are faced with a decisionmaking situation should be explicitly considered when evaluating the error-forcing contexts.
8.	The application of ATHEANA to a fire-initiated accident scenario does not make clear its advantages over existing, less complex methods. More examples of applications need to be developed.

Discussion

Understanding human errors and evaluating their probability of occurrence have been active areas of research since the Three Mile Island accident. "First-generation" models, i.e., those developed in the 1970s and 1980s, varied in their depth of modeling human performance. No serious attempt was made to incorporate concepts from the behavioral and cognitive sciences into these models. The focus was on "human error" with its connotation of blame.

In the late 1980s, a need for "second-generation" models that would delve deeper into the causes of human error was recognized. Attention shifted toward an examination of contextual elements that could trigger cognitive error mechanisms which could lead to unsafe crew actions. ATHEANA is the first major effort to develop a model for human performance based on this new paradigm. We believe that this shift in paradigm is appropriate and commend the staff for carrying out this work.

ATHEANA focuses on the analysis of human performance after a plant event . This is natural, since this has been the main perceived need for improving human reliability analysis. Errors made during routine activities, such as maintenance and testing, are analyzed satisfactorily by using the methods of the human reliability handbook (NUREG/CR-1278, Revision 1). Normal plant activities that may lead to plant events, such as the reactor coolant drain-down event at the Wolf Creek Generating Station, Unit 1, on September 19, 1994, are not currently addressed by ATHEANA.

The principal premise of ATHEANA is that "plant conditions" and "performance-shaping factors" may produce an "error-forcing context" that could trigger an error mechanism such as the refusal to change an initial misdiagnosis when contradictory evidence is received. The performance-shaping factors reflect human-centered influences such as training and communications.

The search for error-forcing contexts is a major effort. A multidisciplinary team consisting of human-reliability experts, plant operators, PRA specialists, and possibly others is needed. Such an extensive effort is not appropriate for all potentially unsafe human acts. We are concerned that the amount of resources required may discourage practitioners from even attempting to use ATHEANA. We believe that a set of screening guidelines should be developed to define different levels of treatment for various unsafe human acts. The qualitative insights gained from the detailed ATHEANA investigations should form the basis for the development of simpler methods for use when appropriate.

We note that a similar situation arises when a decision must be made about the methodology to be used to elicit and utilize expert opinions in probabilistic seismic hazard analysis (NUREG/CR-6372). In some situations of great national interest in which the uncertainties are large, a very formal methodology that is implemented by a multidisciplinary team is required. In other situations, experience has shown that a single technical integrator using informal input from experts is sufficient.

The process of searching for error-forcing contexts starts with a base-case scenario that describes the expected plant and operator behavior for a given initiator. The error-forcing contexts are, then, identified by searching for deviations from the base-case scenario. A great deal of work along these lines was done when the industry developed symptom-based emergency operating procedures. We believe that ATHEANA should take advantage of this experience.

ATHEANA defines an error-forcing context as "the combined effect of PSFs [performance-shaping factors] and plant conditions that create a situation in which human error is likely." Yet, in Chapter 10 of NUREG-1624, Revision 1, it is stated that an error-forcing context may be "so noncompelling that there is no increased likelihood of the UA [unsafe act] compared with the routine PRA context." We believe that the use of clear, accurate terminology is essential, especially when concepts from the behavioral sciences are brought into the practice of engineering. We believe that an alternative terminology should be developed to replace "error-forcing context."

The error mechanisms are developed from a cognitive model that consists of detection, situation assessment, response planning, and response implementation. All of these activities involve decisions that the plant crew must make, especially in the response planning phase. Although the discussion of error mechanisms clearly assumes that decisions are being made, e.g., establishing wrong goals is identified as a possible error, no formal attempt is made to investigate either the decisionmaking process or the impact of time. The decisionmaking processes (as well as the error-forcing contexts) are expected to be different for event sequences that evolve in a relatively short time, e.g., in less than about 30 minutes, and for sequences taking place over longer periods. In addition, decisionmaking may involve balancing conflicting safety and economic objectives; therefore, the plant's safety culture is a critical element in these decisions. Safety culture should be explicitly considered when evaluating the error-forcing context.

The application of ATHEANA to a fire-initiated accident scenario failed to convince us that the results obtained were sufficiently better than those obtained through other, presumably less resource-intensive methods to justify the use of ATHEANA . There are some inconsistencies between this application and the theoretical development in NUREG-1624, Revision 1. For example, the error-forcing contexts that the methodology claims are its foundation were not identified explicitly. We believe that a number of applications are urgently needed to convince the human reliability community and the end users that ATHEANA is a practical model that represents an improvement over existing models. These applications will also serve to guide the development of the screening process that we mentioned above.

A major motivation for the development of ATHEANA is the need for adequate models to support risk-informed regulatory applications. The guidance provided currently for evaluating the probabilities of unsafe human acts is very general. The HEART model (NUREG-1624, Revision 1, Chapter 10), whose quantitative results are proposed as one way for assessing the probability of a given error-forcing context, was developed several years before the ATHEANA project started and there is no effort to adapt it to ATHEANA. If the HEART model is to form the basis for quantifying the error-forcing context in the ATHEANA process, then ATHEANA should include sufficient information to assess the appropriateness of using this model for such purpose.

We acknowledge that any attempt at quantifying probabilities of error-forcing contexts will necessarily involve expert judgment. However, the guidance given by ATHEANA does not build on the large amount of work that has been done on the elicitation and utilization of expert opinions, e.g., in NUREG-1150, NUREG/CR-6372, and NUREG/CR-3518.

A more serious effort on probability evaluation will also help in developing the screening process that we recommended above. We expect that a lot of the details that are now investigated in the analysis of plant conditions, performance-shaping factors, and error mechanisms will not affect the quantification process, thus suggesting ways for limiting the qualitative investigation. While we recognize that the likelihood of plant conditions can be estimated, we believe that the probabilities of performance-shaping factors are much more difficult to evaluate. Thus, ATHEANA must demonstrate the feasibility of evaluating probabilities of error-forcing contexts, of which the performance-shaping factors are an important component.

We believe that the development of the screening process and the application of ATHEANA to several realistic accident scenarios are critical to its success. We look forward to working with the staff on these matters.

	Sincerely,
	/s/ Dana A. Powers Chairman

References:

1.	U.S. Nuclear Regulatory Commission, NUREG-1624, Revision 1, "Technical Basis and Implementation Guidelines for a Technique for Human Event Analysis (ATHEANA)," September 1999.
2.	U.S. Nuclear Regulatory Commission, NUREG/CR-1278, "Handbook of Human Reliability Analysis with Emphasis on Nuclear Power Plant Applications," Final Report Prepared by Sandia National Laboratories, A. D. Swain and H. E. Guttmann, August 1983.
3.	U.S. Nuclear Regulatory Commission, NUREG/CR-6372, "Recommendations for Probabilistic Seismic Hazard Analysis: Guidance on Uncertainty and Use of Experts," Prepared by R.J. Budnitz, G. Apostolakis, D.M. Boore, L.S. Cluff, K.J. Coppersmith, C.A. Cornell, and P.A. Morris, April 1997.
4.	U.S. Nuclear Regulatory Commission, NUREG-1150, "Severe Accident Risks: An Assessment for Five U.S. Nuclear Power Plants," June 1989.
5.	U.S. Nuclear Regulatory Commission, NUREG/CR-3518, "SLIM-MAUD: An Approach to Assessing Human Error Probabilities Using Structured Expert Judgment," Prepared by D.E. Embrey, P. Humphreys, E.A. Rosa, B. Kirwan, and K. Rea, Brookhaven National Laboratory, July 1984.
6.	International Atomic Energy Agency, International Nuclear Safety Advisory Group (INSAG) Safety Series No. 75-INSAG-4, "Safety Culture," Vienna, 1991.