Importance Measures Derived from Probabilistic Risk Assessments

February 11, 2000

The Honorable Richard A. Meserve
Chairman
U.S. Nuclear Regulatory Commission
Washington, D.C. 20555-0001

Dear Chairman Meserve:

SUBJECT:

IMPORTANCE MEASURES DERIVED FROM PROBABILISTIC RISK ASSESSMENTS

During the 469^th meeting of the Advisory Committee on Reactor Safeguards, February 3-5, 2000, we met with representatives of the NRC staff and Consumers Energy Company, Southern California Edison, and STP Nuclear Operating Company regarding the use of importance measures in risk-informing 10 CFR Part 50. We also had the benefit of the documents referenced.

This report responds to the Commission request in the December 17, 1999 Staff Requirements Memorandum that the ACRS evaluate the importance measures derived from Probabilistic Risk Assessments (PRAs) that are currently being contemplated for risk-informing Part 50 and, where appropriate, provide recommended additions or alternatives.

We believe that risk-informed decisions are best made using metrics, such as core damage frequency (CDF) or large, early release frequency (LERF), to evaluate the impact of decision options. There are, however, important situations in which this impact cannot be calculated easily. These include the risk-informed determination of special treatment requirements for structures, systems, and components (SSCs). The SSCs are first categorized according to their "importance," and then a decision is made regarding special treatment requirements for each category. The impact of these requirements on CDF and LERF is not quantified.

The risk-important categories of SSCs can be determined in a number of ways. The commonly used importance measures in risk-informed applications are the Fussell-Vesely (FV) and Risk Achievement Worth (RAW), although others, such as the Birnbaum measure, are occasionally used.

In evaluating the robustness of the SSC categorization, it is important to consider two facts: (1) depending on their definition, importance measures provide different insights regarding the SSC importance and (2) the categorization is the result of an integrated decision by an expert panel that takes into account plant information in addition to the insights provided by the importance measures.

Since the determination of what is important, i.e., the definition of the importance measures, is somewhat arbitrary, these measures have limitations that include the following:

1.	Importance measures are typically evaluated for individual SSCs. Yet, some decisions may affect groups of SSCs. While individual SSCs of a group may not be risk significant, the group itself may be.
2.	Importance measures are strongly affected by the scope and quality of the PRA. For example, incomplete assessments of risk contributions from low-power and shutdown operations, fires, and human performance will distort the importance measures. Even with a full-scope, high-quality PRA, the importance measures have limitations, as discussed in our report of October 12, 1999.
3.	The various categories of risk significance are determined by defining threshold values for the importance measures. For example, in some applications, a SSC is in the "high" risk-significant category when FV > 0.005 and RAW > 2.0, or FV > 0.1, or RAW > 100. In other applications, the numerical values are different. Some licensees choose to emphasize one measure over the other, e.g., RAW over FV. The relationship of these choices to CDF and LERF is unknown.

Given that the analysts have freedom in determining the criteria of risk significance, we were not surprised to find out that some licensees are implementing an approach that does not use importance measures at all. The Top Event Prevention Analysis (TEPA) utilizes success paths to determine what is important. We agree that this approach may have desirable defense-in-depth characteristics.

We note that the statistical literature also contains a number of methods for determining the sensitivity of a function, in our case the CDF or LERF, to its basic inputs, e.g., the failure rates. These methods allow us to investigate the issue of importance at a more elementary level (i.e., the parameter level) than that of FV and RAW (i.e., the SSC level).

As stated above, what really matters is the robustness of the SSC categorization that the expert panel produces through its integrated decisionmaking process that includes plant information in addition to the information provided by the importance measures. Since any choice of criteria for risk significance will likely involve some arbitrariness, we believe, as stated in our report of October 12, 1999, that the expert panel that determines the categorization of SSCs should be fully aware of the limitations and constraints of the chosen method. The panel should be provided with the results of sensitivity analyses, the results of alternative approaches, and an evaluation of the impact of these results on CDF and LERF. We recommend that a project be established to identify clearly the limitations of each proposed approach to importance

determination and to provide guidance to the expert panel on its deliberations regarding these matters. We believe that useful results can be produced in a short period.

Sincerely, /s/ Dana A. Powers Chairman

References:

1.	Report dated October 12, 1999, from D. A. Powers, Chairman, ACRS, to Greta Joy Dicus, Chairman, U.S. Nuclear Regulatory Commission, Subject: Proposed Plans for Developing Risk-Informed Revisions to 10 CFR Part 50, "Domestic Licensing of Production and Utilization Facilities."
2.	M.C. Cheok, G. W. Parry, and R. R. Sherry, "Use of Importance Measures in Risk-Informed Regulatory Applications," Reliability Engineering and System Safety, 60, 213-226, 1998.
3.	W. E. Vesely, "Supplemental Viewpoints on the Use of Importance Measures in Risk-Informed Regulatory Applications," Reliability Engineering and System Safety, 60, 257-259, 1998.
4.	W. E. Vesely, "Reservations on ASME Risk-Based Inservice Inspection and Testing: An Outlook to the Future," Risk Analysis, 18, 423-425, 1998.
5.	K. N. Fleming, "Developing Useful Insights and Avoiding Misleading Conclusions from Risk Importance Measures in PSA Applications," PSA '96, Park City, Utah, September 29 - October 3, 1996, pp. 215-221, American Nuclear Society.
6.	R. W. Youngblood, "Applying Risk Models to Formulation of Safety Cases," Risk Analysis, 18, 433-444, 1998.
7.	R. A. White, R. B. Worrell, and D. P. Blanchard, "Using Top Event Prevention Analysis to Select a Safety-Significant Subset of Check Valves for Testing," Presented at TopSafe98, Valencia, Spain, April 1998.
8.	C. E. Nierode, T. Willemson, R. B. Worrell, and D. P. Blanchard, "Use of Top Event Prevention Analysis to Select a Safety-Significant Subset of Air-Operated Valves for Testing," Proceedings of PSAM 4, New York City, September 13-18, 1998, pp. 1358-1363, A. Mosleh and R. A. Bari, Eds., Springer-Verlag, London.