Assessment of the Quality of Probabilistic Risk Assessments

September 7, 2000

The Honorable Richard A. Meserve
Chairman
U.S. Nuclear Regulatory Commission
Washington, D.C. 20555-0001

Dear Chairman Meserve:

SUBJECT:   ASSESSMENT OF THE QUALITY OF PROBABILISTIC RISK ASSESSMENTS

During the 475^th meeting of the Advisory Committee on Reactor Safeguards, August 29-September 1, 2000, we discussed the staff's approach for addressing the issue of quality of probabilistic risk assessments (PRAs) described in SECY-00-0162. We previously met with representatives of the staff to discuss the draft Commission paper on this matter during our July 12-14, 2000 meeting. We had the benefit of the documents referenced.

Conclusions and Recommendations

1.	We agree with the staff's recommendation to continue with the current process of determining the applicability of PRAs to specific regulatory applications.
2.	The staff has appropriately emphasized that the quality of a PRA must be judged in the context of the regulatory decision that the PRA supports.
3.	Attachment 1, "PRA Scope and Technical Attributes," to SECY-00-0162 is a useful high-level tutorial exposition of PRA elements and technical attributes. It is not a "design-to" standard, nor is it intended to be.
4.	The staff should augment its collection of examples of risk-informed decisions and the requisite PRA quality to include a more diverse set of examples and should provide more details on how risk information was used. This would enable generic conclusions to be drawn regarding the role and quality of the risk information utilized in these decisions.
5.	The case-study ("bottom-up") approach in Attachment 2, "PRA Quality in Risk-Informed Regulation," to SECY-00-0162 is a much needed complement to the "top-down" approach that both Attachment 1 and the American Society of Mechanical Engineers (ASME) Standard for PRAs have taken. This two-pronged approach to the issue of PRA quality is necessary for the achievement of consensus regarding this very difficult issue.

Discussion

In the Staff Requirements Memorandum dated April 18, 2000, the Commission requested the staff to provide recommendations for addressing the issue of PRA quality until the ASME and American Nuclear Society Standards have been completed, including the role of an industry PRA certification process. The staff has responded by stating that it will continue with the current process of reviewing PRAs for specific applications. The staff has provided two attachments to further elaborate on its expectations.

In our report dated July 20, 2000, we stated that the quality of PRA is at the heart of a successful risk-informed regulatory system and that PRA quality should be evaluated in the context of the decision it supports. While this recognition is realistic and appropriate, it is also the main obstacle to developing a PRA standard in the traditional sense that the engineering community normally interprets the term "standard." It is unrealistic for a standard to define a high-quality PRA as one that is of full-scope and uses detailed state-of-the-art models because many regulatory applications do not require this level of effort.

We commented on these challenges when we reviewed the proposed ASME Standard for PRA which attempted to define three categories of PRA quality. Attachment 1 of SECY-00-0162 eschews categories and provides what is necessarily a high-level description of basic PRA elements. We note that a PRA could satisfy the functional attributes listed in Attachment 1, and still be of poor quality. This is an inherent problem and is not intended as a criticism of the staff's effort.

Because the critical issue is the support for regulatory decisions, we found the discussion in Attachment 2 to be useful. The examples of PRA elements important in specific decisions were illuminating. For example, the staff states that in reviewing requests for boiling water reactor (BWR) incremental power uprates, it concluded that increased power levels would result in less time for operator actions during an accident. A PRA supporting such decisions has to include an appropriate analysis of how this shorter time would affect the progression of the relevant accidents. It would have been difficult to determine the importance of this particular PRA requirement before the need for making a decision on this issue arose.

The staff has considerable experience with a number of specific risk-informed regulatory decisions. The staff should expand Attachment 2 to provide more details on how risk information was used in such decisions and to identify common themes and frequently asked questions. Such a case-study ("bottom-up") approach is a much needed complement to the "top-down" approach that both Attachment 1 and the ASME Standard have taken. This two-pronged approach to the issue of PRA quality is necessary to achieve consensus regarding this very difficult issue.

	Sincerely,
	/RA/
	Dana A. Powers Chairman

References:

1.	Memorandum dated July 28, 2000, from William D. Travers, Executive Director for Operations, NRC, to The Commissioners, Subject: SECY-00-0162, Addressing PRA Quality in Risk-Informed Activities.
2.	Memorandum dated April 18, 2000, from Annette Vietti-Cook, Secretary, NRC, to William D. Travers, Executive Director for Operations, Subject: Staff Requirements - Briefing on Risk-Informed Regulation Implementation Plan (SECY-00-0062).
3.	Letter dated July 20, 2000, from D.A. Powers, Chairman, Advisory Committee on Reactor Safeguards, to William D. Travers, Executive Director for Operations, NRC, Subject: Proposed Final ASME Standard for Probabilistic Risk Assessment for Nuclear Power Plant Applications.