Proposed Resolution of Public Comments on Draft Regulatory Guide DG-1122, "An Approach for Determining the Technical Adequacy of Probabilistic Risk Assessment Results for Risk-informed Activities"

April 21, 2003

The Honorable Nils J. Diaz
Chairman
U.S. Nuclear Regulatory Commission
Washington, DC 20555-0001

SUBJECT:	PROPOSED RESOLUTION OF PUBLIC COMMENTS ON DRAFT REGULATORY GUIDE DG-1122, "AN APPROACH FOR DETERMINING THE TECHNICAL ADEQUACY OF PROBABILISTIC RISK ASSESSMENT RESULTS FOR RISK-INFORMED ACTIVITIES"

Dear Chairman Diaz:

During the 501st meeting of the Advisory Committee on Reactor Safeguards, April 10-12, 2003, we met with representatives of the NRC staff and the Nuclear Energy Institute (NEI) to discuss the NRC staff's proposed resolution of public comments received in regard to Draft Regulatory Guide (DG)-1122, "An Approach for Determining the Technical Adequacy of Probabilistic Risk Assessment Results for Risk-Informed Activities." We also had the benefit of the documents referenced.

Recommendations

1. The draft final Regulatory Guide should include definitions of the terms "dominant," "important," "key," and "significant."

2. The peer review of the probabilistic risk assessments (PRAs) should include an assessment of the uncertainties and the validity of key assumptions.

3. The draft final Regulatory Guide should include guidance on how to perform sensitivity and uncertainty analyses.

4. To ensure consistency, the draft final Regulatory Guide should prescribe a minimum list of topics to be included in the peer review.

5. The staff needs to clarify how the Capability Categories are consistent with the provision in the Regulatory Guide that the event probabilities reflect the actual operating history and experience of the plant as well as applicable generic experience.

6. The staff should provide guidance on acceptable qualitative characterization of risk contributions not calculated in limited-scope PRAs.

Discussion

Ever since the Commission started its initiative to risk-inform the regulations, the quality of risk information that is input to the integrated decisionmaking process has been a subject of debate. To help the staff evaluate the quality of submitted PRAs in a timely manner, the American Society of Mechanical Engineers (ASME) has issued a standard for PRAs for "internal" accident initiators and the industry has developed a peer review process. DG-1122 and the associated Standard Review Plan (SRP) Chapter 19.1 document the regulatory position regarding these efforts.

DG-1122 provides guidance to licensees in four areas:

• A minimal set of functional requirements of a technically acceptable PRA.
• NRC position on consensus PRA standards and industry PRA program documents.
• Demonstration that the PRA (in toto or specific parts) used in regulatory applications is of sufficient technical adequacy.
• Documentation that the PRA (in toto or specific parts) used in regulatory applications is of sufficient technical adequacy.

The staff has received a large number of comments from ASME and the industry, most of which have been resolved. The true test of the usefulness of this Regulatory Guide is to subject it to pilot applications. We believe that several issues must be resolved before issuing a draft final Regulatory Guide for trial use so that better insights can be obtained.

ASME and NEI disagree with three staff positions. These positions deal with the definition of terms such as "dominant" sequences or events, the assessment by the peer reviewers of key assumptions, and the minimum list of topics that the peer review process should include.

The ASME standard provides an ambiguous definition of "dominant" and uses the term interchangeably with "significant" and "key." This term is critical to the application of the standard because it determines whether certain requirements are imposed and it is part of the definitions of the Capability Categories. ASME and the industry disagree with the staff's proposal to test a quantitative definition of the term.

As stated above, the purpose of the standard and the peer review process is to assist the staff in determining the quality of risk information used in particular regulatory applications. The staff's review of licensee applications will be eased if there is common understanding of key concepts. We believe that clear definitions of the terms "dominant," "important," "key," and "significant" should be included in the draft final Regulatory Guide before issuing it for trial use.

PRAs rely on numerous assumptions that are often critical to the validity of the results. Although the ASME standard requires that the key assumptions be identified, it does not require the peer reviewers to assess the validity of these assumptions. We agree with the staff that such an assessment should be required.

The ASME standard provides a list of PRA "suggestions" that the reviewers should consider in their review. These are not intended to be either a minimum or a comprehensive list of requirements. The staff argues that these suggestions should, in fact, be requirements; otherwise consistency in the reviews cannot be ensured. We agree.

In our report dated July 23, 2002, we recommended that proposed Revision 1 to Regulatory Guide 1.174 and SRP Chapter 19 state that changes to the licensing basis would, in general, require PRAs that conformed at least to Category II of the ASME standard and a Grade 3 of the industry peer review process.

While DG-1122 does not explicitly state that PRAs should conform at least to Category II of the ASME standard, it does state that the PRA model represent the as-built and as-operated plant, and that the event probabilities reflect the actual operating history and experience of the plant and applicable generic experience. It is not clear how this can be consistent with Category I of the ASME standard. The staff needs to clarify how the Capability Categories are consistent with these requirements. Similar clarification regarding the grades of the peer review process specified in NEI 00-02 should be made.

DG-1122 correctly states that understanding the relevant uncertainties is an essential element of risk characterization. A systematic treatment should include rigorous analyses for parametric uncertainties, sensitivity studies to identify the important epistemic uncertainties, and quantification of the latter. In a risk-informed environment, the proper role of sensitivity studies is to identify what is important to the results, not to replace uncertainty analyses. The staff should include guidance in the draft final Regulatory Guide regarding sensitivity and uncertainty analyses.

DG-1122 states that, for many applications that involve total plant risk, the risk characterization should account for all operating states and initiating events either quantitatively or qualitatively. More guidance is needed on this subject.

We would like to review the draft final version of DG-1122 before issuing our letter on its trial use.

Sincerely,

/RA/

Mario V. Bonaca
Chairman

References:

1. Letter dated April 4, 2003, from Scott F. Newberry, Nuclear Regulatory Research, NRC, to John T. Larkins, Executive Director, ACRS, Subject: ACRS Review of Draft Guide-1122, "An Approach for Determining the Technical Adequacy of Probabilistic Risk Assessment Results for Risk-Informed Activities," and the Associated Standard Review Plan Chapter 19.1.
2. U.S. Nuclear Regulatory Commission, Office of Nuclear Regulatory Research, Draft Regulatory Guide DG-1122, "An Approach for Determining the Technical Adequacy of Probabilistic Risk Assessment Results for Risk-Informed Activities," November 2002.
3. American Society of Mechanical Engineers, ASME RA-S-2002, "Standard for Probabilistic Risk Assessment for Nuclear Power Plant Applications," dated April 5, 2002.
4. Nuclear Energy Institute, NEI-00-02, "Probabilistic Risk Assessment Peer Review Process Guidance," Revision A3, dated March 20, 2002.
5. Letter dated July 23, 2002, from G. E. Apostolakis, Chairman, ACRS, to Richard A. Meserve, Chairman, NRC, Subject: Draft Final Revision 1 to Regulatory Guide 1.174 and to Chapter 19 of the Standard Review Plan.
6. Letter dated April 8, 2003, from Dr. Sidney A. Bernsen, Chairman, ASME Committee on Nuclear Risk Management, to Dr. Mario V. Bonaca, Chairman, ACRS regarding reconciliation of ASME PRA Standard with DG-1122.