Proposed Improvements to the NRC Inspection and Assessment Programs

February 23, 1999

The Honorable Shirley Ann Jackson
Chairman
U.S. Nuclear Regulatory Commission
Washington, D.C. 20555-0001

Dear Chairman Jackson:

SUBJECT: PROPOSED IMPROVEMENTS TO THE NRC INSPECTION AND ASSESSMENT PROGRAMS

During the 459th meeting of the Advisory Committee on Reactor Safeguards, February 3-6, 1999, we reviewed the proposed changes to the NRC Inspection and Assessment Programs, including initiatives related to the development of performance indicators and a risk-based inspection program, which are discussed in SECY-99-007. Our Subcommittees on Plant Operations and Reliability and Probabilistic Risk Assessment also reviewed this matter on January 26, 1999. During these reviews, we had the benefit of discussions with representatives of the NRC staff. We also had the benefit of the documents referenced. We provided an interim letter, dated December 16, 1998, to the Executive Director for Operations on this matter.

Conclusions and Recommendations

1. The process outlined in SECY-99-007 represents a substantial positive step in improving the NRC Inspection and Assessment Programs. The proposed improvements should lead to a risk-informed, efficient process and should improve the objectivity, consistency, and scrutability of these Programs.
2. The objectives of these Programs should be clearly formulated. In particular, the staff should state whether the objectives are to ensure that a specific licensee is maintaining its baseline performance level (related to its licensing basis), or to assess whether any individual plant is an outlier with respect to an expected population-wide performance level.
3. The choice of thresholds for increased NRC attention should be made consistent with the definition of objectives.

Discussion

In response to both the Commission and ACRS concerns, the staff has made substantial progress in improving the NRC Inspection and Assessment Programs for evaluating the performance of nuclear power plant licensees. Since our interim letter, the staff has issued SECY-99-007 which presents recommendations for improvement to the Inspection and Assessment Programs (now termed "Reactor Oversight Process Improvements") in a consolidated manner.

During our discussion of SECY-99-007, two different interpretations of the nature of the inspection program emerged. In one interpretation, inspections are viewed as quality control measures, i.e., a plant is viewed as having an acceptable baseline performance and the inspection program is intended to confirm that the performance remains acceptable. The other interpretation is that the program is intended to identify plants that become outliers with respect to an industry-wide acceptable performance level.

The difference between these two interpretations is whether the acceptable performance levels have different values for different plants. In SECY-99-007, the staff identifies a set of performance indicators (PIs) and sets thresholds for each PI at a level such that 95% of the plants have met this threshold of performance.

The use of this type of threshold on the PIs could imply that the second interpretation is the high-level objective of the Inspection and Assessment Programs. This approach could evolve to be a new, de-facto, regulatory requirement. Furthermore, if the 95% thresholds were to be periodically renormalized, this would constitute a process of continual ratcheting to ever more stringent performance expectations. During our meeting, we discussed the possibility that this could be avoided by developing plant-specific PI profiles and using trends to assess the performance status of the plant with respect to its specific acceptable performance level.

If, on the other hand, the 95% thresholds are one-time settings not subject to renormalization, the use of these thresholds will not lead to ratcheting and would serve the additional purpose of identifying potential outliers. In time, the process would evolve to the point that plant-specific considerations could be used to determine if these "outliers" actually have unacceptable performance.

We have also questioned the constraint of allowing only six months for the pilot program to assess the revised process. The concern is that a six-month pilot program could result in "cramming" (acceleration of both inspections and PI findings) a system intended to be exercised over a full year, such that the results may be distorted.

In addition, we believe that there is a need to use replicates in the pilot program to determine the effects of any uncontrolled variables such as the individuals performing the inspection. Clearly, it will be important to avoid confusing "inspector performance" with "licensee performance." As with any pilot program, there will be uncertainty associated with the results. The staff should include strategies for identifying and controlling such uncertainties in the interpretation of the results of the pilot program.

In the cover letter to SECY-99-007, the staff cites four policy issues that need to be addressed in conjunction with implementation of the revised Inspection and Assessment Programs. We have not heard the details of these policy issues, but expect to review them at a future meeting.

Sincerely,

/s/

Dana A. Powers
Chairman

References:

1. Memorandum dated January 8, 1999, from William D. Travers, Executive Director for Operations, NRC, for the Commissioners, SECY-99-007, Subject: Recommendations for Reactor Oversight Process Improvements.
2. Report dated December 16, 1998, from R. L. Seale, Chairman, ACRS, to William D. Travers, Executive Director for Operations, NRC, Subject: Proposed Improvements to the NRC Inspection and Assessment Programs - Interim Report.
3. Memorandum dated November 19, 1998, from John C. Hoyle, Secretary of the NRC, to William D. Travers, Executive Director for Operations, NRC, Subject: Staff Requirements - Briefing on Reactor Oversight Process Improvements.
4. Memorandum dated June 30, 1998, from John C. Hoyle, Secretary of the NRC, to L. Joseph Callan, Executive Director for Operations, NRC, Subject: Staff Requirements, SECY-98-045, Status of the Integrated Review of the NRC Assessment Process for Operating Commercial Nuclear Reactors.