ROP Evaluation

icon/hyperlink for: Evaluation of the ROP page; consisting of a circle target image with light green outline and red and white alternating rings with a large white arrowhead pointer pointing in the center

On this page:

This page includes links to files in non-HTML format. See Plugins, Viewers, and Other Tools for more information.

Annual Self-Assessments

On February 24, 2000, the staff issued SECY-00-0049, "Results of the Revised Reactor Oversight Process Pilot Program". The resulting staff requirements memorandum (SRM), issued on March 28, 2000, approved initial implementation of the ROP as recommended by the staff. The initial implementation of the ROP began on April 2, 2000. In a follow-up SRM issued on May 17, 2000, the Commission directed the staff to report on the ROP results after the first year of implementation. The staff did so and documented the results in SECY-01-0114, "Results of the Initial Implementation of the New Reactor Oversight Process", issued June 25, 2001. SECY-01-0114 also noted the staff's intention to continue to perform an annual self-assessment of the ROP.

The objectives and details of the ROP self-assessment program are contained in IMC 0307, "Reactor Oversight Process Self-Assessment Program" and its appendices. ROP self-assessment activities leverage data analysis and trending tools to (1) assess program-wide uniformity and effectiveness of ROP implementation, (2) evaluate the effectiveness of recent significant changes to the ROP, (3) evaluate the NRC's response to significant licensee events and declining licensee performance, and (4) perform periodic in-depth reviews of specific areas of interest, including the baseline inspection program. These ROP self-assessment activities help the staff determine whether the ROP was implemented in accordance with current governance documents, whether it was implemented uniformly and effectively across the regions and headquarters, whether it met its pre-established ROP goals and intended outcomes, and whether it was implemented consistent with the NRC Principles of Good Regulation. Taken in aggregate, these determinations form the basis for the staff's annual evaluation of whether the ROP remains effective. Finally, the ROP self-assessment approach ensures that the staff appropriately invests resources in addressing lessons learned that improve the efficiency and effectiveness of the ROP.

Each year, the staff issues an ROP performance metrics report and an ROP self-assessment Commission paper in support of the Agency Action Review Meeting (AARM). The staff also briefs the Commission on the ROP self-assessment results following the AARM. The results of these activities are summarized in the table below.

Time Period Commission Paper Commission Meeting Commission SRM Metric Report
CY 2020 SECY-21-0038 June 10, 2021 Meeting SRM M210610 CY 2020
CY 2019 SECY-20-0040 September 30, 2020 Meeting SRM M200930 CY 2019
CY 2018 SECY-19-0037 June 20, 2019 Meeting SRM M190620 CY 2018
CY 2017 SECY-18-0045 June 19, 2018 Meeting SRM M180619 CY 2017
CY 2016 SECY-17-0049 June 15, 2017 Meeting SRM M170615 CY 2016
CY 2015 SECY-16-0047 June 2, 2016 Meeting SRM M160602B CY 2015
CY 2014 COMSECY-14-0030 May 21, 2015 COMSECY SRM

Meeting SRM M150521B 
 
CY 2013 SECY-14-0047 June 3, 2014 Meeting SRM M140603 CY 2013
CY 2012 SECY-13-0037 May 29, 2013 Meeting SRM M130529 CY 2012
CY 2011 SECY-12-0055 June 1, 2012 Meeting SRM M120601 CY 2011
CY 2010 SECY-11-0054 May 27, 2011 Meeting SRM M110527 CY 2010
CY 2009 SECY-10-0042 May 27, 2010 Meeting SRM M100527 CY 2009
CY 2008 SECY-09-0054 May 14, 2009 Meeting SRM M090514 CY 2008
CY 2007 SECY-08-0046 June 4, 2008 Meeting SRM M080604 CY 2007
CY 2006 SECY-07-0069 May 31, 2007 Meeting SRM M070531 CY 2006
CY 2005 SECY-06-0074 May 16, 2006 Meeting SRM M060516B CY 2005
CY 2004 SECY-05-0070 May 25, 2005 Meeting SRM M050525B CY 2004
CY 2003 SECY-04-0053 May 4, 2004 Meeting SRM M040504B SECY-04-0053, Attachment 3
CY 2002 SECY-03-0062 May 15, 2003 Meeting SRM M030515 SECY-03-0062, Attachment 3
CY 2001 SECY-02-0062 May 1, 2002 Meeting SRM M020501 SECY-02-0062, Attachment 2
Initial Implementation SECY-01-0114 July 19, 2001 Meeting SRM M010719B N/A
Pilot Program SECY-00-0049 March 7, 2000 SRM 1 / SRM 2 N/A

Independent Evaluations

In addition to the ROP self-assessment program, several independent evaluations have been performed since the inception of the ROP to analyze its effectiveness and recommend improvements. Over the past several years, the Government Accountability Office (GAO), the Office of Management and Budget (OMB), the Office of the Inspector General (OIG), the Advisory Committee for Reactor Safeguards (ACRS), the Davis-Besse Lessons Learned Task Force (DBLLTF), and the SDP Task Group have all performed evaluations of the ROP. Early on, two evaluation panels performed detailed reviews of the pilot program and initial implementation of the ROP. These evaluations have generally provided favorable results, but have also suggested potential areas of improvement. Several recommendations from these independent evaluations have been or are being incorporated into the ROP as detailed in the annual self-assessments above and staff responses below.

Entity Final Report Staff Response/Program Improvements
Audit of the NRC's Nuclear Power Reactor Inspection Issue Screening OIG-21-A-07 OIG-21-A-07 05/13/2021
Audit of NRC's Nuclear Power Plant Surveillance Test Inspection Program OIG-20-A-11 OIG-20-A-11 02/11/2021, 08/11/2020
Audit of the NRC Oversight of Supplemental Inspection Corrective Actions OIG-19-A-19 OIG-19-A-19 04/13/2021, 11/09/2020, 11/21/2019
Audit of NRC's Cyber Security Inspections at Nuclear Power Plants OIG-19-A-13 OIG-19-A-13 01/13/2021, 12/03/2019, 08/20/2019
Audit of NRC's Special and Infrequently Performed Inspections OIG-18-A-13 OIG-18-A-13 02/03/2021, 07/30/2020, 03/12/2020, 12/11/2019, 08/20/2019, 07/25/2018
Audit of NRC's Fire Protection Oversight for Operating Reactors OIG-17-A-10 OIG-17-A-10 02/01/2018, 05/31/2017
Audit of NRC's SDP for Reactor Safety OIG-16-A-21 OIG-16-A-21 03/16/2020, 08/20/2019, 02/27/2019, 10/15/2018, 03/01/2018, 08/08/2017, 12/01/2016
Audit of NRC's Oversight of 10 CFR 50.59 OIG-16-A-19 OIG-16-A-19 03/29/2017, 10/05/2016
Audit of NRC's ROP: Reactor Safety Baseline Inspection Procedures OIG-16-A-12 OIG-16-A-12 02/10/2017, 10/27/2016, 06/06/2016
Audit of NRC's Oversight of Spent Fuel Pools OIG-15-A-06 OIG-15-A-06 02/10/2021, 02/11/2020, 03/11/2019, 12/04/2018, 03/15/2018, 08/10/2017, 12/08/2016, 06/10/2016, 05/18/2016, 12/02/2015, 04/29/2015
Audit of NRC's Cyber Security Inspection Program for Nuclear Power Plants OIG-14-A-15 OIG-14-A-15 No Recommendations
Audit of NRC's Management of the Baseline Security Inspection Program OIG-12-A-10 OIG-12-A-10 01/04/2018, 10/10/2017,03/16/2017, 11/12/2015, 01/22/2015, 07/09/2013, 11/20/2012, 05/02/2012
OIG audit of Support to Resident Inspectors OIG-14-A-12  03/2014 OIG-14-A-12, Appendix C
Commission-Directed Independent Review of ROP per SRM to SECY-12-0081 Working Group Report  02/2014 SECY-14-0047, Enclosure 2
OIG audit of Active Component Aging OIG-14-A-02  10/2013 OIG-14-A-02, Appendix B
GAO report on Regional Differences and Access to Information GAO-13-743 09/2013 Response  12/2013
Independent Evaluation of the Reactor Oversight and Incident Response Program 12/31/2008 Action Plan 6/09
NRC Cross-cutting review and Palo Verde Lessons Learned 12/2008 Draft dispositions current as of April 17, 2008
GAO report on Nuclear Reactor Oversight GAO-06-1029 09/2006 Response 11/06
OIG audit of the Reactor Program System OIG-05-A-11 04/2005 Status 11/05 (closes items 5 and 10)
Status 02/06 (closes remaining items)
OIG audit of the Baseline Inspection Program OIG-05-A-06 12/2004 Status 09/05 (closes items 1, 5, 6, 9, and 10)
Status 12/05 (closes items 7 and 8)
Status 06/06 (closes items 3 and 4)
OIG audit of the Significance Determination Process OIG-02-A-15 08/2002 Status updates in annual SECYs
SDP Task Group Task Group Report 12/2002 Status updates in annual SECYs
Advisory Committee for Reactor Safeguards Latest Letter 03/2003 Latest response 04/2003
Davis-Besse Lessons Learned Task Force DBLLTF final report 09/2002 Final Report
Status/Closure
Significant Improvements
OMB Program Assessment Rating Tool Evaluation Final Report & Status of Performance Measures and Follow-up Actions 09/2003 Final Report & Status of Performance Measures and Follow-up Actions
Initial Implementation Evaluation Panel IIEP final report 05/2001 SECY-01-0014, Attachment 5
Pilot Program Evaluation Panel PPEP final report 12/1999 SECY-00-0049, Attachment 3

Stakeholder Feedback

External Surveys/Feedback

External stakeholder engagement opportunities are not limited to the following; however, these are the primary Communication events:
 
Monthly Public Meetings: The staff conducts monthly public working-level meetings with the Nuclear Energy Institute (NEI), the industry, and interested stakeholders to discuss the status of ongoing refinements to the ROP.   This is an opportunity to solicit for feedback from NEI, industry, and interested stakeholders on initiatives or status of projects. Summaries from these meetings are located on the NRC website.
 
Local public meetings and engagements in vicinity of each operating reactor: The staff conducts public meetings or other engagement activities in the vicinity of each operating reactor to discuss the results of the NRC’s assessment of the licensee’s performance. These annual engagements provide an opportunity to engage interested stakeholders on the NRC’s role in ensuring safe and secure plant operations.
 
Annual Regulatory Information Conference (RIC): The staff typically sponsors a technical session at the annual RIC focused on ROP-related issues and topics of interest, or sponsors an interactive ROP exhibit so that interested stakeholders can gather information and ask questions about the ROP.  Additional ROP topics are typically discussed during the regional technical session. Please see RIC website
 
External Feedback Forms: In addition to sharing staff perspectives and gathering feedback during public meeting discussions, the staff compiles feedback forms from ROP public meetings (i.e., the working group meetings, focused meetings/workshops, and annual assessment meetings) to evaluate the effectiveness of public communications and use of plain language. These feedback forms are used to inform external communication initiatives to ensure that external content and publications relative to the ROP are clear, comprehensible, and understandable.
 
ROP “Contact Us” Form: External stakeholders can provide feedback or ask questions regarding ROP implementation across a wide array of topic areas using the ROP “Contact Us” link. 
 
Historical External Surveys:  From the inception of the ROP through CY 2011, the staff published regular surveys in the Federal Register for external stakeholders to specifically provide feedback on the ROP.  The staff has since discontinued this practice as discussed in SECY-14-0047.  The general results of those surveys were discussed in the respective annual ROP self-assessment Commission papers.  From CY 2004 through CY 2011, the staff provided comprehensive detailed responses to the feedback received in those surveys and those staff responses are listed below.
 

CY 2011/2010 Consolidated Response
CY 2009/2008 Consolidated Response
CY 2007 Consolidated Response
CY 2006 Consolidated Response
CY 2005 Consolidated Response
CY 2004 Consolidated Response

 

Internal Surveys/Feedback

Internal stakeholder engagement opportunities are not limited to the following; however, these are the primary Communication events:

  •  NRC staff and management conduct biweekly conference calls with regional management and staff to discuss current issues associated with the ROP. The staff also meets periodically with regional managers to discuss more complex ROP topics and issues.

  • Headquarters staff participate in each region’s inspector counterpart meeting and/or end-of-cycle meetings so that regional staff and management can provide feedback on ROP implementation.

  • The ROP feedback process described in IMC 0801 allows the NRC staff to identify concerns or issues and recommend improvements related to ROP governance or guidance documents. Responsible staff will respond to and address the feedback in accordance with the requirements and expectations of IMC 0801.

  • NRC staff maintains an internal Contact Us Form where internal stakeholders can provide feedback or ask questions regarding ROP implementation across a wide array of topic areas, including but not limited to resident support concerns. 

 

Page Last Reviewed/Updated Friday, September 17, 2021