Issue Date: 1/17/02

NRC Inspection Manual IIPB


Inspection Procedure 71152


Table Of Contents


Identification and Resolution of Problems

PROGRAM APPLICABILITY: 2515
CORNERSTONES: ALL
INSPECTION BASIS: A fundamental goal of the NRC's reactor oversight process is to establish confidence that each licensee is detecting and correcting problems in a manner that limits the risk to members of the public. A key premise of the revised oversight process is that weaknesses in licensee's problem identification and resolution (PI & R) programs will manifest themselves as performance issues which will be identified during the baseline inspection program or by crossing predetermined performance indicator thresholds. However, several aspects of PI & R are not specifically addressed by either the individual cornerstone performance indicators or other baseline inspections. These are detailed in the following objectives. Completion of the inspection objectives is accomplished by sampling issues during each inspectable area inspection, by performing focused reviews of three to six samples per year, and by performing a biennial focused PI & R inspection.
LEVEL OF EFFORT: Periodically screen active temporary modifications on systems which are ranked high in risk. Review the details of 4 to 6 temporary modifications a year for a 1-unit site; 5 to 7 for a 2-unit site; or 6 to 8 for a 3-unit site, respectively. Although the sample sizes are an annual goal, the inspection effort can be distributed on a quarterly basis.

71152-01 Inspection Objective

01.01 To provide for early warning of potential performance issues that could result in crossing thresholds in the action matrix
01.02 To help the NRC gage supplemental response should future action matrix thresholds be crossed
01.03 To provide insights into whether licensees have established a safety conscious work environment
01.04 To allow for follow-up of previously identified compliance issues (e.g. NCVs)
01.05 To provide additional information related to cross cutting issuesthat can be used in the assessment process
01.06 To determine whether licensees are complying with NRC regulations regarding corrective action programs

71152 Inspection Requirements

Within the baseline inspection program, problem identification and resolution (PI&R) activities are reviewed in three discrete but interdependent locations: as part of the "plant status" reviews described in Appendix D to Inspection Manual 2515 and as part of specific inspectable area inspection procedures (discussed in section 02.01 of this procedure); during follow-up to selected issues via paragraph 02.02 of this procedure; and during a biennial team inspection as specified in paragraph 02.03 of this procedure.
02.01 Routine Review of Identification and Resolution of Problems
As described in Appendix D to Inspection Manual Chapter 2515, "Plant Status" and by baseline inspectable area inspection procedures, conduct inspections of problem identification and resolution activities to:
a. Verify that equipment, human performance, and program issues are being identified by the licensee at an appropriate threshold and are being entered into the problem identification and resolution program.
b. Verify that corrective actions commensurate with the significance of the issue have been identified and implemented by the licensee.
The primary focus of these routine reviews should be on verifying that licensees are identifying issues at an appropriate threshold and entering them into their corrective action program. This can be assessed by comparing those issues identified by the NRC during the conduct of the plant status and inspectable area portions of the program with those issues identified by the licensee. This requirement is normally to be accomplished by Resident Inspectors and Region based inspectors responsible for conducting Plant Status and baseline inspectable area inspections. These routine reviews, along with those reviews conducted via section 02.02 of the procedure allow for follow-up to selected issues and operational occurrences to ensure that corrective actions commensurate with the significance of the issue have been identified and implemented by the licensee.
During inspector reviews of plant status and during inspections, inspectors should be alert for potentially significant conditions adverse to quality, such as equipment failures, performance deficiencies, inadequate maintenance work practices, personnel errors, inadequate risk assessment, management and emergent work control problems, procedure deficiencies, or noncompliances with procedures or regulatory requirements. When inspectors note such conditions, inspectors should examine the licensee's corrective action program records and/or attend licensee corrective action program meetings, to verify that the licensee identified the conditions noted by the inspector, and entered those conditions into the licensee's corrective action program.
Inspectors should also routinely review other deficient conditions that the licensee has entered into the corrective action program and the classification and immediate disposition of those conditions, to verify that the classification of the conditions is based at least in part on the relative risk/safety significance of the condition.
Inspectors should remain alert to problems/conditions which are potentially not minor and for which the licensee's investigation, conclusions, and/or corrective actions appear to be in some way inadequate. The inspector should also review the circumstances associated with the licensee's investigation and disposition of the problem/condition, to determine the reason(s) why the licensee's results were not adequate. The selected samples should be reviewed against the performance criteria contained in paragraph 03.01.b, as applicable.
When an inspector finds that the licensee's identification, classification, immediate disposition, and/or final disposition of a condition adverse to quality is not in compliance with the licensee's procedures and/or regulatory requirements, the inspector should assess the significance of that finding in accordance with IMC 0609. Document the team's results in accordance with the guidance contained in Appendix D to IMC 0612 (formerly 0610*)and section 03.01.c. of this procedure.
02.02 Selected Issue Follow-up Inspection
In addition to the above reviews which are performed as part of "plant status" or other baseline inspection procedures, a sample of three to six issues per year should be selected for a more in-depth periodic review. These samples need not be directly tied to the other baseline inspection procedure attachments and should generally be spread across the cornerstones of safety.
Use the general guidance contained in section 71152-03 as an aid in selecting samples for review. The selected samples should be reviewed against the performance criteria contained in paragraph 03.02.b. Document the team's results in accordance with the guidance contained in Appendix D to IMC 0612 and section 03.02.c. of this procedure.
02.03 Biennial Problem Identification and Resolution Inspection
Perform a biennial inspection of the problem identification and resolution activities as follows:
a. From among conditions adverse to quality which the licensee has identified and processed through its corrective action process and are associated in some way with risk-significant SSCs, select a sample for review. The sample selected should include maintenance effectiveness issues, if available. Also select for review the conditions adverse to quality which are in the licensee's corrective action program and are associated with cited or noncited violations of regulatory requirements. Use general guidance contained in sections 71152-03 and 03.03b as an aid in sample selection.
b. Review each condition/problem selected for review using the performance attributes contained in paragraph 03.03.c of the procedure.
c. Review recent audits and/or assessments of the licensee's corrective action program, and compare and contrast the results of those audits and/or assessments with the results developed through this inspection.
d. Using the guidance contained in paragraph 03.03.d assess whether there is indication that licensee personnel may be reluctant to report safety issues.
e. Develop an assessment of the licensee's corrective action program, based on the inspection results developed through steps a. through d.. Structure the assessment as summaries of inspection results and conclusions suggested by those results.
f. Document the team's results in accordance with the guidance contained in Appendix D to IMC 0612.

71152 Inspection Guidance

General Guidance

To the extent possible, this inspection should follow a performance based approach. Emphasize the products and results of the licensee's PI & R program. Inspections performed under this procedure should concentrate on the identification of problems and the effectiveness of corrective actions for risk significant issues rather than on reviewing the administrative aspects of the corrective action program and associated procedures.

This inspection will examine, in part, a sample of licensee corrective action issues to provide an indication of overall problem identification and resolution performance.

In selecting issues for inspection, the inspectors should seek the broadest range of examples within the cornerstone, including the following considerations:

  1. Licensee identified issues (including issues identified during audits or self assessments).
  2. NRC identified issues.
  3. Issues related to NCVs (for the biennial inspection it is mandatory to review the licensee's response to a sample of NCVs unless no NCVs were issued in the cornerstone).
  4. Issues identified through NRC generic communications.
  5. Issues identified through industry operating experience exchange mechanisms (including Part 21 reports, NSSS vendor reports, EPRI reports, experience reports from similar facilities, LERs).
  6. Specific or cross cutting issues identified by safety review committees or other management oversight mechanisms.
  7. Issues identified through employee concerns programs.

The above considerations are presented as guidance and should not be construed as a requirement to select one of each type of issue listed. The guidance is intended to help ensure that, over the course of an assessment cycle and through the performance of the baseline inspections, an appropriate sample will be obtained by which the NRC can obtain indication of the performance of the various elements of a licensee's corrective action program.

In selecting issues for review, inspectors should also use relevant risk insights such as:

  1. Maintenance Rule program basis documents,
  2. Current licensee risk analysis results or insights, and
  3. Significance Determination Program (SDP) worksheets for the plant.

For example, in considering the inspection of licensee corrective actions associated with post maintenance testing (as required by IP 71111, Attachment 19), inspectors should review issues associated with high risk mitigating systems. Additional insights for determining appropriate samples can be obtained by region based inspectors through discussion with resident inspectors or regional inspectors who are familiar with site issues and who are familiar with the licensee's problem identification and resolution process.

Detailed Review Guidance

The following additional guidance should be used in conducting a review of licensee problem identification and resolution activities.
03.01 Routine Review of Identification and Resolution of Problems
a. Relationship to Baseline Inspectable Area Procedures. Most of the attachments to baseline inspection procedures contain a requirement to inspect problem identification and resolution performance within the attachment's area. The routine inspection of PI&R performance as part of baseline inspections is intended to ensure that, over the course of an assessment cycle, a sample of PI&R performance in all cornerstones is obtained. As stated in paragraph 02.01, the primary focus of this portion of the PI & R review should be on verifying that licensees are identifying issues at an appropriate threshold and entering them into their corrective action program.
b. Performance Attributes. When evaluating the effectiveness of licensee corrective actions for a particular issue, the licensee's actions must be viewed against the nature and significance (or potential significance) of the identified problem. While licensee corrective action programs may appropriately consider monetary, plant availability, and other concerns as factors in determining significance, risk should be a primary factor in the licensee's significance determination. Attributes to consider during review of licensee actions associated with individual issues include:
  1. Complete and accurate identification of the problem in a timely manner commensurate with its significance and ease of discovery.
  2. Evaluations and disposition of performance issues associated with maintenance effectiveness, including maintenance errors, maintenance practices, work controls, and risk assessment.
  3. Evaluation and disposition of operability/reportability issues.
  4. Consideration of extent of condition, generic implications, common cause, and previous occurrences.
  5. Classification and prioritization of the resolution of the problem commensurate with its safety significance.
  6. Identification of root and contributing causes of the problem (this attribute will typically only be assessed for significant conditions adverse to quality and may be deferred to the biennial inspection or in-depth reviews performed elsewhere in this procedure.)
  7. Identification of corrective actions which are appropriately focused to correct the problem (may be deferred to biennial inspection).
  8. Completion of corrective actions in a timely manner commensurate with the safety significance of the issue (may be deferred to biennial inspection). If permanent corrective actions require significant time to implement, then verify that interim corrective actions and/or compensatory actions have been identified and implemented to minimize the problem and/or mitigate its effects, until the permanent action could be implemented.
It is not expected that the inspectors assess each attribute for every issue selected for followup during these routine reviews. Rather, inspectors may choose to assess licensee performance against selected attributes, as necessary to be most effective.
c. Documentation. In order to support a more complete assessment of the effectiveness of the licensee's PI&R program, it is important that the NRC document the results of PI & R inspections conducted as part of the baseline procedure attachments. It is expected that documenting reviews of PI&R will help focus the biennial PI&R inspection on areas where concerns have been identified. In general, issues associated with the PI & R program itself should be documented in the PI & R section of the inspection report. Technical issues associated with the inspectable area and cornerstone should be documented in the associated areas of the inspection report. Specific guidance regarding documentation of inspection scope and thresholds for PI&R issues is contained in IMC 0612.
d. Level of Effort. While it is expected that routine reviews of PI & R activities should equate to approximately 10-15 percent of the resources estimated for the associated baseline cornerstone procedures, this is a general estimate only based upon the overall effort expected to be expended in each strategic performance area. It is anticipated that the actual hours required to be expended may vary significantly from attachment to attachment, depending upon the nature and complexity of the issues that arise at the particular facility. Overall, an effort should be made to remain within the 10 to 15 percent estimate on a strategic performance area basis. Inspection time spent assessing PI & R as part of the baseline procedure attachments should be charged to the corresponding baseline procedure attachment.
03.02 Selected Issue Follow-up Inspection
a. An additional sample of three to six issues per year should be chosen for more in depth review, as necessary to verify that the licensee has taken corrective actions commensurate with the significance of the issue. This sample can be chosen using information obtained from plant status reviews and from reviews conducted as part of the baseline inspection procedure attachments, but need not be limited to those issues that are directly related to the inspection procedure attachments. Samples may also be chosen from the list contained in section 71152-03 of this procedure. The selected sample may also be an issue that is tracked by a performance indicator, for which a color change has yet to occur.
b. Performance Attributes
When evaluating the effectiveness of licensee corrective actions for a particular issue, the licensee's actions must be viewed against the nature and significance of the identified problem. While licensee corrective action programs may appropriately consider monetary, plant availability, and other concerns as factors in determining significance, risk should be a primary factor in the licensee's significance determination. Attributes to consider during review of licensee actions associated with individual issues include:
  1. Complete and accurate identification of the problem in a timely manner commensurate with its significance and ease of discovery.
  2. Evaluations and disposition of performance issues associated with maintenance effectiveness.
  3. Evaluation and disposition of operability/reportability issues.
  4. Consideration of extent of condition, generic implications, common cause, and previous occurrences.
  5. Classification and prioritization of the resolution of the problem commensurate with its safety significance.
  6. Identification of root and contributing causes of the problem (this attribute will typically only be assessed for significant conditions adverse to quality).
  7. Identification of corrective actions which are appropriately focused to correct the problem.
  8. Completion of corrective actions in a timely manner commensurate with the safety significance of the issue. If permanent corrective actions require significant time to implement, then verify that interim corrective actions and/or compensatory actions have been identified and implemented to minimize the problem and/or mitigate its effects, until the permanent action could be implemented.
In addition to the general performance attributes contained above, the inspector should refer to Inspection Procedure 95001 for additional guidance on assessing licensee evaluations of significant performance issues. It is not expected that the inspectors assess each attribute for every issue selected for followup during these routine reviews. Rather, inspectors may choose to assess licensee performance against selected attributes, as necessary to be most effective.
c. Documentation
The basis for selection and the scope of review of each sample should be documented in the "Scope" section of 4OA2 of the inspection report. In general, issues associated with the PI & R programs implementation should be documented in the "Findings" section of 4OA2 of the report. This documentation may include factual information that does not rise to the level of an inspection finding. Assessments of PI&R effectiveness will not be done during these inspections. Such assessments will only be done during the periodic team inspection. Only green or greater findings will be included in the summary of findings of the inspection report. Technical issues associated with other inspectable areas and cornerstones should also be documented in those section of the report.
d. Level of Effort
The review of three to six samples should take an average 50-70 hours per year of direct inspection effort and may be completed by resident and/or regional inspectors. Inspection time associated with this review should be charged to this procedure.
03.03 Biennial Problem Identification and Resolution Inspection. The biennial inspection of problem identification and resolution is intended to complement and expand upon the reviews described in Section 03.01 and 03.02 of this procedure by:
  1. Evaluating additional examples of licensee problem identification and resolution.
  2. Reviewing the resolution of issues that earlier had been assessed for the licensee's identification efforts only.
  3. Comparing the NRC's results against the licensee's own assessment of performance in the PI & R area.
  4. Assessing whether PI & R deficiencies exist across cornerstones that might indicate potential programmatic issues.
a. Planning. Obtain licensee administrative procedures that control the identification, evaluation, and resolution of problems. Selected licensee documents needed to support the inspection may be obtained prior to the inspection. These documents should only be reviewed to provide the inspectors with sufficient knowledge of the licensee's programs and processes, as necessary to conduct an effective and efficient inspection.

Obtain and review documents for the in-office review, such as a list of corrective action documents issued from the time of the last PI & R inspection (e.g. a list of work orders, work requests, temporary modifications, calibration failures, condition/problem identification reports, operability evaluations and determinations, etc.). Also obtain relevant licensee corrective action program assessments, program performance information, and trend reports.

Obtain and review procedures and documentation on licensee efforts to identify, resolve and prevent structure, system and component (SSC) performance problems through performance monitoring, root cause analysis, cause determination and corrective action to meet the monitoring requirements of the MR (10 CFR 50.65).

Obtain and review all NRC inspection reports issued since the last PI&R inspection and:

  1. Determine the extent to which all cornerstones have been sampled by routine reviews of licensee PI&R activities and determine if additional PI&R samples are warranted in any cornerstone(s).
  2. Determine the extent to which licensee actions to NCVs have been sampled by routine reviews of licensee PI&R activities.
  3. Identify any trends or patterns in corrective action program issues or performance which may warrant additional sampling to confirm. For example, a series of issues associated with "failure to follow procedures" within one cornerstone may indicate a corrective action performance deficiency within a portion of the licensee's organization; a series of issues associated with failure to follow procedures in multiple cornerstones may indicate a broader concern. Also, a lack of licensee identified corrective action issues within a particular organization may be indicative of a problem with the identification threshold.
b. Biennial Inspection Sample Selection. Based on the planning review, identify a sample of licensee corrective actions for review. The samples chosen for review should include a range of issues selected from the list in section 71152-03.

No specific number of previously reviewed or additional samples is specified. Rather, the biennial inspection team leader should choose as many examples as warranted to complement the routine PI&R inspections and ensure a sufficient basis for evaluating the effectiveness of the licensee's PI&R program. The inspectors, as an option, may consider selecting one or more risk significant systems and using a "vertical slice" approach to picking the inspection sample, so long as the system(s) selected will provide adequate coverage across all cornerstones. In such cases, additional samples may be required to ensure adequate coverage across all cornerstones. An effort should however be made to maintain the total hours expended in completing this procedure to within the estimated level of resources contained in paragraph 71152-04.

c. Performance Attributes. When evaluating the effectiveness of licensee corrective actions for a particular issue or issues, the licensee's actions must be viewed against the nature and significance of the identified problem. While licensee corrective action programs may appropriately consider monetary, plant availability, and other concerns as factors in determining significance, risk should be a primary factor in the licensee's significance determination. Attributes to consider during review of licensee corrective actions include:
  1. Complete and accurate identification of the problem in a timely manner commensurate with its significance and ease of discovery.
  2. Evaluation and disposition of operability/reportability issues.
  3. Consideration of extent of condition, generic implications, common cause, and previous occurrences.
  4. Identification of significant negative trends associated with human or equipment performance.
  5. Classification and prioritization of the resolution of the problem commensurate with its safety significance.
  6. Identification of root and contributing causes of the problem for significant conditions adverse to quality.
  7. Identification of corrective actions which are appropriately focused to correct the problem (and to address the root and contributing causes for significant conditions adverse to quality).
  8. Completion of corrective actions in a timely manner commensurate with the safety significance of the issue (included within this attribute would be justifications for extending corrective action due dates). If permanent corrective actions require significant time to implement, then verify that interim corrective actions and/or compensatory actions have been identified and implemented to minimize the problem and/or mitigate its effects, until the permanent action could be implemented.
  9. In addition, for the above samples that involve maintenance effectiveness, the inspector should verify the following:
(a) The licensee has designated those SSCs as 10 CFR 50.65(a)(1) as appropriate,

(b) Determine if corrective actions for 10 CFR 50.65(a)(1) SSCs are adequate,

(c) Review MR repetitive maintenance preventable functional failures (MPFFs) for indications of weaknesses in the licensee's corrective action program. In addition, identify any problems with root cause analysis or cause determination and corrective action for SSCs experiencing repetitive MPFFs or exceeding their goals or performance criteria,

(d) Ensure that risk assessment, risk management, and emergent work control problems associated with maintenance are identified and resolved promptly.

Additional guidance relative to the MR can be found in NUMARC 93-01, "Industry Guideline for monitoring the Effectiveness of Maintenance at Nuclear Power Plants," and Regulatory Guide 1.160 which endorses the industry guidance.
In addition to the performance attributes contained above, the inspector should refer to Inspection Procedure 95001 for additional guidance on assessing licensee evaluations of significant performance issues.
d. Assessment of Safety Conscious Work Environment. In conducting interviews with or observing other activities involving licensee personnel during the inspection, be sensitive to areas where employees may be reluctant to raise concerns. Although the licensee may be implementing an employee concerns program regarding the identification of safety issues, the possibility of existing underlying factors that would produce a "chilling" effect or reluctance to report such issues could exist and the inspector should be alert for such indications.

Appendix 1 to this procedure provides a list of questions that can be used when discussing PI & R issues with licensee individuals to help assess whether there are impediments to the establishment of a safety conscious work environment. It is not intended that inspectors conduct formal interviews solely for the purpose of assessing the work environment, but rather, that the inspectors make use of the questions in Appendix 1 during discussions with licensee individuals concerning other attributes of the inspection. It is expected that during this inspection, discussions/interviews will be held with both licensee management and staff. If, as a result of the interviews or observations, the inspector becomes aware of specific examples of employees being discouraged from raising safety or regulatory issues within the licensee's or contractor's organization or to the NRC, the inspector should get as complete a set of facts as possible. If the inspector becomes aware of a reluctance of employees to raise safety or regulatory issues unrelated to a specific event or incident, continue pursuing the issue during the remaining interviews and try to determine the reason employees are reluctant to raise issues. However, if any indication of a "chilling" effect is suspected, inform regional management for further review and follow-up. Inspectors should be sensitive to the need to appropriately capture and forward any allegations that may be received during the inspection.

e. Development of PI&R Program Performance Insights. By reviewing a sufficient number and breadth of samples, the inspection team should be able to develop insights into the effectiveness of the licensee's corrective action program. Compare the result of the team's review of corrective action issues with licensee performance reviews, including specific licensee reviews of the corrective action program. Determine whether licensee reviews are consistent with the NRC review of corrective action issues.

The intent of this inspection procedure (both the routine and biennial inspection effort) is to provide insights into licensee performance in the PI & R area based upon a performance based review of corrective action issues. More detailed programmatic reviews of licensee performance in the PI & R area will be conducted during supplemental inspections, in accordance with the assessment action matrix, should established performance thresholds be crossed.

f. Documentation and Evaluation of Program Effectiveness. At the completion of inspection activities, the team should develop a clear and concise discussion of the results of their review. This discussion should be supported by the inspection activities conducted over the assessment cycle including routine inspections, selected sample follow-up inspections, and the biennial inspection of PI&R activities. The discussion should be documented in the inspection report for the biennial PI & R inspection and should be included in the PIM. Included in the documentation should be any issues associated with establishment of a safety conscious work environment that may have been detected during the inspection.

Additional evaluation of the licensee's PI & R programs will be conducted as part of the mid-cycle and/or end of cycle plant performance review by assessing licensee performance using the results of this inspection, as well as other information, including performance indicator data and the results of any supplemental inspections. Additional guidance on documenting the biennial problem identification and resolution inspection is contained in IMC 0612.

71152-04 Resource Estimate

The biennial team inspection will involve on average 212 to 288 hours of direct inspection. Participation (either full or part time) on the inspection team by a member of the resident inspector staff should be strongly considered.

The review of the three to six samples per paragraph 02.01 should take an average 50-70 hours per year of direct inspection.

END