Issue Date: 09/29/98

NRC INSPECTION MANUAL HQMB


INSPECTION PROCEDURE 35702


Inspection of Quality Verification Function

SALP FUNCTIONAL AREA: ENGINEERING (ENG)
PROGRAM APPLICABILITY: 2515

35702-01 INSPECTION OBJECTIVE

To assess the effectiveness of licensee quality verification organizations in identifying technical issues and problems having safety significance and in following up to ensure that issues and problems are resolved in a timely manner.

35702-02 INSPECTION REQUIREMENTS

02.01 Analyze Existing Problems. The NRC inspector should perform a detailed analysis of selected existing problems relative to plant performance, and evaluate why the licensee's quality verification organizations were not effective in identifying issues affecting plant reliability and safety; or similarly, why licensee quality verification organizations have not been effective in identifying and addressing the precursors or root causes of events.

02.02 Assess Licensee Quality Verification. On the basis of the evaluation of the results of the activities cited in paragraph 02.01, the NRC inspector should assess and draw conclusions on how well the licensee quality verification organizations accomplished the following:

      a. Planning for each quality verification activity.
b. Providing individuals with adequate technical experience and expertise to conduct the quality verification activity.
c. Conducting the quality verification activity in adequate depth and with the appropriate emphasis on technical activities.
d. Ensuring effective corrective action.

35702-03 INSPECTION GUIDANCE

General Guidance

This inspection procedure would be implemented when quality verification issues affecting plant reliability and safety are identified. This inspection can be performed by an individual inspector or by a team of inspectors. Licensee management has a number of organizations that perform quality verification functions. The organizations include Quality Assurance (QA), Quality Control (QC), Quality Engineering (QE), and independent review groups, such as the Independent Safety Engineering Group (ISEG). Quality verification organizations perform various types of verifications, such as audits, surveillances, and third-party observations. These organizations perform their verifications in a number of functional disciplines, such as maintenance, operations, and design.

Quality verification organizations perform a measurement and advisory function, monitoring the overall performance of the plant; identifying substandard or anomalous performance and precursors of potential problems; reporting findings in an understandable form and in a timely fashion to a level of line management having the authority to effect corrective action; and verifying promptly the effectiveness of the corrective action and reporting those verification results back to line management. An effective quality verification organization is technically and performance oriented; its primary focus tends to be toward the end product with a secondary effort toward processes and procedures. The organization should have adequate technical resources available to it, and it should be aggressive in searching for, identifying, and following up on problems.

This NRC inspection will use either significant plant problems that have recently occurred or significant NRC technical inspection findings to assess licensee quality verification effectiveness. This technique is appropriate to assess the roles of the quality verification organizations and line management before and during resolution of an issue. Quality verification organizations are responsible for ensuring that quality is achieved and problems are avoided. Thus, the presence of problems reflects on the performance of quality verification organizations, as well as that of line management.

Specific Guidance

03.01 Analyze Existing Problems. This analysis should be accomplished through direct observation of in-process activities, interviews of plant personnel, and documentation reviews of a selective sample of completed licensee quality verification activities performed within the past 12 months. For review of maintenance activities, the assessment period should cover the most recent maintenance evaluation cycle for implementation of 10 CFR 50.65(a)(3) (i.e., the last refueling cycle not to exceed 24 months). In accordance with NUMARC 93-01, Industry Guideline for Monitoring the Effectiveness of Maintenance at Nuclear Power Plants, Section 13, documentation developed to implement the maintenance rule is not subject to the utility quality assurance program unless the documentation used has been previously defined as within the scope of the quality assurance program.

The NRC inspector should perform a detailed analysis of the selected issues (i.e., problem areas and occurrences of events) to understand them technically, why they occurred, and what roles the quality verification organizations and line management played in the issues identification and resolution of issues. For selected problems and events, this analysis should include: (1) determining the chain of events leading to the occurrence of the problem, (2) developing an understanding of the technical and work activities associated with the problem, (3) determining the information that is needed to understand its generic implications, (4) determining the extent to which the licensee identified precursors and the root cause of the problem and investigated the facts surrounding its occurrence, and (5) identifying the licensee's actions to correct the problem and the remedial actions taken to preclude its recurrence.

The NRC inspector should use plant-specific information, including the findings of previous NRC inspections and information obtained from discussions with the NRC resident inspectors, to perform the analysis. Other sources, such as Systematic Assessments of Licensee Performance, Licensee Event Reports, Information Notices, NRC Bulletins, employee concerns, probabilistic risk assessments, current internal license plant status or problem reports, and industry-wide operating experience, also should guide the inspector.

As practical, for maintenance issues, the inspector should also evaluate the extent to which the licensee's quality verification organizations are monitoring the effectiveness of the licensee's maintenance rule (10 CFR 50.65) implementation program. Following the guidance contained in Regulatory Guide 1.160, Monitoring the Effectiveness of Maintenance at Nuclear Power Plants, and NUMARC 93-01, licensees (1) monitor the performance or condition of structures, systems, and components (SSCs) within the scope of 10 CFR 50.65(b), (2) set goals commensurate with safety taking into account industry-wide operating experience for SSCs monitored under 50.65(a)(1), (3) perform root cause analyses, cause determination and corrective actions that preclude recurrence of repetitive maintenance preventable functional failures and goals or performance criteria not being met for SSCs monitored under 50.65(a)(1) and (a)(2), (4) perform periodic evaluations of performance and condition monitoring and preventive maintenance activities, and (5) assess overall plant safety prior to taking SSCs out of service for maintenance activities.

These activities should be reviewed on a sampling basis by the licensee's quality verification organizations, as part of their monitoring program. Additional guidance can be found in NUMARC 93-01 "Industry Guideline for Monitoring the Effectiveness of Maintenance at Nuclear Power Plants," Regulatory Guide 1.160, "Monitoring the Effectiveness of Maintenance at Nuclear Power Plants," which endorses the industry guidance, and Inspection Procedure (IP) 62706, "Maintenance Rule". When reviewing quality verification of maintenance activities, the inspector should concentrate on risk significant performance examples that are a product of poor maintenance programs or implementation. The inspector should be aware that certain non-safety related SSCs may be scoped within the maintenance rule which are not governed by the licensee's Appendix B QA program and should therefore exercise caution when inspecting to and enforcing the maintenance rule. If significant problems are identified within the auspices of the maintenance rule, the inspector should consult with and identify any concerns to regional management who may consider a more detailed inspection in accordance with IP 62706.

The NRC inspector should choose a selective sample of activities and documents to provide a quick picture of performance, thus preparing for a more effective inspection. Items should be selected in areas where known or suspected deficiencies exist. The size of the selective sample is established when the inspector is satisfied that sufficient data have been reviewed and documented to support a conclusion about the area sampled.

It is important that the NRC inspector be accompanied by individuals having the necessary technical or operational experience and expertise when assessing areas other than their own areas of expertise.

03.02 Assess Licensee Quality Verification

      a. The NRC inspector should assess the licensee's planning to ensure that it takes into account trends in performance and responds to events. The NRC inspector should also review the development of a written audit plan and the selection and orientation of the auditors to ensure that the effort is well organized and efficiently completed.
Scheduling should be dynamic. Successful verification programs use a "living" schedule that permits important plant activities and events to be verified and documented as they are occurring. This process provides for the greatest impact and most effective follow-up.
b. The importance of having licensee personnel with direct experience and expertise in the area they are reviewing cannot be overemphasized. Personnel interviews will provide useful information. Likewise, qualification records, which should be current and readily available, are useful for evaluating an individual's capability to competently verify a specific area. However, interviews and records should not be used as the sole bases for questioning an individual's ability. The fewer technically-experienced personnel a quality verification organization has, the more closely the NRC inspector should assess the licensee's verifications to determine its ability to detect operational and technical quality flaws.
For members of the licensee's quality verification organization to function effectively in areas other than their own areas of expertise, it is essential that they be accompanied by individuals who have the necessary technical or operational background to find the obvious as well as the not so obvious quality conditions requiring corrective action.
c. Organizations perform various types of quality verifications. Audits are one of the most obvious. The inspector should be aware of the various types of verifications and their relation to other parts of the quality verification program.
The licensee's quality verifications should be in depth, not superficial, and should emphasize technical achievement more than programmatic conformance. Time spent on a verification by licensee personnel, when compared to the complexity of the subject matter being verified, should provide insight into the depth of the verification and its importance to the licensee. The NRC inspector should determine the level of the verification's detail. For example, is the licensee's design verification activity only a check to verify that the calculations and analyses have been performed, or is it a check of the validity of the calculations and a verification of the engineering basis for them? The results of an achievement-oriented quality verification may indicate that wider problems exist. In such a case, the quality verification organization should examine the programmatic implications and identify other technical areas where review is warranted.
It is important for the quality verification activity report to accurately reflect the collected data. Otherwise, management would be receiving an incomplete or biased perspective of the plant's quality achievement. A report that overstates the effort expended or draws positive conclusions from superficial verifications should be identified as a weakness in the verification program. Problems with timeliness, clarity, root cause identification, or accuracy in reporting may also hamper the corrective action process and should be watched for by the inspector.
Through personnel interviews, the NRC inspector should evaluate the adequacy of communication and cooperation among the licensee's quality verification organization, the functional disciplines, and line management.
d. The NRC inspector should evaluate each response to a quality verification organization finding and the adequacy of the licensee organization's actions to resolve the finding. The actions should correct root causes, not merely the symptoms of those causes. The implications of the deficiency on other plant activities should be a consideration in the assessment of the response. Appendix B to 10 CFR Part 50 requires that the cause of the condition be determined and remedied to preclude repetition. For each analysis reviewed, the inspector should evaluate the reported root cause and the adequacy of the steps taken to determine the actual root cause.
The licensee's quality verification organization should follow up on corrective actions in an aggressive manner. The organization should have access to upper line management and the authority to effectively use that access. The NRC inspector should determine the extent to which the quality verification organizations are actually meeting with line management, as this is a key indicator of their effectiveness.
The NRC inspector should assess the effectiveness of the licensee's quality verification organizations and line management to determine whether identified quality deficiencies are responded to promptly and effectively. Although a separate escalation procedure may not be required, the organization should have established a means to bring the attention of upper management to those problems that are not promptly and adequately corrected. Significant corrective actions remaining open for excessive periods or being extended repeatedly without apparent reason are examples of an ineffective corrective action follow-up program. Those examples also may be indicative of a line management problem.
Closures should not be based on superficial reviews or completed without a determination that the corrective action will be effective in preventing either recurrence of the problems or emergence of similar problems.
The licensee organization should have an established and effective program to identify significant trends.
Because various deficiency tracking and corrective action systems exist inside and outside of the licensee quality verification organization, the NRC inspector should identify the total extent of those systems and their overlapping responsibilities and requirements, if any. The more overlapping and ambiguous the system control, the more likely it is that problems will occur, such as a deficiency being tracked in a system that is not monitored by the group responsible for the corrective action.

35702-04 RESOURCE ESTIMATE

This inspection can be performed by an individual inspector or by a team of inspectors, depending on the issues identified. This inspection is assumed, for resource estimate purposes, to be a team inspection that will evaluate two or more functional areas over a 2-week onsite inspection. The inspection will average 360 hours of direct inspection.

END