Home NRC Library Document Collections Events, Reports Associated with Part 21 Reports 1996

Part 21 Report - 1996-731

ACCESSION #: 9610040032 Anchor/Darling Valve Company o Williamsport, PA 17701 W. G. KNECHT Technical Director September 19, 1996 FAX TO: Dave Skeen U.S.N.R.C. 301-415-2279 SUBJECT: Part 21 Reported Incident on FP&L, 2" Lift Check Valve Attached: --Copy of A/DV letter being sent to customers who were furnished 2" lift check valves --Copy of FP&L Final Engineering Disposition William G. Knecht Technical Director Attachments DATE TO: Utilities that have been furnished 2" 1878 Piston Check Valves Gentlemen: There was a reported failure of a 2" 1878 piston check valve at the Florida Power and Light, St. Lucie site on July 18, 1996. This valve failed to close and it was determined that the valve disc was stuck in a partially open position. It appears that this failure resulted from fretting wear damage between the piston (disc) and valve body caused by pressure pulsations of the St. Lucie reciprocating pump. Anchor/Darling Valve Company has furnished (Utility) with similar valves of this size on the following order(s): (Fill in A/DV S.O. and Customer Order #) We suggest that the valve applications be evaluated for potential pressure pulsation that could result in fretting of the valve body, In addition, FP&L and A/DV need assistance to finalize the root cause of the St. Lucie problem. If an A/DV 2" piston check is in pump service, we would greatly appreciate receiving the following information: 1. Pump type - centrifugal or reciprocating 2. Forward flow rate(s) 3. Approximate length of service time 4. Description of any found anomalies Should you have any questions regarding this matter please call Mr. Floyd Bensinger, our Engineering Manager or Bill Knecht, our Technical Director. Sincerely, Contract Administrator CR 96-1774 Attach 8 Page 1 of 11 FINAL ENGINEERING DISPOSITION Scope: Previous interim dispositions provided an operability assessment for 2" Anchor Darling check valves installed in Unit 1 (Attachment 1) and Unit 2 (Attachment 7). Attachments 2,3,4 & 6 provided supporting documentation, Attachment 5 was superseded by Attachment 7. This attachment provides a final disposition for the CR. Attachments 9- 18 provide supporting documentation. The additional information developed in this final disposition does not adversely affect the previous operability reviews. Corrective actions in this disposition supersede corrective actions listed In prior attachments. Background: CR 96-1738 and In-House Event 96-059 identified loss of Unit 2 charging flow associated with the potential failure of V2167. V2167, the check valve for charging pump 2C discharge, was suspected of socking open which would result In the charging flow recirculating back to the volume control tank rather than into the RCS. The valve model is a 2" stainless steel Anchor Darling piston check which was installed In 1994. It is a piston check valve (straight/T-pattern) located downstream of the reciprocating pump. V2167 (serial # E-T401-9-19) was disassembled for inspection and was found stuck open approximately 1/2". In addition, the body bore machining in the area where the disc is guided was abnormal (looking from top with discharge port at 12:00, machining/casting anomalies were identified from 10:00 to 12:30). The first candidate replacement valve inspected also exhibited machining anomalies in the same location. A second spare for V2167 was inspected, found to be acceptable, installed and successfully tested. Based on initial review of the valves in stores, a working theory for the degradation cause was developed and used to disposition the operability of the Unit 1 valves per Attachment 1. The operability of to Unit 2 valves was subsequently reviewed in Attachment 7 based on a refinement of the same working theory. This attachment provides a final CR disposition by documenting tasks performed to date, developing a root cause evaluation, addressing regulatory reportability & industry notification and planning follow-up activities to be pursued under specific PMAI's. Initial Review of Machining Anomalies and Stuck Open Check Valve By Manufacturer The machining anomalies associated with 2" Anchor Darling check valves and the shack open check valve in the charging system were reviewed with Anchor Darling. Their initial review is provided in Attachment 2. Anchor Darling states that the selection of the subject 2" piston check valve as the charging pump discharge valve is appropriate. They also state that pressure pulses experienced at the discharge of a reciprocating pump may tend to force the valve disc toward the downstream port. They conclude that this system application combined with the machining anomaly on the body bore could cause the valve to remain open (i.e., the disc cocked and caught on machining anomaly). They further stated that other applications may not be as susceptible. Review of FPL's Surveillance Testing of Installed ADV Valves Twenty-eight SS & CS Anchor Darling piston check valves (2" and smaller) are in-service in PSL 1 & 2 for periods ranging from 0 to 36 months (Attachment 18). A review of the testing records (Attachment 3) shows approximately 450 valve-months of satisfactory operation (Exceptions are the Unit 2 V2167 and V2168 failures). The testing population includes 16 of the 17 two-inch piston check valves reviewed in Attachments 1 (Unit 1) and 7 (Unit 2). Note that V18193 is not included in the tested population as it is outside the LLRT boundary for its instrument air containment penetration. CR 96-1774 Attach 8 Page 2 of 11 Initial Inspections of Valves In Stores: Inspections were performed on spare Anchor Darting 1" & 2" Carbon Steel (CS) & Stainless Steel (SS) piston check valves that were in the stores inventory for use with the small bore valve replacement program. Similar machining/casting flaws were found in multiple valves as described in more detail in Attachment 7. Based on these inspections, Attachments 1 & 7 concluded: 1) 2" CS & SS ADV piston check valves in stores are susceptible to machining anomalies. 2) 1" and under CS & SS ADV piston check valves are not susceptible to machining anomalies. 3) All sizes of CS ADV piston check valves in stores are susceptible to internal rusting. Review of Reportability of Discrepant Material Received from ADV Under 10CFR21 Nuclear Engineering Quality Instruction ENG-QI 2.2 governs evaluations conducted to determine whether an issue/concern could result in a 10CFR21 Substantial Safety Hazard. This document states: "Note that SSH issues already reported (by FPL or a vendor to the NRC) do not require SSH evaluation (although operability evaluation may be warranted)." As Anchor Darling Valve Co reported the subject situation with their 20 piston check valves to the NRC on 7/25/96 (Attachment 9), an SSH evaluation by FPL is not required. FPL independently alerted the industry to the Loss of Charging event caused by the failure-to-close of the 2C charging pump discharge check valve via tie Nuclear Network (Attachment 11). Review of Available NPRDS Information The Nuclear Plant Reliability Data System was searched for information concerning Anchor Darling check valves with an 1878 model number. With the exception of the Shearon Harris event, all other events (see below) were deemed to involve foreign material within the valve. Information to support an NPRDS entry for the PSL event was developed and provided to Joe Cimino (Attachment 10). NPRDS Search Criteria: Subject Valve Selected Manufacturer Anchor/ Darling Valve Co Words equal to or beginning with: check valve, ck vlv, chk vlv Mfr Model Numbers: 1878 Company, Plant, Service, Noted Problem Valve, Valve # CP&L, Shearon Harris 1, CVCS Failure to provide full flow, failure 2 in piston check, 1CS-193 to seat 10CFR21 issued. This event is discussed separately. Northeast Utilities, Con Gross Leakage, O-Ring seal came Yankee 1, CVCS loose from disc, Valve placed back in 1 in piston check, CH-CV-405C service GPU, Oyster Creek 1, Unusual noises, Disc disengaged due to Containment Spray missing nut/cotter pin, Valve replaced 2 in swing check, V-3-0133 with swing check GPU, Oyster Creek 1, Unusual noises (chattering), leaking Containment Spray past sear Telephoned to clarify 2 in piston check, V-3-0133 NPRDS data: Valve is in seawater service, no damage to valve, parts not degraded, normally open -150 gpm, debris behind seat (silt and shells), have changed valve design & manufacturers several times. Info per John Galanto (609-971-4000) GPU, Oyster Creek 1, Stuck Open, Marine Growth and Containment Spray Foreign Material, Valve placed 2 in piston check, V-3-1033 back in service Engineering requested to evaluate possible valve replacement GPU, Oyster Creek 1, Containment Spray Stuck Open, Marine Growth and Foreign 2 in piston check, V-3-131 Material, Valve placed back in service, Plan to install upgraded valve Niagra Mohawk, 9 M Point, FW, Failed LLRT, Dirt and Debris Inside 31-01R 18 in swing check Valves, Valve placed back in service Niagra Mohawk, 9 M Point, FW, Failed LLRT, Flutter During Flow 31-01R 18 in swing check Condition Inherent to Piping Design, Valve placed back in service Niagra Mohawk, 9 M Point, FW, Failed LLRT, Flutter During Flow 31-02R 18 in swing check Condition Inherent to Piping Design, Valve placed back in service Portland GE, Trojan 1, SA, Failed LLRT, Foreign Material on SA-2005 2 in unknown style Seating Surfaces, wire and rubber gasket foreign material. Valve placed back in service NYPA, Fitzpatrick 1, Combustible Failed LLRT, Found no off-normal Gas Control, 27 CAD -68, 1.5 in conditions, piece of dir believed unknown style to be responsible returned to service NYPA, Fitzpatrick 1, Combustible Failed LLRT, Cloth-like debris Gas Control, 27 CAD -68, 1.5 in prevented closures, valve placed unknown style back in service CR 96-1774 Attach 8 Page 3 of 11 Review of Shearon Harris 10CFR21 Report - CSIP Miniflow Check Valve Deficiency A CP&L 10CFR21 report to the NRC (Reference 7) describes an event where their 1B Charging Safety Injection Pump (CSIP) was placed in service when it Was inoperable (due to the inability of its mini-flow check valve to meet forward flow testing requirements). One CSIP is normally operating in their system alignment to provide charging flow, The subject ADV valve failure was detected because it exhibited back leakage on at least two different occasions. The back leakage was caused by cocking of the piston toward the valve discharge port and/or damage to the resilient seat. CP&L attributed the damage to the roar instantaneous opening of the piston check due to large hydraulic forces caused by pump startup. Subsequent to the initial failure, three different disc designs were installed (resilient seat, hard seat, full body guided) with no improvement. Originally, the resilient seat was suspected of causing the valve to stick open because it was found partially off the disc and degraded. The valve was inspected, measured, and found to be within the vendor specifications. While no internal machining or casing anomalies were noted, CP&L was not specifically looking for the problem seen at PSL. The valves from CP&L are not available for further inspection. After the problems with the ADV piston check valves (stuck open, insufficient forward now) were not remedied by subsequent disc modifications, CP&L ceased efforts to modify the ADV valve and reinstalled me original Y-pattern piston check valve. No subsequent problems were identified. CP&L did not develop the root cause conclusively, CP&L provided the following information regarding the CSIP recirculation check valves: 1. CP&L changed valve from Rockwell Y-pattern piston check valve to ADV T-pattern piston check valve in 1994 due to ALARA concerns (eliminate seal welded bonnet). Experienced problem with ADV valve (stuck open, insufficient forward flow) that was not remedied by the disc modifications (see below), ceased efforts, and reinstalled Rockwell Y-pattern piston check valve. 2. Changed the disc design (resilient seat, hard seat, full body guided) with no improvement. Originally, the resilient seat was suspected of causing the valve to stick open because it was found partially off the disc and degraded. The valve failed-to-open with the full body guided disc. 3. X-Ray of valve showed disc was cocked with the bottom of the disc towards the discharge port. CP&L believes disc was always cocked to discharge port (i.e. the first time the pump was started the disc would cock). CP&L presented no data to support or refute this contention. 4. CP&L states the most probable root cause is significant hydraulic forces acting on the check valve when the pump starts; pump takes approximately 3 Sec to come up to speed, the valve closes within the first 1/2 second. 5. The valve was not originally backseat leak tested by the plant. They noticed the pipe was hot after the ADV valves were installed and initiated backseat leakage testing. 6. Inspected the body bore of suspect valve with no indication of roughness or machining marks. Inspected intersection on body bore and discharge port and found no problems. (Not clear if fretting damage was identified) 7. Disc Dimensions were within vendor specification: Vendor 1.867, - 0.002, As-found 1.866 Bore Dimensions were within vendor specification: Vendor 1.875, +0.00.4, As-found 1.875 8. Design Pressure/Temperature: 2735 psig, 200 Degrees F; Operating: 2712 psig, 130 Degrees F: Flow 60 gpm CP&L has two other ADV 2" piston check valves installed (BA Transfer Pumps Discharge Check Valves) that have not exhibited the noted problem. These valves are tested quarterly at 35 gpm (forward flow and backseat leakage) with no reported failures. The operating conditions of these valves are different from the CSIP check valves; operating pressure is lower (120 psig vs 2712 psig) and flow is lower and variable (35 gpm vs 60 gpm), CP&L provided the following information regarding the BAT recirculation check valves. 1. Anchor Darling T-pattern piston check valve installed in 1994, No problems reported. 2. Valves tested quarterly for flow and backseat leakage at 35 gpm. 3. Design Pressure/Temperature, 150 psig, 250 Degrees F. Operating, 120 psig, 250 Degrees F, Flow 140 gpm max, varies The Information obtained from ADV and CP&L is insufficient to conclusively determine the root cause the Shearon Harris event. There is insufficient information on which to base any further conclusions. ADV conducted flow testing but could not replicate the problem. Further testing is planned for the CP&L scenario, and FPL has supplied a suspect valve which could also be tested. When further information CR 96-1774 Attach 8 Page 4 of 11 becomes available, it will be evaluated as appropriate. Review of Engineering's records indicates JPN was notified of the Shearon Harris 9/29/95 event by ccMail from FPL Licensing on 10/2/95 due to CPL's Nuclear Network entry. JPN contacted CPL on 10/7/95 to obtain a copy of the Part 21 notice and to discuss the event. The cause of the event was not well understood and no further action was appropriate at that time. It has been determined that the NRC did not make any formal disclosure of the Shearon Harris event. Recent conversation with the PSL site resident indicates that NRC issuance of an information Notice noting both the CPL and FPL events is likely. Inspection of V2167 After Sectioning The initial conclusion that machining anomalies in the 2" ADV piston check valves were directly responsible for the field failure is not supported by more recent inspection of V2167 following valve sectioning. The inspection, as documented in Attachment 14, concludes that local fretting wear occurred in the body contact area with the disc which likely led to a local groove, sticking/disc cocking and the valve's failure-to-close. As the casting flaw above the discharge port and the machining marks below the fretting zone did not exhibit any significant wear marks. they are not believed to have led to the failure of V2167 to close. The machining marks below the fretting zone, however, could well have contributed to piston socking for different valve operating conditions. However, had the machining anomalies been located adjacent to the observed fretting location (ie., the valve's normal disc location during flow) they would have affected valve operability/reliability. Review of the V2167 anomalies indicates the acceptability of casting/machining anomalies is largely dependent on their location. While such anomalies are certainly undesirable in a finished product, their presence can be accepted on a case-by-case basis based on their extent and location. Inspection of V02134 After Sectioning V02134 was removed from service due to galling of the bonnet threads. The valve was subsequently scrapped and sectioned for internal inspection. The Inspection of this valve, as documented in Attachment 16, was performed in the presence of ADV. It identified internal machining anomalies near the discharge port which are not believed to have affected valve operation. Wear bands in the guide surfaces (fretting) were identified at the disc land locations. No local grooving was identified as a result of fretting wear. Local scuffing (abrasive wear) of the guide area at the edge of the discharge bore was observed. Review PWO Work History for Trending Per FRG request, the work order history of V2167 and similar valves was reviewed to determine whether there was sufficient data to trend a developing problem with the Anchor Darling valve. Attachment 17 documents this review. An initial report in December of 1994 indicated V2167 failed to close (flow decreased as recirculation valve opened) but problem could not be replicated. Further inspections did not identify valve internal problems and testing was inconclusive, Engineering concludes there was insufficient data prior to the 7/13/96 event (with its determination the valve was cocked and stuck open) to identify an incipient problem. CR 96-1774 Attach 8 Page 5 of 11 Disposition of ADV Piston Check Valves In Stores Based on the conclusions drawn from the inspection of V2167, the 20 piston check valves remaining in stores were re-inspected against a criteria that would allow raw casting surfaces and/or machine marks in valve guide areas not in contact during valve operation. This review was based on the criteria depicted in Attachment 13 which provides the basis for accepting certain valves that would otherwise be returned to ADV as non-conforming. Attachment 12 lists all 2" piston check valves received from Anchor Darling and the remaining 1" and under piston check valves in Stores. This information for the 2" valves is summarized below: Table omitted. As described in Attachment 12, a number of packaging anomalies were identified from review of tie valves in Stores; these included many valves that were not yet unpacked from the initial factory shipment. Anomalies included water droplets inside the majority of the piston check valves, internal rusting and absence of desiccant. ADV indicates that disassembly of the valve following hydrotest for dry out and reassembly, with the valve pressure seal shipped as a loose part, would be an appropriate corrective action to the first item. These inspections supported the conclusions from the initial inspections (see page 2) that: 1) 2" CS & SS ADV piston check valves in stores are generally susceptible to machining anomalies. 2) 1" and under piston check valves are not susceptible to the machining anomalies noted in the 2" valves. 3) All sizes of CS ADV piston check valves in stores have various levels of internal rusting. The anomalies found in the 1" & under piston check valves (a casting lip) were different from the generic problems associated with the 2" piston check valves and were deemed to be an isolated cases. These two valves will be returned to ADV. CR 96-1774 Attach Page 6 of 11 Root Cause Evaluation: Plant Status When Event Occurred Plant Location Where Event 100% Power Occurred 2C Charging Pump Activity in which Event Occurred Related Activity In Which 2C Charging Pump Shutdown Event Occurred 2B Charging Pump Startup after Oil Addition Type of Inappropriate Action/Job Category Type of Equipment Not Applicable 2" Anchor Darling Piston Check Valve How Human/Admin Error Occurred How Equipment Not Applicable Degraded/Failed Failed to Close Why Event Occurred Charging application developed fretting wear on disc/body interface leading to disc sticking open The failure-to-close event for the 2C Charging Pump discharge check valve (V2167) initially appeared to be caused by internal casting/machining anomalies. Physical inspection of a spare 2" valve also indicated internal machining anomalies which required the consideration of the continued operability of all installed 2" ADV piston check valves (Attachments 1 & 7). Based on a series of verbal discussions with ADV, the cause of the machining anomalies is believed to be due to the past use of a drilling process to perform rough machining of the guide area and to develop the seat pocket recess. This process tends to follow the geometry of the rough casting. Following brazing of the seat and heat treatment, a boning process is used for final machining. The centerline differences between these two processes is believed to account for the observed anomalies. The quality controls within ADV's manufacturing process identified the subject anomalies and these were subsequently accepted following ADV's internal review. Further review of V2167 internal conditions (Attachment 14) has concluded that the noted internal casting/machining anomalies are not the primary cause of the valve's failure-to-close. This conclusion is based on the location of the casting/machining anomalies in V2167. However, the types of anomalies noted in other ADV piston check valves (Attachment 12) may well affect valve operability/reliability. The present understanding is that the 2C Charging Pump discharge check valve's (V2167) failure-to-close was due 10 internal fretting damage within the valve. Fretting damage is believed to be primarily caused by the valve's material combination and the charging system application (long service times in a pulsating flow). Increased tolerances due to casting anomalies may also be a contributing factor. The machining anomalies in V2167 would not affect valve function. The charging pumps are triplex positive displacement piston pumps manufactured by Union Power Pump Company. These pumps have a 2 1/16" bore & a 5 inch stroke and operate at 213 rpm. The number of pressure pulsations per minute of run-time is -640 at a nominal discharge pressure of 2335 psig. Review of the run time hours for the 2" ADV check valves installed in charging system service does not indicate that runtime hours, in of itself, is a good indicator of impending failure. Charging Pump Discharge Check Valve Check Valve Run Time 1B Charging Pump V02133 1,200 hours 1C Charging Pump V02134 6,500 hours (est) 2A Charging Pump V2169 3,100 hours 2B Charging Pump V2168 9,700 hours (est) 2C Charging Pump V2167 5,700 hours Unit 2 Common Header V2462 5,850 hours (est) CR 96-1774 Attach 8 Page 7 of 11 Review of the valve application, knowledge of the disc cocking and inspections of valves V2167 (Unit 2), V02134 (Unit 1), and valves in Stores lead to the following failure description. This description has been updated to reflect additional information; the basic explanation, as contained in Attachments 1 & 7, remains essentially unchanged. Failure was due to a combination of causal factors: 1) The constant flow of the charging pump (44 gpm) results in the valve disc opening to move off its seat approximately 1/2" - 5/8". The constant flow rate of the positive displacement pump results in a single disc position leading to the concentration of wear damage at a single location, 2) As the bottom edge of the disc is not fully supported by the body bore due to the presence of the discharge port, the disc's bottom skirt is supported at two relatively wide points on its arc. At 68 gpm, the angle between the arc points (-140 Degrees) is at its maximum (44 gpm for charging system/-110 Degrees). 3) The pulsating nature of the charging flow (three piston, positive displacement, reciprocating pump) results in a small continuous oscillation of the valve disc. The disc movements within the bore guide surfaces (vertically and side-to-side) result in wear bands that correspond to the disc land locations. 4) The raw casting of the body bore was apparently oversized/off-center which led to incomplete machining of the bore and greater clearances between the disc and bore. Absence of significant wear marks within these regions indicates the rough guide surfaces did not contribute to the disc wear. Greater clearances result in larger disc movements and may lead to higher disc velocities and impact forces. 5) Disc movement led to wear of the disc and the body guide surfaces. The wear band from the disc's upper end is distributed around the entire circumference but is more noticeable on the outlet side of the valve, A lower wear band is also present but more rapid wear occurs at the two lower land's contact points on the bore/discharge port intersection. 6) Due to the long in-service time, constant flow, two point support and softer material of the body, grooving (due to fretting) and scuffing (due to abrasive wear) occurs at the body bore/discharge port intersection. Eventually, the fretting wear reaches a point where the damage results in a cocked disc that is stable under loss of flow; the spring force is not sufficient to reclose the valve when the flow stops/reverses. Based on the below review of the failure description, a reciprocating pump application is expected to be more susceptible to fretting damage than a typical centrifugal pump application. Causal Factor Criteria for Selection / Implications Constant flow rate Results in a single check valve disc position - lift is a function of flow velocity and density Results in concentration of wear at one discharge port location Maximum angle of contact occurs with -68 gpm or 12 ft/sec of water Pulsating flow of Results in small oscillation of the plug to create a reciprocating fretting at the points of contact within the guide pump surface, flow turbulence with a centrifugal pump may also produce fretting but is expected to result in a lower wear rate Mis-cast valve Mis-casting causes greater clearances; more disc movement may result in more rapid wear depending on location Piston & valve Harder disc provides for more rapid wear at points of body materials contact in bonnet bore/discharge port intersection; item with greater wear (body) is non-replaceable. Duration of Wear process is time related; Duration of service at valve's service flow governs effect of wear mechanism. More frequent at flow inspections are warranted for valves normally at flow Absence of one or more factors will increase the time to failure and/or may prevent the failure mechanism. It is difficult to state whether fretting will not occur for a centrifugal pump application - given a sufficient time duration for the damage to be observed. The duration of time required for failure will also be dependent on the pre-existence of casting/machining anomalies. Additional field data for centrifugal pump applications is needed to fully address this potential concern. CR 96-1174 Attach 8 Page 8 of 11 Fretting is an adhesive wear mechanism between two surfaces having oscillatory, relative motion of small amplitude. It involves the repeated formation and fracture of cold welds between two surfaces. Abrasive wear involves the plowing or cutting of grooves in a soft material by a harder one. Factors which affect both adhesive and abrasive wear include relative hardness, surface finish and microstructure. Generally, harder materials are more wear resistant and greater hardness differences lead to increased wear. Rough surface finishes usually cause increased wear, however, very smooth finishes can increase the tendency for cold welding. Also, similar microstructures can increase the tendency for cold welding. The cause of the fretting appears to be small disc oscillations due to the pulsating flow coupled with a less than optimum material combination of disc and body materials. Discussion of the material selection with ADV centered on a rule of thumb of achieving a difference of more than 4 HRC between material couples. This criteria is typically used to prevent the onset of galling, an adhesive wear mechanism, but does not address other wear mechanisms. For the material combination of the resilient seated valves (SA-564 Gr 630-1075 disc and SA 351-CF8M or SA 216-WCB body) wear data indicates a less than optimum couple. For the material combination of the hard seated valve (SA-638 Gr. 660 Type 1 disc and SA 351-CF8M or SA 216-WCB body) no wear data was found in the literature. Based on the hardseated valve disc material hardness (SA-564 Gr 630:43 HRC) compared to resilient valve disc material hardness (SA-564 Gr 630- 1075: 32 HRC), it is reasonable to conclude that wear rates for the resilient seated valve body will be less than those observed for the, hardseated valve. Anchor Darling has been requested to review wear characteristics of their material combinations. Anchor Darling has stated that piston check valve use in non-pulsating flow applications, may not be susceptible to the above degradation mechanism. For a centrifugal pump application (assuming variable flow through the check valve), the periodic travel of the disc across the machining anomaly would tend to smooth the damaged area. For larger flow applications, the greater stroke would result in more available spring closure force. Anchor Darling suggests that this force may be sufficient to overcome sicking due to fretting damage and coupled with the disc's closing momentum may well result in closure of a piston check despite casting/machining anomalies at tower elevations. ADV suggests a smaller valve would be beneficial. Root Cause Summary The root cause for the V2167's failure-to-close is fretting damage between the disc and the valve body at two contact points adjacent to the valve's discharge port. The raw casing surfaces and machining anomalies noted within the piston guide areas are not presently believed to have significantly contributed to the failure of V2167 based on their location and the lack of significant wear marks. However, these type of manufacturing anomalies may well contribute to failure-to-close/open in other valves depending on their location, degree and the valve flow rate application. The fretting damage was likely caused by the large difference in hardness between the valve disc (43 HRC) and valve body (78 HRB) and the pressure pulsations & constant flow application of the charging system. These conditions would lead to small disc oscillations (due to pressure pulsations) at a single point (due to constant now) in the valve discharge port. Implications of Root Cause Determination Based on the above description of the failure causes, it would follow, but is not certain, that fretting damage would be significantly less pronounced for a centrifugal pump application, This presumption is based on pressure pulsations from a centrifugal pump being less than for a reciprocating pump and the variation in pump flow distributing any postulated fretting wear across a wider surface area. The first presumption, though reasonable, may delay rather than prevent the onset of significant fretting damage. The presumed beneficial effect of flow variation is reasonable for a centrifugal pump discharge check valve application where system flow rate varies with time. For a mini- recirculation line check valve serving a Centrifugal pump that is normally operated in a near dead-headed application, the flow variation through the recirculation line would be negligible. This latter application is descriptive of the Shearon Harris CSIP application. CR 96-1774 Attach 8 Page 9 of 11 Based on the above, the Shearon Harris event may bear more similarity to the St. Lucie event than first realized. Based on discussions with ADV and Shearon Harris, fretting damage was not identified within the valve discharge bore. At this point in time, Shearon Harris and ADV continue to believe the cause was related to the initial insurge of the flow from the CSIP pump. Until further data is obtained, it would be prudent to inspect 2" ADV piston check valve internals on a more frequent basis for applications where valves see long periods of service at flow. Based on the current level of understanding, 2" ADV piston check valves are suitable for intermittent service or service in varying flow regimes without restriction. Further use of ADV piston check valves in near continuous use, constant flow service must consider the potential for the valve to fail-to-close after long periods of service (2500-3500 hrs at constant flow). An in-depth design review and in-service applications of the 2" piston check valve is warranted and has been requested by letter to Anchor Darling (Reference 8). Specific long term corrective actions will be required for the installed valves in constant flow service. These corrective actions are under review and will be determined based on the work to be performed by Anchor Darling. The changes will likely include a reduction in valve size of specific ADV piston check valves. In the interim, the following specific countermeasures are recommended: 1. Perform yearly internal inspections of the ADV piston check valves in charging pump discharge check valve service until further notice. These inspections will develop trending information and should be sufficient given the rotation of the pumps. 2. Perform an inspection each refueling outage of other ADV piston check valves in charging service until further notice. These inspections should be based on GMP-01 and specifically target fretting damage in the discharge bore. Inspection by SCE personnel is recommended. General anions have been specified below to prevent recurrence. These include a QA audit of ADV discrepancy identification, evaluation practices, & packaging practices to prevent recurrence at the manufacturing facility. Training of FPL welders is also planned so that similar potential valve problems would be identified prior to implementation. Corrective Actions: Corrective actions In this disposition supersede corrective actions listed in prior attachments. 1. Inspect and, replace if necessary, ADV 2" piston check valves installed in Unit 1 & 2 per the schedule requirements of Attachment 15. All Work Orders for these inspections have been written by SCE. No further action is required at this time for this item. 2. Following completion of the initial valve inspections under Attachment 15, perform periodic inspections of ADV valves in charging service until further notice: a) Perform yearly internal inspections of the valves in charging pump discharge check valve service. Recommend pump discharge valve inspection be bed to charging pump maintenance with instructions to disassemble valve and inspect if not already performed in last 6 months. b) Perform an inspection each refueling outage of other ADV piston check valves in charging service. Inspections should be based on GMP-01 and specifically target fretting damage in the discharge bore Localized fretting of the type seen in V2167 (a groove on each side of the discharge port) is cause for replacement. Inspection by SCE valve component specialist is recommended. Issue a PMAI to ENG-Systems Engineering to institute GMP-01 testing on ADV piston check valves in charging service on a more frequent basis. Recommend pump discharge check valve inspection be tied to charging pump seal maintenance with instructions to disassemble ADV discharge check valve and inspect if not performed in last 6 months. Recommend other downstream Charging System ADV piston check valves be inspected during each refueling outage until further notice. Due Date 10/18/96. CR 96-1774 Attach 8 Page 10 of 11 3. Safety Evaluation JPN-PSL-SEMS-96-052 allows temporary manual isolation of the Unit 2 recirculation valves while the associated charging pumps are operating. Implementation of the safety evaluation shall be implemented and Controlled by approved changes to plant procedures in accordance with section 9.0 of the safety evaluation. The recirculation valves for all 3 Unit 2 charging pumps shall be closed until V2462 is inspected and/or replaced as appropriate. This action provides an additional compensatory measure to prevent backflow (intersystem LOCA), in the event V2462 fails to close. Closure of the manual stop valve on the discharge of a charging pump (V2336, V2464, V2339) to allow pump maintenance is an equally acceptable compensatory measure for that specific pump. With the manual stop valve closed, the respective charging pump and its recirculation valve may be disassembled or tested as required. The recirculation valve shall be closed prior to reopening the manual discharge stop valve. In addition to the above requirement: a) The recirculation valve for pump 2A (V2555) shall be kept closed until valve V2169 has been inspected or replaced with a suitable spare. Closure of V2336 is an equally acceptable compensatory measure. b) The recirculation valve for pump 2B (V2554) shall be kept closed until valve V2168 has been inspected or replaced with a suitable spare. Closure of V2464 is an equally acceptable compensatory measure. The above requirements have been accommodated by a permanent change to OP 0210020 which provides for a normal charging system alignment with the three recirculation valves in the closed positions. This system alignment change is acceptable for one fuel cycle per safety evaluation JPN-PSL-SEMS-96-052. Issue a PMAI to operations to revise OP 0210020 to return to the design basis use of the charging pump recirculation valves after the inspection and replacement, as appropriate, of Anchor Darling check valves V2462, V2167, V2168, and V2169. This PMAI is to include a Mode 4 hold for the power ascension following the 1997 refueling outage (Cycle 15). Due Date 5/15/97. 4. Issue a PMAI to Nuclear Materials Management to return rejected ADV piston check valves to the vendor as identified in CR 96-1774 Attachment 12. Valve Serial, # E-T401-9-25 is to be released to ENG. Valve Serial # E-T401-9-30 is to be tagged "Restricted Use - requires Engineering Approval (Reference CR 96-1774) Gordon McKenzie ext 7276". Valves designated as "Rework" are to be segregated for rework by MM prior to field use. Due Date 8/30/96. 5. Issue a PMAI to Mechanical Maintenance to perform rework of selected ADV valves as identified in CR 96-1774 Attachment 12. Engineering will provide support as required. Due Date 10/25/96. 6. Issue a PMAI to Mechanical Maintenance to perform training for welders to ensure they would identify and report casting and machining problems of the type noted within the 2" ADV piston check valves. Engineering will provide support as required. Due Date 9/27/96. 7. Issue a PMAI to QA to perform an audit of ADV manufacturing and packaging process controls. Manufacturing controls should have prevented shipment of valves with casting/machining anomalies. A potential solution would be written guidelines governing evaluation of discrepant bore surfaces. Packaging controls should have prevented the water observed within shipped valves. Disassembly of the valve following hydrotest for dry out and reassembly with the valve pressure seal shipped as a loose part is a potential solution Due Date 10/18/96. 8. Issue a PMAI to Engineering to provide appropriate long term corrective actions (valve replacements and/or procedure/inspection program changes) based on the root cause determination within CR 96- 1774 and ADV's subsequent input requested by Reference 8. Review plant documents (eg., SOER 86-03 inspection Database, GMP-01) for Potential revision to address the root cause determination Due Date 11/15/06. Attach 8 Page 11 of 11 Attachments: 1. Engineering Disposition related to Unit 1 Operability Review 2. Anchor Darling Letter dated July 19, 1996 3. Inter-Office Correspondence, "Testing of Anchor Darling Check Valves", July 20, 1996 4. Anchor Darling Piston Check Valves Summary of Inspection Findings, July 20, 1996 5. Engineering Disposition related to Unit 2 Operability Review (Superseded) 6. Anchor Darling Letter dated July 23, 1996 7. Engineering Disposition related to Unit 2 Operability Review (Revision 1) 8. Final Engineering Disposition 9. ADV 10CFR21 Letter to NRC dated July 25, 1996 10. NPRDS System Notification 11. Nuclear Network Notification 12. Disposition of ADV Piston Check Valves in Stores 13. Acceptance Criteria for Raw Casting/Machining Anomaly Locations 14. FPL Met Lab Report 96-170, dated August 5, 1996 15. Inspection Results for Installed Valves (8/16/96) 16. FPL Met Lab Report 96-180, dated August 15, 1996 17. Review of PWO History for Trending 18. Installed Anchor-Darling Check Valves, August 16, 1996 References: 1. Unit 1 FSAR, Amendment 14 2. Unit 1 Technical Specifications, Amendment 142 3. Anchor Darling Drawing W9323936, Rev A 4. Unit 2 FSAR, Amendment 9 5. Unit 2 Technical Specifications, Amendment 82 6. Design Basis Document DBD-CVCS-2, Rev 0 7. CP&L Letter to NRC dated Sept 29, 1995 8. FPL Letter to ADV dated August 16, 1996 9. Union Pump Technical Manual 2998-3414, Rev 15 10. Crane Technical Paper 410, 1991 Printing Prepared by Verified by Approved by Date *** END OF DOCUMENT ***

Page Last Reviewed/Updated Wednesday, March 24, 2021