United States Nuclear Regulatory Commission - Protecting People and the Environment

ACCESSION #:  9708210028

                       LICENSEE EVENT REPORT (LER)

FACILITY NAME: San Onofre Nuclear Generating Station      PAGE: 1 OF 9

               Unit 2

DOCKET NUMBER:  05000361

TITLE:  Charging Subsystem Check Valve Failure

EVENT DATE:  06/26/97   LER #:  97-010-01   REPORT DATE:  08/13/97

OTHER FACILITIES INVOLVED:  Unit 3                  DOCKET NO:  05000362




50.73(a)(2)(i), 50.73(a)(2)(vii), & Other: Part 21


NAME:  R.W. Krieger, Vice President,        TELEPHONE:  (714) 368-6255

       Nuclear Generation






On 6/26/97 while Unit 3 was shutdown for refueling, an investigation of

the cause of high auxiliary spray flow found a check valve in the

Charging Subsystem that would not open completely.  This would cause the

charging flow distribution to the Reactor Coolant System to be different

than that assumed in the safety analysis.  Because of the similarity of

the Unit 2 Charging Subsystem, Edison immediately entered the Technical

Specification action statement for one or more trains of Emergency Core

Cooling System (ECCS) inoperable, but 100 percent of the ECCS equivalent

flow to a single operable ECCS train available.  As a conservative

action, Edison also reduced Unit 2's power to about 90 percent, where

charging flow is not required for accident mitigation.  On 6/28/97,

Edison tested Unit 2 and found that a similar check valve also would not

open completely.  As required by the Technical Specifications, Edison

shutdown Unit 2 to repair the valve.  Completion of that shutdown is

being reported as required by 10CFR50.73(a)(2)(i).  Because this valve

failure was caused by a design defect, Edison is including 10CFR21

information in this report.

Edison replaced the faulty design valves in the charging injection lines

and auxiliary spray line in each unit with another type valve.  Edison's

evaluation of the cause(s) is on-going.  This LER will be revised if the

evaluation determines a cause other than faulty design.

The 1979 computer model requires charging system flow for mitigation of

some small break loss of coolant accidents.  However, this model assumes

plant conditions that are not allowed by the Technical Specifications.

Using an EPRI analysis program and actual, allowed, plant parameters,

Edison concluded that the condition resulted in a negligible increase in

plant risk and had no actual safety significance.


NOTE:     This text document was processed from a scanned

          version or an electronic submittal and has been

          processed as received.  Some tables, figures,

          strikeouts, redlines, and enclosures may not have

          been included with this submittal, or have been

          omitted due to ASCII text conversion limitations.  In

          order to view this document in its entirety, you may

          wish to use the NUDOCS microfiche in addition to the

          electronic text.

TEXT                                                          PAGE 2 OF 9

Plant:              San Onofre Nuclear Generating Station, Units 2 and 3

Reactor Vendor:     Combustion Engineering

Event Date:         June 26, 1997

Event Time:         1413 PDT

               Unit 2                             Unit 3

Mode:          Mode 1 (Power operation)           Mode 5 (Cold shutdown)

Power:         100 percent                        000 percent

Temperature:   NOT                                93 degrees F

Pressure:      NOP                                Atmospheric


This LER reports Edison's discovery of a design defect in Kerotest [K085]

2 inch Y-type Series 1513 check valves [V].  This design defect was not

known by the vendor and caused two check valves (one in each unit) in the

Charging Subsystem [CV] to be able to open only a small fraction of their

full stroke.  This increased flow resistance and caused the charging flow

distribution to the Reactor Coolant System (RCS) [AB] to be different

than that assumed in the safety analysis.

As discussed below, to resolve this issue, Edison:

o    Proactively evaluated vendor supplied valve information to determine

     the impact on the charging subsystem and is continuing its

     evaluation of the design defect.

o    Immediately discussed the issue with the NRC Staff.

o    Tested both units to determine the impact on actual plant


o    Shut down Unit 2 and delayed startup of Unit 3 to effect system


o    Identified and is correcting deficiencies in the In-Service Testing

     (IST) and Technical Specification (TS) Surveillance Requirements

     (SR) testing which prevented earlier detection.

Because Edison determined that the valve failure was caused by a design

defect, 10CFR21 information is being included in this report.


The Charging Subsystem (Figure 1) is part of the Chemical and Volume

Control System (CVCS) [CB].  During normal plant transients, the CVCS

maintains RCS inventory.  Following a Safety Injection Actuation Signal

(SIAS), charging is used to supplement the Emergency Core Cooling System

(ECCS) flow.

The three charging pumps (Train A, Train B, and a swing pump) discharge

to a common header.  A single line from the header enters containment and

splits into three lines; one charging line to each of RCS loops 1A and

2A, and one auxiliary pressurizer spray line.  Each charging line has a

check valve (2(3)MU020 and 2(3)MU021) and a normally open motor operated

block valve.  The auxiliary spray line has a check valve (2(3)MU019) and

a normally closed motor operated block valve.  These check valves isolate

the RCS from the CVCS in the event of a pipe break in the CVCS or when

charging is not running.

The limiting licensing basis Small Break Loss of Coolant Accident

(SBLOCA) assumes only one charging pump is providing flow, and that the

flow from only one charging line is discharged into the RCS (see the

Safety Significance section, below).  To maintain these two assumptions,

the flow through the two charging lines must be balanced (at least 15.8

gpm per line with one pump operating).

TEXT                                                          PAGE 3 OF 9


On 6/19/97, while bringing Unit 3 to Mode 6, RCS pressure control did not

respond as the operators [utility, licensed] expected.  When auxiliary

spray valve 3HV9201 was opened with both charging injection valves

3HV9202 and 3HV9203 open, the operators expected to be able to maintain

pressure.  However, RCS pressure started to decrease.  Engineering's

investigation determined that, while fewer pressurizer heaters were

available than in previous cycles, the available heaters were operating

as expected, and should have been adequate to control pressure.  However,

auxiliary spray flow appeared to be higher than expected.  Increased

auxiliary spray flow would likely be caused by the flow resistance of the

two parallel charging lines being higher than expected.  The result would

be some charging flow would be diverted into the auxiliary spray line.

One immediate difference between the two charging lines was check valves

3MU020 and 3MU021.  3MU020 was the originally installed Kerotest valve,

while 3MU021 was a Kerotest redesigned equivalent replacement valve.

Based on vendor supplied design information, Edison decided the higher

resistance was caused by 3MU020 having a significantly higher cracking

pressure (pressure required to open the valve from its closed position)

than 3MU021.  (This Kerotest information was later found to be incorrect.

See the Additional Information section, below.) With a single charging

pump running, 3MU020 was postulated not to open.  All charging flow would

be divided between one charging line and auxiliary spray.

At the time of discovery (1413 on 6/26/97), Unit 2 was in Mode 1, power

operation, at 100 percent power.  Unit 3 was in Mode 5, cold shutdown,

for refueling.

Realizing that unbalanced charging flow did not satisfy the assumptions

of the licensing basis SBLOCA analysis and that Unit 2 had the same valve

types, Edison immediately:

1.   Entered Unit 2 into Condition A of TS 3.5.2.  See the Technical

     Specifications and Additional Information sections, below.

2.   Conservatively reduced Unit 2 to about 90 percent power.  At this

     power level, charging flow is not required to mitigate the

     consequences of a SBLOCA.  Additional information regarding the

     power reduction to 90 percent is provided in the Safety Significance

     section, below.

On 6/27/97, while attempting to measure and then balance the charging

flow distribution in Unit 3, Edison discovered that the increased flow

resistance was actually caused by 3MU021 not opening completely.  Edison

immediately tested Unit 2 and found that 2MU021 also would not open

completely.  on 6/29/97 at 1102 PDT, in anticipation of entering

Condition B of TS 3.5.2, Edison began to shutdown Unit 2 to repair

2MU021.  A 1-hour telephone report was made as required by

10CFR50.72(b)(1)(i) (NRC Operations Log #32545).  Completion of that

shutdown is being reported here as required by 10CFR50.73(a)(2)(i).


TS Limiting Condition for Operation (LCO) 3.5.2, ECCS-Operating, requires

two operable trains of ECCS in Modes 1 and 2, and in Mode 3 whenever the

pressurizer pressure is greater than or equal to 400 psia.  If one or

more trains are inoperable, but at least 100 percent of the ECCS flow

equivalent to a single operable ECCS train is available, Condition A

requires the inoperable train(s) be restored to operable within 72 hours.

If that action is not met, Condition B requires the unit be in Mode 3

within 6 hours and pressurizer pressure reduced to less than 400 psia

within 12 hours.

The Bases of TS 3.5.2 describes an ECCS train as a train of high pressure

safety injection (HPSI) [BQ], a train of low pressure safety injection

(LPSI) [BP], and a

TEXT                                                          PAGE 4 OF 9

train of the Charging Subsystem.  A charging train consists of a charging

pump with the capability of taking suction from the refueling water

storage tank or the boric acid makeup tank, and a flow path to the RCS

via the normal charging lines.  Due to the single containment

penetration, the two normal charging lines are common to both Train A and

Train B.


While Edison's evaluation of the valve failures is on-going, the limited

travel of the piston assembly of 2(3)MU021 is believed to be caused by a

faulty design of the new style disc assembly.  This LER will be revised

if the evaluation determines a cause other than faulty design.


With the assistance of the valve vendor, Edison attempted to repair

2MU021 with a redesigned piston assembly with a longer length-to-diameter

ratio.  However, the valve still failed to open properly.

2(3)MU019 and 2(3)MU021 were replaced with 2 inch Borg-Warner "Y" type

lift check valves, either Mark #558 or #559.  These valves are used

successfully in similar applications at another nuclear unit.  The

hydraulic parameters of the Borg Warner valves for 2(3)MU021 were

verified to be similar to 2(3)MU020.  Also, the charging line flows were

balanced by throttling HV9203 before returning the unit to service.


In 1979, Combustion Engineering's (CE) SBLOCA calculations predicted fuel

cladding temperature would exceed 10CFR50.46 limits (2200 degrees F) for

three break sizes (0.025 sq.  ft., 0.05 sq.  ft., and 0.075 sq.  ft.)

without crediting charging flow.  In response, the NRC approved analysis

credits 15.8 gpm charging flow to supplement HPSI flow, and complies with


For larger breaks (0.1 sq.  ft.  and 1.0 sq.  ft.), charging flow is not

required because RCS depressurization is rapid enough that HPSI alone can

provide sufficient ECCS flow prior to initiation of safety injection tank

(SIT) flow.  SIT flow rapidly refloods the core and maintains cladding

temperatures well below 2200 degrees F.  For smaller breaks (0.01 sq.

ft.), break flow is low enough that HPSI alone is sufficient to limit or

even prevent core uncovery and maintain acceptable cladding temperatures.

The 1979 SBLOCA analysis results are overly conservative because:

o    The computer model uses a positive moderator temperature coefficient

     (MTC) even though TS 3.1.4 prohibits a positive MTC above 70 percent

     power.  A non-positive MTC would decrease or eliminate the

     calculated power increase at the start of the transient.  This would

     result in a faster depressurization, earlier reactor trip and SIAS

     on low pressure, and less RCS inventory lost.

o    The computer model uses 15.0 kW/ft for the peak linear heat

     generation rate (PLHGR) even though TS 3.2.1 limits PLHGR to 13.0

     kW/ft.  Using the lower value in the hot rod heatup analysis would

     result in a lower calculated peak cladding temperature.

o    The SBLOCA methodology itself contains additional conservatisms.

     ABB-CE has submitted for NRC approval an improved SBLOCA model which

     will demonstrate that charging is not required at San Onofre to

     mitigate the consequences of any SBLOCA.  (Reference letter, C.  B.

     Brinkman, ABB-CE, to NRC, Revision to Small Break LOCA Evaluation

     Model, CENPD-137, Supplement 2-P, May 23, 1996.

TEXT                                                          PAGE 5 OF 9

Best estimate LOCA analyses performed for Edison's PRA/IPE using EPRI's

Modular Accident Analysis Program (MAAP) code indicate that one train of

ECCS without charging will maintain peak cladding temperatures below 1500

degrees F.

Therefore, while the current computer model predicts unacceptable results

for some SBLOCA sizes without charging flow, Edison concludes that

2(3)MU021 not completely opening resulted in a negligible increase in

plant risk and had no actual safety significance.


Different Valve Designs:

2(3)MU019, 2(3)MU020, and 2(3)MU021 originally installed in both units

were 2 inch Kerotest Y-type Series 1513 valves (See Figure 2).  In 1985,

2MU021 was replaced and, because the original style valve was no longer

available, Edison used a redesigned series 1513 valve recommended by the

vendor (See Figure 3).  3MU021 and 2(3)MU019 were similarly replaced with

the new style valve.

During the investigation of the high auxiliary spray flow, Edison

obtained design information from Kerotest, including details of the

valves' closing springs.  Based on that information, Edison concluded

that the cracking pressures for the old and new style valves were 20 psid

and 2.5 psid, respectively.  If valves with different cracking pressures

are used in parallel flow paths, a flow imbalance can be created in low

flow conditions (i.e., when the valves are not fully open).  Because the

SBLOCA analysis assumes balanced flow and low flow (one pump running),

Edison concluded this was a condition that alone could have prevented the

fulfillment of a safety function required to mitigate the consequences of

an accident.  A 4-hour telephone report was made on 6/26/97 at 1803 PDT,

as required by 10CFR50.72(b)(2)(iii) (NRC Log #32545).

On 7/14/97, while continuing the review of this event, Edison noticed

that the vendor-supplied design information was incorrect.  The cracking

pressure for the old style valve was re-calculated to be about 3.0 psid,

essentially the same as the new style.  Edison concluded that the

as-designed and built charging design did comply with the licensing basis

of the SBLOCA analysis.  Therefore, this was not a condition that alone

could have prevented the fulfillment of a safety function required to

mitigate the consequences of an accident and Edison is retracting the

4-hour report made on 6/26/97 (NRC Operations Log #32545).

Generic implications:

Edison reviewed other uses of the new style Kerotest check valves in the

plants.  Seventeen safety related applications have been identified.

Fifteen of these, ranging in size from 1/2 inch to 1 inch, provide

pressure boundary isolation functions only and have no forward flow

safety function.  Their failure to open would not prevent the fulfillment

of any safety function.  The observed failures of the new style check

valve did not prevent the valves from closing.

There are two similar 2 inch check valves, 3MU067 and 2MU017, which have

a forward flow safety function (see Figure 4).  The use of these valves

in the charging pump discharge piping has been evaluated as acceptable

for the following reasons:

o    The application is in a single line so that a failure to open

     completely does not affect the flow distribution in the charging


o    The valves have been and continue to be tested quarterly by the IST

     Program.  Both valves have passed their IST by passing full flow at

     normal pump discharge pressure.  One of these valves (3MU067) has

     been in service since 1987 without incident.  Some seat back leakage

     has been observed.  This is acceptable

TEXT                                                          PAGE 6 OF 9

     because the back flow requirement is to assist in isolating the

     charging pump for on-line maintenance and does not affect charging

     pump operation.

o    The charging pumps are positive displacement pumps and will deliver

     their required flow up to the piping system's relief valve set point

     (2735 psig).  The observed check valve failures were the valves not

     opening enough to maintain balanced flow.  If these valves failed to

     fully open (as 2(3)MU021 failed), the affected pump would still have

     adequate pressure margin to deliver full flow (about 40 gpm) through

     the partially open valve.  In addition, in the SBLOCA analysis which

     credits charging flow, the RCS pressure is substantially below

     normal operating pressure, providing additional pressure margin.

o    The charging pumps themselves are protected from overpressure by a

     relief valve in the unlikely event these valves should fail to open.

     The resultant low charging system flow alarm would immediately alert

     plant operators in the control room.  Excessive discharge pressure

     is also indicated in the control room.

o    These two valves will be replaced during the next unit outage of

     sufficient duration, currently expected to be the scheduled

     mid-cycle outages.

Entry into TS 3.5.2 Condition A:

On 6/26/97, Edison management was briefed on the status of the charging

subsystem check valves and the impact on the plant.  At that meeting,

Edison management concluded that only one train of charging was

inoperable (cognitive personnel error) and that entry into TS 3.5.2

Condition A was the appropriate action.  That action was taken at 1413

PDT that day.  The following day, operations determined that, due to the

balanced flow requirement, both trains of charging should have been

declared inoperable.  Nevertheless, because the TS 3.5.2 Condition A

applies for one or two trains of ECCS inoperable, the actions already

taken by Operations were in compliance with the TS and no further actions

were required.

To ensure a correct understanding of TS 3.5.2, Edison will clarify its

Bases to address charging flow considerations and will review this

information with appropriate personnel.

Preoperational Testing, Technical Specification Surveillance Requirements

and In-Service Testing Program: It is not known when the check valves

first failed to open.

During preoperational testing in the early 1980s, the minimum charging

pump flow was verified.  However, the flow split between the two charging

lines was determined by analysis only, and was never tested.  The flow

split would be tested only if design changes were made to the lines which

would alter the flow split.  Because the replacement of MU021 was with a

"like-for-like" valve, the flow distribution was not tested at that time.

The surveillance requirements of TS 3.5.2 specifies the correct positions

and surveillance frequency of numerous ECCS valves.  Although not

specifically named, HV9202 and HV9203 are included in TS SR,

which verifies that "each ECCS manual, power operated, and automatic

valve ...  is in the correct position." MU020 and MU021 have no specific

TS SR requirements, and there is no specific SR in section 3.5.2 for the

charging flow split.  However, the Bases section does identify the

charging flow commitment of 44 percent of 36.2 gpm (15.9 gpm) to each


TS 3.1.9, Boration Systems - Operating, requires two RCS boron injection

flow paths (one path via the charging system and one path via the HPSI

header) to be operable, primarily for shutdown margin (SDM)

considerations.  TS SR requires that each boration flow path be

verified operable each 31 days.  Although Edison had been

TEXT                                                          PAGE 7 OF 9

complying with the literal requirements of TS SR for a boration

flow path, the TS SR did not ensure that the required ECCS flow balance

was maintained.  One of

these two SR sections (3.5.2 or 3.1.9) should include verification that

each charging line is operable.

TS SR requires each charging pump develop at least 40 gpm.  The

IST program performs this test.  Because the two parallel flow paths are

tested together during the pump IST, a closed check valve in one line

could not cause a test failure (opening the pump discharge safety valve

thereby reducing charging flow).  The test for the open function does not

specify the differential pressure for developing the acceptance flow.

Because the charging pumps are positive displacement pumps and the

testing is normally completed at an RCS pressure of 350 psia, adequate

pump discharge pressure (up to the discharge safety valve set point) is

available to pass the acceptance flow through a partially open valve.

The IST Program and TS SR procedure will be revised to ensure these check

valves operate as intended and that the charging line is operable.

Similar previously reported conditions:

In the past three years, Edison has not reported the failure of an ECCS

system because of the failure of a single component or component design


Figure 1 "Charging Subsystem:" omitted.

TEXT                                                          PAGE 8 OF 9

Figure 2 "Original Style Check Valve:" omitted.

Figure 3 "New Style Check Valve:" omitted.

TEXT                                                          PAGE 9 OF 9

Figure 4  "Charging Pump Diagram (typical of each unit):" omitted.


Page Last Reviewed/Updated Monday, March 30, 2020