Reliability and Probabilistic Risk Assessment and Plant Operations - May 30, 2002
Official Transcript of Proceedings
NUCLEAR REGULATORY COMMISSION
Title: Advisory Committee on Reactor Safeguards
Subcommittees on Reliability and Probabilistic
Risk Assessment and Plant Operations
Docket Number: (not applicable)
Location: Rockville, Maryland
Date: Thursday, May 30, 2002
Work Order No.: NRC-399 Pages 1-216
NEAL R. GROSS AND CO., INC.
Court Reporters and Transcribers
1323 Rhode Island Avenue, N.W.
Washington, D.C. 20005
(202) 234-4433 UNITED STATES OF AMERICA
NUCLEAR REGULATORY COMMISSION
ADVISORY COMMITTEE ON REACTOR SAFEGUARDS
(ACRS)
SUBCOMMITTEES ON RELIABILITY AND
PROBABILISTIC RISK ASSESSMENT AND
PLANT OPERATIONS
+ + + + +
DEVELOPMENT OF RELIABILITY/AVAILABILITY
PERFORMANCE INDICATORS & THE INDUSTRY TRENDS PROGRAM
THURSDAY, MAY 30, 2002
The Subcommittees met at the Nuclear Regulatory
Commission, Two White Flint North, Rockville Pike,
Rockville, MD, at 1:00 a.m., Mario V. Bonaca,
Chairman, presiding.
COMMITTEE MEMBERS:
MARIO V. BONACA, Chairman
GEORGE APOSTOLAKIS,
THOMAS S. KRESS
GRAHAM M. LEITCH
VICTOR H. RANSOM
WILLIAM J. SHACK
STEPHEN L. ROSEN
JOHN D. SIEBER
GRAHAM B. WALLIS
ACRS STAFF PRESENT:
AUGUST W. CRONENBERG, Senior Staff Engineer
MAGGALEAM W. WESTON, Staff Engineer.
ALSO PRESENT:
TOM BOYCE
RON FRAHM
DON HICKMAN
MARK SATORIUS
JOHN THOMPSON
PETTERI TIIPPANA
PATRICK BARANOWSKY
BENNETT BRACY
HOSSEIN HAMZEHEE
DALE RAMUSON
LINDE CARPENTER
D. COE
TOM HOUGHTON
A-G-E-N-D-A
ACRS Introduction
Chairman Bonaca. . . . . . . . . . . . . . . 4
NRC Staff Presentation
Introduction
Mr. Satorius . . . . . . . . . . . . . . . . 5
Mr. Baranowsky . . . . . . . . . . . . . . . 5
Mr. Thompson . . . . . . . . . . . . . . . . 6
Mr. Hamzehee . . . . . . . . . . . . . . . .61
Industry Trends Program
Mr. Boyce. . . . . . . . . . . . . . . . . 168
Mr. Rasmuson . . . . . . . . . . . . . . . 198
P-R-O-C-E-E-D-I-N-G-S
1:00 p.m.
CHAIRMAN BONACA: This meeting will now
come to order. This is a Joint Meeting of the ACRS
Subcommittees on Reliability and PRA and Plant
Operations.
I'm Mario Bonaca, Chairman of this Joint
Meeting.
The purpose of this meeting is to discuss
staff progress related to risk in forming the reactor
oversight program and the agency's pilot program to
assess the adequacy and trends associated with safety
system and availability using the performance
indicator PI approach.
Dr. August Cronenberg is the cognizant
ACRS staff engineer for this meeting, while Mrs.
Maggalean Weston is the designated federal official.
Rules for participation in today's meeting
have been announced as part of the notice of this
meeting previously published in the Federal Register
of May 8, 2002.
A transcript of this meeting is being kept
and the open portions of this transcript will be made
available as stated in the Federal Register notice.
It is requested that the speakers first
identify themself and speak with sufficient clarity
and volume so that they can be readily heard.
We have received no written comments or
requests for time to make oral statements from members
of the public.
We will now proceed with the meeting, and
I call upon Mr. Baranowsky of RES to begin.
MR. SATORIUS: My name is Mark Satorius.
I just have a couple of words. I'm a Chief of the
Performance Assessment Section in the Inspection
Program Branch of NRR.
We're very pleased to be here to address
the Subcommittee today and talk about the performance
indicator pilot program that we intend on beginning
later this summer.
And with me is Mr. John Thompson, who is
a member of my staff, who will give an outline on some
of the background and go forward with some of the
details of the pilot program as well as Hossein
Hamzehee who is here from the Office of Research, and
he'll go into more depth on some of the details of how
the pilots program is actually going to work.
And I think, Pat, you did have something
that you wanted to mention?
MR. BARANOWSKY: Yes. I just wanted to
mention that we have briefed this Subcommittee and the
full Committee several times in the past about the
programs that we have underway in the Operating
Experience Risk Assessment Branch that involved the
collection of data and the analysis of data that
relates to risk analysis. And what we're going to be
doing today is discussing two projects that the Office
of Research has been heavily involved in supporting
NRR and a lot of the technical basis for the work that
we're presenting here is founded on work that we have
done, as I said, over the last several years.
So I just wanted to give that as a piece
of background. It includes not only the databases, but
the system and component reliability studies of some
aspects of the action sequence precursor program and
in particular some use of the SPAR models. And I
think you'll see that a few places.
So having said that, I'd like to just turn
it over, I guess, to John Thompson to get started on
what the first issue is about.
MR. THOMPSON: Thank you, Pat.
Good afternoon, members of the
Subcommittee. Can you hear me?
I'm going to be talking today about an
overview of a pilot program to develop a replacement
performance indicator for the mitigation systems,
safety system unavailability.
We've been in the revised oversight
process for about 2 years now and we've observed some
problems with the current SSU indicator and has led us
to develop a working group to address some of these
issues. We are here today to brief you on status of
where we are in a pilot program to replace this
indicator, and hopefully your knowledge level will
come up and understand where we're trying to go and
what we're trying to do for the near future with this
pilot program.
To se the background for this program,
SECY 99-007 addressed the need to further refine the
use of the performance indicators by developing risk
based performance indicators and taking steps in that
direction.
During the first 2 years of the ROP staff
and industry, like I said, identified problems with
the current indicator and make several interim changes
as well as trying to address the need for the longer
term. We formed a working group and have met
regularly since the spring of 2001 to discuss these
issues with industry and with members of NEI and with
the staff.
MR. APOSTOLAKIS: SSU is safety system
unavailability?
MR. THOMPSON: Correct.
We made a formal decision --
MR. APOSTOLAKIS: Which was really safety
train, not system. The way it was defined it was a
train.
MR. THOMPSON: But the indicator gives us
system indication.
MR. APOSTOLAKIS: I thought it was a
train?
MR. BARANOWSKY: I think you're right,
George. It was a train level indication that was meant
to be a surrogate for the system's performance. It was
never put together into a --
MR. APOSTOLAKIS: A system.
MR. BARANOWSKY: A system type thing. But
the threshold for performance were based on
understanding of the implementations of that train in
a risk model, if you will. That's all we originally
did.
MR. APOSTOLAKIS: But you didn't know in
a particular case whether the system was a one out of
two or one out of three
MR. BARANOWSKY: Yes. We've had a lot of
problems, and those are some of the things we're going
to try and correct with the methodology we're
proposing here.
MR. APOSTOLAKIS: And also the
unavailability was only the maintenance, really. I
mean, how long was it out? It didn't include human
error or error -- probability of failure to start,
right? It was just that --
MR. BARANOWSKY: I think it included some
of those things, but the way it included it was
somewhat problematic, and we're going to try and go
over that also.
MR. APOSTOLAKIS: Okay. All right.
MR. THOMPSON: Going hand-in-hand with
this effort was the development of the risk based
performance indicators, and you were briefed on that
in 2001. And that'll be part of Hossein's presentation
following mine.
And this pilot is recognized as an
evolutionary step toward enhanced PI development.
MR. APOSTOLAKIS: So what your heading
calls mitigating system is what your main body says
safety system? It's the same thing?
MR. THOMPSON: Yes.
MR. APOSTOLAKIS: Okay. If they were
considering it, it would MSU?
MR. THOMPSON: Yes, it's the cornerstone
title that we borrowed from, and to give it a
different title than the current indicator.
MR. APOSTOLAKIS: All right. So it's the
same thing?
MR. THOMPSON: Yes.
MR. APOSTOLAKIS: Good.
CHAIRMAN BONACA: Now, I'm sorry. The
heading said "Mitigating System"?
MR. THOMPSON: Yes.
MR. APOSTOLAKIS: But inside you see the
second bullet says "with current SUU"?
MR. THOMPSON: Yes.
MR. APOSTOLAKIS: That's safety system
unavailability, but it's mitigating system.
CHAIRMAN BONACA: Okay. So it is the same
thing?
MR. THOMPSON: Yes, we gave it a different
title to denote that it's a different indicator with
different systems and stuff.
MR. APOSTOLAKIS: So your new indicator
will be mitigating system?
MR. THOMPSON: Yes, that's the title of
it.
MR. BARANOWSKY: And it's going to be a
system level indicator, whereas the other one was more
of a train level indicator --
MR. APOSTOLAKIS: Good. Good.
MR. BARANOWSKY: -- with some implications
for a system.
MR. APOSTOLAKIS: Very good.
CHAIRMAN BONACA: But, you know, one could
contend that a trip function is a separate system. I
mean, yet you're counting that as an initiator. Isn't
it confusing, I mean, to --
MR. APOSTOLAKIS: I wonder whether -- I'm
sorry, I don't want to interrupt.
CHAIRMAN BONACA: No, I just was trying to
understanding.
MR. APOSTOLAKIS: Is a high safety
injection system a mitigating system?
MR. THOMPSON: Yes.
MR. APOSTOLAKIS: I thought it was a
safety system.
MR. SATORIUS: But it's in the mitigating
cornerstone of the ROP.
MR. THOMPSON: Yes.
MR. APOSTOLAKIS: Right.
MR. SATORIUS: It's considered because it
mitigates the effect of an accident, so it's in the
mitigating portion of the cornerstone of the ROP.
MR. APOSTOLAKIS: Well, in other contexts
prevention and mitigation; prevention is everything
before core melt and mitigation and everything after
core melt. Now your point of reference is the
initiating event.
MR. BARANOWSKY: Agreed. From a risk
analyst point of view that's the same way I talk, too.
But the way the direct oversight process has been set
up, mitigating systems are the preventive ones. And
we're not going to try and go back and change 99007.
CHAIRMAN BONACA: That was because -- I
mean before PRA. But before we believed that there
could be a core damage, I mean --
MR. APOSTOLAKIS: Remember, 007 has a
license to kill.
CHAIRMAN BONACA: Exactly.
MR. APOSTOLAKIS: So we can't go back--
CHAIRMAN BONACA: With the old terminology
of the FSAR. Okay.
MR. ROSEN: But we don't need a license
here.
MR. BARANOWSKY: By the way, the names are
subject to change if they aren't pleasing in some way.
MR. APOSTOLAKIS: Well, it's just in
different context we use different names. I mean, you
know, the standard thing about prevention and
mitigation refers to core melt. You guys refer to the
initiating event and you're mitigating the initiating
event. It's a good usage of English.
CHAIRMAN BONACA: I'm only saying that
before PRA became of age, that was the definition they
were using for the FSAR.
MR. WALLIS: Mitigating the accident,
George.
CHAIRMAN BONACA: Mitigating was
mitigating an initiator --
MR. APOSTOLAKIS: Not the initial event.
MR. WALLIS: Now you've got accident.
MR. APOSTOLAKIS: Well, but --
MR. WALLIS: ECCS mitigates the accident.
MR. THOMPSON: That's right.
MR. APOSTOLAKIS: But what is the
accident, that's the question.
MR. WALLIS: Accident's already underway.
MR. APOSTOLAKIS: But it has not reached
core melt.
MR. WALLIS: That's right.
MR. SHACK: The mitigating systems work,
it may not.
MR. THOMPSON: We just wanted to have a
slide to go over some of the known problems that we've
been trying to deal with with the current indicator.
The first is that the risk insights are not accounted
for in the current indicator because it uses design-
basis function instead of PRA or unit functions.
At that thresholds developed or the SSU
were not plant-specific nor were they risk informed
thresholds. The new --
MR. APOSTOLAKIS: They were not? They
were. They were not the right thresholds, but they
were --
MR. HAMZEHEE: Were not plant specific.
MR. APOSTOLAKIS: Right. And now you guys
are saying they should have been plant specific?
MR. HAMZEHEE: That's right. As we go
along with this presentation, we say --
MR. THOMPSON: We are moving in that
direction.
MR. WALLIS: We're trying to agree with an
ACRS point. They should be more plant specific.
MR. APOSTOLAKIS: Several centuries ago.
MR. WALLIS: Exactly. Well, we've finally
caught up.
MR. APOSTOLAKIS: See, that answers my
earlier question. We do have an impact after a few
years.
MR. ROSEN: It takes a while.
MR. APOSTOLAKIS: They're mitigating it.
MR. ROSEN: Now, it's going to be plant
specific. Does that mean that a plant that has three
trains of safety systems versus a plant that has two
trains will get some more credit?
MR. HAMZEHEE: That's correct, yes. And
when we get to the technical aspects of it, you see
that, yes.
MR. ROSEN: I think I'm done here. I think
I'm done here.
MR. KRESS: Mission accomplished.
MR. APOSTOLAKIS: I have complained from
day one.
MR. THOMPSON: Actually, I don't want to
leave you with the impression that the thresholds are
plant specific. They are standard thresholds but the
margin to the thresholds are plant specific.
MR. BARANOWSKY: The risk thresholds are
standard. Just like REG Guide 1.174 has what you might
call thresholds of regulatory acceptance, the
thresholds of risk that are associated with the
performance of these systems are the same. But the
performance within any given system can change
depending on the --
MR. APOSTOLAKIS: How much margin you
have?
MR. BARANOWSKY: Right.
MR. THOMPSON: The level of redundancy or
the risk associated with that system.
MR. APOSTOLAKIS: You'll explain though,
probably.
MR. BARANOWSKY: And he's going to explain
it all if we get there.
MR. APOSTOLAKIS: So now I see this thing
that it was inside NRC, risk manage, does it replace
risk informed?
MR. THOMPSON: Yes.
MR. APOSTOLAKIS: And the Commission has
agreed to that?
MR. THOMPSON: I don't know that.
MR. BARANOWSKY: This is also news to me.
I'm hoping to get some definition other than from
inside NRC.
MR. APOSTOLAKIS: There was something
inside NRC. You guys are changing it already.
MR. HAMZEHEE: Well, we're proactive.
MR. APOSTOLAKIS: I'm not sure risk manage
is a good idea.
CHAIRMAN BONACA: And before we change
everything, you know --
MR. APOSTOLAKIS: At least we can discuss
it a little bit.
MR. THOMPSON: Certainly.
MR. APOSTOLAKIS: We've been talking about
risk informed initiative now for what? Five or six
years? The least you can do is ask the Commission
whether they agree with the change. Okay. I know
that one person wants that, according to inside NRC,
but I think it's a good idea, as Pat said, not to get
our marching orders from inside NRC.
MR. BARANOWSKY: Well, anyhow, I do think
it's important to note that we do have five points
that we're trying to correct in at least skipping some
of the terminology problems here. One of them I think
was that second one, was a pretty important one.
So why don't you go on, John?
MR. THOMPSON: Yes. The demand and demand
failures are not properly accounted for by the current
indicator either. It's kind of a mix, a hodgepodge
that doesn't match well. And the new indicator, the
new pilot more properly accounts for those instances.
And the other big issue with the current
indicator was that use of fault exposure hours can
over estimate the significance and then result in a PI
that can no longer measure further degradation of
performance because the indicator's already yellow or
red and smaller increments of degradation is not
measured.
MR. APOSTOLAKIS: So you're going to tell
us later why these are valued points?
MR. THOMPSON: Yes.
MR. APOSTOLAKIS: Because it's not clear
to me what you mean by this?
MR. THOMPSON: That's correct.
MR. APOSTOLAKIS: And the first bullet is
unclear, too.
MR. THOMPSON: Yes, all of that will be
cleared up later.
And then the last issue is that cascading
of support system unavailability to the monitored
system overstates the actual unavailability of the
monitored system which end up measuring is not only
the availability of the monitored system, but the
availability of the support system that maintains that
system. And it kind of defeats what you're trying to
measure. The new pilot corrects that by monitoring
separately the support system.
MR. APOSTOLAKIS: So you're becoming more
PRA oriented?
MR. THOMPSON: This is definitely a step
in that direction.
MR. APOSTOLAKIS: Yes.
Another question. I mean, PRA's have been
distinguishing between support systems an front line
system now for 25 years. I don't understand why we
have to stop -- anyway, go ahead.
This should be -- the whole thing should
be PRA based.
MR. BARANOWSKY: Well, remember, this is--
we're trying to bring what we learned from the risk
based performance indicator work into this project;
just the things that we tested out and believe work.
And so you're seeing a lot of risk concepts here
today.
MR. APOSTOLAKIS: Okay.
MR. BARANOWSKY: Yes.
MR. ROSEN: The objectives of this program
simply stated was that we wanted to create a better
indicator and a more accurate one of performance that
adds value and solves the known problems that I just
over without also adding undue burden both to the
inspectors that got to oversee this PI as well as
industry which has to implement it.
We want to calculate in this pilot the
revised unavailability -- unreliability values.
MR. APOSTOLAKIS: Now you're mentioning
unreliability for the first time. That's not part of
your five bullets? Are you adding it as a PI
MR. BARANOWSKY: Yes.
MR. APOSTOLAKIS: So the previous five was
corrections to something that existed?
MR. THOMPSON: Yes.
MR. APOSTOLAKIS: But this will be added
as a sixth?
MR. ROSEN: You note that we mentioned
unreliability in one of our letters, George.
MR. APOSTOLAKIS: Oh, yes.
CHAIRMAN BONACA: That was already part of
the Phase-1, right? Phase-1 development?
MR. THOMPSON: Yes, that's one of the
objectives of this new PI is to calculate both and to
compare the results from that to the existing PI data
and to ascertain whether or not the differences
observed in the changes address our concerns and suits
the needs of the revised oversight --
MR. ROSEN: That implies you're going to
do that retrospectively? You're going to go back?
MR. THOMPSON: We're going to do both.
MR. ROSEN: And calculate all this stuff?
MR. THOMPSON: We're doing both.
MR. ROSEN: And compare it to what you got
-- what you have from -- the existing SSU PI data?
How you could to do that when they haven't captured --
many places haven't captured that data respective, you
know?
MR. THOMPSON: Well, we're going back to
look at the pre-ROP data and run that data through the
mechanics of this new PI and then look at it, see how
the SDP looked at it.
We're also going to look at the last two
years of data through tabletops exercises. And then
we'll look at it as the actual data comes in through
in the pilot.
MR. ROSEN: And then in answer to my
question, how you going to get the manuscript --
MR. HAMZEHEE: Well, Steve, I think there
are two parts of your question. One is as part of the
RBPI we looked at some previous data to see if this
concept worked. And we looked at 44 plants, and I'll
talk about them in more detail and demonstrated that
these are reasonable and they do provide adequate
performance indication.
The other thing that John is talking about
is insights learned and the improvements we want to
make the existing PI. So these are mainly for the
future PI. But to make sure that they do work, we are
going to do a visual validation of going and looking
at some, whatever data we can get for the on demand
failures and a number of demand basically based on
EPICS, that is the only available database in the
industry. And in order to see how these things work
and what kind of results we get.
But the main objective is not to be
retroactive and try to regenerate the past 5 or 10
years of performance.
MR. ROSEN: You think EPICS has captured
enough demand data to do that?
MR. HAMZEHEE: For a validation purpose,
yes. And we have a section in phase I RBPI report
NUREG 1753 that shows that. But for the future in
order for these PIs to be fully implemented, then one
of the conditions is for the industry to report
accurate reasonable data to support this performance
indicator.
MR. ROSEN: Yes, I would just put some
measure of grain of salt on the past EPICS data.
MR. HAMZEHEE: That's correct.
MR. ROSEN: Once the staff puts this in
place, then you as a regulatory requirement, then you
can have more confidence.
MR. HAMZEHEE: That's correct. And as
part of this pilot program industry is going to work
with the NRC staff to provide the actual data for some
time period so that we can go through all these "what
if" questions and try to validate the results.
So you're right. In the past we did not
have enough information on the on demand failures.
CHAIRMAN BONACA: Yes, and that's good
just as a baseline for us. Two years ago commenced
our work that was being done and two recommendations
were made. One was that we work with the industry to
improve the ethics to the point where there will be
consistent reporting, otherwise --
MR. ROSEN: That's correct.
MR. HAMZEHEE: And the other one was to
provide some -- to work towards a common definition of
unavailability and reliability with the industry so
that we're comparing apples and oranges there.
MR. ROSEN: Yes.
MR. HAMZEHEE: And it would be worthwhile
for us to understand if any progress has been made on
those two issues.
MR. BARANOWSKY: Now, for sure we have had
some progress on the EPICS thing, except I want to
point out that not all plants are embracing the EPIC
system, so they're not necessarily providing complete
data. But a lot are and we've worked pretty closely
with INPO in their working groups. And that involves
the technical folks from the plants that are providing
this information.
So even though it maybe has on a few
places some errors on the order of 10 to 20 percent on
counting demands, that's probably good enough to get
an idea. Because even if the demands are off 10 or 20
percent, we're in the ball park. Now, a few plants
might be off by 50 percent, but most of them are based
on what we can understand providing the bulk of the
data that's been requested. And we think we've come
together on the unavailability unreliability
definitions through this project, I believe, and
Hossein will cover that in a few minutes.
CHAIRMAN BONACA: Good.
MR. WALLIS: Are you going to tell us what
the first bullet means? Are you going to tell us
that? I mean, I have no idea what a better indicator
performance is, and I don't know how you can tell when
it's good and when it's better. And more accurate
means nothing to me. I mean, if number of SCRAMs is--
if I measure SCRAMs to 2 significant figures, is it
better than 4 significant or 1 significant figures, or
something. Accurate isn't a good word.
MR. HAMZEHEE: Yes, we are going to talk
about this. This is for him to set the ground for us
to know what is our --
MR. WALLIS: And having value, I have no
idea what your value is so I don't know how to add it.
MR. BARANOWSKY: Okay. I think he's
saying that this will provide a method that fixes some
of the problems that we've have in the past where
there were approximation that resulted in many so
called frequently asked questions where because of the
lack of rigor in the development of the PI from a
methodological point of view, we went into more and
more permutations on how to deal with those slight
deficiencies. And what we're trying to do is get away
from that, have something that has more rigor up front
so we don't have to come up with special cases on how
to deal this to make it sort of fit like what we would
expect if we had a more rigorous formulation for the
indicator.
MR. THOMPSON: And not only that, a lot of
these cases ended with a conclusion that the indicator
was not a good way to do this. And we would --
MR. WALLIS: I think you need to start
with a definition of what an indicator should do.
What it's purpose is, how you measure one being better
than another in terms of what it does. Then we can
tell whether the new one is better than the old one.
MR. THOMPSON: Well, I think this new
indicator is going to do something different than what
the old one was trying to do. It's trying to
accurately capture the unavailability and
unreliability which was not necessarily the purpose of
the original indicator.
MR. BARANOWSKY: Yes, I guess what we
haven't done is sat here and other than identifying
those five points on the prior viewgraph, gone through
the methodology things in the current indicator that
are problematic. And I guess we're more focusing on
what's in the one that we're proposing. But if I step
back a bit, just the fact that we didn't have demands
and failures in the prior indicator, it was missing
something. Just the fact that there was a single
model with a single threshold for every plant, we knew
that was a problem.
So we've identified a number of things
that we know are flawed in the current indicator and
what he's trying to say is we're going to make
progress in removing those things. And if those are
flaws and we fix them, then we know this is a better
indicator.
MR. WALLIS: Yes, I think you're
proceeding by solving problems which have been
recognized.
MR. BARANOWSKY: Right.
MR. THOMPSON: Exactly.
MR. BARANOWSKY: That's our approach.
MR. THOMPSON: That's right.
Another objective of the pilot is to
minimize the differences and increase the consistency
where we can between this pilot, the maintenance rule,
PRA and the SDP. And as I go this, you'll see where
we're trying to address those things.
We also want to exercise the methodology,
the actual reporting, the mechanics as licensees would
really do it if we went to full implementation with
this PI such that the data will come in, we'll
actually see the data as it would be for full
implementation.
And then we want to identify after the
pilot any unintended consequences that might result
and assess their impact, if any.
Now, we do have a list of questions,
predetermined questions that we want answered during
the course of the pilot. These probably aren't all
inclusive. They were just some of the ones that the
working group has come up with.
And the first one is one we just went over
in the last slide, is this a better indicator of risk
than using the SDP which we're forced to do through
the frequently asked question resolution such that the
staff may need only rely on the PI indication for the
risk significance and not do the SDP. That is a big
issue with industry right now. That is something they
want, and they think and the working group believes
that if the indicator works as we designed it, we may
be able to achieve that.
MR. APOSTOLAKIS: To achieve what?
MR. THOMPSON: To be able to use the
outcome or the color characterization from the PI as
the appropriate risk characterization.
MR. APOSTOLAKIS: This touches on
something I think that's much bigger, which is -- I
mean, it's not clear to me that the PI should actually
deal with risk. One of the major comments we made in
our last letter, I think, was that the PI by itself as
a measure of risk is not very minimal. That's why the
red threshold was so bad. Because you had to go
through 23 SCAMS to see a significant change in risk.
And as many of my colleagues here have
been saying, we are looking at the PIs to marshal our
resources. Now when you get an early indication that
something's going wrong, and we're going to send more
NRC inspectors or we're going to look more carefully
and so on. But this is not really related to risk.
So it seems to me that this kind of
approach has to be resolved because you may be going
down the same path as the previous PIs. See, the
prblem there was that in order to see a change of 10-3
or 4 -- not 3; 4 or 5 in the CDF because of one PI,
you have to change the PI so much that it was
unrealistic. And we know that accidents don't happen
that way. In accidents you have usually a combination
of events. Right? It's not one thing that you have
too many SCRAMs. The accident is really that you have
one SCRAM and you have other things that are failing.
So I'm not sure that trying to pursue the
PIs as better indicators of risk is a good idea.
MR. BARANOWSKY: Let me clarify a couple
of things on that. In other words, there's another
point maybe.
MR. HAMZEHEE: No, go ahead.
MR. BARANOWSKY: It is a conditional
measure of the risk, which is basically what I think
you were pointing out, George. Holding other things
constant.
In terms of whether some of the indicators
require many failures, if you will, or incidents to
occur before the indicator trips a threshold, I think
that's a measure of the risk significance of the
functions that we're trying to monitor performance on.
And maybe we're not monitoring the right performance
or maybe we need to monitor things differently.
The MSPI indicator is meant to provide an
accumulation of unavailability/unreliability and what
the implications are with regard to performance as it
relates to how risk is changing if that performance
declines.
The SDP looks at individual instances such
as something failed or something was out of service,
and that unavailable contributes to an amount of risk,
such as a risk meter might look at it. I don't know
of any risk meter, for instance, that has a change in
the failure rate or demand failure probability as a
function of time every time one has a failure you
update. I don't think they do that. That's what this
indicator does.
So it accumulates information on a
performance of whatever system or function we're
looking at instead of looking at individual points.
MR. APOSTOLAKIS: I understand that. But
I think the notion that everything in the -- should be
tied to risk is questionable.
For a long time I was advocating -- not
advocating as it should be, but I was working on the
assumption that it was. And some of my colleagues here
said no, that's not the purpose of this. Whether I
have three SCRAMs in a year or not tells me something
else. It doesn't really tell me much about risk, but
it tells me that I should be going there and looking
and that something is not proper, you know. That's
very different from tying it to risk.
MR. SHACK: Yes. I mean, are the PIs
measuring the safety status of the plant or are they
measuring the performance? And I think, you know, you
could argue that looking at one of these PIs, yes, I
can tolerate a very large variation before that in
itself is a measure of an unsafe condition. However,
as an indicator of the performance of the licensee,
you might have -- you know, the risk informed
indicator I think is a reasonable thing. I think the
thing that we're getting at is how do you set the
threshold. And setting the thresholds on a delta CDF
we argued led to problems because you were isolating
something. And you're really not trying to measure
the plant safety status, you're trying to measure
performance.
MR. APOSTOLAKIS: Right. And performance
is in the sense that this fellow deviates too much
from the industry.
MR. SHACK: And as an indicator of
everything else that he's doing. I mean, we're
looking at one thing.
MR. APOSTOLAKIS: Right.
MR. SHACK: And it's indicator of how he's
performing overall because we're not measuring
everything that's related to the plant.
MR. BARANOWSKY: Yes. Well basically what
we've done, of course, in this whole oversight process
is aggregated some things. The cornerstones themself
are disaggregation. I mean, we could have just put
down plant safety status and put containment barriers,
mitigating systems and initiating events all into one
thing; plant safety status. But then you've got a
problem with figuring out well when the safety status
is bad, how do you know what to go look at.
So there was a judgment call that was
made. And we're not trying to revisit that issue
today, even though I think I understand what the
nature of the concern is because in fact in the risk
based performance indicator report we did talk about
an integrated indicator. That would be essentially at
the plant level --
MR. APOSTOLAKIS: Oh, all right. Yes.
MR. BARANOWSKY: But I don't think we're
ready to go there yet.
MR. APOSTOLAKIS: I like that.
MR. BARANOWSKY: We're not ready to go
there yet.
MR. APOSTOLAKIS: I know.
MR. BARANOWSKY: We have to take some
steps, and this is the first step. If this step works
and it looks like it makes sense to address the issues
of plat status versus monitoring performance, then I
think there's probably something we can work on.
MR. APOSTOLAKIS: The ideal situation
would be to have PC model where I can input the
findings and get the delta CDF. Isn't that what --
MR. HAMZEHEE: We already have that.
MR. THOMPSON: Yes, we call that PRA.
MR. BARANOWSKY: Yes, we call that a PRA,
right?
MR. BARANOWSKY: Right. And we're having
enough trouble changing things like 5046, Appendix J
and everything else so if we attack that --
MR. APOSTOLAKIS: It runs already, though-
-
MR. BARANOWSKY: -- none of you will be a
member of this committee by the time we end up coming
with a new indicator.
MR. APOSTOLAKIS: Because the findings are
not always PRA.
MR. HAMZEHEE: And all the plants have
that already and they do it for internal purposes.
And I think Steve had it as his --
MR. APOSTOLAKIS: No, but a lot of the
findings in the inspections require additional
processing.
MR. BARANOWSKY: You're going to see a lot
of use of PRA in this that you haven't seen before as
far as I can tell. And I think we're just going --
MR. APOSTOLAKIS: Let me take one point,
and I did try last time we were writing the letter but
it was pretty much -- I think the way the action
matrix is put together causes a lot of
misunderstandings and leads you the wrong path.
Because it tries to -- it has two different purposes,
really.
One purpose is to look at performance and
the other is to look at risk. And I think the
attitude so far has been risk. Let's try to make
everything risk related. And, you know, sometimes
that leads to an unrealistic result. But that's
something to think about with other guys, not you.
MR. BARANOWSKY: There has to be a nexus
between performance and risk or else we can't --
MR. APOSTOLAKIS: At some point.
MR. BARANOWSKY: -- set the performance
targets in any rational way.
MR. SIEBER: Well, I sort of disagree with
that to some extent. If you look at thresholds, the
ones that seem to be outrageous are the ones
associated with initiating events. And perhaps that's
because initiating events are designed, there's a
design in the plant to cope with them. And so on that
basis initiating events like reactor SCRAMs for
example are better off being performance based than
risk based. But perhaps mitigating systems might have
some value in being risk based.
And so I would treed carefully in this
area. But the way I see it, I see applications for
both performance basing and risk basing in the PI.
MR. APOSTOLAKIS: Sure. Sure. But right
now I think most people look at the action matrix and
they really think in terms of risk.
MR. SIEBER: Risk.
MR. APOSTOLAKIS: Except some members of
this committee.
MR. SIEBER: That's right.
MR. KRESS: And I think it's very
difficult to take an individual performance indicator
and convert that into risk. I know if it's a
mitigating system, you can plug in the change in
unreliability -- in the PRA. But that's not what
we're after here, I don't think.
That as an indicator of what things --
other things may go wrong, and you don't know what
those other things are and you don't know how to input
those into the PRA at the same time.
So I think we make a mistake in actually
saying selecting a delta CDF due to this
unavailability to represent our threshold. Because I
don't think we have a way to establish that threshold
that way. I don't think you have any basis for
choosing. You could probably arbitrarily choose it,
but I don't know how you would do it.
MR. APOSTOLAKIS: The problem I think
comes back to what I said earlier. Accidents don't
occur because of a single thing.
MR. SIEBER: That's true.
MR. KRESS: Yes, that's exactly right.
MR. APOSTOLAKIS: It will not be the
unavailability of something. It will not be an
initiating event, it will be a combination of some
hardware or some human error, or some of this and some
of that, and all of a sudden you have a problem.
MR. BARANOWSKY: Could I just point out --
MR. APOSTOLAKIS: That is difficult to
capture, though. It's very difficult.
MR. BARANOWSKY: We're talking about not
one performance indicator for one system or component.
We're talking about performance indicators that cover
several systems and components. So the issue of --
MR. KRESS: But that's my whole problem is
you don't have a way to conglomerate those together at
the same time.
MR. BARANOWSKY: Well, actually, we do
have a way and that's through the PRA.
MR. APOSTOLAKIS: So great. Let's wait
until we see how you do it.
MR. BARANOWSKY: If we can get --
MR. KRESS: You don't have a way to set
the threshold.
MR. BARANOWSKY: Yes we do.
MR. APOSTOLAKIS: That's the way to do it.
MR. KRESS: Well, okay. I'll wait.
MR. APOSTOLAKIS: But I think the first
bullet, though, I mean really at this time it seems to
argue for the SDP, because the SDP is --
MR. BARANOWSKY: The SDP is looking at
single incidents.
MR. APOSTOLAKIS: Well, it's not done
correctly?
MR. BARANOWSKY: What?
MR. APOSTOLAKIS: We also recommended that
if they find three things, they should do one SDP for
the three things, not separately.
MR. HAMZEHEE: Or should they be looking
at them at the same time.
MR. BARANOWSKY: Oh, no. If they find
three things and you want to know the risk
significance, then you've clearly you've got to have
an integrated model.
MR. APOSTOLAKIS: Exactly.
MR. BARANOWSKY: But if you want to know
about performance, how has performance changed, do you
go every single time there is a failure or an
unavailability if you do -- equivalent, do an ASP
analysis to the SDP.
MR. APOSTOLAKIS: I agree with you. And
I think the action matrix --
MR. BARANOWSKY: I mean, if that's true,
then what's the role of reliability and unavailability
in the PRA itself?
MR. APOSTOLAKIS: Yes. I think you're a
little ahead of the ROP. You are ahead of the ROP.
Because I really think the action matrix should make
that very explicit that performance and safety, you
know, they overlap a lot, sure. But there may also be
different objectives.
MR. BARANOWSKY: There is a bunch of
simplifications in the reactor oversight process
framework. Okay. We knew they were there when we put
them, and what we're trying to do is slowly but surely
improve on those.
We can't just overhaul everything at once.
So we're going to take them on --
MR. APOSTOLAKIS: Well, let's see what you
have solved already. I mean, we've never been there.
MR. BARANOWSKY: Based on one of the
thorniest issues.
MR. APOSTOLAKIS: The way we're going,
we'll never get there.
MR. HAMZEHEE: We're on page 3, I think.
MR. KRESS: What did you say?
MR. HAMZEHEE: We're still on page 3 and
we have 20 some pages.
MR. APOSTOLAKIS: Can you use, I mean with
the permission of the Chairman, your judge on this,
keep the motherhood statements.
CHAIRMAN BONACA: Yes.
MR. THOMPSON: Let's go to the next
viewgraph.
MR. BARANOWSKY: And I'll turn it over to
you.
MR. THOMPSON: All right. This is an
overview of the workings of the pilot.
The MSPI comprises the four existing
systems currently monitored by the SSU PI, plus we're
adding in the support cooling system as monitored
systems for the pilot. That really means the central
service water or its equivalent and the component
cooling water or its equivalent for the boilers.
It's a 12 quarter rolling average like the
current PI, but we're going to monitor and calculate
the unavailability and unreliability in relative terms
of a new unit that we're calling delta CDF index. I
know it doesn't say index here, but we've come up with
an equivalency to a conditional CDF.
MR. BARANOWSKY: And that's because of the
issue of Dr. Kress and Dr. Apostolakis raised about it
being an incomplete measure of the risk.
MR. THOMPSON: Yes. The PI incorporates
plant specific models and uses data to calculate the
index. And Hossein will go over that in detail with
his slides.
The thresholds were developed using the
standard risk insights and are defined as 1E-6 for the
green/white, 1E-5 for white/yellow and 1E-4 for
yellow/red with the units of CDF Index.
MR. APOSTOLAKIS: So they are risk
related?
MR. THOMPSON: Yes, and not plant specific
thresholds.
One of the big differences, too, is that
discovered conditions that prevent fulfillment of the
safety function of the monitored system will be
specifically accounted for in the unreliability
portion of the PI.All right.
MR. LEITCH: I think I missed a subtlety
here. You said it, but CDF index as compared to delta
CDF? I'm not sure I understand the difference.
MR. THOMPSON: As Pat said, it's because
it's a conditional look at what we're trying to
monitor and not a broader look like what the SDP would
do.
MR. ROSEN: Another way to say that I
think, Graham, is that if you really wanted to
calculate delta CDF, you have to use the plant model,
the PRA model and calculate the whole CDF. Here
they're just looking at a couple of systems. It
doesn't take into account the interrelationships
between all of the plant's components and the
different initiating events.
MR. BARANOWSKY: And we're only looking at
level one, so we don't have other factors.
MR. ROSEN: And you're not looking at
shutdown risk.
MR. THOMPSON: Or operator recovery, or
anything like that.
MR. ROSEN: So it's an index, it's just
not the whole deal. Ultimately the right way to do
this is to use the plant PRAs.
MR. SATORIUS: Right.
MR. ROSEN: But they're just taking -- you
know, before they could just crawl, now they're
standing up and trying to walk. And then ultimately
something else will happen.
I would like to ask a question about the
last bullet on that slide. "Discovered conditions
that prevent the fulfillment of the safety function"
now are counted in the unavailability. They're
considered in the unavailability index. That is, you
go back to the last time you knew it worked and
usually take half of that time.
MR. SATORIUS: Right. The fault exposure.
MR. ROSEN: The fault exposure time and
the unavailability. And now what does this mean?
That it's being accounted for in the unreliability PI;
I don't get it.
MR. THOMPSON: Let me take a first stab.
The problem with what we're doing now is
while true that the PI as we have now was designed to
do that, it's the so called T over 2 issue and the PI
is not measuring that no longer. We're using the SDP.
We're taking those instances because the PI can over
estimate the significance of that issue and having it
monitored using the SDP.
So the PIs really not accounting for those
things no longer, especially with the interim fix that
we use with the 99-02 Rev. 2.
MR. APOSTOLAKIS: Now, the unreliability
will be over a period of time, right?
Unavailabilities at the given time? So this will
include the operator intervention to stop the system
if it doesn't stop?
MR. ROSEN: I think we're getting --
MR. THOMPSON: I think you'd better try
that answer.
MR. BARANOWSKY: No, it's really simple.
If the condition was such that had you tried to
initiate a start of the piece of equipment, it
wouldn't have functioned. It would have failed on
demand. So we call it a failure on demand. And since
we're looking at performance over a long period of
time, we just take that as one failure, one demand and
we put it in with the others that have occurred. And
we compute an unreliability for failure on demand.
Whereas before we were looking at it as a single
incident by itself without looking at any other prior
history and just saying how significant is that
condition.
MR. APOSTOLAKIS: So unreliability is not
used in the sense of a PRA. Unreliability is the
failure to start on demand?
MR. BARANOWSKY: That's what this would
be. Let's say turbine driven pump, somebody walked by
the turbine driven pump and found that some valves
were in the wrong position --
MR. APOSTOLAKIS: But you guys are
producing these other reports that are very good that
are looking at the operating experience and you're
calculating unreliability as the probability of the
system not working over a period of time.
MR. BARANOWSKY: Well, we do it both ways.
We look at the probability that it will not operate
when called upon, at start up in order words, and the
probability that it will not continue to operate to
fulfill its safety mission.
MR. APOSTOLAKIS: Right.
MR. BARANOWSKY: That's all included in
this indicator now.
MR. APOSTOLAKIS: But the probability of
not starting any continuing to operate --
MR. BARANOWSKY: Yes.
MR. APOSTOLAKIS: -- is the unreliability.
MR. BARANOWSKY: That's right. And that's
exactly the way we're defining it.
MR. APOSTOLAKIS: Not just the probability
of failure to start?
MR. BARANOWSKY: If we get there, I think
you're going to see it.
MR. APOSTOLAKIS: All right.
MR. ROSEN: You may not get there, but
you'll have fun on the way.
MR. HAMZEHEE: And I think, George, the
way we're doing it now is we are very consistent with
the PRA approach. So it's nothing --
MR. APOSTOLAKIS: Then your earlier answer
was not exactly right. But that's fine.
MR. ROSEN: Okay. So you're solving the
PI with two problems you think by doing it this way?
MR. APOSTOLAKIS: Correct. Fixing a
terminology problem.
MR. ROSEN: So you discover a component --
it's easy. I mean, you have to do a monthly test or
something, you go do the test and it doesn't work,
doesn't start. Okay. That's a failure on demand.
And that's the only way you count that. You don't say,
mmm, we tested this last week and it worked, it passed
the test, so it can't have been out more than a week.
Now what did we do during this week? Oh, it must have
been when we repacked this valves yesterday afternoon.
MR. APOSTOLAKIS: Right.
MR. ROSEN: Now it turns out it doesn't
work because this valve doesn't stroke. We repacked
the valve. Oh, yes, we asked the guy who repacked
whether he knew how to do it. It turns out he was
unqualified.
We've got a lot of problems here, but we
think the time that that valve was made inoperatable,
when that step was taken, was really when that -- it
would have worked up until that time. So if you have
that, you can say fault exposure hours. In that case,
it's just a day.
All right. And then you add that into
unavailability. Now you're saying we're not going to
do that. We're just going to say, okay, it didn't
work, bang. We've got one failure on demand. Forget
all the unavailability, we're going to consider that
it's available throughout that whole time up until
this test. Is that right?
MR. BARANOWSKY: That's right.
MR. HAMZEHEE: That's correct.
MR. ROSEN: Don't you lose something, is
my point? Don't you lose some real unavailability by
doing that? Now before you might have been over
estimating -- you might have been overestimating
unavailability if you're using the T over 2 algorithm.
Might have been. You might also have been under-
estimating. That's why we use T over 2 because we
didn't when in some cases you would over estimate it,
some cases you under estimate; on the average you'd
hit it right on the nose.
MR. BARANOWSKY: No.
MR. ROSEN: Now this new way, you're
almost certainly going to estimate unavailability.
MR. BARANOWSKY: No, I disagree with you
on that.
MR. ROSEN: Almost there's no question
that you're going to under estimate unavailability.
Because the only case in which you are not going to
under estimate unavailability is the case where it
just failed. It would have worked a microsecond, an
epsilon in time before we did this test it would have
worked, but now it won't.
MR. BARANOWSKY: There's going to be some
cases where you over and under estimate it and the
assumption is that it's basically a constant failure
rate process. You'll under estimate by about a factor
of 2.
If you assume that the inspection occurs
at the end of the test interval. The inspection
occurs randomly within the interval, then the T over
2 approximation is correct.
What this solves, though, is the issue of
taking a potential single failure for something that
only has a limited number of demands in, say, a year
in which if you take only, say, one failure in one
year and a limited number of demands in one year; then
you know that for any given one year period of time
you're going to have things like perfectly reliable,
perfectly reliable, highly unreliable, then back to
perfectly reliable again.
The approach that Hossein is going to talk
about is how we are going to bring in Bayesian
statistics to account for this high increase or zero
situation which is basically a sparse statistics
issue.
MR. SIEBER: Do you take into account post
maintenance testing since maintenance generates a lot
of the future failures to start?
MR. HAMZEHEE: Yes. As part of the
unavailability.
MR. SIEBER: Right.
MR. HAMZEHEE: That's correct, yes. They
are all unaccounted for.
MR. APOSTOLAKIS: Well, we should wait and
see until --
MR. ROSEN: I'm still not convinced that
you're not going to lose unavailability. You're
measuring unavailability and unreliability separately
now, right?
MR. APOSTOLAKIS: That's correct.
MR. ROSEN: That's a wonderful, wonderful
thing to do. It's very good you moved in that
direction. But now that you've changed the algorithm
for how you account for the unreliability, you're
going to lose real unavailability hours. You're not
going to account for times that the machines were
really unavailable by doing it this way.
MR. THOMPSON: But there's another issue.
You don't want to double count. You don't want to take
a demand failure and take the unavailability. That's
like double counting.
MR. ROSEN: No, it isn't.
MR. APOSTOLAKIS: Let's raise the issue
when we see the actual --
MR. BARANOWSKY: I mean there's a definite
-- there's an equivalence with a constant failure rate
assumption between a failure on demand in the T over
2 situation. It's a very simple mathematical formula
and we're trying to account for it here.
MR. APOSTOLAKIS: And it depends very much
on what value for lambda you use.
MR. BARANOWSKY: It has to be a small
value of lambda and it has to be a constant failure
rate process.
MR. THOMPSON: Next slide.
MR. APOSTOLAKIS: Yes, let's move on.
MR. THOMPSON: As of today this is the
list of plants that have volunteered to participate in
the pilot. We've tried to --
MR. APOSTOLAKIS: Next time you should
have the plants that are not participating and show
them this time.
MR. THOMPSON: We tried to get a good
random representation, but it was voluntary so these
are the ones that we ended up with.
MR. APOSTOLAKIS: So what does 1/2 mean?
Both units one and two?
MR. THOMPSON: Yes. Units one and two.
The next slide shows the monitored systems
in the pilot. The first four are identical to the
systems currently measured by the SSUPI. The last
line of each of the Bs and Ps are the support system
that is the additional system that the PI is going to
monitor.
MR. APOSTOLAKIS: What are the support
systems?
MR. THOMPSON: Like under B column the
support system cooling, which is essential service
water plus the building closed cooling water and the
turbine building.
MR. APOSTOLAKIS: I don't see any of those
here. Is there?
MR. THOMPSON: The last line.
MR. APOSTOLAKIS: Oh, down at the bottom.
MR. HAMZEHEE: The very last line.
MR. THOMPSON: Any questions? Okay. Next
slide.
MR. APOSTOLAKIS: Actually, support system
cooling is wrong English, isn't it? You're not
cooling the support systems, are you?
MR. HAMZEHEE: That's true, we're not.
MR. THOMPSON: True. It's the support --
MR. APOSTOLAKIS: Well, like all the
component cooling water system, right?
MR. THOMPSON: The support cooling --
MR. ROSEN: But you have a very high
standard. You want the staff to speak English.
MR. BARANOWSKY: We just factor in here
English is our second language, you know.
MR. ROSEN: I didn't think we have any
such expectation.
MR. THOMPSON: The next slide is to give
you the definitions of unavailability and
unreliability as we're defining them for the pilot.
The MSPI unavailability is the sum of the
planned and unplanned maintenance reported by train,
corrective unavailability. It's not all
unavailability. I wanted to make that point.
MR. BARANOWSKY: We're going to show
equations for all this.
MR. APOSTOLAKIS: Wait a minute. This is
the MSPI train unavailability, correct?
MR. THOMPSON: That's correct.
MR. APOSTOLAKIS: Yes, with the word
"train" in front of unavailability?
MR. THOMPSON: Well, it says reported by
train
MR. APOSTOLAKIS: Well, it says MSPI --
MSPI train unavailability.
MR. HAMZEHEE: George, but I think if you
wait until we go through the algorithm, you realize
these are system --
MR. APOSTOLAKIS: I understand that.
He took away the microphone.
MR. THOMPSON: The next bullet is the
train unavailability is the ratio of hours that the
train was unavailable to perform its risk-significant
function or functions as defined in the PRA, the plant
specific PRA, due to planned or corrective maintenance
or testing during the previous 12 quarters with
reactor critical, by the way, which is the current
definition of the SSUPI to the ratio to the number of
critical hours that the train was required to be
available.
MR. ROSEN: Now wait a minute. Now you
told RHR was one of the MSPI indicators you're going
to look at.
MR. THOMPSON: Yes, it is.
MR. ROSEN: Well, how often is RHR needed
when the plant is critical?
MR. HAMZEHEE: The RHR for at power
functions, there are so many functions as you know for
RHR. What we're talking about here are those
functions that are required during at power mode.
MR. THOMPSON: Okay. Right.
MR. BARANOWSKY: And some plants use RHR
in power.
MR. HAMZEHEE: Well, for a mitigating
purpose, yes, under --
MR. ROSEN: No, it's not all plants, but
some do.
MR. HAMZEHEE: That's correct, yes
MR. APOSTOLAKIS: So there should have
been a third bullet there saying that the train
unavailability will be put together somehow to get the
MSPI unavailability. Okay. Okay.
MR. THOMPSON: Yes. For unreliability the
MSPI unreliability is a measure of the demand failure
probability.
MR. APOSTOLAKIS: So it's both. Good.
MR. THOMPSON: Yes. Of the monitored
system and the failure probability during a mission
time.
MR. APOSTOLAKIS: Yes. Okay.
MR. THOMPSON: And that the component
unreliability is the failure on demand probability
that the system would not perform its risk-significant
function when called upon during the previous 12
quarters.
Now all of this Hossein will go over in a
little more detail with his presentation.
Now our schedule, which has turned out to
be an ambitious one, at the end of next month we have
prepared a workshop for the pilot participants, both
for the licensees and for the inspectors that will
partake in the pilot, a three day workshop.
MR. APOSTOLAKIS: Right.
MR. THOMPSON: We will go over the details
and bring everybody up to speed.
August 1 is the start of the pilot and we
plan to come back to the ACRS and brief you on the
pilot progress in two or three months.
MR. APOSTOLAKIS: See, that's confusing to
me. Every bullet is a noun, public workshops, start
of MSPI pilot. So brief there I thought it referred to
a brief subcommittee. You mean to brief or briefing
the ACRS. It's not that you're declaring the
Subcommittee --
MR. ROSEN: We have to deal with this all
the time. You only have to deal with it during this
meeting.
MR. APOSTOLAKIS: So briefing the ACRS
Subcommittee.
MR. THOMPSON: It would require two lines
and two slides otherwise.
MR. APOSTOLAKIS: Okay.
MR. THOMPSON: In February the pilot ends
and we start the analysis period.
MR. APOSTOLAKIS: So what do you expect to
learn from the pilot?
MR. THOMPSON: That we solved the
problems.
MR. APOSTOLAKIS: Certainly those guys are
not going to tell you anything about the theoretical
basis of this. They will probably tell you that that
it was difficult to get the data you thought you were
going to get. I mean, have you thought about it?
MR. THOMPSON: Yes.
MR. APOSTOLAKIS: What is the value of the
pilot?
MR. HAMZEHEE: Yes, there are pilot
objectives. I think if you go over the objectives of
your pilot program --
MR. APOSTOLAKIS: There will be a slide on
that? Fine.
MR. BARANOWSKY: We gave those to you
earlier.
MR. APOSTOLAKIS: Oh, you did?
MR. THOMPSON: It was I think the third
slide.
MR. HAMZEHEE: There's a slide on the
objective of the pilot program.
MR. APOSTOLAKIS: The third slide.
MR. BARANOWSKY: I mean, a major thing is
this is a new and somewhat more complex way of doing
the calculation using a plant specific PRA, if you
will. And so we've got a lot of technicalities that
we have to go over.
MR. APOSTOLAKIS: Yes, these are not
really objectives I had in mind. I mean --
MR. BARANOWSKY: Well, we want to
determine how difficult this is to do in order to get
consistent results.
MR. APOSTOLAKIS: Right.
MR. BARANOWSKY: Because while the
industry is doing their thing --
MR. APOSTOLAKIS: It's the doing of it?
MR. BARANOWSKY: -- the staff is going to
be doing their thing as a check.
MR. APOSTOLAKIS: But let's not over
estimate the value of the pilots. They will tell you
how difficult it is.
MR. BARANOWSKY: Yes.
MR. APOSTOLAKIS: Or of it's impossible,
or whether it's impossible to do what you want to do.
But they will never tell you or question the
theoretical basis of what you're doing.
MR. BARANOWSKY: No. During the pilot
we're also going to do what we are calling "table top
studies," issues that are raised in this meeting or by
other stakeholders that are related to methodology.
And what would happen if you treated it this way or
that way. We're going to run a bunch of case studies
on them. But we want to get the industry into trying
to collect the date and exercise the basic method,
even though it might change a little bit. There's
some technicalities and what Hossein's going to show
you that we're going to have some questions on and we
may want to try other ways.
But we know that collecting the data and
getting everyone to compute in a consistent way the
same thing is a little bit of a problem.
MR. APOSTOLAKIS: Right. So what in July
of 2003?
MR. THOMPSON: We'll collect the results
and roll them up into a RIS and communicate that to
the members of the public.
MR. ROSEN: What's a RIS?
MR. APOSTOLAKIS: What's a RIS?
MR. THOMPSON: Regulatory information
summary. It's kind of like an information notice.
MR. SATORIUS: That's the vehicle that we
report to the public and our other stakeholders. And
it's also the pilot, whether we consider it a success,
whether it met the pre-agreed upon success criteria
and attributes.
MR. THOMPSON: Right.
CHAIRMAN BONACA: Now let me understand
now. You wrote, you know, a NUREG in which you had
Phase-1 development of the RBPIs and you're really
testing some of these RBPIs that you have in that
Phase-1 development, right?
MR. HAMZEHEE: Which were customized for
the ROP, yes.
CHAIRMAN BONACA: That's right.
MR. HAMZEHEE: That's correct.
CHAIRMAN BONACA: No decision has been
made on the part of NRR yet whether to use them in the
ROP or not?
MR. SATORIUS: That's correct. That's the
purpose of the pilot.
CHAIRMAN BONACA: Okay. The pilot really
is to either --
MR. SATORIUS: We'll test what we believe
will be a working system. And, as George had
mentioned, it will exercise the mechanics of reporting
the data for licensees to be able to go out and
collect the data.
CHAIRMAN BONACA: You're doing that?
MR. SATORIUS: And for us to check that
the algorithms are actually working and, as Pat had
mentioned, to do a table top exercises to validate or
to verify that it's doing what --
CHAIRMAN BONACA: With some sensitivity to
the changes you made. I mean, some of them may not be
significant.
MR. SATORIUS: That's correct, some of
them may not be.
CHAIRMAN BONACA: They may not be, you
know, worth the time that -- the collection on the
part of the licensees. Okay. I understand.
MR. SATORIUS: I think, Hossein, you're
ready to start on the technical discussion.
MR. APOSTOLAKIS: Do we want to take a
break?
CHAIRMAN BONACA: No. We have scheduled
a break at 2:30. Let's go on.
MR. HAMZEHEE: Okay. I am Hossein
Hamzehee in research NRC. And I think the purpose of
this presentation is mainly to provide the technical
aspects of this new approach of mitigating system
performance index. And as part of this presentation
I will first talk about the major insights from our
Phase-1 RBPI study and then I will go in a little more
detail into the technical aspects of the approach.
And then at the end we'll summarize the conclusion.
And just to make sure that I can finish
this, I'm going to try to be very focused so that I
can get to the meat of this presentation. And then
I'll be more than happy to answer any easy questions.
MR. APOSTOLAKIS: What about the difficult
ones?
MR. HAMZEHEE: Now first, let's talk about
the insights from the Phase-1 risked-based performance
indicator study.
As you may have all seen in the report,
this study really demonstrated that there are enough
planned risk-significant differences among different
plants in the industry that would make it necessary to
develop some kind of plan specific thresholds for
unavailability and unreliability performance
indicators. And the main reason, as we all know, is
because many of these plants even though they may be
Westinghouse, BNW, PWRs, they all have significant
design features and operating characteristics. And
during this Phase-1 study that was demonstrated.
And the way MSPI will work is this
algorithm will account for those plant specific
features.
And then we also found that this
unavailability and unreliability indicators that were
treated separately in the RBPI study were found to
provide objective and risk informed indication of
plant performance. Again, they were all mainly risk
informed. And they also provide broader risk
coverage, mainly because they had more systems and
they had unreliability in addition to unavailability.
And as I mentioned earlier, the approach
that was described in the Phase-1 report was tested by
evaluating plant specific data for 44 nuclear plants
over a three year time period, which was basically
from '97 to '99 and reused our available SPAR model.
And I'm assuming we all know what SPAR models are.
And we use EPICS for unreliability information and
ROPPI for unavailability information.
MR. WALLIS: I don't understand this. How
would you know if they were not objective? They have
to be risk informed because that's what they're based
upon, aren't they?
MR. HAMZEHEE: That's correct.
MR. WALLIS: Well, how would you know that
they were not risk informed? You're reaching a
conclusion that you discovered that they were risk
informed.
MR. HAMZEHEE: No. Because if you read the
report, at the beginning we set some criteria that
would be based on risk.
MR. WALLIS: Oh, so you have a criteria to
determine whether they were risk informed or not?
MR. HAMZEHEE: Exactly. We defined up
front before we started the study.
MR. WALLIS: Okay.
MR. HAMZEHEE: We said all right how can
we develop objective risk informed indicators. So we
went ahead and defined the criteria.
MR. WALLIS: Okay.
MR. HAMZEHEE: And then we developed the
approach and we would go back and look at those
criteria to make sure that --
MR. WALLIS: So you found that they did
what they were intended to do?
MR. HAMZEHEE: That's exactly correct.
MR. WALLIS: Okay. Good.
MR. HAMZEHEE: Yes, sir.
MR. WALLIS: Thank you.
MR. HAMZEHEE: We also realized as part of
this that support systems are very important. And we
looked at the significance of those support systems.
And CCW and service water system or their equivalent
were found to be some of the most risk significant
support systems. But we also realized that they were
difficult to develop PIs for mainly because of the
variation of designs among the industry. And they
were so plant specific that it would not be easy to
develop PI generically. So with that in mind, when we
started this pilot program as part of preparation, we
worked with the industry and we have come up with some
approach that would be used to developed performance
indicators for those two support systems or their
equivalent.
And the last bullet talks about the fact
that in order to have a good estimation of component
unreliability we used Bayesian update approach. And it
was found to minimize the likelihood of false-positive
and false-negative indications. And as you may all
know, because of the monitoring period it is very
difficult based on statistics, scarcity of data, the
nature of these PIs to develop a PI in a time frame
that could give you 100 percent accuracy. So you
always have to deal with some false-positive and
false-negative probabilities.
MR. APOSTOLAKIS: I don't understand that.
What's a false-negative?
MR. HAMZEHEE: False-negative, I have one
slide on this one. But false-negative means if you
shows a performance indicator that would indicate the
performance is green when in reality it's non-green.
It's either yellow or red, or what; that's called
false-negative. And false-positive means if your
performance indicator indicates red, yellow or white
where in reality it's at the baseline or green
performance.
MR. WALLIS: How do you know what the
reality is? You're measuring something other than the
performance indicator which is more real?
MR. HAMZEHEE: Well, there are two parts.
One is based on -- for instance, you look at the
statistics of the information because you're always
dealing with numbers and then probabilities. And you
estimate. And then you go back and see based on this
estimation. And then availability among these
estimations. What is the likelihood that you are
within your 90 percentile, 95 percentile of actual
performance and what's the probability that if your
performance is green, you're going to demonstrate non-
green performance.
So it's basically looking at data and
statistics.
MR. WALLIS: I thought green was defined
by the output from the performance indicator and there
was no other measure of it to compare it with.
MR. HAMZEHEE: That's correct
MR. WALLIS: Absolute measure --
MR. BARANOWSKY: Remember, the performance
indicators, the current ones and even these are going
to basically have a mean value. And the mean value
allows for some probability, of course, that you're
over or under predicting what the performance is.
And so he took the statistical
characteristics associated with the probability
distributions as to whether or not they were --
MR. WALLIS: I thought that whatever came
out of the process was the measure of meanness.
MR. BARANOWSKY: No.
MR. WALLIS: And there's no other measure
of that.
MR. BARANOWSKY: No. It's like you said,
if you're coming up X number of things and Y is your
threshold, you just ask if you're going over that
threshold.
Here what we're saying is instead of
counting X number of things, we're computing
perimeters and we're having an uncertainty on those
perimeters and that uncertainty expressed in terms of
a distribution was used to derive the probability that
we had a false-positive or a false-negative outcome.
MR. WALLIS: That's correct, yes.
MR. BARANOWSKY: And then we varied a
number of things associated with the distribution in
some assumptions, the time period that we looked at to
come up with the smallest possible range of both
false-positive and false-negative in which we balance
them out.
If you just use the mean, you know like
about half the time you're going to be over or under
estimating whether or not you passed that threshold.
MR. APOSTOLAKIS: Another way of putting
it, if I see a failure, is that a random failure or is
it really a real thing that shows a trend?
MR. HAMZEHEE: That's exactly right, yes.
MR. APOSTOLAKIS: I mean, I may be wrong.
They don't have another indication.
MR. WALLIS: I still don't understand.
MR. HAMZEHEE: And we have a section in
the appendix in RBPI if you would like later on, look
at it and let us know. We'll be --
MR. WALLIS: I guess what you're saying is
if you used a more sophisticated measure which
included uncertainty, then you might reach a different
conclusion. But if you've already chosen to use the
mean as your measure, then that's it.
MR. BARANOWSKY: That's correct. Suppose
we want to have 95 percent confidence, that would be
different than the mean, obviously, different point in
the distribution.
MR. APOSTOLAKIS: If I flip a coin ten
times and I get ten heads, one logical conclusion
would be that it's not a fair point. But I may be
wrong. It may still be fair and I just witnessed a
rare event. All ten trials resulted in heads. That's
what they're addressing.
MR. WALLIS: I don't know. It seems to me
if you have three strikes, you're out, and that's it.
And you define the rules --
MR. APOSTOLAKIS: If I see ten heads in
ten tries, I can either conclude it was not a fair
coin, it was biased toward heads, or it was fair but
I witnessed something that's extremely rare. Because
it's allowed in a fair coin to have ten heads in ten
trials. But the probability is so low so that
essentially that's the problem they're addressing.
MR. HAMZEHEE: That's correct.
MR. APOSTOLAKIS: It's not that they have
a different piece of information because the real coin
will do this. Because that's all they have.
MR. BARANOWSKY: This is a big issue with
the industry. They don't want to have the chance of
false-positives.
MR. APOSTOLAKIS: This is standard in
quality control. I mean, what I see; is that a random
weird occurrence or is it something that shows a
trend? For example, coming from an earlier
discussion, is Bayesian an aleatory thing that is
unusual or rare or does it show a trend of some sort.
CHAIRMAN BONACA: Well, after we have
three or four of those, we will make a decision.
MR. APOSTOLAKIS: Okay. Hossein?
MR. HAMZEHEE: And again, another
challenging part that would add to this is the
monitoring period. Of course, if you have 20 years of
monitoring period, then these indications become more
and more accurate. But when you narrow down the
monitoring period, then you need to understand what
are the probability of these false indications.
MR. ROSEN: Move faster.
MR. HAMZEHEE: You want to go faster? I
can. All right.
Now let's just put --
MR. ROSEN: It's not your fault you're
going slowly, it's George Apostolakis' fault.
MR. HAMZEHEE: Thank you, Steve.
Now quickly let's go over some concepts
before we get into the equations so you're all
familiar with what we're talking about.
Again, the mitigating system performance
index monitors the risk impact of changes in
performance of selected mitigating systems. And this
impact is based on change in core damage frequency.
And then as was already mentioned earlier,
the MSPI includes Level-1, internal events for at-
power mode. And this is consistent with the current
ROP performance indicators.
And again, as was mentioned earlier, the
MSPI for a given system consists of two elements,
unreliability and unavailability. And the MSPI is the
sum of the changes in a simplified CDF evaluation that
shortly I'll show you how from changes in the system
unavailability and system unreliability relative to a
baseline values. And again soon I'll tell you what
those baseline values are.
MR. ROSEN: Now it's my fault.
MR. HAMZEHEE: That's okay, please.
MR. ROSEN: Why do you sum them, a simple
sum? Is that mathematically correct?
MR. HAMZEHEE: Yes. Because in a right
format, yes. Because in reality a piece of equipment
could not -- could be unable to perform its function
either due to unavailability. It's unavailable
because they're doing some maintenance activities on
it or because they tried to start it and it failed to
start. So either one of those is going to add to the
probability that that piece of equipment is unable to
perform its function. So you have to --
MR. APOSTOLAKIS: Well, in theory you
should subtract the probability --
MR. HAMZEHEE: That's correct.
MR. BARANOWSKY: And also we've done some
checking. It's linear approximations that we're
making in order to make this a fairly simple equation
to work with. And we're testing it with full blown
models to see how much potential error is being
introduced.
MR. WALLIS: Well, you might get numbers
bigger than one.
MR. BARANOWSKY: I doubt it.
MR. HAMZEHEE: You're going to be very,
very surprised if you get anything even close to one.
MR. APOSTOLAKIS: The only place where you
have to do this is in seismic analysis where the
probabilities of failure are fairly high. So you have
to --
MR. HAMZEHEE: Conditional probability
given you have a seismic.
MR. APOSTOLAKIS: You have to subtract the
probable. But here it's -- go ahead.
MR. HAMZEHEE: Well, you're my teacher.
MR. APOSTOLAKIS: Go ahead.
MR. HAMZEHEE: All right. The next one,
the other concepts, again, the risk impact of --
MR. APOSTOLAKIS: Where's the equation.
I'm looking --
MR. HAMZEHEE: The risk impact of these
changes on plant performance are estimated using
plant-specific performance data and a Fussell-Vesely
importance measure. And I assume you know what
Fussell-Vesely importance measure is. If not, we'll
go over it.
And again, I think this is very important
to realize that those aspects of the MSPI that are not
-- that those aspects of safety performance that are
not covered by MSPI will be evaluated through our
normal inspection and significant examination process.
Because this MSPIs don't cover all performance areas.
And some example are, for instance, common
cause failures, concurrent failures of more than one
component in a system, passive components that are --
MR. APOSTOLAKIS: Passive components?
MR. HAMZEHEE: Passive components --
MR. ROSEN: We had a discussion this
morning that BRA doesn't cover passive components.
MR. HAMZEHEE: Next time, yes. And then
the other -- and these are the typical things that are
not included in the MSPI and it will be covered by a
SDP and inspection.
MR. WALLIS: There's also passive
management.
MR. APOSTOLAKIS: Yes, I think cultural
issues are not covered. And Fussell I think is double
L.
MR. HAMZEHEE: That's correct, yes.
Now the next one is the scope of MSPI. Let
me quickly go over the scope.
For unreliability calculations
calculations only active components within a system
are included in the performance indicators. And we
all know what active components are. A good example
is a normally closed valve that has to open on demand
to allow flow through a system. We call that an active
component.
And all pumps and diesels in the monitored
systems are considered as active components even if
they're normally running because of their
significance.
Active failures of check valves are not
included in the MSPIs and they'll be covered under
inspections and SDP.
MR. APOSTOLAKIS: Why is that? Why aren't
they included?
MR. HAMZEHEE: Check valves?
MR. APOSTOLAKIS: Yes.
MR. HAMZEHEE: Well, because --
MR. APOSTOLAKIS: If the valve is close,
is suppose to open?
MR. HAMZEHEE: Yes.
MR. APOSTOLAKIS: Then why not?
MR. HAMZEHEE: There are a few reasons.
That was one of the issues that in the last year we
reviewed and analyzed and discussed with the industry
and the conclusion was not to include them; (1) is
because the risk-significant failure of the check
valves are not failure to open, but rather a failure
to prevent reverse leakage from high pressure to low
pressure systems.
MR. APOSTOLAKIS: Oh. So essentially what
you're saying it's a passive system?
MR. HAMZEHEE: Exactly. And it doesn't
happen often. But if it does, then the consequence
could be severe and we evaluate it through SDP
inspection. So that's important because --
MR. APOSTOLAKIS: Active failures. No,
this is --
MR. HAMZEHEE: If it has to open on
demand, this is active failure mode. Because in a --
MR. APOSTOLAKIS: No, no, no, no. If it
has to open and it doesn't open, that's an active
failure and that should be included.
MR. HAMZEHEE: That's correct. And what I
am saying --
MR. APOSTOLAKIS: But if it fails to
remain closed --
MR. HAMZEHEE: That's correct.
MR. APOSTOLAKIS: -- then it's passive.
MR. HAMZEHEE: That's correct. Exactly.
MR. APOSTOLAKIS: You shouldn't be using
the word active?
MR. HAMZEHEE: No. What this bullet is
saying is that the active failure mode of a check
valve, which is failure to open on demand, is not
included in the MSPI intentionally.
MR. APOSTOLAKIS: But it should be
included.
MR. HAMZEHEE: I understand. You know, I
understand. But as I said, the decision was not to
include that because the failure probability of a
check valve to open on demand is very low. So we said
since it does not happen often and it's not risk-
significant, we included it in the inspection and SDP
so it's not forgotten. It's treated in a different
place.
MR. BARANOWSKY: Let me also point out
that the likelihood of seeing one of these valves fail
to open is small and that's why we said we would only
rarely have to look at it. So we would use a
different tool instead of collecting data, data, data
that we're never going to get any pay off from.
Now we're going to study this a little bit
further during the pilot, okay. But the idea is to
keep the number of components that we have to collect
data to a manageable set. And you're going to see
that we're going to have to collect really a lot of
information, and so that's what the decision was here.
These are the things that show up rarely.
And if they show up rarely, let's treat them with some
tool that's good for treating rare events.
MR. APOSTOLAKIS: But the baseline
inspection program will inspect these check valves how
often?
MR. SATORIUS: Typically these would be as
a result of a failure to open and there would be an
event. So there would be event follow up and then --
MR. APOSTOLAKIS: Oh. So it's not
something that will be done routinely?
MR. SATORIUS: Not necessarily. There are
-- back there to help me here, but there are specific
inspections that we do where we can select
surveillances that are either performed or maintenance
activities. So that there is an opportunity within
the inspection program to routinely take a look at
these components. But typically we see a failure here
and we react to that using an event follow up type of
an inspection.
MR. APOSTOLAKIS: Because if it's done
routinely, I mean essentially what you're saying is
the probability of failure is so low that we'll
inspect it every time. Now come on. The inspection
is not risk informed. But if what Mark said is what's
happening, then it's okay.
MR. SIEBER: Generally when a check valve
fails to open, and in 40 years I've never seen that,
that what it would do would be reduce the performance
of the system. But where you do see check valve
failures is in failures to close. They may come off
the pins or get caught in there.
MR. HAMZEHEE: That's correct. That's
exactly --
MR. SIEBER: And generally -- and that is
tested in surveillance tests.
MR. HAMZEHEE: That's correct.
MR. SIEBER: It's the Appendix V tests and
so forth.
MR. HAMZEHEE: It's also covered under the
maintenance rule.
MR. SIEBER: That's right. And it's not
a mitigating system function either.
MR. APOSTOLAKIS: This is philosophical.
If something is so rare that it can't be in the PI,
then it would be logically inconsistent to say I will
move it to the inspection program and inspect it every
time.
MR. SATORIUS: We're not saying that,
George.
MR. HAMZEHEE: We're not saying that.
We're saying it's fairly impassive --
MR. APOSTOLAKIS: Somehow the inspection
has to be risk informed, too.
MR. SIEBER: If it fails to open, you're
going to get a system failure.
MR. HAMZEHEE: That's correct.
MR. SATORIUS: Exactly, and then you would
have an event and you would do event follow up and use
the inspection program.
MR. SIEBER: And that would be
unreliability instance, too, that would count against
a system.
MR. HAMZEHEE: That's correct, yes.
MR. APOSTOLAKIS: So what does the last
bullet mean?
MR. HAMZEHEE: And the last bullet is --
let me just give you an example. For instance, let's
say you have a high pressure safety injection system
in a PWR which has two trains, train A and train B.
Sometimes in train A it had more than one flow path
from discharge of a pump. So you may have two fully
redundant parallel flow path from discharge of train
A.
We're saying that those two valves, even
if they're active components because of the failure of
both valves at the same time has a very low
probability. We're not going to include it in the
MSPI, but rather we do exactly what we said like the
check valve.
MR. APOSTOLAKIS: Which brings up another
issue. What does it mean to include it? I mean,
you're looking at the whole train, aren't you?
MR. HAMZEHEE: No, and we show you what we
-- we were going to tell you soon what is the scope of
a system.
MR. APOSTOLAKIS: Oh.
MR. HAMZEHEE: Because usually what we are
going to do is we are going to look at a full system
and within that system we are going to highlight the
active components. And based on the preliminary
studies that we have done, a typical system train has
a pump and between one to three valves. That's the
scope of a given train.
MR. APOSTOLAKIS: Okay.
MR. HAMZEHEE: And then it's talking about
component boundaries that are consistent with PRAs and
then the SDP will be used for the performance areas
that are also MSPI. We already talked about those
areas.
Now, should I get into equations, Mario,
or you want to take a break?
CHAIRMAN BONACA: Is it a good time,
because you have still quite a bit of --
MR. HAMZEHEE: That's correct.
CHAIRMAN BONACA: -- material to go
through, right?
So let's take a break now. And let's get-
-
MR. SIEBER: This requires more thinking.
CHAIRMAN BONACA: So that in the meantime,
George can review all this upcoming algorithms. Let's
get back here at 20 minutes of 3:00.
(Whereupon, at 2:23 p.m. off the record
until 2:39 p.m.)
MR. HAMZEHEE: Should I go ahead?
CHAIRMAN BONACA: Yes, please.
MR. HAMZEHEE: All right. So the next one
is talking about equations. And the equation number
one mainly says that the mitigating system performance
index for a system is the summation of unavailability
index and unreliability index.
MR. APOSTOLAKIS: I don't understand why
it says changes in train unavailability. You're not
calculating changes, you're calculating the actual
unavailability, aren't you?
MR. HAMZEHEE: Yes, but it says system
unavailability index due to changes in train
unavailability.
MR. APOSTOLAKIS: Yes.
MR. HAMZEHEE: In other words, you're
going to have a baseline unavailability and then based
on that baseline you measure the changes.
MR. APOSTOLAKIS: Why? How.
MR. HAMZEHEE: Will you let me go through
the equations, George, at the end of it if you don't
understand it, then I'll try to explain it.
MR. APOSTOLAKIS: All right.
MR. KRESS: Equation three is a fractional
change in unavailability.
MR. HAMZEHEE: If you guys don't mind, let
me just quick run through the equations quickly and
then I'll be more than happy to stop.
So it's UAI, which is the system
unavailability index due to changes in train
unavailability and URI system unreliability index due
to changes in component unreliability. And you see
one is train, one is component and we'll explain why.
Now, let's see how we find UAI. UAI is
the summation of the UAIT, which UAIT is
unavailability index for train T and the summation is
over the number of trains in the system. So for the
two train system, it's one T, 1 plus T2.
I'm sorry, Steve.
MR. ROSEN: I said it's victory.
MR. HAMZEHEE: And then UAI sub t is equal
to DCFp and equation. Fussell-Vesely, sub UAp divided
by USRP; the whole thing multiplied by that USRp minus
URBLT. And let me explain.
MR. KRESS: That's essentially a plant
specific change in CDF?
MR. HAMZEHEE: Exactly. You got it.
That's exactly what it is.
And let me quickly go over the terms and
then I explain a little more.
CDFp is plant specific internal events at
power core damage frequencies that is going to be
obtained from licensees.
Fussell-Vesely --
MR. APOSTOLAKIS: The mean value, right?
MR. HAMZEHEE: Yes. Right now we're merely
talking about mean values.
And FVUSRP is the train specific Fussell-
Vesely value for unavailability based on plant
specific PRA. So if you have four different planned
PRAs with four different design characteristics, most
likely you're going to have four different Fussell-
Vesely. And each represents a different plant.
And then USRP is the value of
unavailability for train 2 T from plant specific PRAs.
In other words, you're going to go to that system and
ask licensee what they have used for that train in
their PRA models for that system. And that's what we
use. And then later one we talk about how NRC is
going to validate or confirm, make sure those are the
right numbers.
MR. KRESS: Then the question I might have
there is that may be what they used in their PRA, but
what was their normal value? Do they have a standard
value that might be different from what was used?
MR. HAMZEHEE: Yes. I'm going to talk
about them. Yes. You're right. There's going to be
some variations, some differences. We'll talk about
it.
MR. ROSEN: When you say you're going to
go ask them what they used, do you mean in their last
model update?
MR. HAMZEHEE: In their most recent
updated PRA, yes.
MR. ROSEN: Because it changes over time.
MR. HAMZEHEE: That's correct.
MR. KRESS: But you used plant specific
data?
MR. HAMZEHEE: That's correct.
MR. ROSEN: Because we're basing an update
here.
MR. HAMZEHEE: That's exactly right. But
then we also have to define some frequency at which
you can do those. Otherwise, you don't want to change
these everyday, but you're right.
MR. ROSEN: No. But a update frequency,
it's a function of -- it's in our license.
MR. HAMZEHEE: That's right.
MR. ROSEN: Because at South Texas it's
part of the exemption.
MR. HAMZEHEE: That's right.
MR. ROSEN: So now what you're saying is
we will be using that number during a cycle? Okay.
Well, that'll work.
MR. HAMZEHEE: And right now I think I'm
going to focus on how technically we're going to
calculate these. But then later on as part of pilot
with NRR we have to define how often and why, and how
we're going to --
MR. ROSEN: That's right. South Texas is
in the pilot, so you'll get some feedback on how that
works in a plant with an exemption.
MR. HAMZEHEE: Exactly.
MR. SATORIUS: But we understand that
there are reasons for licensee to change their PRA as
they may modify the plant, as they make changes to the
plant. It's a reasonable thing for them to do. And
we're mindful of that.
MR. ROSEN: Change the model as well as
change the data.
MR. HAMZEHEE: Exactly.
MR. KRESS: Now your summation, is that
the summation over the four trains --
MR. HAMZEHEE: If you have a South Texas
three train system, then that's train A plus B, plus
C. For Comanche two train system, it's only two of
them.
MR. KRESS: Okay. But you don't sum it
over systems?
MR. HAMZEHEE: No. This is a system. This
is MSPI, which is mitigating system performance index
for a given system.
MR. KRESS: A specific system?
MR. HAMZEHEE: In other words, we're going
to have one for HPI, one for 00 feed water, one for
RHR.
MR. KRESS: And the way you'd summate
those would be in your --
MR. HAMZEHEE: Within the system.
MR. KRESS: -- multiple performance
indicators in your matrix?
MR. HAMZEHEE: That's correct, yes.
MR. KRESS: You'd have a sort of a
summation?
MR. ROSEN: But RHR is only for RHR
systems that are used for an at power function?
MR. HAMZEHEE: Correct.
MR. ROSEN: So plants that use their RHR
only for shut down, only in shut down modes --
MR. HAMZEHEE: Those functions are not.
MR. ROSEN: -- they will not have an MSPI
for shut down or for RHR?
MR. SATORIUS: It should be the same way
it is right now, Steve. Because we're making no
change to RHR and how it's viewed between the way that
unavailability is measured today and the way
unavailability will be measured under the pilot.
MR. ROSEN: I thought you said RHR was in
the pilot?
MR. SATORIUS: It's in this pilot. It's
also in the PIs that we're gathering unavailability
data for today.
MR. ROSEN: All right. Then why are we
measuring RHR?
MR. HAMZEHEE: Tom, would you like to ask
something?
MR. ROSEN: Is there some RHR function at
power?
MR. BARANOWSKY: No. Are we talking about
low pressure safety injection?
MR. HAMZEHEE: I think Tom Houghton is
here from NEI.
MR. HOUGHTON: Let me try to help. The
two functions of RHR that we're thinking about are the
accident mitigation function, okay, which we would
include for RHR while at power because it's needed
immediately upon initiation of the accident.
MR. ROSEN: That's the low pressure
coolant injection mode of RHR.
MR. HOUGHTON: Right. That's exactly
right.
And the shut down cooling mode we would
not include as a function of RHR.
MR. ROSEN: Now I'm trying to remember in
the South Texas design the low pressure coolant
injection. South Texas has a separate and completely
independent low pressure coolant injection system.
MR. SATORIUS: Steve, I would bet you, and
I don't know for sure, I can find out that South Texas
is providing that low pressure safety injection system
of unavailability today. Because that function, that
system provides that high volume, low pressure --
MR. ROSEN: And labeled paren RHR which
it's not?
MR. SATORIUS: Right.
MR. ROSEN: Because the RHR system at
South Texas is a very different plant, remember, but
it's inside containment -- totally motor's inside
containment, it's used only for shutdown. So, you
know, what we have is separate LPCI.
MR. SATORIUS: Yes, you do.
MR. ROSEN: Which functions in the mode
that other plants or two loop plants typically use
their RHR for. So I think your point, Mark, is that
South Texas is reporting performance of LPCI in lieu
of RHR?
MR. HOUGHTON: Yes.
MR. ROSEN: Because functionally they're
equivalent.
MR. HAMZEHEE: I'm confident that that's
what they're reporting. I can double check that, and
will.
MR. ROSEN: I think that's the right
answer.
MR. APOSTOLAKIS: Can we go back to the
equations so I can understand it.
MR. WALLIS: Could we go back to the
equations and explain to someone stupid like me what's
going on here.
MR. HAMZEHEE: Yes.
MR. WALLIS: What you seem to be doing is
trying to get a measure of of the effect on CDF of
system unavailability and unreliability.
MR. HAMZEHEE: That's exactly right, yes.
MR. WALLIS: And you never said that. And
so the units of this MSPIR are delta CDF?
MR. HAMZEHEE: Correct.
MR. WALLIS: And the rest of it I can sort
of believe what you're doing. Why don't you just
calculate all the CDF directly?
MR. HAMZEHEE: Well, that's what it is.
MR. WALLIS: This is a very round about
way of doing it.
MR. BARANOWSKY: No, no, no. That's not
answering your question. Your question is answered by
this: The full blown, you know, fault tree models all
linked together and everything are going to be fairly
large. This is just a couple of simple --
MR. WALLIS: Well, doesn't Fussell-Vesely
do that for you?
MR. BARANOWSKY: Well, the Fussell-Vesely
incorporates all that modeling in a simple perimeter
so we can just work with some simple --
MR. KRESS: It's almost a precalculation--
MR. BARANOWSKY: Right. So do the
calculation and then only when you change your model
do you go back and mess with this.
MR. WALLIS: Yes.
MR. HAMZEHEE: Now let me then go over the
rest of them. I think at the end it may be more
clear.
The USRP then is the actual unavailability
of train T during the previous 12 quarters. And that
is what we're going to measure as part of this MSPI
for that system.
MR. APOSTOLAKIS: So let's see now, if I
multiple CDFp time the parenthesis, what do I get?
MR. HAMZEHEE: Well, if you get CDF times
Fussell-Vesely, then Fussell-Vesely tells you that
that multiplication gives you the change in CDF do to
that system. And then you multiply that by the --
MR. APOSTOLAKIS: No, no. Fussell-Vesely
is the ratio of the minimal that contain that train,
right?
MR. HAMZEHEE: Yes, it is --
MR. APOSTOLAKIS: Divided by the CDF.
MR. HAMZEHEE: Exactly.
MR. APOSTOLAKIS: So you multiply it by
the CDF and you --
MR. HAMZEHEE: So you get the ultimate
CDF.
MR. BARANOWSKY: No, that's like saying
what fraction of the CDF was due to this.
MR. APOSTOLAKIS: Right. Due to this --
MR. BARANOWSKY: Right.
MR. APOSTOLAKIS: So now I have a fraction
of the CDF that's due to this train. Not the faction,
I don't have the fraction -- I have the --
MR. BARANOWSKY: A portion of the CDF.
MR. APOSTOLAKIS: It's not even a portion.
MR. HAMZEHEE: No, no, no. You are taking
CDF, you multiply it by Fussell-Vesely. Fussell-
Vesely is the change in CDF. You're saying that
what's the CDF if a piece of equipment or that system
is perfect minus the CDF for the base case.
Oh, I'm sorry, base case minus if that
system is perfect. So you get a delta CDF that you
divided by the base case CDF. That's the fraction --
the condition of Fussell-Vesely. So when you divide
it by CDF, up front you multiplied it by CDF. So the
result of those first two terms are the change in CDF.
That if --
MR. APOSTOLAKIS: If the system is always
good.
MR. HAMZEHEE: It's perfect, yes.
Now, this is means -- the maximum
contribution that a system can have on CDF.
MR. APOSTOLAKIS: Right.
MR. HAMZEHEE: Right. Okay. Now hold
that portion.
MR. APOSTOLAKIS: Divide by UAB.
MR. HAMZEHEE: Yes.
MR. APOSTOLAKIS: And what do I get now?
MR. HAMZEHEE: Now, hold that portion now.
The second term is the USRT minus USRBBLT divided USRP.
MR. APOSTOLAKIS: But before I go, are the
UAs that be divided --
MR. HAMZEHEE: Exactly, it doesn't matter.
Yes.
MR. SHACK: Well, he'd be better off to
write it that way.
MR. APOSTOLAKIS: Yes, you'd better write
it that way.
MR. KRESS: He just have to --
MR. SHACK: You see what I'm saying?
MR. HAMZEHEE: Yes. But it was just
easier when I was using WordPerfect.
MR. KRESS: Yes, that's right. WordPerfect
don't let you do it.
MR. APOSTOLAKIS: So logically then the
UAB should be dividing the last parenthesis?
MR. HAMZEHEE: That's correct. So have
you a fraction dimension that's times change in CDF
which is going to give you change in CDF --
MR. APOSTOLAKIS: Now what was the last
parenthesis dividing UAB, what does that --
MR. HAMZEHEE: Okay. Now let's go over
that. The first term USRT is the actual
unavailability of train T during the previous 12
quarters. That is the actual measure of
unavailability as John defined what those things are
for planned and unplanned unavailability minus the
USRBLT which is the baseline unavailability value for
train T. And in a couple of pages I explain what the
baseline unavailability is.
Quickly just to make sure you understand,
the baseline is something that is based on the
industry average over some period of time.
MR. APOSTOLAKIS: Is that the one that was
used in calculating in CDF?
MR. HAMZEHEE: No. Not the baseline.
MR. KRESS: No. That's just sort of an
industry -- that's the old industry average that you
want to see did not depart too far from.
MR. HAMZEHEE: That's exactly right.
MR. APOSTOLAKIS: And why do I bring the
industry average --
MR. HAMZEHEE: Because now if you look at
the USRP, that's the unavailability of the train based
on plant specific PRA. So that term with Fussell-
Vesely plant specific PRA are going to be in terms of
the same perimeters.
MR. APOSTOLAKIS: Wait a minute. The CDF
that you have there, CDFp.
MR. HAMZEHEE: Yes.
MR. APOSTOLAKIS: Is the plant specific
internally events PSA?
MR. HAMZEHEE: Correct.
MR. APOSTOLAKIS: And then you bring the
industry average there for some reason in the last
parenthesis.
MR. KRESS: It's like your quality control
concept.
MR. WALLIS: Yes, I don't understand that.
MR. HAMZEHEE: Well, yes. Because what
we're trying to do here is that we have the plant
specific CDF, plant specific Fussell-Vesely, plant
specific USRP which is going to be under dominator.
And then you look at the delta.
Now the delta is the actual performance
minus something that is a baseline performance. You
have to have some baseline --
MR. SHACK: Why did you choose UAp instead
of --
MR. APOSTOLAKIS: That's right.
MR. HAMZEHEE: We tried that and I'll give
you the answer. We've done a lot of work. We didn't
just use this equation.
The reason for that is when we use UAp for
the minus, then the results when you compare it to the
actual using the full scope PRA don't compare. You get
a much closer approximation when you use this
equation.
MR. APOSTOLAKIS: Well, what do you mean
that they don't compare? I don't understand that.
MR. HAMZEHEE: In other words, if you
use--
MR. BARANOWSKY: No. I know what he's
saying. He's saying remember this is an
approximation. And first of all what's not clear I
think to this Committee is why we're using a baseline
to start off with.
The baseline is the same situation that
was used when the original performance indicators were
instituted. We have to identify what is an acceptable
level of performance to measure departures from.
MR. HAMZEHEE: Right.
MR. BARANOWSKY: So the baseline is
measured against what the performance was in the '95
to '97 time frame consistent with what was known at
the current PIs as documented in 99-007 SECY. So we
didn't want to deviate from that philosophy. So we
have a baseline of if you're at this baseline, you're
okay.
Now the reason for using the Ap in the
denominator is since this is a linear approximation,
we wanted to agree as best as possible over the
realistic range with the full model. We ran the full
model and we found out that when we used the value
that was in the PRA for the unavailability in this
equation, it gave us a better agreement for that
linear approximation over the range of values that
were realistic to be expected in terms of deviations
from the baseline.
MR. WALLIS: I thought the whole idea was
to be plant specific.
MR. BARANOWSKY: It is plant specific.
MR. WALLIS: Now you're bringing in a
baseline which is an average of all the other plants
and it's not plant specific.
MR. KRESS: That's the acceptance
criteria.
MR. BARANOWSKY: The acceptance criteria
is that -- I mean, it would be very hard for us to
pick a baseline -- I mean, that's the right to do it
is to pick a baseline for every single plant that
matches up with where they were in '95 to '97. But
going back and getting that information seemed to be
outside of what we were able to do. And this is
practical and I think people are satisfied.
MR. KRESS: Yes, George, this is not your
concept of plant specific acceptance criteria at all.
MR. APOSTOLAKIS: No, wait, wait, wait.
Well, I could see something like this: I have an
industry wide average which is a baseline and then I
have this plant. For this plant I have a plant
specific CDF, right? I can work backwards now and say
this plant deviates from the industry average by this
much; it's higher, say, by this much. And then I will
use that as part of my acceptance criteria for the
actual deviation. But I would not use the deviation
in equation three, the actual performance.
In other words, derive the acceptance
criterion first and then you look at the actual
performance and somehow you adjust it to the plant
specific situation.
Let's take the situation where this
particular plant is extremely redundant, okay? So the
UAt calculated using the baseline and the nominal that
they use at the plant gives me --
MR. SHACK: The UAt is real.
MR. APOSTOLAKIS: Yes.
MR. SHACK: The UAt is real.
MR. APOSTOLAKIS: UAt is real, but I would
not be using UAt in equation three.
MR. SHACK: I sure will.
MR. APOSTOLAKIS: No, no, no, no. One way
of developing the acceptance criteria, if you want to
compare with the baseline, is to say because this
plant is so redundant, I will allow the actual
performance to deviate more from the nominal
performance. Because I'm already low. But that's not
what they're doing unless it's built into it, and I
don't see.
MR. SHACK: I think it is. It's saying
that they're measuring their performance versus the
industry performance and this guy's extremely
redundant. He can let his unavailabilities go up and
he doesn't get penalized as much.
MR. BARANOWSKY: That's correct.
MR. SHACK: So that if he's worse than
average, he's going to have to --
MR. APOSTOLAKIS: Well, if that idea is
built into this, and it's not clear to me right now --
MR. KRESS: It seems possible to me that
UAt minus UABLT could be a negative number.
MR. BARANOWSKY: That's right. Yes, it
can. That in essence says the risk associated with
unavailability for the time period of interest is
declined. That's possible. Performance improved, if
you will.
MR. KRESS: No, I maintain that --
MR. APOSTOLAKIS: No, the performance is
not improved.
MR. KRESS: I maintain the UAt could be a
decrease in performance and still have a negative
number there.
MR. BARANOWSKY: A decrease in
performance.
MR. KRESS: For that plant.
MR. SHACK: If he's good enough and he
could even decrease his performance if he's good
enough.
MR. APOSTOLAKIS: There are two points of
reference, Pat. One is the industry average and one
is the nominal unavailability of the train of this
plant.
MR. BARANOWSKY: That's the p value.
MR. APOSTOLAKIS: Yes, let's not go there
now.
MR. BARANOWSKY: Right.
MR. APOSTOLAKIS: You can be measuring
deterioration with respect to the industry average.
MR. BARANOWSKY: That's right.
MR. APOSTOLAKIS: Or with respect to the
nominal of the plant.
MR. BARANOWSKY: Oh, that's true. That's
true.
MR. APOSTOLAKIS: Now, what is three
doing? Compared to what?
MR. BARANOWSKY: Okay. Here's the other
reason for doing it this way.
MR. APOSTOLAKIS: I may be deteriorating
with respect to --
MR. BARANOWSKY: The other reason for
doing it is the current value, UAp is changing in
time. Not quite rapidly as UAt.
Now, the idea was instead of saying well
let's just look at how things deviate from some
current performance, which means every time you make
an improvement you then have a little racket look at
this thing instead of saying what's the baseline upon
which I want to measure your performance change from
so that there's some acceptance criteria that doesn't
change forever. That's what it is.
MR. ROSEN: So this is to avoid the
racket?
MR. BARANOWSKY: This is to avoid
ratcheting.
MR. ROSEN: Self-ratcheting?
MR. BARANOWSKY: That's right. You would
be self-ratcheting every time you updated your model.
MR. ROSEN: Let's take a plant that is
updating his model routinely and its performance is
gradually improving. And then over time at one point
it no longer improves. It's been improving for five
years and now they have a bad quarter or a bad half a
year.
MR. BARANOWSKY: Right.
MR. ROSEN: Now their performance
indicator is going to go --
MR. HAMZEHEE: That's exactly right.
They'll be penalized --
MR. ROSEN: Even though they've been
improving for six years and they're below the industry
average?
MR. APOSTOLAKIS: Let's understand first
of all what three does. Not what it should be doing,
what it does.
You remember the way the threshold for the
industry was set, it was the 95th percentile of the
plant-to-plant variability curve. They put 103 units
unavailability and said 95 percentile. Right? That
was the threshold.
MR. BARANOWSKY: Yes.
MR. APOSTOLAKIS: Okay. It does not say
automatically that the thing in parenthesis is
negative for 95 percent of the plants.
MR. KRESS: That's what I would have
thought.
MR. HAMZEHEE: No. Not really.
MR. APOSTOLAKIS: Why not?
MR. BARANOWSKY: Because the UA -- the
baseline UA has nothing to do with a 95 percentile
point.
MR. APOSTOLAKIS: No. But that's what --
MR. BARANOWSKY: This is a different
number.
MR. HAMZEHEE: This is a different number.
MR. APOSTOLAKIS: This is not the
threshold that was set at the time?
MR. BARANOWSKY: These numbers are derived
from different information because the whole
formulation is different.
MR. APOSTOLAKIS: But I thought you said
that you wanted to maintain that 007 approach?
MR. BARANOWSKY: No. What we wanted to
maintain was the fact that we're baselining it to the
'95 to '97 time period. But this unavailability
definition is different than the one that was used in
the 007 unavailability definition. Therefore, the
number's different.
MR. SHACK: You said it was an industry
average, right?
MR. BARANOWSKY: This is an industry
average. What we were looking for before was the
deviation from the 95 percentile measurement. This is
what's the deviation from an acceptable baseline
performance that gives me a delta CDF index of some
value.
MR. WALLIS: What's the baseline
unavailability value for a train T which is unique to
a given plant?
MR. BARANOWSKY: What we said was we
didn't have that --
MR. WALLIS: Never happen?
MR. BARANOWSKY: Well, that's not such an
easy thing for us to go and get.
MR. HAMZEHEE: Did you say what's the
value for UAt?
MR. WALLIS: How do you get an average,
the industry average for a particular train which is
unique to a given plant. That's a certain plant that
has different design.
MR. HAMZEHEE: Remember, though, I think
we talk about in a couple of pages. But that specific
curve is the summation of unplanned and planned
unavailabilities. Okay. And then the planned version
of it is plant specific baseline.
In other words, because plant A and plant
B have different maintenance practices; some do more
PMs, some do less PMs. So the plant portion of it is
plant specific and the unplant portion we are going
to use a three year average based on industry
information.
MR. APOSTOLAKIS: Plant specific or
industry?
MR. HAMZEHEE: No. There are two terms for
that UA baseline. One is a plant unavailability which
is based on preventive maintenance, surveillance
testing. Those are going to be plant specific values.
This other portion of the unavailable are
due to unplant maintenance activities such as
corrective maintenance. Those corrective maintenance
consistent with PRAs are going to be average of the
industry. And we picked '95 to '97 based on what Pat
explained.
MR. APOSTOLAKIS: But why? I mean, I
don't understand that. Why is some part plant
specific --
MR. HAMZEHEE: Well, because the answer is
because for the plant maintenance activities they are
very plant specific. Every plant has a different PM
program, different --
MR. APOSTOLAKIS: But the unplanned
presumably are correlated with the planned ones.
MR. HAMZEHEE: That's not true. Now --
MR. APOSTOLAKIS: That's not true?
MR. HAMZEHEE: Well, there's some
relationship --
MR. APOSTOLAKIS: It's not true for
preventive maintenance. I expect not to have many
unplanned --
MR. HAMZEHEE: Right, there's some
relationship. But the unplanned portion is based on
some failure; either random failures or some dependent
failures based on your maintenance activities.
MR. APOSTOLAKIS: Right. So you're going
to penalize this guy who has an excellent PM program
because some other guy doesn't do that and has a lot
of unplanned SCRAMs. I mean, that's very arbitrary,
isn't it?
MR. HAMZEHEE: I don't think so.
MR. KRESS: You have sparse data on the
unplanned.
MR. BARANOWSKY: I don't understand what's
going on here, to be honest with you. All I can tell
you is this is a measure of the change in CDF that's
associated with departures from a baseline value of
unavailability that's determined to be acceptable.
Okay. That's all this is, nothing more.
MR. APOSTOLAKIS: Okay.
MR. BARANOWSKY: And if we had not used
this formula that you see here, you would have seen
the full blown PRA model with all the Bayesian
expressions and you would have still seen delta UA--
MR. SHACK: Pat, let's just pick it a
different way. Why didn't you pick a baseline that
was the plant's average for '95, '97 and it would
never change?
MR. BARANOWSKY: Because it's a sparse
data.
MR. KRESS: Because it's sparse data that
you don't have.
MR. BARANOWSKY: If you only take that
many years, some plants are -- even though
statistically are going to have a high number and some
are going to be low because they were going into
outages or not in outages --
MR. KRESS: Exactly.
MR. BARANOWSKY: So we needed to take more
data.
MR. KRESS: And then they did the best
they could with that --
MR. BARANOWSKY: Right.
MR. KRESS: -- by using the planned for
plant specific.
MR. BARANOWSKY: Exactly.
MR. KRESS: And they knew that wasn't all
of it, and they got the rest of it --
MR. APOSTOLAKIS: But let me change the
statement. Instead of using the '95, '96, '97 data,
use the number that the utility used in calculating
the CDF. Because that number included all these
considerations.
MR. KRESS: That might be another choice.
MR. APOSTOLAKIS: And that could serve as
the baseline.
MR. SHACK: You say UAp 2000 and then that
would be fixed forever.
MR. APOSTOLAKIS: Whatever.
MR. BARANOWSKY: Well, as it turns out,
this a mute discussion because the numbers are about
the same whether you use the '95 to '97 or the '99 to
2001, as Hossein is going to tell you. Okay.
We only did the '95 to '97 to be
consistent with the philosophy that was espoused in
99-007.
MR. APOSTOLAKIS: You know, this is a
discussion that at least on the Committee we've had
several times in the last two or three years. It
comes back to the fundamental objective of this ROP.
In quality control in manufacturing, forget about
reactors for a moment, the whole idea of quality
control is to measure deviations that could be
unacceptable from the expected performance of this
machine. I don't care that there other 10,000
machines in the world. My machine. Okay?
Now, we come to reactors. If we bring
this philosophy over to ROP, then we would be saying
I want to know whether San Onofre has deviated from
San Onofre's expected performance as reflected in the
PRA.
The 007 said no, we're not going to do
that. We're going to look at San Onofre and say is
San Onofre deviating from the industry average.
Different approach.
MR. BARANOWSKY: Well, George, I think --
MR. APOSTOLAKIS: Different approach. And
that's what you're trying to --
MR. BARANOWSKY: George, wait a minute.
No, not exactly. I'm not sure I think I agree with
you.
If I'm making parts for airplanes in four
different factories on machines and I have a
specification for what's acceptable, I apply a QA that
includes deviations from the specification to those
four different machines. I don't have a different
deviation spec for each machine. So I'm not sure I
agree with your --
MR. APOSTOLAKIS: No, but your machines in
manufacturing are identical. Here you have different
device.
MR. BARANOWSKY: They're never
identifiable, just like all these --
MR. APOSTOLAKIS: No, no. Here you have
different designs here. I mean, the curves in 007
show that.
MR. BARANOWSKY: I think the principle I
espouse is correct.
MR. APOSTOLAKIS: The fundamental question
is do I want to know how much South Texas deviated
from South Texas' performance or do I want to know how
much Texas deviated in the industry average?
MR. HAMZEHEE: And I think Steve made a
good point.
MR. SHACK: No, you want to know how much
South Texas deviated from acceptable performance.
MR. HAMZEHEE: That's exactly the point.
MR. SHACK: The question is how do you
determine acceptance performance.
MR. HAMZEHEE: That's exactly the point.
MR. APOSTOLAKIS: Well, it's stating the
same question a different way.
MR. SHACK: No, it's a different question.
MR. APOSTOLAKIS: Okay. If acceptable is
with respect to -- is acceptable performance with
respect to industry average. Because acceptable is
also the fact that I have licensed it. That was
acceptable independently of industry average. I
licensed South Texas. So if I licensed it and I do
the PRA, the PRA tells me that this is what the NRC
licensed. And then acceptable for me --
MR. KRESS: George, let's look at UAp for
a moment; that's the unavailability that's in the PRA.
Now supposedly they try to keep that updated in a
basin sense.
MR. APOSTOLAKIS: Sure.
MR. KRESS: Because they want to use plant
specific data and say well I didn't know what to use
when I first started my PRA for this, but I'm going to
adjust it as I go along.
UABLT, it's almost the same thing, which is
what you're asking for it to be. It's almost the same
thing because they're using the plant specific planned
maintenance and are saying "Well, I still don't have
enough plant specific data to add to that to make it
the total, which I would put in for my UAp anyway. So
they're trying to get those two to approach the same
thing, like you're talking about, but they have to use
industry wide data because they don't have enough
plant specific data to do it.
I think it's probably a pretty good way to
do it.
MR. SHACK: I mean, the plant is free to
make UAp get better and better every day. I mean
that's their choice.
The NRC says there's some level of
unavailability below which we will not let you go.
And those are different numbers. Well, they are
approximating it here but the performance that they
had in '95 to '97.
MR. HAMZEHEE: That's correct, yes.
That's correct.
MR. APOSTOLAKIS: What is the level of
unavailability? I don't understand that. I see the
difference.
MR. ROSEN: It's not exactly what you
said, Bill. The level of unavailability below which
you may not go is defined by your tech specs. And it
is a level well above --
MR. SHACK: No. This is a level by which
we think your performance is unacceptable. It's not
legally unacceptable, but it's a level below we will
start coming and looking --
MR. ROSEN: It will change your color of
one of your --
MR. SHACK: That's right. This is a level
that if you go beyond it we will change the color of
one of your indicators. You will get management
attention. If you don't think that gets a lot of
attention, you're wrong. It does.
MR. APOSTOLAKIS: Let's talk talk about
UAp minus UABLT divided by UAp. What is that?
MR. HAMZEHEE: That tells you that based
on your actual performance or actual unavailability
during a 12 quarters time frame what fraction of that
unavailability was changed or increased with respect
to baseline. In other words, UAt minus UABLT divided
by UAp gives you the fraction of change in your
unavailability.
MR. APOSTOLAKIS: But why do I divide by
UAp? That's not in the numerator.
MR. HAMZEHEE: No, no, no. If you look at
the equation and I am moving USRP to the last term,
make it easier.
MR. APOSTOLAKIS: Right. Right.
MR. HAMZEHEE: And if you look at that,
you see that you get a fraction.
MR. APOSTOLAKIS: But usually to get a
fraction of deviation from something, you subtract the
actual performance from the something and then the
something is in the denominator, too.
MR. HAMZEHEE: That's correct.
MR. APOSTOLAKIS: Instead you're dividing
by something else.
MR. SHACK: But he's got to get the UAp
out of there because he's built that into the delta
CDF and he's going to compute from CDF times Fussell-
Vesely.
MR. BARANOWSKY: Right. In order to make
this agree with the PRA model we have to use that
value. This is an approximation. We could have
written 18 pages of fault trees here. Okay. And
probably would have gotten away with it. But instead
we took something simple and said assume this stuff in
the brackets is 18 pages of fault trees. All right?
And we changed only one perimeter, the UA, and we've
compared it to baseline and we did the delta
calculation. That's it.
MR. SHACK: Now, if I was writing this
equation, I would write parenthesis CDF star FV
divided by UAp. And that's really the CDF DUA and
then I would have delta U multiplying --
MR. HAMZEHEE: And we have some backup
information in our Appendix F that does have all those
derivation and how we got to this point. We just
didn't want to bore you with those things. But if you
like, we can give you the information after the
meeting.
MR. ROSEN: You don't know our problems.
MR. BARANOWSKY: Well, could I suggest
that we're also looking at just to see if there are
some other better approximations. And it's not, you
know, closed book necessarily. But I think this is
the way we want to go. And we can come back to this
at another meeting and let's get some --
MR. KRESS: George might have been happy
if you would have had UABLT in the denominator. Would
that be just as bad or good --
MR. HAMZEHEE: Yes, we tried that.
MR. KRESS: You tried that?
MR. HAMZEHEE: And I tell you why we got
erroneous results. The problem we're having is your
Fussell-Vesely is based on your plant specific PRA.
And if the USRP is not, then you divide them by each
other, you get different contributions.
MR. KRESS: But the plant specific UAp
should have been something like a UABLT
MR. SHACK: It shouldn't be very
different.
MR. KRESS: It shouldn't have been very
different.
MR. HAMZEHEE: That's correct, yes.
MR. SHACK: This seems like the correct
first order --
MR. KRESS: Yes.
MR. HAMZEHEE: That's correct.
MR. KRESS: It's a very good first order
approximation.
MR. APOSTOLAKIS: CDF times FV by itself
is the change -- the maximum change in CDF I can get
if I assume the component is always good.
MR. HAMZEHEE: That's correct.
MR. APOSTOLAKIS: Without dividing by
anything?
MR. HAMZEHEE: That's correct.
MR. ROSEN: It's like the inverse of --
MR. APOSTOLAKIS: So now I want to somehow
take a fraction of that and consider it acceptable in
my plant?
MR. HAMZEHEE: Yes.
MR. APOSTOLAKIS: So I will multiple by
some fraction.
MR. HAMZEHEE: By fraction of change in
your unavailability.
MR. APOSTOLAKIS: Right. But you see
that's the problem, that it's not a fraction of change
if I subtract UABLT and divide by UAp.
MR. KRESS: It's an approximation, though.
MR. APOSTOLAKIS: To what?
MR. KRESS: It's an approximation to that
fraction you --
MR. APOSTOLAKIS: Well, why don't I divide
by UABLT then?
MR. HAMZEHEE: If you are mathematically
want to have an exact equation, you're right, it
should be USRT minus -- BLT divided by ULBLT or USRT
minus USRP divided by USRP. That mathematically the
exactly term. But when we did the validation and
compared these with the actual full scope PRAs, this
equation would give us the best approximation.
MR. APOSTOLAKIS: Sol you have the actual
-- the exacting somewhere?
MR. HAMZEHEE: Yes. Well, we use the full
scope from all those and then we run them this
equation to see if you get acceptable results. You
know, you want to make sure what you get here is
somehow close to what you may get if you use your full
scope PRA.
MR. KRESS: Now you want to get what you
would have gotten if the full scope PRA had been
corrected to have the right value of UA, which would
have been in my mind a UABLT.
MR. HAMZEHEE: That's correct.
MR. KRESS: But I maintain that UAp is
close enough.
MR. HAMZEHEE: That's correct.
MR. KRESS: It just makes a good
approximation.
You know, I'm not surprised. Because the
full scope's formula has a UAp in it which probably
isn't an updated UABLT at all. So I'm not surprised
that this gives you better approximation to what you
get by the PRA. It may not be a better approximation
to the delta CDF.
MR. HAMZEHEE: But we also had the utility
folks at our pilot plants -- there are how many, Tom?
Ten some plants that are working with us.
A few of them went back and validated this
through using their own updated PRA models. And they
also came out with --
MR. KRESS: So they had the updated UAp?
MR. HAMZEHEE: That's right. They used
theirs and they ran it through their models to see
what kind of approximation we get. And they all came
out with positive indications.
So this is a joint effort with the
industry. WE're not doing this in isolation.
MR. KRESS: Yes, I think this is a pretty
good approximation myself.
MR. APOSTOLAKIS: To what?
CHAIRMAN BONACA: We need to move on.
MR. APOSTOLAKIS: It's not clear to me
what we're approximating.
MR. KRESS: You're approximating the delta
CDF due to the departure of the unavailability from a
baseline level.
MR. HAMZEHEE: That's correct.
MR. KRESS: In fact, the delta CDF you get
may be negative.
MR. HAMZEHEE: That's correct.
MR. KRESS: Even though you've decreased
performance.
MR. WALLIS: That's why the word "change"
is misleading. You say index due to changes in train.
You can have an equation three with a positive or
negative value without changing anything.
MR. KRESS: That's right.
MR. WALLIS: So the word change is a bit
misleading here. It's really really the deviation of
train unavailability from some rather arbitrary chosen
baseline.
MR. KRESS: It's a difference in your CDF
from some baseline CDF.
MR. WALLIS: Right. It's very different
from the idea of a change.
MR. APOSTOLAKIS: Yes, it's not a change.
MR. WALLIS: Not a change in anything.
MR. FORD: Reality it's a second
difference, isn't it? It's a change of a change.
It's a change in delta CDF. It's defining what you get
by multiplying CDF times Fussell-Vesely, doesn't that
give you delta CDF?
MR. APOSTOLAKIS: Right. Delta CDF.
MR. FORD: And then you're multiplying by
this on another change ratio --
MR. APOSTOLAKIS: Because they want to
normalize it to something else. That's really what
they're doing in their second difference. Their
second fraction. And my problem is that the
denominator seems to be different from what you have
in the numerator and you say it's an approximation.
And I don't know what it approximates.
MR. BARANOWSKY: Dale Rasmuson has a
comment.
MR. RASMUSON: The first part there, if
you take CDFp times Fussell-Vesely divided by UAp is--
if you work it out, is just really the Bernbaum
importance measure. That's what it comes down to.
MR. HAMZEHEE: That's another way of
presenting it.
MR. WALLIS: That doesn't help me at all.
MR. HAMZEHEE: I know. That's why it's
presented like this.
MR. WALLIS: Doesn't the DCFD -- isn't
that what it is?
MR. APOSTOLAKIS: Well, why do we let the
man finish.
MR. RASMUSON: You know, the Bernbaum
importance measure is just the partial derivative of
the basic event in question with respect to the core
damage frequency.
MR. WALLIS: Using the UA?
MR. RASMUSON: Right.
MR. WALLIS: It would be simpler if we
started with that.
MR. RASMUSON: Right. I mean, that's what
it's equivalent to.
MR. WALLIS: I think what we're all arguing
about is what you multiple it by to get anything that
means anything.
MR. SHACK: -- CDF DU, then the delta
they've chosen is the right one. It's the actual
version what you find acceptable.
MR. APOSTOLAKIS: Why do you say DU? All
I see is UAp which is the value of unavailability.
It's not the DU.
MR. SHACK: He's arguing that that does in
fact give you DCDF DU.
MR. APOSTOLAKIS: No. I don't see why. I
mean it's just the value of unavailability.
MR. SHACK: The delta.
MR. APOSTOLAKIS: UAp in the denominator
is not a delta. It's the unavailability itself.
MR. WALLIS: You're looking at the
significance as far as CDF is concerned of the plant
deviating from some baseline.
MR. APOSTOLAKIS: I think we said earlier
is the more appropriate.
CHAIRMAN BONACA: We have been stuck on
this equation for 40 minutes flat. And I think we
need to make some progress.
MR. APOSTOLAKIS: Now let's to equation
two.
CHAIRMAN BONACA: Right.
MR. APOSTOLAKIS: Let's go to equation
two.
CHAIRMAN BONACA: The next 40 minutes,
yes.
MR. APOSTOLAKIS: If I have one of the
four system, the four trains --
MR. HAMZEHEE: Yes.
MR. APOSTOLAKIS: Then my UAI will be
worse than if I -- in a plant where I have two trains.
I don't understand why.
MR. HAMZEHEE: No. But remember if you
have four train system, then you have four Fussell-
Vesely values.
MR. APOSTOLAKIS: Lower.
MR. HAMZEHEE: Exactly. You got it.
MR. APOSTOLAKIS: But I still don't
understand why UAI is a sum.
MR. ROSEN: Say the rest of that sentence.
You have a four train system, you have four Fussell-
Vesely values and?
MR. HAMZEHEE: You're going to have four
different values for Fussell-Vesely. And when you use
equation three to add four terms, using equation three
then the equation three because of the Fussell-Vesely
is going to be much smaller quantity because of the
importance of one train for the four train system is
going to give much lower Fussell-Vesely value. So it
does very nicely put together the PRA model into a
four train system by having those terms multiplied and
added together.
MR. APOSTOLAKIS: Well, let's look at
equation two now.
Why is the system unavailability index due
to changes in train unavailability, the sum of the
unavailability and this is for the trains? Because
the system unavailability is not the simple product or
the simple sum of train unavailability. There is
coupling there because of the testing schemes. So I
don't understand why it's the sum.
And the word "change" just throws me off
everywhere I see it. It is what Dr. Wallis said, that
these are changes not in performance, they're changes
with respect -- differences actually from some
baseline performance?
MR. HAMZEHEE: Yes, I think --
MR. APOSTOLAKIS: So these are differences
from the baseline?
MR. HAMZEHEE: That's correct.
MR. APOSTOLAKIS: Not changes due to
random --
MR. HAMZEHEE: Well, that's what it said,
changes in train unavailability relative to the
baseline value. That's the more accurate -- well,
you're right. I'm just adding that. It means
relative to the baseline value. That's what it means.
Now, we didn't write everything here
because the intention was not for this to be a
textbook.
MR. APOSTOLAKIS: No. The problem of
saying is that you're dealing with collecting data.
MR. HAMZEHEE: Yes, sir.
MR. APOSTOLAKIS: And changes there mean
that the data collect now should tell me that I
deviate from what was happening before. But that's not
what you mean. You mean from some baseline.
MR. HAMZEHEE: Relative to a baseline
value. That's what these --
MR. APOSTOLAKIS: Anyway, why is equation
two true?
MR. HAMZEHEE: Because the way -- you're
right. Now, you look at the system from a reliability
perspective, what you said is very true. But the way
we look at the PRA models and we wanted to evaluate
the contribution of a system unavailability on total
CDF, in reality what we do is we go to that system
components and trains and change their perimeters from
a nominal value to some changed value at the same
time.
Once we make those changes, then we go
back and requantify the CDF to get a revised CDF.
So the way we're doing it here is in a
sense doing the same thing. You're saying if the
unavailability of train changes by X percent or by X
hours and B by Y hours, you go back in reality when
you do the full scope evaluation of those changes by
changing the system train A increasing or decreasing
it by some value. At the same time you go use the
train B increase or decrease unavailability. And then
you requality your core damage frequency.
MR. APOSTOLAKIS: So what's this? This
sub.
MR. HAMZEHEE: Yes.
MR. APOSTOLAKIS: It doesn't say --
MR. HAMZEHEE: But the way we have
formulated the equations, you sum it but using
equation three gives you exactly what you do in
reality when you try to measure the impact of a change
in a system performance on CDF.
CHAIRMAN BONACA: Let me just say this.
Okay. This is not the proper approach to go about
this equations. I mean, clearly maybe the best thing
to do is to give you a chance to go back and write --
all this discussion in writing, maybe 3, 4, 5 pages,
so that we can review it and then we much better can
understand. Because I think we're getting back to the
same definitions.
I wrote down deviation availability from
the baseline about half an hour ago and now the same
point is being made again. I mean, so otherwise we're
going in circles.
MR. KRESS: It wasn't exactly the same
point. But owing to the comment to George that the
summation -- George. George is not listening.
MR. APOSTOLAKIS: I have to listen. I
think he's complaining. I'm sorry.
MR. KRESS: The reason the summation is
all right is because this is an index, it's not a real
thing. It's an index.
MR. APOSTOLAKIS: I don't understand that.
I mean, the system consists of two trains. They
probably listed performance of the trains is not
independent. They're not independent because they
have coupled them by the way I do tests.
MR. KRESS: I don't care. I just want an
index that tells me there's something different than
what I used to have.
MR. APOSTOLAKIS: But it has to have some
basis --
MR. BARANOWSKY: It has a basis, but I
think that we're not going to solve it here, as Dr.
Bonaca said.
MR. APOSTOLAKIS: Absolutely.
MR. BARANOWSKY: I suggest that we do
produce a document that you can look at.
MR. APOSTOLAKIS: Yes, good.
MR. BARANOWSKY: Which I think we have to
do anyhow.
MR. APOSTOLAKIS: Are we here in this next
week as well?
CHAIRMAN BONACA: No.
MR. HAMZEHEE: It may be the short version
of this presentation to full Committee next week.
CHAIRMAN BONACA: I don't want to see any
equations next week.
MR. HAMZEHEE: No, no. Just text, you're
right.
MR. BARANOWSKY: Because we're coming back
to this Committee --
CHAIRMAN BONACA: In fact, I heard before
the plan is to have a couple of more updates before we
have a product that we would comment on. So that --
MR. APOSTOLAKIS: A two or three pager
explaining the basis of these equation. And when you
say approximation, explaining what you're
approximating would go a long way towards --
CHAIRMAN BONACA: Well, most of all, I
mean they've made an effort to explore different
possibilities. They did that. And I think if you can
describe that and explain to us why you made the
choice you made.
MR. ROSEN: In your white paper how a
system with three trains in a system with two trains
gets -- how the system with three trains gets credit
for the --
MR. HAMZEHEE: Yes. All right.
MR. ROSEN: You know, with some arrows so
the stupid ones like me can understand to see how
you're dividing it out.
MR. APOSTOLAKIS: Another thing I would
appreciate is if I do consecutive testing of the
trains or I do staggered test, would that make any
difference anywhere here. Don't answer now.
MR. HAMZEHEE: All right. All right. We
will provide some backup documentation and background,
and then we'll give it to you.
MR. BARANOWSKY: We now we have something
to do for the next time we get together.
MR. WALLIS: But I disagree with Pat's
suggestion that the next presentation have no
equation. I mean, these are such simple equations
that we ought to understand.
MR. BARANOWSKY: At the full Committee?
That's next week.
MR. WALLIS: Yes, based on what you're
doing.
MR. BARANOWSKY: Well, we're not going to
have a white paper put together by next week.
MR. WALLIS: But there's nothing
complicated.
MR. BARANOWSKY: I agree with you. It's
not a problem, it's just that --
CHAIRMAN BONACA: Wait a minute. They can
state what the meaning of the equation is. Okay. But,
you know, we shouldn't be there taking it apart and
questioning every single step. There is no time for
doing that. I mean simply we have to have a better
understanding before we get to Committee.
MR. APOSTOLAKIS: If we have your letter
next week, then it's okay.
MR. HAMZEHEE: And I think the idea was to
provide some concept and give you background
information, you read them and come back to us if you
have more questions. There are some background
information, yes.
MR. ROSEN: But they're the same
equations?
MR. HAMZEHEE: We are not going to go over
any detail, but let me just spend a few minutes and
tell you what the next page equations are if you don't
ask me any questions.
The next one is very similar to
unavailability index, but this is not for
unreliability. The only difference is the other one
was done at the train level, this is done at the
component level for active components within a system.
MR. APOSTOLAKIS: Now why on earth is the
same here, this is Bayesian-updated component
unreliability for previous 12 quarters and the
previous one you don't say Bayesian-update?
MR. HAMZEHEE: It's a very good question.
But, George, you realize that in any PRAs for
unavailability we always use exact numbers for
unreliability, we use Bayesian-update.
MR. APOSTOLAKIS: Why don't we use
Bayesian for unavailability?
MR. HAMZEHEE: Because unavailability you
have more data and more information, the uncertainty
is lower than if you use the unreliability that is
very scare and you don't have as much events.
MR. APOSTOLAKIS: So you are taking the
approach now that use Bayesian when I have lot of
uncertainties. That doesn't make sense. I mean, you
can use it all the time.
MR. HAMZEHEE: Is that done in PRAs?
MR. APOSTOLAKIS: Not some PRAs, not all
PRAs.
MR. HAMZEHEE: The majority of the PRAs
they don't do Bayesian on unavailability.
MR. APOSTOLAKIS: This is not a democracy.
We are not voting here. Okay. Most PRAs don't do
uncertainty analysis. That doesn't mean they're
right. Okay?
MR. HAMZEHEE: All right.
Now, that's the concept. Do you want me
to spend anymore time or are you happy with just what
I said?
MR. APOSTOLAKIS: I don't want to spend
anymore time, but that doesn't mean we're happy.
Don't draw conclusions.
MR. KRESS: I'd like to hear about
equation five.
MR. APOSTOLAKIS: Oh boy.
MR. HAMZEHEE: Okay. Equation five now
for unreliability you see that you have that URBC
which is the actual unreliability that is going to
measure for any period, in this case 12 quarters.
Now, to calculate that if you remember
John's definition the unreliability, the way we define
it to make sure that the combination of unavailability
and unreliability is complete, we define it as a
failure of the component to function on demand or
given that it is started successfully on demand, it
continues operating for the mission time. So this is
what that equation is.
And the PD give you the probability of the
components failing to function on demand. And the
second term gives you the probability that it's going
to fail during the mission time. And the Tm is the
mission time, which usually is about 24 hours, in some
cases different.
MR. KRESS: Well, really the question was
why wasn't that by the -- and you answered it it's the
mission time.
MR. HAMZEHEE: That's correct. Not a
fault exposure time. That's correct.
MR. APOSTOLAKIS: Now, the failure on
demand probability. Isn't that one of the views of
before?
MR. HAMZEHEE: No. None of these things
were used in the previous page. Those were all
unavailabilities only, which were the direct
calculation of unavailable hours divided by the number
of hours that the reactor was critical.
MR. APOSTOLAKIS: So it's only
unavailability due to some reason that --
MR. HAMZEHEE: Planned and unplanned
maintenance.
MR. APOSTOLAKIS: Okay. So it then
includes that plus something else?
MR. HAMZEHEE: That's correct.
MR. APOSTOLAKIS: And that something else
is what?
MR. HAMZEHEE: No. PD does not include
unavailability. It only includes on demand time.
MR. APOSTOLAKIS: On demand. On demand.
MR. HAMZEHEE: Remember we said that up
front the big equation was that the mitigating system
performance index has two terms. Unreliability index
plus unavailability index. The unavailability index
had all those planned and unplanned maintenance
activities that would make the equipment unavailable.
The second term is what you see here, the
unreliability index.
MR. APOSTOLAKIS: So if there is a -- I
mean PD would be something -- you could put the zero
for many cases?
MR. HAMZEHEE: No, it won't. Because you
use Bayesian-update. It can never be zero.
MR. APOSTOLAKIS: Physically. Look at it
physically.
MR. HAMZEHEE: Physically, yes, it could
be zero actually for many quarters, yes. Yes. And
that's why we use Bayesian, because you don't want to
use zero when it comes to probability.
MR. APOSTOLAKIS: No, no, no, no. That's
okay.
MR. BARANOWSKY: Well, if it's more than
that, I think the issue about using a Bayesian-update
has to do with sparse statistics of where you only
have a few numbers of demands and then you have a
failure.
If you take a piece of equipment that
normally, say, doesn't fail but once every 600 demands
and you only have then demands per year. Okay? Now
every once in a while there'll be a failure, but if
you look at only a three year period with 30 demands,
it's going to look like your performance was pretty
bad. But, let's face it; you have to have a way of
judging whether or not that's an outlier or not. And
so this allows us to do that, which is exactly the way
the PRAs are done.
MR. APOSTOLAKIS: PD represents failure
modes.
MR. HAMZEHEE: Component failure on demand
probability based on --
MR. APOSTOLAKIS: Failure modes that are
there because you are trying to start the thing?
MR. HAMZEHEE: That's correct, right. Or
open a valve or close a valve.
MR. APOSTOLAKIS: Well, start. Do
something.
MR. HAMZEHEE: Possibly start or it's
failure to run.
MR. APOSTOLAKIS: It's available in the
previous equation.
MR. HAMZEHEE: That's correct.
MR. APOSTOLAKIS: But it doesn't start
because there's some extra stresses or some --
MR. HAMZEHEE: Yes. As Pat said, very
consistent with PRAs. That's the best way of
approximating equipment on reliability or
unavailability. We're not presenting anything new
here.
MR. ROSEN: And this is the solution for
the T over 2 problem?
MR. HAMZEHEE: That's exactly right.
That's why don't you need T over 2 and it gives you a
better approximation of equipment unreliability.
MR. WALLIS: Now, I have to say this. I'm
very happy that you're giving us equations, but I have
great difficulty getting the dimensionality right. N
and A seem to be numbers. And T is hours?
MR. HAMZEHEE: Yes.
MR. WALLIS: And P cannot be numbers and
hours and be compatible --
MR. HAMZEHEE: Yes. Let me just give you
a quick explanation. The whole thing has to be
dimensioned. The PD you agree that's dimension,
right?
MR. WALLIS: Well, everything is dimension
except this mission time, which is in hours.
MR. HAMZEHEE: Yes. But lambda is barely
a rate per hour.
MR. WALLIS: Well, then T is in units of
hours.
MR. HAMZEHEE: Correct. Mission time is in
hours.
Are you looking at (5b) or are you looking
at (5)?
MR. APOSTOLAKIS: 3 and B are not the same
units.
MR. BRANCH:
MR. HAMZEHEE: Yes. Now let's go back.
A and B are the perimeters of the industry priors.
MR. APOSTOLAKIS: They're numbers.
MR. HAMZEHEE: B is hours also. These are
not the same As and Bs from (5a) to (5b).
MR. APOSTOLAKIS: Then B is not compatible
with A.
MR. BARANOWSKY: B is in hours.
MR. HAMZEHEE: Let me explain it.
MR. APOSTOLAKIS: Please do.
MR. HAMZEHEE: For the simplification of
it, (5a) and (5b) are two different equations. As and
Bs in (5a) are different from As and Bs in (5b).
MR. APOSTOLAKIS: Then these?
MR. HAMZEHEE: Correct. Just for
simplicity I used the same nomenclatures.
MR. APOSTOLAKIS: Okay. With two
different calculations.
MR. HAMZEHEE: Exactly.
MR. WALLIS: You're going to make it clear
when you go do it next time?
MR. HAMZEHEE: Yes.
CHAIRMAN BONACA: I think it would be
important again for these equations to understand what
the purpose was in the index. But you tried, because
clearly this is an approximation, as you're saying,
but you try. You try different things and why you
chose -- you rejected most of them and you chose the
product that you had. And I think we'll have it in
writing, I think it will be easier to evaluate for
discussion at that time. We need to look at it again.
MR. KRESS: I personally think you're on
the right track here.
MR. HAMZEHEE: Thank you.
MR. KRESS: And don't have these problems
that the other guys have.
The one thing that I see that I worry
about is I don't want to know the technical basis of
the one times 2 to the minus 6, the thresholds. You
know, I understand you've taken a one times 2 to the
minus 4 and dropped off two orders of magnitude. This
is sort of arbitrary thing to me. And I would like a
little more explanation of that at some point.
MR. BARANOWSKY: Okay. But in a nutshell,
that came from what we did in 99-007.
MR. KRESS: Yes, I know. I didn't like it
then either.
MR. BARANOWSKY: And it's supposed to be
consistent with reg guide 1.174.
MR. KRESS: Yes, and I didn't like that
either.
MR. BARANOWSKY: Well then you won't like
our explanation.
MR. SHACK: If you read the Brunwick
letter, you'll see why?
MR. RANSOM: Just one quick question.
MR. BARANOWSKY: Yes, sir.
MR. RANSOM: On the URBLC that you use
here, is that also an industry average?
MR. HAMZEHEE: That is the same concept.
It's baseline value based on industry average for some
period of time. Same logic, same concept.
MR. ROSEN: I was wondering if you were
hungry when you did this, because you've orders of
magnitude BLTs. I'll have one order of that.
MR. BARANOWSKY: I'm going to suggest that
we probably don't need to go over the rest of the
viewgraphs. Because we've covered the issued i them,
you know, the thresholds, baselines and all that other
kind of thing. So instead of plowing through it,
we've got the record. It's replete with stuff for us
to look at and take into consideration.
CHAIRMAN BONACA: Would you just go
through the final page.
MR. HAMZEHEE: Do you want to go over
conclusions?
CHAIRMAN BONACA: Yes, just the
conclusions.
MR. HAMZEHEE: All right.
MR. APOSTOLAKIS: Well, do you want to say
anything about technical areas currently under
evaluation? Well, say something that we haven't said
before.
MR. HAMZEHEE: You want to go on the one
page before the last page?
MR. APOSTOLAKIS: What?
MR. HAMZEHEE: He said they wanted --
MR. APOSTOLAKIS: Well, there is technical
areas under evaluation, I'd like to hear about the
thresholds. Second to last page.
MR. HAMZEHEE: Again, these are the areas
that we're currently working with the industry to
further refine and improve. And one thing they try to
determine the acceptable of level of false-
positive/false-negative indication. And the following
is the example. And what the first one means is the
probability of calling in performance indicator Y when
it's actually the baseline performance.
And the second one is calling something
green when it's either white or yellow and calling
something green when it's yellow or red. And I don't
even want to go over more discussion. We talked about
this more earlier.
And then the next one is issues, again, as
you all raised questions and concerns or mainly
questions. We want to again go back during the pilot
and revisit the baseline values for these index--
indices. So we're still trying to come up -- you
know, make sure that these values are the best values
to use.
And the third bullet talks about the fact
that as part of this pilot program we also want to
have some independent calculations of these perimeters
and indices using the SPAR models versus the
licensee's PRAs.
And last but not least is to evaluate any
potential differences that we may get using these
indices versus using the SDP process.
These are the current areas that we're
working on.
And the last page are just summary of the
conclusion. I think based on what we've done and all
the work that we've done in the last year or so and
using a lot of insights from RBPI, we believe that
this MSPI approach is based on risk insights and it
does account for plant-specific design and operating
characteristics through the use of plant-specific data
and risk models.
And again, those are the things that we
think it does.
MR. APOSTOLAKIS: Wait a minute. If you go
back to your -- oh, remember the studies that you guys
do, the reliability studies?
MR. HAMZEHEE: Yes.
MR. APOSTOLAKIS: They do not define under
reliability as your equation five. You include the
human intervention. Here you don't, huh?
MR. BARANOWSKY: We put recovery in there.
That's correct.
MR. APOSTOLAKIS: You will put recovery in
there?
MR. BARANOWSKY: No, no. We do put
recovery --
MR. APOSTOLAKIS: In those --
MR. BARANOWSKY: But there are rules in
the data collection for when to include recovery and
not include recovery. I mean, there's a definitions
document, there's going to be documents this thick.
MR. APOSTOLAKIS: I remember. But are you
going to do the same thing here?-
MR. BARANOWSKY: It's going to have a
procedure for crediting recovery that's consistent
with some guidelines that we're putting in.
MR. APOSTOLAKIS: So it will not be just
lambda Tm. It would be plus other things.
MR. BARANOWSKY: The data will be adjusted
so that those with recovery are treated differently.
MR. APOSTOLAKIS: Okay.
MR. HAMZEHEE: And that's actually how we
get some of the As and Bs.
MR. APOSTOLAKIS: Okay.
MR. HAMZEHEE: We apply some recovery so
we adjusted failure rate.
MR. APOSTOLAKIS: Oh, through the failure
rate?
MR. HAMZEHEE: Yes. Because some are
recoverable, some were not using the NUREG reports
that we have generated. So we're consist in that
area.
Now, let's quickly go over the first
bullet and look those sub-bullets.
Use of Fussell-Vesely importance measures
does account for plant-specific features. That's the
beauty about it.
Treatment of demand failures in
unreliability indicators, that's no more fault
exposure time.
Use of Bayesian update for unreliability
indicators because i the short period of time you have
less data points, so Bayesian gives a better
approximation of equipment unreliability and it uses
risk-significant functions rather than design-basis
functions, MSPI.
And then, again, having PI for support
systems mainly for the cooling water, such as service
water and CCW.
And then I think --
MR. APOSTOLAKIS: The English is correct
here.
MR. HAMZEHEE: I'm sorry?
MR. APOSTOLAKIS: The English is correct
here.
MR. HAMZEHEE: Oh, good, I'm glad.
Well, the second bullet MSPI approach
allows -- now, that's why some of you were saying well
one may be negative, one may be positive. But another
good aspect of this approach is that you can balance
between component reliability and component
unavailability. So that the plant that is doing a lot
of PMs is going to have higher unavailability, but
hopefully he has low on reliability. So they're going
to balance each other in a very logical fashion and
it's also consistent with the maintenance rule.
MR. APOSTOLAKIS: Again --
MR. HAMZEHEE: Uh-oh.
MR. APOSTOLAKIS: If I do PM on one train,
don't I make sure the other train is good?
MR. HAMZEHEE: Definitely, by tech specs.
MR. APOSTOLAKIS: Right.
MR. HAMZEHEE: So?
MR. APOSTOLAKIS: So somehow you're
penalizing those people. You say, you know, your
unavailability is very high because you're doing a lot
of PM. I mean --
MR. HAMZEHEE: But at the train level or
the component level, not the system level.
MR. APOSTOLAKIS: AT the train level, yes.
And you're penalizing them for that?
MR. HAMZEHEE: No. And you have a high
unavailability, but remember you sum them up. The
other terms which is unreliability is going to go
down. So you balance between unreliability and
unavailability.
MR. ROSEN: As long as you do good
maintenance.
MR. HAMZEHEE: Exactly. Now, if you do a
lot of maintenance but you're doing the wrong thing,
then both are going to go up. Then that's a good
indication, too.
MR. APOSTOLAKIS: But isn't there some
situations where if one thing is down, one train is
down, then people actually take extra measures and
make sure the other train is up.
MR. HAMZEHEE: That's correct, yes.
MR. ROSEN: What we do is start the other
train --
MR. APOSTOLAKIS: There's absolutely no--
MR. ROSEN: Start the other train first.
No, you don't take it out of service to check that,
you start it. You put it in test.
MR. APOSTOLAKIS: Yes, and there is no
credit for that.
MR. ROSEN: You start it and run it and
make sure it's okay, shut it back down. And now you
take the other train and you're going to do
maintenance out.
MR. APOSTOLAKIS: And so why am I going to
be penalized for that maintenance?
MR. HAMZEHEE: We are not penalizing.
Maybe either I did not state it clearly or something.
The way these are consistent with PRA models, when you
look at the PRA model you never allow for simultaneous
either maintenance or test on more than one train at
the same time. And you know better that even if you
do at the end, you go back and zero out those -- that
have two trains out due to maintenance or testing.
The same thing is done here. We don't
penalize them. We use the same PRA model, the same
approach, but given that one train is unavailable,
that's what we're counting. The other train, the
models don't allow to be unavailable. So there is no
mismatch or no penalizing --
MR. APOSTOLAKIS: But your approach I
argue doesn't include the fact that the other train is
available.
MR. HAMZEHEE: Oh, it does.
MR. APOSTOLAKIS: No, it doesn't. In FV
only. In FV.
MR. HAMZEHEE: Yes, that's because it's
not allowed to be unavailable.
MR. APOSTOLAKIS: It doesn't matter it's
not allowed. In FV it --
MR. HAMZEHEE: It does. In FV it does
account for it.
MR. APOSTOLAKIS: All right.
Now, in the bullet here, the before last,
the limitations. What is the biggest limitation of
what you're doing?
MR. HAMZEHEE: Oh,I'm sorry. Which one
are you?
MR. APOSTOLAKIS: The --
MR. HAMZEHEE: Well, remember the
limitations, for instance, is one is we do not include
concurrent failure of multiple components.
MR. APOSTOLAKIS: Okay. So that's the
biggest one?
MR. HAMZEHEE: Sure.
MR. APOSTOLAKIS: Good.
MR. HAMZEHEE: One is we don't look at
common cause failures. Now, they don't happen often
but --
MR. APOSTOLAKIS: No, you answered the
question I wanted. You know, there was an expected
performance and you met it.
MR. HAMZEHEE: And the last one --
MR. APOSTOLAKIS: That's to be a joke, per
se.
MR. HAMZEHEE: I don't know what is joke
and what's not anymore. Sorry.
MR. ROSEN: Now we've achieved --
MR. APOSTOLAKIS: See, I don't base your
expected performance on the NRC staff average. Would
you like me to?
MR. HAMZEHEE: Well, no comment.
And the last one is that it we believe
based on all the work we've done that it provides
appropriate risk categorization of performance
degradations that are covered by --
MR. APOSTOLAKIS: Okay. When you write
the white paper, when you say approximation, would you
please tell me what the exact thing is. Don't assume
I know, okay?
MR. ROSEN: You have been subjected to the
average level of abuse that we normally subject NRC
staffers to. But --
MR. APOSTOLAKIS: No. This group is --
MR. ROSEN: Maybe you need more. Maybe one
star. The average level times a star.
But I would like to say that --
MR. HAMZEHEE: Yes, sir.
MR. ROSEN: -- this is very good work.
MR. HAMZEHEE: Thank you.
MR. ROSEN: And it's very much in the
right direction and solves many of the problems that
I have had of the discriminatory treatment of plants
that exists in the current mitigating system.
MR. APOSTOLAKIS: What's beyond means
though for --
MR. HAMZEHEE: And thank you.
MR. APOSTOLAKIS: -- why for 2 or 3 years
now people have been telling me that plant specific is
not the way to go and now all of a sudden it is the
way to go?
MR. HAMZEHEE: We are learning.
MR. BARANOWSKY: George, I don't know
whose been telling you that, but remember we're trying
to factor in some of the latest things that we did in
the risk-based PI program and from what we learned
from doing all those studies. And we think we've got
enough basis to go forward with this kind of thing on
a plant specific basis.
MR. RANSOM: And one thing I don't
understand is why this is tied to the industry trend,
you know, rather than being an absolute measure of
whether the plant is improved or not.
I mean, the industry is presumably a value
that will float and could either get worse or better.
And the measure now is relative to that rather than to
where the plant is actually operating.
MR. HAMZEHEE: Well, again, maybe I did
not explain it clearly. But once we defined the
baseline, we're not planning to change that. The
baseline we're planning to keep it constant. Because
as Pat said, we at some point in time basically
between '95 to '97 we agreed that the industry
performance was acceptable. So if everybody agreed and
we used the industry average for that period, then
that's going to be fixed. You don't change it anymore.
MR. RANSOM: Oh, okay.
MR. HAMZEHEE: That's the acceptable level
of performance.
MR. RANSOM: That's not changing with
time.
MR. HAMZEHEE: That's right. No, no, no.
And I did not explain that. I apologize.
MR. APOSTOLAKIS: I don't know what it
means that the industry performance was acceptable
when there is some plants that have CDFs higher than
10-4. Why is that acceptable? It's just that legally
we can't do anything about it. That's unacceptable.
MR. BARANOWSKY: They did not pose undue
risk and the Commission said that's okay.
MR. APOSTOLAKIS: Right. So now it's
undue risk and I can go from 10-3 all the way to 10-7.
And we're saying this is nice and acceptable.
MR. LEITCH: Have you considered the
treatment of --
MR. APOSTOLAKIS: We have a goal of 10-4
and yet being above the goal by an order of magnitude
is fine.
MR. ROSEN: Because it's an average goal
for the whole fleet.
MR. HAMZEHEE: That's right. We're not
taking one plant --
MR. APOSTOLAKIS: On the average,
everybody's an average person.
MR. LEITCH: In the treatment of shared
systems, for example, a diesel that's shared between
two units.
MR. HAMZEHEE: Yes. And we are -- actually
that's a good point. We have talked about this
several times during our public meeting with the
industry. There are some criteria that we define in
9902, the NEI document that said if you follow and
omit certain criteria, those shared equipment like a
diesel for 2 units per plant can be credited into the
PR if they're credited in the PRAs, we do allow their
operation and that is built into the Fussell-Vesely.
So that's a very good point. Those plants are going to
have ore flexibility with the diesel generator
reliability.
MR. LEITCH: All right. In a PWR, for
example, where you're looking at HIPSI RCIC, RHR in an
HIPSI mode perhaps and diesels and cooling water
systems. Are we talking about 12 indicators? In
other words, there would be six for each one of those
things for unavailability and six for unreliability?
In other words, are we developing a whole family of
indicators here or all this data somehow assimilated
into one indicator?
MR. HAMZEHEE: Well, no, the one
unreliability and unavailability are going to be
combining to one. So you're going to have one PI for
the system. That includes unreliability contribution
and unavailability contribution of the system, for the
given system.
Now talking about the cooling system, as
you mentioned there are variations of them. And if
you remember, we mentioned that was one of the
difficulties that we encountered during the RBPI
Phase-1 study. And now as part of this effort with
the industry we are planning to combine all the
cooling water such as service waters, CCW or their
equivalent into one or two PIs. So you're not going
to have more than maybe two PIs.
MR. LEITCH: So for example to go back to
HIPSI just to make sure I understand, there would be
one HIPSI PI?
MR. HAMZEHEE: In this PI, correct.
MR. LEITCH: That would reflect both
unavailability and unreliability?
MR. HAMZEHEE: Correct. So in other
words, we estimated if we have six systems on the
average that we monitoring, we're estimating that we
have anywhere from 4 to 6 PIs.
Am I right?
MR. SATORIUS: Yes, that's correct.
MR. LEITCH: But then in order to -- I
mean, it's quite a different problem whether it's
unavailability or unreliability. They're different
actions the utility would take to solve those
problems.
MR. HAMZEHEE: Sure.
MR. LEITCH: They're different areas the
regulator may want to look at. So you have to go back
another level. In other words, if that indicator is
going off, you need to look back one level further
down the chain, so to speak.
MR. HAMZEHEE: Exactly.
MR. LEITCH: To say is this unreliability
problem or an unavailability.
MR. SATORIUS: Absolutely. And that would
be part of the inspection program. Because if this PI
would cause a system to change colors from green to
white.
MR. LEITCH: Right.
MR. SATORIUS: Then the action matrix
would indicate that there would be a supplemental
inspection that would take place. And that
supplemental inspection would be charged to do just
what you've indicated.
MR. ROSEN: But there's no question,
Graham, that the utilities will monitor this not at
the indicator level. They're monitor at the
unreliability and the unavailability.
MR. LEITCH: Yes, they already are.
MR. ROSEN: Yes.
MR. LEITCH: And what I'm saying is if you
just looked at the indicator that you're producing, it
wouldn't be intuitively obvious what the problem was.
I mean, you would know it was our problem, but --
MR. HAMZEHEE: But the beauty about it is
that if you look at the equations, you can easily go
back and find out where they come from.
MR. LEITCH: Right.
MR. HAMZEHEE: So you can look at the
contributions and do your root cause analysis all the
way down to the component level. Because that's how
these things are built up.
MR. LEITCH: Yes.
MR. HAMZEHEE: So that would allow you to
go back and look at the root cause of the problems.
MR. LEITCH: Okay. Another question I had
was do you now have or do you plan to assess as part
of the pilot what is the level of effort on the part
of the utility to collect all this data?
MR. SATORIUS: You know, Mr. Houghton
whose available with NEI might be in a position to
speak for industry on that.
The question, Tom, was are we mindful of
and is the pilot going to investigate the level of
effort that licensees are going to have to expend in
order to capture this data. And the answer to that
is, yes, we've gotten very good feedback. WE have a
working group that's been formed for approximately a
year and it consists of members of industry as well as
NEI. And we are reminded monthly that the level of
effort that is going to be involved with collecting
this data.
CHAIRMAN BONACA: And there was an issue
that was raised, in fact, by the industry that, you
know, if you increase the number of performance
indicators are you going to take back something from
the inspection program.
MR. APOSTOLAKIS: They need some sort of
guidance. If you increase the number of performance
indicators, what else are you decreasing?
MR. SATORIUS: Well, and I think it's our
view, it's the staff's view that this creation of
adding reliability and unavailability -- because at
one point in time we were looking are reliability and
an unavailability PI so that industry was, quite
frankly, balking because they were looking at going
from 4 PIs to a dozen, as many as a dozen. And there
was reluctance. And I think it was quite an
innovative fix to come up with this addition to where
you come up with really not increasing the actual PI
burden significantly on industry.
MR. APOSTOLAKIS: Okay. This is not part
of the oversight process yet, right? This is still
research?
MR. SATORIUS: Oh, yes, this is a pilot.
And during the pilot program, George, which we would
intend to be about six months in length or longer,
depending upon the other PIs and the other program
would continue in parallel such that the pilot plants
would be burdened to not only continue to report the
information required by the current PI, but also to go
outside of that and gather this additional
information.
MR. APOSTOLAKIS: Okay. Now, one of your
objectives -- changing the subject a little bit -- was
to do -- let's see, to minimize to the extent
practicable the differences and increase the
consistency between this approach, the maintenance
rule, the PRA and the SDP.
Now, the maintenance rule, as far as I
know, doesn't use any industry averages, does it?
MR. HAMZEHEE: Mark, correct me if I'm
wrong. And, John, you guys know better.
I think here what we meant was the data
collection, some of the definitions --
MR. APOSTOLAKIS: But the maintenance rule
doesn't use industry averages. And I wonder why this
agency does one thing with industry averages and
another thing with plant specific numbers.
MR. HAMZEHEE: Here it's talking about the
industry average. It's talking about the areas that
are common to both.
MR. APOSTOLAKIS: Isn't it plant specific?
MR. HAMZEHEE: No, I understand. I am
saying what this bullet covers is those areas that are
common to the maintenance rule, to SDP and to ROP.
MR. APOSTOLAKIS: But let's go beyond the
bullet.
MR. SATORIUS: Let me give it a try.
Right now licensees are required or because of the
maintenance rule are required to collect data and
monitor the performance of systems.
MR. APOSTOLAKIS: Right.
MR. SATORIUS: And one of the pushes by
industry was to the extent that's practicable that we
make the data that they go out ad collect for the
maintenance rule be similar or the same as the data
that they would collect for this PI, that the data
that they would collect for WANO, for the data they
would collect for INPO -- so it lessens the burden.
MR. APOSTOLAKIS: It goes beyond the
bullet. Why in the maintenance rule we are happy with
the number, the baseline number -- used in its PRA and
in this context we are -- in the maintenance rule we
asked the utilities tell us what you want to put for
this train or this system. The utilities looked at
their plant specific PRA. They massaged it a little
bit according to more recent information and they said
X. We didn't ask them to go to the industry average
and do something and give us X star.
So all of a sudden we're saying we have a
major rule that everybody's hailing as being great
because it's risk informed, but is plant specific and
now we're doing this which in addition to being plant
specific evokes something that's called industry
average performance. And the philosophy here it seems
to be inconsistent and I'm wondering why.
Now, maybe it's not your job to do that.
MR. HAMZEHEE: You're right, it's not.
MR. SHACK: But he argues that that's only
a surrogate for the data that he needs.
MR. APOSTOLAKIS: No. He says
MR. SHACK: I mean we've had more
arguments here this afternoon. But I mean one of the
arguments was they tried to make -- that industry
average they talk about is part plant specific and
industry average.
MR. APOSTOLAKIS: But why --
MR. SHACK: They made it as plant specific
as they could, they just didn't have enough data to do
the whole --
MR. APOSTOLAKIS: And why did they have
enough in the maintenance rule? I don't understand
that.
MR. BARANOWSKY: George, we're --
MR. APOSTOLAKIS: We're not talking about
regulation here. It's the maintenance rule. Oh,
those Americans are doing this -- and all of a sudden
there is another major rule that says no, we're going
to do it different this time. The rest of the world
doesn't do that.
MR. BARANOWSKY: Yes, I know. There are
differences in the way the maintenance rule does some
calculations of things. They're rigorously controlled
then the reactor oversight process in terms of any
standards, so the comparability between plants and the
identification of when plants on a consistent basis
exceed an unacceptable or a point in which we should
engage them more, let's say, would be inconsistent.
And so what we're trying to do here is identify ways
to get that consistency.
We've made a number of impacts on I think
the maintenance rule, including especially the way
unavailability information is collected and how
unavailability is defined which was a significant
challenge for us, okay.
MR. ROSEN: A significant area for the
industry.
MR. BARANOWSKY: Yes. And I guess I'm a
little concerned that we're not hearing that we've
made a lot of progress here, and we don't have
perfection but we do have a lot of progress.
Okay. So let's identify the few things
that we've got to still work on, we'll be glad to work
on them.
MR. APOSTOLAKIS: No. You're
misunderstanding my comment. I'm not saying you
didn't progress. What I'm saying is that as an agency
we seem to be happy using one philosophical approach
in one major piece of regulation, the maintenance
rule, which is consistent with my example earlier of
the machines, the quality control.
We told them for your plant tell me what
this unavailability should be, but then make sure you
meet it. And the industry says fine, we'll do that.
So that was plant specific. There was no question of
industry average. South Texas didn't give a damn of
what the other plants were doing. They said this is
us.
Now we are coming with this other major
piece of regulation and we're saying well, you know,
it's nice to know what you guys are doing, but we
really want you to compare with some industry average.
And I'm having a problem with this
different philosophical approach to two major pieces
of regulation.
MR. ROSEN: It's an accommodation. I
think they've said that they even had sparse data, so
they just did what they could.
MR. SATORIUS: And, George, I don't think
-- I can't speak authoritatively on the maintenance
rule, and I think we're a little bit light on that, so
we might want to have to get back to you on that just
because --
MR. APOSTOLAKIS: And maybe we should be
asking different people, you know.
MR. SATORIUS: That's just it. And I'm
thinking maybe we ask some maintenance rule people to
be with us when we discuss this with the full
Committee in case there's a short discussion on that.
CHAIRMAN BONACA: If I understand it, you
have two more presentations to go through today?
MR. SATORIUS: Yes.
CHAIRMAN BONACA: That's on the industry
trends program. And we have an hour left. So, I
mean, we intend to cover this ground?
MR. APOSTOLAKIS: I thought we were just
discussing now.
CHAIRMAN BONACA: Well, you go into
discussion and the discussion is over.
MR. SHACK: Trending time.
CHAIRMAN BONACA: How long?
MR. SATORIUS: WE need to change two
people and can start in 30 seconds.
CHAIRMAN BONACA: Okay. Do you want to
take a break before we do that? Let's take a ten
minute break. But then that means that you have ten
less minutes.
MR. SATORIUS: We can talk quickly. This
is an update on the trends.
(Whereupon, at 4:01 p.m. off the record
until 4:12 p.m.)
CHAIRMAN BONACA: With that, let's --
MR. KRESS: It's the reasonable men are
left.
CHAIRMAN BONACA: We have two
presentations to go through.
I would like to ask members when you ask
questions to make sure we don't talk simultaneously
because our transcriber has pointed out that it makes
it very challenging for her to distinguish who said
what.
With that, this is about industry trends
program.
MR. BOYCE: Thank you. Yes. Good
afternoon. I'm Tom Boyce of the Inspection Program
Branch of NRR. And I'm going to present the industry
trends portion of this briefing.
Before I get started, after looking at the
dynamics that occurred and the level of questioning
over the last three hours I had two thoughts. The
first is I'm glad my colleagues went first and the
second is, given we only have 50 minutes left and it
was an average level of abuse, we won't lower the
average too much if you under perform in this last
hour.
I would like to set a framework, a mindset
for you. We're no longer at the plant specific
threshold level. What we're trying to do is take all
the indicators that we have at the plant level,
aggregate them into an industry average. And that was
alluded to earlier, but it's not an industry average
like we talked about in terms of this equation. It's
an overall level of performance that may or may not be
directly related to the baseline values you saw here.
And I'll get into that in a moment, but I wanted to
just -- it's easy to lose sync between a plant
specific indicator and a plant specific threshold
versus an industry indicator and an industry
threshold.
So with that also I'll be followed by Dale
Rasmuson of Research, and he'll talk more about
threshold development right after I talk about an
overview of the industry trends program.
This is what I'll be talking about. I will
skip going over each bullet. Just let you know it
follows the normal path of introduction and conclusion
at the end.
As background, improving industry trends
contributed to the decision to revise the reactor
oversight process. This is about 1998/1999 time frame
and simultaneous the NRC's strategic plan was revised.
And in that revision was included a performance goal
measure of no statistically significant adverse
industry trends in safety performance. The NRC is
required to report to Congress on the state of
achieving that measure every year, and we do it as
part of the NRC's Performance and Accountability
Report.
Responsibility for this particular
performance goal measure shifted from Research to NRR
in late calendar year 2000. Subsequently NRR
developed a formal ITP in early 2001. And we
initially used existing indicators from the former
Office of AEOD, their PI program, and we also are
using the Accident Sequence Precursors, which I think
you've been briefed on several times.
We've provided two reports to the
Commission. I think you've also gotten copies of
those. They're SECY papers; one in June of 2001, one
in April of this year.
Bottom line, we have identified no adverse
industry trends to date.
And if I haven't said so, ITP is industry
trends program. Sorry if I omitted that.
MR. LEITCH: Have you identified any
positive industry trends?
MR. BOYCE: Each of the indicators that we
have monitored has shown improving trends. But we
have not -- as a negative agency, we haven't gone out
of our way to say these are all positive trends.
We've just said that we've had no adverse trends.
MR. APOSTOLAKIS: As a negative agency?
MR. LEITCH: But aren't the positive
trends statistically significant. But what I'm saying
is are they statistically significant, the positive
trends?
MR. BOYCE: Yes, they are actually. The
improvements are statistically significant, and if
you've got the SECY papers, each of the AU DPIs, and
I believe there are 7 of them, have all shown
improving trends and they've all been statistically
significant. In addition, the ASP program, and we use
the total counts of ASP events, well that's almost
statistically significant through FY 2001. I think
the P value is like .08 and a statistically
significant P value, I think, is .05.
MR. LEITCH: So I guess I'm just trying to
develop a feel then. If the industry performance had
declined as much as it has improved in the past. Are
you saying it's something -- there is an adverse trend
then?
MR. APOSTOLAKIS: Right.
MR. BOYCE: Yes, I would. Let me -- a
picture is worth a thousand words, and actually when
we get to Dale's presentation he'll be talking about--
he has some graphs and will be able to illustrate this
point I think more clearly with automatic SCRAMs while
critical.
MR. LEITCH: Yes. Okay.
CHAIRMAN BONACA: But you use certain
number of years of performance, I mean to develop a
trend?
MR. BOYCE: That's correct. Jumping to a
little bit about what Dale's going to get into, we had
to respond to the performance goal measure very
quickly, and that's why we used the existing
performance indicators. What we decided to do was go
back to as long as we had what we felt was good data
and had confidence in the data. That year turned out
to be 1998 for the AEOD indicators. I believe the ASP
program is going back to 1993.
As far as those two main purposes of the
industry trends program, the first is to provide a
means to confirm that the nuclear industry is
maintaining the safety performance of operating
reactors. And the second is by clearly communicating
industry performance, we believe we will enhance
stakeholder confidence in the efficacy of the NRC's
processes.
The industry trends program actually
complements existing NRC processes. AS I've described
earlier, the reactor oversight process takes a look at
safety of plants on a plant specific basis. What
we're doing is aggregating the data and trying to look
for the big picture. Are we missing anything by
focusing on each of the 103 operating units out there.
While we're looking at the big picture if
we do discover an adverse trend in any of our
indicators, we would respond and take a response in
accordance with our existing processes for addressing
generic issues. Those process are the generic
communications process and the generic safety issues
process.
This slide shows how we communicate with
stakeholders, and it's in a variety of ways. We've
been briefing our ongoing development efforts to an
NRC industry working group that looks at the reactor
oversight process, and those have been periodic,
perhaps quarterly type of briefs.
We published the industry indicators on
the NRC's website.
There's an annual review of the industry
trends program and results at the agency action review
meeting, and we provide annual reports to the
Commission in those two SECY papers so far that I
alluded to on an earlier slide.
We also provide an annual report to
Congress and the graphs of the industry indicators
were included in the most recent report to Congress.
And these industry indicators are also presented at
various conferences with industry. The most example
is the Regulatory Information Conference this past
March.
This slide depicts some of the concepts
that we used when we developed the ITP. We tried to
adopt a hierarchal approach. I had alluded to earlier
that we used a qualified set of indicators in
reporting to Congress. And the reason we use that term
is, actually we ran into a situation where we had just
a multiplicity of potential indicators for use in the
program. And so the hard part was identifying what
was the correct level of reporting and what indicators
acted as, if I could use the term, macroscopic type of
indicators that would give us good insights across the
spectrum and were not so detailed that we might be
missing something.
If we do find a problem in those
macroscopic indicators, we would use all the other
multiple indicators that we think are subordinate to
investigate why we got an up-tic in the more
macroscopic indicator. And I'll try and show you this
on the next slide a little bit more what I'm talking
about.
WE did use these existing programs. And
what we're trying to do is flush out these -- I say
these existing indicators. We're trying to flush out
these existing indicators and make them give us
insights in all the cornerstones of safety. Right now
we have seven cornerstones of safety. Much of the
previous discussion this afternoon focused on the
initiating events and mitigating system's
cornerstones. We are also trying to develop industry
level indicators for the other cornerstones such as
occupational radiation exposure, public radiation
exposure, physical protection and that sort of thing.
WE're trying to flush that out by deriving
these indicators from the information that was
submitted for the plant specific reactor oversight
process, performance indicators. And we've also
tasked research to update some of the studies that,
again, were alluded to earlier this afternoon such as
initiating events and reliability studies.
This slide is intended just to illustrate
the concept of hierarchy of potential indicators.
What we're trying to get to is representative industry
risk here. But where we are is in terms of our
thinking is right here. And three hours of discussion
this afternoon focused on right here in plant level
risk. And the challenge is to aggregate 103 units into
something that's representative of industry risk
without causing a distortion of the indicator.
If there is a problem up here, we can go
down and break the indicators into their constituent
parts right here. An example might be for initiating
events we have reactor SCRAMs. Well, if you have up-
tic in reactor SCAMs, you got to take a look at the
cause. Do you have an up-tic due to automatic SCRAMs,
do you have an up-tic due to manual SCRAMs. Is the
cause due to loss of off site power, or is it due to
instrument air issues, is it due to steam generator
tube ruptures. You get into that. And so the
question becomes do you want to track all these
subordinate initiating events or can you stay with
that one roll up indicator of SCRAMs. So, that's what
this is intended to illustrate.
And if you got any questions on this, I
borrowed it from Pat. So I'm going to make Pat answer
the question.
MR. WALLIS: Well, I guess someone's going
to ask eventually how the things that we worried about
recently fit into this pattern? The Sumner and
Davis-Besse Is that human error or where does that fit
into the --
MR. BARANOWSKY: Those would normally show
up -- there's actually one more thing that's not shown
here, and that's the accident sequence precursor
program, which is another slightly different way of
trying to capture performance information. And we
would sum it through those. Because those things
aren't in the PRA models, they're hard to account for.
So we have --
MR. LEITCH: But those are the things
which obviously we worried about most in the recent
times.
MR. BARANOWSKY: Right. That's why the
ASP program exists and why we have things there.
MR. APOSTOLAKIS: The ASP is not an
indicator.
MR. BARANOWSKY: What's that?
MR. APOSTOLAKIS: I mean this -- I thought
you were using indicators to do this trending.
MR. BARANOWSKY: No. ASP --
MR. APOSTOLAKIS: ASP doesn't use
indicators.
MR. BARANOWSKY: It's an industry
indicator and it's been reported to the Commission for
several years.
MR. BOYCE: Yes, the total counts of ASP
events. The total counts of ASP events can be trended
from year-to-year, and that's the index that we're
using.
MR. APOSTOLAKIS: Well, yes, they could.
MR. BARANOWSKY: And not only that, I mean
it's the total counts where we can look at trends.
And I don't know, Dale, if you have something on that
in here. But we're looking at what's the right way to
present ASP information to show what safety
implications are, whether they're getting better or
worse. And we're going to see some interesting things
going on with ASP when we add some of these new events
in.
MR. WALLIS: It's not just counts, it's
the severity.
MR. BARANOWSKY: Yes, the severity's
important.
MR. APOSTOLAKIS: But if I wanted to
understand whether human performance at U.S. nuclear
plants is improving or deteriorating, or doing
anything I really have no way of learning that, do I?
MR. BARANOWSKY: Right.
MR. BOYCE: Yes, not through the industry
trends program.
MR. BARANOWSKY: That's correct.
MR. BOYCE: And the same question applies
to the reactor oversight process, because that's one
of the cross cutting issues.
MR. APOSTOLAKIS: Sure, they will be asked
the same question, don't worry.
MR. BARANOWSKY: Right.
MR. BOYCE: And the answer I believe is
that we expect that human performance errors would
manifest themselves in performance issues, such as
reactor SCRAMs or in the case of maintenance, lower
reliabilities and higher unavailabilities.
MR. APOSTOLAKIS: Yes. And at least I hope
that this expectation will be revised. Because, you
know, in the case of Davis-Besse, yes, it would have
been revealed in a medium size LOCA. That doesn't
help me at all.
MR. BARANOWSKY: Well, it was revealed
through the identification of the degradation at the
plant.
MR. APOSTOLAKIS: Sure.
MR. BARANOWSKY: And, you know, clearly
there's some relationship to human performance there.
So, any of these things that occur, common
cause failure or whatever, you can go and see whether
there's a human element involved. And if you get a few
hits on these things, that tells you, Ops, that's an
area to focus on. And I think that's one of the things
that's been going on in the oversight process.
MR. SATORIUS: Yes, that's right.
MR. KRESS: An ASP then is one that would
have a conditional CDF of 10-3.
MR. BARANOWSKY: Six.
MR. APOSTOLAKIS: Six.
MR. BARANOWSKY: Or greater. Or greater.
MR. KRESS: There is no CDF of 10-6.
MR. APOSTOLAKIS: Is somebody doing the
Davis-Besse thing, the ASP, or you've not yet --
MR. BARANOWSKY: No, it's in the
preliminary stages, but we're working with NRR because
it's so complicated I think we're just going to work
together on it for now.
MR. ROSEN: Isn't it true in principle
that that second level down on your diagram, plant
risk; plant 1 plus 2 all the way up to N, if everybody
had a good PRA done to the same standards and that was
updated, couldn't you just sum all those and divide by
the total number of plants?
MR. APOSTOLAKIS: Or just look at the 103
units.
MR. BOYCE: Probably.
MR. BARANOWSKY: We could do that if we
were able to trust -- the reason we disaggregate it,
we don't think we have enough trust in these
individual models to just take that one number. So
we're looking at some of the individual subelements
and it's a judgment call as to how many elements you
look at.
But, you're right. In the future one could
in theory have highly qualified PRAs and you could
track just performance at the risk level. And when
there was a problem, then you'd disaggregate the thing
down to get at the root cause in terms of areas of the
plant or issues.
MR. WALLIS: The cracked pipes aren't in
PRAs, are they?
MR. BARANOWSKY: Well, those are some of
the reasons why we have other things.
MR. WALLIS: Yes.
CHAIRMAN BONACA: But you would pick up
that one, for example, significant events. You're
plugged into significant events as a function of here.
So that would pick up, for example, there is Besse
under that, wouldn't you? Or would you have to go
necessarily only to the precursor --
MR. BOYCE: I would think those would be
counted under both. There is slightly different
criteria that NRR uses for the significant events
program and the ASP program. I would expect they'd be
counted in both areas.
CHAIRMAN BONACA: Yes, some of the
indicators like equipment forced outages, forced
outage rate; these are pretty insightful. Average
exposure to plant.
MR. BARANOWSKY: Well, I think we're going
to look at making some changes. The old AEOD
indicators are in there because they're well developed
and understood. We're working on a number of things
that maybe have a better nexus to safety and risk.
And in the future when they're proven, I think Tom may
mention something about that, that might replace them.
MR. BOYCE: That's right. What Pat's said
is correct. We are working to be more risk informed
in as many cornerstones as we can. Those are
primarily the reactor safety cornerstones, initiating
events, mitigating systems and barrier integrity.
MR. APOSTOLAKIS: One of the things that
really concerns me is that as you said earlier, we
have a number of ways of looking at performance. We
have your program. We have the ROPs, we have the ASP,
we have -- you know -- and that's fine. I agree that
we should have a multi-pronged approach.
The thing that really bothers me is that
all of these programs are hardware oriented. And the
industry operating experience is telling us that's not
where the problems are. Now, again, that's a question
that's bigger than you, than your issues here.
MR. SIEBER: AEOD had some indicators that
had human performance factors in them where they did
the percentage comparison to causes.
MR. BOYCE: Yes.
MR. SIEBER: But those are not being done
anymore, right?
MR. BOYCE: Right. That's correct.
MR. BARANOWSKY: I think the approach
we're taking now is if it impacts a functions
availability, then we go down and dig down into
causes. It's just too expensive to have all the
causes tally and ready to go for any possible number
of things. It's a lot more practical to look at
safety functions. If the functions are okay, you don't
have to dig down. If they're not okay, then you dig
down and find out what's going on.
CHAIRMAN BONACA: Yes, the only thing is
that, I mean again the performance of active
components that the utility has focused on for 20
years may not be the best hook on human reliability.
MR. BOYCE: That's right.
CHAIRMAN BONACA: And, you know, for
example certainly, I mean issues that -- decisions
during outages may cause latent effects because you
have a comforter between, you know, the economics and
safety issues. I mean, and you have those kind of
competition, and time. I mean, we've seen it in the
root cause evaluation of Davis-Besse.
Those things, I mean right now they're not
being --
MR. BARANOWSKY: But they're going to
manifest themselves in terms of availability of
equipment or reliability of equipment.
And one that I have a concern about is
operator performance in an accident situation where we
really have nothing more than the training and the
simulators.
MR. APOSTOLAKIS: That's true, Pat. But
if you look at Davis-Besse, I mean you can't really
say, "Okay, I would catch it if the vessel were
breached." I mean, that's too late. Something like
that you can't afford to say I will wait until I see
the impact on the hardware.
MR. BARANOWSKY: Well, I think when we see
something like Davis-Besse we go through lessons
learned, and we ask ourselves what is it about our
inspection program or indicators that suggests we
should make some changes.
MR. APOSTOLAKIS: But I think you will
never make the right changes as long as you are
forbidden from getting into cultural issues and
organizational issues.
MR. BARANOWSKY: Well, I think you can
inspect that stuff. For instance, if we weren't
looking at how licensees were inspecting their heads
and other passive components and we spending a whole
lot of time worrying about the reliability of diesel
generators where we have good data --
MR. APOSTOLAKIS: Right.
MR. BARANOWSKY: -- maybe what we should
be doing is inspecting those softer areas where the
human element is important and I can't get a good
indicator or quantification that changes in time line.
MR. APOSTOLAKIS: Yes, I agree.
MR. ROSEN: Let me tell you the last
things I'm worried about, are diesel generator
performance and operator performance during
transients. What I'm worried about mostly is human
performance other than operator performance.
MR. APOSTOLAKIS: Me, too.
MR. ROSEN: Meaning the maintenance
people, engineering people, managers, supervisors and
executive.
CHAIRMAN BONACA: And, you know, it's
interesting, you know, if you look at the root cause
for Davis-Besse where you have -- you read the guys
did go and cleaned up and then they stopped cleaning
when the schedule said that's it, that's the time you
got, so they didn't complete the activities because
time was up. And, you know, now from the root causes
it's hard to understand if there was a real contention
there. If somebody said oh we should be doing more,
an somebody said no you're not going to do more. Or
if you simply everybody was in lock-step doing that.
There are many opportunities there.
But this is really typical in outages.
And you know one concern I've always had is right now
there has been the race of the industry to have
shorter and shorter outages. Okay. And I think that
those kind of pressure to have shorter outages are
going to put some pressure on some critical activity
and decisions there. Because that's what you got.
MR. BOYCE: I can only add one more thing
that will not be a disagreement with what you said,
but we did attempt to take a look at whether there was
commonality in some of the events that we were seeing.
And so in the first SECY paper that we did we took a
look at issues that were what were called greater than
green and we tried to look at some of the factors like
what was going on at the time. Was it at power, was
it shutdown, what systems were involved and what the
apparent cause was. And as you know, we do follow up
inspections every time you cross that green white
threshold. So we had some data that we could go look
at.
And we concluded, and all the results are
in there, that was not sufficient commonality to be
able to draw any conclusions or have any reasonable
indicators that would tell you anything in advance.
Anyway, I'm not going just make that
statement, but I'm not going to disagree with your
premise.
CHAIRMAN BONACA: And I'm not saying it is
an easy thing to do I'm only saying that somehow we
have to get there.
MR. APOSTOLAKIS: I think there is one
area where we are reluctant to get into because it's
not thermal hydraulic structure or mechanics, or
reactor physics, and that's the soft area
organization, cultural. We keep getting those
messages from the plants that that's where the
problems are and we're not getting into that for some
reason.
Now you might say, "Well, gee, tell me
what to do." I don't know what we should do. That's
why there's an Office of Research. But, you know, we
need to spend money doing more work on performance
indicators that deal with hardware, because that's
where we know what to do.
But this is a really different subject.
I mean we should --
MR. ROSEN: Yes, we are going to move on.
MR. BOYCE: All right. This lays out in
detail the process for industry trends. And
essentially what we're trying to do is use statistical
techniques to identify the adverse trends using
indicators that have been qualified for use. And what
this does is is you just fit a trend line to it to
whatever the indicator is and in general, based on
what we've had so far, if the trend line is pointing
down or is flat, you do not have an adverse trend. If
the trend line is going up, you have an adverse trend
in general.
We've also got a statistical technique
where we look for one year deviations from the norm,
and it's called prediction limits. And I don't want
to get bogged down into it. But recognizing that we
started in 1988, it would take a significant trend to
give us an adverse trend, and that's why we used the
prediction limit method is to look for that short term
trend. We're not calling that an adverse industry
trend, we're calling it we exceeded the prediction
limit and we will investigate it. It's a bit of a
nuance, but I wanted to point it out.
If we identify an adverse trend, the next
step is to evaluate the underlying issues and their
safety significance. And then based on that safety
significance, take the appropriate agency response in
accordance with our existing processes.
And finally, there's an annual review by
the senior managers of the agency at the agency action
review meeting.
Just to give you snapshot of the results
of the program today, as I said there's no
statistically significant adverse trends identified in
fiscal year 2001.
In looking at the indicators that we're
deriving from the ROP data given that the ROP started
in April of 2000, we did not have sufficient data to
do long term trending. However, an inspection of the
data that's been submitted so far, and there's 18 of
those indicators, didn't reveal any significant
issues.
In the most recent SECY paper you'll see
examples of initiating event indicators that the
Office of Research has developed. Those are
essentially an update of the initiating events
indicators in NUREG 5750. We took a look at those,
and again those statistically significant adverse
trends were identified.
MR. APOSTOLAKIS: I think the issue of
statistical significance, it is meaningful it seems to
me to talk about statistically significant trends when
you talk about indicators that are a fairly low level
if I take, you know, what you showed earlier about
risk. This is core damage and I'm way down there.
Again, statistically significant trends make sense.
I wouldn't apply this criteria to core
damage. I'm not going to say, "Look, I only had one
core damage event last year out of 103 plants, that's
not statistically significant, right?" The higher I
go the notion of statistically significant becomes
less valued, it seems to me.
MR. BARANOWSKY: Yes, but the NRC has some
performance standards that say that's unacceptable.
MR. APOSTOLAKIS: For low level
indicators.
MR. BARANOWSKY: No, for like core damage
frequency. We don't say if we get a statistically
significant one. No, we say none is the accepted
performance level.
MR. APOSTOLAKIS: But you're right.
My question is let's say next year nothing
else changes and the only piece of information in 2002
is Davis-Besse. Are we going to still conclude that
the NRC says that there is flat -- or the indicators
show that there are flat or improving trendliness,
trend lines, and this and that? In other words, is
that an outline there that doesn't effect any of our
conclusions?
MR. BOYCE: Well, the ASP program does in
fact, there's actually two outputs in the ASP program.
MR. APOSTOLAKIS: But that --
MR. BOYCE: Actually, there's a second
performance goal measure that uses the ASP program.
And it's in the strategic plan. And it says no more
than one event that exceeds 10-3 delta CDF. So
there's a -- okay. So that's an example of a
threshold base criteria.
We're looking at a trending base criteria.
The ASP program would pick that up because it --
MR. APOSTOLAKIS: But not this program?
MR. BOYCE: Well, that's correct.
MR. APOSTOLAKIS: Because this relies on
AEOD?
MR. BOYCE: Because the index that we're
using for ASP counts total number of ASP events. So
this program would not pick up that significant event.
MR. APOSTOLAKIS: So I wonder whether we
could still claim next year that the industry's
performance is improving? If I have one ASP event, as
10-2.
MR. BARANOWSKY: Well, you have to be
careful about talking about the industry versus a
single licensee that's had a problem. I mean, that's
part of the assessment that I believe you would do,
Tom. If some one or two events occur, you want to
know is this an industry problem or a one plant? You
know, just because one kid's chewing gum, we don't
make the whole class stay after school.
MR. APOSTOLAKIS: That's true. But, I
mean, you know the caution should be both ways. One
is to generalize and say, gee, this is an industry
problem. But also I think you should be cautioned not
to say this is an isolated event too quickly.
So let's do it both ways.
MR. BARANOWSKY: Well, I mean, that part
of their assessment will be whether or not we have
weaknesses in the way we either inspect our licensees,
implement program. And if it's generic, we'd have to
say we have a generic issue that was identified.
MR. WALLIS: Well, the generic issue would
probably be symptoms which because for various reasons
which you can go to at Davis were ignored.
MR. BARANOWSKY: Yes.
MR. WALLIS: And other places where
symptoms are being ignored, maybe because they're not
in the PRA or something.
MR. APOSTOLAKIS: They are not in whatever
we're doing.
MR. SIEBER: Well, one of the interesting
things with performance indicators is as soon as you
establish them, people manage them.
MR. BOYCE: Yes. Right.
MR. SIEBER: And other things were thrown
off that your performance indicators --
MR. ROSEN: George, I think we'll get to
this discussion on Saturday I think when we talk about
our response to Davis-Besse. In particular, I want to
be sure I address the culture issue and how we use
Davis-Besse as an example of where we don't want to
go.
MR. APOSTOLAKIS: Exactly.
MR. ROSEN: And why the regulatory system
needs to do something to respond to those culture
issues which I think is at the root of the Davis-
Besse. And then we define that as corrective action,
system failures and a lot of other kinds of failures.
CHAIRMAN BONACA: Well, it's interesting.
I mean, there -- criteria that were used there. Like,
for example, the pressure was -- on the flanges. The
pressure was simply through the start. So therefore,
how many flanges can you fix in these many hours, and
the answer was --
MR. ROSEN: Rather than the opposite
approach, which is to shut the plant down until you
fix all the flanges, clean the head up completely,
make sure you have no degradation and restore the
initial design basis.
CHAIRMAN BONACA: For the prospective of
a regulator, however, the question then comes should
it be an option left to the corrective action program
or should it be a requirement that if you have leakage
from the head or from somewhere in the primary system
you're going to fix it before you start.
If you have that, you're helping the
technician whose doing the work up there who doesn't
have to stop and question whether or not he should go
beyond the point.
So the issue is broader. There are things
that really help facilitate the process if you take
the decision away from the hands of some intermediate
management or management.
MR. ROSEN: I don't want to be here three
years from now with another plant, XYZ plant, that's
had a serious incident, maybe even an accident, whose
root cause was the same kind of safety culture
deficiencies that happened at Davis-Besse.
MR. APOSTOLAKIS: Yes, of course.
MR. ROSEN: And that we didn't do
something different. That we just saw Davis-Besse,
knew what the root cause was and safety culture and
said "Okay, we'll just keep doing the same regulatory
stuff we have now."
CHAIRMAN BONACA: Exactly. Exactly.
MR. ROSEN: Because what that is is an
embodiment of the commonest definition of insanity,
right? Doing the same thing over and over and
expecting different results.
MR. APOSTOLAKIS: I'm with you. I'm with
you.
MR. BOYCE: Let me press on.
CHAIRMAN BONACA: Yes, let's press on.
MR. BOYCE: One of the problems that we
identified early on was with the indicators that we've
looked at so far, given that we started in 1988, the
indicators all have like an expediential type of
curve. And it appears, just visually, that they might
be approaching an -- and if you get to that point, it
would be very easy, particularly if we shortened the
period of time that we were looking at, to get just
almost like a random up-tic in the indicator. And by
our process we would then start reporting events to
Congress that may or may not have safety significance.
So what we are attempting to do is
establish thresholds for reporting and thresholds for
monitoring and actions so that -- an example would be
SCRAMs. In 1988 we were on the order of about 3
SCRAMs per plant per year. The most recent or Fiscal
Year 2001 we were at .85 SCRAMs per plant per year. So
if we go up to 1 SCRAM per plant per year, is that
much less safe or are we maintaining safety at one
SCRAM per year. And that's the philosophical issue
you face.
And so we are trying to have a rational
basis to establish thresholds where we would monitor
what's going on below the threshold so that we
wouldn't ignore an emerging problem, but at the same
time we would only be reporting issues of safety
significance to the appropriate stakeholders.
All right. So that's the problem. The
Commission helped us out with an SRM directing us to
develop risk informed thresholds as soon as
practicable. We've asked Research to help us out with
that for the appropriate cornerstones and indicators.
We're going to take on the other cornerstones, NRR is,
for like occupational radiation exposure, physical
security, etcetera.
We'll engage the appropriate stakeholders
including this body as we develop them.
The near term indicators that are concern
we think are the ROP indicators because we think they
will have in excess of 4 years worth of data which we
have somewhat arbitrarily said are our threshold for
starting to use these indicators.
The nuance here is to start the ROP in
April of 2000. We asked licensees to submit
historical data, but that data was best estimate type
of data, so we're taking a hard look at that data and
seeing if we can use it. And so that looks like the
near term issue.
If we get these thresholds, we would then
move we think towards a different performance measure,
one that was more oriented towards crossing thresholds
such as the ASP program rather than the totally based
on trends in the indicators.
And with that, I'll turn it over to Dale
Rasmuson whose got some ideas on developing
thresholds.
MR. RASMUSON: I will try to move fast
here, just pick out some selected slides. I did
colored ones because I know George likes colored
slides.
We wanted to share with you our thoughts.
We've just started on this process --
MR. APOSTOLAKIS: You guys are taking
abuse, but you are giving it out, too, you know.
MR. RASMUSON: Oh. Okay.
MR. APOSTOLAKIS: Give and take.
MR. RASMUSON: We wanted to explain some
of the technical approaches we've identified for
generating information for thresholds and to receive
any ideas or suggestions that you may have as a body.
One of the points I want to make is the
second bullet here, is that industry thresholds differ
from plant-specific thresholds. And to illustrate
that, for instance, we'll stay with the unplanned
SCRAMs. In the ROP process, the green/white threshold
is 3 SCRAMs. When I look at the performance on an
industry basis, we're down at about .6 SCRAMs per
plant for average for automatic SCRAMs. And when I
get a trend in this type of fashion here, if I were
going to use three, I'm clear up here. To set a -- I
would probably think that I would be down here for the
industry in some sense.
And so those are the types of things that
we're working and focusing on.
MR. APOSTOLAKIS: Now, have you read the
ACRS letter on the reactor oversight process of last
October?
MR. RASMUSON: I have not.
MR. APOSTOLAKIS: You should, because we
have attacked this threshold.
MR. RASMUSON: Right.
MR. APOSTOLAKIS: We don't like it at all.
MR. RASMUSON: Yes.
MR. APOSTOLAKIS: There's no sympathy
here, you know. Three means nothing.
MR. RASMUSON: Right. Right. And all I'm
trying to do is use that for an example here. Three
may be meaningful for a plant, but for the industry
average when we're taking industry behavior, we tend
to -- you get average behavior and so forth. And so
the threshold probably ought to --
MR. APOSTOLAKIS: And what we said in that
letter is that even for a plant --
MR. RASMUSON: Right.
MR. WALLIS: Well, just the average is 95
percent. And that's more significant than the
average, and that's always way below one or it's
around one.
MR. RASMUSON: I don't know. I haven't
looked at --
MR. WALLIS: I mean, I thought you had it
on your picture there.
MR. RASMUSON: Well, that was a prediction
-- right. And I don't know whether I would use that
value or not. We're looking at two types of
thresholds, one an action threshold and an early
warning threshold.
The action threshold would be sort of the
one that would be used for reporting to stakeholders.
The early warning one would be used as a tool for
helping us in the industry here to monitor performance
and to take some action as we start to see some upturn
in the process.
MR. APOSTOLAKIS: Now, why can't the
reactor revised oversight process do that? Early
warning? Marshaling my resources and then have
something else that says "Boy, am I in trouble."
MR. SATORIUS: I would argue that that's
exactly what the ROP does, George.
MR. APOSTOLAKIS: It doesn't do that.
We'll see that. The action matrix doesn't say things
like that. It may be doing it --
MR. SATORIUS: With the green/white
thresholds and the white/yellow thresholds, and the
inspection findings manifest themselves into colors
that force -- or produces more inspections and further
review by the staff, I would say it does exactly that.
A measured approach.
MR. APOSTOLAKIS: If you move the
thresholds--
MR. SATORIUS: It's a graded approach.
MR. SIEBER: You can have an increasing
industry trend and not change colors on anything at
any specific plant. But something's going on and I
think that's part of regulation to figure out why is
this uptake occurring.
MR. SATORIUS: Tom, you want to answer
that?
MR. BOYCE: Yes, that's the point of the
ITP is to catch those sorts of things.
MR. SIEBER: And outcomes that would be --
or something like that that it says we've looked at
this and analyzed it and here's steps that the
industry ought to take.
MR. BOYCE: Right. And we would follow up
with inspections if appropriate.
MR. SIEBER: Right. So there's a
difference to me, anyway, between the ROP and an
industry wide effort.
MR. APOSTOLAKIS: We'll come back to that.
MR. BOYCE: I have every confidence you
will at some point.
MR. RASMUSON: Okay. Threshold
characteristics. Here thresholds should have a
rational basis, they should be practical, they should
be conceptually simple, they should be consistent with
the existing regulatory framework and they should
reflect risk, safety and regulatory perspectives.
Those are sort of the criteria that we're working
towards.
Our protocol that we're proposing to use
for this is to develop risk and statistical
information related to the trends for input to an
expert panel, provide associated safety and regulatory
information to the expert panel, and then the expert
panel would set the thresholds based on the inputs.
Inputs for the expert panel would be some
of the things, we could start with the values set for
the ROP indicators. We'll look at values from the
risk based performance indicator report, those if
they're applicable. Other risk insights. We'll look
at the current industry performance, the trends,
estimate some of the characteristics from that using
various statistical methods such as prediction
intervals, Bayesian predictive distributions and
different things like that can help us to give some
insights.
Using these we will select some values and
that. We'll evaluate the risk implications of some of
these things is applicable using some selected SPAR
models to just give some idea of what the risk is on
these. And then suggest the values to the panel.
Technical approaches we're looking at are
prediction limits, Bayesian predictive distributions,
percentiles from industry distributions, insights from
PRAs, rate-of-change of the trend, expert panel input,
modification of current ROP thresholds.
There, for instance, some people have
suggested that we take a percentage of, say, the
green/white threshold and use that for an industry
trend threshold.
Integrated risk measure concept being
developed in the enhanced PIs that was talked about
today. That's an idea that we may use for some of
these, but we'll see how that develops and comes
along, and that. That may be applicable for rolling up
some of the indicators like initiating events or
something. Or a combination of all of those.
Technical questions that we are
considering are how many years should be included in
a trend. For instance, if we look at this one here,
certainly this trend right here is certainly driven by
the early years. If we take something in a shorter
interval, that model changes and the prediction
interval changes there.
MR. WALLIS: With all of these things fit
a very simple expediential --
MR. RASMUSON: This particular model here
is strictly a linear one. This is an expediential
model here.
MR. WALLIS: Yes.
MR. APOSTOLAKIS: So if they look at the
last 6, 7 years, they really have a horizontal line,
don't they?
MR. RASMUSON: They have a horizontal
line. And so that's one of the questions that we're
interested in looking at there.
What level is appropriate for reporting to
Congress.
What level is reported for the agency
action to an adverse trend or the start of one.
How should some of the PIs be grouped. We
have initiating events, should we group all of those
into one over -- say super indicator or something and
use that for reporting to Congress and then maybe
trend some of the other things that maybe don't occur
as frequently, such as steam generator tube ruptures.
Certainly we can have information from looking at some
of these lower level initiating events or
characteristics that can help us an agency.
How does the safety goal influence setting
thresholds or should it?
And should the concepts in Reg. Guide
1.174 be used in setting thresholds, if they're
applicable.
Those are some of the questions that we're
kicking around right now and moving forward on. And
so those are the types of things that we're doing.
MR. BARANOWSKY: So it's early and we've
just really started this, but there's a lot of
questions that we've raised and we're going to be
soliciting information. And I think the expert panel
approach is something that we wanted to mention here.
MR. WALLIS: One thing that will keep you
alive is if you have a requirement that every year you
introduce one new PI and discharge one old one.
Because existence of PIs themselves conditions the
sort of behavior in management of a plant to some
extent, which means that something else may be
forgotten. So if you force every year to bring in a
new PI, you have to think about what's important that
you haven't been trending.
MR. KRESS: You've got to have four years
for a good trend, so by then it's --
MR. WALLIS: Yes.
MR. SHACK: I missed, what was the time
frame for this? I mean, when do you plan to have at
least a preliminary version of this in place?
MR. BOYCE: Well, of course, the AEOD
indicators and the ASP program are ongoing.
Personally I'd like to do it sooner rather than later
because I think it's just a question of when we get an
up-tic that we'll be explaining to Congress and having
to fully understand the safety significance, which
isn't easy when you're just dealing with numbers being
submitted.
And so I'm asking research to go as
rapidly as possible. I think it's going to take maybe
to the end of FY 03 before we're able to do it.
MR. SHACK: It's that kind of time frame
you're talking about?
MR. BOYCE: Yes.
CHAIRMAN BONACA: This was actually on the
line of -- you know, when you come up next week it
would be interesting to have a time line for -- you
know, you mentioned before, Pat, that you were
planning to have a couple of more updates to us as you
get progress going.
MR. BARANOWSKY: Yes. Yes, we were going
to put together some tentative dates on what would
happen with that. But our plan is to come back here
at least a few times before to change this up.
MR. APOSTOLAKIS: Actually, I was about to
recommend that once you have, you know, a couple of
ideas as to how to proceed on each of the questions,
it would be a good idea to have a Subcommittee
meeting. I mean, if you are interested in getting our
feedback. Because if you get it after you invest a
year and a half into it, it's kind of late.
MR. BARANOWSKY: Okay. I think we want to
look at some concepts, try some things out showing you
what it looks like. And then show you what it looks
like and solve, and you tell us what other problems we
missed.
MR. APOSTOLAKIS: So you're going to show
us the approximations?
MR. BARANOWSKY: Yes.
MR. SATORIUS: We could probably have some
of those dates, Pat, I think put together for the
Committee briefing next week, don't you think?
MR. BARANOWSKY: I think so.
MR. SATORIUS: Okay.
MR. RANSOM: IS there anything to be
learned from other maturing industries, you know, in
terms of what could be expected? I mean, because
there are obviously reflecting learning curves that go
with time and I think there is quite a bit of
information available, isn't there? My idea would be
you never can set zero as the goal, so there's got to
be some realistic expectation on where these things
can go.
MR. BARANOWSKY: But there's two things
that happen. It's true that people manage the
indicators. I mean, that's a well known thing.
And the second thing is lessons that were
learned long ago that caused people to fix things tend
to be forgotten after a period of time when they don't
appear to be important anymore. Sol those kind of
things have to show up.
But managing the indicators is an issue
for sure.
MR. APOSTOLAKIS: Does the airline
industry have anything like this?
MR. SATORIUS: Or the railroad industry?
I mean, that's probably --
MR. BOYCE: I don't think we've taken a
really hard look at that. You know, some of this,
again we're only a year and a half into this program.
Some of it is you got to be able to be consistent with
like WANO indicators, which worldwide plants --
everybody contributes to that system. And so if we
come up with new ideas, it's got -- we're trying to
work within the worldwide framework. And INPO has got
its ideas. And so we're trying to work so far within
that sort of existing framework. But I think there
might be some value in doing a compare and contrast.
MR. WALLIS: I'm not being factious about
bringing in new indicators. It seems to me that after
a while there may be so few unplanned SCRAMs it
doesn't mean anything, it makes no sense anymore. And
something else is going to be much more significant
and you ought to look at it.
MR. RASMUSON: For example, I mean this is
sort of the yearly distribution of unplanned SCRAMs.
And starting over here you can see how it's quite flat
and here that it's tightened up.
Another way of looking at that is to look
-- just plot it by year. This is the total number of
plants. This one right here is plants with two or
more SCRAM. This one is with one SCRAM only, and this
is with no SCRAMs. And you can see how the industry
has improved in that.
I mean, just looking at some of these
things you sort of see as you get in and cut the data
a little bit different, you sort of see that, hey, you
know, the industry has learned some things. And maybe
we ought to replace this one. I don't know.
MR. SATORIUS: From a historical
perspective, we choose SCRAMs and unplanned power
changes because our history had taught us from
monitoring these plants in the past that typically
plants that are changing power often or are SCRAMing
often have other problems, whether it be maintenance,
whether it be operational. But they were good
indicators of plants that were having problems.
MR. ROSEN: Did Davis Besse change power
often?
MR. SATORIUS: I don't know.
MR. APOSTOLAKIS: See, that's why Graham's
point becomes irrelevant now. Because now we have to
look for a new indicator.
MR. BOYCE: Yes. I would just add -- I
mean, I think that's an intriguing thought. I would
just add that at least as we're bringing on 18 ROP
indicators on line at the industry level. And so
those are still relatively new. So, you know, at
least for a couple of years I think we're bringing in
new stuff.
MR. APOSTOLAKIS: I think there is no
question that for the indicators on which we have
focused in the past the trends are the right ones. So
what we're saying now with the new incident is, you
know, do we have a complete set. Is there something
that we're leaving out.
MR. WALLIS: And are there some other
possible indicators for which the trends are bad?
MR. APOSTOLAKIS: Yes. Exactly. Exactly.
MR. BOYCE: And then the other point I was
going to add, remember this is a voluntary program for
the ROP PIs to submit data. And we don't have
unlimited access to unlimited data.
MR. APOSTOLAKIS: Yes.
MR. BOYCE: And so anything we do, you
know, we're relying on existing sources because it's
hard to justify just for the sake of data asking
licensees to collect and submit it. And so we're
reviewing LERs as one of our sources of data. We're
trying to leverage the ROP PIs, and we're trying to
also go to the EPICs IMPO realm.
But all I'm doing is telling you some of
the problems that we're facing. I think your idea is
intriguing in trying to change them.
MR. WALLIS: I think the licensees
themselves should be sources of PIs. It doesn't have
to be something you know about. I mean, if they
observe themselves that something else is a better
indication of the state of their plant, they ought to
be willing to upgrade it to a PI eventually.
CHAIRMAN BONACA: And they trend a lot of
things and you can really see what's going on.
MR. ROSEN: Open corrective maintenance
items that don't require an outage to correct. That's
just a measure of what they're not getting to, even
though they could. I mean, there's a whole lot of
different things that licensees watch that are not in
this program.
MR. SATORIUS: I'll add, too, that we
provided a paper to the Commission I think in the
January time frame where we had gave some historical
perspectives and also acknowledged that we've worked
with some of our international colleagues and looked
at some of the performance indicators that some of our
international colleagues are tracking. And they look
at things, Steve, similar to what you had noted
yourself that are outside of the scope of what we're
looking at. And we're mindful of those and are aware
of those, and work with NEI and industry on developing
different looks and different PIs.
MR. KRESS: I think your threshold idea is
good because you don't want to be reporting things
that aren't statistically significant trends.
I think the words "statistically
significant" ought to be a strong input into your
thinking on thresholds, which implies to me you need
a distribution of the trends and use standard
statistical measures of what's a significant change in
a distribution.
I'm very skeptical of trying to tie it to
risk at all for these things. I think you're going to
have the same problem we had with the plant specific
ones.
I think safety goals would be useless to
you here. I can't see any way you can factor them
into your thinking at all.
So, I would think in terms of statistical
significance based on actual data and get a threshold
that is from your thinking there that says this thing
is beyond what we expect for the random variations and
it wouldn't necessarily risk significant at all, but
it's an indicator of statistical change in the
industry's distribution.
MR. BARANOWSKY: One of the other things
that we were thinking about doing, you know with the
SCRAMs right now we have like, say three is the ROP
number. But not all SCRAMs are equally important.
And the three is based on them having about the same
importance. So look at this list of initiating events.
You have a wide variation in the safety
significance of these things. The indicator might
somehow take into account and you may end up coming
with a different type of indicator that drops it down
a notch to catch the most risk significant ones, and
that's the kind of thinking we're trying to do.
MR. APOSTOLAKIS: Right.
MR. BARANOWSKY: Which changes the
indicator, as you were saying, from a simple a beans
to something else, although it's in essence trying to
count the same thing.
MR. APOSTOLAKIS: I think our letter on
the ROP, though, is relevant. So maybe you should
consult it. In October, was it?
CHAIRMAN BONACA: So how much time do we
have allowed for doing this ACRS meeting?
MR. CRONENBERG: An hour and a half I
believe.
CHAIRMAN BONACA: One hour and a half. So
clearly you would want to take -- well, handle this
equation. I don't know what the agreement was. I
mean, if you bring them up, there have to be a way to
deal with them. We don't --
INTERVIEWER: Yes, sir, we understand our
challenge.
MR. KRESS: You know, everybody's here but
two members. And I don't think you'll get the same
question on those. We've already -- so I think you'll
be all right with it.
MR. BARANOWSKY: I mean, this was an
information briefing, both of them, by the way.
Because we have quite a bit of work to do as we go on
the road. So --
CHAIRMAN BONACA: So we will expect a
letter.
Any other questions of this program? That
was an interesting update, and I appreciate it
personally.
If not, the meeting is adjourned.
(Whereupon, at 5:10 p.m. the meeting was
adjourned.)
Page Last Reviewed/Updated Monday, July 18, 2016