United States Nuclear Regulatory Commission - Protecting People and the Environment
Home > NRC Library > Document Collections > ACRS > Meeting Schedule and Related Documents > 1999 > Plant Operations - January 26, 1999

Plant Operations and Reliability and Probabilistic Risk Assessment

                       UNITED STATES OF AMERICA
                        U.S. Nuclear Regulatory Commission
                        11545 Rockville Pike
                                           Room T-2B3
                        Rockville, Maryland
                        Tuesday, January 26, 1999
         The subcommittee met, pursuant to notice, at 8:30 a.m.
         JOHN BARTON, Member, ACRS
         MARIO FONTANA, Member, ACRS
         THOMAS KRESS, Member, ACRS
         DON MILLER, Member, ACRS
         DANA POWERS, Member, ACRS
         ROBERT SEALE, Member, ACRS
         WILLIAM SHACK, Member, ACRS
         GRAHAM WALLIS, Member, ACRS.                         P R O C E E D I N G S
                                                      [8:30 a.m.]
         DR. BARTON:  The meeting will now come to order.  This is a
     joint meeting of the ACRS Subcommittee on Plant Operations and
     Reliability and Probabilistic Risk Assessment.  I am John Barton,
     chairman of the Subcommittee on Plant Operations.  Dr. George
     Apostolakis is chairman of the Subcommittee on Reliability and PRA.
         ACRS Members in attendance are Mario Fontana, Thomas Kress,
     Don Miller, Dana Powers, Robert Seale, William Shack, Robert Urich, and
     Graham Wallis.
         Also in attendance is Mario Bonaca, who has been selected as
     a new Member.  He is expected to be appointed to the ACRS in the near
         The purpose of this meeting is to continue the subcommittee
     reviews of proposed improvements to the NRC inspection and assessment
     programs, including initiatives related to development of a risk-based
     inspection program and performance indicators.  Subcommittees will
     gather information, analyze relevant issues and facts, and formulate
     proposed positions and actions as appropriate for deliberations by the
     full Committee.
         Michael T. Markley is the cognizant ACRS staff engineer for
     this meeting.
         The rules for participation in today's meeting have been
     announced as part of the notice of this meeting previously published in
     the Federal Register on December 21, 1998.  A transcript of the meeting
     is being kept and will be made available as stated in the Federal
     Register notice.
         It is requested that speakers first identify themselves and
     speak with sufficient clarity and volume so that they can be readily
         We have received no written comments or requests for time to
     make oral statements from members of the public.
         I note that we've got Bruce Mallett on video from Atlanta. 
     Bruce, can you hear us?
         MR. MALLETT:  I can hear you fine.  Can you hear me?
         DR. BARTON:  Yes, we can.  And I also understand that you
     have another commitment, and we lose you at what time?
         MR. MALLETT:  I have to be out of here by ten o'clock.
         DR. BARTON:  All right.  So at this point we'll proceed with
     the meeting, call upon Frank Gillespie, Mark Cunningham -- Mark's not
     here -- who's got the lead here this morning?
         MR. JOHNSON:  Frank.
         MR. GILLESPIE:  I do.
         DR. BARTON:  All right.  Do you intend, since we're going to
     lose Bruce, we're going to kind of get his inspection stuff tied in
         MR. GILLESPIE:  Yes.
         DR. BARTON:  Since we did meet in December, and sent a
     letter up, the things that we were interested in that you were still
     working on at that point I believe was the work on the assessment
     process.  So if we can spend some time on that today, the transition
     plan, and hopefully can answer some questions on the package that you're
     asking the Commission to proceed with has got some, you know, future
     work needed, some holes in it, how you plan to address those issues. 
     We'd be interested in that also.
         So at this point, Frank, you've got the show.
         MR. GILLESPIE:  In order to get Bruce on pretty quickly, and
     we're not going to repeat everything we said the last time, we've got
     that fat package out, and it pretty much lays out all the details.  So
     rather than go through all the viewgraphs we have here, which are kind
     of a subset of the Commission viewgraphs we use, there is one on the
     concepts.  We're going to I think just get right into it.
         Our one big hole we identified with the Commission just to
     highlight it, the Commission said that they're interested in, is a
     risk-informed screening measure for dealing with inspection results that
     aren't really conducive to quantification.  And we're still working
     that, and Alan Madison's going to be here a little bit later to go over
     the details of that.
         DR. BARTON:  All right.
         MR. GILLESPIE:  And that was probably the one biggest policy
     hole that we had I think when we addressed this with the Commission that
     they were looking forward to seeing in March.  And probably one of the
     more -- I don't want to say intellectually, academically challenging
     things to do is to come up with something that the inspector in the
     field, who's basically only had a two-week risk course, can apply and
     will get a meaningful screening process out of.  So Alan will come. 
     We've made some progress on that.  And he'll be explaining that a little
         So with that, rather than go through all of these, the
     schedule is there, our transition plan is there, we'll hit that at the
     end.  We're on kind of a tight schedule in that we're going to try to
     have eight pilot plants started in June.
         DR. BARTON:  These are six-month pilots?
         MR. GILLESPIE:  Yes, we're going for six-month pilots.  We
     feel we need at least a half a year to just exercise the system and work
     the bugs out of it, that just doing a three-month pilot isn't going to
     give us a chance really to observe it.
         DR. POWERS:  I guess the question that came to my mind was
     the opposite.  Why isn't it a full cycle, a full-year pilot?
         MR. GILLESPIE:  There's a certain amount of impatience on
     people's parts --
         DR. POWERS:  I understand that.
         MR. GILLESPIE:  To get on with it.
         DR. POWERS:  But it seems to me you don't get through the
     whole shooting match in one consistent pilot by going with six months. 
     You're guessing a lot.
         MR. GILLESPIE:  Probably the -- it's not guessing.  Six
     months actually gets us through the basic inspection cycles, gets us
     through a planning cycle, and because the reporting is going to be on a
     quarterly basis with a one-year running average, once you've reported
     one or two quarters' worth of material, you've basically exercised it
     twice.  The first time you've made a lot of mistakes, the second time
     hopefully you've made fewer, and now you can kind of clean it up.  So we
     would have exercised reporting through two cycles.  We would have
     exercised inspection planning.
         What we are likely going to be missing will be those things
     that might only be done once a year or once every two years, like the
     engineering inspections, but the engineering inspection is, quite
     honestly, because there's no PIs in engineering, is fundamentally what
     we're doing today, because we're picking high-risk systems.  So what
     we're not testing in the six-month period is actually the things that
     might be very similar to what we're already doing.
         With that, I'm going to turn it over to Bruce right away,
     because he's got an interesting section and one that's got a lot of
     implementation questions still left on how we're going to actually
     organize it to get it done.  So Bruce --
         MR. MALLETT:  Thank you, Frank.  Mike, were you able to
     receive the two slides that I sent?
         MR. BARANOWSKY:  No, I did not yet.  I'll get someone to run
     for them.
         MR. MALLETT:  Mike Johnson.
         MR. JOHNSON:  Yes, Bruce.
         DR. BARTON:  George, do you have a question at this point?
         DR. APOSTOLAKIS:  Yes.
         DR. BARTON:  Bruce, can you hold a second?  We've got a
     question here at the table.
         MR. MALLETT:  All right.
         DR. APOSTOLAKIS:  Well, I want to ask, as I was reading this
     report, the question of the objectives of the whole exercise came to
     mind, and I'm wondering when it would be a good time to raise that
     question.  Shall we let Bruce go on first, or --
         MR. GILLESPIE:  No, I think that's probably a good time,
     because there's a good lead-in to one of the criticisms we got of the
     inspection section was it did not -- and, Bruce, chime in if I get this
     wrong, but it did not clearly state the objectives we were trying to
     meet when you read the inspection section by itself.  So, no, this is a
     good time to ask --
         DR. APOSTOLAKIS:  Okay.  Let me tell you then what the
     concern is.  On page 2 of the document it says this paper presents
     recommendations for improving the NRC's reactor oversight processes
     including inspection, assessment, and enforcement, and includes a
     transition plan.
         Now it doesn't really say what the objective of the
     oversight process is, but then on page 3 it says, the first full
     paragraph, the objective of the inspection task group was to develop
     recommendations for a baseline inspection program blah blah blah blah
     blah to determine whether plant performance is at an acceptable level.
         Now I am not sure that this is what the objective of an
     oversight process should be.  I see this as equivalent to what people
     call quality control in manufacturing.  You have a process which is
     acceptable.  I mean, the plants have already been licensed, right. 
     They're operating.
         MR. GILLESPIE:  Right.
         DR. APOSTOLAKIS:  It seems to me that the purpose of the
     oversight exercise it to make sure that what you have licensed, what you
     think you have licensed is indeed the truth.
         So, now, then we go to Attachment H.  If that premise is
     correct, then if I look at the figures H1 through 6, which are pages
     H-18 on, these are the actual data that I believe NEI supplied you or
     some of it is -- yes, okay.  So, for example, figure H1 on page H-18 is
     the number of unplanned scrams per year for a number of units.  It's not
     all the units, right?  For a number of units.  And the same thing, the
     next one is the BWR high-pressure injection system unavailability and so
     on.  This is real life.
         In the text it says that the way to define one of the goals,
     I think it's the -- I don't remember the colors now, green to --
         MR. GILLESPIE:  Green to white.
         DR. APOSTOLAKIS:  Green to white.  Is to consider that
     number of unplanned scrams that about 95 percent of the plants meet or
     have a smaller number of scrams.  And then we make that part of the
     assessment process and say this is a threshold.
         It seems to me that this runs counter to the notion that the
     purpose of the oversight process is to make sure that what the plants
     have been doing, what we think they have been doing, is indeed correct. 
     In other words, you are forcing now this way the five percent of the
     plants that are above to actually come below the threshold.  In other
     words, you are regulating them instead of just overseeing them.  Isn't
     there a difference?
         You are telling the -- in fact, if you go to one of the --
     yes, for example, figure H4, BWR RHR system unavailability, the
     threshold is set at .015, and there are 1, 2, 3, 4, 5, 6, 7, 8 plants
     that in fact have an RHR unavailability greater than .015.  So aren't
     you sending them a message that they better make sure that their RHR
     unavailability comes below .015, which may be a good message, but it's
     not the job of the assessment process to do that.
         In other words, the way I would do it, I would say okay, Mr.
     Plant 70, you have an RHR system unavailability .025, it's the highest,
     this is what you have in your IPE, and your whole IPE was accepted by
     the staff or it was reviewed by the staff, and you have a license.  So
     all I have to do now from now on is make sure that you don't become
     worse than .025, not that you reduce it down to .015.  That's a separate
     regulation.  The oversight process' job is to make sure that what we
     have licensed and the assumptions and all that is still the way we
     thought it was.  It's quality control, in other words, not regulation.
         Which brings up now another.  If that is the case, that's
     the second point now, the first point is we are in fact intervening with
     this process, it's not oversight anymore.  The second part I think is
     also a conceptual difficulty.  Let's say you agree with me.  I don't
     expect you to agree with me right away, but let's say you agree with me.
         DR. SEALE:  You're going to give him a little time.
         DR. APOSTOLAKIS:  So you say okay, fine, so even for this
     plant where the RHR unavailability is kind of high, they probably have
     other things that are much better on the average, so the total core
     damage frequency came out to be some reasonable number.  So now we make
     sure that the oversight process will monitor the plan to make sure that
     the IPE -- the IPE's assumptions and the numbers and so on remain the
     way they were submitted.
         But the plant has not been licensed on the basis of the IPE. 
     Right?  It's the whole --
         MR. GILLESPIE:  Right.
         DR. APOSTOLAKIS:  Deterministic.  So can we really do that? 
     Can we regulate, can we have an oversight process that is based on a
     study that is not part of the licensing basis?  Is that allowed?
         Which then creates the question, should we again go back to
     Part 50 and make sure we change things there before we develop this?
         MR. GILLESPIE:  Two things, George.
         DR. APOSTOLAKIS:  Yes.
         MR. GILLESPIE:  Yes, we're out of sync, because inspection
     is ahead of Part 50 -- risk-informing Part 50.
         DR. APOSTOLAKIS:  That's correct; yes.
         MR. GILLESPIE:  So that's true.  The other piece is we don't
     have the sophistication to do this plant by plant by plant right away.
         The third piece is what's probably missing in clarity, and
     we -- I tried to make this clear at the Commission briefing -- is that
     this piece and the way these are used and the way they're used in
     inspection isn't intended to be a backwards ratchet.
         The baseline program is an indication program.  Even the
     inspection program is an indication program.  It is not diagnostic.  It
     is not intended to force someone to be better than they can be.
         DR. APOSTOLAKIS:  Aren't we de facto doing this though?
         MR. GILLESPIE:  Could that happen?
         DR. APOSTOLAKIS:  Yes.
         MR. GILLESPIE:  Yes, and if you set up a standard and say,
     you know, pass this standard it means be go into more of a diagnostic
     mode to understand why and that is the real point here.  That is the
     objective of the thresholds is if someone breaks the threshold it is to
     engage them to ensure that we as regulators understand why you might say
     in this case that they are different from the expected norm.
         DR. APOSTOLAKIS:  So it becomes plant-specific at that
         MR. GILLESPIE:  And at that point you become plant-specific
     and engaged doesn't mean send out a 20-man team.  It means first you'd
     probably end up asking a question -- why?  If there is an explanation,
     we understand it -- then there's an explanation.  We understand it, and
     that is when it becomes plant-specific.
         DR. APOSTOLAKIS:  I understand that but again there is a
     fundamental concern here that -- and there is a third point -- but first
     of all, I don't think it's such a monumental job to do it on a
     plant-specific basis, because we don't have to do it.  We can ask the
     utilities to do it.  We can give them guidelines how to do it but we
     don't have to do it on a plant by plant basis.
         For example, if I go to Figure H-2, look at Plant Number 62
     on page H-19.  It is a BWR.  High pressure injection system
     unavailability is the lowest of all the plants that are listed here. 
     It's less than .01 and you have already set the threshold at .04, right? 
     It's really low.  Now that plant probably wants to get some credit for
     it.  Maybe somewhere else -- they are not named, right? -- somewhere
     else they have an unavailability for something else that is higher than
     the threshold, so they may come back or they should come back and say,
     look, on balance our risk is acceptable and so on and so on, but if we
     look at each one of these individually, which is the third point I
     wanted to make and I forgot, we are regulating -- I mean the idea of the
     cornerstones was not really to continue to be as intrusive as we have
         We don't want to regulate initiating events and the pressure
     boundary and inspect the same way we were doing it before and I think
     that is the path that this is going down, because now we have thresholds
     for initiating events, thresholds for unavailabilities of various
     systems, and so on, so it will be as intrusive as the regulatory system
     and inspection process, as it was.
         How can we give them more flexibility, which is the whole
     idea of performance-based regulation?  Remember, these plants have
     already been licensed.  Even if the core damage frequency is greater
     than the goal, there's nothing we can do about it, unless it is 'way up
     there, which is it not.  Right?  Because that was not part of the
     licensing basis.
         So can we relax a little bit this and instead of looking at
     the systems and the initiating event frequencies and so on propose a
     methodology perhaps that the utilities can implement for them to come up
     with individual -- not goals.  Thresholds?  I don't know -- using some
     sort of systematic approach.
         MR. GILLESPIE:  What we have generated here is kind of a
     generic profile of certain operating parameters --
         DR. APOSTOLAKIS:  Yes.
         MR. GILLESPIE:  And could you generate a plant-specific
     profile for each site, given site differences and engineering
     differences?  In the future the answer is probably yes, you could.
         We started from the general and now we are going to the
         DR. APOSTOLAKIS:  I understand.
         MR. GILLESPIE:  And right now we are at the generic.  I
     don't disagree as a future goal it would be very nice to have a
     plant-specific profile but we don't have it --
         DR. APOSTOLAKIS:  But aren't we over-stepping the bounds
     though of the oversight process -- telling those 5 percent of the plants
     that they better shape up.
         MR. GILLESPIE:  We are not saying that.
         DR. BARTON:  George, I don't think that is what they are
     trying to do here.  I think they are looking for trends.  They are
     looking for plants that all of a sudden are above a norm -- and the
     industry is using one of the performance indicators which looks an awful
     lot like this kind of stuff so the industry is already familiar with
         I think what they are looking for is a plant that is below
     this .04 and all of a sudden it shoots up to, you know, .07, and you
     say, well, why?
         DR. APOSTOLAKIS:  If I owned a plant that already has at BWR
     HP injection system unavailability of .07 -- Plant 75 -- and the NRC as
     part of its oversight process says the threshold is .04, what do I do?
         MR. GILLESPIE:  Could this system be abused to that point? 
     I guess the answer would have to be yes, but that is clearly not the
     intention.  The intention is it's an indication system.  At that point
     as the regulator we would want to understand why they are different and
     what offsets it.  It is not clear.
         Now maybe if we had a detailed, plant-specific profile that
     was completely justified, it would be clear, but we don't have it and
     right now the industry hasn't really proposed to do a plant-specific
     kind of profile.
         MR. MALLETT:  But Frank, let's be careful here.  Let's make
     sure we understand the process here is that we have a band of operation
     for plants and those plants in that band of operation might be, as
     George said, from .04 to .07 unavailability of that system and therefore
     when you look at them you may need nothing more than a baseline
     inspection program if they are in that band of performance.
         It is only supposed to be an indicator of whether we would
     increase our inspection above a minimum level, so the concept is every
     plant will get a baseline minimum inspection and if the indicators show
     they are outside those bands or thresholds we would increase at least a
     part of that.  They could very well operate within that band and we'll
     leave them alone.
         DR. APOSTOLAKIS:  But Bruce, you are bringing up another
     issue now.  You are saying that what counts is the band not the
     threshold, and if I didn't speak to you and listen to you, I wouldn't
     know that.  The document does not say that.  In fact, that may be a way
     out since I fully appreciate, by the way, that you had to do it in a
     generic way.  Perhaps instead of a threshold give a band because what
     you don't want to do is introduce new requirements through a document
     that is supposed to be an oversight and assessment document.
         MR. GILLESPIE:  Yes.  This is fundamentally an internal tool
     for us to know when we have to do and understand more.
         DR. BONACA:  I would like to make one comment.
         MR. GILLESPIE:  Go ahead.
         DR. BONACA:  Simply it seems to me that, yes, I can
     understand it is an internal tool but then it introduces again
     subjectivity and one goal is go away from subjectivity.
         One thing that concerns me, looking at these thresholds --
     that is the same point that George is making -- is they may not be
     equally achievable for all plants at the same level, but again there is
     compensation from other systems that may not -- what I mean is designs
     of plants vary very much for the BWR high pressure injection system.
         Some plants have just one train but they have other things
     that compensate for that, and simply the threshold is not equally
     achievable for all plants, but the reason is there are reasons why,
     however, core damage frequency is equally low because there are other
     systems and so that is what I am trying to understand.
         Is that the point you were making, George?
         DR. APOSTOLAKIS:  Yes.  One of the points.
         MR. GILLESPIE:  We are not disagreeing with that.
         DR. BONACA:  But my only concern is the issue of
     subjectivity, because the question is how is it going to be interpreted
     and used in the field.
         MR. GILLESPIE:  Well, the intention is, just as Bruce said,
     that at that point we have to develop a better understanding, so you go
     from an indication mode into an understanding or diagnostic mode.
         Once we understand it, then it is a done deal.
         Also, this is a starting point, not an ending point and here
     is something I am kind of cautious about and one of the reasons we went
     from generic in that work in the future to specific and that's on the
     other side is all those people who talk about the fidelity of individual
     PRAs and overdependence on them and how can you have two plants, how can
     you have two plants that appear to be virtually identical with two
     different PRA results and different sets of important systems, and how
     do you deal with that uncertainty.
         At this starting point we were not prepared to have
     individual plant by plant specific profiles, so this is our starting
     point.  To do that in the future is a good option, but I can't deal with
     it between now and June very well.
         Is this a de facto different set of performance standards? 
     It could be abused to that extent.  I can't say it couldn't.  It is
     certainly not our intent and that is why we have got it out for comment.
         DR. APOSTOLAKIS:  Well, I appreciate the difficulties you
     have.  I mean don't think that I don't.
         On the other hand, we have to evaluate these things and
     scrutinize them because this is the first time that we are really doing
     such things in a big way.
         First of all, I think that the document, the written
     document, should reflect some of the thoughts that you just gave us,
     Frank.  For example, we had a similar -- not quite the same but similar
     discussion in the development of the Regulatory Guide 1.174 whether the
     lines were bright to different shades of gray and so on, and what Bruce
     said about maybe the band really being more meaningful -- now I don't
     know if you already have several bands of regulatory action and now you
     are making the boundaries between the bands themselves -- what all that
     means -- but also maybe you should give the option to a utility to come
     back and argue on the basis of their IPE or PRA that for them BWR high
     pressure injection system unavailability of .07 makes sense and they
     will maintain that, but they have other things that compensate for that.
         Right now, there is no option like that here.  It is generic
     thresholds and the truth also is that very few people really understand
     the subtleties of all this.
         I mean you are saying there is a danger for abuse.  I think
     the probability is pretty high that well-intentioned people will abuse
     this because they don't understand the fundamental idea of overseeing
     probabilistically -- not that I understand it.  It took me a while to
     understand this and make the points.
         MR. GILLESPIE:  Well, one of the criticisms of the package
     was that we didn't have a clarity of purpose in there, and that is
     something we're going to fix -- the idea that this is strictly an
     indication process and which you then go into a process of understanding
     when the indication says more understanding is needed.
         I can't fix it here today --
         DR. APOSTOLAKIS:  I know.  I am just raising the issue.  I
         MR. GILLESPIE:  The ideal would be plant-specific profiles.
         DR. APOSTOLAKIS:  Yes.
         MR. GILLESPIE:  Because just by the nature of the
     statistics, we are agreeing that 5 percent on any given indicator could
     be outside the band and that's perfectly acceptable.
         DR. APOSTOLAKIS:  Right.
         MR. GILLESPIE:  There's tails on every distribution but
     being in the tail doesn't mean you are bad.  It means we need to
     understand why so we know you are not.  We do need to put that clarity
     in it.
         How we get there from here and how fast we can do it I don't
     know, and the other piece we need to be concerned with as we need to
     part from generic and go to plant-specific is how do we communicate that
     to the other stakeholders.  We may understand it and the industry with a
     PRA group at each facility may understand it, and it's kind of a
     compromise in there in a somewhat more simplistic view in trying to have
     something that we can also communicate to the public.
         There is some simplicity driven into this.
         We also came up with another point of confusion which we are
     starting to get clarified with people, and that is that there are two
     very important matrices, one that Pat is going to go through which apply
     to individual indicators and will then -- also a parallel process to
     apply to individual inspection results.  That is an individual
     indicator.  It is one out of 20 indicators.
         Then there is a matrix that Mike has in the assessment
     process which is the matrix that covers expected agency actions or
     regulatory response.
         Initially when people thought about the concept they talked
     about green plants.  We said but there's no green plant.  There is a
     green indicator.
         Also clarifying the difference between a threshold and a
     band -- well, thresholds establish bands but the context we use it in is
     really a licensee response band, regulatory response band, and the
     terminology if you eliminate the colors, the terminology in there is
     actually focusing on the bands.
         Now the interesting point here is that the industry has -- I
     don't want to say -- I guess I could say has tentatively agreed that the
     licensee response band as a first cut on this is reasonable.  They are
     not saying it is perfect, but they are saying it is reasonable as a
     first start, that people should be able to actually have some things go
     wrong at a facility and they should be able to correct them before they
     pass out of the band.
         And that's a broad expectation of establishing the width of
     the band, be it three scrams or the unavailability numbers.  What we
     don't want is an indicator system that trips on a failure, on a single
     failure, or on a single thing.  So, 95 percent performance over the last
     several years as in industry set, that is kind of how we picked it. 
     Then looked back and said, well, is this low risk?  And the answer to
     both of those was it looks achievable, in general, recognizing there is
     a 5 percent tail on every indicator, and it is low risk.
         I don't know that we can do much more at this point, but
     this is a starting point.  That's the other emphasis here.  This is a
     starting point.  We have got a list of about six different topics from
     research on longer-term on how to mature this and make it better.  So --
         DR. APOSTOLAKIS:  I think that by putting these thoughts in
     the document and starting out by saying, you know, ideally, one would
     like to have a plant-specific PRA base on this.  If a utility wants to
     do it, we are open to suggestions, you are very welcome.  Now, we have
     to do it in a generic way, so we have to select a way for doing it.
         Now, one way might be to say, well, I will take the highest
     value in every chart and go with that.  Well, we don't want to do that
     for some reason.  So we want to be generic.  We have this requirement. 
     We go with the 95th percentile, but we will -- this will be an
     indication, blah-blah, everything you just told us.
         MR. GILLESPIE:  Yeah.
         DR. APOSTOLAKIS:  And maybe soften the impact.  Now, let me
     tell you what really bothers me at a philosophical level without
     necessarily attacking this document.  We can work on a PRA space and be
     as prescriptive as we are now in the deterministic space.  If we ever
     produce a document that will say, okay, it will be risk-informed, so now
     we are going to tell you what a threshold, a regulatory threshold is for
     scrams per year, a regulatory threshold is for unavailability of this
     system and that system, I would really object to that, because now we
     are being very prescriptive in PRA space.  It is none of our business. 
     If the accident sequences have the frequencies that are acceptable, why
     should we regulate the initiating events and so on?
         So that is what really struck me, and I am bringing it up,
     that, perhaps unintentionally, you do some of that here by saying, look,
     95 percent is what we are choosing and the other 5 percent -- now, if I
     am the owner of a facility, what do I do?  I get a signal from the NRC
     that my unavailability is not really good, but that is not a job of the
     oversight process, that is somebody else's job.
         So, now, for oversight purposes, I think it is all right to
     look at each cornerstone, perhaps look at other things below the
     cornerstones, because you are overseeing now, you are making sure that
     what they told you remains that way in the future, so that is not
     intrusive in my mind.  But if you demand, though, that the frequency be
     below a certain level, that is regulating the frequency of initiating
     events and that, I don't think we should do.
         MR. BARANOWSKY:  There is no demand associated with these
     green-white thresholds.  They are exactly what you said, and, that is,
     they are a point of oversight in which we increase our oversight because
     we have indication that we should start to look in those areas.
         When you get to the point of unacceptable thresholds, where
     we talk about talk about taking actions such as shutting down a plant,
     that is a different story.  Then maybe that is the point where we are
     imposing some regulatory action on a licensee.  But this is oversight,
     minimal oversight when you are below that line, a little more oversight,
     which seems to make sense.  Where are you going to spend your resources? 
     Well, where it seems like the performance is not within the norms, and a
     few sigma outside of the norms is the point where you say, is that an
         DR. APOSTOLAKIS:  But you see, Pat, that is exactly my
     point.  That is why I asked what is the objective of this.  Aren't you
     bringing now another element in the regulations, namely, how does your
     plant perform with respect to the others?  When you licensed it, did you
     do that?  No.  When you licensed it, you said here is a set of
     regulations, if you meet them, you get your license.
         Now, we are saying something else.  We are saying but if you
     happen to be outside the bound of what everybody else has.
         MR. BARANOWSKY:  That was just done a convenience because --
     I think you are right, we would like to have a more plant-specific
     approach to this, where some plants are more susceptible to high or low
     reliability depending on system design and operating characteristics. 
     But as a first cut, we said let's just start with this more generic
         DR. APOSTOLAKIS:  I think we need a good discussion up-front
     what the purpose is, that you are fully aware of the fact that there are
     certain risks by going this way, but there are some benefits as well,
     because we can do it is, it is generic and so on, and if somebody wants
     to come in here with a plant-specific PRA and argue that their
     thresholds should be different, you are willing to listen.
         MR. JOHNSON:  George, let's not forget also that we do what
     this process is talking about trying to do, we do it today.  We look at
     the performance of plants, we look at it in some areas that we have set
     up for ourselves.  We make decisions about events as they occur on a
     real time basis.  We decide whether we need to engage, to do more
     inspection based on the instincts of the inspectors.  I am being a
     little bit cavalier, but we use our judgment, based on our experience,
     based on our understanding of what is important from a risk perspective
     today to make decisions about whether we engage, about -- the extent to
     which we engage.  Do we send out individual inspectors?  Do we send out
     a team of inspectors?  So on and so forth.
         And what we need to keep in mind with this process that we
     are proposing is that we are simply trying to add a structure and a
     framework to do that in a way that is a hierarchial approach, as the
     ACRS has pointed out, that we need to go on, we think it is a good
     direction to go in.  That starts with this notion that, given the fact
     that we are working towards this mission, this common mission of public
     health and safety, and then that we have had cornerstones which say,
     hey, given the fact that we have got this robust framework, regulatory
     framework, and all the requirements, we can't possibly look at
     everything.  And even if we could look at everything, what would we do
     with all of that information?
         So we have set up this series of cornerstones and PIs and we
     have established PI thresholds and inspection thresholds.  And the
     entire notion is to be able to take a look, from an objective
     perspective, to see when plants are beginning to wander from what we
     accept as the utility response being -- and this notion that the utility
     can respond, and to the point where we need to begin to take some
         Now, we are not going to be measuring delta CDFs, we are
     going to be looking at numbers of scrams, we are going to be looking at
     some very simple things.  And the whole reason to do that is to be able
     to have some objective way to figure out when we ought to engage or when
     the utility ought to engage, and the extent that we ought to engage. 
     And that is what the action matrix and all those things talk about.
         DR. APOSTOLAKIS:  I understand.
         DR. BARTON:  George, I think utilities are used to working
     this.  You look at the INPO indicator, they have got scrams, they have
     safety system actuation, and they have got goals set for each one of
     those, and utilities try to operate to stay within the goal.  This is
     the same kind of thing.  If you get enough -- above enough of these
     bands, you are probably in trouble.  If you don't meet a lot of the INPO
     indicators, you get a real good look at those areas when they come in
     and do an evaluation as well.
         I think it is a trust thing here.  If get above a couple of
     these targets, is the NRC going to come in and shut you down right away? 
     I think that gives you some nervousness of how this is going to be used. 
     But I -- I don't know, I think it is a good process, to try to figure
     what is a better process than what they have laid out in looking at, you
     know, how are plants performing.  You know, where are plants getting out
     of whack, so to speak, and what is the reason for that?
         If you follow maintenance rule, you have got training,
     procedures and human performance, and all those kinds of things are,
     quote, "okay" at your plant, you are probably not going to have your
     plant above this threshold in these areas.  So all this tells me is I
     see that, you know, what is going at that plant?  What is the
     maintenance history of that system?  What is the problem with the
     maintenance rule at this plant that they are up with this system
     unavailable in the RHR system?  And that is what I think you use this
     for, and I don't have a problem with that.
         Now, if you are going to come in here and beat me up over
     that and shut me down because of it, I think that is the trust issue
         DR. APOSTOLAKIS:  Well, no, first of all, I think this is
     one of the best risk-informed documents --
         DR. BARTON:  I thought you believed that until this morning. 
     I am wondering what is going on here this morning.
         DR. APOSTOLAKIS:  I do believe it.
         DR. BARTON:  This is six months revisited or something.
         DR. APOSTOLAKIS:  And I want it to become better.  But,
     remember, this is -- no, really, you did a great job, don't get me
     wrong.  I am just arguing a couple of points that I think -- three
     points that I think are very important, and some of them can be remedied
     by writing a better background and introduction.
         But I think, John, it is different from an industry -- when
     I have an industry organization like INPO going to a particular plant
     and saying, hey, this is the norm, you are near the tail or you are
     outside, and so on, it is very different from having a regulator come
     and tell me the same thing.
         Let's say that I am a known recover plant, and I tell the
     NRC I have legal documents here that you have granted me a license.  No
     where in the license do I see initiating event frequency and
     unavailability and this and that.  And now you are going to inspect me
     on the basis and make decisions?  I am going to have my lawyers after
         MR. BARANOWSKY:  George, we do that now without this
     objective cut.  We see something happen at a plant, with our own
     personal cult and culture and operation, we go do things.
         DR. APOSTOLAKIS:  Right.
         DR. BARTON:  They do it with the SALP process, which is much
     more subjective than this.
         MR. GILLESPIE:  George, one of the --
         DR. APOSTOLAKIS:  No, no, no, no.  Wait, wait.  The fact
     that we are doing it now, though, let me finish the thought, because
     Michael is already -- the fact that we are doing it now doesn't mean
     that it is the right thing to do, and, in fact, I thought that was one
     of the major complaints at the stakeholder meeting, was the inspection
     process, was it not?
         MR. GILLESPIE:  Yeah, the lack of transparency and the lack
     of a standard.
         DR. APOSTOLAKIS:  And too intrusive.
         MR. MALLETT:  This is Bruce, let me make a comment.  Is that
     I think George is right that this is one of the holes that the
     Commission emphasized to us that we need to work on, is establishing
     these thresholds correctly.  And, also, how the inspection findings
     interplay with these performance indicators.  Let's say somebody drops,
     as you said, George, from .04 to .07, I think you were using
     unavailability of the HPSI system, then they -- we have to look at the
     inspection program and what has it shown us in that area as well. 
     Because it all fits together to say, do we believe the objectives of
     cornerstone were met?  That is what we designed the program to do, is to
     determine is the licensee meeting the objective of that particular
         DR. APOSTOLAKIS:  Bruce, if the four of you gentlemen were
     doing this all the time for other plants, I wouldn't have any problem. 
     Okay.  I wouldn't have any problem.
         MR. MALLETT:  I want to mention a couple of other points,
     Frank, that are different than what George was raising.  But it is close
     to his original question on what was our objective.  In the
     risk-informed baseline program, we wanted to focus on risk, the planning
     inspections and conducting inspections, and we have got that concept in
     the process.  We also wanted to make it clear why we are inspecting
     something from a risk-informed perspective in linkage to our mission. 
     We have got that in there.
         And, last, we wanted to have risk information put into how
     we select our samples, which is different than today, and we wanted to
     have that in there.  So when you look at why we did what we did, those
     were some of the original reasons that we put into the program.  So I
     recognize we have whole lot of thresholds, but we can't forget some of
     those objectives on why we wanted to what we wanted to do.
         DR. BONACA:  One comment I have, if I could.  I don't see
     why this discussion should not be reflected somewhere in the early
     portion of the document to this nation, because you are using a lot of
     PRA, risk-informed information coming from PRA concepts.  So you have
     got to put some kind of insight in how this is going to be used, the way
     you are describing here.  And I think that that would enhance the
     understanding of this approach for inspectors, in fact, and for
     everybody else.  So it would improve the scrutability of the process
     that you are talking about.
         DR. APOSTOLAKIS:  One of my real concerns here is that we
     will send a message to the rest of the NRC that when you develop
     risk-informed regulations, it is okay to put thresholds on initiating
     events, on unavailability.  Well, it is not okay.  It is not okay at
     all.  That is not the idea of performance-based -- risk-informed,
     performance-based regulation, that is not the idea.  And it is so easy
     to fall into that pitfall, because, you know, it is detailed, it is
     nice, it is something I can measure, but that is not the intent.  For
     oversight purposes, it is okay to look at these things, but not to put,
     you know, regulatory limits, then you are back to a prescriptive system.
         MR. GILLESPIE:  Let me go back to one comment that was made
     here that we didn't address.  And that was, one, this is an indicator
     process, including inspection, and part of what a utility gets, if you
     would, from participating with the indicators, is the inspection program
     won't duplicate the information that that indicator is giving us.
         So there is some dependence on the indicator.  We have a
     reliance that we are not going to duplicate that area.  We're going to
     do some verification inspection.  So there's definitely a backoff, if
     you would, on inspection.  Because the big question was what information
     do we need to make a reasonable assurance finding.
         The other thing is, this is not diagnostic or cause.  When
     an indicator says you need to understand what's happening, the problem
     is still going to have to be discussed and identified in the context of
     the regulations.
         You're right, if we went in and said you've got four scrams
     and therefore we're going to take some action based on that, no.  The
     root cause of the problem is still going to have to have its essence and
     any corrective regulatory action taken is still going to be based in
     regulation and license.
         So when you do get engaged, you tend to fall back on the
     traditional system of what was the cause, what's the requirement, was a
     requirement violated, and if a requirement wasn't violated, then we have
     to say we understand what happened and it's okay, or it's a new
     phenomenon, and we have to do something about it and issue an order or a
     license condition or something.  So it gets us back into kind of the
     today environment when you do become more diagnostic.
         That was the intent.  It's a starting point.  It's not a
     finishing point, and we recognize that.
         The other thing is, it was never our intent to rewrite this
     document, by the way.  This document goes away by June.  This document
     is really the description of the concept that we would use as our
     guidance in writing I'll call it the real program documents.  So this
     description of not abusing things would actually go into whatever we
     would replace, Inspection Manual 2515, which has the philosophy of the
     inspection program in it.  One of the cautions I want to do is I don't
     want to keep rewriting a document that's really not a functional
     document in any sense or any system.  This was a descriptive document. 
     So --
         DR. APOSTOLAKIS:  Is this a public document, by the way?
         MR. GILLESPIE:  Oh, yes, yes.  It's on the Web.  The
     Commission gave us permission to put it out before they even saw it so
     we could get comments.  Comment period has started, and we'll see what
     comments we get.  I agree that -- the Commission said the same thing,
     the context that we were talking to at the meeting was absent by way of
     like an introduction or contextual statement in the front of it.
         DR. APOSTOLAKIS:  Let me ask you another question, Frank --
     or all of you.  One of the particularly sensitive issues with some of us
     on this Committee, and I think with the industry, is that PRA is
     perceived as being a tool for adding regulations or adding burden.  Is
     this document removing any of the burden, the existing burden?
         MR. GILLESPIE:  Yes.
         DR. APOSTOLAKIS:  Okay.
         MR. GILLESPIE:  In fact, if you look in the inspection
     section, there are some hours there, and we didn't project it all the
     way forward, but I had given this information to Sam, and he used it at
     the Commission meeting, so now it's in a public transcript, so I feel
     free to use it again.
         This probably reflects a 15 to 20 percent reduction in
     overall inspection.
         DR. APOSTOLAKIS:  Okay.
         MR. GILLESPIE:  In two phases.  When Pat ballparked how many
     sites would actually meet the entire profile as being in the licensee
     control zone, it's in the ballpark of about 50 percent of the utilities. 
     So about 50 percent of the utilities that are now getting some level of
     reactive kind of inspection from us would no longer be getting that
     level of reactive inspection.  So I think there's going to be kind of an
     immediate incremental benefit from being risk-informed and being
         And now the question would be is -- I think we need to be
     cautious as regulators, and we're generally conservative -- is as you're
     fine-tuning to a plant-specific nature maybe, that as you fine-turn, you
     don't lose some sense of checks and balances that you've tried to build
     into the system.  And if you do lose it, you know what you're losing.
         So yes, it looks like about a 20-percent savings.  The
     immediate piece in the baseline program itself from the core to this
     baseline looks like it's approximately 15 to 20 percent.  So that's 15
     to 20 percent of the core, which represents about half the inspection
     program across the board.  And then there's about another 20 percent
     which is the reactive effort, which might not take place.
         Now what's going to happen, George, and you're absolutely
     right, is I could say the safety gain here is everyone's going to strive
     to be in the best category.  It's a possibility.  That's a business
         DR. APOSTOLAKIS:  Which is not a bad idea.
         MR. GILLESPIE:  Yes, I mean, you know, I mean, I have to
     recognize that.  I'm not living in a vacuum, because utilities every day
     make the choice do I go along, and this is one of the criticisms.  So
     I'm not saying this as a fact, but do I go along with the inspector and
     just fix it, or do I fight it.  I mean, that's the kind of a thing you
     hear from the regulatory impact survey going back ten years.
         It would be a business decision that they would strive to
     stay in the licensee control zone in all of these areas.  Now do I feel
     that we've kind of agreed if you would in setting these thresholds with
     credible thresholds, reasonable thresholds, I think we have.  You know,
     I'd say the best shot is 95 percent.
         DR. BARTON:  They are reasonable thresholds.
         DR. APOSTOLAKIS:  Yes.
         MR. GILLESPIE:  They're reasonable thresholds.  Now that's a
     business decision.  It could happen.  And that could happen without us
     doing any arm-twisting.  Someone could say you know, if there were
     really --
         DR. APOSTOLAKIS:  All you have to do is show up, Frank.
         MR. GILLESPIE:  Okay.  Okay.  All we have to do is --
         DR. BARTON:  I think we need to get on with Bruce.  We're
     going to lose him.
         MR. GILLESPIE:  Let's get to Bruce.  Bruce only has about a
     half-hour.  I think we're in violent agreement.  And the Commission was
     also in agreement that we need some contextual words in the front of
     this to put it in context.  I don't want to rewrite the document,
     because the document actually will go away within six months.
         MR. MALLETT:  You should have in front of you -- I think the
     easiest way without repeating our last meeting is one slide that I call
     a flow diagram that's used -- the scope of the program, planning
     inspections and selecting.  Do you have that in front of you?
         DR. APOSTOLAKIS:  Yes.
         MR. MALLETT:  Okay.  Let me walk through that just briefly. 
     I don't want to bore you, but I want to highlight some of the concepts
     we had previously in the program.
         First, we've been talking about an oversight framework where
     we have performance indicators, a baseline inspection, and an assessment
     process.  That led us to the cornerstones of safety concept in which we
     have seven cornerstones.  Those then defined inspectable areas and gave
     us inspectable areas out of that.  And George Apostolakis asked us
     before to put in charts showing that linkage from the inspectable areas
     to the mission of the Agency and to the cornerstones of safety, and,
     George, we did that there in Appendix 2 to Attachment 3 on the document. 
     There's a chart for each of the cornerstones that shows inspectable
     areas and performance indicators.
         DR. APOSTOLAKIS:  I've seen that, Bruce.  Yes.
         MR. MALLETT:  Okay.  I just wanted to let you know we do
     listen to you, George.
         We also --
         DR. SEALE:  Never doubted it.
         MR. MALLETT:  The document I'm going to describe these
     concepts out of for the baseline inspection program are risk-informed is
     in Attachment 3 to the Commission paper.  There are nine sections to
     that document.  Each section describes a particular concept of the
     program.  For example, one that you asked us for, Dr. Seale, you asked
     us the last time to make sure we put in resources.  We put in a
     projection of that in section 8 of that Attachment 3.
         Let me continue looking at this chart with you to just
     describe a few concepts.  First of all, the scope of the program is
     defined by the cornerstones and these inspectable areas, but one of the
     goals we had up front was to define why we inspect.  So we put that in
     something called a basis document, and that's why we drew an arrow from
     basis documents up to scope.
         The next thing you would do in this baseline inspection
     program, you would plan inspections.  And we developed something called
     a risk-informed matrix.  We call it RIM No. 1.  Those are also an
     appendix to Attachment 3 to the document.  I believe it's Appendix 3.
         If you look at RIM No. 1, it outlines for each inspectable
     area what the frequency of inspections are going to be, how many
     inspections you might do, and how many samples you might select.  We put
     ours in there as a way of budgeting.  One of the points of contention
     for the Commission, you ought to realize, they challenged us on was it's
     a great concept, Bruce, but for a lot of the inspectable areas, you
     don't have much definition of the sample.  And they are right on that. 
     We need their -- as we go through the pilot and using this program
     further develop what samples we'll have and how many.
         The next thing where we put risk into the process is in
     selecting the sample.  Once you've planned on your 12-month cycle and
     your plant performance review in each region, you'll take RIM No. 2 then
     and select the system that you might want to look at.  That's generic. 
     Plant specific, you'll have to modify that by using the SRAs and any
     plant-specific information to determine your sample.
         And then the next thing you do in the process is you
     inspect, and we've got the procedures concept set up as a procedure by
     cornerstone to perform the inspection.  And I'm going to -- procedures
     are listed in section 4 of that Attachment 3.
         And the last thing you'll do is you'll assess those findings
     much like today.  We envision you'll have a plant information matrix
     type summary of the finding, but we envision that you'll have a risk
     scale, and this is another hole right now, that this risk scale has to
     be developed to set what we're thinking of as maybe three areas.  You'll
     have a performance that we believe is not risk-important.  You'll have
     another category we believe that's very risk-important in your finding. 
     And then one that just says it's a finding, but it's an average
         DR. APOSTOLAKIS:  Would you --
         MR. MALLETT:  We haven't developed this much -- yes, George.
         DR. APOSTOLAKIS:  Would you put another arrow there under
     findings assessment to bring up again the plant-specific nature of these
     things?  In other words, you do your inspection, and then when you
     evaluate what you found, you look again at the plant.  Unless that's
     understood.  Maybe that's understood.
         MR. GILLESPIE:  Yes.  George, let me kind of come out of the
     closet a little bit on how we're working this.  We actually right now
     kind of have a draft which has two scales, two steps right now, and then
     there will be a third step.
         One is a generic screening out of totally risk-insignificant
     anyplace kind of items.  Then there's another step which gets you into
     is it risk-significant in a sense at that facility.  And then there's a
     third step that says the inspector isn't in a position to make this
     judgment, we need the SRA or we need some more analytic help on the
     bottom.  So it's kind of a multistep screening process, because we need
     something that an inspector can quickly and rapidly use for the majority
     of the cases.
         DR. APOSTOLAKIS:  Okay.
         MR. GILLESPIE:  So, yes --
         DR. APOSTOLAKIS:  That makes sense.
         MR. GILLESPIE:  We're getting to that point, because it's
     the risk at the facility that we have to focus on.
         DR. APOSTOLAKIS:  Okay.
         MR. MALLETT:  Okay.  The last point I wanted to make is that
     we put down a block there that says cornerstones objective met, yes or
     no.  The idea of that is to show that this is a minimum baseline program
     to be performed at all plants.  If they meet the objectives, they'll
     continue to receive the baseline program.  If they don't, through as we
     talked earlier, through crossing some threshold or from an assessment
     finding, meet the objective, we want to increase the inspection above
     that for that plant.
         So, George, you asked the question of reducing burden.  An
     implication of this is that currently, you know, we have set out at some
     sites rather than inspectors at the N level per number of units, we have
     also in some cases set at the N plus 1 level.  If that plant is in the
     band where we believe they're meeting the cornerstone objectives, the
     new program will say they will only receive the baseline inspection
     program, you'll have to decide what you do with that other that is there
     so that they don't receive more than that.  So that's a big implication
     of this change.  But if we carry it through, that will be certainly a
     reduction of unnecessary burden of a licensee's facility.
         I can mention more, but what I want to do is refresh you on
     those concepts and then leave it to any questions that you might have.
         DR. APOSTOLAKIS:  Which attachment has the inspection areas?
         MR. GILLESPIE:  Three.
         MR. MALLETT:  That's Attachment 3, but the Commission paper,
     the SECY paper, the main concepts are just discussed or described on
     page 12 of the Commission paper, and the implications for policy I just
     mentioned are on page 15 of the Commission paper.
         It also listed -- I'll mention one more thing -- in
     Attachment 3 in the beginning you'll find an executive summary and in
     that executive summary we list the work we believe still needs to be
     done prior to conducting a pilot program.
         DR. APOSTOLAKIS:  Now the inspectable areas, Bruce, like
     Attachment 3, I guess arabic 1, page 1, program -- Part 1, inspectable
         MR. MALLETT:  That's correct.
         DR. APOSTOLAKIS:  Attachment, page 1.
         First of all, this idea of complementary and verification
     inspection I think is very good.  You are really trying to discriminate
     as much as you can.
         MR. MALLETT:  Now I have to give credit to Pat Baranowsky
     and his group for that --
         DR. APOSTOLAKIS:  He just stepped out of the room, so let's
     hope he reads the transcript.
         The inspectable areas by cornerstone -- how did you come up
     with these?  Was it just your group sitting around the table and saying,
     well, what is it that we need to do here or was there some more
     systematic way, because this is really what you are actually inspecting,
         MR. MALLETT:  What we did was originally we sat around the
     room or Pat Baranowsky's group sat around the room and we said what all
     do we believe we need to inspect to meet these cornerstones objectives?
         I think we had a hundred and some areas that we came up with
     originally, and then as the framework team developed the performance
     indicators and as we met with them and discussed them, we said where do
     we have a performance indicator that we don't need to inspect because it
     already covers that, and we crossed the list off for those, and then we
     said -- we made another cut.  What is risk important?  Well, we crossed
     a number of things off the list that we felt were not risk-important.
         I'll give you an example.  Under the Mitigating Systems
     cornerstone we originally had an inspectable area of observing
     maintenance activities.  We said we don't need to observe maintenance
     activities.  The key part of those from a risk perspective is the
     post-maintenance testing, we thought, and so we left that one in but
     took out observing maintenance activities.
         That is how we arrived at inspectable areas, but one thing
     I'll also mention, George, in Appendix 1 to that attachment in the basis
     documents we added something after we met with you all the last time. 
     We described in there why we inspect, but we also discussed performance
         Remember you all suggested we highlight how those interplay.
         DR. APOSTOLAKIS:  Yes.  In fact, let's go to Attach 1, page
     5, because that is where my notes are.  Attachment 1, page 5 -- where
     you also have the inspectable areas by cornerstone.
         MR. MALLETT:  Right.
         DR. APOSTOLAKIS:  For example, if I look at initiating event
     cornerstones, maintenance rule implementation, maintenance work
     prioritization and control -- why do I have to worry about work
     prioritization and control if they comply with the maintenance rule?
         MR. MALLETT:  For initiating events?
         DR. APOSTOLAKIS:  Yes.
         MR. MALLETT:  What we felt there was there are some events
     where the maintenance caused the problem by not deciding the right -- I
     don't want to say sequence, but not looking at one system they are doing
     maintenance on versus another system, because they didn't prioritize
     their work to do it in the sequence that wouldn't cause that problem
     later on -- like if they had an event while they were working on a
     particular system and then they were doing maintenance on another system
     that that could affect, they need to be careful of the interplay of the
         That was the idea was to look to see that the licensee is
     looking at the interactions between systems when they do maintenance so
     that they don't cause initiating events for one system by working on
     another system.  Does that make sense?  That's what we were trying to do
         DR. APOSTOLAKIS:  Yes.
         DR. BARTON:  Understand it.
         MR. GILLESPIE:  George, I think -- isn't there a change to
     the maintenance rule that's been briefed to the committee?  I am not
     sure where it stands.
         DR. BARTON:  We haven't heard that yet.
         DR. APOSTOLAKIS:  We haven't seen it yet.
         MR. GILLESPIE:  It talks about configuration evaluation.
         DR. BARTON:  We know it is coming, Frank.  We haven't heard
     the details of the rule change.
         MR. GILLESPIE:  That may impact this area if it becomes part
     of the rule, but for inspection purposes we're touching it until the
     rule is a rule.
         DR. APOSTOLAKIS:  Now heat sink performance -- we can't have
     a PI for that?
         MR. MALLETT:  We believe you can but we're not going to
     develop --
         DR. APOSTOLAKIS:  Oh, okay, okay, all right.
         MR. MALLETT:  If you look at heat sink performance, we put
     complementary down there --
         DR. APOSTOLAKIS:  Right.
         MR. MALLETT:  -- as saying that you need to do an inspection
     in that area because there is not a performance indicator.
         DR. APOSTOLAKIS:  But there could be one.
         MR. MALLETT:  It's not supplemental to a performance
     indicator but it could be.  Same way for the -- if you look under
     mitigating systems and you look at configuration control, and you follow
     that at inspectable areas, one of them we have there is talking about
     equipment alignment and so forth.  There is nothing for shutdown
     conditions right now, performance indicator, but we maybe could develop
     one for shutdown conditions, for example.
         DR. APOSTOLAKIS:  Anyway, yes, I --
         DR. POWERS:   And that raises a question.  In the document
     there are PIs that you define now and there are PIs that you think you
     might be able to define in the future.
         One of the areas where you did not I think define a PI but
     rather relied on inspection was fire protection, yet the fire protection
     people are off busy trying to create a performance-based code with
     performance indicators.
         Why did you feel that it was unlikely that they would be
         MR. MALLETT:  I am not sure, Dana, that we thought it was
     unlikely they would be successful.  We felt at this point in time that
     there wasn't one developed and so --
         DR. POWERS:  I mean that is not the criterion on shutdown,
     so why should it be the criterion on fire?
         MR. MALLETT:  Well, the same way in shutdown we felt that
     there wasn't one developed yet so we would not rely on that performance
         DR. POWERS:  I understand that, but in some cases you have
     said, well, in the future there might be, and you have allowed for that
     possibility that in the future there might be --
         MR. MALLETT:  Oh, I see your point.
         DR. POWERS:  -- but in fire you apparently had little faith
     that this could be accomplished.  Now that may be a very, very rational
     viewpoint but I just wondered why you had that viewpoint.
         MR. MALLETT:  I don't think it was having little faith,
     Dana, as much as we may have had an oversight inconsistency between the
     two writeups and we may have to look at that.
         DR. POWERS:  I think you should, because if that is a major
     initiative that the Staff is undertaking to vote with your feet on it
     may not give them the encouragement they need at this critical juncture
     in their work.
         MR. BARANOWSKY:  I think we identified the performance
     indicators that we thought we might see some development and acceptance
     of in the next six months, and our impression was that the fire program
     performance indicator probably was going to take longer than that to
     develop and get operable.
         DR. SEALE:  You are less sanguine about shutdown?
         DR. POWERS:  In six months we are going to have a
     shutdown performance indicator?
         MR. BARANOWSKY:  Well, there were some proposed by the way,
     and what we didn't have was the time to get the data and look into the
     details of the indicator to see whether we thought it was a really valid
     indicator or not.  The feeling was that it probably could be developed. 
     It was a judgment call.  That's all.
         DR. POWERS:  Well, it may be very sound.
         MR. BARANOWSKY:  You wouldn't be the first to say that it's
     not going to be possible.
         MR. GILLESPIE:  It wasn't our intention to take a shot at
     the fire people.
         MR. MALLETT:  Yes.  We can take Dr. Powers' comment and look
     at it during the pilot program and see if there is some way we can
     factor that in better, give more credit to them -- good point.
         MR. GILLESPIE:  I think simplistically when you look at
     shutdown, you can ask the question how many water sources are available
     and is there power available to pump the water to where it needs to be
     and is the heat sink available?
         All of a sudden I have got specific things that you can
     count at any given time at a shut down facility and say yes, I have got
     two water sources available -- yes, I have power from two different
     sources ready to go to the pumps.  It's actually something we could
     count, so we saw some near-term achievability in shutdown.
         We were not familiar enough -- in fire protection when you
     look a fuel sources and other things, how would you measure fuel
     sources?  It seemed like a bigger problem, so there was kind of, I
     think, a near-term, longer-term perspective when we were doing it.
         I literally mean "we" -- I mean just the team working on it
     saw shutdown was a place we could make progress and fire just needed a
     whole lot more work before you had something that was countable and
     easy -- something you would see from actually the control room.
         DR. POWERS:  You have a tension that runs throughout the
     document between defense-in-depth and some of the rest of the document,
     the rest of the approaches you have.
         One area where you don't have tension, one area that
     everything that one would do for inspection and regulation of safety is
     lined up with defense-in-depth is in fire protection.  The things that
     you would count are the things that make up the elements of fire
     protection -- of defense-in-depth and fire protection.
         Do you prevent fires from occurring?  Do you detect and
     suppress fires?  Do you protect the equipment?  So in a counting basis I
     think that you will find that you can count and what you count are
     things that are explicitly defense-in-depth items.  There is not a
     tension between defense-in-depth and the things you are inspecting the
     way there is throughout the rest of the document -- and that is because
     fire protection is the one area where they have explicitly defined
         MR. MALLETT:  I guess one other comment I would make on a
     different note than what Dr. Powers was talking about was the last time
     we met NEI said they had two points of contention about the proposed
         One was they had thought when we set out there were going to
     be more performance indicators than there are now in the program, and we
     indicated that we opened the door for that, but we haven't developed all
     those performance indicators and that is why they aren't in the program
         The second point of contention they were making was remember
     when we said we wanted to do an identification and resolution of
     problems inspection as a two week inspection once a year and they
     indicated they didn't know we needed to do that.  They thought it was
         We had left it in the program.  We changed it to once every
     two years but we thought it was important to have in the program to have
     an independent look at the routine inspections we're doing in the
     baseline by individual inspectors and to also have an across-the-board
     look at how they are identifying and resolving problems, but I did want
     to point out to you that that was the point of contention on their part
     and I believe they still have that view even though they support the
     concepts in the program.
         If there aren't any more questions I am going to get off the
     line.  We have an enforcement conference we're going to start.
         MR. GILLESPIE:  That is risk-informed --
         MR. MALLETT:  And they are using risk information.  All
         DR. BARTON:  Any other questions of Bruce before he signs
     off?  Hearing none, thank you very much, Bruce.
         MR. MALLETT:  Thank you for letting me do it by
     videoconference so that was very good.  Thank you.
         DR. POWERS:  Mr. Chairman, I am now fighting with another
     windmill, to the same success most people have fighting with windmills.
         Have we gone into the details of Appendix H?
         DR. APOSTOLAKIS:  Yes.  Not the details, the overall
     structure, but we spent an hour on it.  If you --
         DR. POWERS:  I guess I need to understand a little bit
     better the quality of the tools that have been used in defining things
     in Appendix H and better to understand the benchmarking that was done in
     Appendix I.
         I think I understand the approach that was adopted in H, and
     I will confess to being confused a little bit over Appendix I.
         Within H, I need to understand you utilized the Sapphire
     Code plus some PRAs that were submitted by the licensees or by NEI
     provided them to you?
         MR. BARANOWSKY:  We used the Sapphire Code with PRAs that we
     obtained from licensees.  There were approximately 13 of them that we
         DR. POWERS:  And these PRAs were in some sense certified for
     this particular application?
         MR. BARANOWSKY:  They were not certified.  They had been
     reviewed as part of the IPE process.
         DR. POWERS:  And what we know about the reviews that are
     done in the IPE process is that they serve adequately the objectives of
     the generic letter that initiated the IPE process, but in no sense are
     they what I would call a technical review of how well the IPE or the PRA
     reflects the plant, how good the database is, how complete the scenarios
     considered are.
         MR. BARANOWSKY:  Yes.  We weren't really trying to do a
     plant-specific assessment.  What we were trying to do was get some
     representative models and our belief is that the results from the models
     and the IPEs when one looks at the broad insights are reasonably correct
     in terms of what is important and the kind of levels of core damage
     frequency that one sees.
         In fact, if we look at the results of the action sequence
     precursor program, we see approximately the same order of magnitude core
     damage frequency comparability between that program and what the IPEs
     give us and most of the important sequences are of similar
     characteristics but details do different and there are a few different
     ones, so our feeling was that in terms of orders of magnitude we have
     got a spectrum of plant designs with PRAs that have at least been looked
     over somewhat.  They don't have obvious flaws in them and therefore they
     would be reasonable models to perform sensitivity analyses to understand
     how some of these parameters' variation would affect risk calculations.
         DR. POWERS:  At least when I look at the IPE insights
     program, I see an order-of-magnitude difference between plants of
     nominally the same type.  So you've got an order of magnitude --
     uncertainty band?  I don't know what you call it in this.
         MR. BARANOWSKY:  Yes.
         DR. POWERS:  This context.  Derived from a tool that may or
     may not be qualified for this kind of job.
         MR. BARANOWSKY:  Yes.  I think the reason that it's a
     reasonable tool is because of the way that we used it.  We didn't take
     any one plant.  What we did was we ran a bunch of these calculations and
     then we sort of enveloped them.  And we have no reason to believe that
     the IPEs grossly and in a biased manner underpredict the likelihood of
     core damage for this type of a calculation, but that in some cases
     they're a little high and in some cases they're a little low.  And in
     fact most of the work that we've done in AEOD taking actual operating
     experience and comparing it to IPEs shows that about 90 to 95 percent of
     them have pretty reasonable values when it comes to initiating event
     frequencies and system reliabilities, and that there are a few that are
     unexplained as to why they are higher or lower than what we would get
     from looking at some of the more direct indications from the operating
         So it's our belief that taking a set of these, running the
     sensitivities, and more or less enveloping the results probably is
     conservative for this cut at setting thresholds.  Moreover, I think the
     level of risk that we're talking about looking at is, you know, rather
     low.  We're talking about changes in core damage frequency of 10 to the
     minus 5, 10 to the minus 6, for which there's not a significant public
     risk associated with it, and that one doesn't need to be precise in
     calculating these things because we're not using it in that manner. 
     We're trying to get into the ballpark of what we think is an area of
     increasing risk as we move through the different thresholds and
     performance bands.
         DR. POWERS:  Someone might ask that given the uncertainties
     that are associated with risk estimates, is a differential of 10 to the
     minus 5 in core damage frequency detectable?  That is, maybe you're
     looking at just noise.
         MR. BARANOWSKY:  It probably realistically is the noise, but
     it is something that one can calculate.  Just like one can sit down and
     deterministically say a piece of equipment doesn't meet a certain
     regulatory rule, that probably is well within the noise also with regard
     to whether the plant is safe or unsafe.  So I'm not sure that there's
     any difference there.  The point is that we are able to identify plant
     performance characteristics that seem to show increases of these small
     magnitudes, and it's the best tool that we have available for
     identifying where we should be focusing our look.
         DR. POWERS:  Can I ask you about the status of what,
     validation verification, peer review of the Saphire code?
         MR. BARANOWSKY:  Yes, I guess I'd have to ask someone from
     the Office of Research to give you an up-to-date statement on that, but
     my understanding is that that code has been pretty well developed,
     reviewed, and validated by the Office of Research over about 15 or so
     years at least.  I don't know that anyone here is familiar with that. 
     I'm looking around the audience.  I don't know if John is --
         DR. POWERS:  What I know is that the Office of Research has
     in many cases subjected codes that they have sponsored the development
     of to what I think is a rigorous and outstanding peer review process,
     very disciplined, very structured, published reports, and allowed
     developers of those codes to respond to the peer review comments and
     responses from the commenters.  But I am not familiar with such a review
     for the Saphire code.  That doesn't mean it hasn't been done.  I'm just
     not familiar with it.
         MR. FLOOD:  This is John Flood from the Office of Research. 
     We'll have to get back to you on that, Dana.
         DR. POWERS:  I wish you would, because I think this is
     crucial that the Office of Research in developing its PRA tools has
     tools that the rest of the Agency can put an enormous amount of
     confidence in.  Whereas they can have their doubts and qualms about IPE
     PRAs, they should not have any doubts about the capabilities and
     limitations of their in-house tools.
         MR. BARANOWSKY:  By the way, I would like to point out that
     most of the doubts and questions about IPEs have very little to do with
     the specific computer code that was used.  They usually have to do with
     the assumptions and completeness of the model and the tools generally
     give pretty similar results, and that's the way a lot of them have been
     tested out, by benchmarking against each other.
         DR. POWERS:  If you could give me a ten-minute guide on how
     I should reread Appendix I to understand what's been done, I would
     appreciate it, because I have to admit --
         DR. APOSTOLAKIS:  Well, let me ask, what's the process here,
     Mr. Chairman?  We seem to have demolished the presentation.
         MR. GILLESPIE:  Well, no.
         DR. BARTON:  I don't think so.  I was going to ask Frank,
     where are you going from here, because we --
         DR. APOSTOLAKIS:  We will come back to --
         DR. BARTON:  We've done some bantering back and forth and
     trying to get Bruce's input before he to fall off.  I was going to ask
     Frank where does he see going next, we take a break now, or you get into
         DR. APOSTOLAKIS:  Because I had some comments myself, and I
     don't know when to raise them.
         MR. GILLESPIE:  No, right now what we'd like to do is
     continue the questioning on performance indicators in the oversight
     structure, and then move on to Mike and the assessment process.
         DR. APOSTOLAKIS:  Okay.
         DR. BARTON:  Why don't we finish that, and then before we
     move on to Mike, we'll take the break.
         MR. BARANOWSKY:  Okay.  Well, I mean, the questions --
         DR. BARTON:  Any other questions about performance
         MR. BARANOWSKY:  In this area?
         DR. BARTON:  Do we have any other questions of Pat at --
         DR. APOSTOLAKIS:  I cannot ask any questions.
         DR. BARTON:  Okay.
         MR. GILLESPIE:  I was thinking you were going to continue
     on, Dana, beyond -- the question you've asked also applies, and in fact
     the Commission asked -- David Lochbaum asked it -- relative to the stuff
     Bruce was doing.  I'm sorry we don't have Bruce in line, but we've got
     John Fleck here.  And that was how did you pick the inspectable areas to
     be risk-informed given you had to average across PRAs if similar plants
     could come up with different analysts with different results.
         And John, there was kind of a process of averaging -- it's
     kind of gross, but we did this -- on two plants and less than all of
     them for picking sequences that led to areas, and could you just in a
     few minutes --
         MR. FLECK:  Yes, I'm John Fleck from the Office of Research. 
     We looked across IPE the top ten sequences across plants to draw
     insights and put those into a matrix called RIM 2.  Basically it was to
     pull out the information that drives risk out there onto the table, so
     when we are choosing inspectable areas we could somehow link the area to
     those risk insights.
         So we did that -- the insights were generated, as Frank
     said, using IPE information that we brought in house through the IPE
     data base by scanning across the sequence, the dominant sequences at all
     plants, and the insights contained in NUREG-1560.  And the process more
     or less evolved around -- there was about five of us involved in looking
     at the inspectable areas and weighing it with that information in the
     IPE as well as basic information like how much effort would it take even
     just to go look at that particular area.  So there's always some minimal
     amount of time that you'd have to spend on it.  And that's where we came
     down from 150-some-odd areas to something like 26 in the reactor safety
         Does that address the question, Frank?
         MR. GILLESPIE:  Yes.  Basically what he's describing is kind
     of an averaging process that went across the PWR designs, which meant if
     a sequence showed up in the top --
         MR. FLACK:  Top ten.
         MR. GILLESPIE:  Top ten at at least two facilities.  There
     was more detail to it.  At a couple of facilities, didn't have to show
     up anyplace, then it got to influence an inspectable area, which means
     the inspectable areas at one plant might be from a sister plant of
     similar design where a different analyst found something more important. 
     So it's kind of a generic averaging is kind of how I put it in my
         Now the next step is when you apply those insights to
     plant-specific, and this is a step where I'm trying to develop in the
     next phase, is the areas are defined, there will be a minimum look at
     these areas, now customize your look to the specific sequences from that
         And it's the kind of a process of attempting, if you would,
     in a gross sense, to take the best of the average and say here is the
     areas you need to look at, but when you pick your specific sample, you
     need to look at your specific plant and make sure you have hit your most
     significant sequences at that plant.
         It is not a perfect system because we didn't have perfect
     IPEs and PRAs, but it was a way of trying to deal with the strengths of
     the PRAs and the weaknesses in a process way.  And the research reports
     are in draft form.  They are kind of in for final comment right now.
         MR. FLACK:  Yeah, we have got them for out for comment.  We
     should be able to finalize, certainly, within the next two weeks.
         MR. GILLESPIE:  But the next step is, now, how do you go
     from the generic averaging to the specific application and not lose your
     generic insights, but, yet, do some successful customization, as you
     pick your sample, to the high probability sequences at that facility, or
     high risk sequences?  It is a challenge.  And we are dealing in gross
     levels.  This isn't a highly -- a high fidelity system we are in.
         DR. POWERS:  Is this an exercise undertaken with a pad and
     paper or undertaken with computer?
         MR. FLACK:  The exercise of gleaning out the insights was
     done basically by --
         DR. POWERS:  No, I am talking about the step from going --
     you have got these generic inspectable areas.
         MR. FLACK:  Pad and paper.
         DR. POWERS:  And now you want to customize it.  Is it -- is
     that reasonable to have it as a pad and paper exercise, or should there
     be other kinds of tools in the hands of the people to do this the way
     you envision?
         MR. GILLESPIE:  Again, in kind of the macro sense that we
     are dealing in, pad and paper, given we have nailed it down, we have
     kind of boiled it down to a matrix kind of approach, and then you can go
     to the specific facility and we know to the best we know the dominant
     sequences at those facilities.  It is now a matter of saying, what
     systems are predominant?  And if I have to look in the mechanical area
     in a mitigation system, what is the system I want to be sure at my
     facility will stop a sequence?  What is -- and those risk important
     things are all really available for that most part right at the site.
         So making that match in pad and paper, I am not sure I would
     computerize it.  That is a subjective match of kind of the generic stuff
     that, if you would, that we have put together and the site-specific PRA
         Right now we are not thinking about putting any kind of
     fancy algorithm together to try to do it other than do it.  We also have
     to focus on the people doing it.  The people doing it are likely going
     to be the senior resident inspector and the resident staff, with some
     assistance from the SRA.  And you are looking at about 20 sites per
     region, and a lot of up-front planning to do this right.
         So if an SRA spends -- if the senior resident spends about a
     week, and of that week, he gets a day of the SRA's time to overview his
     recommendations and results, that is probably in the ballpark of the
     expertise that is likely to be on most facilities.  So that can't
     complicate it too much.  It has got to be very user-friendly.
         Again, not to say in the future we won't try to computerize
     it, but to get something done this year, this is the best we
     conceptually could approach it with.
         DR. POWERS:  I was certainly not thinking about this year, I
     was thinking --
         MR. GILLESPIE:  Okay.
         DR. POWERS:  -- ten years from now.
         MR. GILLESPIE:  Yeah.  I think, again, then you are to the
     point of probably having site-specific profiles.  And the degree of
     sophistication ten years from now, and someone will have settled the
     fidelity of the PRA question, I hope, by then.  We will either have an
     ANSI standard or someone will have a standard that these will be
     measured against, and then we can move forward.  Lacking that, this is
     the approach we have taken to do it.
         MR. FLACK:  There is one other thing, what we have raised,
     we have discussed -- we are discussing with Frank, is developing these
     generic matrices to make them plant-specific.  So, it would be a matter
     of going to the site and finding out what additional sequences would
     have to be considered and trying to fold that into a matrix form for
     each site, so each site would have its own risk set of insights that
     then would be updated and used by the inspector, but these are just
     things we are entertaining at this time.
         DR. POWERS:  I think there is no question that that will be
     one of the most interesting aspects of the pilot.  How do we go from a
     generic kind of appendix that, quite frankly, I don't understand right
     now, but that is my fault, to the specific activity will be real
         MR. GILLESPIE:  Yeah, because then you are going from, as
     Bruce said, areas that contain multiple systems to specific systems that
     are going to be sampled.  And we also view that as a significant
     challenge, because the directions on how to do that right now are
         DR. BARTON:  Frank, is this the time to take a break?  Till
     25 after and then we will hear from Michael.
         DR. BARTON:  I think we have got a quorum here, so we will
     get started.  Michael, you are next, with the assessment process.  I
     don't know if there's any other questions for Pat or not at this point,
     I don't think so.
         MR. JOHNSON:  Okay.
         DR. BARTON:  We will find out if he leaves, though.
         MR. JOHNSON:  Okay.  I just have a couple of brief slides to
     put up, and I don't want to -- or I hadn't intended to go over the
     entire assessment process.  We did that the last time we met with the
     ACRS.  I just wanted to remind us that the purpose of the assessment
     process is to assemble the information that we get, both from the PIs
     that we are collecting and from the inspection, the risk-informed
     baseline inspection, plus the additional inspection that we do, assemble
     that information to arrive at conclusions regarding plant performance
     and then, based on those conclusions, to identify resultant regulatory
     actions and then communicate the results of those, those actions, and
     the results of the assessment to the public.
         And then, finally, we want to provide a means of follow-up
     such that we look at future performance, factor that into the assessment
     process and make adjustments as appropriate, based on whether the
     licensee is or is not responding and addressing the problems that we see
     as being important.
         The second slide I will put up is this slide I will call key
     concepts.  I just wanted to remind us that the process is based on
     thresholds.  We have done a lot of talking this morning about thresholds
     for the PIs.  Also, keep in mind that we will have thresholds -- a means
     to evaluate the significance of findings from the inspection program and
     thresholds associated with that in each of the cornerstone areas.  And
     it is those thresholds that really drive us, or put us in the position
     where we consider additional action beyond allowing just the licensee to
     take action based on their performance.  So it is very much driven by
         The assessment process considers the performance for a
     12-month rolling window.  That is not new, we have been talking about
     that for a while.  It is a rolling 12-month period of time that we are
     looking at, PIs being provided each quarter, and the inspection results
     will come in as we accomplish inspections.
         The assessment process provides a graded approach.  One of
     the things that is in Attachment 4 is something called an action matrix. 
     And in looking at that matrix then, and with the PI results and the
     inspection results, we have a graded way of looking at what we
     communicate to the licensee based on performance, the actions that we
     take, whether we do inspection, so and so forth.  All of our actions,
     our management involvement are graded such that plants who perform
     better get, if you will, the minimal level of engagement by the NRC. 
     Plants who perform -- whose performance crosses multiple thresholds get
     a greater degree of involvement by the NRC.
         The assessment process eliminates use of the watch list and
     superior performer recognition.  That was one of the things that even
     the IRAP concept proposed, and we thought it was important then, and we
     think it is still important that we do away with things like superior
     performer recognition and the watch list, we don't think they serve a
         And, finally, one of the things that we needed to do with
     the assessment process was look at plants that are in an extended
     shutdown.  Extended shutdown has an impact on the PIs, as you are well
     aware, and the inspections, the additional inspections that we do, we
     think it is necessary to do, with this assessment process that we are
     proposing, exactly what we did today, and that is to set those plants
     aside for this assessment process.
         And we rely on the Inspection Manual Chapter 0350 process
     that has restart approval requirements for us in terms of what we will
     look at to decide whether a plant is ready to restart, and we will make
     some logical decision about when that plant should rejoin the assessment
     process following their restart at the end of that extended shutdown
         Let me just put up very briefly the action matrix.  I
     mentioned this in terms of talking about the fact that the assessment
     process provides for sort of a graded range of actions.  If you will
     look across the top, that gives you the results from anything from a
     plant that would receive --would not cross any threshold, any PI
     threshold, or any inspection threshold, all the way across to plants who
     have crossed multiple thresholds.  And then if you look down the left
     column, we talk about management meetings, the things that we do in
     terms of response, management meetings, licensee actions, NRC inspection
     and regulatory actions, and also the communication.  And so, for
     example, if you look in the left column, you can see that a plant that
     hasn't crossed any threshold, a plant with all the indicators and all
     the inspections in the green band, simply receives a risk-informed
     baseline inspection program.  And, again, as you move to the right, the
     responses becomes graded in significance, increasing significance.
         Let me mention one last thing, and I will shut up and take
     whatever questions you have.  There's --
         DR. APOSTOLAKIS:  Mike.
         MR. JOHNSON:  Yes, sir.
         DR. APOSTOLAKIS:  Would you put that back on?
         MR. JOHNSON:  Okay.
         DR. APOSTOLAKIS:  Where can I find it here?  Where is it, do
     you remember?
         DR. WALLIS:  Twenty-five.
         DR. APOSTOLAKIS:  Twenty-five.
         DR. WALLIS:  In the handout.
         DR. APOSTOLAKIS:  No, I know.
         MR. JOHNSON:  It's Attachment 4.
         DR. BARTON:  Attachment 4, George.  Attachment 4, page 19, I
         DR. APOSTOLAKIS:  Well, page 9 of the --
         MR. JOHNSON:  And page 9 of Attachment 1.
         DR. APOSTOLAKIS:  Attachment 1.  Page 9 of Attachment 1.  I
     am wondering, under Roman II there, one or two inputs white, but the
     cornerstone objective is fully met.  Why don't we move the regulatory
     action of the commenting response and make it a licensee action, and we
     do nothing?  And I am not even sure that the SRI or the branch chief
     should meet with the licensee.
         MR. JOHNSON:  I think the answer to your question is, when
     we cross a threshold, and that is what we are talking about in that
     second column, we would do some follow-up inspection in that area,
     whether it is a PI or whether it an inspectable area from the
     risk-informed baseline inspection program.
         And so when we talk about documenting the response in the
     graded area and in the inspection report, what we are talking about is
     just making sure that that inspection report captures what the inspector
     has found in that additional inspection that follows up in that area. 
     We would do that.  It is sort of an obvious outcome of that inspection
     that you have done to address that fact that you have crossed that
         DR. APOSTOLAKIS:  Well, yeah, I understand that, but I guess
     what I am saying is in the name of granting relief, to both them and us,
     why don't we start getting involved when one of the cornerstones is
     degraded?  In other words, your third column, Roman III, you have one
     degraded cornerstone, so now the NRC is beginning to provide increased
     oversight, you know, that kind of thing.  It is really a matter of
     policy, but I am just asking why we can -- can we not do --
         MR. JOHNSON:  Yes.
         MR. GILLESPIE:  You're right, George, it is a matter of
     where you set the thresholds, and how much margin you are going to have
     from you get in a more interdictive mode.
         DR. APOSTOLAKIS:  Right.
         MR. GILLESPIE:  And we have basically set the threshold.  In
     fact, if you look in Reg. Guide 1.174, there is a graph in there that
     looks very much like the graph Pat uses but it is upside down.  And we
     are, in essence -- we are, in essence, adapting those same thresholds. 
     The bottom band or the least risk significant band in that Reg. Guide,
     where things would be turned over to licensees, is very consistent with
     the description of what we have got in the licensee response zone, or
     the green band.  And so what we did was try not to invent new policy,
     but adapted that perspective.
         DR. APOSTOLAKIS:  Yes, but in that guide, though, they are
     dealing with core damage frequency in there.
         MR. GILLESPIE:  Yes.  And --
         DR. APOSTOLAKIS:  Here, you are talking about not even a
     cornerstone being degraded.
         MR. BARANOWSKY:  But the thresholds were based on risk
     insights that put us into a core damage frequency level of risk
     comparable to that in Reg. Guide 1.174, where the NRC would provide
     review and approval of changes in licensee design and operational
     features, so --
         DR. APOSTOLAKIS:  Wait a minute.  I mean you can have --
         MR. BARANOWSKY:  That is what the white zone is meant to
         DR. APOSTOLAKIS:  Yeah, but you mean you can have one or two
     inputs white, when, in fact, the cornerstone objectives are fully met,
     and see a difference in CDF?
         MR. BARANOWSKY:  No, the -- well, yes.
         DR. APOSTOLAKIS:  Really?
         MR. BARANOWSKY:  Well, I mean you can calculate the
     difference in CDF, it is small.  I mean, for instance, in Reg. Guide
     1.174, it says the licensees can make permanent changes to their plant
     with a CDF change of up to about 10 to the minus 5.  And the white band
     of this performance model includes core damage frequency changes up to
     10 to the minus 5, so it is meant to be consistent with Reg. Guide
     1.174, where there is regulatory oversight of risk-informed applications
     that occur with changes up to 10 to the minus 5.
         The same thing over here, when you cross any one threshold,
     the implications are comparable to being in that kind of a situation
     with regard to changes or increases in risk.
         DR. APOSTOLAKIS:  Well, I would expect that if the
     cornerstone objectives are fully met, even if one or two inputs are
     white, you wouldn't see much --
         MR. BARANOWSKY:  Well, you are not going to see much.
         DR. APOSTOLAKIS:  Unless the plant was already one of the
     great performers, way below the threshold, so by increasing it a little
     bit, you know, --
         MR. BARANOWSKY:  This isn't a regulatory hammer.  This is
     really just the beginning of taking a look so that we don't have things
     go from everything is all right to shut them down the next day.  This
     way the licensee sees it coming.
         DR. APOSTOLAKIS:  Right.
         MR. BARANOWSKY:  There is plenty of warning.  The NRC sees
     it coming.  And, in theory, it is a very modest regulatory engagement at
     that point.
         MR. JOHNSON:  Yeah, and it is very much -- it is very much
     philosophical.  The notion was if the system is set up on cornerstones,
     you don't wait till you have got a degraded cornerstone before you
     engage.  You engage at a point where you are seeing some degradation,
     but not degradation significant enough to take down the cornerstone, if
     you will.
         DR. APOSTOLAKIS:  Yes, but if you look at 3, it says
     cornerstone objectives met with minimal reduction in safety margin.  So
     it is not that, you know, the next category is something that already is
     pretty bad.  You are still meeting the objectives even in Roman III.
         MR. JOHNSON:  Right, but we are talking about --
         DR. APOSTOLAKIS:  What I am saying is doing nothing in the
     first two columns except being informed and then start doing things 3
     and above.
         MR. BARANOWSKY:  Not meeting a cornerstone objective is
     considered pretty significant, it is not the beginning of engagement for
     us.  I mean that is the equivalent of saying I don't have any
     defense-in-depth or something like that.
         DR. APOSTOLAKIS:  No, but you are still meeting it, though. 
     Cornerstone objectives met.
         MR. BARANOWSKY:  No, I understand that.
         DR. BARTON:  Yes.
         MR. BARANOWSKY:  But if we wait until we get to the point
     where things are really broken, that is one of the concerns that was
     raised about not having leading indicators in the current process in
     which one day the plant is operating, the next day we go there and we
     say, oops, we have to shut you down because this is really broken badly. 
     We are trying to have a process here that slowly steps into a
     recognition of declining performance so that there is plenty of
     opportunity to turn that around before we ever get to these level 4 or 5
     conditions.  I mean we never want to get there.
         DR. APOSTOLAKIS:  I know, 4 and 5, I agree.
         MR. GILLESPIE:  Remember, this is an indication system, and
     the severity or the pervasiveness of the root cause to the reason the
     indication has tripped a threshold is really the question, once you trip
     a threshold.  And, in fact, NEI's proposal, which we have spun off of
     here, up-front said if you meet all the PIs that are proposed, that
     should be evidence that your corrective action problem identification
     system is working.  If you trip a threshold, there is at least some
     essence of evidence that your problem identification corrective action
     system isn't working.  That is all we have got.  We have got some
     evidence that problem identification corrective action isn't working. 
     Now, when we get more diagnostic, the question is, how pervasive is it
     or isn't it?  This is an indication where we ask that question.
         So that is kind of one of the fundamental premises that even
     the industry proposed, actually, similar levels.  If you remember, they
     proposed three scrams and we kind of verified that that looked good to
     us on the same basis.  So that is how we are coming at it.
         Could the root cause -- are we good enough to have created
     the perfect system?  No.  So that is how the thresholds were set, and
     that is the intent of the indication.  The primary indication is there
     is a problem with problem correction and identification of problems.
         MR. JOHNSON:  There is one other thought I ought to mention,
     too, George, and it is in Attachment 4, and it mentions the fact that
     when we go to apply this action matrix, one of the first things we would
     do is to consider what the licensee has already done.
         So, for example, if you -- let's suppose you have one or two
     of these areas where the licensee has crossed the threshold.  The first
     thing you would do is to look to see if the licensee had taken some
     actin to do some diagnostics, had taken some action to address the
     problem, and that would further temper what we would do in terms of the
     follow-up inspection that we would plan.  So, we really see this second
     column as one that is really a very measured response, but necessary as
     licensees begin to cross these individual thresholds associated with PIs
     and inspections.
         DR. APOSTOLAKIS:  But look at the first column.  All
     assessment inputs are green, and there is licensee action, licensee
     corrective action.  To correct what?
         MR. JOHNSON:  Well, that simply says if the licensee is in
     the green band, all of these indicators are in the green band, where the
     inspection would find issues, we would simply put those issues in the
     licensee's corrective action program and the licensee would address them
     in accordance with that whatever prioritization scheme they set up,
     because the licensee has a good success at doing that.  This is the
     routine dealing with issues as they arise.
         DR. APOSTOLAKIS:  So there will be issues even though all
     assessment inputs are green?
         DR. BARTON:  Oh, yeah.
         MR. JOHNSON:  Oh, sure.
         DR. BARTON:  The world is not perfect, they are saying. 
     There may be little blibets, but you are still going to be green because
     you haven't crossed any threshold.
         DR. APOSTOLAKIS:  Like what issues?
         DR. BARTON:  I don't know, give us an example of where it
     could be.
         MR. GILLESPIE:  Tag-out problems.
         DR. BARTON:  All right.  Equipment tag-out.
         MR. GILLESPIE:  Equipment tag-out.
         DR. BARTON:  There you go.
         MR. GILLESPIE:  Not safety significant, didn't compromise
     the system.  Inspector sees it, it is a requirement.
         DR. APOSTOLAKIS:  But that is my point, though.  Why should
     you worry about it if all the assessment inputs are green?
         MR. JOHNSON:  And that is our comment is we don't worry
     about it.  That is left to the licensee.
         DR. APOSTOLAKIS:  But why would you even notice?  Why would
     you even bother to look at it?
         MR. BARANOWSKY:  Because you don't know, until you get all
     the facts together, whether or not the facts indicate there is a
     problem.  You can't go and say there is no problem so I won't look for
     facts.  Your job is to go get the facts and then see if they indicate
     that you should have a more aggressive examination or follow-up
     activity.  So the first step is getting the information, that is all in
     the green is about.  Baseline inspection program to get baseline
     information.  Performance indicators, inspections.
         DR. APOSTOLAKIS:  Wait a minute.  All the assessment inputs
     are green, everything.  All the indicators, the areas, right?
         MR. BARANOWSKY:  Right.
         DR. APOSTOLAKIS:  There is nothing to correct.
         MR. JOHNSON:  No, that's not true.
         MR. BARANOWSKY:  The NRC won't be taken a corrective action.
         DR. BARTON:  But the licensee has some actions.
         MR. BARANOWSKY:  The licensee, when they find tag-out
     problems, will be taking a corrective action.  The theory is if you let
     the little things go, they become bigger, and then they become bigger,
     and then they become bigger.
         DR. SEALE:  Indeed, one of the concerns is that by the time
     they reach -- things reach the level of recognition on the NRC's screen,
     you may already be on a slippery slope that may be very difficult to
     turn around.  It is my understanding that INPO is looking at indicators
     at a much more sensitive level, as many as a hundred per plant in some
     cases, and that they would hope that using those indicators, the
     licensees themselves, with INPO's help, would be able to anticipate
     things so that you never let any -- nothing you ever saw got out of the
         MR. BARANOWSKY:  Yes.
         MR. GILLESPIE:  There's actually -- there's INPO -- WANO and
     INPO are kind of one in this -- and WANO's had a lot of input into an
     IAEA document which is currently circulating in draft -- I don't believe
     it's available for the public -- which takes the next step that we've
     said we wouldn't take.  Not only is it indicators from the facility, but
     it talks about safety culture --
         DR. SEALE:  Yes.
         MR. GILLESPIE:  And it talks about indicators of management. 
     So there is internationally and nationally the industry's themselves
     looking at it.  The IAEA document is a document for station personnel,
     and specifically in the front of it it says this is not intended for the
         DR. SEALE:  Yes.  Yes.
         MR. GILLESPIE:  So there is that activity.
         DR. SEALE:  Yes, but it does address these --
         MR. GILLESPIE:  Yes.
         DR. SEALE:  These culture-related concerns that we've raised
     in the past in some of our discussions.
         DR. APOSTOLAKIS:  Well, again, it's okay for INPO and the
     industry to do things like that.  When we do something, I mean, we have
     to be a bit more careful.  And -- let me put it in a different way.  I
     get the impression that we are changing, you know, it is risk-informed
     and so on, but we still -- we don't want to let go, in other words.  We
     still want to get involved in detail.
         A bolder approach might be to say regulatory actions, none
     in the first column, none in the second.  Just be informed of what's
     going on, and then start worrying about it at 3.  And if they have
     tagging problems, it's their problem, you know.  We don't get involved
     in these things.
         MR. GILLESPIE:  Well, I think, you know, we've kind of got
     -- now we're stuck in the compromise position we're in.  We have a set
     of regulations and requirements currently out there.  An inspector walks
     through the plant, and while he's looking at risk-significant systems
     and risk-significant spaces, and he's looking for safety problems, he
     sees that tags are mistagged and valves are mislocked.  It doesn't rise
     to being -- it doesn't negate a system, doesn't negate a train, doesn't
     maybe rise to that level.  Does he walk away and ignore that and not
     tell anybody?  That's what you're suggesting, George?  We can't tolerate
     that.  He has to document it.
         Now how we dispose of it becomes the question.  And what
     we're suggesting here as the disposition is it gets turned over almost
     as if he was a QA inspector and walking through the same space, he gets
     integrated into the licensee system.  Now we're trying not to put undue
     pressure or undo priority on it, and in fact gets in their system and
     gets fixed in accordance with whatever priority system they're using.
         DR. BARTON:  It's level 4.
         MR. GILLESPIE:  It's like the level 4 policy statement going
     through now.
         DR. BARTON:  Right.
         MR. GILLESPIE:  I mean, that's the approach.  The second
     column on this, the darn senior resident lives at the site.  Him
     finding, asking, you know, why did you bust this threshold, when he's
     already there, doesn't -- is kind of a real measured response.  I mean,
     I would be hard pressed to have someone bust a threshold and have one of
     the two residents we've got at the site say I'm not going to ask.  So
     it's a real measured response.  It's kind of proportional to the idea of
     busting one threshold.
         You have your resident at his normal weekly meeting and they
     generally, although they're at six-week reports, they generally have
     kind of a debriefing weekly, and what their findings are says hey, you
     know, your quarterly report showed this.  What's going on here?  It
     seems like a very measured response.  Could it be none?  I don't know
     how we'd administer none and tell a resident not to ask that question.
         DR. MILLER:  These responses you judge are minimal.
         MR. GILLESPIE:  Yes.  Oh, yes, column 2 is very minimal.
         DR. MILLER:  I don't know how you'd define it, but --
         MR. GILLESPIE:  It is.
         DR. MILLER:  They would just be routine, the senior
     residents walking around or the residents.
         MR. GILLESPIE:  Right.
         DR. MILLER:  And we need to document it, because it does --
         MR. GILLESPIE:  Yes.
         DR. MILLER:  Break the written rule, but it has no effect on
         MR. GILLESPIE:  Right.
         DR. SEALE:  You're letting the licensee know that the
     inspector is not brain-dead.
         MR. GILLESPIE:  Right.  The other thing is you're letting
     the public know that we're still looking.  Now we've got another
     audience.  It isn't just us and the licensees.  And, you know, one could
     say well, the inspectors won't bother writing any inspection reports
     unless they found a complete cornerstone completely degraded.  And
     that's not acceptable in the public nature of our business.
         DR. MILLER:  But the burden on the licensee in column 4 is
     probably equal if not somewhat less than it is today.  Is that right or
         MR. GILLESPIE:  Yes.  In fact, at the Commission meeting --
     yes, column 4 -- at the Commission meeting said --
         DR. MILLER:  Column 2; I'm sorry.
         MR. GILLESPIE:  Oh, okay.  I was going to say the
     Commissioners actually commented on column 4 and 5.  They spent some
     time on this and said move the Commission over to column 4.  That was
     their comment.  So, yes.
         DR. MILLER:  But column 2 is less than it is today, or equal
         MR. GILLESPIE:  Oh, yes.  I would expect that column 2 would
     just crossing one threshold where there's a readily available
     explanation would not be sending a three-man team out to investigate it. 
     The first thing is the SRI, what's happening here.  Do you have a
     reasonable explanation?  If the answer is yes, he documents reasonable
     explanation.  The other nature of the public agency we are, we're going
     to be publishing these PIs quarterly.  We're going to be -- it would be
     irresponsible to publish a PI that shows a threshold crossed and have no
     text that said here's what it was, and it appears to be okay.
         DR. SEALE:  I'm intrigued by that bottom row down there
     where you have a differentiation or a dividing line between regional and
     agency review.  You've got a pilot coming up.  Will all of those pilots
     be in one region?
         DR. BARTON:  They could be in all regions.
         MR. JOHNSON:  No --
         DR. SEALE:  One in each?
         MR. JOHNSON:  Two pilot plants in each region.
         DR. SEALE:  Yes.  Okay.  Some in each region.
         MR. JOHNSON:  Eight.
         MR. GILLESPIE:  Eight pilots.
         DR. SEALE:  Some in each -- okay.  It's going to be very
     interesting to see the degree to which unanimity of what that means
     emerges from the different regions.  And that's going to be a job for
     you, I think.
         MR. GILLESPIE:  Consistency --
         DR. SEALE:  Yes.
         MR. GILLESPIE:  Well, the starting point is we'll have
     something we don't have today.
         DR. SEALE:  Sure.
         MR. GILLESPIE:  We've at least got a first shot at a scale. 
     Now we can adjust the scale in effect.  That's the kind of comments
     we're expecting to get.  But at least we have a starting point to
     measure people against.  One of the major criticisms of the whole
     oversight assessment process was we didn't write our criteria down up
     front, and so here we're attempting to do that.
         DR. SEALE:  You may have heard that from us, too.
         MR. GILLESPIE:  Yes.
         DR. APOSTOLAKIS:  One of the things that still is not clear
     to me, Frank, and maybe you can clarify it, is okay, you go down and you
     do the inspection and the licensee -- the performance indicator on
     scrams is high, above the threshold.  And the licensee says there is
     nowhere in my license that says that I have to keep this below 3.  I'm
     not going to take any corrective action.  It was nine from Day 1.  Why
     are you asking me now to change it?  What is the new regulation that's
     forcing me to do this.  And you say PRA.
         MR. GILLESPIE:  No, we're not going to ask them to change.
         DR. APOSTOLAKIS:  Okay.  You're not going to say PRA.
         MR. GILLESPIE:  No.
         DR. APOSTOLAKIS:  What would you answer -- how would you
         MR. GILLESPIE:  At that point we would end up doing
     additional inspection, since he's refused to do a root-cause analysis,
     and the inspection would likely be on his root-cause analysis problem
     identification process, and we would then leak that back to regulatory
         MR. JOHNSON:  Right.  What caused the scrams.
         MR. GILLESPIE:  And if he's not violating any regulatory
     requirements, then you're right.
         DR. APOSTOLAKIS:  That's right.  That's what I'm saying.
         MR. GILLESPIE:  The nine scrams is an indication we need to
     look further and tie that to what requirements are either being violated
     or what new requirements are needed.  We're not going to force him to
     meet the threshold.  But if anyone has nine scrams in a year --
         DR. APOSTOLAKIS:  Let's say you find that no regulatory
     requirements are violated.
         MR. GILLESPIE:  Oh, we could find that, in a difficult
     startup after a long shutdown.
         DR. APOSTOLAKIS:  What was the frequency of scrams in the
     United States 15 years ago?
         DR. BARTON:  About nine.
         MR. GILLESPIE:  Yes, nine.
         DR. APOSTOLAKIS:  Nine a year.
         MR. GILLESPIE:  Eight, nine a year.
         DR. BARTON:  That's in the ballpark.
         DR. APOSTOLAKIS:  Once we're licensed, the guy says it was
     nine then, it's nine now.
         MR. BARANOWSKY:  Yes, but we have both Appendix B and we
     have the maintenance rule, and both of those regulatory requirements
     force you to take corrective actions when your reliability or
     performance is not up to what's expected.  In the maintenance rule we
     have some targets in it for you to meet.  And clearly this would be
     outside of what would be called for in the maintenance rule, and
     certainly Appendix B has elements of quality in it that this would
         DR. APOSTOLAKIS:  Are the targets of the maintenance rule
         MR. BARANOWSKY:  Yes.
         DR. APOSTOLAKIS:  Okay.
         MR. BARANOWSKY:  And I can tell you one thing, they're
     certainly less than nine.
         DR. SEALE:  Yes.  I think there's another way to look at
     this.  Nine years -- or ten years ago we had nine scrams.  Ten years ago
     we had the inspection system that we're still living with today.  When
     we don't have nine scrams a year, maybe it's time to take a look at the
     inspection system and bring it into more of -- well, in tune with what
     the practices are today.  But you still use that as the thing that opens
     the door to the kind of inspection that is more reminiscent of what you
     have today if somebody does show up with nine scrams.
         DR. APOSTOLAKIS:  My fundamental problem is that the basic
     regulations that people have to comply with are not risk-informed.
         DR. SEALE:  True.
         DR. APOSTOLAKIS:  And the inspection oversight program is
     risk-informed, and nobody sees a possible conflict here?
         MR. GILLESPIE:  No, we do.  We do.  In fact, we highlighted
     that in the Commission paper as a conflict, because we are right now out
     in front of the regulations.
         DR. APOSTOLAKIS:  Yes.
         MR. GILLESPIE:  Absolutely.
         DR. APOSTOLAKIS:  We are changing the way we oversee things.
         MR. GILLESPIE:  And --
         DR. APOSTOLAKIS:  Without knowing why.
         MR. GILLESPIE:  No.
         DR. APOSTOLAKIS:  We have empirical knowledge --
         DR. BARTON:  So we shouldn't improve this process --
         MR. GILLESPIE:  We know why we're changing, but -- and
     that's the difficult part.  The only step we can take is, we can't
     enforce thresholds.  Thresholds are an indication as to when we go
     further.  And then we're forced, if you would, into making a link of
     what is the requirement that links to the root cause.
         MR. JOHNSON:  We're forced to use the existing regulatory --
         MR. GILLESPIE:  Structure.
         MR. JOHNSON:  Structure to take action against the licensee.
         MR. GILLESPIE:  I would love not to have to do that, but
     that's where we are.  That's our only option.
         DR. KRESS:  The link is that you're assuming that you have a
     body of regulations there and you're assuming that if a plant does meet
     all those regulations that are out there, then it is more likely in the
     green zone?
         MR. GILLESPIE:  That's inherent in the assumption.
         DR. KRESS:  And so you've got indicators that you don't have
     to go in and see if they meet all the regulations, you just go look at
     these indicators.  Now if somebody gets out of them, your assumption is,
     well, more than likely they don't meet some of these regulations, so
     we'll go in and see.
         MR. GILLESPIE:  Right.
         DR. KRESS:  And you may find that they still meet the
         MR. GILLESPIE:  That's true.
         DR. KRESS:  Or you may find that they don't.  And your
     response is going to be depending on what you find by that additional
         DR. APOSTOLAKIS:  What do you think the response --
         DR. KRESS:  So I think they're forced to go in and their
     response has to be well, do they meet the regulations or not?  And maybe
     those regulations will change over time, and so forth, but I think
     they're right in saying --
         DR. APOSTOLAKIS:  What do you think the response --
         DR. KRESS:  That they're boxed in to saying meet the
         DR. APOSTOLAKIS:  What do you think the response will be if
     they find that they meet the regulations but two of the performance
     indicators are above the threshold?
         DR. KRESS:  That's a good question.  That's where the
     conflict's going to be.  What do you do about that?  I think they have
     to backfit or --
         MR. GILLESPIE:  No --
         MR. BARANOWSKY:  That's so highly unlikely, because what we
     currently have today is performance that would be well within the green
     band in which we still cite licensees on Appendix B violations.  It's
     exactly the opposite problem as the real world than you're postulating
     right now occurring in the future.
         DR. SEALE:  There's also another response, and that is while
     they may be complying with what the regulations state, they're certainly
     not meeting the utility peer group's aspirations for excellence, and the
     other utilities are going to be all over them, because they have the
     same indicators, they're looking at them for the same kinds of concerns,
     and nine is not a satisfactory way of doing things in their world where
     they think in terms of excellence and performance.
         DR. MILLER:  And that peer pressure by some is perceived as
     being more of a hammer --
         DR. SEALE:  That's right.
         DR. MILLER:  Than is the regulatory pressure.
         DR. APOSTOLAKIS:  What kind of world are you guys
     describing?  Plato came again down here and I didn't realize it.  This
     is so platonic.  This is the ideal world.  You know, you will be shamed
     and your peers will come and correct you?  Has that happened before?
         DR. SEALE:  It's happened before.
         MR. BARANOWSKY:  Actually the PUCs are also important here. 
     When the PUCs end up seeing something happening, there's a question as
     to whether or not --
         DR. SEALE:  That's right.
         MR. BARANOWSKY:  There should be changes in rates and so
         MR. GILLESPIE:  And all of that, we're still an independent
     regulatory agency, and we're setting our -- the fact that our thresholds
     and our perspective is coincident with some societal pressures is very
     nice in a system point of view, but clearly these are our thresholds in
     our action statements.
         DR. KRESS:  When you come to enforcing things, I think you
     really are bound by the existing regulations.
         MR. GILLESPIE:  Oh, yes.
         DR. KRESS:  If you end up having this conflict, George,
     where -- and I'm sure it can exist -- where the risk performance metrics
     appear to be way out of line and they're still meeting all the
     regulations, I think you've got a problem with the regulations.
         MR. GILLESPIE:  The ideal piece where this would be
     explained away --
         DR. KRESS:  Fix the regulation.
         DR. BARTON:  Fix the regulation.  Right.
         DR. KRESS:  But that's a -- you know, that's a backfit and
     that's a rule and you have to make new rules.  But I think that's where
     you have to attack.  If you find this to be a real problem that's
     pervasive, I mean, I think you can stand it --
         DR. APOSTOLAKIS:  Or you can live again with an unfortunate
     situation like after the IPEs go down we realize that 19 PWR units were
     above the goal but they meet the regulations.  There's nothing we can
     do.  Right?  Is that a happy situation for us?
         DR. KRESS:  Probably not.
         DR. APOSTOLAKIS:  Probably not.
         DR. BONACA:  I just have a comment I'd like to make.
         DR. KRESS:  The backfit rule is one of the things that's got
     us into this box.  You know, if you can't satisfy the backfit
     requirement, you can't make a new rule.  And that's one of the things
     that have us in this box.
         DR. BARTON:  Mario?
         DR. BONACA:  Yes, I had a comment regarding -- one thing
     that, you know, looking at these performance indicators, in fact, I
     don't know how many plants would not meet these performance indicators
     quantitatively.  I mean, I haven't seen many plants that have more than
     three scrams per year, I mean, in fact.
         And the concern I have is the reverse one, which is the one
     that rather than being an objective process again there is a lot of
     reliance on the complementary or supplementary inspections.  And so the
     burden becomes again on the staff to show that in fact it is an
     objective process that you are using.
         Maybe the question I have is how did you get these
     thresholds, and they seem to be in fact much more relaxed than what you
     have from INPO.
         DR. APOSTOLAKIS:  In fact, they go to page H-18.
         DR. BONACA:  I know.
         DR. APOSTOLAKIS:  You will find one, two, three, four, five,
     six, seven, eight plants whose number of unplanned scrams for a year is
     above the threshold.
         DR. BONACA:  Yes, that's right.
         DR. APOSTOLAKIS:  By design.  So what do these eight plants
         DR. BARTON:  What they'd better do is have a corrective
     action program to reduce the amount of scrams, but if you're shut down,
     you're not going to be economically feasible, pa da pa da pa da, down
     the path you go.
         DR. APOSTOLAKIS:  As a result of oversight?
         DR. BARTON:  Pardon?
         DR. APOSTOLAKIS:  As a result of oversight they have to do
         MR. GILLESPIE:  No, there's going to be logical explanations
     in some cases.  If you have a plant that's been shut down for two years,
     of which we happen to have several, and you go to start up, it is not
     unprecedented even in the first month you would have three, four, five
     scrams.  But you'll generally identify exactly what it was very rapidly,
     and you'll correct it and move on.  That's an explanation.
         DR. APOSTOLAKIS:  Incidentally --
         MR. GILLESPIE:  It means we have to understand and explain. 
     It doesn't mean we have to take action.  And the licensee just has to be
     able to explain it and it has to make sense and be logical.
         DR. APOSTOLAKIS:  I don't know.  I mean, that is related to
     Dana's question earlier about not regarding the PRA.  I don't know how
     NEI produced these figures.  Was it based on one year's performance?
         MR. BARANOWSKY:  No.
         DR. APOSTOLAKIS:  Was it based on a lot of years'
         MR. BARANOWSKY:  No.  It was based on I believe five years'
     performance and it ran from approximately 1992 through '97.
         DR. APOSTOLAKIS:  So for each plant it was the same
     denominator, five years.
         MR. BARANOWSKY:  It was on a per-year basis, but they
     collected for that five years, and then they took the peak value for
     each plant in that five-year period.
         DR. APOSTOLAKIS:  Oh.
         MR. BARANOWSKY:  Now there is a couple of things that are
         One is performance has been improving over that period and
     since that time, so there's a reasonable likelihood that there would be
     less hits in any given year than would be indicated by the 95 percent --
         DR. APOSTOLAKIS:  Because I was wondering if you have five
     years' worth of data for each plant, I mean the codes are available
     now -- a two-stage Bayesian thing would be --
         MR. BARANOWSKY:  We got 'em.
         DR. APOSTOLAKIS:  You have them already?
         MR. BARANOWSKY:  We can apply a lot more sophistication than
     we did.
         DR. APOSTOLAKIS:  Yes.
         MR. BARANOWSKY:  But we kept it very simple for this
     go-around.  We wanted to keep the mathematical eloquence to a minimum so
     that it could be more easily understood by the everyday inspection kind
     of person and licensees without going through a lot of arguments about
     is this a good computer code, is that a good mathematical method, is the
     prior proper and so forth -- we wanted to get away from that for now
     because we just wanted to try this out.
         It will evolve into something a lot more sophisticated as we
     get more plant-specific and more risk-based and we improve on the
     indicators, but for now, it is a very simple approach.
         The thresholds there have minimal risk implications.  We ran
     some calculations using those 13 models I told you about and for that
     particular threshold we didn't see anything that approached the 10 to
     the minus 5 change in core damage frequency from what would be a mean
     type of performance over that period of time, when we moved up to that
         In some cases we put all the plants' performance at the
     threshold to see if the risk was high, so it was a relatively low risk
     associated with operating within that performance band, and relatively
     low risk, what I mean there is what we have today.  That is low risk.
         Increases in risk when one moves up to those limits are
     small enough that they wouldn't cause us to be concerned with regard to
     the kind of thresholds for risk applications Regulatory Guide 1.174.
         MR. JOHNSON:  So let me just tell you what we'll see --
         DR. APOSTOLAKIS:  Before we go to that though, let's talk
     about the substance.
         MR. JOHNSON:  Which one, George?  Back to the matrix?
         DR. APOSTOLAKIS:  This is pages 14, 15 of the Attachment
     2 -- cross-cutting issues.
         MR. JOHNSON:  Yes.
         DR. APOSTOLAKIS:  Can we discuss those?
         DR. BARTON:  Go ahead.
         DR. APOSTOLAKIS:  You have a very nice discussion of safety
     culture which is under the name Safety Conscious Work Environment, but
     then you dismiss it -- "No separate and distinct assessment of
     licensees' safety culture is needed because it is subsumed by either the
     PIs or baseline inspection activities."
         Does everyone agree with that?
         DR. BARTON:  Well, I think they are driven to that based
     on -- it's the policy of the Commission is if you don't go and inspect
     safety culture, management effectiveness and all that stuff I think you
     are forced into this approach.
         DR. APOSTOLAKIS:  Why don't we say that?
         MR. BARANOWSKY:  That's not really -- I mean that may be one
     reason why you get to that conclusion, but that's not why we got there
     when we did this work.
         The real reason we got there is we're looking at outcomes
     that affect the objectives of the cornerstones that we identified.
         DR. BARTON:  Okay.
         MR. BARANOWSKY:  And our conclusion was that this is an
     underlying cause to outcomes.  As long as we measure the outcomes, which
     is the performance indicators and the inspections, and if those outcomes
     are acceptable we don't have to go and check into whether or not they
     have the right attitude at their plant.
         DR. APOSTOLAKIS:  So you are measuring human performance?
         MR. BARANOWSKY:  Sure.  People make mistakes and cause
     plants to trip.  People make mistakes and cause equipment to fail or be
     unavailable.  To the extent that those things don't occur, we are
     satisfied with the human performance.
         Now thee are some things that we can't measure with the
     performance indicators like how will the operator respond to a real
         That's where we would have inspection or related activities
     associated with the operator licensing program.  That is a complementary
     area for inspection.
         DR. APOSTOLAKIS:  Well, let's see what you are saying about
     post-initiator operator actions.
         You admit that they are far less frequent -- that's page
     14 -- while initial and requalification examinations provide a
     predictive measure of operator performance during off-normal and
     emergency operations, follow-up inspections of risk-significant events
     will provide a more direct indication of the adequacy of post-initiator
     human performance.
         In other words, you are saying that, look, we really -- I
     mean there is no way we can know how the guys are going to perform.
         MR. BARANOWSKY:  Actually, what we are saying is that we
     have had a very successful operator qualification and training program
     and that unless we have objective information or evidence given to us as
     a result of abnormal events that really challenge the operations and
     show that they didn't handle the situation properly, we are going to
     rely on that program to give us confidence that that is an acceptable
     area of licensee performance.
         We had some incidents occur in which operators have made
     some errors during these incidents, and those would be the things that
     we would look into to see whether or not those are an occasional human
     error problem or whether they are indicative of deeper problems with
     their program.
         DR. BARTON:  I think inspectors go look at, if you put those
     into the corrective action program, they go look at what is going on at
     the site with respect to improving human performance and reducing
     operator error.
         The inspectors look at that, so that's in real life.
         DR. BONACA:  Yes, the industry is moving, however, in a
     direction where you can group elements of human performance,
     safety-conscious work environment, problem identification in safety
     culture and you have very clear indicators of it.
         For example, you can look at operator challenges, temporary
     mods, operator work-arounds.  Those are typical indications of poor
     culture if you don't correct those, or for example, corrective action
     programs that doesn't solve you issues or doesn't have the proper root
     cause.  That is an indication of that.
         They may be worthwhile in fact at some point to group this
     other indicator that you have inside here under some heading -- because
     the industry is moving in that direction and I think they should be
     encouraged to move in that direction, and I think something could be
     done inside here.
         MR. BARANOWSKY:  In fact, we would encourage them to do that
     kind of thing because that is like an early indication of an indicator,
     if you will.
         DR. BONACA:  That's right.
         MR. BARANOWSKY:  And what they are trying to do is maintain
     their performance above that threshold and so by having these kinds of
     additional performance indication or inspection activities that they
     perform themselves, which go really beyond what the NRC does, they have
     the opportunity of controlling their own destiny.
         DR. BONACA:  That's right.
         MR. BARANOWSKY:  That's really the idea that we are trying
     to foster here is that the NRC does one to five percent of all the
     inspections and evaluations that go on at a plant.  The licensee does
     the other 95 or 96 percent or whatever it is, and it is their ability to
     do that stuff correctly that we are trying to take sample information on
     to see what the indications are.
         MR. JOHNSON:  Right.  In fact, the industry was very
     interested in looking at the things that you have just mentioned.  They
     were also very interested in having us not look at them if they don't
     reflect themselves in actual performance, and so that is sort of how we
     have come out.
         DR. BONACA:  But you do have significant expectations in
     safety-conscious work environment and to me it seems you should
     translate those expectations in measurable things.  You know, I gave you
     some example of those because there are relationships there, so I
     understand the interest of the industry also not having you quantify
     some of those, but to the degree to which you are interested in those
     specific assessments, some correlation with the specific metrics could
     be important.
         DR. APOSTOLAKIS:  When the Commission decided that it's none
     of our business to look at safety culture, they interpreted safety
     culture the way you interpret it.
         Let me be a little more clear.  If you read reports from
     Vienna on safety culture, safety culture consists of two elements they
     say.  One is the organizational structure and the other is the
     attitudes, the norms and so on.
         You seem to be making a distinction.  You are saying
     safety-conscious work environment, otherwise known as safety culture,
     really refers to the norms and attitudes because you still worry about
     the organizational structure on the problem identification and
     corrective action programs.
         What is not clear to me is whether the Commission also
     thinks about it that way.
         DR. BARTON:  That's a good point.
         DR. SEALE:  Well, didn't they really say don't look at
     management --
         DR. BARTON:  Don't look at management effectiveness and
         DR. APOSTOLAKIS:  But what does management mean?  It means
     your work processes and the way you manage business or only the
         DR. BARTON:  I think it is the former.
         DR. APOSTOLAKIS:  Well, then, how come we have a full
     section here on problem identification that says the scope of problem
     identification corrective action programs includes processes for
     self-assessment, root cause analysis, safety committees, operating
     experience, feedback and corrective action.
         These are processes.  They are part of the organization
     structure, so from this document I get the impression that this is
     outside safety culture, whereas the INSEC documents is part of the
     safety culture.  You have to make sure you have good processes and so on
     and so one, which is fine with me, but now that explains the
     Commission's decision because I am a little puzzled by the decision if
     it includes processes for self-assessment root cause analysis and so on.
         I mean obviously we do care about these things --
         DR. SEALE:  It's my impression, George --
         DR. APOSTOLAKIS:  -- so these are not safety culture.
         DR. SEALE:  It is my interpretation, this personal
     interpretation, that they don't -- do not want to be in the position of
     having appeared to have told a utility, whether it has a vice present or
     a -- whatever -- and exactly how many people report to that person and
     whether their training organization reports to a Nuclear Executive Vice
     President for the whole organization or the guy in charge of a given
     plant, those kinds of questions.
         DR. APOSTOLAKIS:  Okay, so somehow we have to make a
         DR. BARTON:  George, I think if you look at this, this is
     really looking at safety culture, if you are looking at the thing that
     you read.
         DR. APOSTOLAKIS:  Yes.
         DR. BARTON:  If you look in the corrective action program,
     self-assessment, root cause analysis, safety committee, you are really
     looking at what is the safety culture of that organization.  You really
     are.  I don't care what you call it.
         DR. APOSTOLAKIS:  And that is what INSEC said, but the
     previous section here said "a safety-conscious work environment, also
     referred to as a safety culture" and then it concludes by saying, "In
     short, no separate and distinct assessment of licensees' safety culture
     is needed."
         Then there is immediately another section that says, ah, but
     for processes that have to do with identification and correctional
     programs, we worry about those, so I think a lot of this is semantics.
         By the way, I don't disagree with you and that clears up a
     lot of things in my mind because I think Bob is right, Bob Seale, that
     the Commission did not want to appear as if they were interested in
     which Vice President reports to the President.
         DR. SEALE:  And the driving force behind that is that there
     are documentable cases in the past where certain people who are
     employees of the Commission maybe overstepped their authority and
     appeared at least to have interfered with that sort of thing and it did
     raise a considerable amount of ire and fury and heat.
         DR. APOSTOLAKIS:  Yes.
         DR. BARTON:  That came out in the Towers-Perrin Report.
         DR. APOSTOLAKIS:  And there are other parts of the safety
     culture that I do worry about.  For example, operator training and so
     on.  Yes -- in the broad definition, that is part of it, so we really
     are excluding here, and I think rightly so, the attitudes, goal
     prioritization, things that you can't really see, right?
         MR. GILLESPIE:  Right.
         DR. APOSTOLAKIS:  So maybe just change the words a little
     bit to make sure that that is what you are doing.     
         This is not really -- safety-conscious work environment, the
     way you describe it here, is not also referred to as "safety culture."
         MR. GILLESPIE:  I don't think we used the words "safety
     culture" but if we did we shouldn't have.
         DR. BARTON:  Yes, you do.
         MR. GILLESPIE:  We do.
         DR. BARTON:  You have got it thrown in there a couple of
         DR. APOSTOLAKIS:  Page 14.
         MR. BARANOWSKY:  That's a throw-away.  I am glad to have
     something the ACRS thought of something here.
         DR. SEALE:  You put it in quotes so it is --
         MR. BARANOWSKY:  I admit it -- we made a mistake.
         DR. APOSTOLAKIS:  I am not asking you to admit you made a
     mistake.  It's good that you did.
         DR. APOSTOLAKIS:  But it is a very important part though
     that you really worry about -- in fact, if you go to a plant or your
     resident inspector realizes there is no mechanism, there is no process
     for operating experience feedback, he probably won't like it.  Right?
         DR. SEALE:  Well, any root cause analysis that doesn't
     address concerns having to do with human action is idiocy.
         DR. APOSTOLAKIS:  Which is most root cause analysis.
         MR. BARANOWSKY:  When you cross the threshold and the
     resident inspector goes and says why is this occurring, and they see
     that they don't have a good root cause analysis activity, then you are
     getting into these things, but that is a cause of the problem kind of
     thing.  That's why we are saying you won't have an indicator for that by
     itself because if you did I mean that would be fine but we don't know
     how to relate some of these things to the cornerstones directly.  There
     isn't a simple algorithm for doing that.
         DR. SEALE:  The thing you detect is the consequence, not the
         MR. BARANOWSKY:  Right.
         MR. GILLESPIE:  And in fact independent of which cornerstone
     or which PI or which inspection activity might cause us to engage
     further in doing a root cause analysis, the end of the root cause
     analysis might find that problems exist in places not first indicated,
     because at that point you are in an analytic mode and how pervasive is
     it is the kind of question you are asking, how limited is it, was it
     isolated, was it not isolated, so you roughly go from one kind of
     inspection process and procedure to a different one when you get to that
         DR. BARTON:  I have seen you put your light out.  Does that
     mean you are done?
         MR. JOHNSON:  That means --
         MR. GILLESPIE:  I think he gave up on the last viewgraph. 
     He's not going to put it up.
         MR. JOHNSON:  We have moved beyond the last one.  I can put
     it up but I don't see a point.
         DR. APOSTOLAKIS:  I am done with my questions, Mr. Chairman.
         DR. BARTON:  You are done with your questions?  All right.
         Throughout the document, and I think you mentioned this
     morning you still have got some work to do and there's some not just
     shoe-strings to tie but you actually have some other open issues that
     are mentioned throughout the document.
         When do you plan to have that supplemental information
     ready?  When can we hear?  Will that be after pilot programs, after this
     thing goes out for comment?  What is the timing on when we'll hear what
     is missing in here?
         MR. GILLESPIE:  Let me ask Al Madison to come up -- Alan is
     the Team Leader for the Transition Task Force -- and ask him to address
     part of the schedule.
         DR. BARTON:  Is that the last slide you weren't going to
         MR. GILLESPIE:  No.  This is one that wasn't even on the --
         MR. MADISON:  I am Alan Madison with NRR.  We have met
     before.  Your specific question, so I can address it?
         DR. BARTON:  You go through the document and it says thee is
     more work to be done; we've got to do something here -- throughout, all
         My question is when will you fill in those blanks and when
     will you have that work done?
         MR. MADISON:  Well, I am glad you recognize there is a lot
     of work to be don, a lot of work left, and we're, as the paper alludes
     to, we are going to implement or establish a task force which we have
     begun to do.  Because of the broad scope of activities that are involved
     there will be a lot of folks involved in that task force.
         Some of the items -- Mike, you'll probably want to list the
     key transition tasks.
         These are some of the key tasks we think are involved in
     there and you probably noted a few others.  But we have broken it out
     into the work remaining and developing the procedures and the policy
     documents that are going to need support both the inspection procedure
     and the assessment procedures.  Enforcement policies also fit into
     there.  The performance indicators, how we are going to collect the
     performance indicators, how licensees -- the format they are going to
     report them in.  What are we going to do with it?  How are we going to
     get that out to the public?  All that is yet to be developed, the
     support infrastructure for those changes, as well as other changes in
     various areas.
         We have got a lot of training to do, we have got
     development.  Some of the jobs that were left to be done, including --
     well, it was called benchmarking, we are now really calling that a
     feasibility study.  Is the process -- is it feasible that we can take
     the performance indicators, that we can take inspection data and compare
     those and come up an assessment in the cornerstone areas?  We plan to do
     that during the transition.
         One of the biggest things, and that would be the next slide
     with the transition strategy.  One of the biggest things we recognized
     right away is there is a lot of communication and change management
     issues that we need to deal with in this.  And so we, as part of
     establishing the transition task force, we have named what we call a
     change champion who is going to be kind of our voice out in the industry
     and the agency, and to the public, and that is Sam Collins, Director of
     NRR.  He will be our primary communicator of what we are doing and where
     we are going.
         To assist him, to help him out in that, and to help us out,
     we are in the processing a change coalition, which we feel are the -- we
     are hoping these folks are what we are calling the opinion leaders in
     the various areas, and they may or may not be in management.  We are
     focusing right now on the regions, but we are also going to develop some
     folks here in headquarters that will be on that change coalition.  We
     see them as our primary source of information back from the field, how
     the transition, how the processes are being perceived by the people in
     the field, and changes, recommended changes and improvements that we can
     make to the process.
         We are also going to try to leverage off of this a group of
     individuals, the senior individuals in this change coalition, who may be
     able to provide some management oversight and some management control
     for the process implementation that we go through.
         We also recognize there is industry and other external
     stakeholders, such as the states, that we are going to have to
     communicate with and bring on board.  And we are developing some -- or
     we are working with some opportunities, I guess, where we can take some
     action in that area.
         One of the transition task force members will be -- that
     will be his primary job, to look after the change management and the
     communication issues that we have.  Another member of the task force
     will look at the training issues that we have and develop training for
     the staff.  And we will probably end up developing some joint training
     with industry to get the plants up to speed on the performance
     indicators, how they are going to report them, and how we are going to
     -- what we are going to do in that area.
         Now, the schedule.  There's what we see as our major
     milestones as we go along.  Now, throughout that time period we expect
     to be communicating and keeping the ACRS informed, and we will try to
     schedule some opportunities that we can do that.  We look at having a
     Commission paper sometime in the March timeframe and receiving final
     Commission approval on the recommendations, not on the whole package. 
     We are not going to have all the details filled in.
         We plan to start a pilot in June.  Now, to support the
     pilot, we are going to have to have a good draft of the inspection
     procedures and the program documents that will support those inspection
     procedures.  And we are also going to have to have provided some
     training to those selected pilot plants and the NRC staff that is going
     to be involved in that, that includes the senior resident inspectors,
     the branch chiefs, and probably some DRS personnel, and others in
     various areas.
         And we are also looking at, in October, providing a workshop
     that would be a joint industry-NRC workshop, public workshop, to provide
     training for everybody involved.
         Does this sort of answer some of your question?
         DR. BARTON:  That's part of it, Alan, but I was under the
     impression that you owed the Commission some more supplemental paper,
     some more documents by next month sometime.
         MR. MADISON:  The March paper, I think, promised -- the
     March paper is -- the promised March paper has to deal with completing
     some of the items, I think, and the Office of Enforcement is to come up
     with a recommendation, a final recommendation of where they went ahead
     with the enforcement policy.  I think we are going to come up with a
     better draft.  We didn't provide them any documentation on what we are
     going to do with the inspection data to relate it to performance
     indicators, and we owe them a document that at least outlines the
     concept that we have and gives some direction of where we are going with
     that.  I am not sure how much detail we owe them on that.
         DR. BARTON:  Well, are we going to get to see any of this
     stuff?  Do you need a letter from us before this goes out for public
     comment or what?  Where are we in your kind of chain here?
         MR. GILLESPIE:  Let we tell you where are in the cycle of
     things.  It is out for public comment.  To maintain this aggressive
     schedule, it went out for public comment before the Commission even got
     it, and the Commission went on faith that the staff had a good document,
     which was -- I appreciate that because it allowed us to move forward.
         We will be going back to the Commission, I believe the
     meeting is tentatively scheduled for March 18th.  About the first week
     in March, we will be going back with a smaller paper -- we are not
     sending 400 pages back up -- that will deal with the exceptions and the
     comments.  And that will include dealing with any comments now we get
     from an ACRS letter that comes to the Commission from this meeting in --
         DR. BARTON:  February.
         MR. GILLESPIE:  February, I forget when the date is.
         DR. BARTON:  Fourth, 5th, something like that?
         MR. GILLESPIE:  Fourth or something.  Yeah.  So the ACRs
     letter will be factored in to this comment period.
         DR. BARTON:  We are going to really -- we are really
     commenting on this policy.
         MR. GILLESPIE:  Yes.
         DR. BARTON:  SECY-99-007, January 8th document.
         MR. GILLESPIE:  Yes.  Yeah.  And now the hole, the real big
     hole that the Commission focused on was this subjective or potentially
     rule-based scale that an inspector could rapidly assess his results
     against and know whether -- I just turn this over to a licensee, or does
     it warrant further pursuit.  That was the significant hole that was
     identified.  We are going to be working on that.
         As a minimum, our best insights on that will have to go in
     the March paper.  As a maximum, maybe we will get it done sooner and get
     it out for comment to everybody, including the ACRS for information and
         DR. SEALE:  When the March paper is available, we will get a
     copy of that?
         MR. GILLESPIE:  Yes, absolutely.
         DR. POWERS:  Mr. Chairman, I think we should devote some
     time to this letter.  I think we have to go beyond just the details and
     the specifics, but point out what we think are the strengths of the
     approach that they have adopted in what I think is an extraordinary
     piece of work.
         DR. BARTON:  Yes.
         DR. POWERS:  And that stands as a template for a lot of
     other activities that the NRC is undertaking.
         DR. MILLER:  However, you are going to -- you recommend we
     comment on the process of getting to this document as well as the
     document itself?
         DR. POWERS:  That's right.  I think that -- I think some of
     the ideas that emerged in this document are strokes of genius.
         DR. SEALE:  They are worthy of emulation.
         DR. POWERS:  And could be used in a lot of other contexts to
     get over some of the rough spots of moving in the direction of more
     risk-informed regulations.
         DR. SEALE:  I have one comment --
         DR. POWERS:  Other than that, we ought we praise the
     document wherever we can.
         MR. GILLESPIE:  Well, we also appreciate, because this sets
     us up for the future.  I promised research, you have always got to give
     them a plug, this is the beginning, and we know there's weaknesses in
     the approach that deserve future consideration.  So any insights as to
     where, first, future consideration might go would also be appreciated. 
     Because once this goes in place, the next step is figure out to make it
         DR. SEALE:  I have one comment on the schedule or the
     outline of future action you made.  I heard about task force members or
     team members.  I heard things about piggy-backing or some term like
     that, some administrative or leadership, or something like that in all
     of this process, and I never did hear any word about the regional
     administrators.  And it seems to me that while this is going to be in
     the pilot stage, only a small part of what is going on in each region,
     that it is very important that the regional administrators be brought up
     to speed on this process very early, because they can either make it or
     break it.
         MR. GILLESPIE:  Yes, absolutely.  And that is -- I hope
     nothing I said led you believe that we weren't going to involve the
     regional administrators.  Yes, and we are wrestling with that.  And the
     agency right now is a quandary, and the quandary is we have taken on, as
     an institution, what the consultants tell us we probably shouldn't have,
     all at once.  We are reorganizing.  We just went through a massive
     number of retirements in the management ranks, including the regions. 
     And at the same time, at least as far as the regions are concerned, we
     are maintaining a current program and we are trying to completely turn
     it upside down.  And we are trying to do it all at once.
         DR. BARTON:  Business as usual.  Right.
         MR. GILLESPIE:  Yeah.  And it has put strains on the system. 
     While we would have liked to have actually had a regional administrator
     volunteer to come in and work with us for six months and run this show,
     I would have liked the relief, it wasn't in the cards, because two of
     the regional administrators are basically new.
         DR. SEALE:  Well, it is not in the nature of those beasts.
         MR. GILLESPIE:  Yeah.  But we are fostering, as part of the
     change coalition, and we are trying to -- we have been brainstorming
     this.  I will throw this out on the table.  This is a concept.  I won't
     even say I have talked it over very much with people, at least we have
     touched it with Sam, and that would be putting a senior level committee
     together which might be deputy regional administrators, regional
     administrators, or projects division directors, who are the guys who
     really have to make this work when we get it out -- one person from each
     region who is kind of the senior member of this concept of a change
     coalition -- and we wouldn't be on it, and, actually, subject ourselves
     about once every two to three weeks to using them like a murder board,
     where we would spend one day briefing them, and a half-a-day letting
     them tear us apart.  And at the Commission table at the end, in March,
     we would ask the chairman of that group to come and sit at the table
     with us.
         Now, that is kind of a conceptual concept that we are
     looking at.  We haven't really run that by the regions yet, and they
     haven't volunteered for it.  But that is kind of the degree of thinking,
     as we are putting the task force together, that we are going to.  So any
     suggestions, any -- that is kind of the gelling, it is as close as we
     have come right now, conceptually, to how we would both get critical,
     very, very critical -- Can this be applied?  Can we do it? -- advice,
     and have them at the table to say, hey, they took my criticisms and they
     told me why not, or they took them into account, here's my concerns that
     still exist.
         DR. SEALE:  I have another question I want to ask, and it
     goes back to -- I don't know, for some reason, I have always had a
     picture of autonomous fiefdoms as you move from one region to the other,
     and I don't think that is an entirely erroneous picture.  In this -- in
     that chart you had up there earlier, you talked about the dividing line
     between regional review and Commission review.  In all of these mergers,
     acquisitions and things like that that have gone on, how many cases do
     we now have where an operating company has plants in more than one
         MR. GILLESPIE:  I couldn't answer.  It is getting worse.  I
     mean with Entergy buying Plymouth, we have got Region IV to Region I,
     which is the maximum extent.  I don't know.  We could look it up.
         DR. SEALE:  Yes.  Well, it is interesting to me because it
     seems to me that that -- and I am not sure how you get it, but it might
     be interesting to try to find out, in the eyes of a victim, whether or
     not they see equal treatment from the regions.
         MR. GILLESPIE:  Yeah, that's an interesting point.  It has
     happened, by the way.
         DR. SEALE:  I am sure it has, but, you know, I think it is
     something that the regions ought to be aware of as a reasonable and
     perfectly correct source of performance data that they have to be
     prepared to live with.  Just a wild suggestion.
         MR. GILLESPIE:  Yes, and actually putting on another hat. 
     Within the same branch, we are putting together a program performance
     group of four or five people.  We used to do this about four or five
     years ago, got strapped for resources, couldn't do it.  And with the
     demise, if you would, of the end of the A&E program, we now have some
     people available to get back into more of a routine -- How is the
     program going?  And that is a good point, because we would be visiting
     some utilities and plants, and I hadn't necessarily, I don't think
     anyone has suggested that we ask that particular question.
         I am going to guess there is a half-a-dozen companies.
         DR. SEALE:  I would think so, yeah.  Yeah.
         MR. GILLESPIE:  That cross some line, some way, shape or
         MR. MADISON:  There is a -- NEI is having a user survey
     which they are going to survey pre-implementation of this process and
     post-implementation, which may address that question.  We are going to
     try to utilize that survey.
         DR. POWERS:  I want to go back to your question about the
     cultural change that you have to make within the agency itself.  You are
     not the first institution to undertake a re-engineering of its
     procedures.  It has gone on long enough that there is a body of
     management science that has been assembled on the difficulties of these
     things.  I, myself, am unfamiliar with that literature, but I wondered
     if it had anything that was of use to you.
         MR. GILLESPIE:  Yes.
         MR. MADISON:  Definitely.
         MR. GILLESPIE:  In fact -- okay, go ahead, Alan.  He is the
     holder of the textbook, he has got the formula.
         MR. MADISON:  Well, I don't know if I have got the formula
     or the answer, but we are trying to get the answer now.  We are
     utilizing the contractor that the NRC does, Nick Mann, as well as his
     subcontractors, to give us some insight into where we ought to go.
         As I said earlier, we have one individual who is going to be
     on the task force, August Specter, who has a background in this area. 
     He is going to be the focus for this so that we have somebody who is
     really concentrating on this full-time.  He and I are going to go to --
     actually, outside to the industry, to Salem, to try to get some input
     and some insights from the changes that they have gone through and the
     processes they have used.
         Yeah, we are going to try to collect some of this body of
     knowledge and use it.
         MR. GILLESPIE:  Well, no, no, no.  Think about this, and it
     is funny, the people from Salem were very, very open with us, and they
     said we have done it twice and we messed it up the first time.  And, so,
     I am not sure that they might not be kind of -- when you are going for
     insights, and they gave us some interesting insights and documentation
     already which has caused us some thinking.
         One was people want to hear from the first-line supervisor. 
     They don't want to hear from outsiders.  Which is interesting, because
     the traditional way NRC does things is we kind of collectively as a team
     would have gone out to the regions and lectured the first-line people
     with the supervisors in probably the same group.
         And you start to develop well, we've got to go talk to the
     branch chiefs and division directors separately first and make sure not
     that they're saying yes, we'll do what you're telling us to do, but yes,
     we understand the underlying principles and can explain it to our staff. 
     So some of the insights of what Alan's been doing are already -- it
     slowed us down a little bit, because we said we need to think about
     this.  If you do it in the wrong order, you could mess it up badly.
         DR. BARTON:  I thought it was a surprise.
         DR. SEALE:  There's a variation on that that the INPO people
     use in their plant assessments now, their visits, namely they use local
     peers --
         DR. BARTON:  Right.
         DR. SEALE:  On their visiting team, and the presentation of
     the results of the visit is made by the local peer.
         DR. BARTON:  Right.
         DR. SEALE:  Which is kind of the same thing.  The guy that's
     telling you the finding knows your plant.  He's not somebody from Mars
     that came in and --
         MR. GILLESPIE:  Yes.  And maybe we should have known this
     right away.  We didn't.  And to me that was, if nothing else, an
     extremely valuable insight that said slow down, understand the steps,
     adjust who you're talking to, you want to talk to different people with
     different levels at different times.
         And then you have to say what are they going to do with the
     information?  And the first-line supervisor, division director, do
     different things.  So we're trying to do it right, and in fact we're
     looking at the literature and trying to use the right experts to give us
     the input.
         DR. POWERS:  I mean, I think that's the answer is simply not
     be arrogant enough to think that you know all the pitfalls to this,
     because I think more than one company has had serious difficulties
     making these kinds of changes.
         MR. GILLESPIE:  Yes.
         DR. POWERS:  The other thing is, of course, to recognize I
     think the personality of your organization and the personality of the
     Salem organization are about as orthogonal as two organizations can be. 
     So I laud you for taking the words from them, but I caution you they may
     not be 100 percent transferable.
         DR. MILLER:  They say they messed it up the first time.  I
     don't think the second time the story's been told totally yet.
         DR. POWERS:  Yes.  I mean, this is --
         DR. MILLER:  We don't know the result in the second time.
         DR. POWERS:  Right.  This is a classic theory x and theory y
     organization.  They're just not going to mesh well.
         MR. MADISON:  But they do have some interesting concepts
     that we want to find out what they -- how they really made or are making
     them work and see if we can apply them to our own case.
         DR. POWERS:  I think you're doing the right thing, going and
     collecting information from wherever you can and then use that part
     that's useful to you and throw away that part that's not.
         MR. MADISON:  Exactly.
         DR. POWERS:  That's all you can do, and pray.
         MR. MADISON:  We may not have the answer yet, but we're --
     this is probably the first time that in my recollection with the NRC
     that we've made this large of an attempt at managing the change rather
     than just making it.
         DR. POWERS:  The other piece of wisdom that I've heard
     frequently expressed in this area is that it takes three years, that if
     you try to bring about a change in the culture and do it in six months,
     and then we'll make another change at the end of that six months, you
     make no change at all.
         MR. MADISON:  Um-hum.
         DR. POWERS:  And it's -- it is so tempting for managers to
     say okay, here's the time line when we will have made this change, and
     lay it down by the calendar rather than the personality of their
     organization, and I think that has been one of the biggest discoveries
     people have made is that --
         DR. MILLER:  Each organization has a different personality. 
     I'll guarantee any university will take a lot more than three years.
         DR. BARTON:  Three to five years --
         DR. MILLER:  Some organizations might take less.
         DR. POWERS:  Well, I think you have certain advantages in
     some organizations would change and certain you don't, and NRC is likely
     to be slow, simply because conservatism has been the watchword of the
     way you do things, not only within this agency but in all of nuclear
     power, the whole thing is think carefully about making a step.  Always
     make sure you've got your foot on firm ground before you take a step
     forward.  I mean, we tell people to do this all the time, and that
     infects the nuclear business, whereas in Silicon Valley they don't --
     not too concerned about whether they're floating on air or not.
         DR. MILLER:  Dana, wouldn't you expect with all the changes
     in the upper management they reflect and so forth, that could be in some
     cases disadvantageous, and in other cases advantageous in this culture?
         DR. POWERS:  I think you're right that whenever you change a
     high-level boss, people anticipate change and are more open to it.  On
     the other hand, they're always suspicious.
         DR. MILLER:  But aren't there like somebody reflected
     there's fiefdoms, I think Bob Seale, at the different regions, which is
     probably true.  Now you've had two regional administrators who are
     probably in control of those fiefdoms depart.  If the fiefdoms are going
     to change, it's a good time to change them, if indeed that's what we're
     going to do.
         DR. BONACA:  I have one more question regarding the report. 
     I realize that I got this report, I've seen it for the first time just a
     few days ago, so -- and the question I have is the report sets
     essentially four objectives for this change in the inspection process. 
     One is improved objectivity, scrutability, risk-informed, and also a
     secondary goal seems to be the reduction of burden.  And clearly reading
     it scrutability is improved, risk-informed is there, okay.
         The only thing that I was asking was, I would have liked to
     see some comparison of the areas of inspection that we used to do before
     that would not be done by the new process and vice versa to convince me
     that in fact we have enhanced objectivity and also that we have reduced
     burden.  Without that it was a little bit difficult to make a judgment. 
     And I wonder if you have done that kind of comparison, just as a
     minimum, you know, large pockets of perspective, what we're not going to
     look at anymore that we used to look at before, and vice versa, what is
     new that we were not looking at before.
         MR. GILLESPIE:  Yes, let me -- we actually kind of -- I wish
     I could give a flat yes or no -- we actually did a check going the other
     way, and the question was would this program as proposed in the areas of
     inspections catch the things that we've seen at plants in the past which
     are not covered by PIs.  And therefore you still see a large -- almost
     the same emphasis as we have today on engineering.  So the engineering
     area didn't change very much.
         For the other areas, we did it, but didn't write it in, and
     there's a chart in there that talks about the risk-informed baseline and
     section, it comes up to like 1850 hours.  We actually have a table
     that's in the inspection manual today that we check each year to see
     that we're kind of in the ballpark on the current core inspection
     procedure, which is kind of the equivalent.  And for a two-unit site,
     that comes out to be about 2200 hours.  The best estimate on if you did
     the sampling process and looked at the things that are in here was about
     1850.  So we are anticipating that it will be as a first estimate about
     20 percent less.
         But we didn't write that down as a formal report.  But it's
     really comparing this table to the table that's in there.  And that was
         DR. BONACA:  The main concern I had was that, you know, I
     see a lot of restructuring.  The question is, are we really changing
     what we're going to look at?  And if we are, what are we going to lose
     in the process to gain in that process?
         MR. GILLESPIE:  Yes.  If there was a major area of loss, and
     this may sound counterintuitive, but I think in the whole set of
     inspectable areas, the major reduction is in operational areas.  I'll
     give you an example relative to maintenance.  The current procedures
     would cause maintenance procedures to be observed on some periodic
     basis, and I think it's once every two months or something now.
         Well, what's the point of that?  What's the safety
     significance of it?  I mean, those kind of questions are asked.  I think
     you'll find in the inspectable areas that the focus is on observing the
     postmaintenance testing to assure that the proper characteristics will
     work, which doesn't mean sitting there eight hours and watching a
     maintenance procedure.  But it does mean watching the test procedure,
     which will be much truncated.
         The PIs tended to be very operational PIs, numbers of
     scrams, and risk insights that Research came up with focuses that --
     John, help me out -- some what percentage of the risk actually looks
     like in real experience came from maintenance, wasn't it?  You used to
     have a pie chart you showed --
         MR. FLACK:  Yes, that was a -- and actually that was the big
     discrepancy.  A lot of the events, the risk-significant events that we
     were observing through the S program was coming from maintenance and not
     so much from operations.
         MR. GILLESPIE:  Operations.
         MR. FLACK:  And I think that's where you're seeing the
     change, the tradeoff now is they're going to be looking more at
     maintenance and less on operations.
         MR. GILLESPIE:  Yes, and the way we look at maintenance in
     this process becomes different, hopefully more intelligent way of
     looking at it rather than just the gross approach of we're going to
     watch this guy do maintenance on a valve from beginning to end.  And you
     know when we watched him do the maintenance we didn't hang around to see
     the postmaintenance test?  Because that wasn't in the procedure.
         DR. POWERS:  You have to advertise this.  I mean, you just
     really have to advertise that you've taken advantage of these insights
     that you've been getting out of your past inspection program.  I mean --
         MR. GILLESPIE:  And some of this is practical insights.
         DR. BONACA:  I'm saying that as you go forward in a
     transition, I think it would be helpful to have somewhere a brief
     discussion of this, because you are leaving behind issues or reviews
     that may be significant, and the question is you have a burden of proof
     that this transition in fact is effective --
         MR. GILLESPIE:  Yes.
         DR. BONACA:  And also you have a burden of proof of closing
     on these objectives that you set in your, you know --
         MR. GILLESPIE:  Right.
         DR. BONACA:  And the other thing is that it would be
     certainly useful to the industry and certainly to me to understand the
     differences there and to really -- it would give me much more comfort
     that in fact the objectivity is truly improved in the process.
         MR. GILLESPIE:  Let me -- I'm only hesitant because this is
     an opportunity lost, you might say.  If this was a research project,
     literally doing a kind of a more academic research project, an
     experiment that doesn't work still is a successful experiment, and it
     tells your hypothesis was an incorrect hypothesis.  And some of us only
     look at -- only if it proves a hypothesis is it successful.
         And when Bruce was putting together the program, he went
     through a process of identifying over 100 inspectable areas which
     basically encompassed the program as his group who were actually
     inspectors, sorry, today.  And so they basically took today's program
     and cut it up into all these pieces, and we ended up with about 40 areas
     when you implied all the insights.  What we lost was they didn't write
     down the results of the brainstorming sessions which necessarily caused
     them to filter those things out.
         And I don't know how recoverable that is, but I don't mind
     going back and asking the question, and going back through some early
     drafts.  Steve Stein was on the group, and I talked to Steve, and I
     said, Steve, did you save the file folders to show the traceability of,
     basically, how do you go from everything we look at now to what you look
     at here, and why?
         DR. BONACA:  As a minimum, if you had written that here, I
     would have been more comfortable.
         MR. GILLESPIE:  Yes, and that was a reaction a number of us
     had when we started reading, and we said, oh, no, we left off the
     negatives.  We left off the reasons why we left things off.
         We will look at the file folder and see if it is
     recoverable.  I don't want to promise it, that it is.
         MR. FLACK:  Just to clarify, they are not really lost.  I
     mean for the baseline inspection, it does a certain level of
     inspections, but, again, you revisit these depending on your performance
     indicators and findings.  So it is not totally lost, it is just that you
     will get into it after something else leads you there.
         MR. GILLESPIE:  Yes, but what is lost is -- What did we
     trade off?  What would we be inspecting today that we are not going to
     inspect a year from now because we have this performance indicator?
         MR. FLACK:  As part of the baseline.
         MR. GILLESPIE:  As part of the baseline.  And that, I am
     afraid, is lost.
         MR. JOHNSON:  But I almost think a better way to do it, or
     at least as good a way to do it, and one that we had anticipated, was
     that we will go look -- we have talked a lot about success criteria, and
     one of the success criteria is do we meet the objectives.  And I think
     it is going to be important that we go back a year from now, and we look
     in terms of what it is we are actually inspecting.  And what did the old
     core provide?  What are we looking at now?  Do we have a sense that we
     are more objective?  Do we have a sense -- I mean have we hit our
     targets in terms of burden reduction?  Have we hit those things that we
     outlined for ourselves?  And we have promised to do that all along, and
     the Commission is holding our feet to the fire on that.
         DR. BONACA:  The other thing is that I would expect some of
     your resident inspectors would want to know why I am not going to
     inspect this anymore, I think is important.
         MR. GILLESPIE:  And that is going to have to be factored
     into the training.  And I will give you one last bit of insight that is
     hidden in this paper, in Pat's write-up some place, and it goes -- of a
     thousand LERs that came in about a year, only 1 percent, or 10, were
     significant enough to be screened at a 10 to the minus 6 or greater risk
     number, which is significant in that --
         DR. POWERS:  It is not hidden, it is highlighted in there.
         MR. GILLESPIE:  Yeah.  I'll tell you the truth, I have
     talked to a lot of people who said they read the paper and they missed
     this sentence, and I remembered it, because it stuck.
         DR. POWERS:  It stands right out.
         MR. GILLESPIE:  If that is in any way reflective of also the
     kind of things that an inspector sees as he is walking around a plant,
     then that is going to be the most significant cultural change that we
     will have, that 99 percent of what we have been observing in the
     facilities is likely not risk significant.
         Now, I don't know that I can take that and directly make
     that conclusion, but, certainly, it says that many of the things that we
     see going on day to day, even though they are viewed as incorrect or not
     right, may not necessarily have a risk significance.  That is the one
     most significant cultural change that is going -- that we are challenged
     with right down to the lower levels, because we have trained ourselves,
     we have trained the industry, that this is a problem, regulatory
     significance, procedural compliance.
         DR. BARTON:  Verbatim compliance with regulations, it is not
     quite that bad, but it is almost there.
         MR. GILLESPIE:  So I don't know the answer, but this is the
     change process.  It is not just changing how you inspect and what you
     inspect, but it is the fundamental understanding of why I am just
     turning this over to a licensee, when only a year ago, this might have
     been a level 4, level 3 violation.
         DR. SEALE:  It is interesting.
         DR. APOSTOLAKIS:  We are not requiring verbatim compliance
         DR. BARTON:  I hope not.
         DR. APOSTOLAKIS:  If you read the nuclear safe culture
     thing, we do.  We do.  We do.
         MR. BARANOWSKY:  They might be NCVs, non-citable violations.
         DR. BARTON:  That is how to handle it.
         MR. GILLESPIE:  Anyway, it is that kind of data that needs
     to be put together in an understandable -- that is going to be a
     challenging training course, if you would, or adjustment to the P-111
     course that we currently have.  To get this kind of information, and the
     context, that also fits with what people feel they are intuitively
     seeing at the facilities is going to be difficult.
         Even Sam Collins gave me an incident, it was a morning
     report that came into the EDO.  A BWR flange had a steam leak on it, it
     was going up in power.  A 10 percent power tech spec says check your
     high pressure injection turbine-driven pump.  They let steam in.  Steam
     flange wasn't tightened.  Steam leak.  They turned off the steam, they
     tightened the flange, it continued up in power.  The system got very
     exercised.  It happened to be the most important thing that happened
     that day in the morning report.
         It was not risk significant.  So everyone got exercised all
     the way up through the EDO, you might say, in a communications sense,
     for something that was not -- the system worked.  The tech spec said
     test, they tested.  Yes, they have to go back and find out why weren't
     the bolts tightened the first time around.  And Sam looked at me when I
     said, this isn't risk significant.  He said I can't believe you, go get
     a risk guy.  So I took it down to the risk guy and I said --
         DR. BARTON:  Well, it is going to take years to change that
         MR. GILLESPIE:  Is this significant?  And the risk guys
     says, you know that is not significant.  I said, yeah, but you have to
     write it down for me.  So he wrote it down, he put his name on the
     bottom.  I went back in to Sam, and said, see, Sam, it is still not risk
     significant.  And he just looked at me and said, we are really going to
     have to change the way we think around here.
         DR. BARTON:  You are darn right you are.  It is also not
     only risk, it is also you are afraid that you are going to be accused of
     not knowing what is going on out there.
         MR. GILLESPIE:  Well, and that is -- yeah, and that is the
     other sense.
         DR. BARTON:  That is their part, whether it is risk or not.
         MR. GILLESPIE:  When you start stepping back, if you do less
     inspection, you have less information.
         DR. BARTON:  Right.
         MR. GILLESPIE:  When you are less intrusive, you have less
     knowledge on a day to day basis.
         DR. BARTON:  Yeah.
         MR. GILLESPIE:  It is a different approach.  And it is going
     to be a challenge to get this into the culture of this organization.
         DR. BARTON:  Oh, yeah.
         DR. SEALE:  Somehow you have to bring into the attributes of
     this system the idea that you are inspecting smarter.
         MR. GILLESPIE:  Yes.
         DR. SEALE:  And maybe that 1 percent example is a good
     lead-in to that line, to that point.
         DR. MILLER:  The same number of hours in inspection on the 1
     percent rather than on the 99, is that?
         DR. SEALE:  Well, --
         MR. GILLESPIE:  That is really the focus of a risk-informed
     inspection program is you are inspecting somewhat less, but you are
     actually inspecting the risk important things more.
         DR. MILLER:  That's the whole objective.
         DR. POWERS:  I think it is important to consider a human
     element of the inspector's life, it is what I call my proof-reading
     theory of facility inspection.  I sometimes get to inspect facilities. 
     If I don't -- if I have people working for me developing a facility and
     they don't make a certain number of errors, my inspection becomes very
     poor.  As soon as they cross a threshold of a certain number of errors,
     then I get them all.  And so it seems to me that I would tend to portray
     this as -- it is true, 99 percent of these things are not risk
     significant, but they may well portend a degradation, and you need to
     bring them still to the licensee's attention, it is just that we don't
     do anything about them.
         MR. GILLESPIE:  That was our earlier discussion.
         DR. POWERS:  Yeah, I think we had this earlier discussion.
         MR. GILLESPIE:  Exactly.
         DR. POWERS:  That what you don't want is inspectors being
         MR. GILLESPIE:  Yeah, you don't want the inspector stepping
     aside, because what you are going to have -- you are going to miss
     something you should have picked up.  And that is actually the challenge
     to the screening process, which is the big IOU to the Commission of
     inspection results.  Having some false positives when you screen is
     probably okay.  In other words, some things that you do a little more
     work on, you might not have had to do a little more work on, because I
     feel comfortable we are talking about fairly small numbers of things. 
     But having a false negative is totally unacceptable.
         So the challenge in setting up -- it has got to an
     asymmetrical uncertainty in the screening process, and Alan has taken
     this challenge on, so now, officially, he is hooked.  So -- and that is
     why it might end up being kind of a multi-stepped thing, a rule-based
     thing that is looking at certain questions.  Is it a train, is it a
     system?  If it is a program failure, it is kind of like asking a
     question -- Are there enough program failures that the coincident effect
     at any given point in time is risk significant?  That is a real tough
     question to answer, and that starts getting at the sense of regulatory
     significance or not, because it is not -- Was it wrong and fixed, wrong
     and fixed, wrong and fixed?  If it is wrong and fixed, and things don't
     happen coincident to degrade several systems or several trains, then it
     won't hit a safety threshold, yet, you have to give it to the licensee
     because you don't want all those things happening.
         So that, to me, is the one major, major, major intellectual
     challenge in this whole system that we have ahead, is doing that for the
     inspection findings.  Because what I said doesn't depend on numbers of
     findings, it depends on the severity of the finding.  So, in inspection
     space, we may be dealing with the safety significance of the finding,
     not the number of findings.  And, again, that is a different approach. 
     If you have a lot, it has to be safety significant, right, that is the
     intuitive feel.  It may not be.  And it is going to be very difficult,
     because now we are counter-intuitive with our inspection force, and that
     is going to be -- that is going to be real hard.
         DR. MILLER:  So an inspector who found one very risk
     significant situation would get more credit, so to speak, that somebody
     who found a hundred insignificant ones, is that the thing you are
     alluding to here?
         MR. GILLESPIE:  No, it is a matter of -- the general sense
     is, gee, I found eight things wrong in procedural compliance, and the
     typical enforcement letter that would go out, signed by a division
     director, would declare it a programmatic failure, and it would be a big
     deal in today's environment.  I don't know what it would be in
     tomorrow's environment, because if those eight procedural items happen
     disconnected in time and never actually affected a piece of equipment,
     and we are trying to develop a safety outcome score, or goal, or
     judgment, it puts it in a different perspective.  And it is not the way
     we have been trained.
         DR. POWERS:  I have a question about public perception.  If
     I am a member of the public sitting next to this -- living next to a
     plant, and I see less stringency on the part of the NRC, the number of
     level 4 violations has gone to zero, lots of things that on the face of
     it would have been violations today are not tomorrow, and I, as a
     reasonable person, come to you and I say, how come?  And you say, oh,
     they weren't risk significant.
         And I respond, gee, how would I know that?  And you say,
     well, it is real simple, you run the sapphire code with the right kinds
     of inputs and outputs, and you take the things and you do a risk
     assessment worth on this, and you look at your 95 percent confidence
     level on a T-scale with 5 degrees of freedom and utility comes this
         Do you think you have really persuaded me that you are not
         DR. MILLER:  Have you persuaded anybody on this Committee
         DR. POWERS:  You're not relieving a burden upon your
     licensee at the expense of increased risk on my part?  And by risk in
     that context, I mean my qualitative perception of the risk.
         MR. GILLESPIE:  Actually, we are working very closely with
     public affairs on trying to answer that question.  But there's some
     flaws in your basic assumption, they are different than our basic
     assumptions.  One, we expect the inspector will write down when he sees
     something wrong.  Now, what we have to talk about is what do we call it. 
     The current enforcement policy change, as yet unapproved, but I
     understand the vote sheets are in, would take all these level 4s and
     call them non-cited violations, would not have us follow up on every
     single one of them, but would have us audit, on an audit basis, to
     follow up on them.  We would expect --
         DR. POWERS:  If I am a member of the public who knows a
     little bit about your processes, what I see is violations used to be
     cited, now they are not.  Violations used to be followed up, now they
     are not.  And when I -- as a reasonable question, I mean I don't think
     it is an outrageous question, before I make any accusation, is I ask you
     why, and you give me this claptrap about risk analyses and things like
         MR. GILLESPIE:  Like I said --
         DR. POWERS:  What other conclusion can I arrive at?
         MR. GILLESPIE:  Well, I think, first, the intent is that we
     are not going to ignore the items.  Now, what we are doing is calling
     them something different, and that gives us, you might say, a public
     perception challenge to deal with, because we are calling them something
     different, and we will not be following up on every single one we used
     to follow up on.  So we are going to have to very articulate in saying
     why.  We haven't been there yet.
         In fact, one of the criticisms of this whole report, that
     they opened up the meeting with, was a member of the public would have a
     very difficult time reading this report.  It is not user-friendly.
         But let me go back and say why this is better than what we
     have, and that is kind of where we are at right now.  What we have right
     now in judging overall safety of the facility is the SALP process.  The
     SALP process is actually very, very, very subjective, there is no doubt
     about it.  In fact, the insights we would have from this program that we
     have kind of run so far would be that a lot of facilities, or a large
     group of facilities, are run so safely that it is almost impossible to
     discern between them, yet SALP was trying to force itself to discern
     between several very well run facilities because we were obligated to
     write a report.
         This will allow us now to have more information out to the
     public directly related to their operations and more frequently, and
     that is the plus.  The minus from the system is we are not going to
     follow up on every one of the items.  The plus to the system is, by
     publishing the performance indicators, and an interpretation of what
     they mean on a quarterly basis, and something bigger on an annual basis,
     we will be displaying to the public more and current information on the
     safety of the facility.  So we are substituting one process for another
         How we put that together is going to be a real challenge,
     and how we communicate that to the public is going to be another real
     challenge.  In fact, public affairs offered that maybe we could go talk
     to the League of Women Voters and a whole bunch of other people to try
     to deal with the concerns.
         DR. POWERS:  I think the important thing, I mean I hate to
     be so complimentary, and I will try to avoid that --
         MR. GILLESPIE:  Oh, no, but this is a hard one.  You know,
     it is easy to deal with the darn technical issues.  It is very difficult
     to deal with the translation of what you think is a good judgment into
     you are backing off, you are not doing as much.
         DR. POWERS:  One thing I liked about the draft response you
     articulated here, you went for about two minutes without ever once
     bringing up the word "risk" in that, and I think that is the start of
     formulating a good response, is bring up the risk after you have said
     all those things you did, more information, better information, better
     interpretations and things like that.
         Get to the results of going in this direction and then if
     someone wants to ask you why do you have all this good insight, then you
     can bring up risk.  Rather than saying, oh, well, we have used these
     analyses that it is impossible for me to reproduce, or even understand,
     to focus my efforts so that I get the most bang for the buck.  You know,
     that just leaves me cold as a response, whereas, when you tell me -- ah,
     you are getting much more, you are really getting much more out of this,
     and much -- using much the words you did and don't bring up this arcane
     technology that requires a degree in statistics to understand.
         DR. SEALE:  A little confession may be good for the soul
     here, too.  I have heard that it is, at least in some circles, although
     it is not too popular these days.
         DR. POWERS:  But in both the states that you and I live in
         DR. SEALE:  Very definitely.  And the confession may be, and
     it may take a little bit of homework to verify it, but the confession
     may be that, of all those violations that you had previously cited, the
     ones that you are no longer publicizing -- or going into the record
     with, and so on, were ones that had no impact on -- when you finally get
     around to the point of public risk.
         DR. POWERS:  But I don't think I would try to bring that
     story to the public --
         DR. SEALE:  Not up front.
         DR. POWERS:  -- because he didn't think -- he didn't know
     whether they did or not.  All he knew was that they were being stringent
     and now they are not.  I think -- I really like this idea of no, no, no,
     it is not less, you are getting way more.  I really like that approach.
         DR. SEALE:  Well, we are looking at the things that count.
         DR. POWERS:  I wouldn't even go that far.  I would say we
     are looking --
         DR. MILLER:  You are getting more.
         DR. POWERS:  We have always looked at the things that count,
     and we are continuing to look at the things that count, and we are
     giving you far, far more than you ever got before, because you used to
     get these funny looking SALP scores that meant absolutely nothing to
     anyone on the face of the planet, not even those in the NRC.  Now, you
     are getting all this other wonderful stuff.  I think that is the germ of
     a public -- of risk communication that may actually work.
         MR. GILLESPIE:  It's funny, that's where Bill Beecher came,
     in a lengthy meeting we had with him yesterday, trying to brainstorm. 
     It was tell them what you are giving them, don't --
         DR. POWERS:  That's exactly right.
         DR. MILLER:  Tell them the glass water is getting fuller not
         MR. GILLESPIE:  It is getting fuller, not emptier, yeah.
         DR. BARTON:  I think we need to wrap this up, and figuring
     out what do we want to talk about at the full Committee meeting, I guess
     next week.
         DR. APOSTOLAKIS:  We are all here.
         DR. BARTON:  Pardon?
         DR. APOSTOLAKIS:  We are all here.
         DR. BARTON:  Yes, we are all here.  What else?  What --
         DR. APOSTOLAKIS:  Let's take the time to write a letter. 
     Well, I don't think we need a presentation, we are all here.
         DR. SEALE:  Well, the only thing I can think of is if we ask
     them if they could, at that point, tell us what additional information
     you are liable to be able to put in the Commission letter.
         DR. BARTON:  The supplemental information.
         MR. MADISON:  I think at this time it is premature.  We have
     got a draft concept that we are refining and I really want to put it
     through a lot more exercises before we make it -- bring it here.
         DR. SEALE:  Okay.
         DR. BARTON:  What about the policy issues that remain that
     you mention in this document, evaluating interface of Part 50,
     revisiting event response evaluation, revisiting N-plus-one,
     organizational impact.  Is that something that you are prepared, that we
     could talk about, that we could address in our letter?  Since the
     Commission has to make decisions on those policy issues?
         MR. MADISON:  No, the Commission -- those four policy issues
     were a heads-up to the Commission.  And we actually had a fair amount of
     discussion on.  These are not today's decisions.
         DR. BARTON:  Okay.
         MR. MADISON:  They are next year's decisions.  In some
     cases, they are six months away.  But the basic question was, if you do
     go down this path, and there is less inspection, but you keep
     N-plus-one, residents tend to be generalists, not specialists.
         DR. BARTON:  Right.
         MR. MADISON:  If you need engineering specialists and what
     you have got is generalists living at the plants, that gives you an
     organizational problem and causes you to need, if you go down this path
     very far, to step back and say, I have got an institutional problem I
     have to address, not a technical problem.
         The reconciliation or harmonization with Part 50, that was
     our number one issue on the list.
         DR. BARTON:  Right.  Right.
         MR. MADISON:  It is just highlighting to them, we are
     getting out in front and if Part 50 changes their scale, then we are
     going to have to have feedback from that, and they are going to have to
     be cognizant of Commission decisions made.  If the Commission approves
     our basic scale of where our bands are, then it is going to be
     hard-pressed for the Commission to necessarily change their mind without
     some justification later. So approving what we are doing actually has
     future implications that says hey, the bands are set.  Here's the
     licensee response zone for whatever it is.
         So, no, those four issues were a heads-up.  I think they
     need to be dealt with in the course of probably the next 12 months. 
     They can't be ignored, particularly if we are successful --
         DR. BARTON:  That's true.  Right.
         MR. MADISON:  -- with the pilot program, but not in the next
     three months.
         DR. BARTON:  Okay.  What else do we want to hear?  George, I
     think the problem we've got is it's already in the Federal Register,
         DR. APOSTOLAKIS:  You can take that time to write a letter.
         DR. POWERS:  Or at least a summary from the Commission
     Chairman in order to -- or from the Subcommittee Chairman at the very
     least.  I don't think it would be a bad idea to bring up these policy
     issues even though they are six months down the line, because that is
     about our time schedule for thinking about the subject.
         DR. BARTON:  Okay.
         DR. POWERS:  And it would be opportune for putting it on our
     agenda so that when you come in here, we are not having to spin up from
     zero -- that and a summary from the Subcommittee Chairman would fulfill
     our obligations.
         DR. BARTON:  At the full Committee meeting.
         DR. POWERS:  At the full Committee meeting.
         DR. BARTON:  Okay.
         DR. APOSTOLAKIS:  How much time do we have?
         DR. BARTON:  I don't know.
         MR. MARKLEY:  10:30 on Wednesday.
         DR. MILLER:  One of the policy issues -- it may not be a
     policy issue -- we raised a concern about the fact that the inspection
     assessment process is leading the regulatory situation.  Is that an
     issue you want to raise?
         DR. POWERS:  Yes.  I think that's a --
         DR. MILLER:  I am beginning to think that maybe one could
     look at the advantages of this leading the regulatory process -- if we
     want to look at the half a glass of water fuller.
         DR. POWERS:  And I will remind you --
         DR. MILLER:  It might lead them in the right way --
         DR. POWERS:  I would still like to know about your
     validation of your tools for your Appendix H and Appendix I analyses.
         MR. GILLESPIE:  That's an IOU we'll take away and get back
     on Sapphire Code and what happened to that.
         DR. WALLIS:  Well, I took a vow of silence when I came in
     this morning.
         DR. POWERS:  Why?
         DR. MILLER:  It's about those hundred Senators being quiet.
         DR. WALLIS:  But Dana woke me up when he said how would the
     public perceive this, and I think it is not just a question of public
     perception but too many of these changes, which I think you have done
     very well, appear to be motivated by what is good for the utility or
     what is good for the administrators in the NRC, and it would be good if
     somehow it could be put in perspective to give a really fair measure of
     what has the effect of all this been on nuclear safety.
         I don't know how you would do that but really that is a
     measurement I would like to see or get some better feel for.  I like
     what you are doing, but how do I tell if it's been good for nuclear
     safety or are we backing off or what?
         Maybe we back off but it's still better for nuclear safety,
     but how do I know?
         MR. GILLESPIE:  Yes, that's -- let me explore that a little,
     because this is kind of a key point underlying the paper.
         In the paper it talks about this is a process to ensure that
     safety is maintained so we are clearly not putting something in place,
     and from the way Pat described his statistical and the way the first
     threshold was set, we are clearly trying to maintain what we believe is
     an acceptable level of safety.  We are not forcing it to be better.
         Quite honestly, I don't think we could force less than the
     .8 scrams on the average per plant that exists today, so how is this
     good not to do that?  That is a public perception problem.  You mean the
     NRC is not forcing it to be better and better and better and better?
         I don't have a good answer for that because we haven't
     digested it.  We understand it but that is kind of one of those things
     that is hidden in the paper.
         DR. KRESS:  Well, let me take a stab at that at the risk of
     breaking my vow of silence.
         DR. POWERS:  What is this vow of silence?
         DR. KRESS:  This process we are going through could lead up
     to this bit of a problem that George pointed out that you will find some
     plants for example that may get outside your inspection lines but then
     you go in and look at it more closely and find that they still comply
     with all the regulations, and what you will have done there is perhaps
     this particular plant while it still complies with the regulations as a
     specific plant has -- let me put it this way -- not quite as good a
     desired risk status.
         MR. GILLESPIE:  Yes, that's --
         DR. KRESS:  And it is because your regulations have allowed
     that.  Now this allows you to focus on the problem of risk, which is
     really your regulatory mission, and you can say, well, have our rules
     done the job or not ,and which part of the rules are keeping us from
     doing our job and it will allow you to go back and say now do we need to
     change the rules some way and in what manner do we need to change them
     in order that we can get plants within a risk status that we would
     better desire.
         Now of course that raises questions of backfits or
     substantial improvements in regulatory analysis, but I mean you're stuck
     with that, but it will allow you to focus your attention on the rules
     and whether they are doing the job correctly or not, and I think it is
         I mean that is a public benefit that is going to come from
     it in my mind.
         MR. GILLESPIE:  I'm going to say it kind of harshly.  I
     don't mean it -- I don't know how else to say it, but if the industry
     disagrees with the basic premise that we have tried to pick thresholds
     that in fact they could fairly operate within and in normal
     circumstances at a reasonably well-run plant, not a perfect plant, it
     can operate within those thresholds.  If that basic premise is
     incorrect, the industry has to stand up and be counted and say that.
         DR. KRESS:  That may be the way -- your hands may be tied --
         MR. GILLESPIE:  I can only take on so much burden and this
     team in developing an approach that has to be somewhat generic, and it
     has to give the public the ability quite honestly to compare one plant
     to another, but in comparing one plant to another when you get
     structured like this, it's there and, yes, one plant could have, if it
     continually violated a threshold but was in compliance with all the
     rules, it could have a less desirable public image, and that is kind
     of -- you know, the comment on SALP today -- a similar comment.
         I hope it will be less of a problem than the system today
     gives us in that same arena but we will likely have not fixed it
     perfectly, so the industry now has to speak up and that is why this
     thing is out for comment.
         DR. BONACA:  Your point is a concern I had before when I
     talked about the fact that most units meet those kinds of quantitative
     criteria that we have here.  From past experience I can remember exact
     cases where units met those criteria that were provided by INPO and so
     on and so forth entirely, and yet their performance was very negative
     because of other issues which are tied to inspection.
         Now to the degree to which you have a much more scrutable
     process here, it's going to be very hard to deal with that kind of
         I am trying to understand, you know, because I do believe
     that the inspections bring about a lot of insights that tell you whether
     or not a plant is doing well.
         What I am trying to say is that the numerics may force you
     to disregard a lot of these other insights from inspections if
     consistently they give you all greens.
         See what my concern is now?
         MR. GILLESPIE:  Yes, and this is the crux of a past
     criticism we have gotten from GAO in past years.  Let me take what you
     have just said, if I could, to the extreme, and that is when we go in
     and shut a plant down because all of a sudden -- Cook.
         We did an A&E inspection, raised questions about
     engineering, went in and looked at engineering even more.  They went in
     and looked at their engineering even more.  The ice columns became -- it
     just -- it kind of, you might say, expanded to the point where the
     facility has been shut down for an extended period now to get the design
     process straightened out.
         One of the things we were being pressed several years ago
     was to always identify a problem to allow a licensee to correct it
     before it got to the point where you had to shut down to correct it.  I
     believe -- and I am not going to commit to the team, unless they want to
     chime in, that these indicators plus the inspection -- and I can't take
     my eye that there's still substantial inspection being done in very
     focused areas, particularly engineering, will still give us an early
     enough indication that I think we can beat that goal.
         Now we looked back retrospectively and Cook was one of the
     ones that Bruce Mallett looked back on and said there is no indicator
     for this problem they had in engineering -- are we still looking at
     engineering?  And Bruce said yes, we are still looking at engineering,
     and we would have had an opportunity to pick that up.
         Cook still would have been shut down, because you would have
     found a problem in engineering, they would have pursued it more, the
     problem would have gotten bigger, and they'd be shut down.  The Cook
     shutdown by this new process likely would not have been avoided.
         DR. MILLER:  But it might have been shorter.
         MR. GILLESPIE:  It might have been shortened because we
     would have a different perspective on it possibly but I don't think the
     shutdown itself would have been avoided, so for Cook we wouldn't have
     made this goal that was set up for us.
         Inspection insights are still going to be there.  I hope
     they are the insights in the right area.  We have tried to focus them
     that way.
         I would like to bring up one other thing that is in the
     paper and no one here touched upon.  It's something that Mike is going
     to grimace when I say it, and it's the -- what do we call it? -- the
     executive exemption?  No --
         MR. JOHNSON:  Executive override.
         MR. GILLESPIE:  Executive override.  We are allowing a five
     percent executive override and I regret the terminology but I am pleased
     with the concept.
         In fact, if there are significant insights that cause us in
     a very limited number of cases to question the performance indicators,
     we do need to go in and examine those cases and if we find the
     indicators are correct, fine.  That reinforces where we're at.  If we
     find they were wrong, the executive override concept has great value in
     program evaluation space relevant to being kind of like a proof test.
         One of the things that we are going to be kind of committed
     to is kind of looking at the terminology and the basis we had in there
     for that.
         David Lochbaum, UCS, says I don't like these executive
     overrides at the Commission meeting -- there should be zero executive
     overrides -- because it had a negative connotation, but actually having
     an occasional question to test the system is probably a positive move,
     so we are going to need to go back and re-look at how we have termed
     that in there to take care of that concern, not everyplace but I think
     as a programmatic group in Headquarters we have to be conscious that we
     need to test our assumptions somehow, and the only way to do that is
     from the outside looking in.  You can't necessarily do it always from
     the inside looking out.
         DR. SEALE:  Are you familiar with the term "the halo
     effect" --
         MR. GILLESPIE:  Yes.
         DR. SEALE:  The halo effect was on the basis of
     personalities.  The executive override that you are talking about is on
     the basis of specific physical observables.  I think that is a
     reasonable trade, and not have the personalities which have been a
     problem in the past.
         MR. GILLESPIE:  So that is one aspect in there.  There was
     some discussion on that.  You know, if that is a piece -- you know --
     those are the kinds of pieces and kinds of comments that we're searching
     for that will help us hone the system down or in, get it in tune, and
     move forward with it.
         I think the concept was right, the move was right, it's the
     right thing to do, but we worded it wrong.
         DR. SEALE:  Yes.
         MR. GILLESPIE:  And that just comes as time evolves and you
     say, well, why did we say it that way?  Why do we really think this is
     the right thing to do?
         That is kind of an evolving nature, which is why we are not
     committed to rewriting this entire 400 pages, but now taking the
     comments, adding them to these 400 pages, and then moving forward and
     writing the actual program documentation.
         With that, I am worn out.
         DR. BARTON:  Well, one thing.  I know you said you didn't
     want to rewrite the document but there's areas -- the document could
     probably be streamlined and a lot thinner because there's a lot of
         Also, I don't remember whether there is a master table of
     acronyms in there, but you use ERO in two different sections, Emergency
     Response and then Effluent something and Radiological Emergency.  You
     may want to either put a master acronym thing in here or else change one
     of the two because you use it in two separate --
         MR. GILLESPIE:  You can tell we had multiple authors putting
     this --
         DR. BARTON:  Yes, and that is probably the result and why it
     got that way, but you want to look at that so you don't end up with
     confusion down the road or at least put a master --
         MR. GILLESPIE:  Pat's comment was more than multiple.  In
     fact, there was about 24 authors who wrote different sections.
         DR. MILLER:  There is strength in the weakness.
         DR. BARTON:  Anything else we want to talk about next week
     or here that we haven't heard before?  If not, then --
         DR. SEALE:  I have an entirely different matter to ask the
         DR. BARTON:  While these guys are here?
         DR. SEALE:  No, no.  Just don't you guys leave, that's all I
     am saying, the members.
         DR. BARTON:  And we are done and we will break for lunch.
         Thank you for all the hard work and coming in here and
     putting it up again.  I guess we are adjourned.
         [Whereupon, at 12:31 p.m., the meeting was concluded.]

Page Last Reviewed/Updated Tuesday, July 12, 2016