Advisory Committee on Nuclear Waste 133rd Meeting, March 19, 2002
Official Transcript of Proceedings
NUCLEAR REGULATORY COMMISSION
Title: Advisory Committee on Nuclear Waste
133rd Meeting
Docket Number: (not applicable)
Location: Rockville, Maryland
Date: Tuesday, March 19, 2002
Work Order No.: NRC-283 Pages 1-116
NEAL R. GROSS AND CO., INC.
Court Reporters and Transcribers
1323 Rhode Island Avenue, N.W.
Washington, D.C. 20005
(202) 234-4433. UNITED STATES OF AMERICA
NUCLEAR REGULATORY COMMISSION
+ + + + +
ADVISORY COMMITTEE ON NUCLEAR WASTE (ACNW)
133RD MEETING
+ + + + +
TUESDAY,
MARCH 19, 2002
+ + + + +
ROCKVILLE, MARYLAND
+ + + + +
The meeting commenced at 10:00 a.m. in
Room T2B3, Two White Flint North, Rockville,
Maryland, George M. Hornberger, Chairman, presiding.
COMMITTEE MEMBERS PRESENT:
GEORGE M. HORNBERGER, Chairman
RAYMOND G. WYMER, Vice Chair
B. JOHN GARRICK, Member
MILTON N. LEVENSON, Member
STAFF PRESENT:
JOHN T. LARKINS, Executive Director, ACRS-ACNW
SHER BADAHUR, Association Director, ACRS-ACNW
HOWARD J. LARSON, Special Assistant, ACRS-ACNW
LYNN DEERING, ACNW Staff
LATIF HAMDAN, ACNW Staff
MICHAEL LEE, ACNW Staff
RICHARD K. MAJOR, ACNW Staff
WILLIAM HINZE, ACNW Staff
CAROL A. HARRIS, ACRW/ACNW Staff
RICHARD P. SANVIO, ACRS/ACNW Staff
Also Present:
CAROL HANLON, DOE
PETER SWIFT, Bechtel SAIC
WILLIAM BOYLE, DOE
. I-N-D-E-X
Opening Statement. . . . . . . . . . . . . . . . . 3
ACNW Planning and Procedures
Update on DOE Performance Assessment
Program (BJG/MPL). . . . . . . . . . . . . . 6
Carol Hanlon . . . . . . . . . . . . . . . . 6
Peter Swift. . . . . . . . . . . . . . . . .14
Bill Boyle . . . . . . . . . . . . . . . . .46
Adjourn. . . . . . . . . . . . . . . . . . . . . 116
. P-R-O-C-E-E-D-I-N-G-S
(10:05 a.m.)
CHAIRMAN HORNBERGER: The meeting will
come to order. This is the first day of the 133rd
meeting of the Advisory Committee on Nuclear Waste.
My name is George Hornberger, Chairman of the ACNW.
Other Members of the Committee present are Raymond
Wyner, Vice Chairman, John Garrick and Milton
Levenson. And also present we have a consultant with
us today, Bill Hinze.
During today's meeting, following the
planning and procedures session the Committee will (1)
hear an update from DOE on its performance assessment
program; (2) finalize the annual research report to
the Commission, and (3) discuss preparations for
tomorrow's meeting with the Commissioner.
John Larkins is the designated federal
official for today's initial session.
This meeting is being conducted in
accordance with the provisions of the Federal Advisory
Committee Act. We received no requests for time to
make oral statements from members of the public
regarding today's session. Should anyone wish to
address the Committee, please make your wishes known
to one of the Committee staff. We have received one
written comment from Mr. Mel Silberberg, on the
research program. His letter will be inserted into
the record at this meeting.
It is requested that speakers use one of
the microphones, identify themselves and speak with
sufficient clarity and volume so that they can be
readily heard.
Before proceeding, I would like to cover
some brief items of current interest. Items of
interest, (1) Dr. Victor Ransom has been appointed as
the eleventh Member of the ACRS. He is a Professor
Emeritus of Nuclear Engineering, Purdue University.
Prior to this, he was a Scientific and Engineering
Fellow at the Idaho National Engineering and
Environmental Laboratory. Mr. Timothy Cobetz and Mr.
Robert Elliott have been selected the ACRS/ACNW
Technical Staffs. Rob, who returns to the ACRS staff
having previously served on a rotational assignment
comes from NRR and will replace Noel Dudley on the
ACRS staff. Tim, who joins the Staff from the Spent
Fuel Project Office, will assist both Committees as
the work load dictates.
Dr. Margaret Chu has been approved by the
Senate as Director, Office of Civilian Radioactive
Waste Management. She comes to DOE from Sandia
National Laboratories where she has been in charge of
the Nuclear Waste Management Program. Prior to that
she was Deputy Manager for WIP.
The attached, at least attached in our
book here, February 13, 2002, a paper by Commissioner
Dicus, "The Future of Environmental Protection, a U.S.
Regulator's Perspective" provides a most interesting
perspective on this topic and I'm sure that anyone who
wants it can get a copy of this document.
Any other items? Okay, good. We are
going to move to our first topic which is an update on
DOE performance assessment and John Garrick will chair
this section of the meeting.
MEMBER GARRICK: I'm going to waive any
opening remarks for the benefit of having the time to
ask questions and what have you and I think we have
three people that we're going to hear from: Carol
Hanlon, Peter Swift and Bill Boyle. And I would ask
each of them to give us a quick statement of their
assignment or their role for the benefit of the record
and the Committee and those in attendance.
So Carol?
MS. HANLON: Thanks, Dr. Garrick. Is this
on? Can you hear me? Good morning. I am Carol
Hanlon with the Department of Energy. I'd like to
introduce to you my colleagues, Dr. William Boyle and
Dr. Peter Swift and ask them to join us up here. They
will be giving the main presentations.
Peter is with the Sandia National Labs,
Performance Assessment, and he has had a very main
role in helping us with our performance assessment
activities as well as the prioritization effort going
forward.
Dr. Boyle, as you know, is a Technical
Advisor, with Yucca Mountain and has strong
underground geotechnical expertise.
So the gentlemen will be making the
presentations.
You know that the Committee has been
carefully following our process and are particularly
concerned both with the technical aspects as well as
the performance assessment. We've briefed you many
times and especially last year on several of these
topics, including the Supplemental Science and
Performance Analysis Document, the Preliminary Site
Suitability Sites and Engineering Report and I know
you've been at many of the key technical issue
technical exchanges. So you're very familiar with
these issues.
We're also familiar with and we have
carefully considered the letters that you provided,
especially the letter on performance assessment and
we're hoping that you will see some of your
recommendations included in our path forward.
So I'm pleased to be able to speak with
you today and give you an update on some of the
information that has come out, some of the reports
that have come out since last summer.
I've introduced Dr. Boyle and Dr. Swift
and if I may just briefly cover some of the
information as an introduction.
This is our snapshot on our home page
which is available at www.yuccamountain --
www.ymp.com and it pretty nicely captures the major
efforts, the major accomplishments we have had during
the last year or so, the release of the Yucca Mountain
Site Suitability Evaluation, Rev. 1 of the Science and
Engineering Report, the SR Comment Summary Document,
Supplemental Comment Summary Document, those
responding to and summarizing comments that we
received during our comments period; the final
environmental impact statement and some other
information as well as the state and county impact
reports.
CHAIRMAN HORNBERGER: In the spirit of
engelbrecht, you did say ynp.com and I didn't know
that DOE had become a dot com.
(Laughter.)
MS. HANLON: Thank you very much for
helping me. Did I say dot com? Thank you.
Everyone will correct me. YMP.gov. And
I will never use an acronym again.
(Laughter.)
So the presentations that follow address
these technical updates and comments on preliminary
site suitability evaluation. There are two types of
them. One that evaluates the evaluation, the impacts
of the final regulatory standards including the
Environmental Protection Agency Standard 40 CFR Part
197 as well as Nuclear Regulatory Commission's 10 CFR
Part 9, excuse me, 63.
In addition, the technical updates
consider the evaluations of additional information
which was available since release of the supplemental
science report and analysis, the science and
engineering report and the preliminary site
suitability evaluation report, that information that
was continuing to be collected and analyzed over the
summer.
Another topic that we will discuss is the
treatment of uncertainty in the total system
performance assessment for the license application,
both the uncertainty analysis and strategy and
discussion of treatment forward of uncertainty and
finally, the path forward for the Yucca Mountain
performance assessment focusing on uncertainty that
matters and risk-informed prioritization for
performance assessment.
And you have in your book and in the
presentation again these major developments, on-going
technical exchanges with the staff during the year and
we had another technical exchange last week in San
Antonio; the release in May of the Science and
Engineering Report which was based on the total system
performance assessment in July; in August, releasing
supplemental science and performance analyses as well
as a preliminary site suitability evaluation; and
including the updates later to total system
performance assessment, staff recommendation and the
technical basis which Peter will say something about.
CHAIRMAN HORNBERGER: Carol, what was the
technical exchange last week?
MS. HANLON: It was on -- what was the
title again?
DR. BOYLE: Laboratory design.
MS. HANLON: And the final regulatory
standards not in July, but in June, 40 CFR 197 which
was finalized in November, 63 -- 10 CFR Part 63 was
finalized and also in November, the Department's 10
CFR 963 was released.
In December 2002, we had -- I think that's
an error -- 2001, additional information documented
was presented in four Letter of Reports which we'll
discuss with you today and in February, the site
recommendation went forward from the President. So
we're in the process of realigning our science and
performance assessment activities within BSC and
moving forward with a consistent direction on
treatment of uncertainty as well as focusing on the
risk-informed performance-based approach.
So with, unless you have any questions on
that brief introduction, I'd like to turn the
microphone over to Dr. Swift.
CHAIRMAN HORNBERGER: Just a quick one,
Carol. What's BSC?
MS. HANLON: Bechtel.
DR. SWIFT: BSC is Bechtel SAIC Company.
It's the management operating contractor and this
first presentation is the four letter reports that
Carol mentioned. I'll go through them, fairly
quickly, but just summarize what new information there
is relevant to performance assessment since the major
documents of last summer.
I should credit many other people, Jerry
McNish, the manager of the Total System Performance
Assessment Department, in particular; and Mike DeLugo,
who was the lead on one of the four letter reports,
the largest, that Update Impact letter report.
And just to clarify, there was one mention
made there on Carol's side on realigning science and
performance assessment activities within Bechtel SAIC
and what has been done is that the Post-closure
Science Programs have been brought together with
performance assessment into a single organization
called the Performance Assessment Project. Bob
Andrews is the manager of that.
And the performance assessment
calculations, the TSPA, Total Systems Performance
Assessment, is one department within that larger
Performance Assessment Project. In fact, there are
several subprojects. TSPA now actually reports to me
in this group called Performance Assessment Strategy
and Scope.
The science programs we're familiar with
for years also now report directly to Bob Andrews
within Performance Assessment.
A couple of overview slides here, just to
go through quickly. What we have here first, there's
a body of information that is the Total System
Performance Assessment for the Site Recommendation,
TSPASR, documentation and with that I'm including the
Supplemental Science Performance Analyses from last
summer, last spring and summer.
This, the SSPA and the other documents
that are associated with that, I believe have already
been presented to this group, so what I'm focusing on
are things that follow that, that's this page and the
next one in the handout. A Letter Report in
September, completed in September, looking at the
impacts of the final EPA rule and also supporting the
final environmental impact statement and then a Letter
Report in December on the impacts of the NRC's final
rule which was, we felt, there are enough things in
that to run traditional analyses.
And then this technical update impact
Letter Report, known by its acronym as the TUILR.
These, so you can -- a graphic showing you what the
documentation is, two pages of this. First page is
performance assessment documents, going all the way
back to September of 2000, a document called the TSPA,
SR-Rev 0 and it's updated. ICN stands for Interim
Change Notice. That's basically the revision.
Updated in December, that's the version which people
are most familiar with. That supported the site
recommendation and the upper-tier documents that were
released that spring and summer, but it was also
updated in the spring, the supplementary analyses were
published in July in something called the
Supplementary Science Performance Analyses Volume 2,
SSPA Volume 2.
Then September and December, new results
that you probably have not seen yet. The Part 197
update and the Part 63 update.
MEMBER GARRICK: Peter, when you get
around to doing the TSPA-LA, will it integrate all of
these documents into the TSPA-LA?
DR. SWIFT: The TSPA-LA will be stand
alone in the sense that it will be a complete
documentation of its own set of analyses. It will
probably most closely resemble the models used in
these ones, but does that answer your question?
We don't have to keep sending you back to
a lower tier or older documents.
MEMBER GARRICK: Okay, thank you.
DR. SWIFT: This talk is about TSPA, but
it's worth keeping tracking of the non-TSPA documents
also, the upper tier documents of the science
documents.
I've lumped them both together on this
side. Go back to 2000, you have the Process Model
Reports and the Analysis Model Reports prior to the
scientific basis or TSPA-SR. They fed into an upper
tier DOE document released last May, the Yucca
Mountain Science and Engineering Report. These were
contract reports. This is a DOC document. This is a
primary technical basis for site recommendation, a
thing called the Science and Engineering Report,
published in early May 2001.
The scientific basis was updated again in
the spring of 2001 in this Supplementary Science and
Performance Analyses by one which was a scientific
basis. This document, published in July as a DOE
document, I believe, has new science that was not in
this one. And also in Volume 2 it has new TSPA
analysis.
Together, these two supported the
preliminary site suitability evaluation. This is the
document that actually makes the site recommendation
case. That was a DOE document published in August.
Thecover date is July,but wasn'treleased until August.
And this is all material you've seen or
was available. This is the new part over here, the
November 2001, Technical Update Impact Letter Report.
(Slide change.)
DR. SWIFT: Now the Letter Report on the
Final EPA Rule and it's worth actually noting the
footnote. If you try to do a search in any records,
data base, looking for that document, you'll discover
that the title of it says it's input to the final
environmental impact statement. That's correct.
Informally, we think of it as the update report on the
EPA rule and it was originally planned prior to the
completion of the EPA rule. It was originally planned
as an EIS update.
So the TSPA was modified to meet
specifications in Part 197. We went from the average
member of the critical group to the reasonably
maximally exposed individual. We went from 20
kilometers to 18 kilometers, both for groundwater
release and for the volcanic disruption scenario and
ashfall. And the EPA rules specified 3000 acre/feet
per year for groundwater protection. So we ran those.
(Slide change.)
DR. SWIFT: We also, these were the ones
that were aspects of the analysis that was planned for
the EIS, looked at both the base case waste policy act
inventory and a possible expanded inventory. And that
was the main point of the EIS.
We also ran some updated igneous activity
scenarios. We reran human intrusion which we had not
run since December of 2000 and we looked at two
different times for human intrusion.
(Slide change.)
DR. SWIFT: So far those changes were all
driven by regulation or assumption. We also did make
changes in the model itself since the model used in
the spring of 2001. I listed the most important one
here first. Waste-package corrosion calculations for
the results of that show -- used a general corrosion
model that was independent of temperature.
In the SSPA, the supplementary results
from next spring, we had used a temperature
independent corrosion model which basically showed
corrosion slowing at lower temperatures. We felt
there was insufficient technical basis to support that
for the site recommendation. You know it was already
published in the SSPA, so we took it back out and that
one change there counts for most of what you're going
to see in these slides.
We found an error in our in-drift
thermal-hydrology work which we omitted heat transfer.
It made a whole lot of difference. So we put it back
in and got it right.
We had omitted portal transport from the
portal due to intrusion. We corrected that. We had
an updated version of a waste package degradation
model. And we modified the inventory slightly at the
request of the Naval programs to treat their fuel as
part of the commercial inventory, whether it's a DOE
inventory. It's a small fraction anyway and would
make no difference.
(Slide change.)
DR. SWIFT: Results. These are mean
annual doses and millirems per year. I'm not showing
the complete panel of the doses that generated that,
but these are means drawn from 300 realizations. The
black curve here is the mean from TSPA-SR in December
of 2000. The red here is a single curve shown from
SSPA June-July of 2001. And this happens to be for
the high temperature operating mode that we looked.
This was only the high temperature mode here. In
SSPA, we looked at high and low. And then here, blue
and green, you can hardly tell the difference between
them, this new modified model run for both high and
low temperature for the updated model.
MEMBER GARRICK: Has the red curve not
reached its peak yet?
DR. SWIFT: That is correct, the red curve
-- unless that is its peak. We don't know that. But
in the actual highest point on the curve is here.
That's due a climate spike. By inference, we believe
that -- we can't rule out the possibility it might
have achieved a higher peak if it ran longer, but
there actually is a peak in there.
Taking out the temperature-dependent
corrosion, basically moves the time of large scale
package failure from here to 740,000 years and what
that has done is basically it leaves the -- in the red
curve corrosion rates slowed as temperature dropped
later and the green and blue curves, they do not.
They stay at a higher corrosion rate throughout.
CHAIRMAN HORNBERGER: What's the change to
explain the differences at early times?
DR. SWIFT: It'll come to me in a minute.
CHAIRMAN HORNBERGER: Is it an assumption
on juvenile failures or is it igneous activity or what
is it?
DR. SWIFT: No, it's juvenile failure.
For the SR, we had input from our waste package
engineers, but they saw no credible mechanism for
juvenile failures, so we had none. This is the
earliest general corrosion failure showing up here on
the black curve.
For the updates for both SSPA and the more
recent work early last fall, we have the first general
corrosion failures later. They're out in here. But
we do now have a model for juvenile failures, early
failures, due to improper heat treatment of lid wells.
The number of failures, in about a quarter of our
realizations, we had one or two packages out of 11,000
failing. So it's a very small failure rate, but it
produces a non-zero dose. It gives you small numbers.
This is a non-zero dose out to there that is largely
driven by igneous iodine and Carbon 14 in groundwater
transport.
MEMBER GARRICK: Are you going to later
get into a little more detail about impact of the
changes in this -- in the model in relation to the
difference in the assumptions between the TSPA-SR and
these results? I'm thinking of things like if you've
introduced this corrosion model now, has that brought
seepage back into the picture as an important
phenomena because in the TSPA-SR it was not an
important phenomena.
DR. SWIFT: It's still not particularly
important here. It matters for transport away from a
package, but the corrosion model is still independent
of water saturation. As long as you have humidity,
you have corrosion.
MEMBER GARRICK: And you still have the
same model inside the waste package of the saturated
water environment, those kinds of things?
DR. SWIFT: Uh-huh. Yes. The end package
transport model, I think is when --
MEMBER GARRICK: So it's still diffusive
transport that's the main?
DR. SWIFT: Yes. One significant
difference between and this applies for both the red
and blue-green here. A significant difference between
these two curves and this one is that the -- in an
attempt to put a little more realism in that diffusive
transport pathway of the package, we now split the
transporting waste that are transported by diffusive
properties when they reach the drift wall, the rock.
We put the diffusive transport fraction into the
matrix of the rock and we put the effective, if there
is effective, transport and that would that synchrony.
We put that fraction and it fractures.
Previously, we put it all into the fractures, this
curve, put all the waste and the fractures and that
didn't seem realistic, simply based on the surface
area available for diffusive transport, most of it is
going to go into the largest part of the surface area
which is matrix.
So that's the only change that I think
comes to mind for me anyway, between -- for the in-
drift transport model, between this, these curves and
that one. Probably more realistic with the splitting
of the diffusive.
Ask questions as I go. The time and the
fact that I'm only one person, I wasn't planning to go
out for a lot of detail in this stuff, so go ahead and
ask question.
VICE CHAIRMAN WYMER: I have a question.
How important was the microbiological corrosion? Was
it important at all?
DR. SWIFT: Bill, do you want to field
that one?
DR. BOYLE: I'm sorry, I don't have the
answer.
VICE CHAIRMAN WYMER: Is it a minor
player?
DR. SWIFT: No, I don't think it's a
player at all.
VICE CHAIRMAN WYMER: The other question
is what is meant by aging multipliers of inside out
corrosion?
DR. SWIFT: Aging multipliers for inside
out corrosion. That is pretty cryptic. The model
does not have an explosive treatment of the behavior
of alloy 22 as it ages. Instead, we apply a
multiplier to the corrosion rate to account for aging,
changes in the alloy aging. I don't know what the
update was, but someone felt, I suspect that in the
SRR model we had an aging multiplier only on outside
in corrosion.
Somebody pointed out we should have it on
the inside out corrosion also. But it's an uncertain
parameter. It's a parameter that has a range on it to
account for our uncertainty in the effects of aging
and corrosion.
VICE CHAIRMAN WYMER: So the multiplier
then --
DR. SWIFT: Accelerates the rate.
VICE CHAIRMAN WYMER: Accelerates the
corrosion, in some arbitrarily decided way?
DR. SWIFT: Uh-huh. In some -- I hope
it's more than arbitrary, but it's not physics based.
MEMBER LEVENSON: Is the uncertainty ever
symmetrical? It's always in a more dangerous
direction?
DR. SWIFT: We'll come to that in the next
set of talks. For these analyses, I believe it is
asymmetrical in many cases. I would like to see more
symmetry.
MEMBER GARRICK: Generally, more of a log
normal than a uniform?
DR. SWIFT: I know where you're headed
with the question. Keep asking it.
(Slide change.)
DR. SWIFT: The igneous activity results.
I think that's the next -- yeah. This same
presentation or say similar presentation was given by
Jerry McNish to the Review Board in January and this
figure drew quite a lot of attention from the Board
who were displeased with the lack of prominence given
to the word "probability weighted" here. I want to
make clear of that right now. These are probability
weighted mean annual doses. This is consistent with
what's in Part 63. This is not being included with
obvious dose you'd expect to see, but it is -- I'll go
through that in a couple of slides here, what it
really is.
This is the regulatory dose of volcanic
activity. The black curve here is what was shown in
TSPA-SR in December of 2000 and the blue and red and
curves here were updated in September and these, by
the way are essentially identical. I was also updated
in the SSPA in June and July.
The red and blue were a perfect overlay
here for high temperature and low temperature
operating modes. The volcanos are pretty insensitive
to temperature of the depository.
Changes here, recent updates, since SSPA,
specifically for this analysis, we move the location
from 20 kilometers to 18 kilometers and we updated the
biosphere dose conversion factors. We also made all
the changes I just talked about in the nominal model.
That's the other feature that's here.
There are a series of other changes not
described here which were updated as part of SSPA in
the spring. They are what account for this vector of
25 increase from here to here, interruptive dose and
the decrease over here at later times.
The smooth curve to here or all the way
out to here is driven by the volcanic ashfall dose and
the irregular curve here and here is from the
groundwater release from damaged packages and at some
point in the future the basic weight, probability
weighted dose from the groundwater pathway from
packages damaged by igneous activity will cross over
and exceed the corruptive ones. If you put edging for
the black line, you adjust the eruptive half, we have
a curve that kept on going out like that where it's a
groundwater curve, goes like that. So at the
crossover point, you see that the curve changed from
being smooth to being irregular.
The sharpness here is due to a long term
climate change in the model, it's spiking here. These
are glacial climates.
The other major changes here between
basically we -- at suggestions from the center and the
NRC staff, we looked at a different wind speed data
set which led to an increase of about, a factor of 2.5
from here to here. We updated our biosphere dose
conversation factors --
MEMBER GARRICK: Does that mean you will
look more at a wind row than a --
DR. SWIFT: No, the wind direction is
still assumed to be fixed towards the location of the
REMI for these. So that would have to be a factor of
4 or 5.
MEMBER GARRICK: Yes.
DR. SWIFT: It's not a huge player, but
yeah --
MEMBER GARRICK: It is quite a huge
player.
DR. SWIFT: The 4 or 5 add up.
MEMBER GARRICK: Yes.
DR. SWIFT: We looked at wind from a
higher altitude. They had pointed out that we had a
data set that went to higher altitude than we could
have used and we used that and that was part of the
difference here.
As a matter of fact, we unrealistically
used only the highest altitude, the 300 millibar data
only went into that, whereas for this one, we used a
somewhat lower altitude data set against the full
column of wind speeds and got the elevation up.
This also has an increase in the number of
packages involved in the eruption, due to a
recalculation of how we did that. Has increased dose
conversion factors due to reconsideration of the nasal
ingestion pathway. That's the larger particles lodged
in the nose. We ended up putting in the long --
MEMBER GARRICK: But you continued to use
the assumption that all the waste packages were
degraded that were in the intersect?
DR. SWIFT: Yes. All packages in the --
we were conceptualizing the volcano as a conduit, a
cylinder that rises up through it. It's also got an
intrusive dike, a tabular body that may cross many
drifts, but the portion that erupts, we're assuming is
a cylinder with a mean diameter of about 50 meters of
- or medium diameter.
Yes, all packages in that cutout by the
cylinder are assumed to be fully destroyed. The
phrase is damaged sufficiently to provide no further
protection. And the waste within them is produced to
the grain size of the particles which is by-products.
MEMBER LEVENSON: Is there any
significant, for this type analysis, is there any
significant difference in the footprint of the high
level versus the low level?
DR. SWIFT: The high-temperature/low-
temperature?
MEMBER LEVENSON: Yes.
DR. SWIFT: It's a simple scaling. It
affects the probability that the event will hit it at
all. And if you need to have a larger footprint for
a lower temperature operating mode, then the
probability of the event scales -- it's not precisely
linear because -- it's close enough. If you double
the footprint, you're going to double the probability.
MEMBER LEVENSON: I would have expected to
see a little difference between the high temperature
--
DR. SWIFT: Oh, thank you. Thank you. We
didn't do it. We said that and that's a caveat that
is in the text and I should say that. We simply used
the one footprint for this and in text we discuss how
to use the weighting factor if you want to.
It's not clear that we will have different
footprints. One of the options was for low
temperature, was to use the same footprint and a
longer and more rapid ventilation period.
So we weren't quite sure what to do with
that and it was going to be a nuisance to --
MEMBER GARRICK: How about the erosion
time of the 1000-year erosion time for the 15
centimeter layer? You're still using that?
DR. SWIFT: No. Let me explain what we
actually did here. This is -- this is a good slide to
do it with.
This is the -- what we call the
conditional dose, the dose that you would get -- this
is a figure that we probably should have showed them
ERB in January, but didn't.
If an event happened at 100 years and
these were calculated by the way it would be SR
modeled, black being the curve on the previous page.
If an event were to happen at 100 years, a person
living after that would receive a dose as shown here.
So a person alive at 2000 years might receive a dose
somewhere in this bandwidth here, the mean being red,
so that would be 95th shown, both shown in black.
Clearly, the dose would be worse if you
were alive at the year of the volcano. There's no
probability weighting shown here. The uncertainty
between the lowest and the highest curves reflects
uncertainty basically in the inputs to our ASHPLUME or
transport model, things like windspeed and the conduit
diameter. That's basically how many packages are
effective. And also in our biosphere conversion
factors.
The slope of the curve, how fast it drops
off through time is a factor of two things. One is
there's radioactive decay and the other is how quickly
that contaminated ash layer erodes away. And -- all
right, that's the top figure.
The bottom figure down here is just mean
curves. The red here. Now it's just the mean curve
shown for condition events at different times at 100,
500, 1,000, 5,000 years.
This, if you were to draw a curve,
connects the dots through the tops of them, that's the
radioactive decay curve. So these curves then are our
soil removal factor. That's the rate at which soil
will be contaminated and ash layers being eroded away.
However, our treatment is not quite as
simple, John, as the way you describe it. What we're
doing is we are assuming that the top layer of top
soil erodes at a rate of 6 to 8 millimeters per year.
I believe that's right. However, we're assuming that
soil is plowed annually, so it's constantly being
remixed to 15 centimeters, so that any way -- how
thick the ash layer is, the radionuclides get mixed in
to a 15 millimeter soil layer every year and the top
of it gets skimmed off every year. So it's an
exponential decay in our soil removal rather than a
simple decay. There's always some still left there.
And so if we weren't mixing, we would take
off that 15 centimeters in several hundreds of years.
It would be relatively rapid. We are mixing, so that
we're always creating a -- we've always moving
radionuclides deeper down in that soil layer with each
year's plowing and erosion.
Clearly, we are fairly sensitive to the
way we treat erosion. If we had zero erosion, if the
soil layer stayed there forever, this curve would look
like the connecting the dots from the top of there,
simply go out like that. Would be a radioactive decay
curve.
CHAIRMAN HORNBERGER: This is a pretty
critical part of the model for the first few thousand
years?
DR. SWIFT: Yes, it is.
MEMBER GARRICK: And it struck me as an
extremely conservative assumption.
DR. SWIFT: The assumptions that go into
that have to do with whether you think we're dealing
with agricultural land or stable desert soil. If
we're dealing with agricultural land that really is
being plowed every year, this may not be that
unrealistic. We have a fairly high, compared to what,
for example, the NRC staff has recommended, we have a
relatively rate at which stuff blows off, but because
we're plowing and mixing, that is consistent with what
you'd expect to see on crop land.
On the other hand, if we didn't have this
plowing and mixing going on, we had stable desert
soil, we shouldn't have such air mass loading or such
rapid erosion. We have pretty high air mass loading
in our BACS. It's dusty air people are breathing in,
consistent with agricultural land. It's blowing
around.
It's what we did anyway. I just want to
show one other slide here.
(Slide change.)
DR. SWIFT: This is how you get from those
conditional doses to dose mean doses because this is
something that it's not intuitive and this is just a
question of probability space rather than a real
phenomenon. This is what the role asks for and I
believe it makes sense.
Think of these as mean doses from the
previous curve, the mean conventional dose. If an
eruption happened, Volcano 1 happened in Time 1 and
you dropped the time axis here, this is dose/time, a
person alive in the future could get in the Year T-1,
they would get that dose. If they were in Year T-5,
but the eruption was in Year T-1, they would get this
dose here off that curve there.
If, on the other hand, they were in Year
T-5 and an eruption happened in Year T-6 out here,
they'd get a zero dose. The eruption hasn't happened
yet.
Now put it into probability weighting, the
probability that a person living out here in the Year
T-5 could get a dose from an eruption that already
happened back in Year 1, Year 2, 3, 4 and so on, well,
the probabilities of those were all the same, similar
to the process that has a time constant probability
and so the probability weighted mean dose we'd get out
here is simply the sum of the probability of all the
events in the time interval of interest, 0 to 1,000
years times the doses associated with each one of
those events at the time you're interested in.
So at Time 5, this person living here
could be getting a dose from this event, from this
event, this event, or that event. That one is a zero.
And each one of them has equal probability and you
multiply them and sum them up. And what you get when
you do this, this is actually what we do, but a little
thought experiment suggests that at early times,
although the consequences are highest, the probability
that the event happens in that year or has already
happened, the probability is low. As you go out in
time, the probability accumulates.
So the probability if you're living out
here at the Year 10,000, the probability that the
event has already happened is 10,000 times the
probability in the first year. So in this sum here,
the doses go down at later times, but the
probabilities accumulate and you'd expect to see a
mean curve that starts out low and climbs to some
intermediate peak and then falls off again as doses
decay from radioactive decay.
And that actually is what -- that's what
the blue curve is here or the black one. The peak is
around 3000 years and after that radioactive decay
takes over and starts to drop off.
So that's -- the point of that explanation
is just to say, show how we got from things that look
like this to the probability of weighted sum that the
regulation asks for.
I've got to speed up here.
(Slide change.)
DR. SWIFT: Human intrusion scenarios.
This is the forced assumption that a driller drills
through the waste package. Part 197 says one waste
package, made a pathway to the borehole pathway to the
saturated zone and assume it occurs at a time when the
waste package is degraded enough that the grower would
not recognize it. And this then is -- we picked
30,000 years. And this is an intrusion of 30,000
years as our annual doses, a full set of 300 of them
with a mean shown.
And these are -- we also reran it for the
proposed NRC rules, was prior to finalization of 197.
We used the 100-year time. What you see here, for
example, from 100 out to the first arrivals coming in,
this is basically your minimum saturated zone
transport time. The spread of arrival is out from
time after that show the spread and saturated zone of
transport. So some realizations showed first arrivals
here. Some didn't have them arriving too well out
there.
(Slide change.)
DR. SWIFT: The December Report looked at
the impact of Part -- final Part 63. The main
difference here was the rule now requires us to use
3000 acre/feet per year for individual protection
which is something the EPA has not clarified in their
rule. So now we were using a sample value previously.
We also in this report, if you get a hold of the
report and read it, will discover we ran a couple of
cases that are now moot following clarification of the
word "unlikely" and the new proposed rule.
We went ahead and ran a case with an
igneous intrusion eruption for the groundwater
protection standard and also for the human intrusion.
And with a clarification of the rule in those cases
are moot.
What happened with the 3,000 acre/feet.
The result was to scale those by approximately two
thirds.
CHAIRMAN HORNBERGER: Two thirds or three
halves?
DR. SWIFT: Two thirds. We're diluting
them.
CHAIRMAN HORNBERGER: Oh, that's dilution.
DR. SWIFT: This was a sample value.
There's the range given, with a mean of about 3,000
acre/feet in our -- this is what we found from our
survey of these in the region. Well, this just pushed
us to the upper portion of the range in the rule and
produces a little more dilution.
These -- the numbers shown here are the
nominal performance only. These are the numbers of
the doses due to the juvenile failures of nominal
performance. We took the volcano out to show that.
CHAIRMAN HORNBERGER: It's two-thirds
because you put everything into the volume.
DR. SWIFT: The larger volume.
CHAIRMAN HORNBERGER: Rather than have a
concentration?
DR. SWIFT: Yes.
(Slide change.)
DR. SWIFT: The Technical Update Impact
Letter Report, this is the fourth of them. I said a
letter report. This is -- this report is underlying
science rather than TSPA. I had a much smaller role
in this report, but Mike DeLugo is the person who did
most of the coordinating of it.
The point here was documenting additional
information since completion of the underlying science
for the Science and Engineering Report and the Yucca
Mountain Preliminary Site Suitability Evaluation. So
this updates in science since roughly the spring of
2001. In some cases it goes back a little further
than that. But this was a new work that was going on
last summer and early fall in experimental programs.
And then the impacts of this work were evaluated on
TSPA and preclosure, basically to make sure there
weren't any -- wasn't any new information that would
necessitate a re-evaluation of the said
recommendation.
It's a thick report. It's almost 400
pages long. It includes 11 White Papers in each of
the topical areas where the technical staff was sent
back to just document what is their new information.
Then we had a rapid series of workshops where we
looked at the impacts. To do this, we got the
technical experts who wrote those White Papers
together with TSPA analysts and we had our workshop
setting. We went through each topic and on the spot
estimated elicited impacts on total systems.
A couple of pages here of examples of the
sorts of things. This isn't a very complete list.
Just the sorts of information that was available:
fracture data, seepage data. These were things that
were written up and then people were asked how would
this affect the input for models and the modelers were
asked would this have an effect. And so on, the high
profile one here, the discovery of high concentrations
of chloride in seepage waters at low temperatures.
VICE CHAIRMAN WYMER: In your technical
update, you didn't include a couple of processes.
DR. SWIFT: To the extent that they're
captured in the on-going work related to the --
usually the air field environment and the engineered
barrier system, yes, we did. It was structured around
the existing science programs. We didn't force a
White Paper on a couple of processes.
You want to deal with that?
DR. BOYLE: That fluoride example is
actually -- falls within the realm of a couple of
processes. As it turns out the flourine came from the
materials that were introduced by jackets, but it was
postulated that it could have been coming from the
fluoride. That's the small amount of fluoride that's
present in the rocks.
VICE CHAIRMAN WYMER: So to that extent
you put it in.
DR. SWIFT: Yes. There were some portions
of this were entirely preclosure, for example, updated
data on aircraft activity. We have a new survey of
aircraft traffic in the area, know that that would
change the risk of accidental airplane crash, which
seems moot now.
(Slide change.)
DR. SWIFT: So what were the results of
this and if you've got a copy of the technical update,
the TUILR, I recommend you go straight to the very
back end of it on pages 350 on where there's an
appendix that discusses impacts on post closure
assessment and there are a series of chapters in that
appendix, a weight for each of the various measures of
the components. And here's the conclusion. All
impacts of all the new work are insignificant, except
for these two.
First, the Transport Team believes that
they're now able to show a reduction of nominal dose,
igneous dose, but the nominal performance. They thing
it may show a more delayed retardation in unsaturated
zone perhaps lowering the 10,000 year dose up to one
order of magnitude. This is lowering, but it's also
just pushing it up further in time. It's slowing the
transport.
VICE CHAIRMAN WYMER: What's that due to?
DR. SWIFT: Excuse me?
VICE CHAIRMAN WYMER: What slows it down?
DR. SWIFT: It's a change in the way
they've been treating diffusion near the matrix and
this is not my field. It's out of the realistic case
AMR and so-called realistic case AMR and that it was
a change -- actually, I believe a numerical treatment
in the model of the -- I believe it added more cells
in the matrix, so you -- instead of having the matrix
represented by a single cell and diffusion occurring
all the way to the center at once and way back out
again in a single step, I believe in the numerical
model there, so it takes longer for the diffusion to
get in and out. And it's the back out part where
we're seeing the benefit.
The model probably will not be permitted
in the TSPA because it's numerical intensive. But
anyway, and also we weren't too excited about a
possible one order of magnitude reduction in a dose
that's already 10-5. But it's worth knowing anyway.
It may be there.
Possible increases in the eruptive dose.
Basically, we looked at the impacts of the Center's
model and the conclusion of our staff was that no more
than one order of magnitude increase there. That's
based simply on looking at the total number of
packages that might be involved. The largest
difference is between -- from a performance point of
view, the largest difference is between the Center's
model and the one we're using as the Center's proposed
mechanism has more packages.
MR. HINZE: Peter, if I may, Bill Hinze.
You have eruptive there. What about intrusive? Is
that considered in that?
DR. SWIFT: Yes. The effective is only on
the eruptive side here. That's deliberate to say
eruptive in the dose there.
The Center's model that basically we're
concerned about here, we worry about, is the one that
calls for what I call a dog-leg eruption where magma
rises, hits a drift, flows down a drift and goes back
up again. Ours goes straight through and they
proposed, and we can't rule out the possibility it
would go a dog-leg path and sweep the entire drift and
of course, the eruption. And if so, more packages
will be involved.
MR. HINZE: Are you considering it at all
in terms of the intrusive? Is that going to be coming
along?
DR. SWIFT: We are reevaluating our model
for the intrusive effects. We don't see a big impact
there on dose. We believe our model needs more work
to be ready for the LA, but we don't think that's
going to change much.
MR. HINZE: Are you eliminating the zones
that you had in terms of disruption of the canisters?
DR. SWIFT: That may be modified for LA.
It may not. We're working on that right now. Some
version of that is likely to stay at the Zone 1 of
extreme damage and Zone 2, lesser damage, but in fact,
there are igneous geniuses are working on that
question right now.
MR. HINZE: Thank you.
MEMBER GARRICK: If the new data that's
being talked about now on igneous activity results in
an increase in the likelihood term, what is that going
to do to your results?
DR. SWIFT: In terms of increasing the
probability of a volcanic event or an eruption, those
are separate probabilities.
MEMBER GARRICK: Yes.
DR. SWIFT: Increasing the probability of
either of those at the site is pretty much a direct
scaler on the probability, the way it goes.
MEMBER GARRICK: Right.
DR. SWIFT: There is a question about the
air magnetic data and whether or not that will change
the probability.
MEMBER GARRICK: So if the cases increases
that a 10-7 number is not even justified on the basis
of the supporting evidence and it may be more like
10-6, if that happens it's going to be pretty much a
linear effect?
DR. SWIFT: Yes. We don't think that's
going to happen. Our of our impact assessment, we
don't think that probability is changing much.
And I think that does it. No, I've got
the summary slide.
(Slide change.)
DR. SWIFT: Just for completeness, to note
that we did look at impacts on pre-closure performance
of new data. Also, we didn't see anything there.
I think that's simply a summary side.
Let's me sit down.
The analyses that I've just summarized
here, basically provide confidence in the adequacy,
appropriateness of the SR.
MEMBER GARRICK: I know we're running a
little behind and I want to give the Committee a
chance to ask questions, but you'll be hanging around,
will you not?
DR. SWIFT: Actually, I should have said
that right off. No, I have a 2:55 flight to catch.
MEMBER GARRICK: Oh, I see. Well, then
let's give the Committee the benefit of your presence
and see if there are any questions.
DR. SWIFT: Bill and I are your speakers
until 12:30.
MEMBER GARRICK: And you have to leave --
yes. Okay.
Ray, go ahead.
VICE CHAIRMAN WYMER: In your backup slide
24, you indicate that Carbon-14 is rated Class C.
DR. SWIFT: Yes.
VICE CHAIRMAN WYMER: Why did you put that
in there?
DR. SWIFT: I don't actually know where
that came from, Class C. So I guess I can't answer
your question. I realize it was likely to come. I
don't know.
VICE CHAIRMAN WYMER: That's the first
time I really heard it talked about the Carbon-14
being rated Class C.
DR. SWIFT: I can say that we do not have
a realistic model for groundwater transport. This is
based on the assumption that Carbon-14, carbon, in
general, is a nonreactive species for groundwater
transport, our groundwater chemists just don't like
that.
(Laughter.)
DR. SWIFT: So basically that's an upper
bound on Carbon-14.
VICE CHAIRMAN WYMER: You don't know where
that came from?
DR. SWIFT: No, I don't.
MR. BOYLE: Good morning. Thank you for
this opportunity. Peter had talked about some
updates, and these next two talks -- the first by me
and then I'll be followed by Peter -- are going to
deal with uncertainty analyses and what we're doing
with uncertainties.
For those of you that were present at the
NWTRB meeting at the end of January in Pahrump, I made
this presentation there, and I'm pretty much going to
make the same presentation. And I think Peter will as
well, for the most part.
This report -- it's available at our
website, if you haven't seen it already. And it
represents the work of others, in particular the two
people whose signatures are on the report -- Kevin
Coppersmith of Coppersmith Consulting, and Jerry
McNish of BSC.
And Chapter 2 of the report was prepared
by Jerry and the various process model leads. Chapter
3 was prepared by Kevin, and with input from Peter and
Bob Andrews, comments from them. And Chapter 4 was
prepared by Karen Jenny and Tim Nieman of GeoMatrix.
Now, the overview of the next two talks --
the first is by me on the report itself, "Uncertainty
Analyses and Strategy Report," and Peter is going to
talk about how to -- the implementation of a
consistent treatment of uncertainty in the TSPA, total
system performance assessment, for license
application.
This is the title of Section 1 of the
report. It's "Introduction," and the three main goals
of the report are listed on page 2 of the report.
I've distilled them here in these three bullets. This
is what is done in Section 2 of the report. Summarize
and discuss what we at the project have done to
evaluate, clarify, and improve the representation of
uncertainty in the total system performance
assessment. That's Section 2, and it also gets at
comments made by other groups.
Based on this discussion, Section 3
develops a strategy for how to handle uncertainties,
and it also proposes some improvements for the future.
And then, Section 4 deals specifically with how to
communicate uncertainties to various groups,
decisionmakers, technical people, and also proposes
some improvements for the future.
The next I think it's six or so -- I think
it's up through page 9 -- pages 4 through 9 of the
package you have are a table. And it's related to
something that's in Section 2 of the report. Here is
the title of Section 2 of the report, "Evaluation of
Uncertainty Treatment in the TSPA and the Significance
of Uncertainties."
On pages 30 and 31 of the report, there is
Table 2.2 and it's called "Key Remaining
Uncertainties," and it deals in the table with these
first four columns. And in the report there is in
that table in the report, this fifth column isn't
there. The information that's in the fifth column I'm
showing you here is in the report, but it's in the
text of the report. But we had a request from people
at headquarters to distill down those paragraphs and
pages in the report and create this fifth column.
As I did at the NWTRB meeting in Pahrump
-- we'll be here all day if we go through each and
every item in this table. The main point that I want
to get across with respect to Section 2 is the various
technical investigators were asked to summarize the
state of uncertainties. What I asked them to do is I
asked them, how can you sleep at night knowing that
there was a potential at that time that a decision was
going to be made? How can you sleep at night with the
remaining uncertainties? And that's what this table
and those parts of the report tried to capture.
We got back two very common answers of why
these people were able to sleep at night. One is the
uncertainties really didn't matter. They looked at a
broad range, and for some of the items it didn't
really affect the dose at 18 or 20 kilometers.
MEMBER GARRICK: But, Bill, isn't that
dependent upon the model?
MR. BOYLE: Sure.
MEMBER GARRICK: Because in the VA, for
example, seepage was a very important phenomena.
MR. BOYLE: Right.
MEMBER GARRICK: And so you changed your
corrosion model, so that in the site recommendation
report it's not an important phenomena.
MR. BOYLE: Right.
MEMBER GARRICK: And I think it's those
kinds of connections that are very important.
MR. BOYLE: Right. And that's the point
I would say when they -- when I say that there wasn't
-- it really didn't have an effect or it wasn't
important, it is with respect to the insights that
were being gained by an implementation of either the
TSPA itself or some subsystem.
But, you know, the answer is both.
Sometimes it was -- when carried all the way through
to the end of the TSPA calculation, it showed that it
didn't matter, which then just raises the question,
what if the underlying models really aren't right?
But those were the answers I got back from
the PIs. One is it really didn't matter, it seemed,
over a range of uncertainty. But the second answer
that came back quite frequently, and is represented in
the far right column, in various words is, "Well, I
was conservative." You know, I took a bound, like the
one that deals with the rock -- acknowledge that the
analyses were very conservative.
Whether that's a palatable approach in the
end, that is what was used at this point, and that's
the answer that was given.
So with that, we can't possibly spend a
lot of time on all these technical items. In January
in Pahrump, I jumped up to slide 10. I'm going to do
it here today again. And it's -- we're jumping to a
new section of the report, and this was a very
important section of the report, Section 3, and that's
the title of it up there, "Strategy for the Future
Treatment of Uncertainties."
And Section 3.1 of the report has a
compilation of words from the regulation. It quotes
from the EPA's regulation on how uncertainties should
be treated. It has quotes from what at that point was
-- probably started with the draft of 63, and then we
may have stayed with the draft or perhaps we got the
final comments from 63. I think we did get the final
comments from 63, but also comments from this
committee, the Nuclear Waste Technical Review Board,
the NEA/IAEA peer review group for the TSPA, and also
the peer review group we had for the TSPA-VA.
So we synthesized all of those -- you
know, provided the quotes and synthesized those
comments in Section 3.1. And then, in Section 3.2,
came up with a strategy for the future. And on these
next two slides, slide 10 here and 11, there were
eight recommended things to do. And these are the
quotes from those eight things. The first four are
shown here.
And if you read the report, each of the
eight recommendations starts off with a section in
bold, and that's what's reproduced here. And so they
are develop a total system performance assessment that
meets the intent of reasonable expectation. That's
defined in the EPA rule and also the NRC's word-for-
work exactly the same.
Quantify uncertainties in inputs to
performance assessment. Identify processes that
encourage the quantification of uncertainties and gain
concurrence on approaches with the Nuclear Regulatory
Commission. And provide the technical basis for all
uncertainty treatment.
Also, the fifth recommendation was to
address conceptual model uncertainty. Develop a
consistent set of definitions and methods for bounds
and conservative estimates. Develop and communicate
information that can be used by decisionmakers. And
this is dealt with more explicitly in Section 4 in the
next few slides. And also, develop detailed guidance
and provide for its implementation.
After the report came out, the DOE sent a
technical direction letter over from our contracting
officer over to Bechtel SAIC and told them to develop
this detailed guidance based upon a strategy, either
this strategy or one similar to it, and incorporate
that strategy into the planning exercises they were
doing to get us out to license application. And
that's what Peter is going to talk about in the next
talk.
At the meeting in Pahrump of the NWTRB,
detailed implementation was being developed -- a
document. I have a copy of it here somewhere. It was
being developed at that time, but now it actually has
been developed. And Peter will talk about that.
Now, Chapter 4 -- or Section 4 of the
uncertainty analyses and strategy report -- that's the
title of it -- "Communication of Uncertainties." This
exact figure is not actually in the report. There's
a very similar figure in the uncertainty report. I
think it's -- I wrote it down. It's Figure 2-13 on
page F-18 that's very similar to this. But this is
the slide I showed in January.
And what I wanted to get across -- for
those of you that -- you saw Peter's slide this
morning, slide 9, that showed the black, the blue, the
green, and the red curves. Carol in September showed
a similar such figure when she was making a
presentation for somebody else at a Nuclear Waste
Technical Review Board meeting. Tim Sullivan was
sick, and so Carol made that presentation with a very
similar figure.
And there were comments from Dr. Knoppman,
a member of the NWTRB, on the fact that that figure
doesn't show any uncertainty. It just shows means.
And so we took that comment to heart.
And if you go back and you look at the
preliminary site suitability evaluation document that
was out last summer, that also had figures of that
type which didn't show any uncertainty, where now if
you go and look at the final site suitability
evaluation documents you'll see this figure and some
of the other figures that I'm going to show in this
talk.
At the time of the talk in Pahrump in
January, the site suitability evaluation documents
weren't final yet, so I couldn't reference them. But
I was pretty sure that this figure might end up in it.
This figure is also -- Peter showed, I believe it was
slide 10, this morning, the one that he had labeled as
the probability weighted dose axis.
There was -- you could have read about the
controversy about that figure even in the general
press. It made the Las Vegas Review-Journal, and The
Sun, and also some of the energy-related documents.
The Nuclear Waste Technical Review Board,
even in their most recent letter to DOE, had concerns
essentially that generated from this figure and the
one that Jerry McNish had shown in the presentation
before, in that -- and I'm reproducing it here exactly
how it was shown in January to show -- it's
interesting that it comes up in a talk about
communication of uncertainties.
The concern is is that it's just labeled
as total annual dose, with no recognition that it's
probability weighted. And there were some concerns
perhaps that things were not being communicated quite
clearly. But as I said at the meeting in Pahrump, if
you go to the uncertainty analyses report, you'll see
an explanation down here that does describe it as
probability weighted.
Or if you go to the SSE, the site
suitability evaluation document, you'll see a big
paragraph that explains the fact that it's probability
weighted. But for a PowerPoint presentation, in order
to have a nice, big figure, that was stripped out.
So, you know, there were no ill intentions, but it
just shows that in communicating sometimes there can
be unintended consequences.
Now, all of these charts -- these next few
charts deal, as Peter has already described these
charts -- these have to do -- when it says "total," it
takes the disruptive igneous event doses and adds them
to the nominal. In a sense, they just look -- because
of the magnitude of the igneous doses, they just look
like the igneous doses.
I would much rather show the nominal
results. But by the time I get a few slides in you'll
see that in order to make meaningful graphs of some of
these results we have to go with something like the
igneous results, not the nominal results, because the
nominal results produce too many zero doses and they
don't make very meaningful graphs.
CHAIRMAN HORNBERGER: So I take it you've
solved this problem, and you now know how to
communicate --
MR. BOYLE: Yes.
CHAIRMAN HORNBERGER: -- clearly what a
convolution integral is to the lay public.
MR. BOYLE: We try.
(Laughter.)
I will say the January meeting ended at
midday. I drove right back and met with Ken DeLugo
for -- he was working on the site suitability
evaluation documents. We went through every figure in
there to make sure that the Y-axis was correctly
labeled and that we had the big paragraph explaining
it.
So we did take the Board's comment to
heart. We did not want to be misrepresenting anything
to anyone.
MEMBER GARRICK: Did the Board want you to
continue to show all the realizations, given that --
MR. BOYLE: I'll get to that. But wait
until you get to the next slide. One of the
recommendations in this Section 4 of the report is,
with respect to communicating uncertainties, there are
different audiences. Some people are much more
comfortable with a lot of detail. And decisionmakers,
or those that don't have a background in mathematics,
or in TSPA in particular, perhaps need less.
This is full-blown. But even in the
preliminary site suitability evaluation, we never
showed any such thing, which led to the comment about
Carol's presentation in September. So we did want to
show -- this shows all -- this shows probably a
maximum amount in terms of what you would want to show
in the results.
But since some people have difficulty with
the horsetail diagrams, one of the recommendations of
Section 4 is we'll thin it out some, if you will, you
know, clear it up. So these are essentially the same
results, but it's just shaded in between the 5th and
95th percentile, still showing a mean.
To remove some of the distractions of all
of the horsetails, try and get it across simpler, that
-- if you will, that this is an air band, if you want
to think of it that way, and it was shaded in to show
the possible range of results between the 5th and 95th
results. And this slide also is now labeled
probability.
CHAIRMAN HORNBERGER: Why the mean and not
the median?
MR. BOYLE: Why the mean and not the
median? Because it's the regulatory measure. That's
-- just make it simpler. You know, my wife
understands the difference between the mean and
median. She had to take a course. But many people do
not, so --
MS. HANLON: Bill, before you go on, I
just want to mention, since Bill is -- has talked
about the fact that Dr. Knoppman, as well as Dr.
Cohen, mentioned several times that they were unhappy
with the level of treatment of uncertainty, in the
final site suitability evaluation we did spend a great
deal of time, both in the executive summary as well as
in Chapter 4, going into a discussion of uncertainty
and putting more treatment in with what Bill is
talking about.
MR. BOYLE: Yes. And we may have added
the first -- the two figures, the full horsetail
diagram, which was a change from the preliminary site
suitability evaluation. I believe we added this one,
and I believe we added this one. And this is the
figure that gets across why I'm showing the igneous --
the combined total doses rather than the nominal
results.
This represents a cumulative distribution
function and a relative occurrence of PDF, if you
will, of the 5,000 realizations for the igneous doses.
And we get a nice, smooth cumulative distribution
function based on those 5,000 realizations. It goes
all the way from zero to one.
Whereas, in the nominal, within the 10,000
years, which is what the site suitability evaluation
dealt with, some 70 or 80 percent of all the
realizations for the nominal case are actually zero.
And it makes -- you end up with a funny-looking
cumulative distribution function, which I didn't want
to have to go into all that explanation, so we chose
a data set that gave a nice, smooth one.
And this figure is in the site suitability
evaluation, and what it represents is at the time of
the peak in this plot, at 312 years, right here, we
looked at all 5,000 realizations and plotted them up
as a cumulative distribution function and as a
relative occurrence of probability density function,
if you will.
And it can be seen just at first glance
because of the log scale, but it's a first cut. And
so they look approximately normal, so it's a log
normal result that's a first cut.
Then, my last slide, I ended with a quote
from Charles Darwin. I thought it was appropriate
relative to TSPA and uncertainties first, and so I
don't -- I haven't read this book by Darwin, but I got
the quote out of a book of quotes. And I don't know
in which -- what context he made this.
But it's interesting that it's by Darwin
and that our TSPA has been evolving, not by natural
selection, but we hope by survival of the fittest --
you know, the better models for surviving.
Also, TSPA -- it relates to TSPA in that
we are looking at the future in a TSPA, and we also
must make judgments based with conflicting and vague
probabilities. And with that, I turned it over to
Peter with one last explanation.
I think as perhaps this committee knows
full well, that there apparently are perhaps two types
of analysts, those that are very comfortable with
bounding, conservative approximations, and others that
want a fuller representation of the uncertainties
involved.
And I had -- after I put these slides
together I attended a National Academy of Sciences
meeting, Committee on Geological and Geotechnical
Engineering, where that discussion came up of the
frustrations when the two groups collide.
And it had nothing to do with Yucca
Mountain, but it put it in perspective for me that
we're not the only project that deals with this choice
of, do we just bound it and get on with it, and remove
some of the information, or should we deal with the
uncertainties more fully?
And I said at the January meeting in
Pahrump that the two different approaches, when viewed
in the extreme by the proponents of the other
approach, can be viewed as an unyielding rock, if you
will, one that doesn't yield any sort of information,
whereas the other can be viewed as this big whirlpool
that sucks in all available time and money.
And with that image of a rock and a
whirlpool, between which a path has to be charted,
brought to mind Odysseus sailing between Scylla and
Charybdis. And for us I said, "Peter is our Odysseus,
who is going to tell us how he was to chart a course
and the detailed implementation of how we were to
treat the uncertainties."
And at that point, the guidelines that I
showed you were in the process of being prepared, have
now been prepared. I think they provide a proper
course on how to deal with uncertainties. I'd like to
think that this committee would feel the same way, but
there's always a little caution in that, you know, the
answers in the implementation, you know, that the
guidelines are not that prescriptive in terms of
everybody would follow them exactly the same way.
So time will tell, but I'm heartened by
the approach that Peter and his staff have developed.
And I think he will tell you about it now.
MEMBER GARRICK: Just to telegraph
something that may be for the benefit of Peter is that
the problem is not whether the situation lends itself
to a bounding analysis or a probabilistic analysis.
The problem is that when you do a bounding
type analysis and you try to embed it in a
probabilistic analysis with language that's very
confusing, an example of which is to say, "Well, I
don't know what the solubility is, and I don't want to
put a distribution on it. So I'm going to assume that
this is what it is, and it's an upper bound." And
then you later say that there's no uncertainty
associated with the solubility because you assumed a
point value and as an upper bound.
Now, that's where you throw the system
into total turmoil, and that particular flaw is very
evident in the TSPA-SR. It's one thing to use
bounding analysis in a screening capacity, and what
have you, but it's another thing to use bounding
analysis on something about -- something that's very
uncertain, and then, in the wrap up say that there is
no uncertainty associated with it because you bounded
it.
And that's the same as ignoring the
uncertainty, and that's something that we have real
concern with.
MR. BOYLE: Right. And I think on that
same issue the Nuclear Waste Technical Review Board,
they used different words, but it's the same issue of
how --
MEMBER GARRICK: More elegantly, I'm sure.
(Laughter.)
MR. BOYLE: We ended up, particularly in
the TSPA-SR, they commented on it in a letter of
March 20th, 2000. We have this mix of where we've
incorporated uncertainties for some parts, did not for
other parts, and we've got this mix. The guidelines
that Peter is going to talk about I think will try --
will end up in a better situation.
Hopefully, at the end of the
implementation of those guidelines we won't have this
unknown mix of uncertainties. We may still have some,
you know, approximations and bounds in it, but
hopefully we'll have a better handle on it. And
that's what those eight bullets were supposed to get
at, and then Peter was to implement it.
MEMBER GARRICK: Okay.
MR. HINZE: John, can I ask a -- Bill, can
I get to your fifth column, a detail on your fifth
column, which is kind of an ominous title. On page 9,
you have, "New analyses may lead to reduction of the
probability of explosive, eruptive phenomena." What
analyses are these? Could you explain that a bit to
us?
MR. BOYLE: You know, I would have to --
I didn't --
MR. SWIFT: The question is -- it goes to
the type of volcanic eruption. Some volcanic
eruptions involve violent eruption and ash pushed
quite a long way into the atmosphere. And those are
the ones we're worried about. They're called violent
strombolean eruptions.
They're relatively rare in the geologic
record from Yucca Mountain, but not -- they're there.
But they're not the most common type, which are normal
strombolean eruptions, which produce a cinder cone
directly around the point of eruption and do not
produce ash blankets over a large area.
The question is: what fraction of our
eruptions are actually violent? And when does the
violent phase occur? Is it early in the eruption or
late in the eruption? If it's early in the eruption,
then that's the time we worry about. If it's late in
the eruption, the waste may already have been ejected
into a cinder cone close to the conduit rather than
being pushed out 20 kilometers.
For the SR and for all of the work you've
seen, we took the copout path of bounding it with the
assumption that our eruptions were, indeed, violent --
the strombolean ones. And so if we can justify a
basis for saying that some -- only, say, 10 percent,
20 percent, whatever -- we can justify a value, we'll
try to use that and produce our eruptive probability
that way, our probability of violent eruption.
MR. HINZE: Thanks.
MEMBER GARRICK: Any other questions for
Dr. Boyle before he sits down? Okay. Thank you.
MR. SWIFT: I wasn't completely prepared
for it in Pahrump when Bill introduced me as Odysseus.
I wasn't prepared for that. But it did occur to me
that at least one point was relevant, that Odysseus
had been on the road far too long and -- 22 years, was
it? And whether that was me or the project, I wasn't
quite sure. Also, it didn't have a happy ending
either.
(Laughter.)
So Odysseus is not the analog here. It's
Scylla and Charybdis that we're worried about.
You've got to start thinking about the
treatment of uncertainty with this question of
conservatism versus realism. And these are just some
simple observations here that -- many reviewers of our
TSPA have criticized a lack of realism. There's a
list of them, and this group is right there.
Obviously, there's a common theme. People
are looking for something we're not providing.
The second bullet here is my own
observation that I believe in general these reviewers,
when they review the TSPA and find a lack of realism,
they are in many cases not distinguishing between the
TSPA and the underlying process models. For them, the
TSPA is a window into the process models.
So if our process modelers make the
assumption that they will bound a solubility limit
within a range of uncertainty, we carry that forward
into the TSPA. And, yes, it's a lack of realism.
It's actually, I believe, a lack of realism in the
underlying process models.
This is appropriate. I think a good TSPA
should be a window into the underlying science. It
should be the first place you go to look to see how
well we understood something. But there are
differences, and it's worth keeping those in mind.
There are some places where we may have a more
realistic treatment at the underlying level, and for
good reasons have chosen to simplify it in the TSPA.
All of the reviewers' comments and
expectations with respect to realism -- there's a good
-- excellent summary of them in the Coppersmith and
McNish report that Bill just mentioned, Section 3.1.
But to me, I'm focused on what's in the rule, what has
the NRC asked for in the rule.
And this is the two clauses out of the
definition of "reasonable expectation" that basically
for me sum up the issue pretty well. And I think,
fortunately for the reviewers listed on the previous
slide, these two bullets actually do put the key
thoughts directly into the rule.
Characteristics of reasonable expectation
include -- do not exclude important parameters simply
because they are difficult to precisely quantify. And
this one focused on the full range of defensible and
reasonable parameter distributions, rather than only
upon extreme physical situations and parameter values.
These are the words out of the rule.
I actually take some heart in the site
softness of language here. It's not fully
prescriptive. It doesn't say focus exclusively on the
full range or only use a full range. Rather, it
suggests to me that we're looking for some common
sense here, but, clearly, the goal was -- the goal is
a full treatment of uncertainty.
So what's in our -- the guidance that we
came up with for the project? What we're looking for
is some version of a realistic analysis rather than a
bounding one. But what's admitted right up front,
some conservatisms will remain. Our job is to be
clear about where they are, what the basis is for
them, and what their impact is. There are cases where
the applicant, I believe, is going to end up being
conservative and explaining why and what -- how it
matters.
Focus on a realistic treatment of
uncertainty. That's not the same as a full
understanding of realistic performance. This is a
sticking point within the project. Realistic
treatment of uncertainty sometimes gets equated with
a full deterministic understanding of reality. And
the first here is achievable -- realistic treatment of
uncertainty. The full understanding of realistic
performance is not achievable. That would be the --
that would require 10,000 years yet.
So the bullets that go along with that for
me -- simplified models are okay in the TSPA. Broad
uncertainties are okay, if they're justified and
explained. This is important. Scientists generally
think of their job to be to reduce uncertainty. We
need a shift in mind-set here. Our job for TSPA is
not to reduce uncertainty; it's to make sure we've
adequately included it.
So broaden the range of uncertainty rather
than -- based on present knowledge, if you weren't
confident with the uncertainty bounds you've put in,
make them broader. If you weren't confident in them,
that meant they weren't broad enough. And then see if
they matter.
MEMBER GARRICK: I'm pleased to see that
there. That's a very important issue.
MR. SWIFT: These are just words so far.
We still have to implement these. But that thought --
the shifting of a scientist's mind away from 20 years
of experimental work driven to reduce uncertainty to
the simple statement "give me a broad uncertainty
amount," that's a difficult shift.
Scientists and PA analysts need to work
together to incorporate uncertainty in the TSPA. I'll
have more to say on that. But it -- it can't be done
by either the process scientist or the PA analyst
independently -- and focus on a clear explanation of
what we did, mathematical/conceptual descriptions. If
we're talking about parameter uncertainty, you'll
actually be able to see the equations in which the
parameter was implemented and the traceability.
That's something to strive for.
This thing called the guidelines document.
This is -- Bill described it as having been required
contractually by the DOE in a direction letter in
December. It was delivered on March 1st. It's a
rather dull document. I apologize. Guidelines for
developing a document and alternative conceptual
models, model abstractions, and parameter uncertainty
in TSPA.
It's, I say, dull because we don't want to
call it a procedure. We are not -- it's not a quality
assurance procedure in that sense, but it reads like
a procedure. I wish I knew how to fix that.
It describes the -- it meets the
requirements of the technical direction letter by
implementing the strategy outlined in the report Bill
described. It also addresses some NRC KTI agreements,
and the last page of this handout is the text of those
agreements.
The important thing here is that it uses
a team approach for both models, the alternative
conceptual models and the abstractions. And for the
parameters we set up a three-cornered team -- a
triangular team, with a lead for models, the
abstraction models. I use the same person as the
lead, he or she, same person does the lead for the
alternative conceptual model work. And a parameter
lead, and then a subject matter expert and a TSPA
analyst.
So think of it for parameters, where there
is one parameter team lead, but for each uncertain
parameter in the TSPA there will be a subject matter
expert and a TSPA analyst who -- the three of them
jointly have to agree on the distribution for that
parameter and actually sign off on it. Likewise, the
models.
The model abstractions -- the goal of
abstraction is to capture the important processes, the
processes that are important to system interactions,
and to make sure that the abstraction allows an
appropriate representation of uncertainty.
This is important. The abstraction is
going to use simplified parameters, often lumped
parameters, to capture quite a lot of things. They
have to built with an eye towards, can we actually
assign uncertainty -- representational uncertainty to
those parameters in a meaningful way?
The sections get developed by the subject
matter experts. These are for the scientists --
reviewed by the process model analysts. They're
developed in the scientists' reports. These are the
AMRs, the analysis and model reports.
There is no prescription on how to
actually do an abstraction, recognizing that they can
be everything from -- well, not listed -- you could
just put the full numerical model into the PA. You
could simplify it, simple functions, response
services, parameters.
The implementation in the TSPA gets back-
reviewed by a subject matter expert, and that
implementation gets documented in the TSPA's report.
For alternative conceptual models, there's
a little simple step-through process here that we're
asking our model developers to walk through. For each
process of interest in product alternatives, if any,
with consistent available information, there's no
requirement here to go out and make up alternatives.
In fact, there aren't any that are consistent
available information.
If only one conceptual model is consistent
with all the information, that's good. That means
you've -- you don't have a valid viable alternative to
conceptual models. Instead, you have things that can
be screened out. And you document that at that point.
That basically is part of our FEP screening process.
Things like, for example, seismic rises in the water
table that might flood the repository is not an
alternative conceptual model because it is not
consistent with available information. We believe
that can be ruled out.
If you have multiple viable alternative
conceptual models, evaluate their impacts on subsystem
and component performance. That's the process model
or the specialist in that area. If there are
alternatives, if the alternatives result in the same
subsystem performance, i.e. the same information that
you delivered to the system model, then, again,
alternative conceptual model uncertainty is not a
significant source of uncertainty in the total
analysis. Doesn't matter which alternative we use,
we're getting the same result out of it.
If two or more show different subsystem
performance, develop abstractions for both and deliver
them to TSPA. That takes you back into the
abstraction process. Basically, have them reviewed by
TSPA and implemented.
Here's a point. If the abstractions for
the alternatives are not straightforward, this is a
place where I think you're going to see some
conservative choices come in. I don't really have an
example in my head, but some -- let's suppose somebody
proposes an alternative conceptual model which would
show improved performance but is going to be a heck of
a chore to abstract it into the TSPA. Perhaps the
example I gave earlier of metrics diffusion in the
unsaturated zone might be one.
This is a place where I think the project
will probably take the cost effective approach and
explain why they're being conservative.
TSPA evaluates --
CHAIRMAN HORNBERGER: Peter?
MR. SWIFT: Yes.
CHAIRMAN HORNBERGER: On that point, it
strikes me that what you're -- if what you're saying
is that you have alternative conceptual models that
are consistent with information, then you don't have
a clear way to choose one over the other.
MR. SWIFT: Right. Yes.
CHAIRMAN HORNBERGER: And it strikes me
that all you're saying is that, fine, if they give
different performance we will use the one that shows
the worst performance.
MR. SWIFT: Yes. Well --
CHAIRMAN HORNBERGER: Is that right?
MR. SWIFT: That will do. The question
is, at what level do they show the worst performance?
If they're showing different performance at the
subsystem level, that isn't -- doesn't for sure mean
they're going to show different performance at the
system level. But, yes, other than that I -- same as
what you just said. But the idea is to actually
direct people who document this process of thinking.
MEMBER GARRICK: It seems there's kind of
a corollary rule here that would apply, too, and that
is that if you have multiple conceptual models -- and
let's say that those models provide the same results
-- then you ought to use the simplest model as the
basis. This is the Copenhagen rule for the great
physicist.
CHAIRMAN HORNBERGER: It actually precedes
Copenhagen, because it's William of Ockham in I
believe it was 1674.
(Laughter.)
MEMBER GARRICK: Well, Niels Bohr picked
it up and --
(Laughter.)
-- made the point very elegantly that if
you have multiple theories, and they give you the same
results, we're going to, by damn, take the simplest
one.
MR. SWIFT: The problem is when they don't
give you the same results.
MEMBER GARRICK: Yes, I understand. But
there is that issue that there is a tendency sometimes
for modelers to want to impress you with the
complexity rather than impress you with the
simplicity.
MR. SWIFT: If the two models give the
same subsystem result, my conclusion, alternative
conceptual model uncertainty is not significant. The
under bullet not stated there is that the subject
matter expert then has to document that as to, yes, I
have these multiple alternatives. They all give me
the same result. Therefore, I'm only going to deliver
the simplest one, or the one of their choosing,
forward in the TSPA.
MEMBER GARRICK: Yes.
MR. SWIFT: And that actually is in the
guidance document. That step is there.
VICE CHAIRMAN WYMER: Now, this is turning
out to be a lot of extra work, but -- and looking at
various parameters. But what you haven't said is how
the parameters should be looked at.
MR. SWIFT: Let me get to that on the next
slide when I talk about parameters. Let's imagine
here that the alternative conceptual models are
implemented in TSPA and you actually run a full TSPA
or a subset of TSPA with the different alternatives in
it.
And if the options -- the impacts are
significant, then the options are -- there are
basically two options. One is you can carry the
multiple alternatives all the way through to the
regulatory dose, but then you have to weight the
alternatives, and then you have to be able to defend
those weightings in some way.
So that may not be the -- the first simple
thing, you always give them equal weight, but if they
-- and see if it makes a difference. If they don't
make a difference, then you learn something. If you
can't defend weights, then at that point, again, you
default to the more conservative one if you've gone
through this.
Parameters --
VICE CHAIRMAN WYMER: I'd say selecting a
parameter value is not the same as selecting --
MR. SWIFT: Yes, I'm aware of that. The
assumption here that you just caught me on is we
already know which parameters matter. And the
identification of we're actually -- this is the step
in the process that we're in right now. It's
identifying the parameters we want to treat as
uncertainty parameters in the TSPA.
And we're doing that by -- the TSPA team
is providing a list of the parameters that were
treated as uncertain parameters in previous analyses
back to the subject matter experts in each of their
areas for review and updating.
There are parameters that have been
treated as uncertain parameters in past analyses that
actually aren't doing very much in the analysis. The
analysis would be insensitive to the uncertainty in
them. If that uncertainty was appropriate, you know,
justifiable, defensible, and still was doing nothing,
then that parameter might be a candidate for one to be
switched to a fixed value.
If, on the other hand, the subject matter
expert looks at that list of uncertain parameters and
says, "Whoa. Here's the one that really captures the
process. Better put a distribution on that and get it
in there," that will happen. But the real answer to
your question is that this is -- it's human judgment,
and this is why an iterative analysis better, because
people learn things through time as to which sources
of uncertainty matter at the system level.
We've learned a lot in 10 years of TSPA
and interacting with the process model teams. I
actually do think that we have the right uncertain
parameters, probably more of them than we need, and
there isn't a unique test to make sure you've gotten
them all. That's, from a judgment, an iteration and
review.
But once you've got the list of uncertain
parameters identified, categorized, they get mapped
back to the subject matter experts for documentation
in their AMRs. And the full range of defensible and
reasonable distributions gets documented by the
subject matter experts in their AMRs in that
triangular model with the team lead and an analyst.
There are two things yet to consider in
building uncertainty distribution. First is the
available data. But, second, and this is the part
that typically gets missed, you have to think of how
the parameter is used in the model. Model scaling
issues, what's the cell size in the model. These are
numerical models, and it makes little sense to use
porosity data collected at a sidewall core this big,
use that exactly as is in a model where you might have
cell blocks hundreds of meters on the side. The
distribution is something different.
So think of spatial variability, which is
the example I was offering there, because it affects
the scaling of the parameter, how it's used in the
model. This is the point where you want the modeler,
the person who actually knows what the parameter is
doing in the equation in the model, working with the
subject matter expert most familiar with the data,
working with the team lead who is -- will have -- the
statistician who is supporting them in how to apply
the -- how to build a defensible distribution from the
available data.
And it's typically not a matter of fitting
it with a normal or log normal or some specified
model, because nature doesn't work in statistics like
that. What you want is -- what we want is a
distribution function that doesn't add any new
information, honors information we have, doesn't
create new knowledge.
So the simplest example of such would be
a peacefulized linear distribution. If you actually
thought data itself was appropriate to be used in the
model as was -- as is -- a peacefulized linear fit for
the data is better than trying to force fit a normal
or log normal distribution.
But the distribution is -- ultimately,
it's a subjective decision, and you want it made by
the right experts -- the scientist, the PA modeler,
and a statistician with experience in doing that. And
then you want it documented, and so we'll do that, and
then implement things through a controlled database.
MR. BOYLE: And also, I think in the way
this system is set up, it's the consideration of
alternative models frequently gets at which parameters
are under consideration. For example, if your model
is that the rock is elastic, well, Young's module and
Poisson's ratio is sufficient. If, on the other hand,
you assume that it is viscal plastic, well, then, that
generates a whole new set of parameters for which you
then need values.
MR. SWIFT: The last slide here, this one
actually got edited a little bit in the final review,
and you'll get a kick out of what came out of it here.
This bullet used to say that regulators and reviewers
are not asking for the impossible, and someone felt
that was a little too negative.
(Laughter.)
But I actually believe it. And I'm
getting back to the idea there that if we, the DOE,
were to misinterpret what you're asking for as -- I
hope I'm going to get a head nod here. If we were to
misinterpret that as asking for a full, realistic,
deterministic solution to the future, that is
impossible. We're not going to do it.
But we can commit to a realistic treatment
of uncertainty. Can we actually achieve it? It will
be some version of it, but there will --
pragmatically, there will be conservatisms here and
there. And it's our job to explain what we did.
And this is the last point -- there's no
unique solutions. A lot of credibility comes from how
well we can explain it.
And that's it for that presentation.
MEMBER GARRICK: Before we go to the next,
any comments from members of the committee? Milt?
MEMBER LEVENSON: Well, I'd like to make
one comment. It's sort of a follow-on to the comment
John made earlier, and that is that it's not of major
importance that we reduce uncertainty. The importance
of uncertainty is only to make sure that the true
extent of some risk is not obscure, and that, as in
medical work, false positives are equally to be
treated as false negatives. And the whole objective
is to make sure that you know enough so that you can
evaluate the risk. Now, reducing uncertainty per se
is certainly not an objective of mine.
MR. SWIFT: Others see things differently.
I agree with you completely, but the -- on the
alternative conceptual model side, for example, a
question I got from the ATRB was, well, where is the
step where you go out and design an experimental
program to go back and test those models and throw one
or the other of them out?
And that isn't where I actually was
thinking. I was thinking we're going to make a
decision based on information that we have now. We're
going to decide if the uncertainty matters. And it's
a different way of thinking of things.
MEMBER GARRICK: Ray?
VICE CHAIRMAN WYMER: Well, actually,
you're not going to make a decision based on
information -- based on what you have between now and
the time of your license application. No, I've raised
my questions already.
MEMBER GARRICK: George?
CHAIRMAN HORNBERGER: Again, just a
comment following on what you've just said, Peter. I
think in part, as I read some of the TRB comments, it
would be that the question is whether or not there is
an adequate scientific base to support the models that
you have.
MEMBER GARRICK: Yes. And I like the --
your remarks about -- that the distributions or the
uncertainties need to be driven by the information or
the data. Too often we've seen people spending a
great deal of time and effort and exercise on trying
to choose a distribution that will work for them in
their model.
And there's enough analytical tools
available now that there's no reason for doing that.
We ought to be able to forget about whether it's log
normal or beta or gamma or whatever, and let the
information, however it comes out as a distribution,
be the basis of the model.
And even if it's a histogram, because
there are tools now that very effectively convolute
discrete probability distributions. And that is even
a more -- often a more accurate representation of
what's taking place, and very often much easier to
follow.
So these are encouraging signals that
you're giving in the context of guidance. I think
it's very important. And I think it also addresses
this whole issue of the confusion that sometimes
exists between good science and adequate science with
respect to solving a problem, and we've talked about
that a lot in this committee.
We want good science, but we don't think
it's necessary to reduce an uncertainty between, say,
10-12 and 10-7, even though it's five orders of
magnitude, if the risk is of the order of 10-3. So
that's adequate, even with that wide amount of
uncertainty. So these are steps that are very
encouraging to us.
Any questions from the staff? Yes.
MR. HAMDAN: Uncertainty means different
things to different people. Milt and I and perhaps
everybody in this room understands uncertainty as has
been described, and Milt described it very well. And
on slide 11 from your first presentation --
MR. SWIFT: In my first presentation?
MR. HAMDAN: Yes. Which igneous activity
-- you showed us the slide about the effect of igneous
activity -- if you want to -- this was what obscured
uncertainty. This is very clear. It goes to the
point and evaluates the effects of igneous activity of
a property of one. So this has nothing to do with
uncertainty.
But to the public and people who are on
the street, this is the uncertainty. You are saying
in this slide to them that if -- it were -- this is
the point that you are going to give to them, not to
us in this room. We evaluate the risk and then we
make a recommendation based on risk.
But to the people on the street, this is
uncertainty. And I think this needs to be looked at
and responded to and articulated to the public. So
this is the comment that I make.
I have another question for Peter, and
that is on your slide on the parameter uncertainty.
You probably will need to pull that out. The approach
is fine, and you have articulated it very well. The
real questions come with the -- when you want to
assign probability distribution to a certain
parameter, that's where the rubber meets the road.
There sometimes you don't have enough data
to select a distribution, and that's where the problem
lies. There are a lot of parameters with a
distribution that have validity bases, and I wonder if
you could extend your answers from that -- for
conceptual models to do alternative distributions for
these parameters, and satisfy yourself that, really,
it does not make a difference.
And that will probably be needed because
there is simply a lot of parameters with uncertainty
for which we do not know what the right distribution
is.
MR. SWIFT: My own experience in analyses
like this is that for parameters to which the results
are sensitive, the form of the distribution is less
important than the range. What you're worried about
are the impacts of the tails.
And so the difference between a log
uniform distribution and a log normal distribution
will not be that great. The difference between a log
uniform and uniform distribution may be very
important, but I think picking distributions that span
a broad enough range of uncertainty that the range
itself is defensible -- just what the scientists
believe is a broad enough range -- will take quite a
lot of the concern off the shape of the distribution,
the actual form or function used to fit it.
MEMBER GARRICK: I see this next
presentation is -- yes?
MR. HINZE: Well, I guess wanted to
suggest that the most important, the most critical
phase of this whole thing is making the decision on
when you have done sufficient sites that you have a
model that you can do some calculations with. It's
very hard to put uncertainties on that.
It reminds me a bit of the ore deposit at
Roxby Downs in south central Australia, which is the
most important mineral discovery since the Second
World War. It was found as a result of a very
incorrect model. The answer was just beautiful. It's
correct, but it was a totally incorrect model.
Now, if you use that incorrect model,
which some people have tried to use, in other parts of
the world to find a similar ore deposit, you're just
not going to get there. It seems to me that you can
put parameters -- the uncertainty around these
parameters, but it's the question of when you've done
sufficient science that you understand the process
well enough so that you can make a judgment, and that
judgment will have uncertainties.
MEMBER GARRICK: That's correct. Okay.
I see this next presentation is a big one.
MR. SWIFT: It's not as big as it looks.
I'll give you the fast version.
MEMBER GARRICK: We would like to hold, as
best we can, to our 12:30 recess.
MR. SWIFT: Me, too.
So I will certainly not go through all of
these. I will give --
MEMBER GARRICK: I have an advantage. I
can ask all the questions and extend the time, and
then blame you --
(Laughter.)
-- blame you for overrunning.
MR. SWIFT: Can we have the lights here?
So we can see the screen better. Good. Thank you.
Since December, the project has gone
through a replanning exercise. You'll recall that in
-- starting in the summer of 2001, the project went
through a planning exercise for a multi-year plan
prepared by the M&O contractor, BSC, and then to be
approved by the DOE, for the work to be done to
support a license application.
And the plan, which was submitted in
September, produced a large body of work that was --
scientific work that went out to about -- an
application in about 2006. The dates were somewhat
flexible. But the DOE came back to BSC and said,
"Perhaps you want to replan."
There was not a prescribed date. However,
we felt it was prudent to replan with the idea of 2004
in mind to see if we could, in fact, identify a scope
of work that would allow us to produce a docketable
license application in 2004. Of course, docketable is
the NRC's decision, not ours, but we wouldn't submit
an application unless we felt there was a reasonably
good likelihood that it would be docketed.
So with that in mind, we set out to
prioritize work in the performance assessment and
science activities, which we realigned the project at
the same time, so that the science became part of the
performance assessment project, and focused primarily
on the work that was necessary for license
application, identify and select an overall scope of
work to balance the project management risks. This is
not human dose risk. This is the management risk.
What's our risk of success or failure? And document
it.
And this, then, would be the basis for the
replan that was delivered to DOE March 1st. Back in
December, we were planning ahead. In fact, we did --
BSC did deliver a new multi-year plan to DOE on
March 1st, and that is in DOE review.
And that has not been released yet. Is
that correct? The plan B of -- anyway, it's a big fat
thing, work plans for the outyears, starting
immediately but out to 2004.
And inform these decisions with input from
the TSPA analyses, technical staff working with the
science program staff, line management, project
management -- this will be the senior management team
-- and then project planning. These are the people
who ultimately -- they're the ones that have to
prepare a multi-year plan.
In theory, it's -- no, more than theory,
it is a resource-loaded schedule where you can
actually point to a schedule and see what things cost,
how long they take, and what they do. And that's the
-- so the process we wanted to go through here, just
for -- this is -- remember, we're only prioritizing
work within performance assessment and science.
This does not include design activities.
This does not include licensing activities, quality
assurance activities, the various support activities.
It's only a portion of BSC's budget that was developed
in such studies.
The PA team identified attributes --
basically, a short story here -- we're headed for a
multi-attribute utility analysis. And we've done it.
That's where this clarification is headed.
The PA team defined the attributes at
which the work scope was evaluated. The department
managers of the science departments defined work
scopes to be considered for each model, component.
Think of the unsaturated zone or the saturated zone,
and so on. For each model component, they defined
alternative work scopes they wanted to have
considered. For each of those they should have
estimates of cost and time.
And the department managers and the TSPA
modelers provided initial estimates of the impact of
the proposed work on the attributes. Basically, an
attributes questionnaire. We scored each work scope
description.
There were 25 model components, and each
one had about -- almost each one had three work
scopes, so about 75 different work scope descriptions
were scored against these attributes. And that was
actually done in a workshop in January where we had
the key players all together in a room for three days
and went through scoring the -- first of all, we wrote
final work scope descriptions, we scored them against
the attributes, and I'll go through how we -- what
that means here in a minute.
And then we ran them through the utility
analysis tool that I'll describe in a minute, produced
an initial prioritization, in mid-January had a
management review of it, and provided input to our
budget team at the end of January. And that, in turn,
has gone on to DOE now.
This is a conceptual figure that -- it's
important because this came out of a BSC management
meeting in November -- the idea that, since most of us
think best in only three dimensions, let's find --
think of it in three dimensions. What are the things
that matter to us in making decisions about what
science activities we do on the project?
And we came up with three axes -- a
quantitative performance axis. What is our calculated
total annual dose? And what work are we doing that
moves it up or down? This is, you know, basically,
are we in compliance with 63/113?
Regulatory defensibility and acceptability
-- in a regulatory framework, can we defend the models
and data used to calculate that dose? Have we met the
qualitative requirements of Part 63? This axis is
Part 63 and 197, as implemented through 63.
So things like multiple barrier
requirements to qualitative, descriptive requirement,
but that would live on this so-called X-axis.
Satisfying KTI agreements -- the NRC has
given us a list of what needs to be done to defend the
models. We, in some form or another, need to address
those agreements and produce defensible models. Then
-- and we need, say, for example, quality assurance
requirements, our so-called X-axis attribute.
Then there are the Z -- what we call the
Z-axis out this way. This was a -- Y was up in this
coordinate system when it first appeared on our white
board. So the Z-axis here, qualitative acceptability,
internal and external defensibility, these are issues
that we know we care about them, yet you can't trace
them to anything that's in the rule.
So some of these are -- some of them are
actually quantitative as well as qualitative. But
qualitative things -- defensibility of models, beyond
what's needed for a regulatory framework, the question
of, can we convince people we actually understand the
system well?
Many of the Technical Review Board's
concerns are on this axis, not all of them. I'll
argue that some of the NRC staff and center's concerns
may be on this axis. They're valid. This is not to
say the Z-axis is not important, but there are
technical issues that don't tie directly to the rule.
For a quantitative one, an easy one to
think of is peak dose. There's no regulatory limit
applied to peak dose occurring several hundreds of
thousands of years out. At the peak dose is a
quantitative Z-axis attribute.
So it's not three-dimensional space, and
those axes aren't orthogonal. And they're certainly
not mutually exclusive. We decided to define it as a
16-dimensional space for the purpose of the utility
analysis, and nobody can think in that, but the
spreadsheet does.
There are 16 attributes here that can be
coarsely lumped against those three axes, but, in
fact, for the utility analysis we scored things on
each one of these attributes without considering those
at X-, Y-, Z-axis. That's just there as a
communication tool for our own management team.
For each work scope, we went to the
technical staff and said, "Will your work, if you do
your Level 1, 2, or 3 scope" -- and I'll explain what
those are in a second here -- "will that change
10,000-year mean annual dose?" Which, by the way,
that is driven entirely by the volcano. That's the
10,000-year total. That's the Part 113 dose.
Will it change groundwater concentrations
or human intrusion? And that's it for quantitative
performance.
Regulatory defensibility -- have we
captured all credible FEPs? Have we excluded the ones
that can be excluded according to the criteria? Are
we meeting our requirements to describe -- identify
and describe multiple barriers? And do they link to
specific KTI agreements?
The so-called Z-axis sorts of attributes
impact on conference of internal reviewers, impact on
conference of external reviewers, and some
quantitative ones have come out of the TSPA
calculations. Change in time to 15 millirem. Change
in uncertainty. This would be the distribution spread
from the 9th and 5th, for example. There's no
regulatory driver for that. It's the mean we're
regulating on, but we do care about that spread in the
uncertainty and system outputs.
We looked at a forced early failure case,
peak dose, and this -- I should say associated with
conditional igneous intrusion. We had a question
there about, will your work affect our conditional
igneous dose? Will your work affect a representation
of uncertainty at the parameter level? And our
ability to defend the conceptual models.
The actual questions themselves are shown
in the handout. I have time to show those a little
bit. This goes pretty quick here.
For each model component -- there are 25
of them I'll show up here in a minute -- department
managers define three levels of work. The expectation
was that this would be an increase in cost and/or
time. Level 1 would be the quickest and cheapest.
Level 3 would be the longest and most expensive.
Level 1 -- what work would be required to
complete quality assurance issues and to validate the
existing models? That's not focusing on not
developing new models, but meeting our own internal
validation requirements for the models that we used in
the analyses I showed an hour ago -- the most recent
set of PA models, which are not -- in our own
terminology, they are not qualified and validated yet.
Level 2 scope -- take a so-called risk-
informed approach to going beyond Level 1. Risk-
informed in this sense means to us look at the impact
of the work before you decide if you're going to do
it, and, in particular, this might involve taking PA-
based -- system-level, performance-based approaches to
resolving KTI agreements rather than the literal full
scope of work that was anticipated when we agreed to
the agreement.
In other words, if we can show it doesn't
matter, is that a sufficient way to address a
question?
Level 3 -- and these are both optional.
We went to the work package managers and said, "If you
can close everything at Level 1, don't bother to
define any higher levels for us." Level 3 was
essentially the same as that plan A work scope that
got us to 2006. And with respect to KTI agreements,
it was the full and literal completion of all
activities proposed.
Managers were -- these are the science
managers, department managers, provided input on how
well each proposed work scope meets the defined set of
attribute scores with respect to the defining set of
attributes -- better way of saying that.
And the -- I'm going to skip a couple of
slides here and just go to slide 10, because the --
the same word that's on the intervening slide, where
you've got this helpful equation here. Ignore the
figures. They're not all that helpful.
But for each one of these 16 attributes,
we asked questions of the technical staff, how likely
is your work at this level -- scope of work -- how
likely is it to, for example, increase confidence in
your treatment of parameter uncertainty? How likely
is it to result in an increase in total dose? And the
technical staff provided answers that ranged from very
unlikely to very likely.
We then asked a management team what types
of -- what value they assigned to different types of
answers. That's this V thing here. Actually, it's a
relatively small player in the utility analysis. But
since it's up there -- if someone said their work was
likely to -- or, let's say, it was likely to result in
a change in dose, we then asked the TSPA modelers,
"How big a change might that produce?" These are all
subjective answers, but at least we're asking the
right experts.
And the TSPA modeler might say, "Oh, it
could increase by a factor of greater than 10, or
could have a small change of less than a factor of
10." We wanted to apply -- but this now is a
management decision, what value do you apply to the
different answers.
In this hypothetical example here, we gave
that a weight of one and a weight of zero for a
neutral effect, a small change in dose, and a weight
of .15 for an increase in dose. But that's how the
impact value function was used.
Then this weighting -- this is a
subjective management decision. How important did
management think that question was? And these were
elicited from the management team in the project.
If I go back to that three-dimensional --
you know, this figure -- if our technical staff were
all-knowing, for any scope of work they actually
could, in theory, define a vector in this N-
dimensional space that defined where their work would
put us. Would their work, you know, greatly increase
qualitative defensibility? Would it greatly increase
regulatory defensibility? And so on.
That would be the first term of that
three-term sum that went into the utility analysis.
The other two are management questions of, where does
the project want to be in that -- in this
N-dimensional space? If the project did not care
about qualitative acceptability, external reviewer
type issues, we could truncate the Z-axis and live
entirely in the X/Y plane. It appears to be the bare
minimum needed for licensing under Part 63.
But the project is not willing to accept
that risk, and, you know, the TRB certainly
understands that point. I met with them -- a subgroup
of them, just a couple of days ago, and we talked
about this. Obviously, we're not going to live only
on the X/Y plane.
So where are our decisions headed? And
that's the point of the -- listing the management
weight. And for each of the 16 attributes, for each
of the work scopes, you can define -- we define a
likelihood that the answer will be what we -- what --
that the likelihood of a specified answer will occur.
That comes from the technical staff, the scientists.
And these values and weights comes from managers.
And then, for each work scope you create
a spreadsheet and sum them up. And the -- you get a
utility. It's a dimensionless number. It's
associated with each work scope, and it's a
quantitative -- and fully comparable from one work
scope to the next -- measure of the utility of doing
that piece of work, utility defined in respect to the
questions we asked, what are the attributes, and the
weights management put on those attributes.
Now, caveats that come out of this --
first of all, like any decision analysis exercise, the
results of the model here means the utility model,
utility analysis. It's a decision-aiding tool. The
project had no intention of using it as a direct
decisionmaker. Other -- that's the most important
caveat. It's to inform managers who still have to
make subjective human judgments.
The cost assumptions that went into it
were not always consistent. Results -- we
deliberately did not constrain them by schedule. We
didn't force people to say, "You've got to make
everything end by 2004." There were some differences
among department perspectives and the impacts of their
work, despite the workshop discussions. Surprisingly
few, actually. That workshop was as close to a
consensus as I've seen when people understood what it
was we were doing.
It doesn't include all work scopes. It's
just a -- for those who are looking for the piece of
the management budget or design or testing the
interface with design or the TSPA calculations
themselves. We excluded those from the exercise.
Some questions didn't capture what we were
after. We wrote some bum questions. That happens.
Utility rankings -- we presented utility-
only rankings, and they're in the packet here, and
also utility cost ratio rankings. Utility-only
rankings ignore cost. Utility cost ratios are better.
They're clearly what the tool was designed for.
You're doing a -- we are doing a utility analysis
because cost does matter. We don't have an unlimited
budget.
But when you do a utility cost ratio, you
discover pretty quickly that not all work costs the
same. And very large work packages may perform more
poorly in the evaluation, simply because they've got
a big denominator -- cost -- where, obviously, if
you're a utility people defined their, aggregated it
coarsely, and produced expensive packages. You get a
big denominator in that fraction.
The examples here in the packet are
weights, management decision weights from two people
-- Bob Andrews and myself. In fact, in the back, in
the backup slides, there are different rankings
provided with weights listed from other groups of
people. And in the report that went with this, the
people are identified by name. They were the BSC
management team. We also listed some DOE managers but
did not include their results. That would have been
inappropriate.
What we discovered -- you can see when you
get to the back -- is that actually the management
weights, even though we had some -- quite a broad
range of management types in the exercise, we were not
as sensitive to the management weights as you might
think.
And this last caveat here, that the --
deliberately, we didn't want to emphasize -- didn't
want to focus only on things that show a positive
benefit or a negative impact either way. Both of them
are important. Obviously, we need to know if our work
is going to show poor performance. That's -- we must
know that. But we also want to value work that shows
improved performance.
MEMBER GARRICK: I think we're going to
have to wrap up in about five minutes.
MR. SWIFT: Okay. I'm there.
Just an example here, a couple of
examples. Thank you, Bill. I apologize for this.
These were three different levels of work
defined for engineered barrier system flow and
transport. The way to read this figure -- the bars
here are the amount of utility associated with that
activity for each one of these 16 attributes over
here.
And the first thing that -- we have the
big blue band here -- resolution and closure of KTI
issues. So the experts in the engineered barrier
system department felt that if they did more work, the
Level 2 work, they had a better likelihood of closing
their KTI agreements. And the band is thicker here
than it is here. But they were still -- I believe
this was a likely answer, and that was very likely.
Interestingly, although these two work
scopes are very different, they show an absolute --
this one costs $2 million more, takes two years
longer, and in the space of these questions that we
asked gives you the same answer. So that's an example
where the management decision was pretty much a no-
brainer. We looked at the one that gives you the same
answer more cheaply.
CHAIRMAN HORNBERGER: Peter, you'll have
to forgive Raymond and I. We have another meeting we
have to go to.
VICE CHAIRMAN WYMER: I apologize.
MEMBER GARRICK: That's all right. We'll
carry on here for a while.
MR. SWIFT: The work scopes are sorted by
utility at the Level 2. This was the so-called risk-
informed work scope, the intermediate work scope for
each of these areas. Biosphere scored highest. This
is simply because -- a number of reasons, but
biosphere has a high likelihood of affecting the total
10,000-year dose.
And in the management weighting used to
generate this figure, Bob Andrews and I both felt that
anything that was not moved -- the 10,000-year total
dose was the most important thing. We gave that a
very high weighting, and the biosphere igneous
activity was --
MEMBER GARRICK: Peter, let me comment on
that. Of all the things on here, the biosphere is
probably the most prescriptive. And so -- and one of
the things that prescription does is very often
eliminate a lot of decisionmaking and analysis,
because this -- the biosphere -- the regulations on
the biosphere are pretty binding in terms of how much
flexibility you have in analysis and investigation.
I'm surprised that that would end up on top.
MR. SWIFT: Well, this is a -- let me go
to the next -- a different slide. This one is in the
backups, and it's -- I'll come back to answering that
question.
MEMBER GARRICK: Yes. Okay.
MR. SWIFT: This is incremental utility.
This is --
MEMBER GARRICK: Yes, I understand that.
Yes.
MR. SWIFT: -- how much more you're
getting when you -- how much more utility you get when
you go from Level 1 to Level 2. And biosphere has
dropped well down the list here.
MEMBER GARRICK: Yes.
MR. SWIFT: The difference there is that
our biosphere team felt that even at their lowest
level of work they were likely to show an increase in
the BDCFs. And that's what drove that --
MEMBER GARRICK: I see. Okay.
MR. SWIFT: And I actually -- this is all
subjective. This is judging by work that is not yet
done, and I tried to argue with Tony Smith about this
one in particular. I don't think shifts are going up
as much as he thought they were, but he's the
technical department lead on that. And that's where
his work fell out.
That was the -- did I just put up the
incremental utility plot? I did. Okay. I'm looking
for my summaries here.
MEMBER GARRICK: Are you going to tell us
how this has affected the outcome of the decisions?
MR. SWIFT: Yes, if I can find the slide.
Yes, that was my mistake. They're over here now.
It's actually 22 and 23.
What did we do with this? We brought the
spreadsheet as an electronic tool, and the types of
rankings you see. We brought them to a BSC management
workshop where we had the senior project manager of
the BSC, not the corporate management, but Nancy
Williams, who is the project manager, and her staff --
I could name who they are. She had a -- has an
oversight -- project oversight review board. No, it's
just called project oversight board, POB, that is
people you're familiar with.
It would be Jack Bailey, John Beckman,
Gene Yonker, Nancy Williams herself chairing it.
Representatives from the national laboratories would
be Roland Carson, Joe Farmer, Andrew Worrell, Sal
Peterman from USGS, Tom Cotton, who is on it. These
people met for a fairly intensive three-day meeting to
go through the results of this spreadsheet, quite a
lot of detail.
And then we put the final rankings up on
the screen -- not final, we put the various versions
of the rankings up on the screen. They've been up off
and on for a long time, with the cost utility ratios
displayed and total cost. And we started drawing
lines in the budget where -- what can we afford? And
that would be an example that would be -- slide 20,
for example.
These are -- we do Level 1 scope for each
of them. This is what it costs in FY '02, and then we
start adding in more work at the -- from Levels 2 or
3 at the sort of -- the highest increments of utility
cost ratios first. So the most bang for the buck
principle here.
What did we discover when we started
drawing the line in the budget? That there was not as
much money available as we had happened, and we were
able to come to the conclusion fairly quickly that, in
fact, the Level 1 scope was where we were looking.
The emphasis was going to be, based on the money
available, on validating the models that were already
available. These would be the models we showed
earlier today.
However, we could not afford to move up to
Level 2 and 3 across the board, so we started taking
those work packages apart item by item. And the
management team actually, in real time, went through
all of the work package descriptions and brought
things forward from Level 2 to Level 3 into the budget
on subject human judgment bases.
A primary selection criteria of moving
work forward was to avoid canceling any tests that
were ongoing. One of the lessons learned from other
projects is: don't cancel a test if you've already
paid for startup costs. Go ahead and collect the
data.
So this is just -- this is not all PSF
tests. Some have PSF tests -- some tests are brought
forward, and other examples of testing activities were
brought forward.
And activities that were needed had to be
accelerated to support documentation activities that
could be done. Activities that were at the Level 1
scope of work but were planned to be done too late to
support license application in '04 were accelerated,
and that required bringing extra money forward.
Basically, this was a money management
tool that they exercised about how you use your money
wisely to manage your work.
And then this exercise with project
management took place in late January, and early
February we spent detailing work package descriptions
that were then delivered to DOE on March 1st.
And I apologize for running over. I have
a summary slide here. This was a decision -- the
multi-attribute utility analysis was a decision-aiding
tool, not a decisionmaking tool. We keep saying that
because that last slide included a lot of human
judgment.
You want both the technical and the
management input. The management weights are
important. That's where we decide what it is -- where
we want to be in that X/Y/Z space.
Consideration given to regulatory
requirements, technical defensibility, and money.
And, yes, we will have to reevaluate it as new
information becomes available.
MEMBER GARRICK: Okay. Thank you.
Milt, do you have any comments?
MEMBER LEVENSON: No.
MEMBER GARRICK: Appreciate the
presentations. I think one of the real problems when
you get into this business of trying to come up with
utility functions that contain preference functions is
dealing with the different groups as you have, and
addressing the biases that might exist in those
groups, because all of us think that what we're doing
is the most important. So there has to be some sort
of normalization process.
But you said that this seemed to be --
there seemed to be a lot of harmony in this case. I'm
surprised at that.
MR. SWIFT: There was. The trick I think
was to focus people on giving fair questions -- fair
answers to questions as we asked them. When -- if you
simply say, "Is my work important?" ask somebody to
answer that question, the answer would always be yes.
But if you say, "Will this specific piece of work
change a dose result?" or "Will this close KTI
agreements? And, if so, please name them and explain
how you're going to close them." At that level,
people were quite objective, and they were willing to
say, "No. Actually, this doesn't do anything to
dose."
We had a broad enough set of questions --
MEMBER GARRICK: Were they thinking in the
context of uncertainty when they answered the
question? Because --
MR. SWIFT: Not as much as I had hoped
they would be.
MEMBER GARRICK: Because the risk is
really the uncertainty. And so if what they're saying
is that it doesn't affect the central tendency
parameter, that's one thing. But if it does affect
the tails of the distribution, it could be very
significant.
MR. SWIFT: With respect to the dose
calculations, yes, they were -- they were thinking of
that sort of thing. But they were all thrown for a
curve -- thrown a curve right away by the realization
that 10,000-year total dose is really a question of
igneous activity.
And as soon as people realize that, you
know, if you want to score on the Y-axis, the
quantitative axis in that, you've got to have a
volcano.
MEMBER GARRICK: Yes.
MR. SWIFT: A whole lot of people came
into the room thinking they were going to say,
absolutely, I've got some tail up there that's going
to drive dose. Actually, no, you don't. You may
score on time, 215 millirem. You may score on peak
dose. You may score on the conditional early failure
scenario. But those all get different weights.
And because we had a broad range of
questions, I think every technical staff person was
able to feel like, yes, there is a question that
captures my -- their personal issues. But then, they
didn't know how management was going to weight those
questions, and so they were -- the technical level of
agreement was surprisingly high. People always found
a question they could say, "Yes, that's the one I'm
aiming at."
MEMBER GARRICK: Any questions from the
staff?
One thing I would say is that in the TSPA-
SR you had a couple of appendices that did a very nice
job of delineating the key assumptions, and then going
out on a limb a little bit and indicating what the
impact of these assumptions might be.
I don't know if it's the way you're doing
it, but I think that it would be very helpful with
respect to traceability and transparency for this to
be kind of a reference point and subsequent versions
be measured against this reference point. I think it
would make it very clear what -- which assumptions
have changed and what impact they've had, and which
assumptions are being driven by the decision to go to
an entirely different corrosion model, for example.
By just changing the model, you can end up
with a different set of importance rankings for
contributors. So I found that what you did in the SR-
TSPA very valuable in boiling down just exactly what
the team thinks is important from an assumption set
standpoint.
And I hope that something like that is
carried forward. I'm not saying you should do it,
because we don't advise you; we advise the Commission.
But that was -- I'm just observing that that was an
example of a transparency tool or a traceability tool
that was very helpful. And your presentations were
very helpful, and we thank you very much.
And with that, unless Carol has --
MS. HANLON: Dr. Garrick, Bill has brought
copies of his uncertainty analysis and strategy, as
well as Peter's guidelines. So we're going to leave
these here for you. If you need additional copies,
let us know.
And I'd just like to thank Bill and Peter
again for working around very difficult schedules,
including technical exchanges and Peter's out of the
country trek to be here today.
MEMBER GARRICK: We know they are very
busy men, and we know you're a very busy lady. Thank
you very much.
(Whereupon, at 12:48 p.m., the
proceedings in the foregoing matter went
off the record.)
Page Last Reviewed/Updated Monday, October 02, 2017
Page Last Reviewed/Updated Monday, October 02, 2017