Reactor Operator and Senior Reactor Operator Examinations (Generic Letter 82-13)

                               UNITED STATES 
                       NUCLEAR REGULATORY COMMISSION 
                          WASHINGTON, D. C. 20555 

                               June 17, 1982 

TO ALL POWER REACTOR LICENSEES, APPLICANTS FOR AN OPERATING LICENSE AND 
HOLDERS OF A CONSTRUCTION PERMIT 

Gentlemen: 

Subject:  Reactor Operator and Senior Reactor Operator Examinations 
          (Generic Letter 82-13) 

Enclosed are the results of a meeting held in Bethesda, MD, on January 6, 
1982, to discuss changes to the examination used to license Reactor and 
Senior Reactor Operators. The questions and comments raised during that 
meeting are discussed in the enclosed summary. 

This letter is for your information only and requires no response or action.

                              Sincerely, 


                              Darrell G. Eisenhut, Director 
                              Division of Licensing  
                              Office of Nuclear Reactor Regulation 

Enclosure: 
1. Response to Questions/
     Comments 
2. List of Attendees  





8204210387 
.

General Topic: Written Examination Questions 

Questions/Suggestions: 

Questions should not require lengthy calculations that are not done on the 
job. 

Questions should not be taken from the FSAR since it is worst case and may 
not be up to date. 

Technical Specification questions should not require knowing actions after 
one hour since the SRO can look these up. 

More short answer or multiple choice questions should be used. 

Health Physics questions should concentrate on operating information not 
time-distance-shielding type questions. 

Mitigating core damage should be included in exams. 

RO Exams should not include Tech Specs or Emergency Plan since these are not
her responsibility. 

If new topics are to be included in the exams, the utilities should be 
notified. In addition to submitting good questions, utilities should be able
to identify questions they feel are bad. 

Response: 

As indicated in the January 6 meeting, emphasis in the exam will be on 
operational information. To ensure that exam questions are in line with the 
guidelines given to you, we are reviewing the questions and deleting or 
modifying ones that do not comply. As discussed in the exam consistency 
topic, we are developing a computerized bank of exam questions. When fully 
developed and reviewed, we will make this bank available to the public. We 
encourage submission of questions for the bank and comments on questions 
that you feel are not valid. In this effort we are eliminating the Health 
Physics questions that are not generally the responsibility of the RO or 
SRO, and arranging that Tech Spec questions concentrate on understanding of 
the bases, general knowledge of what actions are required immediately 
(within one hour) and why, and what systems have tech spec limits and why. 
In the operational exam the candidate's ability to find and use the tech 
specs will be examined. Question content may still include calculations to 
determine the candidate's understanding of the principles involved in 
nuclear plant operation, but calculations for the sake of calculations will 
be avoided. 

We are investigating short answer and multiple choice question formats and 
have used both in exams that have been given. We will not, however, shift 
entirely to this format until we have assured that the questions present a 
fair opportunity for the candidate to show his or her knowledge and we have 
notified the industry that we will be changing exam style. 

We are continually developing new questions from the training material 
provided. We will continue to use the FSAR as a source of information, but 
we recognize its limitations. As always, out of date information should be 
pointed out in the exam critique so that questions and answers can be 
updated. We hope in the near future to develop a regular information letter 
to inform the public of new areas of interest in the exam. 
.

General Topic: Examination Administration 

Questions/Suggestions:  

What responsibilities/authority do proctors, especially when it is the 
Resident Inspector, have? 

Will questions be made available to allow trainers to know what areas should
be covered? 

The exam should be open book, allowing access to the materials the operator 
would have in the control room (i.e., Tech Specs). 

Will NRC guidance be updated to reflect the new exam format? 

What options are open for taking the written exam? AM/PM sessions? One 
sitting? And what options for splitting sections are available if a split 
period used? 

Reducing the number of questions (time) increases the importance of each 
question. 

Formula/Equation sheet should be standardized. 

Can the utilities get copies of questions submitted by other utilities? 

NRC should recommend a list of standard texts. 

A firm date for implementing the new format should be set. 

Reporting results should be standardized. Some people get summary sheets, 
some don't know results until licenses or denials arrive. Results should be 
in faster. 

Why not leave a copy of the exam after it has been taken? 

Can the utilities get a copy of the exam in advance to allow for a thorough 
review. 

Why establish a time limit? The exam should find out what you know, not how 
fast you can write. 

Why not use qualified industry people to administer exams at other 
facilities? 

Better guidance is needed on what will be covered in the exam and better 
information on grading criteria and granting waivers should be available. 
There is no way to check on the results of oral exams. 

Are machine prepared, machine graded exams possible? Probable? 

Better clarification of the March 28, 1980 letter on qualifications is 
needed. 
.

Response: 

Your concerns fall into several areas. First, the exam may be taken either 
at one sitting or broken into two parts. The utility has the option, but all
candidates must take the exam the same way. The exam will be given, as 
indicated, section 1, 2, and 3 or 6, 7, and 8 in the first three hours and 
4, 5, or 9, 10 in the second three hours if a split sitting is elected. The 
examiners or proctors have the authority to allow individual breaks during 
the exam, but the exam must still be finished within the allowed time. The 
examiner or proctor will keep track of elapsed time. 

Second, only examiners are authorized to modify an exam. Proctors, even 
resident inspectors, are not authorized to make changes. Proposed changes 
should immediately be called to the attention of the assigned chief 
examiner. 

Third, as discussed in the exam questions topic, we are reviewing the 
questions, investigating multiple choice and short answer questions to allow
more areas to be covered in the limited amount of time, however, a time 
limit will remain. We are investigating machine generated/machine graded 
formats but have no immediate plans for implementing such a system. Before a 
change like that will be adopted, the approach will be validated and your 
comments solicited prior to any action being taken. Once fully developed, we 
will consider making the questions contained in the exam question bank 
available to assist you in your training program and in keeping the 
questions current. 

Fourth, we do not intend to go to an open book exam. We are working to 
ensure that the written exam tests for information that the candidate should 
know without aids and the operational exam tests his ability to use aids 
such as procedures and Tech Specs. 

Fifth, we are in the process of developing updated guidance on exam content,
objective and subjective grading criteria, exam administration and 
application content. We will not endorse a set of reference texts. This 
guidance, coupled with greater accessibility of exam questions, should 
improve information available on what the exam will cover. Any texts that 
suitably cover the material are acceptable. 

Sixth, we are implementing a system to allow automatic, computer-aided 
tracking of applications and exam results. Our goal is to have all results 
reported to the individuals and utilities within two months of completion of
the examination. As more examiners are certified this goal will be reviewed 
to see if we can reduce it even further. You can assist us by refraining 
from calling for results or submitting FOIA requests until after the two 
months have passed. Exam results are not final until all portions of the 
exam have been completed and internal audits for consistency and fairness 
are done. At that point, exam summary sheets will be sent to the utility, 
and licenses or denials and a copy of the written exam will be sent to the 
individual. 

Finally, almost all old format reexaminations have been completed and 
sufficient experience has been gained in preparing new format exams. 
Therefore, only new format exams will be given for examinations scheduled 
after July 1, 1982. 
.

General Topic: Examination Consistency 

Questions/Suggestions:  

Some examiners lack the knowledge and training necessary to give oral exams.
They use poor methods and intimidate the candidates. 

Some examiners are too academically oriented. There are large 
inconsistencies between examiners. You almost have to know who is preparing 
your exam so you can prepare the candidates in the proper areas. 

Some examiners will not accept an operating method that differs from a 
"cookbook" method even though the alternate method may be equally 
acceptable. Therefore, correct answers are marked wrong. 

There are no standards for required level of knowledge for oral exams. 
Therefore, the depth required varies between examiners. 

Response: 

As discussed in other sections, we are working on guidelines for exam 
content and developing a question bank of valid questions. Since these will 
include plant specific, as well as generic questions, the operating 
philosophy of each facility can be reflected in their exam questions. 

For oral exams we are preparing guidelines for the examiners. Appropriate 
portions will be made publicly available. 

We have also established a training and certification program for contract, 
examiners. This will ensure that a minimum competence has been obtained 
prior to conducting exams. To ensure competence and consistency, all 
examiners, NRC and contractors, will be audited periodically by the OLB 
Section Leader. We are sensitive to the issue of competency and consistency 
because we are actively transferring examiner functions to the Regional 
Offices. Any specific feedback from an examination is welcome and will be 
kept confidential. 
.

General Topic: Simulator Exams 

Questions/Suggestions: 

Plant specific simulators should be mandated by NRC. 

What are NRC's future plans for simulator exams? More guidance is needed on 
what will be covered in simulator exams, especially for non-plant-specific 
simulators. 

Response: 

A Commission Paper has been prepared recommending that the requirement for 
non-plant-specific simulator exams be removed. We have studies underway to 
develop valid operational exams, including simulator exams. At this time we 
feel that dynamic transient operation is an important aspect of the exam but
that the limitations imposed by non-plant-specific simulators and the 
scheduling problem encountered reduce the effectiveness of the exam and do 
not justify the resources required. Therefore, we are considering returning 
to the old exam method of performing power transients, startups and 
shutdowns on those facilities that do not have a plant specific simulator 
available. Any change in the exam will be discussed before it is 
implemented. Until the need for operating tests on the facility or other 
testing methods being studied by the staff are validated, the operating 
examination will continue to follow the existing guidelines in NUREG 0094. 
Therefore, there is a continued need for simulator training to comply with 
the guidance in NUREG 0094. The Commission has not acted on the staff 
recommendation at this time. The Commission paper does not include any 
requirements for actual plant operations at this time. 

As discussed under the exam administration topic, we are developing 
guidelines for simulator exams. We are also modifying the operational exam 
summary sheet to facilitate the simulator exam and to ensure that areas 
examined at, the simulator are not duplicated in the plant walk-through. 

Miscellaneous Question: 

Where are guidelines for medical applications? 

Response: 

Basic requirements are in Part 55. Amplifying guidance in Regulatory Guide 
1.134 which endorses ANSI/ANS 546. 

Suggestion: 

Fallback to RO on instant SRO failure isn't worth, much since engineers or 
supervisors aren't in the union and cannot perform RO duties. 

Response: 

We agree that fallback has little value. When a candidate is certified to 
need an SRO license to perform his or her duties, we don't see how having an
RO will help. Under the new format exam, this problem should disappear. 
.

Suggestion: 

NRC should put certification of training instructors on high priority since 
this could have a real impact on training. 

Response: 

We agree entirely. Our highest priority, as always, is licensing new 
operators. As more examiners are more examiners are trained and certified, 
we will be pursuing the area of instructor certification more vigorously. 
All Operating License applicants are required to have certified instructors 
prior to fuel load and we have certified instructors at several operating 
plants. We are monitoring the INPO work on training program accreditation 
and are investigating means to certify vendor and consultant instructors 
that teach systems and operations courses. More information will be 
available on this subject in the near future. 

Question: 

What are NRC's plans for requalification exams? 

Response: 

In the Commission paper on non-plant-specific simulators we discuss the 
subject of requalification at length. To summarize, as directed by the 
Commission, we will start giving requalification examinations in conjunction
with scheduled visits for replacement exams. We are targeting at least one 
site visit to each facility this year and expect to administer 
requalification examinations to 20% of the licensed operators. Our initial 
plans are to conduct the requalification exams during replacement exams; 
however, we will work out availability and schedules with each utility. In 
the Commission paper, the staff proposed that for utilities with a plant 
specific simulator, the requalification exam will be given only on the 
simulator. Otherwise, a written exam and practical test will be given. 
Failure of the exam will require accelerated retraining in weak areas, as is
required now, and NRC reexamination. If significant weaknesses in the 
utility requalification program are revealed by the exams, NRC administered 
requalification exams for all license renewal applicants will be likely 
until the requalification program has been sufficiently upgraded. Details of 
the program are in the Commission paper. No Commission action has been taken 
at this time. 

Suggestion: 

NRC should periodically issue a listing of generic weak points. 

Response: 

When our automated system is operational, we intend to issue quarterly 
information reports that will include observed weak areas. It will also 
include areas of concern at NRC that might affect the content of future 
exams and general information on the exams. We hope to be producing these 
reports in the very near future. 
.

Suggestion: 

Cold plants need amplification of H. Denton's March 28, 1980 letter 
regarding experience requirements. 

Response: 

It is difficult to provide better generic guidance than exists in the letter
Task Action Plan Item I.A.2.1 NUREG-0737 and NUREG 0094. Specific cases can 
be discussed with the appropriate OLB Section Leader. The OLB policy is to 
grant exemptions or waivers only in specific cases, not on a generic basis. 
Therefore, the requirements should be discussed with the Section Leaders.