OIG/96E-13 - Selecting, Managing, and Utilizing the M-Cubed Contract
April 17, 1996
|MEMORANDUM TO:||Hugh L. Thompson
Deputy Executive Director for
Nuclear Materials Safety, Safeguards and Operations Support
|FROM:||Thomas J. Barchi
Assistant Inspector General for Audits
|SUBJECT:||SELECTING, MANAGING, AND UTILIZING THE M-CUBED CONTRACT|
Attached is the Office of the Inspector General's subject report of our special evaluation conducted at your request on the M-Cubed contract. This report identifies lessons the agency can learn in order to more effectively develop systems and oversee contracts in the future. Our review of this contract disclosed control processes that did not work as intended and communication breakdowns among and between key offices. We believe the concerns surfaced during this evaluation raise the question of whether the agency needs a focal point in reviewing key information technology decisions.
We briefed senior NRC managers following our evaluation so they could consider appropriate actions on our observations. This report does not contain recommendations, therefore, we did not solicit agency comments and will not track staff actions to address our observations. However, we will continue to monitor the agency's efforts to strengthen the various processes.
|cc:||J. Taylor, EDO
J. Milhoan, OEDO
J. Blaha, OEDO
J. Hoyle, SECY
K. Cyr, OGC
D. Rathbun, OCA
R. Scroggins, OC
P. Norry, ADM
G. Cranford, IRM
R. Bangart, OSP
W. Russell, NRR
E. Jordan, AEOD
D. Morrison, RES
C. Paperiello, NMSS
J. Funches, ICC
W. Beecher, OPA
T. Martin, RI
S. Ebneter, RII
H. Miller, RIII
L. Callan, RIV
- Evaluation Results
- Contractor Selection
- Contract Requirement Specification
- Contract Administration and Oversight
- U.S. NRC Functional Organizational Chart
- Major Contributors to this Report
- Glossary: Office of the Inspector General Products
At the request of the Deputy Executive Director for Nuclear Materials Safety, Safeguards and Operations Support (DEDO), the Office of the Inspector General (OIG) conducted a special evaluation on the U.S. Nuclear Regulatory Commission's (NRC) performance in selecting, managing, and utilizing a systems development contract with M-Cubed Information Systems (M-Cubed). The DEDO's request stemmed from the inability of key NRC offices to reach a timely agreement on contract ceiling increases needed to complete work on several critical systems. This impasse led to a stoppage of the work in progress on these systems.
Our purpose in conducting this evaluation was to examine the circumstances surrounding the M-Cubed contract and identify processes which need to be strengthened to avert similar future occurrences on NRC procurements. Our evaluation identifies lessons the agency can learn to more effectively develop systems and oversee contracts in the future. It also demonstrates what can happen when control processes do not serve the agency as intended. Our work disclosed that, during the contractor selection process, NRC did not pay close enough attention to potential warning signs and made hasty decisions due to organizational pressures. We also identified instances of communication breakdowns in the work specification and contract oversight processes.
The M-Cubed contract evolved into a high-risk endeavor for NRC for a number of reasons: (1) M-Cubed was a small firm that was probably limited in its capabilities to successfully respond to the large increase in workload desired by NRC; (2) M-Cubed had not finished a $700,000 pilot project before NRC decided to place about another $3 million of client-server work with the firm; (3) the client-server technology work being performed by M-Cubed was new to NRC; and (4) the interactive systems development approach used by the Office of Information Resources Management was relatively new to NRC and we believe it requires unique contract oversight. Despite the need for close scrutiny of the M-Cubed contract, the processes that normally help reduce the risks associated with a contract did not adequately work as designed. We believe that appropriate responses to these lessons can strengthen intra/inter office communications and team work, thus helping the agency to more economically and efficiently develop systems and oversee contracts.
This report provides the results of a special evaluation conducted by the U.S. Nuclear Regulatory Commission's (NRC) Office of the Inspector General (OIG). The Deputy Executive Director for Nuclear Materials Safety, Safeguards and Operations Support (DEDO) was concerned with the adequacy of selecting, managing, and utilizing a systems development contract awarded to M-Cubed Information Systems (M-Cubed). The inability of key NRC offices to reach timely agreement on contract ceiling increases and the resulting work stoppages on several critical systems by the contractor triggered the DEDO's concerns.
NRC's goal in implementing client-server technology is to replace the agency's numerous computer platforms with an integrated set of information systems. Client-server technology will allow the agency to perform onsite computing using data from mainframe, minicomputer, and microcomputer systems rather than relying on timesharing activities at the National Institutes of Health or on its' own aging and underpowered Data General computer systems.
NRC's strategy, as stated in the Fiscal Years 1994-1998 Information Technology Strategic Plan (Strategic Plan), was to move aggressively toward client-server technology. A U.S. General Services Administration contractor independently verified and validated NRC's strategic initiative to adopt and implement client-server technology. Prior to the issuance of the Strategic Plan, the Office of Personnel (OP) requested help from the Office of Information Resources Management (IRM) to integrate its numerous computer systems into one. The desire to meet OP's request, coupled with the Strategic Plan, led IRM to seek immediate solutions using client-server technology.
IRM first looked in-house and to existing contractors to provide support in moving the agency toward a client-server environment. Because these sources lacked experience in this area, IRM turned to M-Cubed, a small 8(a) firm. NRC first learned of M-Cubed from an International Business Machine Corporation representative. After reviewing the client-server capabilities of six other 8(a) firms, NRC awarded a contract worth about $700,000 to M-Cubed in August 1993.
NRC, through this contract, tasked M-Cubed to develop a pilot application to integrate OP systems using client-server technology. In December 1993, IRM requested to expand the scope of the contract to meet the needs of other agency offices, which the Senior Contract Review Board (SCRB) approved. This expansion, dated July 1994, incorporated additional work under the base contract on a task ordering basis at an estimated cost of $2.987 million. In October 1995, IRM requested an additional estimated $2.725 million to complete existing as well as new work covered by this modification.
We believe the agency can learn valuable lessons from the experiences gained with the M-Cubed contract. We observed several processes that did not function as designed which led to the work stoppage under this contact. During the contractor selection process, NRC did not pay close enough attention to potential warning signs and made hasty decisions due to organizational pressures. We also identified instances of communication breakdowns throughout the contractor selection, work specification, and contract oversight processes.
The M-Cubed contract developed into a high-risk endeavor for NRC for a number of reasons: (1) M-Cubed was a small firm that was probably limited in its capabilities to successfully respond to the large increase in workload desired by NRC; (2) M-Cubed had not finished a $700,000 pilot project before NRC decided to place about another $3 million of client-server work with the firm; (3) the client-server technology work being performed by M-Cubed was new to NRC; and (4) the interactive systems development approach used by IRM was relatively new to NRC and we believe it requires unique contract oversight. We also believe senior management should consider enhancing existing processes to help the agency minimize future risks for information technology projects and to improve its overall contract management effectiveness. These matters are discussed in greater detail in the following sections.
NRC's contractor selection process should help the agency minimize risks associated with the selection of a contractor. We found that the agency's desire to meet the needs of the offices and NRC's goal to aggressively move toward client-server technology led the agency to use a Small Business Administration's Section 8(a) Program contractor in lieu of going through "full and open competition". Contracts under $3 million in the 8(a) Program generally are not competed, thus reducing the time for contract award.
NRC's initial impressions were that M-Cubed was a capable company that had experience in client-server technology. In August 1993, NRC selected M-Cubed to conduct a pilot project to met the need of OP to integrate its multiple data bases. The OP pilot project was an opportunity for M-Cubed to demonstrate its client-server capabilities to NRC. However, prior to the completion of OP's pilot project, the SCRB(1) approved IRM's request to use M-Cubed to perform additional client-server integration work for the Offices of Nuclear Reactor Regulation, the Secretary of the Commission, Personnel, and the Office for Analysis and Evaluation of Operational Data. IRM formally modified the M-Cubed contract by $2.987 million to add this expanded work in July 1994.
We believe NRC missed two important pre-contract potential warning signals. MCubed had no prior Federal Government contracting experience and was a small company with three employees when it began contract work for NRC. In addition, the agency moved ahead to significantly expand contract requirements before thoroughly assessing M-Cubed's performance on the pilot project. The SCRB's approval to expand M-Cubed's contract after only four months of work essentially committed the agency to rely on a small company with potentially limited capability to achieve an important part of NRC's Strategic Plan.
Contract Requirement Specification
NRC offices did not effectively communicate and work together as a team to establish and agree to contract work specifications and ultimate system requirements needed by users. Effective communication and understanding of contract requirements early on is essential for all involved in the process.
In addition to introducing the client-server technology to NRC, the M-Cubed contract used a technique relatively new to NRC called rapid application development (RAD). RAD is an interactive approach to systems development in which user feedback drives the development of system features and requirements as a system is designed. Commonly, under the RAD approach: (1) the contractor designs a prototype based on imprecise requirement specifications because not all requirements are known at the onset; (2) the users provide feedback on the prototype; (3) the contractor redesigns the prototype incorporating the requirements resulting from the user feedback; and (4) this process is repeated until the system prototype is acceptable to the users. Using RAD, contract costs may increase as users identify new and expand existing requirements. We believe this systems development method requires unique contract management in order to effectively control costs and evaluate contractor performance.
We believe the Office of Administration's (ADM) Division of Contracts (DC) and IRM did not come to a common understanding of the implications of RAD on contract requirements. Seemingly, this condition contributed to disagreements between DC and IRM during the oversight phase of this contract. (We discuss contract administration and oversight in the next section of this report). Further, DC personnel stated the M-Cubed contract, as written, is not structured to utilize a systems development technique like RAD. We believe this emphasizes the lack of effective communication and understanding between these offices on RAD's impact on contract management.
Also, our evaluation found that not all potential users of a system were identified at the beginning of the development process. Potential system users need to be identified early in the process so that users can provide timely ideas on system requirements. Generally, the identification of additional system users during the development process results in new and/or expanded requirements and increased contract costs. This in turn should be an important consideration in deciding whether additional requirements can be added to an existing contract without impeding a contractor's ability to successfully perform.
Additionally, IRM and at least one user office had different views as to the scope of work required on one system. Since the work scope was not adequately documented to define the work the contractor should perform on that system, disagreements subsequently arose between the user office and IRM. We believe it is imperative that agreement between IRM and user offices as to work scope should be reached early on and changes be continually documented so that expectations are clear to all parties.
We believe improved communications and better understanding of system requirements by all offices involved can reduce risks associated with systems development projects in the future. Interestingly, we note that both DC and IRM personnel cited a recent agency-wide computer equipment update project as an example where the two offices successfully used a teamwork approach. Although we have not audited this project, it seems to demonstrate the ability of the two offices to work well together.
Contract Administration and Oversight
For a properly functioning contract administration and oversight process, communication and an understanding of expectations between the offices involved is a necessity. This relationship takes even a greater emphasis for service organizations. Both IRM and DC are service organizations.
However, our work showed that NRC offices often did not work well together -hampered primarily by a lack of communication. As previously stated, DC and IRM did not come to a common understanding on the implications of RAD, an interactive systems development approach. DC personnel reportedly noted significant and recurring cost increases on the M-Cubed contract as early as May 1995, however, these concerns were not documented formally and raised as major issues until September 1995. Moreover, one DC official told us that it was also difficult to determine from successive work descriptions whether the contractor was doing new work or repeating work that should have been previously performed. This impacted DC's ability to judge whether NRC had encountered cost overruns on this contract and whether the contractor was entitled to fees. Meanwhile, IRM expected contract costs would go up because system requirements were being identified using RAD.
On October 24, 1995, IRM sought to raise the contract ceiling amount by an additional $2.725 million to complete ongoing and new work. On October 3 1, 1995, DC's contracting officer requested a delegation of contractual authority from the DC Director. The request did not include a request for procurement action (RFPA). In this case, the RFPA should have been prepared by IRM to justify their request, however, due to communication breakdowns with DC, IRM managers believed they had provided all the necessary information. The RFPA was neither submitted by IRM nor requested by DC prior to the delegation request. Unresolved questions about the cost increases and ineffective communication between and within these offices resulted in the DC Director questioning the need for additional funds. ADM and IRM are presently working through a plan to resolve the M-Cubed contract.
Further, managers in two sponsoring offices told us they had concerns about whether M-Cubed had the capability and adequate employees to perform NRC's expanded work. However, these concerns were not documented and provided to IRM for resolution. We believe such concerns need to be raised and documented early on so that resolutions can be made in a timely fashion and before scarce resources are potentially wasted.
Closer scrutiny on high risk contracts can help prevent what has happened with the M-Cubed contract. The agency should address indications of concerns in a timely manner, especially ones requiring additional funding. Key to this is the need for improved communications by all parties involved in the process.
The M-Cubed contract was potentially a high risk effort that introduced new client-server technology to the agency, a goal of NRC's Information Technology Strategic Plan. NRC risks increased with the expansion of the work placed with this contractor. Ultimately, breakdowns throughout the contractor selection, work specification, and contract oversight processes resulted in cost increases and delays in system implementation. This demonstrates what happens when agency control processes do not function as intended.
We believe the agency can learn many lessons from its experiences with this contract. When contracting for such critical work, agency managers need to carefully consider whether a firm has the capability and staff to adequately perform the work. Improved planning and communication among and within all offices involved will help reduce unnecessary risks to NRC. We also believe the concerns surfaced during this evaluation raise the question of whether the agency needs a focal point in reviewing key information technology decisions.
U.S. NRC Functional Organizational Chart
Major Contributors to this Report
Corenthis B. Kelley
Michael A. Cummins
Scott W. Buchan
Senior Management Analyst
Glossary: Office of the Inspector General Products
1. INVESTIGATIVE REPORT - WHITE COVER
An Investigative Report documents pertinent facts of a case and describes available evidence relevant to allegations against individuals, including aspects of an allegation not substantiated. Investigative reports do not recommend disciplinary action against individual employees. Investigative reports are sensitive documents and contain information subject to the Privacy Act restrictions. Reports are given to officials and managers who have a need to know in order to properly determine whether administrative action is warranted. The agency is expected to advise the OIG within 90 days of receiving the investigative report as to what disciplinary or other action has been taken in response to investigative report findings.
2. EVENT INQUIRY - GREEN COVER
The Event Inquiry is an investigative product that documents the examination of events or agency actions that do not focus specifically on individual misconduct. These reports identify institutional weaknesses that led to or allowed a problem to occur. The agency is requested to advise the OIG of managerial initiatives taken in response to issues identified in these reports but tracking its recommendations is not required.
3. MANAGEMENT IMPLICATIONS REPORT (MIR) - MEMORANDUM
MIRs provide a "ROOT CAUSE' analysis sufficient for managers to facilitate correction of problems and to avoid similar issues in the future. Agency tracking of recommendations is not required.
4. AUDIT REPORT - BLUE COVER
An Audit Report is the documentation of the review, recommendations, and findings resulting from an objective assessment of a program, function, or activity. Audits follow a defined procedure that allows for agency review and comment on draft audit reports. The audit results are also reported in the OIG's "Semiannual Report' to the Congress. Tracking of audit report recommendations and agency response is required.
5 . SPECIAL EVALUATION REPORT - BURGUNDY COVER
A Special Evaluation Report documents the results of short-term, limited assessments. It provides an initial, quick response to a question or issue, and data to determine whether an in-depth independent audit should be planned. Agency tracking of recommendations is not required.
6. REGULATORY COMMENTARY - BROWN COVER
Regulatory Commentary is the review of existing and proposed legislation, regulations, and policies so as to assist the agency in preventing and detecting fraud, waste, and abuse in programs and operations. Commentaries cite the IG Act as authority for the review, state the specific law, regulation or policy examined, pertinent background information considered and identifies OIG concerns,
observations, and objections. Significant observations regarding action or inaction by the agency are reported in the OIG Semiannual Report to Congress. Each report indicates whether a response is required.
1. The SCRB was composed of the Deputy Executive Director for Nuclear Reactor Regulation, Regional Operations and Research and other senior office directors.