OIG/97A-02 - Review of M-Cubed's Work on the Reactor Program System
- Report Synopsis
- Agency Comments
- Objectives, Scope, and Methodology
- Abbreviations and Acronyms
- Major Contributors to this Report
- Glossary: Office of the Inspector General Products
Office of the Inspector General
U.S. Nuclear Regulatory Commission
Washington, D.C. 20555-0001
April 23, 1997
|Memorandum To:||Anthony J. Galante|
Chief Information Officer
Leonard J. Callan
|From:||Thomas J. Barchi|
Assistant Inspector General for Audits
|Subject:||Review of M-Cubed's Work on the Reactor Program System|
Attached is the Office of the Inspector General's audit report entitled, "Review of M-Cubed's Work on the Reactor Program System."
On April 15, 1997, the Deputy Executive Director for Management Services responded to our draft report. She agreed with our recommendation and outlined corrective actions to be taken.
In 1996, the Office of the Inspector General (OIG) evaluated the U.S. Nuclear Regulatory Commission's (NRC) oversight of the M-Cubed Information Systems, Inc. contract, reported contract management problems, and identified lessons the agency could learn to more effectively develop systems and oversee contracts. Because of continuing concerns about the M-Cubed contract, OIG initiated a further review of M-Cubed's efforts, particularly the agency's controls over work on the inspection planning module of the Reactor Program System (RPS/IP). NRC officials expected M-Cubed to complete this part of the RPS project.
We found that the agency's best interests were not served in continuing work on RPS/IP using M-Cubed as a subcontractor to Applied Management Systems, Inc. (AMS). NRC managers expected that the agency would benefit from AMS oversight, however, little, if any, benefit was gained. As a result, agency resources were not effectively managed and NRC incurred an additional cost in the form of the fee paid to AMS.
The agency does not currently know what work will be needed to complete RPS/IP. Another Office of Information Resources Management contractor will provide NRC with cost and time estimates for completing the work. In addition to providing those estimates, the contractor will evaluate the technical soundness of the existing work and determine whether modifications are needed to ensure the quality of RPS/IP. To date, the cost of work on RPS/IP totals approximately $1 million.
Our report makes one recommendation to improve NRC's management of contracts for developing agency systems.
In late 1995, the Office of Administration (ADM) expressed concerns about the management controls over the M-Cubed Information Systems, Inc. contract. Among other points, ADM officials raised questions about what they believed were significant and recurring cost increases on the M-Cubed contract. Those concerns were detailed in communications between ADM and the Office of Information Resources Management (IRM) regarding justification for increasing the ceiling on the M-Cubed contract--an area in which the two offices did not agree. At the request of a Deputy Executive Director for Operations (DEDO), the Office of the Inspector General (OIG) evaluated NRC's oversight of the M-Cubed contract(1). OIG disclosed problems with NRC's management of this contract and identified lessons the agency can learn to more effectively develop systems and oversee contracts.
Moreover, in September 1996, Office of Nuclear Reactor Regulation (NRR) personnel expressed concerns to OIG about M-Cubed's work on the Reactor Program System (RPS). In general, NRR had concerns with M-Cubed's use of personnel and the rate at which funds were expended under the task order issued for RPS work. NRR's concerns were echoed by the IRM Project Officer. Based on our previous work and the more recent concerns about M-Cubed's work, we initiated further review of M-Cubed's efforts and the controls over RPS work. Appendix I of this report contains additional information on our objectives, scope, and methodology.
NRC contracted with M-Cubed in August 1993 to provide support in moving the agency toward a client-server systems environment by developing a pilot application to integrate Office of Personnel systems. NRC modified the contract in July 1994 to expand the client-server work to several other NRC systems being developed by IRM, including RPS, and to allow NRC to issue task orders under the contract.
RPS is an NRR-sponsored information system that is designed to provide a single, integrated system with one common user interface for all reactor licensing and inspection matters. RPS will allow management and staff to schedule work items, account for effort on and status of those items, retrieve pertinent guidance and historical information, and produce reports describing completion of the work. NRR officials expected to complete the inspection planning module (RPS/IP) and the system design of the long-term RPS project in fiscal year 1996.
In August 1995, M-Cubed began work under task order 7, issued for work on RPS/IP, with a total cost plus fixed fee of about $627,000. Although funding was budgeted to cover work through December 1996, resources were essentially expended by July 1996 and, therefore, M-Cubed's work as a prime contractor stopped. No funding was added to the contract as a result of the disagreement between IRM and ADM. However, IRM decided to continue the work started under task order 7 with M-Cubed as a subcontractor to another IRM contractor.
The ADM/IRM impasse led to the DEDO's request that OIG examine the M-Cubed matter. In April 1996, during M-Cubed's work under task order 7, OIG issued Special Evaluation 96E-13, Selecting, Managing, and Utilizing the M-Cubed Contract. OIG reported that the M-Cubed contract was a potentially high-risk effort for introducing new client-server technology to the agency. That risk level increased with the expansion of work placed with M-Cubed. OIG found that breakdowns throughout the contractor selection, work specification, and contract oversight processes resulted in cost increases and delays in system implementation. The rapid expenditure of task order 7 funds was evidence of problems with NRC controls over M-Cubed's efforts.
During 1995, IRM completed plans for a new systems development strategy, the Comprehensive Information Systems Support Contract (CISSCO). This strategy provides for a single contractor, using a variety of subcontractors, to provide a wide range of services for the development, operation, maintenance, and support of applications software systems and for related operations support. In March 1996, IRM, rather than use the agency's procurement office, decided to use a contract awarded through the General Services Administration to implement CISSCO.
Agency Funds Were Not Put to Best Use
Because the M-Cubed contract ceiling was not increased, IRM elected to continue some of M-Cubed's efforts by using available funding under an existing Applied Management Systems, Inc. (AMS) contract. Under this arrangement, M-Cubed was a subcontractor to AMS. However, the agency's interests were not well served in that it paid unnecessary costs to continue work under the AMS contract.
In late 1995, officials in IRM and ADM exchanged a series of memos aimed at reaching agreement on sufficient justification for increasing the ceiling on the M-Cubed contract. ADM officials were concerned about cost controls over the contract. Given that funds were not added to the contract, ADM officials did not oppose IRM's decision to continue some of the work using M-Cubed as a subcontractor to AMS. However, they expected that AMS, as the prime contractor, would be involved in conducting the work and would provide additional, needed oversight of M-Cubed's efforts.
IRM, a service organization to the agency to develop systems, was focused on continuing work on key NRC systems. In June 1996, IRM issued task order 15, for about $154,000, to AMS for additional work on RPS/IP. AMS subcontracted the work completely to M-Cubed, with the exception of an AMS Task Manager. In July 1996, M-Cubed began work on the task order, which was scheduled to run through January 1997. However, the funds were virtually exhausted in September 1996. AMS and IRM became aware of the rapid use of funds in mid-August. NRR, the office funding the system, was not informed of this until late September.
To evaluate the results of using the AMS contract to continue work on this and other projects, we reviewed task order 15 and 5 TACs(2) issued under a task order of the AMS contract where M-Cubed, as a subcontractor to AMS, continued work it had begun under its own contract. We found that, although AMS was the prime contractor and NRC thought it would benefit from AMS management experience and oversight, AMS charged virtually no time to the efforts, as shown in Table 1. Although AMS performed general contract administration, it did not provide the day-to-day oversight expected by ADM. Daily oversight would appear as hours charged directly to a task order or TAC while contract administration is charged to NRC under a separate AMS task order. We found that M-Cubed performed virtually all of the work charged to these tasks.
Table 1: Hours Charged on Work Subcontracted to M-Cubed
|TAC/Task Order #||Total Hours Charged||AMS Hours||M-Cubed Hours||Other Hoursa||AMS %||M-Cubed %||Other %a|
|Task Order 15||2,550.5||78||2,472.5||0||3||97||0|
|TACs under Task Order 6|
|aOther subcontractors charged hours to these tasks
Source: OIG analysis of status reports for the above tasks.
AMS believed that IRM expected it to minimize its participation and daily subcontractor oversight, and allow M-Cubed to simply continue work on previously begun efforts. IRM management stated that they expected AMS to provide managerial oversight only; they expected no AMS involvement in the technical work. As mentioned earlier, ADM's expectations in agreeing to this arrangement were also that AMS would be involved in conducting the work and providing needed oversight of M-Cubed's efforts. Aside from these varied expectations, the arrangement was also hampered by the fact that AMS and M-Cubed are typically competitors and this strained their ability to work together.
So, although various NRC personnel expected that the agency would benefit from AMS oversight, little, if any, benefit was gained. As a result, NRC incurred, at a minimum, an additional cost in the form of the fee paid to AMS(3). For instance, under task order 15, we calculated that NRC paid an additional cost of about $14,000 (approximately 9% of the total cost) above the cost of the same effort if the work had been continued under the M-Cubed contract. Despite the fact that M-Cubed was a subcontractor under AMS, IRM essentially continued to use M-Cubed like a prime contractor, thus minimizing the benefits that could have been provided by AMS. NRC's experience with accelerated resource expenditure under M-Cubed's task order 7 should have alerted the agency to the need for additional efforts to control similar difficulties with later work.
Given the problems with controls over the expenditure of funds and NRR concerns, we also reviewed M-Cubed's support for billings under Task Orders 7 and 15. We found that M-Cubed's billings were adequately supported, reasonable, and valid.
Currently, the agency does not know what it will cost to complete RPS/IP. Computer Sciences Corporation (CSC), the IRM contractor under CISSCO, will provide NRC with cost and time estimates for completing the work. In addition to providing those estimates, CSC will evaluate the technical soundness of the existing work and determine whether modifications are needed to ensure the quality of RPS/IP. To date, the cost of all work on RPS/IP totals approximately $1 million.
NRC's best interests were not served in continuing work on RPS/IP using M-Cubed as a subcontractor to AMS. NRC managers expected that the agency would benefit from AMS oversight; however, little, if any, benefit was gained. Under the AMS/M-Cubed arrangement, agency managers had varied expectations; there were misunderstandings among agency officials and contractors; and agency resources were not effectively managed.
The agency does not currently know what work will be needed to complete RPS/IP. Another IRM contractor will provide NRC with cost and time estimates for completing the work. In addition to providing those estimates, the contractor will evaluate the technical soundness of the existing work and determine whether modifications are needed to ensure the quality of RPS/IP. To date, the cost of work on RPS/IP totals approximately $1 million. Our review of M-Cubed's support for billings under Task Orders 7 and 15 disclosed that they were adequately supported, reasonable, and valid.
Because of continuing problems in awarding and managing procurements for developing agency systems (as discussed in this and OIG's prior report on the M-Cubed contract), we recommend that the Chief Information Officer and the Executive Director for Operations take appropriate actions to ensure that these problems are not repeated as the agency implements its new system development strategy (CISSCO). In particular, there needs to be agreement among the various offices in terms of what is in the best interests of NRC. The need for close coordination and effective communication should be underscored.
On April 15, 1997, the Deputy Executive Director for Management Services (DEDM) responded to our draft report and agreed with our recommendation. The DEDM stated that the Executive Director for Operations, the Chief Information Officer, and the Chief Financial Officer will collectively take corrective actions to ensure more effective management of information system procurements. We believe these actions will address the intent of our recommendation.
Objectives, Scope, and Methodology
Continuing concerns about M-Cubed Information Systems, Inc.'s work for the U.S. Nuclear Regulatory Commission (NRC) prompted us to look at NRC's management of those efforts. Specifically, we reviewed M-Cubed's work on the Reactor Program System (RPS). The objectives of our audit were to determine whether (1) M-Cubed's charges to NRC were reasonable and valid, and (2) NRC's management controls related to M-Cubed's work on RPS were adequate.
The scope of our audit was limited to M-Cubed's efforts under task order 7, contract 33-93-196 (the M-Cubed contract), and as a subcontractor under task order 15, contract 33-92-203 (the Applied Management Systems, Inc. (AMS) contract). The audit focused on the period subsequent to that covered in the Office of the Inspector General's Special Evaluation, Report 96E-13, dated April 17, 1996. This audit touched on some of the same issues but the audit program was designed to address the management controls involved in the problems that continued after the Special Evaluation was issued and to review M-Cubed's billings, an area not covered in the Special Evaluation.
To determine whether the contractor's charges were reasonable and valid, we identified M-Cubed's total charges under both task orders and reviewed compliance with contract terms, including Federal Acquisition Regulations. We did not fully assess M-Cubed's controls over billings but rather relied on substantive testing of M-Cubed's payroll records to determine the reasonableness and validity of billings. Our review included a comparison of all employee time sheets to hours charged on invoices; a review of canceled checks issued to employees for corresponding pay periods; a check of supervisory review of time sheets and initialing of all changes; and a review of invoices from consultants whose time was charged to NRC.
We reviewed NRC's management controls related to work performed by M-Cubed on RPS. To perform our review of controls, we interviewed senior management and staff from the Division of Contracts, the Office of Information Resources Management (IRM), and the Office of Nuclear Reactor Regulation. We also interviewed management personnel from AMS and M-Cubed. In addition, we reviewed policies and procedures related to the management of the M-Cubed and AMS contracts, the contracts themselves, and other NRC guidance relating to the management of information systems development. Because work on RPS will continue under IRM's new Consolidated Information Systems Support Contract, we also evaluated certain controls related to that program. And, because IRM will be using the also new System Development and Life Cycle Management Methodology for future developments, we reviewed certain controls related to that methodology.
Our audit was conducted in accordance with generally accepted Government auditing standards from October 1996 through January 1997.
|April 15, 1997|
|Memorandum To:||Thomas J. Barchi|
Assistant Inspector General for Audits
|From:||Patricia G. Norry|
Deputy Executive Director for Management Services
|Subject:||Draft Report - Review of M-Cubed's Work on the Reactor Program System (RPS)|
This responds to your March 24, 1997, memorandum transmitting the subject audit report.
The EDO and CIO jointly agree with the key aspect of this report which is the need for close coordination and effective communication among the various offices in terms of what is in the best interests of the NRC when developing agency systems. The responsibility for the successful implementation of any information resources management project is shared among everyone involved: the sponsors, the users, and the administrative and technical support staffs. The EDO, CIO, and CFO will collectively ensure that their respective staffs are working together to guarantee the effective and efficient design, implementation, and operation of all agency information resources management projects.
Please contact me at 415-7443 or Edward L. Halman at 415-6222 should you have any questions regarding our response.
Abbreviations and Acronyms
|ADM||Office of Administration|
|AMS||Applied Management Systems, Inc.|
|CISSCO||Comprehensive Information Systems Support Contract|
|CSC||Computer Sciences Corporation|
|DEDO||Deputy Executive Director for Operations|
|DEDM||Deputy Executive Director for Management Services|
|IP||Master Inspection Planning System II|
|IRM||Office of Information Resources Management|
|M-Cubed||M-Cubed Information Systems, Inc.|
|NRC||U.S. Nuclear Regulatory Commission|
|NRR||Office of Nuclear Reactor Regulation|
|OIG||Office of the Inspector General|
|RPS||Reactor Program System|
|TAC||technical assignment control|
Major Contributors to this Report
Corenthis B. Kelley, Team Leader
Robert W. Moody, Senior Auditor
Cheryl A. Miotla, Management Analyst
Glossary: Office of the Inspector General Products
1. Investigative Report - White Cover
An Investigative Report documents pertinent facts of a case and describes available evidence relevant to allegations against individuals, including aspects of an allegation not substantiated. Investigative reports do not recommend disciplinary action against individual employees. Investigative reports are sensitive documents and contain information subject to the Privacy Act restrictions. Reports are given to officials and managers who have a need to know in order to properly determine whether administrative action is warranted. The agency is expected to advise the OIG within 90 days of receiving the investigative report as to what disciplinary or other action has been taken in response to investigative report findings.
2. Event Inquiry - Green Cover
The Event Inquiry is an investigative product that documents the examination of events or agency actions that do not focus specifically on individual misconduct. These reports identify institutional weaknesses that led to or allowed a problem to occur. The agency is requested to advise the OIG of managerial initiatives taken in response to issues identified in these reports but tracking its recommendations is not required.
3. Management Implications Report (MIR) - Memorandum
MIRs provide a "ROOT CAUSE" analysis sufficient for managers to facilitate correction of problems and to avoid similar issues in the future. Agency tracking of recommendations is not required.
4. Audit Report - Blue Cover
An Audit Report is the documentation of the review, recommendations, and findings resulting from an objective assessment of a program, function, or activity. Audits follow a defined procedure that allows for agency review and comment on draft audit reports. The audit results are also reported in the OIG's "Semiannual Report" to the Congress. Tracking of audit report recommendations and agency response is required.
5. Special Evaluation Report - Burgundy Cover
A Special Evaluation Report documents the results of short-term, limited assessments. It provides an initial, quick response to a question or issue, and data to determine whether an in-depth independent audit should be planned. Agency tracking of recommendations is not required.
6. Regulatory Commentary - Brown Cover
Regulatory Commentary is the review of existing and proposed legislation, regulations, and policies so as to assist the agency in preventing and detecting fraud, waste, and abuse in programs and operations. Commentaries cite the IG Act as authority for the review, state the specific law, regulation or policy examined, pertinent background information considered and identifies OIG concerns, observations, and objections. Significant observations regarding action or inaction by the agency are reported in the OIG Semiannual Report to Congress. Each report indicates whether a response is required.
1. Special Evaluation 96E-13, Selecting, Managing, and Utilizing the M-Cubed Contract, dated April 17, 1996.
2. A technical assignment control number (TAC) is a unique control number assigned by offices to identify work requests. TACs are used to track the status of that work to completion.
3. There were additional cost effects that we did not measure for all of these tasks. For instance, under task order 15, M-Cubed's rates, as charged to AMS, varied from those under M-Cubed's own contract.