OIG/97E-10 - Evaluation of Best Practices for Developing and Implementing Integrated Financial Management System
- Report Synopsis
- Objectives, Scope, and Methodology
- Evaluation Results
- Major Contributors to this Report
- Glossary: Office of the Inspector General Products
Office of the Inspector General
U.S. Nuclear Regulatory Commission
Washington, D.C. 20555-0001
June 9, 1997
|Memorandum To:||Jesse L. Funches
Chief Financial Officer
|From:||Thomas J. Barchi
Assistant Inspector General for Audits
|Subject:||Evaluation of Best Practices for Developing and Implementing Integrated Financial Management System|
Attached is the Office of the Inspector General's Special Evaluation Report on our evaluation of best practices for developing and implementing integrated financial management system. Since this report does not contain recommendations, we did not solicit formal agency comments. However, we briefed senior NRC managers on our observations.
In a February 1997 memorandum, the U.S. Nuclear Regulatory Commission's (NRC) Chairman directed the acting Chief Financial Officer (CFO) to develop a plan for establishing an agency-wide financial management system. In the memorandum, the Chairman specifically stated that (1) the system should be operational within the next two years, (2) it should be designed to integrate financial planning data with performance data, and (3) the new system's goal should be to eliminate the need for multiple financial tracking systems. In March 1997, the acting CFO responded to the Chairman. The Financial Management System Development Plan, endorsed by the Executive Council, identified a proposed approach to developing an integrated financial management system.
Because of the importance of this project, the Office of the Inspector General's (OIG) initiated an evaluation to identify and gain an understanding of best practices for developing and implementing a single, integrated financial management system. Our goal is to share with the agency the results of this best practices effort. Our study was not exhaustive, but intended to provide a basic understanding of such efforts. Therefore, our list of best practices does not include all such practices that may be found. While NRC's Development Plan addresses many of the issues we identified, OIG believes that NRC can benefit from the lessons learned by others that are implementing integrated financial management systems.
We found best practices to a successful project include: (1) ensuring top level commitment and involvement, (2) learning from the experiences of others, and (3) minimizing software modification. Finally, while we recognize that there needs to be a sense of urgency about attaining an effective integrated financial management system, we also found that establishing realistic milestones and cost estimates is equally as important.
According to the U.S. General Accounting Office, seriously inadequate automated financial management systems are currently the greatest barrier to timely and meaningful financial reporting. Agency systems are old and do not meet users' needs. In fact, in March 1995, the Office of Management and Budget (OMB) reported that 39 percent of agency systems were originally implemented over 10 years ago and a little over half (53 percent) need to be replaced or upgraded within the next five years.
On July 25, 1995, the former Comptroller General of the United States reported to the U.S. Congress that agencies have a critical need to greatly improve and modernize financial management systems, which across government, are in abysmal shape today. In doing so, agencies need to reengineer financial management processes, while at the same time implementing new accounting and financial reporting standards.
In a February 1997 memorandum, the U.S. Nuclear Regulatory Commission's (NRC) Chairman directed the acting Chief Financial Officer (CFO) to develop a plan for establishing an agency-wide financial management system. In the memorandum, the Chairman specifically stated that (1) the system should be operational within the next two years, (2) it should be designed to integrate financial planning data with performance data, and (3) the new system's goal should be to eliminate the need for multiple financial tracking systems.
In March 1997, the acting CFO responded to the Chairman. The Financial Management System Development Plan, endorsed by the Executive Council, identified a proposed approach to developing an integrated financial management system. The plan explained the agency's five phased approach with the first phase being completed in six months. Phase one will identify the system's components, associated costs, and the detailed activities and schedules for the remaining phases of implementation. Because of the importance of the Chairman's initiative and NRC's past difficulties in developing management information systems, the Office of the Inspector General (OIG) undertook this evaluation.
Objectives, Scope, and Methodology
OIG's objective was to identify and gain an understanding of best practices for developing and implementing a single, integrated financial management system. OIG's goal is to share with the agency the results of this best practices effort. Because our study was not exhaustive, but intended to provide a basic understanding, our list of best practices does not include all such practices that may be found.
We conducted our work from March to May 1997 by interviewing personnel from the U.S. General Services Administration (GSA), the National Aeronautics and Space Administration (NASA), the U.S. Department of Agriculture, the City of San Diego, and a banking institution. These organizations had recently developed or were developing integrated financial management systems. We reviewed criteria established by OMB, GAO, the Joint Financial Management Improvement Program (JFMIP), and the Chief Financial Officers Council. The CFO Council document, A Strategy of Leadership and Engagement, was a significant source of information.
We did not assess the agency's compliance with laws and regulations concerning the development and implementation of a management information system, nor the agency's management controls related to this project.
To assist NRC in meeting the Chairman's goal of improving financial management, we are providing the agency with examples of best practices in developing and implementing an integrated financial management system. We divided our results into two categories: organizational issues and project implementation issues. While NRC's Development Plan addresses many of the same issues identified, OIG believes that NRC can benefit from the lessons learned of others who have recently developed and implemented an integrated financial management systems.
As the agency begins the task of integrating its financial management system, it needs to consider several issues. Our work identified four organizational issues that can adversely affect a project's implementation if not addressed and effectively managed. An effective organization is vital to success because it should clearly define a project's structure and the lines of communication between the project team and users, as well as the team and senior management.
Agency Commitment to the Project
The organizations with whom we spoke emphasized that an integrated financial management system needs top level commitment, a strong project team, and a dedicated staff for project development. Senior management must provide support and leadership by setting general project direction, managing the agency's political environment, establishing system modernization as a high priority, and overcoming skepticism about the value and likely success of the project. For example, at both GSA and NASA, a Deputy CFO has the responsibility for implementing the agency's integrated financial management system.
In a audit report entitled Report on the Audit of the Administrative Systems Automation Project, the Inspector General for the Board of Governors of the Federal Reserve System, reported that "...senior management involvement has been insufficient." The report concluded that (1) strong sustained and visible support by senior management is one of the critical factors in project implementation, and (2) without top level support the project will face increased difficulty in overcoming natural resistance to change, obtaining end user participation, and resolving issues that require procedural or policy changes.
Several organizations advised that a project team must have the proper level of influence and decision making authority. Giving the team proper authority with senior level support allows the team to effectively interact with other offices and permit the project to be developed from an agency-wide approach. For continuity purposes, a full time staff is needed for project implementation. This full time team should consist of representatives from all user groups as well as representatives from the Offices of the CFO and the Chief Information Officer. GSA and NASA, as well as other organizations, dedicated full-time staff to project development efforts.
Another common element of success is a realistic assessment of milestones and sound cost estimates. Any proposed cost increases and milestone extensions should be reviewed and approved by senior management.
NRC's goal is to implement an integrated financial management within two years. The Development Plan, however, calls for establishing project milestones and cost estimates at the completion of phase one. At that time, the agency should know if a two year schedule is feasible.
Our contacts and research showed that user involvement is standard practice. User participation creates increased ownership and project commitment. Because users are the most familiar with the organization and its processes, they can identify their needs and assess needed changes. The City of San Diego established standard operating procedures that require the implementation team to request information from all potential user groups including management, users, oversight groups, and technical personnel. Information gathered allowed the implementation team to categorize user needs as mandatory, non-mandatory, or optional.
Mandatory requirements are ones that the agency must meet, while non-mandatory requirements are items that help facilitate a process or report. These latter requirements normally relate to features that users have learned or depend on to make the application work more smoothly. Optional items are convenience features such as e-mail, calenders, and music. An implementation team must be able to meet 100% of the mandatory requirements. Implementing non-mandatory and optional requirements involve the discretion of the implementation team with input from users and senior management.
During interviews with the Department of Agriculture, the City of San Diego, and a banking institution, we were told that effective communication is critical to successful system development and implementation. We believe such communication allows the team to clearly understand the needs of management and users. With effective communication, senior management is informed on key information such as project status and cost. In addition to a strong agency commitment, communication helps alleviate the natural resistance to change and provides for greater acceptance of the new system.
Another aspect of effective communication is the need to document key decisions. We believe at a minimum decisions affecting user needs, milestones and costs, and system modifications must be documented.
Project Implementation Issues
The most resource intensive phase of the project is implementing commercial off-the-shelf (COTS) software. OIG issues identified in this area include defining the system and system acquisition.
Defining a Single, Integrated Financial Management System
Because the level of integration varies among agencies, each agency must define its own integrated financial management system. As NRC defines its system, we believe, as a minimum, the agency should consider its requirements for electronic data interchange, the type and frequency of financial reports, and the ability to produce financial statements.
The CFO Council's report explains that the concept of a single, integrated financial management system differs from agency to agency. While no one definition exists, the CFO report stated that many private companies stress a high degree of integration, with the objective of maximizing integration.
In 1994, JFMIP recognized the difficulties of defining specific financial management terms. JFMIP, therefore, sponsored the development and the issuance of the report entitled Framework for Federal Financial Management Systems. This document attempts to define a "single, integrated financial management system" and provide implementing guidance. It defines such a system as:
"A unified set of financial systems and the financial portions of mixed systems encompassing the software, hardware, personnel, process (manual and automated), procedures, controls, and data necessary to carry out financial management functions, manage financial operations of the agency, and report on the agency's financial status to central agencies, Congress, and the public. Unified means that the systems are planned for and managed together, operated in an integrated fashion, and linked together electronically in an efficient and effective manner to provide agency-wide financial system support necessary to carry out the agency's mission and support the agency's financial management needs."
System Acquisition Planning
We identified several areas that NRC should explore during system planning that would help ensure the ultimate success of this key phase. They are: lessons learned, business process reengineering/software modifications, vendor demonstrations, vendor support, and independent verification.
As the agency begins to develop and implement its financial management system, a good first step is to learn from its own successes and failures and those of others. In the past, NRC has encountered problems when developing and implementing management information systems. These problems have included defining users, defining and documenting user requirements, cost management, and implementation delays.
Several federal agencies are currently developing integrated financial management systems. Agencies with whom we spoke were all receptive to discussing their experiences with us. For contract procurement information, representatives from GSA's Financial Management System Software Schedule can provide valuable information. The schedule provides sources for COTS software that comply with JFMIP core financial system requirements. The use of this schedule, established in 1989, is currently mandatory for all agencies in the Executive Branch. The contract allows for procurement actions using a letter of interest (LOI) rather than a request for proposal. Theoretically, the LOI is simpler to use than a traditional request for procurement because clauses covered by the master contract need not be repeated in the LOI. NRC representatives could use LOIs from other agencies, such as the U.S. Court System, as learning tools as it begins to develop its own document.
The objective of a project team in preparing a LOI should be to ensure a successful and timely acquisition. Key aspects of the LOI to meet that goal include:
Define the scope and requirements (layout the expectations).
Define the evaluation process.
Require a demonstration (advise bidders that the agency will develop and control the requirements for the demonstration).
Avoid the possibility of a protest
Business Process Reengineering/Software Modifications
Instead of tailoring its processes to the software available, federal agencies traditionally modified COTS software to meet "unique agency requirements." This practice has been too costly, and according to the CFO Council is "the polar opposite of today's best practices in the private sector."
According to the CFO Council document, the private sector experience provides several useful lessons for federal financial managers. First, a key private sector goal was to reduce the costs of their accounting systems to stay competitive. Second, reengineering existing processes seemed to lead all financial systems efforts. Third, there was unanimous agreement that COTS software should not be modified. Modifying the basic software to accommodate individual or special considerations should normally not occur. Rather, special needs should be met by attaching a module to the basic software. Agencies in the market for COTS software are now trying to limit the need for modifying software.
Another reason to evaluate and reengineer a process is to avoid paving cowpaths. "Paving cowpaths means using information technology to automate existing processes without improving the workflows or other components of these operations. Often, the result is to do the wrong things faster. A natural tendency exists for financial personnel to want to automate processes exactly the way they are now."(1)
Business process reengineering can provide solutions to improve and streamline processes. However, some agencies did not take that path, but opted to take the most expedient approach of modifying software to accommodate existing processes. In these cases, the agency often purchased a faster processing box for antiquated processes.
Several sources including GSA and the U.S. Court System stated that vendor demonstrations are an important step in the system acquisition phase. An agency should not rely on system descriptions and sales pitches. Rather an agency should test the system using a controlled transaction-based test -- a test that is more rigorous than software demonstrations. Software demonstrations usually use only vendor data and do not address detailed agency requirements. A controlled transaction-based test uses an agency developed test transactions with expected results.
Based on our interviews and research, we believe that adequate training, system documentation, and avoiding extensive software modifications may reduce future reliance on a contractor. In its summer 1996 issue, The Government Accountants Journal(2) reported that continuing contractor involvement is, in part, dependent on the adequacy of training. Adequate training (1) ensures proper operation and use of the new system, (2) helps the agency take control of its new system, and (3) lessens dependence on the software vendor. Through adequate training, agency personnel may save money by assuming some vendor functions.
Closely related to training is the need for adequate user documentation and technical procedures. Documentation and procedures must meet the needs of the technical support, system operations personnel and users. In the CFO Council sponsored conference on Upcoming Changes in Federal Financial Management Systems, the U.S. Courts System reported that detailed vendor documentation was a critical factor in assessing vendor competence.
Avoiding costly software modifications will also help reduce contractor reliance. When vendors release software updates, modifications made to the original COTS software must also be updated. Not only can this be costly, it can also lead to continual contractor involvement. While some level of continuing contractor support is realistic, an agency needs to consider the future costs of comprehensive support.
According to a GSA official, an independent verification and validation contractor can also provide support during the implementation phase of the project. The independent verification and validation contractor should monitor the vendor's implementation work as well as perform acceptance testing. This will help ensure that the agency is getting what it is paying for from the software vendor. This function may be performed in-house, if an agency has the requisite expertise.
Our work identified several best practices that can lead to the successful development and implementation of a single, integrated financial management system. While we recognize that NRC's development plan addresses many of the identified issues, we wanted to assist the agency by providing results of our survey of best practices. In summary, our survey found four best practices which can aid in achieving a successful project. They are: (1) ensuring top level commitment and involvement, (2) learning from the experiences of others, and (3) minimizing software modification. Finally, while we recognize that there needs to be a sense of urgency about attaining an effective integrated financial management system, we also found that establishing realistic milestones and cost estimates is equally as important. Because this system will have agency-wide financial management implications, we will continue to monitor the development and implementation of this project.
Major Contributors to this Report
Anthony C. Lipuma
Michael A. Cummins
Glossary: Office of the Inspector General Products
1. Investigative Report - White Cover
An Investigative Report documents pertinent facts of a case and describes available evidence relevant to allegations against individuals, including aspects of an allegation not substantiated. Investigative reports do not recommend disciplinary action against individual employees. Investigative reports are sensitive documents and contain information subject to the Privacy Act restrictions. Reports are given to officials and managers who have a need to know in order to properly determine whether administrative action is warranted. The agency is expected to advise the OIG within 90 days of receiving the investigative report as to what disciplinary or other action has been taken in response to investigative report findings.
2. Event Inquiry - Green Cover
The Event Inquiry is an investigative product that documents the examination of events or agency actions that do not focus specifically on individual misconduct. These reports identify institutional weaknesses that led to or allowed a problem to occur. The agency is requested to advise the OIG of managerial initiatives taken in response to issues identified in these reports but tracking its recommendations is not required.
3. Management Implications Report (MIR) - Memorandum
MIRs provide a "ROOT CAUSE" analysis sufficient for managers to facilitate correction of problems and to avoid similar issues in the future. Agency tracking of recommendations is not required.
4. Audit Report - Blue Cover
An Audit Report is the documentation of the review, recommendations, and findings resulting from an objective assessment of a program, function, or activity. Audits follow a defined procedure that allows for agency review and comment on draft audit reports. The audit results are also reported in the OIG's "Semiannual Report" to the Congress. Tracking of audit report recommendations and agency response is required.
5. Special Evaluation Report - Burgundy Cover
A Special Evaluation Report documents the results of short-term, limited assessments. It provides an initial, quick response to a question or issue, and data to determine whether an in-depth independent audit should be planned. Agency tracking of recommendations is not required.
6. Regulatory Commentary - Brown Cover
Regulatory Commentary is the review of existing and proposed legislation, regulations, and policies so as to assist the agency in preventing and detecting fraud, waste, and abuse in programs and operations. Commentaries cite the IG Act as authority for the review, state the specific law, regulation or policy examined, pertinent background information considered and identifies OIG concerns, observations, and objections. Significant observations regarding action or inaction by the agency are reported in the OIG Semiannual Report to Congress. Each report indicates whether a response is required.
1. Phillips, Warren R., Bonnie L. Brown, C. Morgan Kinghorn, Andrew C. West, Public Dollars, Common Sense. Washington DC; Coopers & Lybrand, 1997. p.63.
2. Myaing, Robin T., The Government Accountants Journal, Summer 1996, pp. 24-28.