Our review of the California State University's (university) Common Management System (CMS) revealed the following:
With 23 campuses and an annual budget of more than $5.2 billion, the California State University (university) provides a broad education along with undergraduate and graduate instruction for professional and occupational goals to more than 400,000 students each year. Supporting this instruction is a structure for handling numerous administrative details, including the following: student services such as admissions, registration, and grades; human resources activities of processing and paying faculty, classified staff, and student workers; and financial services related to purchasing, billing, inventory, and accounting for funds. To enhance administrative productivity and quality, the university is developing and implementing an administrative software project, the Common Management System (CMS), which is replacing all university finance, human resources, and student administration systems with the PeopleSoft suite of administrative software. The university plans to support this software with one outsourced data center to process all CMS data. One expected benefit of CMS is enabling students to go online to look up admissions status and grades. Recent estimates for the CMS project's total costs are about $662 million-$393 million for one-time costs and $269 million for maintenance and operations-for the nine-year development and implementation period from fiscal year 1998-99 through 2006-07.
The university originally anticipated significant cost savings from its CMS and data center efforts, even believing these savings could fund other information technology initiatives. It now says it did not have cost savings as a primary goal, but expected CMS would avoid or minimize costs for improved and expanded administrative software services over the long term. Regardless of its reasons for pursuing CMS and its supporting data center, the university has not established a business case for the CMS project by preparing a feasibility study report or a similar analysis that clearly defined its intended benefits and associated costs when compared with the pre-CMS environment. In addition, when the university decided to direct a single administrative system at all campuses, it did not evaluate the cost variance in alternatives that allowed for implementation plans ranging from running one shared copy of the software to running several copies focusing on differing campus needs. Without compelling evidence of its need for new administrative software, or a cost-benefit analysis for the project, the university cannot ensure that the CMS project is a worthwhile expenditure of resources.
When asked why it never conducted a formal return-on-investment analysis on the CMS project, the university explained that the magnitude of potential savings estimated by its consultants, IBM and Pacific Partners Consulting Group (Pacific Partners), led it to believe that such a formal analysis was unnecessary. However, the university cannot rely on its consultants' reports as justification for not developing the business case for CMS because these studies were not intended for such a purpose-as evidenced by their scope. In fact, both studies recommended that the university conduct subsequent cost-benefit analyses before proceeding with the project. Further, although the university may have intended to conduct its own cost-benefit analysis for the CMS project, it never completed one.
Also, the university's stated reasons for CMS, given in its "Why CMS?" document, are insufficient to justify a significant investment without analysis demonstrating that the problems it described were severe or systemic. Further, the university cannot support that most of its campuses were planning to replace prior administrative systems in July 1999 when the decision was made for all campuses to implement CMS. The university's chancellor cites various reasons for pursuing CMS, such as drawing all campuses into a more common set of business practices. However, these reasons are inadequate to forgo documenting the problems being addressed by CMS and the associated costs to fix them. The strategic benefits the chancellor is hoping to achieve come at a significant cost annually; the most recent university estimates indicate it will cost more than $65 million each year to maintain the fully implemented CMS software.
Recent data indicate that the university's 1998 cost projections of $332 million to $400 million and 1999 projection of $440 million for its CMS project understated the project's costs. Because it did not collect actual project cost information from campuses, the university, at our request, expedited a comprehensive cost survey of actual CMS expenditures and projections. This survey revealed that the total project cost for the types of expenses it initially estimated-what the university considers to be "new" costs-now total $482 million. The $482 million includes maintenance and operations of the data center during the implementation period, but it excludes certain campus project costs the university did not estimate. The university maintains these are not "new" costs. They include $63 million in implementation costs charged to other campus budgets and $117 million related to campus maintenance and operations costs for CMS during the nine-year project period. However, the accuracy of these figures is uncertain given that 73 percent of the projected $662 million in implementation and maintenance and operations costs through fiscal year 2006-07 is estimated.
Additionally, the university has not established a mechanism to monitor overall systemwide project costs adequately, contributing to a lack of complete project cost information for university management and for the Legislature. The university told us it planned to gather cost data from campuses in fall 2002. After we expressed our concern about the incomplete information, the university surveyed campuses in June 2002 for this cost information. Also, the university lacks a systemwide funding plan for the CMS project and, therefore, lacks a full picture of how the project may affect future funding priorities.
Further, we noted problems that cast doubt on whether CMS will achieve all the objectives the university intended, nor offer what could have been achieved from such a systemwide project. Some problems stem from the university's weak efforts early in the planning process. For instance, one business objective was to minimize costs and time to implement and maintain the software. One of the ways the university intended to do this was by limiting modifications to the vendor software to only those needed to meet its business needs. However, the university had no basis to anticipate the modifications it needed to make because, before it purchased the software, it did not sufficiently evaluate its specific business processes to understand which business processes the potential vendors' software products could accommodate and which software products would require modification to meet its business needs. Additionally, it must often continue to reapply these modifications when the vendor software is updated, thus increasing the costs to maintain the CMS software. Further, the university plans to continue to use existing processes for systemwide reports because it did not design CMS to replace these processes. Finally, its piecemeal approach of identifying, procuring, and implementing its own CMS solution did not share risk with vendors and consultants. Thus, it assumed substantially all the considerable financial and business risk involved in ensuring that the software meets its business needs and is implemented successfully.
Additionally, the CMS software procurement process raises questions about whether the university used a fair and objective competitive process. Originally, the university planned to identify one or more vendors that campuses could select, and its solicitation document did not provide a method to select only one vendor. When it decided late in the process to recommend one software vendor, the university did not use a quantitative scoring process to select objectively between the two finalists and could not demonstrate that it had resolved questions raised by the procurement evaluation teams. After procuring the software, the university and the campuses hired CMS consultants through sole-source contracts that appear to be contrary to its own policy on when such agreements are appropriate. The university also has not required solicitations for offers from various consulting firms under its master agreements. Without such additional offers, the university cannot demonstrate that it procured best-value services.
Unlike these procurements, the university used recommended procurement practices to select the best-value vendor when procuring the outsourced data processing services needed to run CMS. Also, it shared risk with the outsourced data center vendor by establishing contract terms aimed at holding the vendor accountable for meeting preestablished service levels. When it experienced inadequate service from the data center in the early months of the contract, the university used procedures in the contract to help raise services to agreed levels, and recent months show improvements in the levels. However, although it has worked to address its CMS data processing needs, the university only now is starting to address campus CMS data warehousing needs. Data warehousing can provide for optimum data storage and reporting, such as enabling the production of reports that contain historical analysis of university operations. Earlier in the project planning, the university removed data warehousing services from the CMS project scope. It is now revisiting and starting to address campus interest for those services.
Finally, the university did not do enough to ensure that individuals participating in the project's procurement decisions were free from apparent conflicts of interest, casting a shadow on the project. The university did not designate certain university positions, such as some CMS project directors, as responsible for filing annual forms to disclose economic interests. Also, the university did not provide appropriate guidance to employees to identify potential conflicts. Finally, the university lacks a policy that spells out for employees what constitutes "incompatible activities" and does not require designated employees to receive regular ethics training. Conflicts of interest or incompatible activities could compromise the university's reputation for honest and fair business practices and undermine public confidence in the university's procurement decisions. In fact, we found an employee who appeared to have had a conflict of interest while participating in a CMS procurement, and one employee who possibly may have used nonpublic information to benefit personally.
The university should adopt policies and procedures that require a feasibility study before the acquisition and implementation of significant future information technology projects.
The university should ensure that it monitors systemwide project costs adequately, including establishing a mechanism to collect data on campus costs periodically. Also, it should establish a systemwide funding plan for CMS that includes campuses.
The university should take steps to ensure that it meets its business objectives for the CMS project, including taking action to minimize the costs and time associated with implementing the software. Further, it should determine how it could improve the design of CMS to report systemwide information.
The university also should use recommended practices, such as ensuring that it shares project risk with vendors and using a quantitative evaluation method to select best-value vendors. Further, when procuring information technology systems or software in the future, the university should evaluate its specific business processes against vendor products before procurement, then select vendors that best accommodate the university's specific needs.
Finally, it needs to strengthen its procedures for preventing and detecting potential conflicts of interest for individuals participating in procurement decisions. For example, it should conduct periodic ethics training for designated employees. Additionally, it should establish an incompatible activities policy that it formally communicates to employees.
The university states that it agrees in nearly all cases with our recommendations but does not agree fully with all the findings of the audit. The university believes the audit recommendations will be beneficial for the continued development and improvement of the CMS effort, and states it has already implemented or begun to implement some of the recommendations and will be acting on the others.