Report 2014-105 Summary - March 2015
Los Angeles Department of Water and Power - Consequences Linked to Its Premature Launch of Its Customer Information System May Push Total Costs Beyond $200 Million
Our audit of the Los Angeles Department of Water and Power's (department) customer information system (CIS) highlighted the following:
- Over time, the department more than doubled its original budget for implementing CIS—from $87 million to nearly $181 million.
- The department minimized or ignored the severity of issues raised regarding CIS's readiness before launching it.
- The department's quality assurance expert warned that no aspect of the project was ready.
- The department shared little to no specific information with the Los Angeles Board of Water and Power Commissioners (board) about the severity of CIS's issues, limiting the board's oversight ability.
- A recent department-sanctioned analysis confirmed that the department's decision to launch CIS was premature and its workforce unprepared.
- The department is still struggling—more than a year later—to normalize important business practices and to collect unpaid accounts.
- Although the department has improved its ability to issue timely bills based on actual meter reading, its customers owe more than $681 million as of November 2014 for past-due bills.
RESULTS IN BRIEF
In early September 2013 the Los Angeles Department of Water and Power (department) launched its new customer information system (CIS) to provide billing and customer service functions for its roughly 3.8 million customers. Following more than three years of integration and testing, CIS replaced a 40-year-old, highly customized system that was technologically outdated. However, numerous and profound problems with CIS arose immediately after its implementation, reasonably causing the public to question whether the system was actually ready for everyday use. In fact, both independent quality assurance reports and the department's own assessment of the system's readiness demonstrate that the department minimized or ignored the severity of the issues that existed at the time it made the decision to launch CIS. Further, in the months leading up to the launch, the department consistently failed to disclose the issues with CIS's development when presenting the status of the Customer Information System Connection project (project)1 to the Los Angeles Board of Water and Power Commissioners (board), effectively denying the board the opportunity to delay the system's implementation.
The department originally budgeted $87 million for implementing CIS; however, it more than doubled that budget to nearly $181 million over time. Nonetheless, immediately after CIS's launch, it became clear that the system was not yet ready and that the department's decision to implement it was questionable at best. Consequently, the department's customers began complaining of late utility bills, unwarranted shut-off notices, and excessive wait times to speak with customer service representatives.
Although the department had received feedback about CIS's significant problems before its launch, the department chose not to heed this information. Reports from the department's quality assurance expert warned that no aspect of the project was ready; in fact, the quality assurance expert reported that the project's scope, quality, and schedule were all at the lowest possible rating and needed immediate attention. However, the department shared little to no specific information with the board about the severity of CIS's issues, limiting the board's oversight ability. A recent department-sanctioned analysis of the causes of CIS's failures confirmed that the department's decision to launch CIS in September 2013 was premature and that the department's workforce was not prepared to ensure the system's proper operations.
More than a year later, the department is still struggling to normalize important business practices and to collect unpaid accounts. Although the department has improved its ability to issue timely bills based on actual meter readings, its customers owe more than $681 million as of November 2014 for past-due bills, an increase of over $245 million from July 2013, two months before CIS was launched. This increase in past-due bills is linked to the department's attempts to control its long call wait times. Specifically, the department reduced the number of past-due accounts it placed in collections after the launch of CIS because customers whose accounts are in collections frequently call to resolve their payment issues. In other words, the department limited its efforts to collect past-due bills in order to decrease its call volume and thus reduce its long call wait times. By September 2014 the department had already spent $187 million on implementing and stabilizing CIS. If it is unable to resolve and ultimately collect on its past-due accounts, these uncollectible debts, linked in part to the department's launching the system prematurely, could add in excess of $40 million to CIS's overall price tag.
In the year following CIS's launch, the department was also ineffective at addressing system issues. Specifically, it has yet to resolve a number of defects—issues that cause a system to not perform as expected by negatively affecting its operations—in CIS that existed before it was launched. Some of these continuing defects are severity level 1 defects, the most significant because they interrupt or make a system's normal operations impossible. The department's recent steps to address CIS's remaining defects and other system issues potentially involve its entering into another multimillion dollar contract. In November 2014 the department issued a request for proposal for work valued at between $13 million and $15 million over three years. The department has asserted that the request for proposal is for new work, but we question this characterization. Our IT expert noted that the request for proposal includes an assessment of the current state of CIS—which has been in use for less than two years—and that it suggests that the original implementation was incomplete and incorrect and requires remediation, which the request for proposal intends to address.
Despite its collections difficulties, the department's current financial situation appears to be sound. In fact, in December 2014, the department's board approved a transfer of $265.6 million to the Reserve Fund of the city of Los Angeles, which is consistent with the amount it has transferred in previous years. Nonetheless, the department's poor decision making and poor communications with its board regarding the launch of CIS may unnecessarily cost it millions of dollars from unpaid customer bills—costs that it will ultimately need to either absorb or pass on to its customers in the form of rate increases.
To ensure that the board can more effectively exercise oversight for the department's significant information technology projects, the board should take the following actions:
- Establish a standing committee composed of board members to oversee and critically evaluate the status of the department's various information technology projects.
- Develop reporting standards for the department's management to follow when discussing the status of information technology projects. Such reporting standards should, at a minimum, specify the frequency with which the department's management makes such reports and require the following disclosures about each information technology project:
- The amount of project growth, in terms of both budget and scope of work, from initial project estimates through current projections.
- The results from system testing and a listing of the critical defects that exist and must be fixed prior to system use.
- The concerns the quality assurance contractor has raised and how the department is addressing them.
- Develop a process for the board to designate certain information technology projects as having a potentially significant effect on business operations or customer relations, and require that department managers first obtain the board's approval before launching such critical new systems.
In its response to the audit, the board stated that it and the department agreed with the recommendations; however, it stated that it disagreed with the basis on which we made them. The board asserted that the department had not misled it about the project, but, rather an ongoing independent investigation has preliminarily found that a vendor hired to assist in implementing CIS intentionally misled the department. The board noted that it has made progress in addressing concerns with CIS and will continue to work with all interested parties to ensure issues this audit raised are addressed.
1 While the new system was under development, it was referred to as the Customer Information Service Connection project. Since its launch, it has been called simply CIS. In this report, we use the term "project" to refer to the system during its development and "CIS" to refer to it after it was launched.