Report 2014-116 Recommendations

When an audit is completed and a report is issued, auditees must provide the State Auditor with information regarding their progress in implementing recommendations from our reports at three intervals from the release of the report: 60 days, six months, and one year. Additionally, Senate Bill 1452 (Chapter 452, Statutes of 2006), requires auditees who have not implemented recommendations after one year, to report to us and to the Legislature why they have not implemented them or to state when they intend to implement them. Below, is a listing of each recommendation the State Auditor made in the report referenced and a link to the most recent response from the auditee addressing their progress in implementing the recommendation and the State Auditor's assessment of auditee's response based on our review of the supporting documentation.

Recommendations in Report 2014-116: California Department of Consumer Affairs' BreEZe System: Inadequate Planning and Oversight Led to Implementation at Far Fewer Regulatory Entities at a Significantly Higher Cost (Release Date: February 2015)

:
Recommendations to Consumer Affairs, Department of
Number Recommendation Status
15

To make certain that Consumer Affairs has a project team that consists of staff trained in managing IT projects, it should ensure that all the BreEZe project team leads obtain all required project management training.

Fully Implemented
16

Consumer Affairs should develop a process to ensure that it prepares all required project management documents and undertakes all oversight activities related to BreEZe as CalTech requires so that it can prevent or identify and monitor future problems as they arise. This includes taking steps to sufficiently respond to any concerns that the IPO specialist and the IV&V consultant raise.

Fully Implemented
17

To the extent that Consumer Affairs chooses to implement BreEZe at the phase 3 regulatory entities, it should first complete a formal cost-benefit analysis to ensure that BreEZe is a cost-effective solution to meet these regulatory entities' business needs. To make certain this analysis is complete, it should include an assessment of the potential changes these regulatory entities may require to be made of the BreEZe system and the associated costs. Consumer Affairs should complete the cost-benefit analysis before investing any more resources into the implementation of BreEZe at the phase 3 regulatory entities, and it should update this analysis periodically as significant assumptions change.

Pending
18

If Consumer Affairs determines that a new IT project is warranted in the future, it should develop a process to ensure the success of that project that includes, at a minimum, system requirements that are specific to each regulatory entity as applicable.

Fully Implemented
19

If Consumer Affairs determines that a new IT project is warranted in the future, it should develop a process to ensure the success of that project that includes, at a minimum, a project team that is qualified and experienced.

Fully Implemented
20

If Consumer Affairs determines that a new IT system is warranted in the future, it should develop a process to ensure the success of that project that includes, at a minimum, development of and adherence to all project management plans.

Fully Implemented
21

If Consumer Affairs determines that a new IT project is warranted in the future, it should develop a process to ensure the success of that project that includes, at a minimum, planning and implementation of effective organizational change management.

Fully Implemented
22

If Consumer Affairs determines that a new IT project is warranted in the future, it should develop a process to ensure the success of that project that includes, at a minimum, timely response to concerns that the IV&V consultant and the IPO specialist raise.

Fully Implemented
23

If Consumer Affairs determines that a new IT project is warranted in the future, it should develop a process to ensure the success of that project that includes, at a minimum, sufficient staffing.

Fully Implemented
29

To ensure that it complies with all terms of the BreEZe contract, Consumer Affairs should develop and follow a process for doing so, including documenting how it has met applicable contract terms, such as forwarding all change requests to CalTech.

Fully Implemented
34

Consumer Affairs should continue to work with the phase 1 regulatory entities to ensure that the issues they are facing with BreEZe are being resolved in a timely manner, with particular attention to understanding their reporting needs and improving the system's reporting capabilities.

Fully Implemented
35

To ensure that future training for BreEZe system rollouts is timely and effective, Consumer Affairs should provide training on the BreEZe system as close to the rollout date as possible to ensure that staff retain the information for using the system as it is implemented.

Partially Implemented
36

To ensure that future training for BreEZe system rollouts is timely and effective, Consumer Affairs should work with the regulatory entities to develop training that is specific to each entity's business processes.

Partially Implemented
Recommendations to Legislature
Number Recommendation Status
1

To ensure that it receives timely and meaningful information regarding the status of the BreEZe project, the Legislature should enact legislation that requires Consumer Affairs to submit a statutory report annually, beginning on October 1, 2015, that will include Consumer Affairs' plan for implementing BreEZe at those regulatory entities included in the project's third phase, including a timeline for the implementation.

No Action Taken
2

To ensure that it receives timely and meaningful information regarding the status of the BreEZe project, the Legislature should enact legislation that requires Consumer Affairs to submit a statutory report annually, beginning on October 1, 2015, that will include the total estimated costs through implementation of the system at the remaining 19 regulatory entities and the results of any cost-benefit analysis it conducted for phase 3.

No Action Taken
3

To ensure that it receives timely and meaningful information regarding the status of the BreEZe project, the Legislature should enact legislation that requires Consumer Affairs to submit a statutory report annually, beginning on October 1, 2015, that will include a description of whether and to what extent the system will achieve any operational efficiencies resulting from implementation by the regulatory entities.

No Action Taken
Recommendations to Registered Nursing, Board of
Number Recommendation Status
30

To ensure that BRN has adequate data to effectively use its resources and manage its workload, it should formally track and monitor the timeliness of its processing of applications by type and the cause of any delays.

Fully Implemented
31

To ensure that BRN has adequate data to effectively use its resources and manage its workload, it should formally track and monitor its pending workload of applications by type and original receipt date.

Fully Implemented
32

To ensure that BRN has adequate data to effectively use its resources and manage its workload, it should conduct an analysis no later than June 30, 2015, of its application processing since implementing BreEZe in order to identify the workload capability of each of its units, such as the licensing support unit; to the extent it determines additional resources are necessary, BRN should submit a request for these resources that is appropriately justified.

Fully Implemented
33

To ensure that BRN continues to process applications within regulatory time frames, it should continue its efforts to refine its business processes to increase efficiency and reduce the amount of time applications are pending its review.

Fully Implemented
Recommendations to Technology, California Department of
Number Recommendation Status
4

To help ensure the success of the BreEZe project going forward, CalTech should ensure that Consumer Affairs responds promptly to, and adequately addresses, concerns raised by its IPO specialist and its IV&V consultant.

Fully Implemented
5

To help ensure the success of the BreEZe project going forward, CalTech should require Consumer Affairs to develop and follow all project management plans and ensure that the BreEZe project team leads receive all required training.

Fully Implemented
6

To help ensure the success of the BreEZe project going forward, CalTech should document key discussions with Consumer Affairs in which significant concerns are raised about Accenture and assessments of the department's performance by the IV&V consultant and the IPO specialist.

Fully Implemented
7

If Consumer Affairs receives the necessary funding and resources to successfully implement BreEZe at the phase 2 regulatory entities and the project continues to face escalating costs, CalTech should require Consumer Affairs to conduct an analysis of the costs and benefits of moving forward with the project as planned or of suspending or terminating the project.

Resolved
8

Continue implementing the STAR project for state departments and ensure that its framework requires departments undertake detailed business process analyses before submitting FSRs to CalTech for approval.

Fully Implemented
9

Continue implementing the STAR project for state departments and ensure that its framework requires departments use detailed business process analyses to justify the type of IT solution they are proposing, such as a COTS or a fully customized system.

Fully Implemented
10

Continue implementing the STAR project for state departments and ensure that its framework requires departments use detailed business process analyses to inform the resulting RFPs and that they ensure that the RFPs contain adequate requirements.

Fully Implemented
11

To ensure that IT projects have the oversight needed to better position them for success, CalTech should require state departments to follow its IT policies, including developing all necessary plans and receiving all required training.

Fully Implemented
12

To ensure that IT projects have the oversight needed to better position them for success, CalTech should ensure that departments obtain IPO and IV&V services as soon as an IT project is approved. Additionally, CalTech should document key discussions with any department in which the IV&V consultant and the IPO specialist raise significant concerns about a project, and ensure that the respective department responds promptly to, and adequately addresses, the concerns that the IPO specialist and the IV&V consultant raise.

Fully Implemented
13

To ensure that IT projects have the oversight needed to better position them for success, CalTech should require state departments to appropriately address deficiencies identified in CalTech's approval of any SPRs as conditions that must be met, and to do so according to specified timelines. If the deficiencies are not adequately addressed within the specified timelines, CalTech should take action to suspend the project until the department has either resolved the identified deficiencies or adequately documented its justification for not addressing the deficiencies.

Fully Implemented
14

To ensure that IT projects have the oversight needed to better position them for success, CalTech should develop thresholds relating to IT project cost increases and schedule delays to inform and better justify its decision to allow an IT project to continue. If a department's IT project reaches or exceeds these thresholds, CalTech should require the department to conduct a cost-benefit analysis for the project and include this analysis in an SPR. CalTech should consider the results of this analysis in its decision to approve or deny the SPR and, if warranted, take action to suspend or terminate the project so that it does not allow projects with significant problems to continue without correction.

Pending
24

To ensure that future IT project procurements do not jeopardize the State's financial interests, CalTech's procurement division should document its reasons for approving any deviations from standard contract language.

Fully Implemented
25

To ensure that future IT project procurements do not jeopardize the State's financial interests, CalTech's procurement division should ensure that contract language gives departments the rights to the source code necessary to complete the project if a contract is terminated for convenience.

Fully Implemented
26

To ensure that future IT project procurements do not jeopardize the State's financial interests, CalTech's procurement division should ensure that contract language contains assurances that state funds will not be used for unauthorized purposes.

Fully Implemented
27

To ensure that future IT project procurements do not jeopardize the State's financial interests, CalTech's procurement division should ensure that contracting departments can appropriately manage IT maintenance support contracts by including service-level agreements in these contracts.

Fully Implemented
28

To ensure that contracting departments comply with requirements, CalTech should periodically monitor the status of contracts and enforce adherence to procurement requirements. For example, CalTech should ensure that contracting departments forward all change requests to it and that it conducts reviews of all change requests to determine whether the changes exceed the scope of the respective contract.

Pending


Print all recommendations and responses.


Report type

Report type
















© 2013, California State Auditor | Privacy Policy | Conditions of Use | Download Adobe PDF Reader