Report 2022-114 All Recommendation Responses

Report 2022-114: California Department of Technology: Weaknesses in Strategic Planning, Information Security, and Project Oversight Limit the State's Management of Information Technology (Release Date: April 2023)

Recommendation for Legislative Action

The Legislature should revise state law to clarify CDT's role, responsibilities, and priorities for strategically guiding the State's acquisition, management, and use of IT. The revised priorities should require CDT to do the following:
Follow best practices in its 2024 strategic plan and all future strategic plans by developing measurable objectives to achieve goals and incorporating performance measures for those objectives. Further, it should pursue accountability by monitoring the State's progress toward achieving the plan's goals.

Recommendation for Legislative Action

The Legislature should revise state law to clarify CDT's role, responsibilities, and priorities for strategically guiding the State's acquisition, management, and use of IT. The revised priorities should require CDT to do the following:
Develop a plan by July 1, 2023, for satisfying its statutory requirement to identify, assess, and prioritize modernizing high-risk, critical IT systems.

Recommendation for Legislative Action

The Legislature should revise state law to clarify CDT's role, responsibilities, and priorities for strategically guiding the State's acquisition, management, and use of IT. The revised priorities should require CDT to do the following:
By March 2024, develop and maintain an inventory of the State's IT systems or components of systems that agencies can reuse to avoid duplication of efforts.

Recommendation for Legislative Action

The Legislature should require CDT to create and lead an interorganizational task force to assess IT staffing problems in the State and to issue recommendations to increase the State's hiring and retention rates of highly qualified IT personnel. The task force should be composed of CDT staff, state IT staff, and state human resources staff.

Recommendation for Legislative Action

The Legislature should require CDT to develop a plan for determining the overall statewide information security status of the State's reporting entities by January 2024. This plan may entail CDT's assessing reporting entities through its existing oversight lifecycle or through alternative processes. It may include increasing the number of CDT staff, revising CDT's review process, or pursuing enforcement measures and corrective actions for reporting entities that do not address information security deficiencies. For example, when appropriate, CDT could require reporting entities to address outstanding information security deficiencies before implementing new IT initiatives.

Recommendation for Legislative Action

The Legislature should make changes to improve the independence of the State's IT project oversight. One option it could consider is creating a new state entity, such as an independent board, that is specifically tasked with certain oversight responsibilities for IT projects. If the Legislature pursues this option, the majority of the board members should be selected independently of the Governor by, for example, leaders of the Legislature or other elected state officers. The board could include representatives from state agencies, the Legislature, and the private sector. Alternatively, CDT could continue to perform its oversight responsibilities and the Legislature could create a committee to review CDT's oversight reports. The new board or committee should be tasked with making recommendations to CDT about the remedial measures and corrective actions that CDT should require of the agency performing the project to resolve problems in a timely manner, as well as recommendations about suspending, reinstating, and terminating IT projects. The new oversight board or committee should report regularly to the Legislature and project stakeholders on each project's progress in meeting its approved objectives.

Recommendation for Legislative Action

If it decides to create a new oversight board or committee, the Legislature should ensure that board or committee's ability to provide effective oversight by requiring it to do the following:
Include, in the project oversight reports, substantive analyses of the key indicators of a project's progress—such as schedule, scope, cost, and staffing resources—that are based on the original approved project plan. The oversight reports should also identify any changes made to the project plan by a special project report, a contract amendment, or department change orders.

Recommendation for Legislative Action

If it decides to create a new oversight board or committee, the Legislature should ensure that board or committee's ability to provide effective oversight by requiring it to do the following:
Establish a knowledge group composed of IT industry experts, CDT staff, agency information officers and chief information officers, and state policymakers to establish clear, data-driven guidelines and metrics for suspending, reinstating, and terminating IT projects to decrease the frequency and severity of IT system failures, cost overruns, delayed implementation, and limited functionality. The knowledge group should base the guidelines on industry best practices for determining IT project success.

Recommendation for Legislative Action

If it decides to create a new oversight board or committee, the Legislature should ensure that board or committee's ability to provide effective oversight by requiring it to do the following:
Periodically analyze the lessons learned that are included in agencies' post implementation evaluation reports to identify trends or patterns. The new oversight board or committee should also require state agencies to complete post implementation evaluation reports for projects that are terminated before implementation. The board or committee should use the information from both types of reports to improve its oversight processes.

Recommendation #10 To: Technology, California Department of

To ensure that it consistently applies best practices when conducting strategic planning, CDT should develop a policy or procedure that documents the required elements of its strategic plan. These elements should include key goals, strategies for achieving those goals, measurable objectives, performance measures, and processes to monitor progress.

6-Month Agency Response

In July, CDT published the California State Digital Strategy: An Innovation Roadmap. This strategy provides an actionable framework for California government and education systems to innovate using technology to improve resident experiences and services for their residents. It is another example of statewide strategies that ensure work occurring at each state department is in alignment with Vision 2023. In addition, CDT has awarded a contract with a vendor to assist in developing the state's next technology strategic plan. The vendor's work will include developing the new strategic plan with measurable outcomes and helping to improve CDT's annual report process as well as updates to policy and procedure documentation to ensure best practices are followed for all future strategic planning efforts. Current efforts also include a close-out of the current Vision 2023.

California State Auditor's Assessment of 6-Month Status: Pending

We look forward to reviewing CDT's new strategic plan to ensure it includes elements such as key goals, strategies for achieving those goals, measurable objectives, performance measures, and processes to monitor progress.

60-Day Agency Response

CDT will update policy to highlight each department's CIOs responsibility to develop and maintain an IT Strategic Plan which aligns with their department's mission and the California Department of Technology Statewide IT Strategic Plan.

Furthermore, an annual progress report process and template will be developed that enables CDT to provide a year-over-year comparative analysis of key performance indicators of statewide adoption and progress toward the statewide IT strategic plan. CDT will leverage our current metrics, such as CalSecure and Broadband Action Plan and Digital Strategy, and develop new measurements, where applicable.

CDT seeks a vendor partner to assist in evolving the vision, mission, goals, and guiding principles for IT organizations statewide. A solicitation was published on 5/19/2023, with a response due 6/30/2023.

California State Auditor's Assessment of 60-Day Status: Pending

Recommendation #11 To: Technology, California Department of

To expand its knowledge of threats to the State's information security and more effectively leverage the State's resources for threat monitoring, CDT should perform increased outreach with reporting entities. Specifically, CDT should learn what reporting entities are currently doing for monitoring and alerting other agencies of cybersecurity threats and educate them about its no-cost threat monitoring service.

6-Month Agency Response

There are six state entities verified to have existing mature SOCs that perform 24x7 monitoring and alerting and are out of scope for our services at this time. Regarding all others, OIS has surveyed departments regarding their security programs. At least 132 departments have completed these surveys providing insight into their security solutions, reporting, and compliance. These survey results have provided insight into how organizations manage essential security solutions. We continue to proactively reach out to state entities to discuss our monitoring service. Since the last update we have increased enrollment from 18 to 51. Since the last update, we have also provided introductory presentations for over 60 organizations interested in learning more about how the service works and how to onboard. We recently published a major update to CDT's SOCaaS website to provide more thorough and detailed information about our SOCaaS monitoring service.

Additionally, please see the attached (excel spreadsheet) document which lists our P0 or outreach meetings with customers where we describe our service. The spreadsheet also has email communications where we still have the date.

The chart below shows the significant increase in customers in the past 10 months:

SOCaaS Customers - Jan. 2022 to Oct. 2023

Total Depts 2022 Cust's Current Cust's % Increase

159 18 51 21%

Reporting 106 15 32 113.33%

Non-Reporting 5 0 1 N/A

Independent 26 1 7 113.33%

Constitutionals 9 0 1 N/A

Un-affiliated 6 1 3 200.00%

Non-State 7 1 7 600.00%

California State Auditor's Assessment of 6-Month Status: Partially Implemented

CDT has made some progress, but has not fully implemented this recommendation.

60-Day Agency Response

During the next year CDT will implement a plan with processes and events to perform increased interaction with State reporting entities. CDT will increase its efforts to alert State entities/agencies of cybersecurity threats and educate these same entities about its no-cost threat monitoring service during this same time period. CDT has plans to increase its outreach with State reporting entities. The Office of Information Security will continue to hold online meetings and training to assist entities with threat monitoring.

In addition, CDT's Office of Information Security has a team that meets with reporting entities and works side by side to ensure best practices are used for monitoring cybersecurity threats. CDT also educates the entities about its no-cost threat monitoring service. CDT will continue to promote the continuous monitoring Security Operations Service monthly at the Security Advisory Committee meetings. In addition, progress reports on detection effectiveness are being provided to the departmental ISOs. To date, 25 entities have been onboarded in an optimized state, with an additional 20 that are sending internal security logs and are currently being optimized.

California State Auditor's Assessment of 60-Day Status: Pending

Recommendation #12 To: Technology, California Department of

To improve the effectiveness of the PAL process at ensuring the success of projects, CDT should take the following actions:
Revise the PAL process to promote the use of modern approaches, such as modular or agile, when developing new systems. Further, CDT should maintain awareness of new development approaches and update its approval process to encourage their use, whenever feasible.

6-Month Agency Response

OSPD continues to evaluate changes to the Project Approval Lifecycle (PAL). Some changes have been implemented in the last six months. Other changes intended to further improve and streamline the planning and oversight processes are underway. For example, Financial Analysis Worksheets (FAW) were updated in May 2023 to allow for tracking operational, M&O costs within the project lifecycle. Another FAW project effort also kicked off to modernize the FAW from using manually populated spreadsheets to an improved digital solution enabling better cost planning and tracking for modern development methods.

OSPD has also initiated the Project Services Modernization effort and created a roadmap for improvements to the planning, procurement, and project oversight processes.

OSPD is also engaged in another pilot effort with departments to enhance project change and reporting mechanisms to allow for modern development methods using an Iterative Project Report (IPR) similar in nature to the Special Project Report (SPR) but designed to enable adaptive project development methods (agile, iterative, incremental, etc.).

Finally, Departments are now required to identify their implementation approach in the planning documentation submitted to OSPD and check points have been added to the project approval and oversight processes.

California State Auditor's Assessment of 6-Month Status: Pending

60-Day Agency Response

In the next year, CDT/OSPD will have updated PAL policies, procedures, and processes to identify opportunities for the greater use of modular or agile development approaches. OSPD will enhance its project-approach monitoring including performance trends and project outcomes. Annually, OSPD will assess new modern approaches and partner with PDAC and ITEC to determine if the state should adopt.

CDT's Office of Statewide Project Delivery (OSPD) initiated a review and identified changes to the project approval lifecycle (PAL) and will also review policy for possible changes. OSPD will also determine funding/resource availability during this time period. CDT, as appropriate, will establish processes to require state entities to include their implementation approach in their planning alternatives. CDT will add the necessary checkpoints in its portfolio and project approval and oversight processes over the next few months.

California State Auditor's Assessment of 60-Day Status: Pending

Recommendation #13 To: Technology, California Department of

To improve the effectiveness of the PAL process at ensuring the success of projects, CDT should take the following actions:
Revise the PAL process to require agencies to ensure, and CDT to verify, that proposed projects align with statewide strategic initiatives so that all approved projects are contributing to the State's strategic goals.

6-Month Agency Response

CDT has fully implemented Procedure/Standards (PS-027) and issued the portfolio template to include instructions and data capture of statewide strategic alignment.

CDT has reviewed policy and procedure areas and determined no State Administrative Manual (SAM) policy changes are required but has enacted procedures (State Information Management Manual (SIMM) and internal) enhancements to further ensure projects clearly align with Vision 2023, the statewide technology strategic plan.

CDT has also revised its budget change proposal (BCP) review, project planning and portfolio instructions and checklist templates to verify and validate that projects align with Statewide Strategy.

California State Auditor's Assessment of 6-Month Status: Pending

60-Day Agency Response

OSPD will develop a Statewide IT Strategic Plan checklist which requires departments to identify which goal/initiative their project aligns to. Prior to approval, OSPD will include OTech, ODS, EA, and other CDT program experts to ensure that the project and proposed technical solution align with CDT strategies, including but not limited to State's Digital Strategy, CalSecure, and statewide technical architecture and standards.

CDT issued a Procedures/Standards Update (PS-027) to its portfolio report template, including specific reporting and guidance on statewide strategic alignment.

CDT will review and determine where additional policy and procedure changes (e.g., PAL, Agency Portfolio, BCP review) to further ensure projects align with statewide strategic initiatives during the budget review and planning phases.

CDT/OSPD will revise the budget change proposal review, project planning, and Agency Portfolio instructions and templates checkpoints to verify and validate that projects align with Statewide strategic initiatives.

California State Auditor's Assessment of 60-Day Status: Pending

Recommendation #14 To: Technology, California Department of

To improve the effectiveness of the PAL process at ensuring the success of projects, CDT should take the following actions:
Develop internal metrics that include information on each project's size, the timeliness with which a solution was procured, the length of time to complete each stage of PAL, the degree to which an implementation was successful, and the degree to which the project was completed on time and within budget. CDT should trend the results of these internal metrics over time and include them in its annual report to the Legislature.

6-Month Agency Response

CDT/OSPD has implemented tools and metrics to provide reporting on each project's size and the degree to which the project was completed on time and within budget and trend analysis. CDT/OSPD has piloted (currently in testing and data population) a searchable lessons-learned database to enable the State to evaluate the degree to which an IT project implementation is successful.

CDT/OSPD has also enhanced its tools to track and report on the length of time to complete each PAL stage including the timeliness with which a solution was procured. CDT OSPD continues to report on the degree to which a project is successful in the Final Independent Project Oversight Report (IPOR) and Post Implementation Evaluation Report (PIER).

Finally, CDT/OSPD continues to enhance tracking and reporting tools to provide project information on each project size (in cost), the length of time to complete each PAL stage, the degree to which a project was successful, and the degree to which a project was completed on time and within budget.

California State Auditor's Assessment of 6-Month Status: Pending

60-Day Agency Response

OSPD will develop PAL dashboard with pertinent metrics, which will be reviewed by CDT directorate and OSPD leadership bi-annually for process improvements.

CDT/OSPD has implemented tools and metrics to provide reporting on each project's size and the degree to which the project was completed on time and within budget and trend analysis. CDT/OSPD has implemented a searchable lessons-learned database to enable the State to evaluate how successful an IT project implementation is.

CDT/OSPD will continue developing metrics and tool enhancements to report on the timeliness with which a solution was procured, the time to complete each PAL stage and the degree to which implementation was successful.

CDT/OSPD's goal is to implement enhanced metrics and tools to provide information on each project's size, the timeliness with which a solution was procured, the length of time to complete each PAL stage, the degree to which implementation was successful, and the degree to which the project was completed on time and within budget; and Trend analysis and reporting of project success metrics over time in an annual report to the legislature.

California State Auditor's Assessment of 60-Day Status: Pending

All Recommendations in 2022-114

Agency responses received are posted verbatim.