To ensure that it maximizes its use of CaRES, the board should address the structural and operational flaws that prevent identification of erroneous information and implement edit checks and other system controls sufficient to identify errors.
The Cares2 system improvements have been completed. The system successfully went live March 2017.
According to an Independent Verification and Validation (IV&V report) provided by the board, because the legacy CaRES application did not meet all business requirements and had documented database and application design issues, the board initiated a project to modify the CaRES application. This project modified the legacy CaRES application and produced Cares2. The IV&V report notes that Cares2, among other things, enhanced the reporting functionality and includes multiple master reports which provide management the ability to monitor and track critical performance issues and verify and validate compliance.
The requirements analysis, design, and development activities are complete. Project management best practices, knowledge transfer, and technical documentation activities continue. System testing, user acceptance testing, end user training, and organizational change management activities are underway. As a result of statewide security requirements and legislation regarding data privacy and security, an encryption solution is underway for the new system. The additional security and data encryption work will extend the project schedule. System go live is expected in Spring 2017.
To better ensure a successful implementation, the Board leveraged services from the California Department of Technology and in partnership conducted a complete assessment of the CaRES Modification Project in early Spring 2015. As a result, a more defined go-forward approach was developed to adequately incorporate system and user acceptance testing, training, as well as project management best practices, disciplines and documentation as applicable. The go-forward approach is scaled to meet the needs of the department and the current state of the project, while ensuring compliance with new legislative mandates, control agency IT project requirements, and mitigating risks for the State. The estimated completion for system implementation is Fall 2016.
The scheduled implementation of the final increment of the CaRES Modification Project has been delayed. Based on initial user acceptance testing earlier in 2014, the Board determined that the complexity of the improvements in this final phase required additional time for adequate system and user testing and subsequent training of all headquarter and Joint Powers staff across the State. This refinement of our testing processes and training schedule is important to the successful completion and operation of the system. We are developing these testing and training plans and will be able to report an estimated completion date by December 31, 2014.
In an effort to correct the underlying database and application architecture issues in CaRES and to ensure the maximum use of the system, the Board authorized the CaRES Modification Project in June 2010. The final phase of the project is underway and will be completed in early Spring 2014. When complete, CaRES will by fully functional, scalable and can be expanded with additional functionality.
To correct the underlying database and application architecture issues in CaRES and to ensure the maximum use of the system, the Board authorized the CaRES Modification Project in June 2010. Phase 1 of the project, the Discovery and Validation process, was completed in June 2011. Phase 2 of the project, implementation of the database and application architectural changes, is currently underway. The implementation is divided into 3 increments. Increment one will be completed by December 2012. Increments 2 and 3 will be completed in 2013. When complete CaRES will by fully functional, scalable and can be expanded with additional functionality. The estimated date of completion remains December 2013.
To ensure that the board appropriately carries out its outreach efforts, it should define the specific procedures to accomplish its action strategies for outreach and establish quantitative measures to evaluate the effectiveness of its outreach efforts.
In 2013, the Restitution Fund reserve was stable enough to continue to pursue an aggressive outreach program that had been placed on hold from 2009. The Comprehensive Outreach Plan was updated with new statistics and implementation began. The strategies and tactics created were designed following extensive research that involved; demographics and crime statistics statewide, application data collected at CalVCP, input from Joint Power agencies, stakeholders, law enforcement and marketing professionals. Furthermore, the Plan also called for the development of a baseline account of the public awareness of the program in order to measure success in future years. The updated version provided 8 primary objectives:
1. Develop metrics, awareness surveys and statistical analysis to target and evaluate communications activities.
2. Provide outreach to underserved and vulnerable populations.
3. Educate and partner with California first responders
4. Strengthen partnerships with victim service programs
5. Provide members of the legislature with information on CalVCP
6. Conduct outreach for key awareness months
7. Provide digital Outreach
8. Enhance media relations
The baseline survey was conducted in February 2014 and all aspects of the Plan are being implemented including, media relations, advertising, public service announcements, community outreach, advocate and partner outreach as well as legislative outreach. All aspects of the audit recommendations are met and fulfilled.
The Board's outreach efforts continue to be consistent with the current Comprehensive Communications and Outreach Plan. The Board recently developed and began implementing an updated 2013-2015 Strategic Communications and Outreach Plan that encompasses reaching underserved and vulnerable audiences throughout the State in an effort to raise awareness of CalVCP and its services. The plan calls for developing a baseline measurement to evaluate outreach effectiveness by capturing quantitative data on program awareness levels pre and post campaign efforts. Furthermore, the Board is leveraging funds received from a Federal Office of Victims of Crime grant to better identify underserved victims, define gaps in services and examine more innovative means to reach and educate victims of crime to the services offered by the State program. The Board expects to meet the deadlines provided in the Federal agreement of the grant by early 2014.
As noted in previous progress reports, the board has substantially complied with this recommendation. The board's Comprehensive Communications and Outreach Plan identified metrics to establish benchmarks for awareness levels, prioritized projects and targeted underserved populations. The remaining element of the recommendations that has yet to be fully implemented is the performance survey to establish a baseline from which the board may accurately measure its goals. As previously noted the board had planned to contract with a vendor in 2010 but postponed plans due to budget constraints related to the condition of the Restitution Fund. During this difficult economic climate where all but necessary spending is restrained, our plans to conduct a formal performance survey remain on hold. We will continue to conduct our outreach efforts including using tools such as social and digital media and will actively seek feedback from stakeholders and advocates as to the success of these efforts.
The board should establish a complementary set of goals designed to measure its success in maximizing assistance to victims and their families. These goals should include, but not be limited to, one that focuses on the correlation of compensation payments to program support costs and one that establishes a target fund balance needed to avoid financial shortfalls. As it monitors the goals it has created, the board should ensure that its cost structure is not overly inflexible and that it is carrying out its support activities in the most cost?effective manner possible.
To demonstrate that it makes appropriate eligibility decisions on applications, the board should ensure that it correctly considers reports from other entities, such as law enforcement, and that it sufficiently documents the basis for its decisions.
In its six-month response, the board reported that it updated training modules to include an emphasis on correctly documenting the basis for eligibility decisions. The board also reported that it provided refresher training to staff in May 2009. In its one-year response, the board reported that it has provided additional training sessions covering claim processing and eligibility determination, which includes training on evaluating information to determine eligibility and properly documenting the results. Additionally, the board reported that it completed its new procedure manual, which is accessible to staff on-line. (2010-406, p. 219)
To ensure that it complies with state regulations for notifying applicants, the board should modify its process for when it notifies applicants of decisions, or it should seek regulatory change.
The board agrees with our recommendation concerning notification of applicants of the board’s recommended decisions, and this change was incorporated into a proposed regulation package. In its one-year response, the board reported that the regulation changes were adopted in April 2009. (2010-406, p. 220)
To ensure that the board has accurate information to measure its success in meeting statutory deadlines for processing applications, it should correct the problems with the “accepted date” data field in CaRES.
To improve its processing time for making decisions on applications and for paying bills, the board should identify the primary problems leading to delays and take action to resolve them. Additionally, it should consistently document its reasons for any delays in processing applications or bills.
To ensure that the board processes appeals of denied applications within a reasonable time, it should establish written procedures and time frames.
To improve its success at obtaining requested information from verifying entities, the board should develop specific procedures for staff to use when following up with verifying entities, including appropriate time frames for following up as well as the number of attempts the staff should complete.
In its 60-day response, the board identified two problems that led to delays in processing. The first was the lack of necessary information on applications that precludes the board from beginning to process the applications. The second was the untimely submission of information from providers regarding verification information and from law enforcement regarding crime reports. The board stated that it was developing an on-line application to deal with the problem regarding incomplete applications. The board also stated that it was developing a new procedure manual that would provide step-by-step instructions for staff to follow when verifying applications and bills, including time frames for follow-up. In its one-year response, the board reported that it has completed the on-line application design and testing and successfully piloted the application in four counties. The board stated that it is planning the rollout to the remaining counties in the second quarter of 2010. The board stated that it expects the on-line application to speed processing because it provides help to applicants so they can provide all the required information. Additionally, the board reported that it has completed the procedure manual. The procedure manual provides specific guidance for when and how often staff should follow up with verifying entities. (2010-406, p. 220)
To improve its success at obtaining requested information from verifying entities, the board should continue its outreach efforts to communicate with verifying entities the importance of responding promptly to its requests for information.
To ensure that the board complies with state law requiring the program to pay only amounts not covered by other reimbursement sources, the board should ensure that staff consistently verify and document their efforts to ensure that there are no other reimbursable sources.
Additionally, the board should consistently maintain documentation of its formal approval of applications and bills.
To ensure that it maximizes its use of CaRES, the board should develop goals, objectives, and benchmarks related to the functions it carries out under CaRES that will allow it to measure its progress in providing prompt, high quality service.
To ensure that it maximizes its use of CaRES, the board should continue identifying and correcting problems with the system as they arise.
To ensure that it maximizes its use of CaRES, the board should seek input from and work with relevant parties, such as assistance centers and JP units, to resolve issues with the transition.
In its one-year response, the board stated that it is continuing its effort to maximize its use of CaRES. The board stated that it has developed a corrective action plan that it uses for identifying issues that must be addressed and is tracking the progress of issues. Additionally, the board stated that it hired a database architect to identify structural problems and provide detailed recommendations on how to address these issues in CaRES. The board expects the architect’s final assessment and recommendations in December 2009. Further, the board stated that it established a CaRES Change Control Board to review and prioritize modifications and that this is an ongoing process. The board also reported that it is in the process of developing system documentation and dependency diagrams of CaRES. Finally, the board reported that it continues to work closely with JP office staff to resolve CaRES issues as they arise. The board stated that it conducts regular conference calls with county JP offices and problems relative to CaRES are communicated and tracked in a bi-weekly operational meeting. The board also stated that it actively solicits feedback from a cross-section of representatives relative to CaRES performance problems. (2010-406, p. 222)
To ensure that it maximizes its use of CaRES, the board should develop and maintain system documentation sufficient to allow the board to address modifications and questions about the system more efficiently and effectively.
To increase the number of applicants who work through assistance centers, the board should emphasize the advantages of doing so whenever possible.
To ensure that the board effectively manages the program workload and can report useful workload data, it should develop written procedures for its management of workload.
To ensure that the board effectively manages the program workload and can report useful workload data, it should implement the reporting function in CaRES as soon as possible.
To ensure that the board effectively manages the program workload and can report useful workload data, it should establish benchmarks and performance measures to evaluate whether it is effectively managing its workload.
To ensure that the board effectively manages the program workload and can report useful workload data, it should review the applications and bills converted to CaRES from VOX that are showing excessively lengthy processing periods and determine whether problems with the data exist or whether the board has significant time processing problems.
To ensure that the board appropriately carries out its outreach efforts, it should establish a comprehensive outreach plan that prioritizes its efforts and appropriately focuses on those in need of program services. As part of its planning efforts, the board should seek input from key stakeholders such as assistance centers, JP units, and other advocacy groups and associations to gain insight regarding underserved and vulnerable populations.
To ensure that the board appropriately carries out its outreach efforts, it should consider demographic and crime statistics information when developing outreach strategies.
To ensure that the board appropriately carries out its outreach efforts, it should use information from applicants regarding how they heard about the program as part of its overall efforts to measure outreach effectiveness.
In its one-year response, the board reported that its Comprehensive Communication and Outreach Plan identifies the use of 10 existing metrics and the development of additional metrics that are and will be used to establish benchmark awareness levels, prioritize projects, target underserved and hard-to-reach populations, and evaluate the effectiveness of overall outreach efforts. The board also reported that to more definitively measure its success in achieving outreach goals, it is in the process of establishing a baseline from which it may accurately measure goals. The board stated that it has developed the methodology to perform a survey to establish a baseline and plans to execute the survey by late 2009. Additionally, the board reported one of the metrics in its plan is an evaluation of applicants responses to how they heard about the program and that it is using the responses to focus and evaluate research efforts. (2010-406, p. 224)
To ensure that the board appropriately carries out its outreach efforts, it should specifically budget for and report actual outreach expenses.
The board reported that it had established an outreach budget for fiscal years 200809 and 200910, incorporating all the elements of the Comprehensive Communication and Outreach Plan. (2010-406, p. 224)
Agency responses received after June 2013 are posted verbatim.