When planning future intermediate sanction programs, the parole division should decide on appropriate benchmarks for monitoring performance, identify the data it will need to measure performance against those benchmarks, and ensure that reliable data collection mechanisms are in place before a program is implemented. After implementing a new intermediate sanction program, the parole division should analyze the data it has collected and, if relevant, use the data in existing databases to monitor and evaluate the program's effectiveness on an ongoing basis.
November 28, 2012
In August 2011, the following community-based programs were transferred from the California Department of Corrections and Rehabilitation (CDCR), Division of Adult Parole Operations (DAPO), to the CDCR, Division of Rehabilitative Programs (DRP): Parolee Service Centers, Residential Multi-Service Centers, Day Reporting Centers, Community-Based Coalition, Substance Abuse Treatment and Recovery, and Computer Literacy Learning Labs.
As mentioned in previous responses to the Bureau of State Audits, DRP uses a
“Key Performance Indicator” (KPI) system to measure performance outcomes, and was in the process of incorporating these programs into its KPI system in order to measure performance one year ago. It is important to note that these programs joined those already under the jurisdiction of DRP, including the Substance Abuse Services Coordination Agencies, the Female Offender Treatment and Education Program, and the Parolee Services Network.
Recommendation No. 1
In October 2012, DRP established appropriate performance indicators/benchmarks for the community-based programs it inherited from DAPO. These include, but are not limited to, the following:
• Program Completion
All community-based programs are required to submit these data to DRP on a monthly basis, via a secure upload to a database provided by CDCR. DRP's Performance, Data and Fidelity Unit then publishes the data in its monthly KPI reports. DRP's executive and management team analyzes the KPI reports to assess utilization. Where issues or concerns are identified, DRP shares that information with DAPO, in an effort to increase enrollment, attendance and completion.
However, at this time, DRP publishes only the “Enrollment” data collected from the programs transferred from DAPO. The Division is in the process of analyzing how best to group the “Attendance” and “Program Completion” data, since there is a fair amount of variation in the way that the current contractors track these data elements.
In its prior response, DRP indicated that “Recidivism Rate/Reduction” would also be established as a performance indicator. This element has not yet been fully implemented, for a number of reasons. Though CDCR's Office of Research publishes annual reports examining the recidivism rates of individuals released from prison, CDCR does not yet have the resources to track recidivism by individual program.
DRP and CDCR's Office of Research continue to explore options to do this, so that we can differentiate outcomes by the various interventions. Unfortunately, given current staffing and priority workload issues, this is not something that CDCR can commit to accomplishing in the near future.
Other potential performance indicators could include employment, education levels, and characteristics of those served. DRP anticipates KPIs will be finalized by July 2013.
The parole division should ensure that the savings estimates developed during program planning are based on reasonable assumptions. If those assumptions change, it should update the savings estimates promptly.
The parole division concurs with our recommendation and indicates it will ensure that any discussions with legislative staff or other researchers include reasonable projections or estimates, and that it updates and reassesses projected savings in a timely manner. Specifically, when developing its fiscal year 2006-07 budget, the parole division indicates adjusting the assumptions and savings estimates related to its parole programs based on current data. (2007-406, p.241)
The parole division should consider analyzing the effect programs have had on parolee behavior and should use the knowledge it gains from the analyses to make future intermediate sanction programs more effective. The analysis should include the benefits of adding features to make these programs more effective.
Agency responses received after June 2013 are posted verbatim.