Report 2012-122 Recommendation 14 Responses

Report 2012-122: Mental Health Services Act: The State's Oversight Has Provided Little Assurance of the Act's Effectiveness, and Some Counties Can Improve Measurement of Their Program Performance (Release Date: August 2013)

Recommendation #14 To: Mental Health Planning Council, California

The Planning Council should take steps to ensure that it annually reviews the overall effectiveness of MHSA programs in accordance with state law.

1-Year Agency Response

The Planning Council has released the Data Notebook to all the county mental health boards for their use in reporting their review of program outcomes for their county to the Council. The Notebooks are due July 31, 2014 although some counties have asked for an extension. Additionally, the Council has begun holding public forums around the state to obtain input from stakeholders at the local level regarding the various changes in the mental health delivery system over the past several years. The first forum was held in May in Merced, in San Bernardino in July and a 3rd is scheduled for early September in Yolo County. The Council plans to hold mono-lingual forums with interpreters, also to reach out to native communities and the LGBTQ communities for designated forums to receive their input. Lastly, the Council is working with the CA Behavioral Health Directors Association to finalize the protocol and format for program reviews of children's programs in small counties. It is anticipated that the reviews will begin in late 2014/early 2015. Aggregated reports from the Data Notebook and Public Forum projects are expected to be released late 2014/early 2015.

Until county-reported data becomes available for the Council to use in fulfilling its responsibilities to review and report on program performance, we will continue to use these optional methods of gathering program performance data at the local level. These projects have been launched so this Audit Review item is fully implemented.

California State Auditor's Assessment of 1-Year Status: Fully Implemented

6-Month Agency Response

Over the past months, the Council continued to meet regularly with both the DHCS and the MHSOAC and the group has identified some possibilities to address the need for an effective data collection and outcomes reporting system. Necessary research, testing and documentation are underway as initial steps to future implementation. Meanwhile, the Council is in the process of implementing several activities over this year to obtain input at the local level regarding the effectiveness of mental health services, including MHSA-funded services, through public forums, a data notebook project with the local mental health boards and also on-site county program reviews as alternative ways to fulfill the above mandate until there are sufficient sources of outcomes and data for the Council to review.

We are currently developing the data notebook and training curriculum, identifying the format and topic areas for the public forums, and designing the review protocol for the on-site program reviews. These will be pilot tested in the Spring and early Summer 2014. Final reports are expected by the end of the year.

California State Auditor's Assessment of 6-Month Status: Partially Implemented

60-Day Agency Response

In its response to the Auditor's findings, the Council reached out to DHCS and the MHSOAC in August, requesting a meeting to explore avenues of collaboration on the mutual mandates. The three entities met in mid-October to discuss their strategies to meet these recommendations, and will continue to meet on a monthly basis to identify and define individual roles and tasks as outlined in WIC 5610 sec 120 (a) and (b).

In addition to the external workgroup with DHCS and MHSOAC, the Council has convened an internal workgroup to better understand the process of evaluation- the available data sources, the criteria for good reporting (i.e. pertinent, accessible, and informative performance indicators, which the Council is mandated to approve) and the critical review of the reports that are generated. When completed, this information will be shared with the entire Council in order to clarify the expectations of the mandate and comply with the Audit's recommendation. (This workgroup is on-going, with no established end-date at this time).

Continuing the development of the new Data Notebook for use by county Mental Health Boards in reporting on mental health access and service provision by age, gender, and ethnicity relative to the county population using quality of life indicators such as incarceration and hospitalization rates, high school graduation, educational persistence, and employment. We have a target date of March 1, 2014 for this.

California State Auditor's Assessment of 60-Day Status: Partially Implemented

All Recommendations in 2012-122

Agency responses received are posted verbatim.