Report 2012-044 Recommendation 13 Responses

Report 2012-044: California Department of Education: Despite Some Improvements, Oversight of the Migrant Education Program Remains Inadequate (Release Date: February 2013)

Recommendation #13 To: Education, Department of

To determine if the statewide migrant education program is effective, Education should finalize its current evaluation of the program and begin developing the capacity to produce a more robust annual evaluation of the program.

Annual Follow-Up Agency Response From November 2017

Fully implemented. A series of stakeholder meetings were held to develop both the statewide comprehensive needs assessment and state service delivery plan (SSDP), completed in June 2017 and October 2017, respectively. The SSDP identifies strategies to: (1) address migratory students' needs; (2) provide measurable program outcomes (MPO), which evaluate the subgrantees' strategy implementation; and (3) set migrant education specific performance targets (PT). At the end of the first year of the plan's implementation in 2018-19, Education will begin to evaluate the SSDP's MPOs and PTs. In addition, subgrantees will receive annual progress reports to support continuous program improvement.

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

Annual Follow-Up Agency Response From November 2016

Partially implemented. In December 2015, each of Education's 20 subgrantees completed a local Comprehensive Needs Assessment (CNA). Education is utilizing these findings to develop a meta-analysis to serve as a basis for the statewide CNA. Education anticipates the new statewide CNA and State Service Delivery Plan (SSDP) will be available online in the summer of 2017. Once Education identifies measurable outcomes in the SSDP, the local MEPs will provide an annual update to Education as part of the yearly grant application process. This process will encourage the local MEPs to continuously improve processes and services to students, and ensure that necessary data is collected for the next iteration of the evaluation process in three years.

California State Auditor's Assessment of Annual Follow-Up Status: Not Fully Implemented

Annual Follow-Up Agency Response From October 2015

Education completed the evaluation of the MEP for the first cycle of the Comprehensive Needs Assessment. Education provided each of its grantees grants to contract with independent researchers to conduct Local Needs Assessments. Each subgrantee will be required to submit final reports to Education in December 2015. Education will utilize these reports as the basis for a statewide Comprehensive Needs Assessment, which will serve as the foundation for the development of a new State Service Delivery Plan (SSDP). The SSDP will identify new measurable objectives, which will allow for the production of a more robust annual evaluation of the MEP.

California State Auditor's Assessment of Annual Follow-Up Status: Not Fully Implemented

Annual Follow-Up Agency Response From October 2014

Education completed the evaluation of the migrant program and is in the process of implementing two projects that will lead to a more robust annual evaluation. The first project focuses on regular data exchanges between the Migrant Student Information Network and the California Longitudinal Pupil Achievement Data System (CALPADS). This data exchange will facilitate the acquisition of statewide migrant student assessment data, and ensure that eligible migrant program students are identified in CALPADS.

The second project is the implementation of the first phase of a statewide Comprehensive Needs Assessment. In 2014-15, the migrant regions will conduct local assessments that will identify program needs. These assessments will also allow Education to identify performance targets with measurable objectives. In the forthcoming State Plan for Services, Education will ensure that identified performance targets can be measured and migrant student data is available to conduct the analysis.

California State Auditor's Assessment of Annual Follow-Up Status: Not Fully Implemented

1-Year Agency Response

For 2013-14, Education redesigned the RA for regions and districts to complete as part of the grant application process. Education evaluates all submitted RAs; the redesigned RAs provide Education with an increased level of detail for evaluating regions program descriptions and implementation processes. During the initial evaluation, Education identifies informational deficiencies which may entail requesting regions to explain or specify the: (1) alignment of service programs to identified student needs; (2) detail on program descriptions; (3) itemization of service budgets; and (4) reasonableness of administrative cost percentages in relation to the 15-percent goal. Once the informational requests are fulfilled, Education re-evaluates the RAs with the updated information. Education intends to continue this robust program evaluation process and provide any necessary technical assistance to the regions.

California State Auditor's Assessment of 1-Year Status: Pending

It is unclear to us how Education's review of regional applications will ensure it can produce a robust annual evaluation of the statewide migrant program as we recommended.

6-Month Agency Response

Education is in the process of finalizing its current evaluation of the MEP. The collection of quantitative and qualitative data has been completed and is being analyzed; Education anticipates review completion by 2013-14. The data analyses will help Education assess the alignment of regional activities with the MEP State Service Delivery Plan (SSDP) performance measures as provided by Education.

In an effort to produce a more robust evaluation of the MEP, and regions' alignment with Education's performance measures per the SSDP, Education has designed a new Regional Application for regions and districts to complete each school year prior to providing services to migrant students. The new Regional Application incorporates seven critical components of service delivery. These components include: (1) School Readiness; (2) English Language Arts; (3) Mathematics: (4) High School Graduation; (5) Health; (6) Out of School Youth; (7) Parent Involvement. In addition, the new Regional Application requests regions to: (a) evaluate migrant student data; (b) identify student needs; (c) design supplemental programs that address such needs; (d) implement programs; and (e) monitor program performance regularly.

***will e-mail full response

California State Auditor's Assessment of 6-Month Status: Pending

60-Day Agency Response

The estimated completion date for the evaluation of the statewide delivery of migrant education services is October 31, 2013. This information, together with the new regional applications and district service agreements, and the next statewide Comprehensive Needs Assessment, will inform the development of a more robust annual evaluation.

California State Auditor's Assessment of 60-Day Status: Pending

All Recommendations in 2012-044

Agency responses received after June 2013 are posted verbatim.