Report 2012-105 All Recommendation Responses

Report 2012-105: Departments of Public Health and Social Services: Weaknesses in the Administration of the Child Health and Safety Fund and the State Children's Trust Fund Limit Their Effectiveness (Release Date: November 2012)

Recommendation #1 To: Public Health, Department of

To ensure that it does not violate provisions of state law that prohibit contracts for services that state employees can perform, Public Health should establish that it has adequate justification to contract under Section 19130 (b), prior to submitting contracts to the Department of General Services (General Services) for approval.

6-Month Agency Response

On January 15, 2013, CDPH issued to section chiefs and above an email reminder (Attachment A) of Contracts and Purchasing Services Section (CPSS) Bulletin 09-13 (Attachment B), which requires the CDPH Office of Legal Services to review and approve any contracts/procurements that involve personal services.

• The Center for Chronic Disease Prevention and Health Promotion (CCDPHP) Deputy or Assistant Deputy Director, has directed each of its two Division Chiefs to (1) ensure that Branch Chiefs review the CD PH-wide email reminder and emphasize compliance with policy in its two-on-two meetings with Branch Chiefs throughout February, and (2) reminded administrative staff from all Center Branches to follow established policy reviewed in the February administration update meetings. A copy of January 29, 2013 meeting agenda attached (Attachment C).

California State Auditor's Assessment of 6-Month Status: Fully Implemented

60-Day Agency Response

Public Health stated that effective November 3, 2009, its Contracts and Purchasing Services Section Bulletin 09-13, entitled, "Contracts/Procurements involving Personal Services," requires the completion of a form "Justification for Contracting Out Services" for all personal services contracts. Public Health also stated that its Office of Legal Services must review and approve this form before a contract or amendment can by fully executed.

Public Health stated that on January 15, 2013, it issued an e-mail reminder of this bulletin to its section chiefs and above. In addition, Public Health stated that by February 28, 2013, either the deputy director or assistant deputy director of its Center for Chronic Disease Prevention and Health Promotion (center), which oversees the Kids' Plate Program, will direct each of its two division chiefs to (1) ensure via two-on-two meetings with branch chiefs that they review the e-mail reminder and emphasize with their staff compliance with the policy, and (2) remind administrative staff from all branches to follow the established policy.

California State Auditor's Assessment of 60-Day Status: Partially Implemented

Recommendation #2 To: Public Health, Department of

To comply with state contracting laws and policies that protect the State's interest, Public Health should ensure that its staff do not allow contractors to work before the General Services has approved the contracts.

6-Month Agency Response

By June 20, 2013, the Deputy Director, CCDPHP, will issue a memo (copy of draft memo attached, Attachment D) to all CCDPHP staff reminding them that the Public Contract Code (PCC) sections 10295 and 1 0335, State Contracting Manual (SCM) section 4.09, and Public Health Administrative Manual (PHAM) section 3-1140 prohibit commencement of work without a fully executed contract.

- In their regular February 2013 two-on-two meetings, the Deputy Director, CCDPHP, directed Division Chiefs to discuss compliance with contracting policy with their respective Branch Chiefs.

- During the February 4, 2013, administrative update meeting, the CCDPHP Assistant Deputy Director reminded all branch administrative staff of contracting policy requirements and provided handouts of pertinent State Contract Manual and Public Health Administrative Manual provisions. See meeting agenda and copies of handouts attached (Attachment E).

California State Auditor's Assessment of 6-Month Status: Fully Implemented

60-Day Agency Response

Public Health stated that by February 2013, the center will take the following actions to ensure that its staff comply with this mandate and with the internal contracting policies that speak to this mandate: (1) issue an e-mail reminder of the relevant Public Contract Code, State Contracting Manual, and Public Health Administrative Manual sections that prohibit commencement of work without a fully executed contract; (2) direct division chiefs to discuss compliance with contracting policies with branch chiefs in their regular two-on-two meetings; and (3) remind administrative staff from all branches, via administration update meetings and e-mails, of the contracting policy requirements.

California State Auditor's Assessment of 60-Day Status: Pending

Recommendation #3 To: Public Health, Department of

To comply with state contracting laws and policies that protect the State's interest, Public Health should ensure that its staff do not use service orders to circumvent the State's contracting policies.

6-Month Agency Response

By June 20, 2013, CCDPHP Deputy Director or Assistant Deputy Director, will issue an all staff memo directing staff to review the Service Order Manual and the appropriate uses of service orders (see Attachment D).

• By June 20, 2013, CCDPHP will direct its lead staff involved in procurement processes to attend the CDPH Procurement Training series and provide a certificate of completion to program supervisors at the end of their last session. Many of these staff are currently enrolled (see Attachment D).

• In October 2012, the Administrative Support Unit Chief of the Safe and Active Communities Branch completed the CDPH Procurement Training series (Attachment F).

• During the February 12, 2013, administrative update meeting the CCDPHP Assistant Deputy Director reminded all branch administrative staff that service orders were not to be used to circumvent State Contracting Manual requirements. The meeting agenda is attached (Attachment G).

• On April 30, 2013, the Program Support Branch released Contracts and Purchasing Services Bulletin #13-03 announcing the Department's new Service Order Policy (Attachment H). Effective July 1, 2013, new policies will be instituted to centralize tracking, review, and approval of service orders within the Program Support Branch. The Program Support Branch has conducted two "Service Order Policy for FY 13/14" training sessions. The first was held on May 14, 2013, and was attended by 17 CCDPHP staff. The second was held on May 22, 2013, and was attended by 15 CCDPHP staff.

California State Auditor's Assessment of 6-Month Status: Fully Implemented

60-Day Agency Response

Public Health stated that by February 28, 2013, the center will issue an e-mail directive for program staff to review the Service Order Manual and the appropriate use of service orders. In addition, Public Health stated that the center will direct its lead staff involved in the procurement process to attend Public Health's Procurement Training series and provide a certificate of completion to program supervisors at the end of their last session. Finally, Public Health reported that the administrative support unit chief within its Safe and Active Communities Branch (branch) completed the series in October 2012.

California State Auditor's Assessment of 60-Day Status: Partially Implemented

Recommendation #4 To: Public Health, Department of

To comply with state contracting laws and policies that protect the State's interest, Public Health should recoup the overpayment made to the San Diego State University Research Foundation (research foundation), if applicable.

1-Year Agency Response

CDPH has completed its review of the personnel expenditures billed under the contract from July 1, 2007 through June 30, 2010 and has determined that there was no overpayment and a recoupment of funds is not necessary.

BSA found that two individuals had a budgeted time base in the contract of 25% time. In 14 of the invoices BSA reviewed, salaries were billed and reimbursed in excess of 25% time.

The amounts billed were for appropriate costs incurred in completion of the agreed-upon services. The contract stated that "Cumulative line item shifts of up to $25,000 or 10% of the annual agreement total may be made, whichever is greater, up to a cumulative annual maximum of $50,000, provided the annual agreement total does not increase or decrease." The contractor was required to seek prior approval from CDPH for such line item changes, but the contract stated "line time shifts meeting this criteria shall not require a formal agreement amendment."

Over the course of the three-year contract, the contractor identified several instances when these individuals spent more time on a project deliverable than originally anticipated and identified other budgetary savings to cover the increased salaries, in accordance with the contract payment provisions. CDPH approved these requests and line-item shifts did not exceed the contract limits. In Years 1 and 2, sufficient savings were available within the budgeted salary amounts to cover increases in some staff time. In Year 3, savings from "Other Operating Expenses" covered the increased staff time.

California State Auditor's Assessment of 1-Year Status: Resolved

6-Month Agency Response

• On December 14, 2012, CDPH submitted a public records request to the Bureau of the State Audits (BSA) to obtain the working papers that speak to this overpayment (Attachment 1). On January 15, 2013, BSA mailed the working papers to CDPH. By June 30, 2013, CDPH will complete its review of these records and determine whether a recoupment of costs is necessary.

California State Auditor's Assessment of 6-Month Status: Partially Implemented

60-Day Agency Response

Public Health submitted a Public Records Act request to the state auditor to obtain the documentation that supports the overpayment cited in our report. Public Health stated that, once it receives the information, it will review the documents and initiate a recoupment process, if deemed necessary.

California State Auditor's Assessment of 60-Day Status: Partially Implemented

Recommendation #5 To: Public Health, Department of

To determine whether the appropriation to administer the Kids' Plates Program is sufficient, Public Health should continue its plans to evaluate the costs of the regional grants Request for Application (RFA) process and its monitoring of the awards for fiscal year 2012–13. If Public Health determines that the appropriation is insufficient, it should seek an amendment to state law.

1-Year Agency Response

CDPH evaluated its administrative costs for the Kid's Plates Program for FY 2012-13 and did not seek an amendment to state law.

California State Auditor's Assessment of 1-Year Status: Fully Implemented

6-Month Agency Response

The Safe and Active Communities Branch (SAC) is documenting the time spent on administrative tasks associated with the regional contracts. SAC completed the RFA process on November 8, 2012 (the date on which the appeal period ended; Attachment J), and is preparing contracts.

• CDPH has encountered a number of issues in processing the new KP regional contracts, including specific challenges to CDPH's authority to fund pedestrian safety projects and the need to contract directly with a proposed regional subcontractor. CDPH has not yet determined what the ongoing administrative costs will be. To date, we estimate we have spent over 600 staff hours developing the RFA, reviewing applications, and negotiating and processing contracts, at an estimated cost of $23,000 in salaries alone.

California State Auditor's Assessment of 6-Month Status: Partially Implemented

60-Day Agency Response

Public Health stated that its branch is documenting the time spent on administrative tasks associated with the regional grants. Further, Public Health stated that the branch completed the RFA process on November 28, 2012, and is preparing contracts with a start date beginning in March 2013.

California State Auditor's Assessment of 60-Day Status: Partially Implemented

Recommendation #6 To: Public Health, Department of

To comply with the State Contracting Manual, if Public Health chooses to use contractors for the Kids' Plates Program, it should direct its staff to substantiate the expenditures contractors claim. For example, Public Health could ask the contractors to submit for review detailed records substantiating all or a sample of their invoices.

1-Year Agency Response

In a January 17, 2013, email to all Administrative Unit staff, CDPH instituted a new policy requiring a periodic review of detailed expenditures on Kid's Plate invoices. Branch management will reinforce the policy with each of the four project coordinators as part of a project orientation scheduled for December 19, 2013.

A June 18, 2013, memo to all Center for Chronic Disease Prevention and Health Promotion staff requires periodic, detailed reviews of contract expenditures. In addition, CDPH requires the new Kid's Plate contractors to submit detailed receipts with their June invoices.

California State Auditor's Assessment of 1-Year Status: Fully Implemented

6-Month Agency Response

SAC has instituted a new policy requiring a periodic review of detailed expenditures on KP invoices. A January 18, 2013 email outlines this new policy for all Administrative Unit staff (Attachment K). Branch management will reinforce the policy with each of the four new project coordinators as part of a project orientation to take place after the contracts are fully executed (July- August 2013).

• By June 20, 2013, the Deputy Director, CCDPHP, will issue a memo (Attachment D) that includes a directive to institute periodic, detailed reviews of contract expenditures.

California State Auditor's Assessment of 6-Month Status: Partially Implemented

60-Day Agency Response

Public Health stated that the branch instituted a new policy requiring a periodic review of detailed expenditures on Kid's Plate Program invoices and sent an e-mail outlining the policy to all administrative staff on January 17, 2013. Public Health also stated that the branch will reinforce the policy with each of the three new project coordinators as part of a project orientation to take place after the contracts are fully executed in April or May 2013.

California State Auditor's Assessment of 60-Day Status: Partially Implemented

Recommendation #7 To: Public Health, Department of

To ensure that it is able to measure its progress toward fulfilling the requirements of the health and safety fund, Public Health should establish performance measurements for the Kids' Plates Program.

Annual Follow-Up Agency Response From October 2015

CDPH's Kid's Plates Program awarded four contracts for the implementation of an evidence-based program, Safe Routes to School, which has demonstrated success in reducing childhood bicycle and pedestrian injuries. Each contract included a scope of work with measurable objectives and delineated performance measures and/or deliverables for major activities. At the conclusion of the contract period, CDPH evaluated the overall performance and impact of the Kid's Plates contracts, prepared a Final Report Summary and posted this summary on the Kid's Plates website: http://www.cdph.ca.gov/programs/Pages/KidsPlates.aspx

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

Annual Follow-Up Agency Response From October 2015

The status of this recommendation is unchanged.

California State Auditor's Assessment of Annual Follow-Up Status: Not Fully Implemented

Annual Follow-Up Agency Response From October 2014

The status of this recommendation is unchanged.

California State Auditor's Assessment of Annual Follow-Up Status: Not Fully Implemented

1-Year Agency Response

The following performance measures are in place for the Kids' Plates Program (KPP) 1. Performance Measure--CDPH will ensure the KPP funds are utilized to implement evidence-based/informed programs or best practices that can contribute to reducing childhood injuries and fatalities. Consistent with recommendations on page 28 of the report, each contract scope of work contains activities that support the Program performance measure. For example, each KPP contractor will be conducting bike/pedestrian injury reduction activities for which there is evidence, research or evaluation demonstrating the injury reduction benefits. CDPH staff will monitor the contracts, and provide technical assistance and other guidance to ensure fidelity. A baseline of collisions/injuries/fatalities will be completed for each of the intervention schools/districts. Changes in these incidents will be monitored during and at the end of the contract to measure changes. 2. Performance Measure-Using available data and surveillance mechanisms, CDPH will monitor changes and trends in childhood injuries to assess the impact of its efforts in reducing childhood injuries. Because of funding amount, the small scale and short duration of the current projects, and the factors that contribute to reductions in injuries/fatalities, it is very difficult to measure the link between contractor/Program performance and overall injury reduction. Over time, with continued investment, and using proven injury prevention approaches, the KPP can help stimulate changes in policy, practice and priorities that will lead to injury reductions. These Performance Measures have been approved by CDPH Executive Staff (see attachment). We have attached the signature page for the contracts. The contract with Monterey County was delayed due to the required time to schedule it for approval by the Board of Supervisors. It has been signed by the contractor and CDPH and is waiting for final approval.

California State Auditor's Assessment of 1-Year Status: Partially Implemented

6-Month Agency Response

Each of the four awardees submitted a scope of work delineating performance measures and/or deliverables for each of the major activities that they will implement. CDPH is incorporating these measures into the contracts. The awardees will submit semi-annual reports to describe progress in achieving these performance measures (December 31, 2013; June 30, 2014; and December 31, 2014).

California State Auditor's Assessment of 6-Month Status: Partially Implemented

60-Day Agency Response

Public Health stated that the Kid's Plate Program RFA requires the completion of a template, which requires a specific "Performance Measure and/or Deliverable" for each major project activity. Public Health stated that each of the three awardees submitted a scope of work delineating performance measures and/or deliverables for each of the major activities that they will implement. Public Health stated it will incorporate these measures into the contracts and the awardees will submit semi-annual reports to describe their progress in achieving these performance measures.

California State Auditor's Assessment of 60-Day Status: Partially Implemented

Recommendation #8 To: Public Health, Department of

To ensure that it is able to measure its progress toward fulfilling the requirements of the health and safety fund, Public Health should periodically assess its progress toward meeting its measureable outcomes.

Annual Follow-Up Agency Response From October 2015

At the conclusion of the contract period, CDPH evaluated the overall performance and impact of the Kid's Plates contracts, prepared a Final Report Summary and posted this summary on the Kid's Plates website: http://www.cdph.ca.gov/programs/Pages/KidsPlates.aspx

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

Annual Follow-Up Agency Response From October 2015

The status of this recommendation is unchanged.

California State Auditor's Assessment of Annual Follow-Up Status: Not Fully Implemented

Annual Follow-Up Agency Response From October 2014

The status of this recommendation is unchanged.

California State Auditor's Assessment of Annual Follow-Up Status: Not Fully Implemented

1-Year Agency Response

The first progress reports are due to CDPH until the end of January, 2014. We will provide copies to BSA when we receive them. CDPH will review these and subsequent progress reports, provide feedback and technical assistance as indicated, and regularly assess the contractors' contributions to the overall program performance measures.

California State Auditor's Assessment of 1-Year Status: Partially Implemented

6-Month Agency Response

• CDPH has mechanisms in place to monitor changes in morbidity and mortality in the three areas covered by the new projects. Beginning in July 2013 CDPH will assess progress with baseline measures of targeted pedestrian and bicycle injuries.

California State Auditor's Assessment of 6-Month Status: Partially Implemented

60-Day Agency Response

Public Health stated that it has mechanisms in place to monitor changes in morbidity and mortality in the three areas covered by the new projects. Public Health stated that it will assess progress beginning in March 2013 with baseline measures of targeted pedestrian and bicycle injuries.

California State Auditor's Assessment of 60-Day Status: Partially Implemented

Recommendation #9 To: Social Services, Department of

To ensure that the office complies with the State Contracting Manual, Social Services should direct the office to substantiate the expenditures that grantees claim. For example, the office could ask the grantees to submit for review detailed records for all or a sample of their invoices.

60-Day Agency Response

Social Services updated its internal grant and contract manuals to strengthen its process for substantiating expenditures. Specifically, Social Services updated its internal manuals to include procedures for requesting random samples of backup documentation on at least a quarterly basis from the grantee or contractor in order to substantiate claimed expenditures.

California State Auditor's Assessment of 60-Day Status: Fully Implemented

Recommendation #10 To: Social Services, Department of

To ensure that the office complies with the State Contracting Manual, Social Services should direct the office to recover the overpayment from the Children's Bureau of Southern California, if applicable.

60-Day Agency Response

Social Services stated that the Children's Bureau of Southern California (Children's Bureau) resolved the overpayment in the billing it submitted for June 2011. Social Services stated it received, reviewed, and approved the Children's Bureau's corrected documentation and found that the expenditures were appropriate and that there was no overpayment.

California State Auditor's Assessment of 60-Day Status: Fully Implemented

Recommendation #11 To: Social Services, Department of

To ensure that it uses trust fund moneys only for permissible uses, Social Services should direct its internal audits staff to periodically perform reviews of the trust fund expenditures.

60-Day Agency Response

Social Services stated that it included a yearly fund review in the 2013 audit plan and that the review includes the trust fund.

California State Auditor's Assessment of 60-Day Status: Fully Implemented

Recommendation #12 To: Social Services, Department of

To ensure that it uses trust fund moneys only for permissible uses, Social Services should revise its invoicing process to clearly identify the objectives in the scope of work section of its grant agreements and their corresponding funding sources.

6-Month Agency Response

Clarifying procedures have been developed to further ensure that funds are expended utilizing the appropriate fund source. (See Attachment A.) Further, as noted in the Grant Manual, the scope-of-work form has been updated to identify the appropriate fund source for each activity grantees are required to perform in each grant. (See Attachment B.)

California State Auditor's Assessment of 6-Month Status: Fully Implemented

60-Day Agency Response

Social Services stated that the office updated its invoice form to identify the objectives in the scope of work section of its grant agreements to the corresponding funding sources.

California State Auditor's Assessment of 60-Day Status: Partially Implemented

Although the form identifies the funding percentages, it does not identify the objectives in the scope of work section of the grant agreements that would correspond to the funding percentages.

Recommendation #13 To: Social Services, Department of

To ensure compliance with the state law governing the trust fund that allows it to fund research and demonstration projects that explore the nature and incidence of child abuse and the development of long-term solutions to the problem of child abuse, Social Services should establish procedures to ensure that all grants it awards for research and demonstration projects clearly demonstrate a linkage to the trust fund's requirements.

6-Month Agency Response

Clarifying procedures have been developed to further ensure that funds are expended utilizing the appropriate fund source. (See Attachment A.) Further, as noted in the Grant Manual, the scope-of-work form has been updated to identify the appropriate fund source for each activity grantees are required to perform in each grant.

(See Attachment B.)

California State Auditor's Assessment of 6-Month Status: Fully Implemented

60-Day Agency Response

Social Services added a section to its grant manual that specifically identifies the allowable uses of the trust fund as set forth in the Welfare and Institutions Code.

California State Auditor's Assessment of 60-Day Status: Partially Implemented

The added section does not identify the procedures it plans to use to ensure that all grants it awards for research and demonstration projects clearly demonstrate a linkage to the trust fund's requirements.

Recommendation #14 To: Social Services, Department of

To ensure that the office complies with the State Contracting Manual, Social Services should direct the office to update its guidelines for grant administration to establish a formal process for reviewing the grantees' progress reports and interim products. This process should include documenting the office's review and assessment of whether the grantees meet the goals, objectives, and measurable outcomes in their grant requirements.

60-Day Agency Response

Social Services updated its grant manual to include a section on reviewing grantees' progress reports. Social Services' office created a new form to document the receipt, review, and approval of grantees' progress reports.

California State Auditor's Assessment of 60-Day Status: Fully Implemented

Recommendation #15 To: Social Services, Department of

To ensure that the office complies with the State Contracting Manual, Social Services should direct the office to retain the documentary evidence of its review and assessment in the grantee files.

60-Day Agency Response

Social Services' office created a new form to document the receipt, review and approval of grantees' progress reports, which will be retained in the grantees' files. In addition, the form instructs consultants to keep any backup documentation (e-mails, document phone calls, etc.) in the grantees' files.

California State Auditor's Assessment of 60-Day Status: Fully Implemented

Recommendation #16 To: Social Services, Department of

To ensure that its efforts funded by the trust fund are preventing or reducing incidences of child abuse and neglect, Social Services should develop universal performance measurements for the trust fund.

1-Year Agency Response

Partially Corrected/Anticipated Correction Date: December 31, 2013. The CDSS has created a plan to develop overarching performance outcome measures for the SCTF that are appropriate for the programs funded. Since the SCTF programs vary widely in purpose and function, the goal of the inclusive process is to build agreement around which measures can be universally applicable. This process will include transmission of a set of proposed outcomes to stakeholders for review and comments prior to their finalization. The performance outcome measures will incorporate the requirements for the SCTF as set forth in WIC section 18969(e).

1) During this reporting period the CDSS convened a stakeholder group that included community-based organizations, OCAP staff, and a parent from the community who is the mother of a special needs child and also the Executive Director of WarmLine Family Resource Center, to discuss ideas for improved accountability and effectiveness, and to review and consider outcome and performance measures for all OCAP fund sources.

2) The performance measures are now being vetted through the CDSS Executive Office. Once approved, a final set of performance measures will be published and implementation across programs will begin.

3) After the performance measures are finalized, they will be reflected in the grants CDSS awards and will be utilized to determine whether the grants are effective in meeting the funds' requirements as well as the efficacy in reaching goals of reducing abuse and neglect.

California State Auditor's Assessment of 1-Year Status: Fully Implemented

The agency provided the state auditor with its approved performance measurements for the trust fund in January 2014.

6-Month Agency Response

CDSS is creating a plan to develop overarching performance outcome measures for the SCTF that are appropriate for the programs funded. Since the SCTF programs vary widely in purpose and function, the goal of the inclusive process is to build agreement around which measures can be universally applicable. This process will include transmission of a set of proposed outcomes to stakeholders for review and comments prior to their finalization.

1) CDSS has begun the process of drafting a set of outcomes with descriptions and accompanying process measures, which include the requirements set forth in WIC Section 18969(e);

2) The draft outcomes will be released to stakeholders for review and comment by August 2013;

3) CDSS will then incorporate that feedback into a final set of outcomes.

California State Auditor's Assessment of 6-Month Status: Partially Implemented

60-Day Agency Response

Social Services stated, by June 30, 2013, it will develop and publish a set of overarching performance measures for the trust fund that will be consistent with the legislative intent of the fund, as described in Welfare and Institutions Code section 18969(e).

California State Auditor's Assessment of 60-Day Status: Pending

Recommendation #17 To: Social Services, Department of

To ensure that its efforts funded by the trust fund are preventing or reducing incidences of child abuse and neglect, Social Services should ensure that the performance measurements are reflected in the grants it awards.

Annual Follow-Up Agency Response From October 2014

During this reporting period, the Office of Child Abuse Prevention (OCAP) implemented the performance measures in the State Children's Trust Fund (SCTF) grant agreements through individual grant amendments with grantee organizations. Each grant has specific measures that correspond to the allowable uses of the SCTF. Amendments were effective as of July 1, 2014, and grantees are currently tracking performance utilizing the new measures.

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

1-Year Agency Response

Partially Corrected/Anticipated Correction Date: December 31, 2013. The CDSS has created a plan to develop overarching performance outcome measures for the SCTF that are appropriate for the programs funded. Since the SCTF programs vary widely in purpose and function, the goal of the inclusive process is to build agreement around which measures can be universally applicable. This process will include transmission of a set of proposed outcomes to stakeholders for review and comments prior to their finalization. The performance outcome measures will incorporate the requirements for the SCTF as set forth in WIC section 18969(e).

1) During this reporting period the CDSS convened a stakeholder group that included community-based organizations, OCAP staff, and a parent from the community who is the mother of a special needs child and also the Executive Director of WarmLine Family Resource Center, to discuss ideas for improved accountability and effectiveness, and to review and consider outcome and performance measures for all OCAP fund sources.

2) The performance measures are now being vetted through the CDSS Executive Office. Once approved, a final set of performance measures will be published and implementation across programs will begin.

3) After the performance measures are finalized, they will be reflected in the grants CDSS awards and will be utilized to determine whether the grants are effective in meeting the funds' requirements as well as the efficacy in reaching goals of reducing abuse and neglect.

California State Auditor's Assessment of 1-Year Status: Partially Implemented

6-Month Agency Response

After the performance measures are developed, they will be reflected in the grants CDSS awards and will be utilized to determine whether the grants are effective in meeting the funds' requirements as well as the efficacy in reaching goals of reducing abuse and neglect.

California State Auditor's Assessment of 6-Month Status: Partially Implemented

60-Day Agency Response

Social Services stated, by June 30, 2013, it will develop and publish a set of overarching performance measures for the trust fund that will be consistent with the legislative intent of the fund, as described in Welfare and Institutions Code section 18969(e).

California State Auditor's Assessment of 60-Day Status: Pending

Recommendation #18 To: Social Services, Department of

To ensure that its efforts funded by the trust fund are preventing or reducing incidences of child abuse and neglect, Social Services should evaluate the performance measurements annually to assess whether the trust fund's programs and services are effective in reducing incidences of child abuse and neglect.

Annual Follow-Up Agency Response From September 2015

The Office of Child Abuse Prevention has received and evaluated quarterly reports for a full year, ensuring that all State Children's Trust Fund (SCTF) grantees meet established performance measures. Grants funded by the SCTF in fiscal year 2014-15 include the California Evidence Based Clearinghouse, Mandated Reporter Training, and the Family Development Matrix.

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

We confirmed that Social Services' Office of Child Abuse Prevention is obtaining and documenting its review of quarterly performance reports from the grantees described in the department's response.

Annual Follow-Up Agency Response From October 2014

Partially Corrected. The first report is due from grantees at the end of the first quarter of the fiscal year, in October. The report period is July-September 2014; grantees report within 30 days of the end of the reporting period.

OCAP staff will review progress with respect to measure on a quarterly, not annual, basis, and work with grantees to monitor progress on outcomes.

The recommendation will not be fully implemented until the end of the first year of implementation, to satisfy the annual review requirement. However, measures will be reviewed at least quarterly.

California State Auditor's Assessment of Annual Follow-Up Status: Not Fully Implemented

1-Year Agency Response

Partially Corrected/Anticipated Correction Date: December 31, 2013. The CDSS has created a plan to develop overarching performance outcome measures for the SCTF that are appropriate for the programs funded. Since the SCTF programs vary widely in purpose and function, the goal of the inclusive process is to build agreement around which measures can be universally applicable. This process will include transmission of a set of proposed outcomes to stakeholders for review and comments prior to their finalization. The performance outcome measures will incorporate the requirements for the SCTF as set forth in WIC section 18969(e).

1) During this reporting period the CDSS convened a stakeholder group that included community-based organizations, OCAP staff, and a parent from the community who is the mother of a special needs child and also the Executive Director of WarmLine Family Resource Center, to discuss ideas for improved accountability and effectiveness, and to review and consider outcome and performance measures for all OCAP fund sources.

2) The performance measures are now being vetted through the CDSS Executive Office. Once approved, a final set of performance measures will be published and implementation across programs will begin.

3) After the performance measures are finalized, they will be reflected in the grants CDSS awards and will be utilized to determine whether the grants are effective in meeting the funds' requirements as well as the efficacy in reaching goals of reducing abuse and neglect.

California State Auditor's Assessment of 1-Year Status: Partially Implemented

6-Month Agency Response

The CDSS will incorporate these performance measures into its Request for Application/Request for Proposal process for future grant awards. For existing grantees, the CDSS will update their scope-of-work to incorporate the performance measures. Fund recipients will be required to report annually on the overarching performance measures, and will be required to modify performance if out of compliance.

California State Auditor's Assessment of 6-Month Status: Partially Implemented

60-Day Agency Response

Social Services stated, by June 30, 2013, it will develop and publish a set of overarching performance measures for the trust fund that will be consistent with the legislative intent of the fund, as described in Welfare and Institutions Code section 18969(e).

California State Auditor's Assessment of 60-Day Status: Pending

Recommendation #19 To: Social Services, Department of

To ensure compliance with the state law that requires the office to publish certain trust fund information, Social Services should require the office to establish procedures to ensure inclusion on its Web site of all programs and services it funded using the trust fund.

6-Month Agency Response

CDSS/OCAP has established procedures to ensure the inclusion on its Web site of all programs and services it funded using the trust fund. See Attachment C for a copy of those procedures. The CDSS/OCAP Web site has been modified to include comprehensive information about the programs funded by SCTF. See Attachment D, which is a printout screen of the document posted on the Web site and that summarizes all of the programs. Each program can be accessed for detailed information by clicking on its title.

California State Auditor's Assessment of 6-Month Status: Fully Implemented

60-Day Agency Response

Social Services stated that the office has ensured that the Web site provides information on all the programs and services financed with all the fund sources it administers, including those financed wholly or in part by the trust fund.

California State Auditor's Assessment of 60-Day Status: Pending

Social Services did not provide a copy of the procedures it established to ensure the inclusion on the office's Web site of all programs and services it funded using the trust fund.

Recommendation #20 To: Social Services, Department of

To ensure compliance with the state law that requires the office to publish certain trust fund information, Social Services should require the office to publish on its Web site the amount in the trust fund as of June 30 each year.

60-Day Agency Response

Social Services' office updated its Web site to include the publication of the amount of, and expenditure data associated with, the trust fund as of June 30, 2012. Social Services stated that it will ensure the Web site is updated annually with the information required by statute.

California State Auditor's Assessment of 60-Day Status: Fully Implemented

Recommendation #21 To: Social Services, Department of

To improve the presentation of the information it publishes for the trust fund, Social Services should establish a link that separately provides descriptions of the types of programs and services it funds using the trust fund and the target populations that benefit from the programs.

6-Month Agency Response

The information requested has already been updated and included on the CDSS Web page. The CDSS has included an additional print out of the Web page which identifies the hyperlinks available to the user which will take the user to a full description of the programs funded. (See Attachment E.)

California State Auditor's Assessment of 6-Month Status: Fully Implemented

60-Day Agency Response

Social Services stated that the office updated its Web site to provide information on all the types of programs and services financed with all the fund sources it administers, including those financed wholly or in part by the trust fund, and to include target populations served.

California State Auditor's Assessment of 60-Day Status: Pending

Our review of the office's Web site (www.childsworld.ca.gov/PG2287.htm) on May 6, 2013, found that the office did not include a description of the programs and services and the target populations that benefit from the programs and services, as state law requires. Specifically, on its Web site, the office has a category titled Annual Expenditures that has a link to "SCTF". This link leads the user to a "Fund Fact Sheet" for the trust fund. The fact sheet has the categories of pilot and demonstration projects, dissemination, training and technical assistance, public education and outreach, and conference sponsorship. However, the office did not include a description of the programs and services provided by the Supporting Father Involvement Study, Evidence-Based Clearinghouse, Parent Leadership-Parents Anonymous, Parent Leadership-Conferences, and Strategies. Instead, the office provided a link to its March 2009 report on the trust fund's legal requirements. Further, for all of the programs and services listed in the previous sentence and the Mandated Reporter Research and Training program, the office did not identify the target populations that benefitted from the programs and services.

Recommendation #22 To: Social Services, Department of

To improve the presentation of the information it publishes for the trust fund, Social Services should present the amount it disbursed from the trust fund in the prior fiscal year by the amount spent for each program or service.

6-Month Agency Response

The information requested has already been updated and included on the CDSS Web page. The CDSS has included an additional print out of the Web page which identifies the hyperlinks available to the user which will take the user to a full description of the programs funded. (See Attachment E.)

California State Auditor's Assessment of 6-Month Status: Fully Implemented

60-Day Agency Response

Social Services' office updated its Web site with information on the amount disbursed from the trust fund in the prior fiscal year 2011-12 (as of June 30, 2012) by the amount spent for each program or service.

California State Auditor's Assessment of 60-Day Status: Fully Implemented

All Recommendations in 2012-105

Agency responses received after June 2013 are posted verbatim.