Report 2017-129 All Recommendation Responses
Report 2017-129: Department of Rehabilitation: Its Inadequate Guidance and Oversight of the Grant Process Led to Inconsistencies and Perceived Bias in Its Evaluations and Awards of Some Grants (Release Date: July 2018)
Recommendation #1 To: Rehabilitation, Department of
To comply with federal and state requirements, and to ensure consistency and fairness in its grant process, Rehabilitation should issue regulations describing its grant process from RFA development through appeals. It should submit its proposed regulations to the Office of Administrative Law no later than December 2018.
6-Month Agency Response
The Department submitted its proposed regulations to the Office of Administrative Law on December 17, 2018.
- Completion Date: December 2018
- Response Date: January 2019
California State Auditor's Assessment of 6-Month Status: Fully Implemented
60-Day Agency Response
The Department is in the process of drafting regulations describing its grant solicitation process from RFA development through appeals. The proposed regulations will be submitted to the Office of Administrative Law by December 2018.
- Estimated Completion Date: December 2018
- Response Date: September 2018
California State Auditor's Assessment of 60-Day Status: Pending
Recommendation #2 To: Rehabilitation, Department of
To ensure that management and staff involved in the grant process are sufficiently informed about the process and their responsibilities, Rehabilitation should require these employees to attend a kickoff meeting before the development of each RFA in which participants discuss the key stages of the grant review process, each individual's roles and responsibilities, and requirements surrounding conflicts of interest and confidentiality. Further, it should record these discussions in meeting minutes to ensure that expectations of employees are clearly defined and documented.
Annual Follow-Up Agency Response From October 2020
The Department revised the GSM in October 2018 to include these recommendations and implemented these practices during the Department's first RFA solicitation process after Report 2017-129 was published. These changes are contained in the GSM Interim Guidance: October 2018.
- Completion Date: February 2020
California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented
Annual Follow-Up Agency Response From October 2019
The Department has revised the GSM to include these recommendations and will carry out these practices in future grant solicitations. Prior to developing the RFA, the Department shall ensure that a meeting is held with all key personnel to discuss roles and responsibilities, conflicts and confidentiality.
- Estimated Completion Date: July 2020
California State Auditor's Assessment of Annual Follow-Up Status: Pending
1-Year Agency Response
The Department has revised the GSM to include these recommendations and will carry out these practices in future grant solicitations. Prior to developing the RFA, the Department shall ensure that a meeting is held with all key personnel to discuss roles and responsibilities, conflicts and confidentiality.
- Estimated Completion Date: August 2019
- Response Date: July 2019
California State Auditor's Assessment of 1-Year Status: Pending
6-Month Agency Response
The Department has revised the GSM to include these recommendations and will carry out these practices in future grant solicitations. Prior to developing the RFA, the Department shall ensure that a meeting is held with all key personnel to discuss roles and responsibilities, conflicts and confidentiality.
- Estimated Completion Date: GSM Interim Guidance- October 2018: Final GSM - After regulatory process has concluded
- Response Date: January 2019
California State Auditor's Assessment of 6-Month Status: Pending
60-Day Agency Response
The Department has revised the GSM to include these recommendations and will carry out these practices in future grant solicitations. Prior to developing the RFA, the Department shall ensure that a meeting is held with all key personnel to discuss roles and responsibilities, conflicts and confidentiality.
- Estimated Completion Date: October 2018 for Interim GSM; Final GSM after regulatory process has concluded
- Response Date: September 2018
California State Auditor's Assessment of 60-Day Status: Pending
Recommendation #3 To: Rehabilitation, Department of
To comply with state laws and regulations and help ensure that staff involved in making governmental decisions during the grant process are impartial, Rehabilitation should ensure that they receive ethics training, which includes conflict-of-interest training, at least every two years.
Annual Follow-Up Agency Response From September 2023
The Department is currently developing a Request for Applications for Traumatic Brain Injury and expects to award grants in or about July 2024. Thereafter, the Department will submit documentation demonstrating full implementation of this recommendation.
- Estimated Completion Date: August 2024
California State Auditor's Assessment of Annual Follow-Up Status: Pending
Annual Follow-Up Agency Response From October 2022
The Department finalized a Request for Application (RFA) for the Traumatic Brain Injury program in March 2022 and will provide the documentation to the CSA contact showing the Department recevied and verified that the ethics training was current for each staff person involved in the RFA process.
- Completion Date: March 2022
California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented
Rehabilitation has established a process to monitor completion of ethics training by its staff members. To demonstrate that it had implemented this process, Rehabilitation provided us with ethics training certificates for seven individuals involved in the Traumatic Brain Injury program grant, which it finalized in March 2022. However, three of the seven training certificates expired before the onset of the grant. As a result, Rehabilitation has not ensured that the staff involved in making governmental decisions during the grant process were impartial. We look forward to Rehabilitation being able to demonstrate full implementation of its process it its future responses.
- Auditee did not substantiate its claim of full implementation
- Auditee did not address all aspects of the recommendation
Annual Follow-Up Agency Response From October 2021
The Department's Human Resources Section is responsible for monitoring that DOR staff receive ethics training in compliance with state laws and regulations. Human Resources Section reminds staff to complete these tasks and notifies the staff member's supervisor and Deputy Director, as appropriate, if the staff member has not timely completed the task.
The Department put out a Request for Application (RFA) for the Traumatic Brain Injury Program September 2021. Per CSA's request the Department will provide documentation that will demonstrate full implementation of the CSA recommendation after grant award.
- Estimated Completion Date: 1/2022
California State Auditor's Assessment of Annual Follow-Up Status: Pending
Annual Follow-Up Agency Response From October 2020
The Department's Labor Relations Officer (LRO) is its Filing Officer responsible for monitoring that DOR staff receive ethics training in compliance with state laws and regulations. The LRO reminds staff to complete these tasks and notifies the staff member's supervisor and Deputy Director, as appropriate, if the staff member has not timely completed the tasks. The Department has attached a description of the process the LRO uses to conduct their monitoring of completion of ethics training and has attached verification that the process has been implemented for compliance period 2017-2018.
- Completion Date: January 2019
California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented
Although Rehabilitation has established a process for monitoring some staff's completion of ethics training, it does not require all staff involved in the grant process to complete such training. Additionally, for the most recent grant that it administered, Rehabilitation did not demonstrate that all staff involved in the grant completed ethics training. Until Rehabilitation can demonstrate it has addressed these shortcomings, we will continue to report the status of this recommendation as not fully implemented.
- Auditee did not substantiate its claim of full implementation
- Auditee did not address all aspects of the recommendation
Annual Follow-Up Agency Response From October 2019
The Department's Labor Relations Officer (LRO) is its Filing Officer responsible for monitoring that DOR staff receive ethics training in compliance with state laws and regulations. The LRO reminds staff to complete these tasks and notifies the staff member's supervisor and Deputy Director, as appropriate, if the staff member has not timely completed the tasks. The Department has attached a description of the process the LRO uses to conduct their monitoring of completion of ethics training and has attached verification that the process has been implemented for compliance period 2017-2018.
- Completion Date: January 2019
California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented
- Auditee did not substantiate its claim of full implementation
Annual Follow-Up Agency Response From October 2019
The Department's Labor Relations Officer (LRO) is its Filing Officer responsible for monitoring that DOR staff receive ethics training in compliance with state laws and regulations. The LRO reminds staff to complete these tasks and notifies the staff member's supervisor and Deputy Director, as appropriate, if the staff member has not timely completed the tasks. The Department has previously provided a description of the process the LRO uses to conduct their monitoring of completion of ethics training, and has implemented verification that the process for compliance period 2017-2018
- Completion Date: January 2019
California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented
Rehabilitation has established a process to monitor completion of ethics training by its staff members. However, we will continue to report this recommendation as partially implemented until it can demonstrate compliance with the new process during its next administration of a grant.
- Auditee did not address all aspects of the recommendation
1-Year Agency Response
The Department's Labor Relations Officer (LRO) is its Filing Officer responsible for monitoring that DOR staff receive ethics training in compliance with state laws and regulations. The LRO reminds staff to complete these tasks and notifies the staff member's supervisor and Deputy Director, as appropriate, if the staff member has not timely completed the tasks. The Department has attached a description of the process the LRO uses to conduct their monitoring of completion of ethics training and has attached verification that the process has been implemented for compliance period 2017-2018.
- Completion Date: January 2019
- Response Date: July 2019
California State Auditor's Assessment of 1-Year Status: Partially Implemented
Rehabilitation has established a process to monitor completion of ethics training by its staff members. Although the department has demonstrated that it is working toward full compliance, we look forward to revisiting its implementation of the process upon its next administration of a grant.
6-Month Agency Response
The Department's Labor Relations Officer (LRO) is its Filing Officer responsible for monitoring that DOR staff receive ethics training in compliance with state laws and regulations. The LRO reminds staff to complete these tasks and notifies the staff member's supervisor and Deputy Director, as appropriate, if the staff member has not timely completed the tasks. The Department has attached a description of the process the LRO uses to conduct their monitoring of completion of ethics training, and has attached verification that the process has been implemented for compliance period 2017-2018.
- Completion Date: January 2019
- Response Date: January 2019
California State Auditor's Assessment of 6-Month Status: Partially Implemented
Rehabilitation has established a process to monitor completion of ethics training by its staff members. Although the department has demonstrated that it is working toward full compliance, we look forward to revisiting its implementation of the process upon its next administration of a grant.
- Auditee did not substantiate its claim of full implementation
60-Day Agency Response
The Department reinstituted the practice to identify any staff who must complete the ethics training and who has not done so timely, through monitoring of the documentation by the Ethics Officer (Labor Relations Officer)who will notify the Deputy Director when any employee is not current. The Department will provide documentation to demonstrate implementation of this practice at the next reporting period.
- Estimated Completion Date: January 2019
- Response Date: October 2018
California State Auditor's Assessment of 60-Day Status: Pending
Recommendation #4 To: Rehabilitation, Department of
To help ensure that staff involved in the grant process adequately protect confidential information, Rehabilitation should develop confidentiality procedures for each grant. Further, it should ensure that staff involved in the grant process sign the conflict-of-interest and confidentiality forms before the development of the RFA for each grant.
Annual Follow-Up Agency Response From October 2020
The Department revised the GSM in October 2018 to include these recommendations and implemented these practices during the Department's first RFA solicitation process after Report 2017-129 was published. These changes are contained in the GSM Interim Guidance: October 2018.
- Completion Date: February 2020
California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented
Annual Follow-Up Agency Response From October 2019
The Department is in the process of developing confidentiality procedures applicable to the grant solicitation process and will carry out these practices in future grant solicitations.
- Estimated Completion Date: July 2020
California State Auditor's Assessment of Annual Follow-Up Status: Pending
1-Year Agency Response
The Department is in the process of developing confidentiality procedures applicable to the grant solicitation process and will carry out these practices in future grant solicitations.
- Estimated Completion Date: August 2019
- Response Date: July 2019
California State Auditor's Assessment of 1-Year Status: Pending
6-Month Agency Response
The Department is in the process of developing confidentiality procedures applicable to the grant solicitation process and will carry out these practices in future grant solicitations.
- Estimated Completion Date: GSM Interim Guidance - October 2018: Final GSM - After regulatory process has concluded
- Response Date: January 2019
California State Auditor's Assessment of 6-Month Status: Pending
60-Day Agency Response
The Department is in the process of developing confidentiality procedures applicable to the grant solicitation process and will carry out these practices in future grant solicitations.
- Estimated Completion Date: October 2018 for Interim GSM; Final GSM after regulatory process has concluded
- Response Date: September 2018
California State Auditor's Assessment of 60-Day Status: Pending
Recommendation #5 To: Rehabilitation, Department of
To ensure that it has received sufficient input and feedback from the disability community to inform the development of RFAs, Rehabilitation should solicit and document stakeholder input and feedback before and during the development of each RFA.
Annual Follow-Up Agency Response From October 2020
The Department revised the GSM in October 2018 to include these recommendations and implemented these practices during the Department's first RFA solicitation process after Report 2017-129 was published. These changes are contained in the GSM Interim Guidance: October 2018.
- Completion Date: February 2020
California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented
Annual Follow-Up Agency Response From October 2019
The Department is in the process of developing confidentiality procedures applicable to the grant solicitation process and will carry out these practices in future grant solicitations.
- Estimated Completion Date: July 2020
California State Auditor's Assessment of Annual Follow-Up Status: Pending
1-Year Agency Response
The Department has revised the GSM to include these recommendations and will carry out these practices in future grant solicitations.
- Estimated Completion Date: August 2019
- Response Date: July 2019
California State Auditor's Assessment of 1-Year Status: Pending
6-Month Agency Response
The Department has revised the GSM to include these recommendations and will carry out these practices in future grant solicitations.
- Estimated Completion Date: GSM Interim Guidance - October 2018: Final GSM - After regulatory process has concluded
- Response Date: January 2019
California State Auditor's Assessment of 6-Month Status: Pending
60-Day Agency Response
The Department has revised the GSM to include these recommendations and will carry out these practices in future grant solicitations.
- Estimated Completion Date: October 2018 for Interim GSM; Final GSM after regulatory process has concluded
- Response Date: September 2018
California State Auditor's Assessment of 60-Day Status: Pending
Recommendation #6 To: Rehabilitation, Department of
To increase transparency and ensure that applicants have the information necessary to understand the grant process, Rehabilitation should include in its RFAs clear scoring criteria and descriptions of the evaluation, award, and appeals processes, including the process it will use to address applications that receive tied scores.
Annual Follow-Up Agency Response From October 2022
The Traumatic Brain Injury RFA awarded in March 2022 included clear tie breaker language. The Department will provide the CSA contact the RFA with highlighted language that addresses the tied score process. Page 15 of the RFA contains tie score language.
- Completion Date: March 2022
California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented
Annual Follow-Up Agency Response From October 2021
The current Traumatic Brain Injury Request or Application (RFA) released in September 2021 includes clear tie breaker language. Per CSA's request the Department will provide documentation to demonstrate full implementation of the CSA recommendation after grant award.
- Estimated Completion Date: 1/2022
California State Auditor's Assessment of Annual Follow-Up Status: Pending
Annual Follow-Up Agency Response From October 2020
The Department revised the GSM in October 2018 to include these recommendations and implemented these practices during the Department's first RFA solicitation process after Report 2017-129 was published. These changes are contained in the GSM Interim Guidance: October 2018.
- Completion Date: February 2020
California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented
Rehabilitation's revised grant solicitation manual states that the method of the tiebreaker must be provided in the RFA. However, Rehabilitation did not include its tiebreaker methodology in its most recent RFA for the Systems Change grant. Because this information was not included, during a bidders' conference related to the grant, a stakeholder inquired about the tiebreaker methodology. Although Rehabilitation provided the methodology at that time, it should have included the tiebreaker methodology in its RFA directly. As a result, we have assessed the status of this recommendation as partially implemented. We look forward to reviewing Rehabilitation's future annual responses to ensure it has fully implemented this recommendation.
Annual Follow-Up Agency Response From October 2019
The Department is in the process of developing confidentiality procedures applicable to the grant solicitation process and will carry out these practices in future grant solicitations.
- Estimated Completion Date: July 2020
California State Auditor's Assessment of Annual Follow-Up Status: Pending
1-Year Agency Response
The Department has revised the GSM to include these recommendations and will carry out these practices in future grant solicitations.
- Estimated Completion Date: August 2019
- Response Date: July 2019
California State Auditor's Assessment of 1-Year Status: Pending
6-Month Agency Response
The Department has revised the GSM to include these recommendations and will carry out these practices in future grant solicitations.
- Estimated Completion Date: GSM Interim Guidance - October 2018: Final GSM - After regulatory process has concluded
- Response Date: January 2019
California State Auditor's Assessment of 6-Month Status: Pending
60-Day Agency Response
The Department has revised the GSM to include these recommendations and will carry out these practices in future grant solicitations.
- Estimated Completion Date: October 2018 for Interim GSM; Final GSM after regulatory process has concluded
- Response Date: September 2018
California State Auditor's Assessment of 60-Day Status: Pending
Recommendation #7 To: Rehabilitation, Department of
To ensure that Rehabilitation maintains all relevant grant documentation and responds fully to requests for public records, it should immediately adhere to its records retention policy and save all grant-related documents, including email correspondence and attachments, to a centralized location.
Annual Follow-Up Agency Response From October 2020
The Department revised the GSM in October 2018 to include these recommendations and implemented these practices during the Department's first RFA solicitation process after Report 2017-129 was published. These changes are contained in the GSM Interim Guidance: October 2018.
- Completion Date: February 2020
California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented
Annual Follow-Up Agency Response From October 2019
The Department is in the process of developing confidentiality procedures applicable to the grant solicitation process and will carry out these practices in future grant solicitations.
- Estimated Completion Date: July 2020
California State Auditor's Assessment of Annual Follow-Up Status: Pending
1-Year Agency Response
The Department has revised the GSM to include these recommendations and will carry out these practices in future grant solicitations.
- Estimated Completion Date: August 2019
- Response Date: July 2019
California State Auditor's Assessment of 1-Year Status: Pending
6-Month Agency Response
The Department has revised the GSM to include these recommendations and will carry out these practices in future grant solicitations.
- Estimated Completion Date: GSM Interim Guidance - October 2018: Final GSM - After regulatory process has concluded
- Response Date: January 2019
California State Auditor's Assessment of 6-Month Status: Pending
60-Day Agency Response
The Department has revised the GSM to include these recommendations and will carry out these practices in future grant solicitations.
- Estimated Completion Date: October 2018 for Interim GSM; Final GSM after regulatory process has concluded
- Response Date: September 2018
California State Auditor's Assessment of 60-Day Status: Pending
Recommendation for Legislative Action
To avoid bias or the perception of bias, the Legislature should enact legislation that prohibits state agencies from selecting as an evaluator of grant applications a representative, former member, or former staff of any organization or person that is applying to receive grant funding from the state agency.
Description of Legislative Action
AB 1013 (Chapter 498, Statutes of 2019) prohibits a state agency from permitting an evaluator to review a discretionary grant application submitted by an organization or a person for which the evaluator was a representative, voting member, or staff member within the two year period preceding receipt of that application.
- Legislative Action Current As-of: July 2020
California State Auditor's Assessment of Annual Follow-Up Status: Legislation Enacted
Description of Legislative Action
AB 1013 (Obernolte) would prohibit a state agency from permitting an evaluator to review a discretionary grant application submitted by an organization or a person for which the evaluator was a representative, voting member, or staff member within the two year period preceding receipt of that application.
- Legislative Action Current As-of: August 2019
California State Auditor's Assessment of 1-Year Status: Legislation Introduced
Description of Legislative Action
The Legislature has not taken action to address this specific recommendation.
- Legislative Action Current As-of: January 2019
California State Auditor's Assessment of 6-Month Status: No Action Taken
Description of Legislative Action
The Legislature has not taken action to address this specific recommendation.
- Legislative Action Current As-of: September 2018
California State Auditor's Assessment of 60-Day Status: No Action Taken
Recommendation #9 To: Rehabilitation, Department of
To ensure consistency and fairness in the evaluation process, Rehabilitation should make sure that it accepts only complete applications submitted before the deadline, unless otherwise specified in the RFA. If the RFA specifies a hard deadline and applicants submit incomplete applications, Rehabilitation should not accept any portions of the applications submitted after the deadline and should assess the penalty for incomplete applications specified in the RFA.
Annual Follow-Up Agency Response From October 2020
The Department revised the GSM in October 2018 to include these recommendations and implemented these practices during the Department's first RFA solicitation process after Report 2017-129 was published. These changes are contained in the GSM Interim Guidance: October 2018.
- Completion Date: February 2020
California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented
Annual Follow-Up Agency Response From October 2019
The Department is in the process of developing confidentiality procedures applicable to the grant solicitation process and will carry out these practices in future grant solicitations.
- Estimated Completion Date: July 2020
California State Auditor's Assessment of Annual Follow-Up Status: Pending
1-Year Agency Response
The Department has revised the GSM to include these recommendations and will carry out these practices in future grant solicitations.
- Estimated Completion Date: August 2019
- Response Date: July 2019
California State Auditor's Assessment of 1-Year Status: Pending
6-Month Agency Response
The Department has revised the GSM to include these recommendations and will carry out these practices in future grant solicitations.
- Estimated Completion Date: GSM Interim Guidance - October 2018: Final GSM - After regulatory process has concluded
- Response Date: January 2019
California State Auditor's Assessment of 6-Month Status: Pending
60-Day Agency Response
The Department has revised the GSM to include these recommendations and will carry out these practices in future grant solicitations.
- Estimated Completion Date: October 2018 for Interim GSM; Final GSM after regulatory process has concluded
- Response Date: September 2018
California State Auditor's Assessment of 60-Day Status: Pending
Recommendation #10 To: Rehabilitation, Department of
To help ensure that evaluators adequately protect confidential information and that the evaluation process is fair, Rehabilitation should develop standardized evaluator training for confidentiality procedures and conflicts of interest, including a discussion of bias or the appearance of bias. Rehabilitation should also ensure that the candidates receive this training and sign conflict-of-interest and confidentiality forms before it selects evaluators. Further, it should prohibit program staff who participate in the development of an RFA from acting as evaluators for the applications Rehabilitation receives in response to that RFA.
Annual Follow-Up Agency Response From October 2020
The Department revised the GSM in October 2018 to include these recommendations and implemented these practices during the Department's first RFA solicitation process after Report 2017-129 was published. These changes are contained in the GSM Interim Guidance: October 2018.
- Completion Date: February 2020
California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented
For its most recent grant, Rehabilitation demonstrated that it provided its evaluators with training regarding its confidentiality procedures and conflicts of interest, including a discussion of bias or the appearance of bias. Additionally, Rehabilitation had each evaluator sign a conflict-of-interest and confidentiality form certifying that they were free from conflicts or biases. None of the evaluators were program staff who participated in the development of the RFA. Based on this information, we have assessed the recommendation as fully implemented.
Annual Follow-Up Agency Response From October 2019
The Department is in the process of developing confidentiality procedures applicable to the grant solicitation process and will carry out these practices in future grant solicitations.
- Estimated Completion Date: July 2020
California State Auditor's Assessment of Annual Follow-Up Status: Pending
1-Year Agency Response
The Department has revised the GSM to include these recommendations and will carry out these practices in future grant solicitations.
- Estimated Completion Date: August 2019
- Response Date: July 2019
California State Auditor's Assessment of 1-Year Status: Pending
6-Month Agency Response
The Department has revised the GSM to include these recommendations and will carry out these practices in future grant solicitations.
- Estimated Completion Date: GSM Interim Guidance - October 2018: Final GSM - After regulatory process has concluded
- Response Date: January 2019
California State Auditor's Assessment of 6-Month Status: Pending
60-Day Agency Response
The Department has revised the GSM to include these recommendations and will carry out these practices in future grant solicitations.
- Estimated Completion Date: October 2018 for Interim GSM; Final GSM after regulatory process has concluded
- Response Date: September 2018
California State Auditor's Assessment of 60-Day Status: Pending
Recommendation #11 To: Rehabilitation, Department of
To increase the transparency of its selection process and to ensure that it receives the most qualified evaluators possible, Rehabilitation should issue a public solicitation for evaluators for each grant that includes a description of essential and desirable qualifications.
Annual Follow-Up Agency Response From October 2022
Consistent with the Department's commitment to improving our grant award process, the RFA for the Traumatic Brain Injury Program awarded in March 2022 included the public solicitation for evaluators. The Department also implemented the use of a matrix to evaluate the evaluator applications prior to final selection. The matrix included an initial review for conflict of interest.
The Department will provide the CSA contact the public solicitation for evaluators and the matrix used to evaluate the applications submitted by potential evaluators.
- Completion Date: March 2022
California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented
Annual Follow-Up Agency Response From October 2021
In regard to the previous solicitation that CSA mentions in the example, we note that the Department determined that there was no evidence of evaluator bias, and there was sufficient evidence that the evaluator understood their responsibilities as an evaluator and completed their duties as an evaluator free of bias.
Consistent with the Department's commitment to improving our grant award process, the Request for Application (RFA) for the Traumatic Brain Injury Program September 2021 includes the public solicitation for evaluators. The Department also implemented the use of a matrix to evaluate the evaluator applications prior to final selection. The matrix included an initial review for conflict of interest.
Per CSA's request the Department will provide documentation that will demonstrate full implementation of the CSA recommendation after grant award.
- Estimated Completion Date: 1/2022
California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented
As we stated in our previous assessment, for its previous grant, Rehabilitation issued a public solicitation for evaluators, which included minimal qualification requirements. According to its Grant Solicitation Manual, Rehabilitation should also identify criteria that will disqualify potential evaluators who have the appearance of bias or conflicts of interest. However, Rehabilitation did not identify such criteria for that RFA. As a result, we found that one of its evaluators had at least an appearance of bias with one of the grant applicants. Aside from having evaluators self-certify that they are free from conflicts of interest or bias, Rehabilitation did not demonstrate that it has made efforts to ensure it receives the most qualified evaluators possible that are impartial and unbiased.
Annual Follow-Up Agency Response From October 2020
The Department revised the GSM in October 2018 to include these recommendations and implemented these practices during the Department's first RFA solicitation process after Report 2017-129 was published. These changes are contained in the GSM Interim Guidance: October 2018.
- Completion Date: February 2020
California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented
For its most recent grant, Rehabilitation issued a public solicitation for evaluators, which included minimal qualification requirements. According to its Grant Solicitation Manual, Rehabilitation should also identify criteria that will disqualify potential evaluators who have the appearance of bias or conflicts of interest. However, Rehabilitation did not identify such criteria for this RFA. As a result, we found that one of its evaluators had at least an appearance of bias with one of the grant applicants. Aside from having evaluators self-certify that they are free from conflicts of interest or bias, Rehabilitation did not demonstrate that it has made efforts to ensure it receives the most qualified evaluators possible that are impartial and unbiased.
- Auditee did not substantiate its claim of full implementation
- Auditee did not address all aspects of the recommendation
Annual Follow-Up Agency Response From October 2019
The Department is in the process of developing confidentiality procedures applicable to the grant solicitation process and will carry out these practices in future grant solicitations.
- Estimated Completion Date: July 2020
California State Auditor's Assessment of Annual Follow-Up Status: Pending
1-Year Agency Response
The Department has revised the GSM to include this recommendation and will carry out these practices in future grant solicitations.
- Estimated Completion Date: August 2019
- Response Date: July 2019
California State Auditor's Assessment of 1-Year Status: Pending
6-Month Agency Response
The Department has revised the GSM to include these recommendations and will carry out these practices in future grant solicitations.
- Estimated Completion Date: GSM Interim Guidance - October 2018: Final GSM - After regulatory process has concluded
- Response Date: January 2019
California State Auditor's Assessment of 6-Month Status: Pending
60-Day Agency Response
The Department has revised the GSM to include this recommendation and will carry out these practices in future grant solicitations.
- Estimated Completion Date: October 2018 for Interim GSM; Final GSM after regulatory process has concluded
- Response Date: September 2018
California State Auditor's Assessment of 60-Day Status: Pending
Recommendation #12 To: Rehabilitation, Department of
To ensure that evaluators have the information necessary to sufficiently and fairly assess and score applications, Rehabilitation should develop training by December 2018 that can be tailored to each grant and includes at minimum the following topics:
-The purpose and relevant regulatory requirements for the grant.
- Instructions on how to score applications, including an applicant's financial information, and direction that they must provide comments to support their scores.
Rehabilitation should provide this training to evaluators before allowing them to score applications.
Annual Follow-Up Agency Response From October 2020
The Department revised the GSM in October 2018 to include these recommendations and implemented these practices during the Department's first RFA solicitation process after Report 2017-129 was published. These changes are contained in the GSM Interim Guidance: October 2018.
- Completion Date: February 2020
California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented
Rehabilitation demonstrated that for its most recent grant, it included the relevant regulatory requirements in its RFA, which it provided to evaluators during its orientation and training session. It also provided evaluators with scoring instructions, which included how to score an applicant's financial information and that evaluators must provide comments to support their scores.
Annual Follow-Up Agency Response From October 2019
The Department is in the process of developing confidentiality procedures applicable to the grant solicitation process and will carry out these practices in future grant solicitations.
- Estimated Completion Date: July 2020
California State Auditor's Assessment of Annual Follow-Up Status: Pending
1-Year Agency Response
The Department has revised the GSM to include these recommendations and will carry out these practices in future grant solicitations.
- Estimated Completion Date: August 2019
- Response Date: July 2019
California State Auditor's Assessment of 1-Year Status: Pending
6-Month Agency Response
The Department has revised the GSM to include these recommendations and will carry out these practices in future grant solicitations.
- Estimated Completion Date: GSM Interim Guidance - October 2018: Develop Training - December 2018: Final GSM - After regulatory process has concluded
- Response Date: January 2019
California State Auditor's Assessment of 6-Month Status: Pending
60-Day Agency Response
The Department has revised the GSM to include these recommendations and will carry out these practices in future grant solicitations.
- Estimated Completion Date: October 2018 for Interim GSM; December 2018 for training development; Final GSM after the regulatory process has concluded
- Response Date: September 2018
California State Auditor's Assessment of 60-Day Status: Pending
Recommendation #13 To: Rehabilitation, Department of
To ensure that it provides sufficient oversight of the grant process, Rehabilitation should ensure that the technical review teams it assigns to grants provide the director and chief deputy with a memorandum summarizing the evaluation process and the evaluators' recommended grant awardees. Rehabilitation should also designate an individual responsible for reviewing and approving the memorandum and recommended awardees before it publishes its notice of intent to award.
Annual Follow-Up Agency Response From November 2023
The Department is currently developing a Request for Applications for Traumatic Brain Injury and expects to award grants in or about July 2024. Thereafter, the Department will submit documentation demonstrating full implementation of this recommendation.
- Estimated Completion Date: August 2024
California State Auditor's Assessment of Annual Follow-Up Status: Pending
Annual Follow-Up Agency Response From October 2022
The Department awarded grants through the RFA for the Traumatic Brain Injury Program in March 2022. The Program Director notified the Chief Deputy by sending an official memorandum summarizing the evaluation process and the evaluation panel's recommendation before posting the Notice of Intent to Award. The Department will provide the CSA contact the memo from the Program Director to the Chief Deputy that was sent via email prior to posting the Notice of Intent to Award.
- Completion Date: March 2022
California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented
Since we made this recommendation, Rehabilitation has modified its grant manual, including the responsibilities for reviewing the evaluations of grant applications and issuing a summary memorandum to the director and chief deputy. Currently, the manual assigns responsibility for reviewing the evaluation process to a Post Evaluation Review individual or team designated by the administrative services deputy director, and the program director administering the grant issues a summary memorandum to the director and chief deputy. The manual does not designate the person responsible for reviewing and approving the memorandum and recommended awardees before publishing the notice of intent to award, as we recommended. Thus, until it does so, we will continue to report this recommendation as not fully implemented.
The intent of our recommendation served two purposes. First, for a separation of duties, the individual or team conducting post evaluation reviews should be appointed and overseen separately from the program staff who select the initial evaluators and oversee the grant administration. Thus, if followed, Rehabilitation's revisions to its grant manual accomplish this goal. Second, to ensure that the director and chief deputy are fully informed of critical aspects of the grant process, we recommended that Rehabilitation designate the person responsible for reviewing and approving the memorandum and intended awardees. However, Rehabilitation has not implemented this portion of the recommendation.
Further, we found inconsistencies in the documentation Rehabilitation provided regarding the Traumatic Brain Injury Program grant mentioned in its response. For example, the program deputy, rather than the administrative services deputy, appointed the post evaluation reviewer. Further, the program deputy inaccurately described the evaluation review in the summary memorandum to the director and chief deputy, stating that the review did not observe any discrepancies in evaluator scoring or noncompliance with scoring instructions, even though the post evaluation review noted several issues with evaluators not appropriately supporting their scores. Additionally, Rehabilitation could not demonstrate that the director or chief deputy reviewed or approved the memorandum and intended awardees before the department published the notice of intent to award.
We look forward to reviewing Rehabilitation's progress in implementing this recommendation during its future responses.
- Auditee did not substantiate its claim of full implementation
- Auditee did not address all aspects of the recommendation
Annual Follow-Up Agency Response From October 2021
The Department put out a Request for Application (RFA) for the Traumatic Brain Injury Program September 2021. The Program Director will notify the Chief Deputy by sending an official memorandum summarizing the evaluation process and the evaluation panel's recommendation before posting the Notice of Intent to Award. Per CSA's request the Department will provide documentation that will demonstrate full implementation of the CSA recommendation after grant award.
- Estimated Completion Date: 1/2022
California State Auditor's Assessment of Annual Follow-Up Status: Pending
Annual Follow-Up Agency Response From October 2020
The Department revised the GSM in October 2018 to include these recommendations and implemented these practices during the Department's first RFA solicitation process after Report 2017-129 was published. These changes are contained in the GSM Interim Guidance: October 2018.
- Completion Date: February 2020
California State Auditor's Assessment of Annual Follow-Up Status: Pending
Rehabilitation's grant solicitation manual does not require its technical review team to issue a memorandum to the director and chief deputy summarizing the evaluation process and the evaluators' recommended grant awardees. Despite its grant solicitation manual assigning responsibility to the review team for reviewing and verifying that the evaluation panel followed applicable scoring instructions and processes, Rehabilitation did not demonstrate that the review team conducted such a review for its most recent grant. As such, it also did not summarize its findings to the director and chief deputy. Further, Rehabilitation did not demonstrate that it appointed an individual who was responsible for reviewing and approving the memorandum and recommended awardees before publishing the notice of intent to award. Until it addresses these concerns, we will continue to report this recommendation as not fully implemented.
- Auditee did not substantiate its claim of full implementation
- Auditee did not address all aspects of the recommendation
Annual Follow-Up Agency Response From October 2019
The Department is in the process of developing confidentiality procedures applicable to the grant solicitation process and will carry out these practices in future grant solicitations.
- Estimated Completion Date: July 2020
California State Auditor's Assessment of Annual Follow-Up Status: Pending
1-Year Agency Response
The Department has revised the GSM to include these recommendations and will carry out these practices in future grant solicitations.
- Estimated Completion Date: August 2019
- Response Date: July 2019
California State Auditor's Assessment of 1-Year Status: Pending
6-Month Agency Response
The Department has revised the GSM to include these recommendations and will carry out these practices in future grant solicitations.
- Estimated Completion Date: GSM Interim Guidance - October 2018: Final GSM - After regulatory process has concluded
- Response Date: January 2019
California State Auditor's Assessment of 6-Month Status: Pending
60-Day Agency Response
The Department has revised the GSM to include these recommendations and will carry out these practices in future grant solicitations.
- Estimated Completion Date: October 2018 for Interim GSM; Final GSM after regulatory process has concluded
- Response Date: September 2018
California State Auditor's Assessment of 60-Day Status: Pending
Recommendation #14 To: Rehabilitation, Department of
If it finds errors in an evaluation that merit restarting the grant process, rescoring of applications, or convening a new evaluation panel, Rehabilitation should resolve any issues before it begins the rescoring process. It should also notify applicants to ensure that they are aware of any changes to the process due to the errors. Further, it should consider promulgating regulations and amending its grant manual to permit staff to request evaluators to rescore applications or convene a new evaluation panel when it finds issues with an evaluation.
Annual Follow-Up Agency Response From September 2023
The Department revised its Grant Solicitation Manual (GSM) based on community input and the recommendations in the audit report. After approval of the rulemaking process, the GSM will be adopted in its final form.
In 2019 and 2022, the Department withdrew its rulemaking packages after consultation with the Office of Administrative Law about the proposed sections related to the development of requests for applications. The Department expects to notice proposed regulations in October 2023 without the proposed sections related to RFA development and submit them to the Office of Administrative Law for approval by July 2024. Regarding the recommendation to promulgate regulations for development of requests for applications, by September 29, 2023, the Department will submit a request to the Principal Auditor for further discussion.
The final GSM will be published by December 2024.
- Estimated Completion Date: December 2024
California State Auditor's Assessment of Annual Follow-Up Status: Pending
Annual Follow-Up Agency Response From October 2022
The Department revised its Grant Solicitation Manual (GSM) based on community input and the recommendations in the audit report. After approval of the rulemaking process, the GSM will be adopted in its final form.
- Estimated Completion Date: June 2023
California State Auditor's Assessment of Annual Follow-Up Status: Pending
Annual Follow-Up Agency Response From October 2021
In regard to the previous solicitation that CSA mentions in the example, we note that the Department determined that there was no evidence of evaluator bias, and there was sufficient evidence that the evaluator understood their responsibilities as an evaluator and completed their duties as an evaluator free of bias.
Consistent with the Department's commitment to improving our grant award process, the Request for Application (RFA) for the Traumatic Brain Injury Program September 2021 includes the public solicitation for evaluators. The Department will provide more guidance to the post evaluation staff person assigned to review the evaluation panel selection thoroughly. Per CSA's request the Department will provide documentation that will demonstrate full implementation of the CSA recommendation after grant award.
- Estimated Completion Date: 1/2022
California State Auditor's Assessment of Annual Follow-Up Status: Pending
As we stated in our previous assessment, for its previous grant, Rehabilitation issued a public solicitation for evaluators, which included minimal qualification requirements. According to its Grant Solicitation Manual, Rehabilitation should also identify criteria that will disqualify potential evaluators who have the appearance of bias or conflicts of interest. However, Rehabilitation did not identify such criteria for that RFA. As a result, we found that one of its evaluators had at least an appearance of bias with one of the grant applicants. Aside from having evaluators self-certify that they are free from conflicts of interest or bias, Rehabilitation did not demonstrate that it has made efforts to ensure it receives the most qualified evaluators possible that are impartial and unbiased.
Annual Follow-Up Agency Response From October 2020
The Department revised the GSM in October 2018 to include these recommendations and implemented these practices during the Department's first RFA solicitation process after Report 2017-129 was published. These changes are contained in the GSM Interim Guidance: October 2018.
- Completion Date: February 2020
California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented
Rehabilitation is in the process of promulgating regulations, and it has amended its grant manual. Rehabilitation states in its grant solicitation manual that, in addition to the preliminary review of the evaluation process by the technical review team, it will designate staff separate from those involved in the grant process to conduct a review for procedural errors, evaluator prejudice, and whether evaluators supported their scores with evidence from the relevant applications. Rehabilitation did not demonstrate that it conducted the preliminary review, and its post-evaluation review is questionable. Specifically, we identified at least one evaluator who had, at minimum, a perceived bias, which the post-evaluation review team did not identify. Had it sufficiently evaluated the grant process, it would have identified this perceived bias, which likely would have merited the convening of a new evaluation panel.
- Auditee did not substantiate its claim of full implementation
- Auditee did not address all aspects of the recommendation
Annual Follow-Up Agency Response From October 2019
In addition to incorporating this recommendation into regulations, the Department has revised the GSM to include these recommendations and will carry out these practices in future grant solicitations.
- Estimated Completion Date: July 2020
California State Auditor's Assessment of Annual Follow-Up Status: Pending
1-Year Agency Response
In addition to incorporating this recommendation into regulations, the Department has revised the GSM to include these recommendations and will carry out these practices in future grant solicitations.
- Estimated Completion Date: August 2019
- Response Date: July 2019
California State Auditor's Assessment of 1-Year Status: Pending
6-Month Agency Response
In addition to incorporating this recommendation into regulations, the Department has revised the GSM to include these recommendations and will carry out these practices in future grant solicitations.
- Estimated Completion Date: GSM Interim Guidance - October 2018: Final GSM - After regulatory process has concluded: Regulations - June 1, 2019
- Response Date: January 2019
California State Auditor's Assessment of 6-Month Status: Pending
60-Day Agency Response
In addition to incorporating this recommendation into regulations, the Department has revised the GSM to include these recommendations and will carry out these practices in future grant solicitations.
- Estimated Completion Date: October 2018 for Interim GSM; Final GSM after regulatory process concluded; Regulations June 2019
- Response Date: September 2018
California State Auditor's Assessment of 60-Day Status: Pending
Recommendation #15 To: Rehabilitation, Department of
To ensure that it consistently and thoroughly evaluates appeals, Rehabilitation should establish in state regulations and its grant manual that staff at the appropriate level of authority are to acknowledge all appeal requests, notify intended awardees that could be affected by the appeals, and inform the appellant of the qualifications of the review committee members. Staff at the appropriate level of authority must also notify all affected parties of the review committee's final decision within the time frame Rehabilitation establishes in regulations.
Annual Follow-Up Agency Response From September 2023
The Department revised its Grant Solicitation Manual (GSM) based on community input and the recommendations in the audit report. The Department went through the rulemaking process and in December 2019 discovered that it was necessary to revise the proposed regulations.
In 2019 and 2022, the Department withdrew its rulemaking packages after consultation with the Office of Administrative Law about the proposed sections related to the development of requests for applications. The Department expects to notice proposed regulations in October 2023 without the proposed sections related to RFA development and submit them to the Office of Administrative Law for approval by July 2024. Regarding the recommendation to promulgate regulations for development of requests for applications, by September 29, 2023, the Department will submit a request to the Principal Auditor for further discussion.
The final GSM will be published by December 2024.
- Estimated Completion Date: December 2024
California State Auditor's Assessment of Annual Follow-Up Status: Pending
Annual Follow-Up Agency Response From October 2022
The Department revised its Grant Solicitation Manual (GSM) based on community input and the recommendations in the audit report. The Department went through the rulemaking process and in December 2019 discovered that it was necessary to revise the proposed regulations.
The Department received written and public comment, including the public hearing on this rulemaking package and is not in the process of completing the rulemaking file to submit to the Office of Administrative Law in accordance with the Administrative Procedures Act. The Department expects approval of this rulemaking package in December 2022.
The final GSM will be published after the regulations are adopted. In the interim, the Department published an Interim GSM on October 29, 2018 for use until the regulatory process has concluded.
- Estimated Completion Date: December 2022
California State Auditor's Assessment of Annual Follow-Up Status: Pending
Annual Follow-Up Agency Response From October 2021
The Department revised its Grant Solicitation Manual (GSM) based on community input and the recommendations in the audit report. The Department went through the rulemaking process and in December 2019 discovered that it was necessary to revise the proposed regulations.
The Department has completed a draft of the proposed text for the grant regulations. Consistent with the Department's mission statement and core principles of partnership and transparency, the Department is engaging with its advisory bodies to obtain their input on the proposed concepts and approach before finalizing the proposed text and commencing the rulemaking process, including publication of the notice of proposed rulemaking and public comment period. The anticipated engagements include, but may not be limited to, the State Rehabilitation Council, Blind Advisory Council, State Independent Living Centers, and Assistive Technology Advisory Committee at their regularly scheduled public meetings.
The final GSM will be published after the regulations are adopted. In the interim, the Department published an Interim GSM on October 29, 2018 for use until the regulatory process has concluded.
- Estimated Completion Date: 12/2022
California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented
Annual Follow-Up Agency Response From October 2020
The Department revised the GSM in October 2018 to include these recommendations and implemented these practices during the Department's first RFA solicitation process after Report 2017-129 was published. These changes are contained in the GSM Interim Guidance: October 2018.
- Completion Date: February 2020
California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented
Rehabilitation is in the process of promulgating regulations, and it has amended its grant manual. Its grant manual provides that staff shall acknowledge each appeal request, notify the intended awardee, inform the appellant of the qualifications of the review committee, notify all affected parties of the review committee's final decision, and notify all affected parties of its decision within 30 days. Rehabilitation demonstrated that it followed each of these steps for its most recent grant. However, we will continue to report this recommendation as partially implemented until its regulations are finalized.
- Auditee did not address all aspects of the recommendation
Annual Follow-Up Agency Response From October 2019
The Chief Deputy Director, responsible for appointing the review committee, will implement these changes. In addition to incorporating this recommendation into regulations, the Department has revised the GSM to include these recommendations and will carry out these practices in future grant appeals.
- Estimated Completion Date: July 2020
California State Auditor's Assessment of Annual Follow-Up Status: Pending
1-Year Agency Response
The Chief Deputy Director, responsible for appointing the review committee, will implement these changes. In addition to incorporating this recommendation into regulations, the Department has revised the GSM to include these recommendations and will carry out these practices in future grant appeals.
- Estimated Completion Date: August 2019
- Response Date: July 2019
California State Auditor's Assessment of 1-Year Status: Pending
6-Month Agency Response
The Chief Deputy Director, responsible for appointing the review committee, will implement these changes. In addition to incorporating this recommendation into regulations, the Department has revised the GSM to include these recommendations and will carry out these practices in future grant appeals.
- Estimated Completion Date: GSM Interim Guidance - October 2018: Final GSM - After regulatory process has concluded: Regulations - June 1, 2019
- Response Date: January 2019
California State Auditor's Assessment of 6-Month Status: Pending
60-Day Agency Response
The Chief Deputy Director, responsible for appointing the review committee, will implement these changes. In addition to incorporating this recommendation into regulations, the Department has revised the GSM to include these recommendations and will carry out these practices in future grant appeals.
- Estimated Completion Date: October 2018 for Interim GSM; Final GSM after regulatory process has concluded; Regulations June 2019
- Response Date: September 2018
California State Auditor's Assessment of 60-Day Status: Pending
Recommendation #16 To: Rehabilitation, Department of
To ensure that Rehabilitation has appropriate oversight of its grant process and can sufficiently demonstrate that it followed the process, it should designate staff, separate from those involved in the respective grant process, to conduct a review of each grant process for procedural errors, evaluator prejudice, and whether evaluators supported their scores with evidence from the relevant applications before it awards grants.
Annual Follow-Up Agency Response From September 2023
The Department is currently developing a Request for Applications for Traumatic Brain Injury and expects to award grants in or about July 2024. Thereafter, the Department will submit documentation demonstrating full implementation of this recommendation.
- Estimated Completion Date: August 2024
California State Auditor's Assessment of Annual Follow-Up Status: Pending
Annual Follow-Up Agency Response From October 2022
The Department finalized a RFA for the Traumatic Brain Injury program in March 2022 and provided guidance and tools for use during the post evaluation review to the post evaluation review staff member. The Department will provide the CSA contact with documentation of the correspondence sent by the program director to the post evaluation review staff member providing guidance and tools to ensure understanding of the process. This correspondence will demonstrate full implementation of the CSA recommendation after grant award.
- Completion Date: March 2022
California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented
Although Rehabilitation provides some guidance to its post evaluation reviewer, it does not describe how the department expects the reviewer to conduct their review. For example, it provides the reviewer with the evaluators' qualifications and self-certifications, indicating that they do not have a conflict of interest. However, the guidance does not describe its expectations of the steps the reviewer should take to ensure that these certifications are accurate. Because we identified potential conflicts of interest in this audit and another subsequent grant through basic internet searches, we expected that Rehabilitation would provide comprehensive guidance to its reviewer with the necessary steps to verify that evaluators are free from prejudice. Thus, until it does so, we will report this recommendation as not fully implemented.
We look forward to reviewing Rehabilitation's progress toward implementing this recommendation during future responses.
- Auditee did not substantiate its claim of full implementation
- Auditee did not address all aspects of the recommendation
Annual Follow-Up Agency Response From October 2021
Please see the Department's response to CSA recommendations #3, #10 and #13 for specific improvements to our process which are reflected in our current Request for Application (RFA) for the Traumatic Brain Injury Program September 2021 process. Further, the Department will provide more guidance to the post evaluation staff person assigned to review the RFA process thoroughly. Per CSA's request the Department will provide documentation that will demonstrate full implementation of the CSA recommendation after grant award.
- Estimated Completion Date: 1/2022
California State Auditor's Assessment of Annual Follow-Up Status: Pending
Annual Follow-Up Agency Response From October 2020
The Department revised the GSM in October 2018 to include these recommendations and implemented these practices during the Department's first RFA solicitation process after Report 2017-129 was published. These changes are contained in the GSM Interim Guidance: October 2018.
- Completion Date: February 2020
California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented
Although Rehabilitation designated staff separate from those involved in the grant process to conduct a review of the process for its most recent grant, the evaluation notes that the team did not detect procedural errors or evaluator bias or prejudice. However, based on our review of available documentation, we found several discrepancies in the grant process, such as the RFA not including a tiebreaker methodology, staff involved in the grant process who may not have completed ethics training, and an evaluator with at least a perceived bias. As a result, we do not believe that Rehabilitation's post-evaluation review was comprehensive enough to demonstrate that Rehabilitation followed its grant process or to ensure sufficient oversight of the process.
- Auditee did not substantiate its claim of full implementation
- Auditee did not address all aspects of the recommendation
Annual Follow-Up Agency Response From October 2019
The Department has revised the GSM to include these recommendations and will carry out these practices in future grant solicitations.
- Estimated Completion Date: July 2020
California State Auditor's Assessment of Annual Follow-Up Status: Pending
1-Year Agency Response
The Department has revised the GSM to include these recommendations and will carry out these practices in future grant solicitations.
- Estimated Completion Date: August 2019
- Response Date: July 2019
California State Auditor's Assessment of 1-Year Status: Pending
6-Month Agency Response
The Department has revised the GSM to include these recommendations and will carry out these practices in future grant solicitations.
- Estimated Completion Date: GSM Interim Guidance - October 2018: Final GSM - After regulatory process has concluded
- Response Date: January 2019
California State Auditor's Assessment of 6-Month Status: Pending
60-Day Agency Response
The Department has revised the GSM to include these recommendations and will carry out these practices in future grant solicitations.
- Estimated Completion Date: October 2018 for Interim GSM; Final GSM after regulatory process has concluded
- Response Date: September 2018
California State Auditor's Assessment of 60-Day Status: Pending
Recommendation #17 To: Rehabilitation, Department of
To comply with federal and state requirements, and to ensure consistency and fairness in its grant process, Rehabilitation should revise and formalize the policies and procedures in its grant manual to incorporate the rules adopted by regulation and to address the recommendations in this report. The grant manual should specify that any deviations from the required grant process must be for good cause and be documented.
Annual Follow-Up Agency Response From September 2023
In 2019 and 2022, the Department withdrew its rulemaking packages after consultation with the Office of Administrative Law about the proposed sections related to the development of requests for applications. The Department expects to notice proposed regulations in October 2023 without the proposed sections related to RFA development and submit them to the Office of Administrative Law for approval by July 2024. Regarding the recommendation to promulgate regulations for development of requests for applications, by September 29, 2023, the Department will submit a request to the Principal Auditor for further discussion.
The final GSM will be published by December 2024.
- Estimated Completion Date: December 2024
California State Auditor's Assessment of Annual Follow-Up Status: Pending
Annual Follow-Up Agency Response From October 2022
The Department received written and public comment, including the public hearing on this rulemaking package and is not in the process of completing the rulemaking file to submit to the Office of Administrative Law in accordance with the Administrative Procedures Act. The Department expects approval of this rulemaking package in December 2022.
The final GSM will be published after the regulations are adopted. In the interim, the Department published an Interim GSM on October 29, 2018, for use until the regulatory process has concluded.
- Estimated Completion Date: December 2022
California State Auditor's Assessment of Annual Follow-Up Status: Pending
Annual Follow-Up Agency Response From October 2021
The Department revised its Grant Solicitation Manual (GSM) based on community input and the recommendations in the audit report. The Department went through the rulemaking process and in December 2019 discovered that it was necessary to revise the proposed regulations.
The Department has completed a draft of the proposed text for the grant regulations. Consistent with the Department's mission statement and core principles of partnership and transparency, the Department is engaging with its advisory bodies to obtain their input on the proposed concepts and approach before finalizing the proposed text and commencing the rulemaking process, including publication of the notice of proposed rulemaking and public comment period. The anticipated engagements include, but may not be limited to, the State Rehabilitation Council, Blind Advisory Council, State Independent Living Centers, and Assistive Technology Advisory Committee at their regularly scheduled public meetings.
The final GSM will be published after the regulations are adopted. In the interim, the Department published an Interim GSM on October 29, 2018 for use until the regulatory process has concluded.
- Estimated Completion Date: 12/2022
California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented
Annual Follow-Up Agency Response From October 2020
The Department revised its Grant Solicitation Manual (GSM) based on community input and the recommendations in the audit report. The Department went through the rulemaking process and in December 2019 discovered that it was necessary to revise the proposed regulations. The department expect to submit a new notice of rulemaking for publication in November 2020.
The final GSM will be published after the regulations are adopted. In the interim, the Department published an Interim GSM on October 29, 2018 for use until the regulatory process has concluded.
- Estimated Completion Date: March 2021
California State Auditor's Assessment of Annual Follow-Up Status: Pending
Annual Follow-Up Agency Response From July 2020
The Department revised its Grant Solicitation Manual (GSM) based on community input and the recommendations in the audit report. The Department held a public call with stakeholders on August 10, 2018 to solicit additional input regarding the revised GSM and offered the public the opportunity to submit written comments through various communication methods after the call.
The final GSM will be published after the regulations are adopted. In the interim, the Department published an Interim GSM on October 29, 2018 for use until the regulatory process has concluded.
- Estimated Completion Date: December 2020
California State Auditor's Assessment of Annual Follow-Up Status: Pending
Annual Follow-Up Agency Response From October 2019
The Department revised its Grant Solicitation Manual (GSM) based on community input and the recommendations in the audit report. The Department held a public call with stakeholders on August 10, 2018 to solicit additional input regarding the revised GSM and offered the public the opportunity to submit written comments through various communication methods after the call.
The final GSM will be published after the regulations are adopted. In the interim, the Department published an Interim GSM on October 29, 2018 for use until the regulatory process has concluded.
- Estimated Completion Date: July 2020
California State Auditor's Assessment of Annual Follow-Up Status: Pending
1-Year Agency Response
The Department revised its Grant Solicitation Manual (GSM) based on community input and the recommendations in the audit report. The Department held a public call with stakeholders on August 10, 2018 to solicit additional input regarding the revised GSM and offered the public the opportunity to submit written comments through various communication methods after the call.
The final GSM will be published after the regulations are adopted. In the interim, the Department published an Interim GSM on October 29, 2018 for use until the regulatory process has concluded.
- Estimated Completion Date: August 2019
- Response Date: July 2019
California State Auditor's Assessment of 1-Year Status: Pending
6-Month Agency Response
The Department revised its Grant Solicitation Manual (GSM) based on community input and the recommendations in the audit report. The Department held a public call with stakeholders on August 10, 2018 to solicit additional input regarding the revised GSM and offered the public the opportunity to submit written comments through various communication methods after the call.
The final GSM will be published after the regulations are adopted. In the interim, the Department published an Interim GSM on October 29, 2018 for use until the regulatory process has concluded.
- Estimated Completion Date: June 1, 2019
- Response Date: January 2019
California State Auditor's Assessment of 6-Month Status: Pending
60-Day Agency Response
The Department revised its Grant Solicitation Manual (GSM) based on community input and the recommendations in the audit report. The Department held a public call with stakeholders on August 10, 2018 to solicit additional input regarding the revised GSM and offered the public the opportunity to submit written comments through various communication methods after the call.
The final GSM will be published after the regulations are adopted. In the interim, the Department will publish an Interim GSM for use until the regulatory process has concluded.
- Estimated Completion Date: June 2019
- Response Date: September 2018
California State Auditor's Assessment of 60-Day Status: Pending
Recommendation #18 To: Rehabilitation, Department of
To ensure that it consistently and thoroughly evaluates appeals, Rehabilitation should establish in state regulations and its grant manual a process for the review committees to request additional information from appellants or program staff. To allow time for an adequate review of any additional information, Rehabilitation should consider extending the time for review committees to issue their decision on appeals from 30 days to 45 days.
Annual Follow-Up Agency Response From September 2023
The Department revised its Grant Solicitation Manual (GSM) based on community input and the recommendations in the audit report. The Department went through the rulemaking process and in December 2019 discovered that it was necessary to revise the proposed regulations.
In 2019 and 2022, the Department withdrew its rulemaking packages after consultation with the Office of Administrative Law about the proposed sections related to the development of requests for applications. The Department expects to notice proposed regulations in October 2023 without the proposed sections related to RFA development and submit them to the Office of Administrative Law for approval by July 2024. Regarding the recommendation to promulgate regulations for development of requests for applications, by September 29, 2023, the Department will submit a request to the Principal Auditor for further discussion.
The final GSM will be published by December 2024.
- Estimated Completion Date: December 2024
California State Auditor's Assessment of Annual Follow-Up Status: Pending
Annual Follow-Up Agency Response From October 2022
The Department revised its Grant Solicitation Manual (GSM) based on community input and the recommendations in the audit report. The Department went through the rulemaking process and in December 2019 discovered that it was necessary to revise the proposed regulations.
The Department received written and public comment, including the public hearing on this rulemaking package and is not in the process of completing the rulemaking file to submit to the Office of Administrative Law in accordance with the Administrative Procedures Act. The Department expects approval of this rulemaking package in December 2022.
The final GSM will be published after the regulations are adopted. In the interim, the Department published an Interim GSM on October 29, 2018 for use until the regulatory process has concluded.
- Estimated Completion Date: December 2022
California State Auditor's Assessment of Annual Follow-Up Status: Pending
Annual Follow-Up Agency Response From October 2021
The Department revised its Grant Solicitation Manual (GSM) based on community input and the recommendations in the audit report. The Department went through the rulemaking process and in December 2019 discovered that it was necessary to revise the proposed regulations.
The Department has completed a draft of the proposed text for the grant regulations. Consistent with the Department's mission statement and core principles of partnership and transparency, the Department is engaging with its advisory bodies to obtain their input on the proposed concepts and approach before finalizing the proposed text and commencing the rulemaking process, including publication of the notice of proposed rulemaking and public comment period. The anticipated engagements include, but may not be limited to, the State Rehabilitation Council, Blind Advisory Council, State Independent Living Centers, and Assistive Technology Advisory Committee at their regularly scheduled public meetings.
The final GSM will be published after the regulations are adopted. In the interim, the Department published an Interim GSM on October 29, 2018 for use until the regulatory process has concluded.
- Estimated Completion Date: 12/2022
California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented
Annual Follow-Up Agency Response From October 2020
The Department revised the GSM in October 2018 to include these recommendations and implemented these practices during the Department's first RFA solicitation process after Report 2017-129 was published. These changes are contained in the GSM Interim Guidance: October 2018.
- Completion Date: February 2020
California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented
Rehabilitation is in the process of promulgating regulations, and it has established a process in its grant manual for review committees to request additional information from the appellants or program staff. However, until it finalizes its regulations, we will continue to report this recommendation as partially implemented.
- Auditee did not address all aspects of the recommendation
Annual Follow-Up Agency Response From July 2020
The Chief Deputy Director, responsible for appointing the review committee, will implement these changes. In addition to incorporating this recommendation into regulations, the Department has revised the GSM to include these recommendations and will carry out these practices in future grant appeals.
- Estimated Completion Date: December 2020
California State Auditor's Assessment of Annual Follow-Up Status: Pending
Annual Follow-Up Agency Response From October 2019
The Chief Deputy Director, responsible for appointing the review committee, will implement these changes. In addition to incorporating this recommendation into regulations, the Department has revised the GSM to include these recommendations and will carry out these practices in future grant appeals.
- Estimated Completion Date: July 2020
California State Auditor's Assessment of Annual Follow-Up Status: Pending
1-Year Agency Response
The Chief Deputy Director, responsible for appointing the review committee, will implement these changes. In addition to incorporating this recommendation into regulations, the Department has revised the GSM to include these recommendations and will carry out these practices in future grant appeals.
- Estimated Completion Date: August 2019
- Response Date: July 2019
California State Auditor's Assessment of 1-Year Status: Pending
6-Month Agency Response
The Chief Deputy Director, responsible for appointing the review committee, will implement these changes. In addition to incorporation this recommendation into regulations, the Department has revised the GSM to include these recommendations and will carry out these practices in future grant appeals.
- Estimated Completion Date: GSM Interim Guidance - October 2018: Final GSM - After regulatory process has concluded: Regulations - June 1, 2019
- Response Date: January 2019
California State Auditor's Assessment of 6-Month Status: Pending
60-Day Agency Response
The Chief Deputy Director, responsible for appointing the review committee, will implement these changes. In addition to incorporating this recommendation into regulations, the Department has revised the GSM to include these recommendations and will carry out these practices in future grant appeals.
- Estimated Completion Date: October 2018 for Interim GSM; Final GSM after regulatory process has concluded; Regulations June 2019
- Response Date: September 2018
California State Auditor's Assessment of 60-Day Status: Pending
Recommendation #19 To: Rehabilitation, Department of
To ensure that it consistently and thoroughly evaluates appeals, Rehabilitation should establish in state regulations and its grant manual that to be able to rescore applications when necessary, the review committee members should be subject-matter experts or, if they are not subject-matter experts, the review committee should have the authority to recommend a new evaluation panel instead of rescoring applications itself when it identifies a reason to invalidate previous evaluations.
Annual Follow-Up Agency Response From September 2023
The Department revised its Grant Solicitation Manual (GSM) based on community input and the recommendations in the audit report. The Department went through the rulemaking process and in December 2019 discovered that it was necessary to revise the proposed regulations.
In 2019 and 2022, the Department withdrew its rulemaking packages after consultation with the Office of Administrative Law about the proposed sections related to the development of requests for applications. The Department expects to notice proposed regulations in October 2023 without the proposed sections related to RFA development and submit them to the Office of Administrative Law for approval by July 2024. Regarding the recommendation to promulgate regulations for development of requests for applications, by September 29, 2023, the Department will submit a request to the Principal Auditor for further discussion.
The final GSM will be published by December 2024.
- Estimated Completion Date: December 2024
California State Auditor's Assessment of Annual Follow-Up Status: Pending
Annual Follow-Up Agency Response From October 2022
The Department revised its Grant Solicitation Manual (GSM) based on community input and the recommendations in the audit report. The Department went through the rulemaking process and in December 2019 discovered that it was necessary to revise the proposed regulations.
The Department received written and public comment, including the public hearing on this rulemaking package and is not in the process of completing the rulemaking file to submit to the Office of Administrative Law in accordance with the Administrative Procedures Act. The Department expects approval of this rulemaking package in December 2022.
The final GSM will be published after the regulations are adopted. In the interim, the Department published an Interim GSM on October 29, 2018 for use until the regulatory process has concluded.
- Estimated Completion Date: December 2022
California State Auditor's Assessment of Annual Follow-Up Status: Pending
Annual Follow-Up Agency Response From October 2021
The Department revised its Grant Solicitation Manual (GSM) based on community input and the recommendations in the audit report. The Department went through the rulemaking process and in December 2019 discovered that it was necessary to revise the proposed regulations.
The Department has completed a draft of the proposed text for the grant regulations. Consistent with the Department's mission statement and core principles of partnership and transparency, the Department is engaging with its advisory bodies to obtain their input on the proposed concepts and approach before finalizing the proposed text and commencing the rulemaking process, including publication of the notice of proposed rulemaking and public comment period. The anticipated engagements include, but may not be limited to, the State Rehabilitation Council, Blind Advisory Council, State Independent Living Centers, and Assistive Technology Advisory Committee at their regularly scheduled public meetings.
The final GSM will be published after the regulations are adopted. In the interim, the Department published an Interim GSM on October 29, 2018 for use until the regulatory process has concluded.
- Estimated Completion Date: 12/2022
California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented
Annual Follow-Up Agency Response From October 2020
The Department revised the GSM in October 2018 to include these recommendations and implemented these practices during the Department's first RFA solicitation process after Report 2017-129 was published. These changes are contained in the GSM Interim Guidance: October 2018.
- Completion Date: February 2020
California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented
Rehabilitation is in the process of promulgating regulations, and it has amended its grant manual providing that the review committee members can rescore applications. It does not distinguish whether review committee members should be subject matter experts. The grant manual also provides the review committee the authority to recommend a new evaluation panel when it identifies a reason to invalidate the previous evaluations. For its most recent grant, despite one evaluator having at minimum a perceived bias, the review committee upheld the award decision, and did not exercise this authority to convene a new evaluator panel to rescore the applications. Until it addresses these shortcomings and finalizes its regulations, we will continue to report this recommendation as not fully implemented.
- Auditee did not address all aspects of the recommendation
Annual Follow-Up Agency Response From July 2020
The Chief Deputy Director, responsible for appointing the review committee, will implement these changes. In addition to incorporating this recommendation into regulations, the Department has revised the GSM to include these recommendations and will carry out these practices in future grant appeals.
- Estimated Completion Date: December 2020
California State Auditor's Assessment of Annual Follow-Up Status: Pending
Annual Follow-Up Agency Response From October 2019
The Chief Deputy Director, responsible for appointing the review committee, will implement these changes. In addition to incorporating this recommendation into regulations, the Department has revised the GSM to include these recommendations and will carry out these practices in future grant appeals.
- Estimated Completion Date: July 2020
California State Auditor's Assessment of Annual Follow-Up Status: Pending
1-Year Agency Response
The Chief Deputy Director, responsible for appointing the review committee, will implement these changes. In addition to incorporating this recommendation into regulations, the Department has revised the GSM to include these recommendations and will carry out these practices in future grant appeals.
- Estimated Completion Date: August 2019
- Response Date: July 2019
California State Auditor's Assessment of 1-Year Status: Pending
6-Month Agency Response
The Chief Deputy Director, responsible for appointing the review committee, will implement these changes. In addition to incorporation this recommendation into regulations, the Department has revised the GSM to include these recommendations and will carry out these practices in future grant appeals.
- Estimated Completion Date: GSM Interim Guidance - October 2018: Final GSM - After regulatory process has concluded: Regulations - June 1, 2019
- Response Date: January 2019
California State Auditor's Assessment of 6-Month Status: Pending
60-Day Agency Response
The Chief Deputy Director, responsible for appointing the review committee, will implement these changes. In addition to incorporating this recommendation into regulations, the Department has revised the GSM to include these recommendations and will carry out these practices in future grant appeals.
- Estimated Completion Date: October 2018 for Interim GSM; Final GSM after regulatory process has concluded; Regulations June 2019
- Response Date: September 2018
California State Auditor's Assessment of 60-Day Status: Pending
All Recommendations in 2017-129
Agency responses received are posted verbatim.