Report 2016-046 All Recommendation Responses

Report 2016-046: Board of Registered Nursing: Significant Delays and Inadequate Oversight of the Complaint Resolution Process Have Allowed Some Nurses Who May Pose a Risk to Patient Safety to Continue Practicing (Release Date: December 2016)

Recommendation for Legislative Action

To ensure that BRN receives timely and consistent notification of nurses' alleged violations of the Nursing Act, the Legislature should require the employers of registered nurses to report to BRN the suspension, termination, or resignation of any registered nurse due to alleged violations of the Nursing Act.

Description of Legislative Action

As of December 13, 2021, the Legislature has not taken any additional actions to fully implement this recommendation beyond those taken in Senate Bill 799 (Chapter 520, Statutes of 2017).

California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented

Description of Legislative Action

As of December 13, 2020, the Legislature has not taken any additional actions to fully implement this recommendation beyond those taken in Senate Bill 799 (Chapter 520, Statutes of 2017).

California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented

Description of Legislative Action

The Legislature has not taken any additional actions to fully implement this recommendation beyond those taken in Senate Bill 799 (Chapter 520, Statutes of 2017).

California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented

Description of Legislative Action

The Legislature has not taken any additional actions to fully implement this recommendation beyond those taken in Senate Bill 799 (Chapter 520, Statutes of 2017).

California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented

Description of Legislative Action

Senate Bill 799 (Hill, Chapter 520, Statutes of 2017) authorizes BRN to investigate at its discretion complaints against registered nurses participating in the intervention program, and prohibits disciplinary action with regard to acts committed before or during participation in the intervention program, unless the registered nurse withdraws or is terminated from the intervention program.

California State Auditor's Assessment of 1-Year Status: Partially Implemented

Recommendation for Legislative Action

If BRN does not develop and implement an action plan by March 1, 2017, to prioritize and resolve its deficiencies, as mentioned in the first recommendation to BRN, the Legislature should consider transferring BRN's enforcement responsibilities to Consumer Affairs.

Description of Legislative Action

As of December 13, 2021, the Legislature has not taken any action to address this specific recommendation.

California State Auditor's Assessment of Annual Follow-Up Status: No Action Taken

Description of Legislative Action

As of December 13, 2020, the Legislature has not taken action to address this specific recommendation.

California State Auditor's Assessment of Annual Follow-Up Status: No Action Taken

Description of Legislative Action

As of December 2019, the Legislature has not taken action to address this specific recommendation.

California State Auditor's Assessment of Annual Follow-Up Status: No Action Taken

Description of Legislative Action

The Legislature has not taken any action to address this specific recommendation.

California State Auditor's Assessment of Annual Follow-Up Status: No Action Taken

Description of Legislative Action

Legislation has not been introduced to address this recommendation.

California State Auditor's Assessment of 1-Year Status: No Action Taken

Recommendation for Legislative Action

The Legislature should amend state law to require BRN to conduct investigations of complaints alleging substance abuse or mental illness against nurses who choose to enter the intervention program.

Description of Legislative Action

As of December 13, 2021, the Legislature has not taken any action to address this specific recommendation.

California State Auditor's Assessment of Annual Follow-Up Status: No Action Taken

Description of Legislative Action

As of December 13, 2020, the Legislature has not taken action to address this specific recommendation.

California State Auditor's Assessment of Annual Follow-Up Status: No Action Taken

Description of Legislative Action

As of December 2019, the Legislature has not taken action to address this specific recommendation.

California State Auditor's Assessment of Annual Follow-Up Status: No Action Taken

Description of Legislative Action

The Legislature has not taken any action to address this specific recommendation.

California State Auditor's Assessment of Annual Follow-Up Status: No Action Taken

Description of Legislative Action

Legislation has not been implemented to address this recommendation. However, it should be noted that Senate Bill 799 (Hill, Chapter 520, Statutes of 2017) requires the California Research Bureau to prepare and deliver a report to the Legislature by January 1, 2019, that evaluates to what extent employers voluntarily report disciplined nurses to the board and that offers options for consistent and reasonable reporting mechanisms.

California State Auditor's Assessment of 1-Year Status: Pending

Recommendation #4 To: Registered Nursing, Board of

To ensure that it promptly addresses this report's findings, BRN should work with Consumer Affairs to develop an action plan by March 1, 2017, to prioritize and resolve the deficiencies we identified.

60-Day Agency Response

On or about December 12, 2016, the Board of Registered Nursing's (BRN) Executive management developed an action plan that documents and tracks each of the enforcement audit recommendations, BRN's initial response to each recommendation, timeline to implement recommendations, responsible parties, and outcome of implementation of recommendations. BRN staff collaborated with the Department of Consumer Affairs (DCA) executive management, DCA's SOLID Training Solutions, DCA's Division of Investigation (DOI) and the Attorney General's Office (AG) to develop solutions including policies and procedures and measurable outcomes to implement the recommendations. The action plan is reviewed and updated on a regular basis to ensure BRN meets the recommendation deadlines.

California State Auditor's Assessment of 60-Day Status: Fully Implemented

BRN, in collaboration with DCA, has created a plan that lists each of our 23 recommendations, including what it has done, or plans to do, to implement our recommendations. Although the plan does not specifically prioritize our recommendations, based on our follow-up with DCA and BRN officials, BRN is prioritizing its work on the recommendations by ensuring each are fully implemented by the completion dates included in our recommendations.

Recommendation #5 To: Registered Nursing, Board of

To ensure that BRN resolves complaints regarding nurses in a timely manner, by March 1, 2017, it should develop and implement formal policies that specify required time frames for each key stage of the complaint resolution process, including time frames for how quickly complaints should be assigned to the proper investigative unit or expert witness, and how long the investigation process should take. BRN should also work with DOI to establish a reasonable goal for the length of time DOI's investigators take to conduct investigations of complaints referred to it by BRN.

Annual Follow-Up Agency Response From November 2019

The BRN developed and fully implemented formal policies that specify required time frames for each key stage of the complaint resolution process, including time frames for how quickly complaints should be assigned to the proper investigative unit or expert witness, and how long the investigation process should take. BRN worked with DOI to establish a reasonable goal for the length of time DOI's investigators take to conduct investigations of complaints referred to it by BRN.

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

1-Year Agency Response

The BRN developed and implemented formal policies that specify required time frames for each key stage of the complaint resolution process, including time frames for how quickly complaints should be assigned to the proper investigative unit or expert witness, and how long the investigation process should take. BRN has worked with DOI to establish a reasonable goal for the length of time BRN and DOI's investigators take to conduct investigations of complaints referred to it by BRN.

California State Auditor's Assessment of 1-Year Status: Pending

Although BRN claims it has fully implemented this recommendation, it did not provide evidence demonstrating that it has. Until it does so, we will continue to assess the status of this recommendation as pending.

6-Month Agency Response

As of March 6, 2017, newly developed BreEZe reports are in full production. Staff is providing individual data reports to management on a monthly basis. Case processing timeframes and the completed case flow maps have been incorporated into the enforcement policy and procedure manuals.

In a meeting on February 10, 2017, with the Division of Investigation (DOI) management and the Board of Registered Nursing (BRN) Intake and Investigation Chief, an agreement was made to establish the Investigation timeframe goal for both DOI and BRN at 240 days. This agreement was confirmed in writing with all parties and documented in the Investigation Unit Procedure Manual.

California State Auditor's Assessment of 6-Month Status: Pending

BRN did not provide us with documentation to support its claim that it has fully implemented this recommendation.

60-Day Agency Response

Through collaborative efforts with DCA's BreEZe Business Integrity Analysts (BIAs), Executive management, SOLID staff, and BRN enforcement managers and staff, the BRN has developed the following business processes:

- A detailed workflow mapping of complaint intake and investigation business processes,

- Updated policies and procedures to direct staff action to ensure the assignment of cases to appropriate investigation unit or expert witnesses,

- Ongoing (bi-weekly) communication with BRN staff regarding process improvement and outcomes,

- Updated policies and procedures to include timeframe expectations for each stage of complaint and investigation resolution process,

- A robust set of milestone reports were developed and implemented to track each key stage of the complaint resolution and investigation processes to provide management with accurate data regarding case processing timeframes,

- An average of 240 days was established as a reasonable goal for DOI and BRN to complete the investigation process. BRN and DOI will continue to work collaboratively to examine ways to improve efficiencies.

California State Auditor's Assessment of 60-Day Status: Partially Implemented

Although BRN developed formal policies that specify time frames for some key stages of the complaint resolution process, it did not specify time frames for how quickly complaints should be assigned to the proper investigative unit. Further, BRN could not provide evidence demonstrating that these policies have been approved by, for example, the board or the executive officer, and have been distributed to staff.

Recommendation #6 To: Registered Nursing, Board of

To ensure that BRN resolves complaints regarding nurses in a timely manner, by March 1, 2017, it should establish a formal, routine process for management to monitor each key stage of the complaint resolution process to determine whether the time frames are being met, the reasons for any delays, and any areas in the process that it can improve.

Annual Follow-Up Agency Response From October 2018

The BRN has established a formal, routine process for management to monitor each key stage of the complaint resolution process to determine whether the time frames are being met, the reasons for any delays, and any areas in the process that it can improve to ensure the BRN resolves complaints regarding nurses in a timely manner.

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

The BRN established a formal, routine process for management to monitor each key stage of the complaint resolution process. Specifically, it developed a number of reports that show how long staff are taking to process complaints. In addition, the process that it developed directs managers to use these reports to routinely monitor each key stage of the complaint resolution process to assess whether staff are meeting set time frames, the reasons for delays, and any areas for improvement.

1-Year Agency Response

The BRN has established a formal, routine process for management to monitor each key stage of the complaint resolution process to determine whether the time frames are being met, the reasons for any delays, and any areas in the process that it can improve to ensure the BRN resolves complaints regarding nurses in a timely manner.

California State Auditor's Assessment of 1-Year Status: Pending

Although BRN claims it has fully implemented this recommendation, it did not provide evidence demonstrating that it has. Until it does so, we will continue to assess the status of this recommendation as pending.

6-Month Agency Response

Timeframe/Milestone reports are completed and in full production mode as of March 1, 2017. Reports are delivered on a monthly basis to all enforcement unit managers.

As of April 14, 2017, instructions for managers to use the Timeframe/Milestone reports when evaluating staff workload are included in the Management Monitoring Plans.

In reviewing the reports recommended by the audit the BRN designed additional reports to track Decisions & Appeals, as well as Citation and Fine workloads. These new reports are in final testing stages with the Department of Consumer Affair's (DCA) Office of Information Services. Scheduled implementation of these new reports is anticipated by July 1.

California State Auditor's Assessment of 6-Month Status: Pending

BRN did not provide documentation to substantiate its claim of full implementation.

60-Day Agency Response

DCA BreEZe BIAs and BRN Subject Matter Experts (SME) worked collectively to develop a series of data reports (i.e. 50 sets) for use in all enforcement units. These reports will allow Managers to identify areas of delays, track team and individual performance, and address process improvements with supervisors and staff.

All reports are in production as of February 1st which will be used by managers to monitor each key stage of the complaint resolution process to ensure time frames are being met, identify reasons for any delays, and potential process improvements.

Each unit has developed a management monitoring plan instructing supervisors in report utilization, timeframe monitoring, and ongoing audits of staff case workload.

California State Auditor's Assessment of 60-Day Status: Partially Implemented

Although BRN developed a number of reports that if used would show how long staff are taking to process complaints, it did not develop policies and procedures that specifically direct managers to, or detail how they should, use these reports to routinely monitor each key stage of the complaint resolution process to assess whether staff are meeting set time frames, the reasons for delays, and any areas for improvement. Until these shortcomings are addressed, we will report this recommendation as partially implemented.

Recommendation #7 To: Registered Nursing, Board of

To ensure that BRN resolves complaints regarding nurses in a timely manner, by March 1, 2017, it should establish a plan to eliminate its backlog of complaints awaiting assignment to an investigator.

Annual Follow-Up Agency Response From October 2023

The BRN will continue to implement this recommendation. As stated in response to Recommendation 3 for Investigative Report 12020-0027 (associated to Recommendation 7 for 2016-046), the BRN "has demonstrated a good-faith effort to work with the audit team to develop a satisfactory approach for fully implementing the 2016 audit recommendation, we deem this recommendation fully implemented." The BRN has not received any additional request for information or documentation from CSA but will cooperate with any such requests to demonstrate full compliance.

This recommendation was fully implemented December 2020. The drop down above for the First Month of Full Implementation will not allow the year 2020; therefore, January 2021 was selected.

California State Auditor's Assessment of Annual Follow-Up Status: Pending

In October 2018, BRN asserted that it had fully implemented this recommendation and provided a report that it represented would support this claim, which caused us, in reliance on those representations, to conclude that this recommendation was fully implemented. However, as discussed at length in report number I2020-0027, we later received a whistleblower complaint alleging that BRN had manipulated data and delivered a falsified report with the intention of misleading the California State Auditor's Office. Our office substantiated that complaint and the full report can be found at http://auditor.ca.gov/pdfs/reports/I2020-0027.pdf. As a result, we have no current assurances that the recommendation has been accurately addressed and, in September 2020, we revised our assessment of the status of this recommendation to pending.

Although BRN has once again asserted that it had fully implemented this recommendation and has made a good-faith effort to work with the audit team to develop a satisfactory approach for implementing this recommendation, we have no assurance that it has actually reduced its investigator caseload and fully implemented this recommendation. Thus, because it was able to falsify data in one of the reports it provided to our office previously, we would need to perform additional work to verify whether it has actually reduced investigator caseloads and fully implemented our recommendation. Therefore, at this time, we are assessing this recommendation status as pending until we are able to perform the additional analysis needed to verify whether the investigator caseloads have actually been reduced.

Annual Follow-Up Agency Response From October 2022

The Board of Registered Nursing (BRN) will continue to implement this recommendation. As stated in response to Recommendation 3 for Investigative Report 12020-0027 (associated to Recommendation 7 for 2016-04), the BRN "has demonstrated a good-faith effort to work with the audit team to develop a satisfactory approach for fully implementing the 2016 audit recommendation, we deem this recommendation fully implemented." The BRN has not received any additional request for information or documentation from CSA but will cooperate with any such requests to demonstrate full compliance.

California State Auditor's Assessment of Annual Follow-Up Status: Pending

In October 2018, BRN asserted that it had fully implemented this recommendation and provided a report that it represented would support this claim, which caused us, in reliance on those representations, to conclude that this recommendation was fully implemented. However, as discussed at length in report number I2020-0027, we later received a whistleblower complaint alleging that BRN had manipulated data and delivered a falsified report with the intention of misleading the California State Auditor's Office. Our office substantiated that complaint and the full report can be found at http://auditor.ca.gov/pdfs/reports/I2020-0027.pdf. As a result, we have no current assurances that the recommendation has been accurately addressed and, in September 2020, we revised our assessment of the status of this recommendation to pending.

Although BRN has once again asserted that it had fully implemented this recommendation and provided several reports similar to the one it provided in 2018 to support its claim, we have no assurance that it has actually reduced its investigator caseload and fully implemented this recommendation. Specifically, in November 2022, BRN provided several reports indicating that its investigators only have 30 or fewer cases assigned to them. However, because it was able to falsify data in one of these reports previously, we would need to perform additional work to determine whether it manipulated the data in these reports to verify whether it has actually reduced investigator caseloads and fully implemented our recommendation. Therefore, at this time, we are assessing this recommendation status as pending until we are able to perform the additional analysis needed to verify whether the investigator caseloads it reported are accurate.

Annual Follow-Up Agency Response From October 2021

The BRN will continue to implement this recommendation. As stated in response to Recommendation 3 for Investigative Report 12020-0027 (associated to Recommendation 7 for 2016-04), the BRN "has demonstrated a good-faith effort to work with the audit team to develop a satisfactory approach for fully implementing the 2016 audit recommendation, we deem this recommendation fully implemented." The BRN has not received any additional request for information or documentation from CSA but will cooperate with any such requests to demonstrate full implementation.

California State Auditor's Assessment of Annual Follow-Up Status: Pending

In October 2018, BRN asserted that it had fully implemented this recommendation and provided a report that it represented would support this claim, causing us, in reliance on those representations, to conclude that this recommendation was fully implemented. However, as discussed at length in report number I2020-0027, we later received a whistleblower complaint alleging that BRN had manipulated data and delivered a falsified report with the intention of misleading the California State Auditor's Office. Our office substantiated that complaint and the full report can be found at http://auditor.ca.gov/pdfs/reports/I2020-0027.pdf. As a result, we have no current assurances that the recommendation has been accurately addressed and, in September 2020, we revised our assessment of the status of this recommendation to pending.

Although, in October 2021, BRN once again asserted that it had fully implemented this recommendation and provided a report similar to the one it provided in 2018 to support its claim, we have no assurance that it has actually reduced its investigator caseload and fully implemented this recommendation. Specifically, in December 2020, BRN provided a report indicating that its investigators only have 30 or fewer cases assigned to them. However, because it was able to falsify data in this report previously, we would need to perform additional work to determine whether it manipulated the data in this report to verify whether it has actually reduced investigator caseloads and fully implemented our recommendation. Therefore, at this time, we are assessing this recommendation status as pending until we are able to perform the additional analysis needed to verify whether the investigator caseloads it reported are accurate.

Annual Follow-Up Agency Response From December 2020

The Board of Registered Nursing (BRN) established a plan to eliminate its backlog of complaints awaiting assignments to an investigator by memorializing the expectation that the number of investigations assigned to a Special Investigator does not exceed 30 investigations. Since the BRN memorialized this expectation, management has monitored and ensured that investigators are not assigned more than 30 investigations.

California State Auditor's Assessment of Annual Follow-Up Status: Pending

In October 2018, BRN asserted that it had fully implemented this recommendation and provided a report that it represented would support this claim, causing us, in reliance on those representations, to conclude that this recommendation was fully implemented. However, as discussed at length in report number I2020-0027, we later received a whistleblower complaint alleging that BRN had manipulated data and delivered a falsified report with the intention of misleading the California State Auditor's Office. Our office substantiated that complaint and the full report can be found at http://auditor.ca.gov/pdfs/reports/I2020-0027.pdf. As a result, we have no current assurances that the recommendation has been accurately addressed and, in September 2020, we revised our assessment of the status of this recommendation to pending.

Although BRN has once again asserted that it had fully implemented this recommendation and provided a report similar to the one it provided in 2018 to support its claim, we have no assurance that it has actually reduced its investigator caseload and fully implemented this recommendation. Specifically, in December 2020, BRN provided a report indicating that its investigators only have 30 or fewer cases assigned to them. However, because it was able to falsify data in this report previously, we would need to perform additional work to determine whether it manipulated the data in this report to verify whether it has actually reduced investigator caseloads and fully implemented our recommendation. Therefore, at this time, we are assessing this recommendation status as pending until we are able to perform the additional analysis needed to verify whether the investigator caseloads it reported are accurate.

Annual Follow-Up Agency Response From October 2018

The BRN has established a plan to eliminate its backlog of complaints awaiting assignment to an investigator.

California State Auditor's Assessment of Annual Follow-Up Status: Pending

In October 2018, BRN reported that it had fully implemented this recommendation and provided "evidence" to support this claim, causing us, in reliance on those representations, to conclude that this recommendation was fully implemented. However, as discussed at length in report number I2020-0027, we later received a whistleblower complaint alleging that BRN had manipulated data and delivered a falsified report with the intention of misleading the California State Auditor's Office. Our office substantiated that complaint and the full report can be found at http://auditor.ca.gov/pdfs/reports/I2020-0027.pdf. As a result, we have no current assurances that the recommendation has been accurately addressed and, in September 2020, we revised our assessment of the status of this recommendation to pending.

1-Year Agency Response

The BRN has established a plan to eliminate its backlog of complaints awaiting assignment to an investigator.

California State Auditor's Assessment of 1-Year Status: Pending

Although BRN claims it has fully implemented this recommendation, it did not provide evidence demonstrating that it has. Until it does so, we will continue to assess the status of this recommendation as pending.

6-Month Agency Response

The BRN eliminated the unassigned case backlog prior to the 60-day response and continues to have no backlog of unassigned cases. 60-day feedback from auditor outlines concerns regarding Investigator workload. BRN believes additional staff will be necessary to address the level of workload assigned to investigators and is in the process of reviewing workload and staffing resources to determine what is needed.

California State Auditor's Assessment of 6-Month Status: Pending

BRN did not provide documentation to substantiate its claim of full implementation.

60-Day Agency Response

As of January 17, 2017, the BRN has eliminated any unassigned backlog in the BRN Investigation Unit.

Immediately following release of the audit report, case assignments were increased for all investigators from 20 to 25 each which aided in the elimination of the backlog. The BRN developed procedures requiring a case received by the investigations unit to be assigned to a Supervising Special Investigator (SSI) within 10 business days. The SSI is responsible for reviewing and triaging the incoming case workload of the unit. Cases will remain assigned to the SSI until the field investigator's workload allows the case to be assigned to the field investigator. This process has been updated and included in the policies and procedures for investigations. In addition, the BRN began strictly adhering to the Division of Investigation's CPEI case referral guidelines which has also contributed to elimination of the backlog.

California State Auditor's Assessment of 60-Day Status: Partially Implemented

BRN indicated that it eliminated its backlog of complaints awaiting assignment to an investigator by assigning them to its investigators. Based on documentation provided by BRN, this increased investigators' caseload, on average, to roughly 24. However, as we describe in our audit report on pages 28 to 29, the chief of investigations acknowledged that a full caseload for BRN's non-sworn investigators is 20 complaints. Therefore, rather than BRN eliminating its backlog, it simply shifted the backlogged complaints by assigning them to its non-sworn investigators who already have full caseloads. Until it can demonstrate that its non-sworn investigators are able to timely resolve complaints, given the increased caseload, we will report this as partially implemented.

Recommendation #8 To: Registered Nursing, Board of

To increase its pool of expert witnesses, by June 2017, BRN should develop and implement a process to track the effectiveness of the methods it uses to recruit expert witnesses, and then focus its efforts on those methods that prove to be the most successful.

Annual Follow-Up Agency Response From November 2019

The BRN developed and fully implemented a process to track the effectiveness of the methods it uses to recruit expert witnesses and is focusing its efforts on using the most successful methods.

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

Annual Follow-Up Agency Response From October 2018

The BRN has developed and implemented a process to track the effectiveness of the methods it uses to recruit expert witnesses, and is focusing its efforts on using the most successful methods.

California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented

The BRN has developed and implemented a process to track the effectiveness of the methods it uses to recruit expert witnesses. However, it has not provided evidence to demonstrate that it is focusing its recruitment efforts on the most successful methods. Until it does so, we will continue to mark this recommendation as partially implemented.

1-Year Agency Response

The BRN has developed and implemented a process to track the effectiveness of the methods it uses to recruit expert witnesses, and is focusing its efforts on using the most successful methods.

California State Auditor's Assessment of 1-Year Status: Pending

Although BRN claims it has fully implemented this recommendation, it did not provide evidence demonstrating that it has. Until it does so, we will continue to assess the status of this recommendation as pending.

6-Month Agency Response

The Expert Witness program has been re-branded and is now called the Expert Practice Consultant (EPC) program. The application has been updated and published to the BRN website.

A recruitment flyer for EPC is now included with all BRN paper renewal notices in the mail. This has resulted in a substantial increase of 61.5% EPC applications received. The success in the number of EPC applications is due to collaboration with DCA's Public Information Office (PIO) and BRN outreach efforts to nursing program Deans and Directors, the California Hospital Association, professional nursing organizations and use of social media.

The internal EPC database has been updated to capture recruiting data and more accurately capture hours and cost. Recruitment data will be evaluated annually or as needed to determine the most effective recruitment technique.

California State Auditor's Assessment of 6-Month Status: Pending

BRN did not provide documentation to substantiate its claim of full implementation.

60-Day Agency Response

In an effort to increase its expert witness pool, the BRN has implemented the following:

- Updated Expert Witness application and uploaded the document on the BRN Website.

- Contacted DCA Public Information Office (PIO) to develop a detailed marketing plan to recruit additional Expert Witnesses.

- PIO will develop an initial proposal, and present marketing materials for review to the BRN by March 8, 2017.

- Final production date for Expert Witness Marketing Plan is March 29, 2017.

- BRN Expert Witness Database was updated to capture recruiting data and track effectiveness of recruitment methods.

California State Auditor's Assessment of 60-Day Status: Pending

BRN did not provide any documentation demonstrating that the actions they have claimed to take thus far, were indeed taken. As a result, we are assessing the status of this recommendation as pending, given BRN indicates it will fully implement the recommendation.

Recommendation #9 To: Registered Nursing, Board of

To increase its pool of expert witnesses, by June 2017, BRN should modify its renewal application process for nurses' licenses to include a question regarding whether they would be interested in serving as an expert witness, and then develop a process to promptly follow-up with those nurses.

Annual Follow-Up Agency Response From October 2018

The BRN has modified the RN license renewal application to include a question regarding whether a licensee is interested in serving as an expert witness and developed a process to promptly follow up with those nurses.

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

BRN modified the RN license renewal application to include a question regarding whether a licensee is interested in serving as an expert witness, developed a tool to track those nurses who indicate they are interested, and developed a process to follow-up with those nurses who expressed interest on a monthly basis.

1-Year Agency Response

The BRN has modified the RN license renewal application to include a question regarding whether a licensee is interested in serving as an expert witness and developed a process to promptly follow-up with those nurses.

California State Auditor's Assessment of 1-Year Status: Pending

Although BRN claims it has fully implemented this recommendation, it did not provide evidence demonstrating that it has. Until it does so, we will continue to assess the status of this recommendation as pending.

6-Month Agency Response

A paper renewal mail insert has been developed and is included in all mail renewal notices beginning February 2017.

On March 6, 2017, a request was made to BRN's BreEZe Subject Matter Experts (SME) to include an additional online renewal question regarding the RNs interest in becoming an Expert Practice Consultant. An information technology ticket was initiated and changes to the system will be made according to the BreEZe schedule.

California State Auditor's Assessment of 6-Month Status: Pending

60-Day Agency Response

A marketing insert was developed for the renewal application and is scheduled for distribution to all renewal licensees beginning with the February 2017 mailing.

In addition, BRN staff is working with the BIAs to develop recruitment efforts using the online renewal application via the BreEZe.

California State Auditor's Assessment of 60-Day Status: Pending

Recommendation #10 To: Registered Nursing, Board of

To increase its pool of expert witnesses, by June 2017, BRN should take the steps necessary to increase the hourly wage it pays expert witnesses.

Annual Follow-Up Agency Response From October 2023

The BRN continues to be successful in recruiting Expert Practice Consultants (EPCs) and does not need to seek an increase in the hourly wage it pays expert witnesses at this time. In the future, should the expert pool not be sufficient to meet the needs of the Board then the BRN will take the necessary steps to increase the hourly wage of the EPCs.

California State Auditor's Assessment of Annual Follow-Up Status: Will Not Implement

Annual Follow-Up Agency Response From October 2022

The Board of Registered Nursing (BRN) continues to be successful in recruiting Expert Practice Consultants (EPC) and does not need to seek an increase in the hourly wage it pays expert witnesses at this time. In the future, should the expert pool not be sufficient to meet the needs of the Board then the BRN will take the necessary steps to increase the hourly wage of the EPCs.

California State Auditor's Assessment of Annual Follow-Up Status: Will Not Implement

Annual Follow-Up Agency Response From November 2019

The BRN has been extremely successful in recruiting Expert Practice Consultants in all areas and does not need to seek an increase in the hourly wage it pays expert witnesses at this time. In the future, should the expert pool not be sufficient to meet the needs of the Board then the BRN will take the necessary steps to increase the hourly wage of the EPCs.

California State Auditor's Assessment of Annual Follow-Up Status: Will Not Implement

Annual Follow-Up Agency Response From October 2018

The BRN has been extremely successful in recruiting expert witnesses in all areas and does not need to seek an increase in the hourly wage it pays expert witnesses at this time. In the future, should the expert pool not be sufficient to meet the needs of the Board then the BRN will take the necessary steps to increase the hourly wage of the expert witnesses.

California State Auditor's Assessment of Annual Follow-Up Status: Will Not Implement

1-Year Agency Response

The BRN has contacted three other healing arts boards, Board of Vocational Nursing and Psychiatric Technicians, Medical Board of California and Board of Psychology, to determine an average Expert Practice Consultant wage to be paid per hour.

On January 6, 2017, BRN completed a report of expert hours by profession in order to project the overall impact to the BRN budget.

The BRN would like to increase the expert fee to $100 for RNs, $125 for Advanced Practice Registered Nurses (APRN), $150 for psychologists and $200 for Psychiatrists, effective July 1, 2018.

The Board is actively working to identify resources to allow it to implement these increased costs.

The BRN will gather additional data to support the expert practice consultant wage increase.

California State Auditor's Assessment of 1-Year Status: Pending

6-Month Agency Response

The BRN has contacted three other healing arts boards, Board of Vocational Nursing and Psychiatric Technicians, Medical Board of California and Board of Psychology, to determine an average Expert Practice Consultant wage to be paid per hour.

On January 6, 2017, BRN completed a report of expert hours by profession in order to project the overall impact to the BRN budget.

The BRN would like to increase the expert fee to $100 for RNs, $125 for Advanced Practice Registered Nurses (APRN), $150 for psychologists and $200 for Psychiatrists, effective July 1, 2018. The Board is actively working to identify resources to allow it to implement these increased costs.

California State Auditor's Assessment of 6-Month Status: Pending

60-Day Agency Response

BRN has benchmarked the hourly rate for expert witnesses of three other healing arts boards (e.g. BVNPT, MBC and Psychology) to determine an average hourly wage.

BRN staff proposes raising the rate for all expert witnesses to include the following: $75 to $100 per hour for RNs, $125 to $150 for psychologists, and $175 to $200 per hour for psychiatrists. A proposal will be submitted in or about April 2017 in accordance with the budget cycle.

California State Auditor's Assessment of 60-Day Status: Pending

Recommendation #11 To: Registered Nursing, Board of

To ensure it does not risk compromising private and confidential information related to ongoing investigations of complaints, BRN should immediately ensure that any email correspondence it has with expert witnesses is transmitted securely.

Annual Follow-Up Agency Response From November 2019

The BRN developed procedures on how to send the investigative report and case materials to the expert witness, which includes procedures that specify that staff must ensure that any correspondence or documents it sends to expert witnesses is transmitted securely, or that sensitive information is redacted.

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

Annual Follow-Up Agency Response From October 2018

The BRN ensures any email correspondence it has with expert witnesses is transmitted securely to ensure it does not risk compromising private and confidential information related to ongoing investigations of complaints.

California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented

Although BRN developed procedures for instructing staff on how to send the investigative report and case materials to the expert witness, the procedures do not specify that staff should ensure that any correspondence or documents it sends to expert witnesses is transmitted securely, or that sensitive information is redacted. Thus, until it establishes this as part of its official policy, we will report this as partially implemented.

1-Year Agency Response

The BRN ensures any email correspondence it has with expert witnesses is transmitted securely to ensure it does not risk compromising private and confidential information related to ongoing investigations of complaints.

California State Auditor's Assessment of 1-Year Status: Pending

Although BRN claims it has fully implemented this recommendation, it did not provide evidence demonstrating that it has. Until it does so, we will continue to assess the status of this recommendation as pending.

6-Month Agency Response

As of November 21, 2016, the BRN began omitting any subject RN's name from e-mails sent to the experts, as was specified in the auditor's recommendation.

Instruction was included in the Complaint Intake policies and procedures manual prohibiting the transfer of personally identifiable information via e-mail.

BRN will pursue an electronic transfer of materials via the secure DCA Cloud to further this recommendation. The BRN is also exploring best practices with other healing arts boards and the Attorney General's Office (AGO) regarding this process.

California State Auditor's Assessment of 6-Month Status: Pending

BRN did not provide documentation to substantiate its claim of full implementation.

60-Day Agency Response

To ensure e-mail correspondence with expert witnesses are transmitted in a secure environment, personally identifiable information related to investigation of complaints is no longer be transmitted in e-mails. Hence, case review material is being sent via the U.S. Postal Mailing system. This process has been updated in the Complaint Intake Policies and Procedures manual.

California State Auditor's Assessment of 60-Day Status: Partially Implemented

Although BRN developed a policy for staff to send by traditional mail the investigative report and case materials to the expert witness rather than email, the policy does not specify that staff should ensure that any correspondence or documents it sends to expert witnesses is transmitted securely, or that sensitive information is redacted. Thus, until it establishes this as part of its official policy, we will report this as partially implemented.

Recommendation #12 To: Registered Nursing, Board of

To ensure that it is able to accurately monitor the performance of its complaint resolution process and that it has accurate data to address its staffing needs, BRN should immediately begin working with Consumer Affairs to implement cost-effective input controls for BreEZe that will require BRN staff members to enter information into a complaint record in a way that is consistent with BRN's business processes, as well as to implement changes that would cause BreEZe to accurately identify the order in which activities occur.

Annual Follow-Up Agency Response From November 2019

To ensure the BRN is able to accurately monitor the performance of its complaint resolution process and has accurate data to address its staffing needs, the BRN worked with Consumer Affairs to implement cost-effective input controls for BreEZe which requires BRN staff members to enter information into a complaint record which is consistent with BRN's business processes and the BRN implemented changes that would cause BreEZe to accurately identify the order in which activities occur.

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

Annual Follow-Up Agency Response From October 2018

BRN's BreEZe subject matter experts worked with the DCA's business integrity analysts to develop activity code functionality to allow stop gap controls in the BreEZe system which will accurately identify the order in which activities occur. Prioritization of BreEZe activities is a global issue that will impact all DCA boards and bureaus that utilize the BreEZe system.

A System Investigation Request (SIR) was submitted by BRN on December 16, 2016 to DCA which was approved to go to the BreEZe vendor for scope-of-work review and approval. The BreEZe vendor returned the SIR with the scope-of-work approved and changes were then submitted as a work authorization to the BreEZe project Change Control Board to approve. The Change Control Board, comprised predominantly of leadership from Boards and Bureaus, approved the Work Authorization to be forwarded for Impact Analysis. The Impact Analysis was completed. The BRN continues to work towards updating the business process and evaluate the cost-effectiveness of implementing this practice.

The implementation of this type of change typically takes at least one year to complete. The enforcement user group approved the final system design for the work authorization in November 2017. The BRN subject matter expert completed the process to approve the final system design in May 2018. Based on this SIR's priority and level of effort, DCA anticipates implementing this change in Spring 2019.

California State Auditor's Assessment of Annual Follow-Up Status: Pending

1-Year Agency Response

BRN Breeze Subject Matter Experts (SMEs) worked with the DCAs Business Integrity Analysts (BIA) to develop activity code functionality to allow stop gap controls in the BreEZe system which will accurately identify the order activities occur. Prioritization of BreEZe activities is a global issue that will impact all DCA boards and bureaus that utilize the BreEZe system. Implementing stop gap controls requires approval from the following workgroups: DCA's Enforcement User Group and Reports User Group, BreEZe Vendor, and Change Control Board.

A System Investigation Request (SIR) was submitted by BRN SMEs on December 16, 2016, to DCA which was approved to go to the BreEZe Vendor for scope-of-work review and approval. Once the BreEZe Vendor returned the SIR with the scope-of-work approved, changes were then submitted as a work authorization to DCA's Change Control Board to approve the BreEZe stop gap controls. The Change Control Board approved the Work Authorization to be forwarded for Impact Analysis. This Impact Analysis is currently pending. The BRN will continue to work with these groups to update the business process and evaluate the cost-effectiveness of implementing this practice.

Per DCA, the implementation of this type of change typically takes at least one year to complete. The enforcement user group has approved the final system design for the work authorization as of November 2017.

Target completion date is September 2018.

California State Auditor's Assessment of 1-Year Status: Pending

6-Month Agency Response

BRN SMEs worked with the DCAs Business Integrity Analysts (BIA) to develop activity code functionality to allow stop gap controls in the BreEZe system which will accurately identify the order activities occur. Prioritization of BreEZe activities is a global issue that will impact all DCA boards and bureaus that utilize the BreEZe system. Implementing stop gap controls requires approval from the following workgroups: DCA's Enforcement User Group and Reports User Group, BreEZe Vendor, and Change Control Board.

A System Investigation Request (SIR) was submitted by BRN SMEs on December 16, 2016, to DCA which was approved to go to the BreEZe Vendor for scope-of-work review and approval. Once the BreEZe Vendor returned the SIR with the scope-of-work approved, changes were then submitted as a work authorization to DCA's Change Control Board to approve the BreEZe stop gap controls. The Change Control Board approved the Work Authorization to be forwarded for Impact Analysis. This Impact Analysis is currently pending. The BRN will continue to work with these groups to update the business process and evaluate the cost-effectiveness of implementing this practice.

Per DCA, the implementation of this type of change typically takes about one year to complete.

California State Auditor's Assessment of 6-Month Status: Pending

60-Day Agency Response

BRN SMEs have worked with the BIAs to develop activity code functionality to allow stop gap controls in the BreEZe system which will accurately identify the order activities occur. Prioritization of BreEZe activities is a global issue that will impact all DCA boards and bureaus that utilize the BreEZe system. Implementing stop gap controls will require approval from the following workgroups: DCA's Enforcement User Group and Reports User Group, BreEze Vendor, and Change Control Board. A System Investigation Request (SIR) was submitted by BRN SMEs on December 16, 2016, to DCA which was approved to go to the BreEZe Vendor for scope-of-work review and approval. Once The BreEZe Vendor returned the SIR with the scope-of-work approved, changes were then submitted as a work authorization to DCA's Change Control Board to approve the BreEZe stop gap controls. The Change Control Board approved the Work Authorization to be forwarded for Impact Analysis. This Impact Analysis is currently pending. The BRN will continue to work with these groups to update the business process and evaluate the cost-effectiveness of implementing this practice. Per DCA, the implementation of the process typically takes about one year to complete.

California State Auditor's Assessment of 60-Day Status: Pending

Recommendation #13 To: Registered Nursing, Board of

To ensure that it is able to accurately monitor the performance of its complaint resolution process and that it has accurate data to address its staffing needs, once it has implemented cost-effective input controls for BreEZe and accumulated six months of data, BRN should analyze these data to determine whether its staffing is sufficient to meet its workload.

Annual Follow-Up Agency Response From November 2019

To ensure that the BRN accurately monitors the performance of its complaint resolution process and has accurate data to address its staffing needs, the BRN analyzed six months of accumulated data after implementing the cost-effective input controls for BreEZe (Recommendation 12) to determine whether its staffing is sufficient to meet its workload.

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

Annual Follow-Up Agency Response From October 2018

Data reports were released to managers for review in March 2017. The six-month data accumulation mark ended in September 2017. Reports have been reviewed for the six-month period. BRN continues to monitor these reports monthly. Data for FY 16-17 has been collected and is being analyzed to determine staffing needs.

California State Auditor's Assessment of Annual Follow-Up Status: Pending

1-Year Agency Response

Data reports were released to managers for review in March 2017. The 6-month data accumulation mark ended in September 2017. Reports have been reviewed for the six month period. BRN continues to monitor these reports monthly. Data for FY 16-17 has been collected and analyzed to determine staffing needs to complete its workload within the time frame goals. The BRN will seek staff resources through the Budget Change Proposal (BCP) process to address additional staffing needs until time frame goals are met.

California State Auditor's Assessment of 1-Year Status: Pending

6-Month Agency Response

Data reports were released to managers for review in March 2017. The 6-month data accumulation mark will be in September 2017. Reports have been reviewed for the previous six months. BRN will continue to monitor these reports monthly until we have the upcoming six months' worth of data. Analysis of the data will be used to determine whether additional staffing is necessary to meet its workload.

California State Auditor's Assessment of 6-Month Status: Pending

60-Day Agency Response

If input controls are approved by the Enforcement User Group and implemented in BreEZe, the BRN will accumulate and analyze six months of data to determine if staffing needs are sufficient to meet its current workload.

California State Auditor's Assessment of 60-Day Status: Pending

Recommendation #14 To: Registered Nursing, Board of

To ensure that it is able to accurately monitor the performance of its complaint resolution process and that it has accurate data to address its staffing needs, BRN should develop and implement training for all BRN complaint processing staff that instructs them on how to accurately enter information in complaint records that are contained in BreEZe, including the date BRN received the complaint, in a manner that is consistent with BRN's business processes.

Annual Follow-Up Agency Response From October 2018

The BRN has developed and implemented training for all BRN complaint processing staff to ensure the BRN is able to accurately monitor the performance of its complaint resolution process and has accurate data to address its staffing needs. The training instructs staff how to accurately enter information in complaint records that are contained in BreEZe, including the date BRN received the complaint, in a manner consistent with BRN's business processes.

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

The BRN provided training to staff instructing them on how to accurately enter information in complaint records that are contained in BreEZe. Further, it included procedures in its desk manual for staff to follow when entering information in complaint records that are contained in BreEZe.

1-Year Agency Response

The BRN has developed and implemented training for all BRN complaint processing staff to ensure the BRN is able to accurately monitor the performance of its complaint resolution process and has accurate data to address its staffing needs. The training instructs staff how to accurately enter information in complaint records that are contained in BreEZe, including the date BRN received the complaint, in a manner consistent with BRN's business processes.

California State Auditor's Assessment of 1-Year Status: Pending

Although BRN claims it has fully implemented this recommendation, it did not provide evidence demonstrating that it has. Until it does so, we will continue to assess the status of this recommendation as pending.

6-Month Agency Response

On February 24, 2017, all BRN Complaint Intake staff completed training on BreEZe input coding. This training ensures unit practice is consistent with the BRN business process and data collection needs.

The training curriculum is saved for future training of new staff or refresher training should it be necessary. All staff completed sign-in sheets for the training.

Training material was incorporated into the Complaint Intake Unit procedure manual for complaint and Consumer Protection Enforcement Initiative (CPEI) guideline evaluation and referral (see pages 77-78). Staff instructions for accurately inputting complaint receipt dates are located on pages 17, 40, 42, and 59 of the Complaint Intake Unit procedure manual.

California State Auditor's Assessment of 6-Month Status: Pending

BRN did not provide documentation to substantiate its claim of full implementation.

60-Day Agency Response

In order to ensure accurate monitoring of performance of the complaint resolution processes, all BRN complaint processing staff members have attended the SOLID BreEZe Basics for Enforcement. Proof of training completion is tracked in the BRN centralized training tracking system. All future new staff will also be required to attend this training.

Additionally, monitoring of staff performance is highlighted in the aforementioned detailed data reports. Manager review of the available data reports will identify issues such as coding errors and process delays.

All policies and procedures have been updated to include the specific BreEZe coding requirements for the complaint resolution process which is consistent with BRN's business processes.

California State Auditor's Assessment of 60-Day Status: Partially Implemented

Although BRN provided evidence of an Outlook calendar appointment for a mandatory BreEZe coding training, it did not provide evidence demonstrating that it provided the training to all BRN complaint processing staff and that the training instructed the staff on how to accurately enter information in complaint records that are contained in BreEZe, including the date BRN received the complaint, in a manner that is consistent with BRN's business processes.

Recommendation #15 To: Registered Nursing, Board of

BRN should immediately comply with state law and adhere to the revised CPEI guidelines that DOI issued in August 2016. Additionally, BRN should establish and maintain a process for communicating with DOI to discuss any questions that arise in assigning a priority to a complaint or referring a complaint to the proper investigative unit.

Annual Follow-Up Agency Response From October 2018

BRN complies with state law and adheres to the revised CPEI guidelines that DOI has issued. BRN has established and maintains a process to communicate with DOI to discuss any questions that arise in assigning a priority to a complaint or referring a complaint to the proper investigative unit.

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

BRN provided evidence demonstrating that it has established procedures to ensure it complies with state law and adheres to the revised CPEI guidelines that DOI has issued. Specifically, BRN developed complaint intake procedures that include a copy of the revised CPEI guidelines that DOI issued in August 2016, and direct staff that they must adhere to the CPEI guidelines. The procedures also outline a process for communicating with DOI to discuss any questions that arise in assigning a priority to a complaint or referring a complaint to the proper investigative unit.

1-Year Agency Response

BRN complies with state law and adheres to the revised CPEI guidelines that DOI has issued. BRN has established and maintains a process to communicate with DOI to discuss any questions that arise in assigning a priority to a complaint or referring a complaint to the proper investigative unit.

California State Auditor's Assessment of 1-Year Status: Pending

Although BRN claims it has fully implemented this recommendation, it did not provide evidence demonstrating that it has. Until it does so, we will continue to assess the status of this recommendation as pending.

6-Month Agency Response

Instructions have been incorporated into the Complaint Intake Unit Procedure manual, which has been implemented into normal staff practice. Data verified in BreEZe shows full compliance with CPEI case referral guidelines as of December 2016.

Staff instructions for the use of CPEI guidelines, resolution of CPEI application questions, and communication with DOI regarding CPEI can be verified on pages 77-79 of the Complaint Intake Unit Procedure manual.

California State Auditor's Assessment of 6-Month Status: Partially Implemented

BRN developed complaint intake procedures that include a copy of the revised CPEI guidelines DOI issued in August 2016, and that direct staff to adhere to the CPEI guidelines when assigning complaints. However, although these procedures direct staff to assign those complaints with allegations that clearly fall into categories 1 or 2 to DOI, they also instruct staff to assign complaints with vague allegations, or those that do not exactly fall within categories 1 or 2, to BRN's investigative unit without discussing them with DOI. By not communicating with DOI regarding the assignment of these types of complaints, BRN risks not assigning these complaints to the appropriate investigative unit.

60-Day Agency Response

Complaint Intake Staff have been trained and provided desk copies of the DCA Case Referral Guidelines for Investigations (Guidelines). Complaint Intake Staff Services Analysts evaluate complaints and refer cases to BRN or DOI investigations as required by the Guidelines.

Complaint Intake Policies and Procedures have been written to formalize the process.

Beginning December 2016, BRN management and DOI have met on several occasions to discuss ongoing case referral and priority issues. Since then, formally scheduled monthly meetings have been conducted to maintain constant communication regarding enforcement matters. Additional meetings will be scheduled as often as needed to ensure that the CPEI case referral guidelines and other matters of concern are being addressed in a timely manner.

California State Auditor's Assessment of 60-Day Status: Partially Implemented

Although BRN developed complaint intake procedures that include a copy of the revised CPEI guidelines that DOI issued in August 2016, these procedures do not direct staff that they must adhere to the CPEI guidelines. The procedures also do not outline a process for communicating with DOI to discuss any questions that arise in assigning a priority to a complaint or referring a complaint to the proper investigative unit.

Recommendation #16 To: Registered Nursing, Board of

To ensure that BRN and DOI consistently conduct adequate investigations and obtain sufficient and appropriate evidence to discipline nurses accused of violating the Nursing Act if warranted, BRN in collaboration with Consumer Affairs should implement a mechanism by March 2017 to track and monitor supplemental investigation requests that result from investigators' failure to obtain required documentation or sufficient evidence and use this information to mitigate the causes of these failures.

Annual Follow-Up Agency Response From November 2019

The BRN in collaboration with Consumer Affairs implemented a mechanism to track and monitor supplemental investigation requests and uses this information to mitigate the causes of these failures.

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

Annual Follow-Up Agency Response From October 2018

To ensure that BRN and DOI consistently conduct adequate investigations and obtain sufficient and appropriate evidence to discipline nurses accused of violation the Nursing Practice Act, BRN Subject Matter Experts worked with DCA BreEZe staff to create additional activity codes which identify the source of supplemental investigation (AGO, Expert, Board, Other), and the type of evidence requested (i.e. obtain records, interviews, subpoenas). Reports are run quarterly in BreEZe to track and monitor supplemental investigation requests. BRN Supervising Investigators and the BRN Complaint Intake and Investigations Chief analyze the data to identify trends in order to mitigate the causes of the failures. The BRN Complaint Intake and Investigations Chief communicates trends for DOI investigations at monthly meetings with DOI.

California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented

BRN implemented a mechanism to track and monitor supplemental investigation requests that result from investigators' failure to obtain required documentation or sufficient evidence. However, it did not provide adequate documentation to fully demonstrate that it uses this information to mitigate the causes of these failures.

1-Year Agency Response

To ensure BRN and DOI consistently conduct adequate investigations and obtain sufficient and appropriate evidence to discipline nurses accused of violating the Nursing Practice Act, the BRN has collaborated with DCA to implement a mechanism to track and monitor supplemental investigation requests that result from investigators' failure to obtain required documentation or sufficient evidence and uses this information to mitigate the causes of these failures.

California State Auditor's Assessment of 1-Year Status: Pending

Although BRN claims it has fully implemented this recommendation, it did not provide evidence demonstrating that it has. Until it does so, we will continue to assess the status of this recommendation as pending.

6-Month Agency Response

BRN Subject Matter Experts worked with DCA BreEZe staff to create additional activity codes which identify the source of supplemental investigation (AGO, Expert, Board, Other), and the type of evidence requested (i.e. obtain records, interviews, subpoenas). Staff coding procedures that require they document the reason for the supplemental investigation can be found on page 83 of the Complaint Intake Unit Procedure manual.

BRN updated the Complaint Intake Unit procedures for staff to identify the reason why a supplemental investigation is conducted in the activity results text box for the supplemental investigation activity code.

California State Auditor's Assessment of 6-Month Status: Pending

Although BRN's Complaint Intake Unit Procedures direct staff to identify the reason why a supplemental investigation is conducted in BreEZe, it did not provide documentation demonstrating that BreEZe indeed contains this functionality.

60-Day Agency Response

BRN SMEs have submitted SIRs to add additional activity codes to include all reasons why a supplemental investigation would be requested (i.e. obtain records, interviews, subpoenas).

The new/updated BreEZe activity codes, which will capture more detailed information regarding the origin and scope of supplemental investigation requests, have been developed and are scheduled to be implemented in the next BreEZe software release by the end of February 2017.

California State Auditor's Assessment of 60-Day Status: Partially Implemented

Although BRN implemented a mechanism in BreEZe on February 21, 2017 to allow it to track who requests a supplemental investigation and whether the request was for an interview, subpoena, or other documentation, this new mechanism does not allow BRN to identify or track whether the reason for the supplemental investigation request was the result of an investigator's failure to obtain required documentation or sufficient evidence during the initial investigation. Without this information, BRN is hindered in its ability to mitigate the causes of investigators' failure to obtain required evidence.

Recommendation #17 To: Registered Nursing, Board of

To ensure that BRN and DOI consistently conduct adequate investigations and obtain sufficient and appropriate evidence to discipline nurses accused of violating the Nursing Act if warranted, BRN in collaboration with Consumer Affairs should coordinate with the Attorney General to develop a biennial training program that includes techniques for gathering appropriate evidence and ensure that all investigators, including DOI's investigators, participate in this training.

Annual Follow-Up Agency Response From October 2018

To ensure BRN and DOI consistently conduct adequate investigations and obtain sufficient and appropriate evidence to discipline nurses accused of violating the Nursing Practice Act, the BRN has collaborated with DCA, the AGO, and DOI to develop and provide biennial training that includes techniques for gathering appropriate evidence and ensures all investigators, including DOI's investigators have participated in the training.

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

BRN provided documentation demonstrating it coordinated with the Attorney General in developing training for all investigators and provided evidence of both DOI and BRN investigators attendance records at the training. It also provided the training manual, which includes techniques for gathering appropriate evidence.

1-Year Agency Response

To ensure BRN and DOI consistently conduct adequate investigations and obtain sufficient and appropriate evidence to discipline nurses accused of violating the Nursing Practice Act, the BRN has collaborated with DCA, the AGO, and DOI to develop and provide biennial training that includes techniques for gathering appropriate evidence and ensures all investigators, including DOI's investigators have participated in the training.

California State Auditor's Assessment of 1-Year Status: Pending

Although BRN claims it has fully implemented this recommendation, it did not provide evidence demonstrating that it has. Until it does so, we will continue to assess the status of this recommendation as pending.

6-Month Agency Response

With the coordination of the AGO and participation of DCA's SOLID Training Solutions, DOI, and BRN staff; the BRN implemented a formal training curriculum developed by the AGO for DOI and BRN investigation staff.

The training was designed to educate BRN and DOI investigators in case development and evidentiary issues related to BRN complaints, as well as a description of what constitutes sufficient evidence when investigating BRN complaints.

As of March 22, 2017, all BRN and DOI investigators completed this training.

The training session on March 22nd was recorded and provided to BRN and DOI to make available to future new investigators as part of their required training.

This training requirement has been added to the BRN Investigator training plan in the Investigation Unit Procedure manual.

California State Auditor's Assessment of 6-Month Status: Pending

BRN did not provide documentation demonstrating it coordinated with the Attorney General in developing the training, evidence demonstrating that the training occurred, or documentation indicating the training is required. Until it provides such evidence, we will continue to report this recommendation as not fully implemented.

60-Day Agency Response

With the coordination of the Attorney General's Office and participation of DCA's SOLID Training Solutions, DOI, and BRN staff; the BRN has worked aggressively over the past six weeks to develop a formal training curriculum for DOI and BRN investigation staff. This training is designed to educate BRN and DOI investigators in case development and evidentiary issues related to BRN complaints, as well as a description of what constitutes sufficient evidence when investigating BRN complaints.

The outline of the training curriculum has been developed. The final curriculum, training materials, and training presentation are being developed by the AG's office. Training for all investigative staff is scheduled to be complete March 22, 2017. The BRN will record one of the sessions which will be used as an exemplar in the future for newly hired investigative staff. Staff will be required to attend biennial training sessions. This requirement has been added to the investigator training plan in the policy and procedure manual.

California State Auditor's Assessment of 60-Day Status: Pending

BRN did not provide any documentation demonstrating that the actions they have claimed to take thus far, were indeed taken. As a result, we are assessing the status of this recommendation as pending, given BRN indicates it will fully implement the recommendation.

Recommendation #18 To: Registered Nursing, Board of

To ensure that BRN and DOI consistently conduct adequate investigations and obtain sufficient and appropriate evidence to discipline nurses accused of violating the Nursing Act if warranted, BRN in collaboration with Consumer Affairs should use this training program to develop a procedural guide that specifies proper evidence-gathering techniques, including a description of what constitutes sufficient evidence, for investigators to follow when investigating complaints. They should then distribute this guide to all investigators, including DOI's investigators, by December 2017, and jointly instruct them to adhere to the guide when conducting investigations.

Annual Follow-Up Agency Response From October 2018

The BRN, in collaboration with DCA, developed a procedural guide that specifies proper evidence-gathering techniques, including a description of what constitutes sufficient evidence, for investigators to follow when investigating complaints. The guide was distributed to both BRN and DOI investigators and they were jointly instructed to adhere to the guide when conducting investigations.

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

BRN provided documentation demonstrating that its claim of full implementation is accurate.

1-Year Agency Response

The BRN, in collaboration with DCA, developed a procedural guide that specifies proper evidence-gathering techniques, including a description of what constitutes sufficient evidence, for investigators to follow when investigating complaints. The guide was distributed to both BRN and DOI investigators and they were jointly instructed to adhere to the guide when conducting investigations.

California State Auditor's Assessment of 1-Year Status: Pending

Although BRN claims it has fully implemented this recommendation, it did not provide evidence demonstrating that it has. Until it does so, we will continue to assess the status of this recommendation as pending.

6-Month Agency Response

The manual developed for Investigator Training as required in Auditor Recommendation #17, specifies proper evidence-gathering techniques, including a description of what constitutes sufficient evidence, for investigators to follow when investigating complaints. The manual was disseminated at the trainings held in March 2017. The training manual has been digitized and is available via the BRN Investigation shared network.

California State Auditor's Assessment of 6-Month Status: Pending

BRN did not provide evidence demonstrating that it has specified proper evidence-gathering techniques in its procedures and did not submit documentation indicating it had provided such direction to staff in its March 2017 training. Further, it did not provide documentation demonstrating that it distributed direction regarding proper evidence-gathering techniques to all investigators, including DOI's investigators. Until it does so, we will continue to report this recommendation as not fully implemented.

60-Day Agency Response

Once the two training sessions for BRN and DOI investigators have concluded, BRN will meet with DOI, SOLID, and AG to assess effectiveness of training materials and convert the training manual into a procedural guide manual that will be distributed to all BRN and DOI investigators with instructions to adhere when conducting investigations. This completion and distribution of this guide to staff is scheduled on or before December 2017.

California State Auditor's Assessment of 60-Day Status: Pending

Recommendation #19 To: Registered Nursing, Board of

To ensure that its enforcement unit employees appropriately address and process complaints in a consistent and efficient manner, by March 2017, BRN should develop a process to centrally track the internal and external trainings its staff participate in. On a regular basis, managers should review this information to ensure enforcement staff are participating in a timely manner in appropriate trainings that address the enforcement activities they specifically perform and the types of complaints they may investigate.

Annual Follow-Up Agency Response From October 2018

BRN developed a process to centrally track the internal and external trainings its staff participate in. BRN managers review the training information regularly to ensure enforcement staff are participating in appropriate trainings that address enforcement activities they specifically perform and types of complaints they may investigate in a timely manner.

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

BRN developed a process to centrally track the internal and external trainings its staff participate in, as well as a process for managers to regularly review this information to ensure enforcement staff are participating in a timely manner in appropriate trainings that address the enforcement activities they specifically perform and the types of complaints they may investigate.

1-Year Agency Response

BRN developed a process to centrally track the internal and external trainings its staff participate in. BRN managers review the training information regularly to ensure enforcement staff are participating in appropriate trainings that address enforcements activities they specifically perform and types of complaints they may investigate in a timely manner.

California State Auditor's Assessment of 1-Year Status: Pending

Although BRN claims it has fully implemented this recommendation, it did not provide evidence demonstrating that it has. Until it does so, we will continue to assess the status of this recommendation as pending.

6-Month Agency Response

Managers were given data from SOLID training solutions which outline all training records for BRN Staff. BRN created a centralized database to track existing and future training data for staff. This database is updated on an ongoing basis by the supervisors as training is completed.

Management Monitoring Plans have been updated to include staff training plans by classification.

California State Auditor's Assessment of 6-Month Status: Partially Implemented

Although BRN developed a database to centrally track internal and external trainings its enforcement staff have taken, it has not developed a process to ensure that managers regularly review that information to ensure enforcement staff are participating in appropriate trainings that address the enforcement activities they specifically perform and the types of complaints they may investigate.

60-Day Agency Response

Data from SOLID training solutions which outlines all training records for BRN Staff was obtained and customized for all areas of BRN to centrally track all internal and external trainings of staff participation.

Management monitoring plans have been updated to include minimum training requirements for all levels of enforcement staff. Managers will review and evaluate on an annual basis, or as needed, all trainings that have been completed by staff as well as any identified additional training related to types of complaints and investigations.

California State Auditor's Assessment of 60-Day Status: Partially Implemented

Although BRN developed a listing of internal and external trainings its enforcement staff have taken, it has not developed a process to centrally track the internal and external trainings or to ensure that managers review that information to ensure enforcement staff are participating in appropriate trainings that address the enforcement activities they specifically perform and the types of complaints they may investigate.

Recommendation #20 To: Registered Nursing, Board of

To ensure that its enforcement unit employees appropriately address and process complaints in a consistent and efficient manner, BRN should implement a formal training program no later than December 2017. In developing this program, BRN should consult with DOI and the Attorney General to identify training that could benefit its enforcement staff, and also solicit input of its enforcement staff on areas of their job duties where they believe they need additional training.

Annual Follow-Up Agency Response From November 2019

The BRN implemented a formal training program to ensure our Enforcement employees appropriately address and process complaints in a consistent and efficient manner. The BRN consulted with DOI, the Attorney General's Office, and Enforcement staff in the development and delivery of this training and additional training if warranted.

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

Annual Follow-Up Agency Response From October 2018

The BRN has implemented a formal training program to ensure enforcement staff address and process complaints in a consistent and efficient manner. The BRN has consulted with DOI and AGO to identify training that could benefit its enforcement staff and solicited input from enforcement staff on areas of their job duties where they believe additional training is needed.

California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented

The BRN provided evidence that it consulted with DOI to help inform the development of a formal training program. It also provided evidence that it solicited input from enforcement staff on areas of their job duties where they believe additional training is needed. However, it did not provide evidence of the formal training program itself or demonstrate that it consulted with the Attorney General to identify training that could benefit its enforcement staff. Until it does so, we will report this recommendation as partially implemented.

1-Year Agency Response

The BRN has implemented a formal training program to ensure enforcement staff address and process complaints in a consistent and efficient manner. The BRN has consulted with DOI and AGO to identify training that could benefit its enforcement staff and solicited input from enforcement staff on areas of their job duties where they believe additional training is needed.

California State Auditor's Assessment of 1-Year Status: Pending

Although BRN claims it has fully implemented this recommendation, it did not provide evidence demonstrating that it has. Until it does so, we will continue to assess the status of this recommendation as pending.

6-Month Agency Response

BRN has consulted with DOI, AGO and DCA to review and revise the DCA Enforcement Academy to address the various areas of enforcement. BRN enforcement management and staff have worked with DCA SOLID staff as subject matter experts in redesigning the DCA Enforcement Academy. The formal training courses will be taught in-person by SOLID staff and Subject Matter Experts (beginning January 2017) will include but are not limited to:

- Basics of Enforcement

- BreEZe Enforcement Training

- Complaint Intake

- Investigation Techniques

- Leadership and Safety in the Field

- Interview Techniques for Investigators

- Investigative Report Writing

- Courtroom Testifying

- Probation Monitoring

- Investigative Subpoena Preparation

Additional courses are currently in development for other areas of enforcement. The training courses will be monitored and managed by supervisors to ensure all staff completes these training courses in a timely manner.

Management monitoring plans have been updated to include minimum training requirements for all staff.

BRN enforcement management will work with DCAs SOLID to disseminate a training survey to solicit input from enforcement staff on areas of their job duties where they believe they need additional training. The information collected from the survey will be considered in developing additional training for staff.

California State Auditor's Assessment of 6-Month Status: Pending

60-Day Agency Response

BRN has consulted with DOI, AGO and DCA to review and revise the DCA Enforcement Academy to address the various areas of enforcement. BRN enforcement management and staff have worked with DCA SOLID staff as subject matter experts in redesigning the DCA Enforcement Academy. The formal training courses (available beginning January 2017) will include but are not limited to:

- Basics of Enforcement

- BreEZe Enforcement Training

- Complaint Intake

- Investigation Techniques

- Leadership and Safety in the Field

- Interview Techniques for Investigators

- Investigative Report Writing

- Courtroom Testifying

- Probation Monitoring

- Investigative Subpoena Preparation

Additional courses are currently in development for other areas of enforcement. The training courses will be monitored and managed by supervisors to ensure all staff completes these training courses in a timely manner.

California State Auditor's Assessment of 60-Day Status: Pending

Recommendation #21 To: Registered Nursing, Board of

BRN should immediately stop overriding fingerprint holds in BreEZe based solely on the fact that fingerprint data is present in BRN's legacy system and, for those cases where it believes it is necessary to override the system, BRN should receive its executive officer's approval to do so and document both the reason for the override and evidence of the executive officer's approval.

Annual Follow-Up Agency Response From October 2018

The BRN only overrides fingerprint holds in BreEZe if the Executive Officer has approved to do so and has documented the reason with evidence of the Executive Officer's approval.

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

BRN developed a procedure for staff to follow that describes the steps to take in BreEZe to override a fingerprint hold. This procedure also describes those instances where it is necessary for staff to override the system, and how staff should document the reasons for the override and evidence of the executive officer's approval. BRN provided evidence of the executive officer's approval.

1-Year Agency Response

The BRN only overrides fingerprint holds in BreEZe if the executive officer has approved to do so and has documented the reason with evidence of the executive officer's approval.

California State Auditor's Assessment of 1-Year Status: Pending

Although BRN claims it has fully implemented this recommendation, it did not provide evidence demonstrating that it has. Until it does so, we will continue to assess the status of this recommendation as pending.

6-Month Agency Response

The BRN has developed and implemented a manual fingerprint renewal process in BreEZe, which requires the Executive Officer (EO) approval. The approval of each record is documented in the renewal transaction notes section of BreEZe as well as a BreEZe report identifying each record that has been authorized by the EO for approval.

LiveScan fingerprint forms are maintained and monitored for results by staff. Once fingerprint results are received, populated to BreEZe, and verified by staff, the LiveScan forms are destroyed via confidential shred.

A "Processing Fingerprint Holds" procedure was developed and disseminated to staff. Only specific staff is authorized to clear the fingerprint business rule.

California State Auditor's Assessment of 6-Month Status: Pending

BRN did not provide any documentation to substantiate the statements it made regarding its claim of full implementation.

60-Day Agency Response

The BRN has developed and implemented a fingerprint renewal process in BreEZe, which requires the Executive Officer (EO) approval. The process documents the reason to approve the license renewal transaction as well as signatory approval by the EO. The documentation is scanned and attached to the licensee's renewal transaction and maintained in the BreEZe system.

California State Auditor's Assessment of 60-Day Status: Partially Implemented

Although BRN developed a process for staff to follow describing the steps to take in BreEZe to override a fingerprint hold, the process does not direct staff to only override the system in those instances where it believes it is necessary, obtain the executive officer's approval, and document both the reasons for the override and evidence of the executive officer's approval.

Recommendation #22 To: Registered Nursing, Board of

BRN should continue working with Justice and Consumer Affairs and finalize its reconciliation, by March 1, 2017, of Justice's fingerprint data with its data in BreEZe to identify any nurses who are missing fingerprint records. Once this reconciliation is performed, BRN must take the steps necessary to immediately obtain fingerprints from those nurses for which Justice has no fingerprint records.

Annual Follow-Up Agency Response From October 2018

The BRN has worked with the Departments of Justice and Consumer Affairs to finalize the reconciliation of fingerprint data between DOJ and BreEZe. BRN has taken steps to obtain fingerprints from nurses who do not have fingerprint records in DOJ's system.

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

BRN provided evidence that it finalized its reconciliation of its data and Justice's fingerprint data, and has taken the steps necessary to obtain fingerprints from those nurses for which Justice has no fingerprint records.

1-Year Agency Response

The BRN has worked with the Departments of Justice and Consumer Affairs to finalize the reconciliation of fingerprint data between DOJ and BreEZe. BRN has taken steps to obtain fingerprints from nurses who do not have fingerprint records in DOJ's system.

California State Auditor's Assessment of 1-Year Status: Pending

Although BRN claims it has fully implemented this recommendation, it did not provide evidence demonstrating that it has. Until it does so, we will continue to assess the status of this recommendation as pending.

6-Month Agency Response

All data reconciliation actions with the Department of Justice (DOJ) are complete and BRN is taking steps necessary to immediately obtain fingerprints from those nurses for which neither BRN nor DOJ has fingerprint records.

BRN identified approximately 16,000 current and active RN's missing fingerprint data. From December 2016 to present, BRN has processed approximately 10,100 notification letters to licensees with deficient fingerprint data. There are approximately 5,900 licensees remaining to be contacted. BRN is currently in the process of contacting the remaining individuals not in compliance with the fingerprint requirement.

Approximately 440 current and active licensees who failed to comply with the fingerprint notifications were referred to the Enforcement Division for possible disciplinary action.

California State Auditor's Assessment of 6-Month Status: Pending

60-Day Agency Response

Both BRN and DCA have been in communication with Department of Justice (DOJ) to complete the reconciliation of fingerprint records. To reconcile fingerprint data, the BRN needs DOJ's fingerprint database results in order to compare BRN BreEZe fingerprint records to determine which licensees still require fingerprint results. BRN will continue to work with DCA to identify any fingerprint data not available in BreEZe and that require licensee's to furnish fingerprints to the DOJ as required by law. BRN will also continue to collaborate with DCA to obtain updated fingerprint information from DOJ to reconcile all licensee records.

Using the fingerprint data received from DOJ in May 2016 the BRN is actively seeking fingerprint results for any licensee in BreEZe where records do not exist in an effort to update the data in the BreEZe system. The BRN has taken the following steps to notify licensees about the fingerprint renewal requirement:

- Initial and follow-up letters and e-mails have been mailed out to licensees where no fingerprint results exist in BreEZe or DOJ

- The BRN website has been updated with information regarding this requirement

- Announcements have been made at recent BRN board meetings held in January and February 2017

- A social media campaign has been launched in collaboration with DCA on FaceBook and Twitter regarding the fingerprint renewal requirement

- BRN executive management has met with nursing professional organizations (e.g. Hospital Association, ACNL, CNA, SEIU, UNAC, and ANA-C) who have shared this information with their nursing constituent groups throughout the state

- A telephone hotline and e-mail address were established to respond to the effected licensees

- Nurses who received the notification letters and do not comply with the requirement will be noticed and referred to enforcement for possible disciplinary action

California State Auditor's Assessment of 60-Day Status: Partially Implemented

Although BRN has taken steps to obtain the fingerprints from those nurses for which it knows are lacking fingerprints from the May 2016 Justice data, it still has not finalized its reconciliation of Justice's fingerprint data with its data in BreEZe to identify any nurses who are missing fingerprint records. Until it has done so, this recommendation is not fully implemented.

Recommendation #23 To: Registered Nursing, Board of

To ensure that it has prompt access to adequate information that could affect the status of a nurse's license, by June 2017, BRN should establish formal agreements with other agencies and other health boards that have information pertaining to a nurse's misconduct.

Annual Follow-Up Agency Response From October 2018

BRN has finalized an agreement for data sharing between the California Department of Public Health (CDPH) and the BRN. The final agreement was approved in November 2017. The BRN also has a data sharing agreement with the Bureau for Private Postsecondary Education (BPPE). The BRN finalized an agreement with the Los Angeles Superior Court in March 2018 to provide licensee information to the BRN.

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

BRN provided evidence demonstrating that it reached out to several health boards in an attempt to establish MOUs regarding the sharing of enforcement information. It also provided evidence demonstrating that it is in the process of developing an MOU with one health board. Further, BRN provided MOUs that it has entered into with certain agencies regarding the sharing of information, which, according to BRN's executive officer, will allow the sharing of information regarding nurse misconduct.

1-Year Agency Response

BRN has finalized an agreement for data sharing between the California Department of Public Health (CDPH) and the BRN. The final agreement was approved in November 2017. The BRN also has a data sharing agreement with the Bureau for Private Postsecondary Education (BPPE). The BRN finalized an agreement with the Los Angeles Superior Court in March 2018 to provide licensee information to the BRN.

On May 31, 2017, the BRN met with DCA Executives to discuss Memorandums of Understanding (MOU). The DCA is currently exploring the process of sharing enforcement information between all boards and bureaus.

Any agreement entered into by the board/bureau will conform to Business and Professions Code section 11180 etc. to ensure any agreement or MOU drafted either by DCA or as a result of a delegation granted by the Director is compliant with Business and Professions Code section 11183 so as to maintain the confidential nature of the information shared.

California State Auditor's Assessment of 1-Year Status: Pending

Although BRN claims it has fully implemented this recommendation, it did not provide evidence demonstrating that it has. Until it does so, we will continue to assess the status of this recommendation as pending.

6-Month Agency Response

BRN is currently finalizing an agreement for data sharing between the California Department of Public Health (CDPH) and the BRN. The document is currently being reviewed by CDPH legal. A final agreement is anticipated by July/August 2017.

BRN and the California Department of Corrections and Rehabilitation (CDCR) have discussed preliminary procedures for exchange of information for BRN complaints. Discussions are ongoing.

BRN's EO met with various Nursing and Hospital Associations to discuss possible solutions in an effort to obtain timely evidence during BRN investigations. Discussions are ongoing.

On May 31, 2017, BRN's EO, Assistant EO and Chief of Discipline, Probation and Intervention met with DCA's Director and Chief Deputy Director to discuss Memorandums of Understanding (MOU). The Director stated DCA will take the lead on this to assist all boards and bureaus regarding sharing enforcement information.

Any agreement entered into by the board/bureau will conform to Business and Professions Code section 11180 etc. to ensure any agreement or MOU drafted either by DCA or as a result of a delegation granted by the Director is compliant with Business and Professions Code section 11183 so as to maintain the confidential nature of the information shared.

BRN considers this item to be partially implemented pending additional action from community partners.

California State Auditor's Assessment of 6-Month Status: Pending

60-Day Agency Response

To ensure the BRN has prompt access to adequate data, the BRN has met with the California Department of Public Health (CDPH), California Department of Corrections and Rehabilitation (CDCR), and DCA's healing arts boards to discuss establishing formal agreements and/or memorandums of understanding (MOU) to share data that includes a nurse's misconduct and other pertinent information.

The BRN's EO met with several healing arts board's EO's in December 2016 to initiate discussion of establishing routine meetings regarding mutual issues of concern. Meetings will be scheduled on a quarterly basis. A more formal MOU will be presented to DCA's Legal Office for review which will allow all healing arts boards within DCA to share information.

Initial meetings were held with CDCR and CDPH during the month of January 2017. Follow-up meetings are scheduled to formalize the MOU process pending legal approval of the agreements by all involved parties.

California State Auditor's Assessment of 60-Day Status: Pending

Recommendation #24 To: Registered Nursing, Board of

To ensure that it has prompt access to adequate information that could affect the status of a nurse's license, by June 2017, BRN should work with Consumer Affairs and other health boards to determine whether modifying BreEZe to include a capability that would allow it to promptly notify BRN when another health board receives a complaint or takes disciplinary action against a licensed nurse is cost-effective. If it is, add this functionality to BreEZe.

60-Day Agency Response

The BRN SMEs worked with DCA's BIA to develop an Alert function that is ready to be deployed in the next BreEZe software release at the end of February 2017.

The BreEZe update will include an "Entity Level" alert which will notify BRN staff when another board and/or bureau in BreEZe has taken one of the following actions on a mutual licensee:

- Opened a new complaint, or

- Discipline against a license.

California State Auditor's Assessment of 60-Day Status: Fully Implemented

BRN provided documentation demonstrating that functionality was added to BreEZe to allow BRN to receive automatic notifications when another health board using BreEZe receives a complaint or takes disciplinary action against a licensed nurse.

Recommendation #25 To: Registered Nursing, Board of

To ensure that it promptly and appropriately sends notifications to complainants as state law requires, by March 2017, BRN should develop desk procedures that describe the actions enforcement staff members should take when processing incoming complaints and when BRN reaches a final disposition on a case.

6-Month Agency Response

Alerts were created based on the original auditor recommendation and are working as expected. Complaint intake policies and procedures were updated and include timeframe expectations.

As a result of feedback from the auditor, the BRN suggested additional changes to the Alerts, including the following:

- Update acknowledgement alert to 7 days

- Update closure letter alert to 21 days

- BRN submitted a ticket based on the Auditor 60-day response which was deployed in BreEZe release 2.18 on May 24, 2017.

California State Auditor's Assessment of 6-Month Status: Fully Implemented

BRN established an alert in BreEZe to automatically alert staff before the due date to send an acknowledgement letter within 10 days after BRN has received a complaint and a final disposition letter within 30 days after it has resolved a complaint. This alert will help ensure that BRN sends required notifications promptly. Additionally, BRN has developed procedures that describe the actions enforcement staff should take when processing incoming complaints and when BRN reaches a final disposition on a case.

60-Day Agency Response

The BRN SMEs worked with DCA's BIA to develop an Alert function within BreEZe if a complaint acknowledgement letter has not been created and sent within 10 days of a complaint being received by the BRN. Additionally, a second Alert has been created to notify when a closure letter has not been created within 30 days of the case closure date. These Alerts are ready to be deployed in the next BreEZe software release at the end of February 2017.

The Complaint Intake Policies and Procedures desk manuals have been updated to include complaint received acknowledgement and notification of closure timeframe requirements.

California State Auditor's Assessment of 60-Day Status: Partially Implemented

BRN established an alert in BreEZe to notify staff that they had not yet created an acknowledgement letter 10 days after BRN has received a complaint and when it has not sent a final disposition letter 30 days after it has resolved a complaint. However, because it established the alert to notify staff after it has already passed the deadline set by State Law for acknowledging receipt of a complaint, this alert will not ensure that it sends required notifications within the 10-day time frame specified in state law. Further, BRN has not developed procedures that describe the actions enforcement staff should take when processing incoming complaints and when BRN reaches a final disposition on a case.

Recommendation #26 To: Registered Nursing, Board of

To ensure that it promptly and appropriately sends notifications to complainants as state law requires, by March 2017, BRN should establish formal procedures, such as managers performing routine audits of complaint files, to monitor incoming complaints and final dispositions.

Annual Follow-Up Agency Response From November 2019

The BRN established formal procedures to monitor incoming complaints and final dispositions. These formal procedures include managers performing routine audits of complaint files and prompt notifications to complainants as appropriate.

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

Annual Follow-Up Agency Response From October 2018

To ensure BRN is promptly and appropriately sending notifications to complainants as state law requires, the BRN has established formal procedures for managers to perform routine audits of complaint files and monitor incoming complaints and final dispositions.

California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented

Although BRN developed high-level management monitoring procedures that include conducting regular audits and reviews of staff workload to ensure timely completion of complaint processing, these procedures do not specify how management will ensure staff promptly and appropriately send required notifications to complainants. In addition, BRN did not demonstrate evidence of reports it uses to monitor and verify that its staff send notifications as the law requires. Until it does so, we will continue to assess this recommendation as partially implemented.

1-Year Agency Response

To ensure BRN is promptly and appropriately sending notifications to complainants as state law requires, the BRN has established formal procedures for managers to perform routine audits of complaint files and monitor incoming complaints and final dispositions.

California State Auditor's Assessment of 1-Year Status: Pending

Although BRN claims it has fully implemented this recommendation, it did not provide evidence demonstrating that it has. Until it does so, we will continue to assess the status of this recommendation as pending.

6-Month Agency Response

Complaint intake policies and procedures have been updated to include staff timeframe expectations. The management monitoring plan outlines the process for the manager to audit all staff within the unit on an ongoing and routine basis to ensure staff is appropriately managing workload and adhering to timeframe expectations.

California State Auditor's Assessment of 6-Month Status: Pending

BRN did not provide any evidence that it has established formal procedures, such as requiring managers to perform routine audits of complaint files, to monitor incoming complaints and final dispositions. Until it does so, we will continue to report this recommendation as not fully implemented.

60-Day Agency Response

In addition to the Alert function and Policy and Procedure updates included in item 25 above, BRN Complaint Intake has developed formal management monitoring procedures which require monthly audits of complaint files to monitor incoming complaints and final dispositions. Moreover, the management monitoring procedures also highlight the use of the new data reports to evaluate pending workload and evaluate whether cases are progressing throughout the enforcement process in accordance with established milestones.

California State Auditor's Assessment of 60-Day Status: Partially Implemented

Although BRN developed high-level management monitoring procedures, these procedures do not specify how management will monitor incoming complaints and final dispositions, such as by performing routine audits, to ensure it promptly and appropriately sends required notifications to complainants.

All Recommendations in 2016-046

Agency responses received are posted verbatim.