Report 2013-302/2013-303 All Recommendation Responses

Report 2013-302/2013-303: Judicial Branch Procurement: Semiannual Reports to the Legislature Are of Limited Usefulness, Information Systems Have Weak Controls, and Certain Improvements in Procurement Practices Are Needed (Release Date: December 2013)

Recommendation for Legislative Action

To improve the usefulness of the Judicial Council's semiannual reports, the Legislature should amend the Judicial Branch Contract Law to require the Judicial Council to:

- Make the semiannual reports available in an electronic format that can be read by common database and spreadsheet software products that allow users to readily sort and filter the data.

- Include new contracts and the complete history of contracts amended during the reporting period in its semiannual reports, including the date of the original contract; the original contract amount and duration; all subsequent contract amendments; and the date, amount, and duration of each such amendment.

- Include information on whether a contract was competitively bid, the justification if it was not competitively bid, and whether the contract was with a Disabled Veteran Business Enterprise. For information technology contracts, the Judicial Council should identify whether the contract was with a small business.

Description of Legislative Action

The Legislature has not taken action to address this specific recommendation.

California State Auditor's Assessment of Annual Follow-Up Status: No Action Taken

Description of Legislative Action

Legislation has not been introduced to address this specific recommendation.

California State Auditor's Assessment of Annual Follow-Up Status: No Action Taken

Description of Legislative Action

Legislation has not been introduced to address this specific recommendation.

California State Auditor's Assessment of Annual Follow-Up Status: No Action Taken

Description of Legislative Action

Legislation has not been introduced to address this recommendation.

California State Auditor's Assessment of Annual Follow-Up Status: No Action Taken

Description of Legislative Action

Legislation has not been introduced to address this recommendation.

California State Auditor's Assessment of Annual Follow-Up Status: No Action Taken

Description of Legislative Action

Legislation has not been introduced to address this specific recommendation

California State Auditor's Assessment of 1-Year Status: No Action Taken

Recommendation #2 To: Administrative Office of the Courts

To improve the usefulness of the Judicial Council's semiannual reports until a statutory requirement is enacted, the AOC should work with the Judicial Council to pursue a cost-effective method to provide the semiannual reports in an electronic format that can be read by common database and spreadsheet software products that allow users to readily sort and filter the data, beginning with the semiannual report covering the July 1, 2013 through December 31, 2013, reporting period.

Annual Follow-Up Agency Response From November 2016

The Judicial Council of California (Judicial Council) has posted on its website vendor contract and payment data in Microsoft Excel, thus allowing the public to search for specific information such as vendor name, payment amount, and general purpose. This information from the website can be accessed at "www.courts.ca.gov/7466.htm." These contractor and payment reports pertain to both the activities of the trial courts, as well as the courts of appeal, the Habeas Corpus Resource Center, and the Judicial Council.

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

1-Year Agency Response

The Phoenix report has multiple lines for each contract that CSA has requested be changed to one line for sorting. The Phoenix team met with CSA on October 30 in Sacramento and a new sortable summary layout was agreed upon, is in development, and will be delivered going forward starting with the report to be issued on Feb. 1, 2015.

The Oracle report will have the blank fields filled in for the February 1, 2015 reporting as the change is minor to accomplish.

California State Auditor's Assessment of 1-Year Status: Pending

6-Month Agency Response

The semiannual report was provided on the California Courts web site in an excel format starting with the Feb. 1, 2014 report covering the period July 1, 2013 through December 31, 2013. While awaiting pending changes in legislation regarding additional data reporting requirements, no programming changes were made in the reporting to make them more readily sortable. As the legislation was suspended, programming changes will be initiated for the sorting for the Feb. 1, 2015 reporting.

California State Auditor's Assessment of 6-Month Status: Partially Implemented

60-Day Agency Response

The semiannual report covering the July 1, 2013, through December 31, 2013 reporting period is on the California Courts web site in an Excel format. The web address is www.courts.ca.gov/7466.htm.

Please note that the narrative report provides specific tables for 1) large contracts (over $650K for the AOC) and 2) payments by type for the trial courts.

California State Auditor's Assessment of 60-Day Status: Partially Implemented

Although the AOC has provided the semiannual report covering the July 1, 2013, through December 31, 2013 reporting period in an electronic format on the California Courts web site, several issues still limit the ability of users to readily sort and filter the data. For example, the report for trial court contracts describes aspects of each contract, such as the total amount, duration of contract, or service provided, on multiple lines. Thus, if a user sorted trial court contracts by total amount from highest to lowest, and became interested in learning more about one particular contract, the user would be unable to see additional information about that contract. Fixing these issues would not take an inordinate amount of time or effort, and would greatly increase the usefulness of the semiannual reports.

Recommendation #3 To: Administrative Office of the Courts

To improve the usefulness of the Judicial Council's semiannual reports until a statutory requirement is enacted, the AOC should work with the Judicial Council to pursue a cost-effective method to do the following:

- Include new contracts and the complete history of contracts amended during the reporting period in the semiannual reports, including the date of the original contract; the original contract amount and duration; all subsequent contract amendments; and the date, amount, and duration of each such amendment. The AOC should present this information beginning with the semiannual report covering the July 1, 2014, through December 31, 2014, reporting period.

1-Year Agency Response

The implementation of the changes including additional data elements was predicated on the proposed legislation becoming statute and funding being provided. The legislation was suspended and no funding has been provided. Without additional funding, the costs involved in the implementation are significant when budget cuts have resulted in other priorities for the funding we have.

California State Auditor's Assessment of 1-Year Status: Will Not Implement

We maintain that the AOC should improve the usefulness of the Judicial Council's semiannual reports by including new contracts and the complete history of contracts amended during the reporting period in the semiannual reports.

6-Month Agency Response

The implementation of the changes including additional data elements was predicated on the proposed legislation becoming statute and funding being provided. The legislation was suspended and no funding has been provided. Without additional funding, the costs involved in the implementation are significant when budget cuts have resulted in other priorities for the funding we have.

California State Auditor's Assessment of 6-Month Status: No Action Taken

60-Day Agency Response

Much of the information relating to the complete history of amended contracts recommended by the Auditor for inclusion in the semiannual reports beginning with the semiannual report covering the July 1, 2014, through December 31, 2014, reporting period resides in physical files maintained by individual judicial branch entities files that are not accessible electronically without modifications to systems and processes and procurement of software licenses estimated to be in excess of $1 million dollars, not counting the cost of additional time required of staff at each judicial entity to cull and upload the information from paper files.

The AOC can attempt to pursue a cost-effective method to provide in the semiannual report the additional information recommended by the Auditor that is not currently statutorily mandated. However, in light of the significant ongoing budget reductions to judicial entities, and specifically to trial courts, that have negatively impacted the public services they provide, whether this recommendation can be fulfilled in the timeframe requested (July 1 December 31, 2014 reporting period) without further impairment of the publics access to justice is unclear, unless additional funding is provided.

California State Auditor's Assessment of 60-Day Status: No Action Taken

Recommendation #4 To: Administrative Office of the Courts

To improve the usefulness of the Judicial Council's semiannual reports until a statutory requirement is enacted, the AOC should work with the Judicial Council to pursue a cost-effective method to do the following:

- Begin tracking additional information in its data systems for inclusion in the semiannual reports. This information should include whether a contract was competitively bid, the justification if it was not competitively bid, and whether the contract was with a Disabled Veteran Business Enterprise. For information technology contracts, the AOC should identify whether the contract was with a small business. The AOC should present this information beginning with the semiannual report covering the July 1, 2014, through December 31, 2014, reporting period.

1-Year Agency Response

The implementation of the changes including additional data elements was predicated on the proposed legislation becoming statute and funding being provided. The legislation was suspended and no funding has been provided. Without additional funding, the costs involved in the implementation are significant when budget cuts have resulted in other priorities for the funding we have.

California State Auditor's Assessment of 1-Year Status: Will Not Implement

We maintain that the AOC should improve the usefulness of the Judicial Council's semiannual reports by tracking additional information in its data systems for inclusion in the semiannual reports, such as whether a contract was competitively bid, the justification if it was not competitively bid, and whether the contract was with a Disabled Veteran Business Enterprise.

6-Month Agency Response

The implementation of the changes including additional data elements was predicated on the proposed legislation becoming statute and funding being provided. The legislation was suspended and no funding has been provided. Without additional funding, the costs involved in the implementation are significant when budget cuts have resulted in other priorities for the funding we have.

California State Auditor's Assessment of 6-Month Status: No Action Taken

60-Day Agency Response

Much of the information relating to whether a contract was competitively bid or whether the contract was awarded to a Disabled Veteran Business Enterprise (or a small business for information technology contracts) resides in physical files maintained by individual judicial branch entities files that are not accessible electronically without modifications to systems and processes and procurement of software licenses estimated to be in excess of $1 million dollars, not counting the cost of additional time required of staff at each judicial entity to cull and upload the information from paper files.

The AOC can attempt to pursue a cost-effective method to provide in the semiannual report the additional information recommended by the Auditor. However, in light of the significant ongoing budget reductions to judicial entities, and specifically to trial courts, that have negatively impacted the public services they provide, whether this recommendation can be fulfilled in the timeframe requested (July 1 December 31, 2014 reporting period) without further impairment of the publics access to justice is unclear, unless additional funding is provided.

California State Auditor's Assessment of 60-Day Status: No Action Taken

Recommendation #5 To: Administrative Office of the Courts

This recommendation has been superseded by a recommendation from a subsequent report. See 2015-302 #2.

The AOC should implement all of the best practices related to general and business process application controls as outlined in the U.S. Government Accountability Office's Federal Information System Controls Audit Manual no later than December 31, 2014, thereby strengthening and continuously monitoring the effectiveness of the controls over its information systems. In addition, the AOC should immediately begin implementing improvements to its controls over access to its information systems and place these improvements into effect by February 2014. Finally, the AOC should provide guidance and routinely follow up with the superior courts—requiring updates every six months until all identified issues are corrected—to ensure that they make the necessary improvements to their general and business process application controls.

Annual Follow-Up Agency Response From November 2016

The Judicial Council of California (Judicial Council) was able to secure the Governor's approval for $3.1 million in additional funding during fiscal year 2016-17 to strengthen information technology security and disaster recovery programs, with additional ongoing funding of $1.9 million in subsequent years. This funding will result in the implementation of user access auditing tools at the courts, the establishment of annual information system risk assessments, and the implementation of a formalized information technology security plan. The BCP also provides funding for three full time employees to support the IT security and disaster recovery programs within the Judicial Council. The Judicial Council anticipates having additional information on an anticipated timeline in our next annual update.

California State Auditor's Assessment of Annual Follow-Up Status: See 2015-302 #2 for the most current assessment

1-Year Agency Response

In June 2014, the Judicial Council (formerly the AOC) adopted a framework of information systems controls that was based on the industry standards. Federal Information System Controls Audit Manual (FISCAM) control activities. Subsequent to the adoption of this framework, a number of efforts were initiated to facilitate its implementation: (1) a risk management overview in Sept. 2014; (2) an IT operational risk assessment was conducted to determine areas where the development of policies and procedures were required and a presentation of the findings will occur in Dec.2014; (3) a consolidated policies and procedures manual has been drafted and is currently being reviewed by IT Office staff prior to its release and publication expected in the third quarter of this fiscal year. While implementation of best practices as demonstrated above will continue, full implementation of all of the practices will require resources and time. It is expected that substantive compliance will be completed in 2015 but further work to be compliant in all areas may be necessary and additional funding will be requested. Budget Change Proposals (BCP's) may be prepared to request funding to implement some of the more complex tasks and to be able to address all of the issues. The JC may submit the BCP for its activities in Jan. 2015. As noted in our prior response, access changes are controlled through an automated process and issues in audits are periodically followed-up. Periodically guidance has been provided through posting of audit reports on the California Courts web site, issuance of systemic issues in audit reports advisory committees and issuance of guidance to trial courts such as the Jan. 31, 2014 memorandum to superior court CEO's, HR officers, and information officers. JC staff also continues to contact courts with audit reports issued during the last two years concerning the status of incomplete items. Status reports will be prepared periodically.

California State Auditor's Assessment of 1-Year Status: Partially Implemented

6-Month Agency Response

The Information Systems Controls Framework (ISCF) for the Administrative Office of the Courts (AOC) has been approved for the AOC by the AOCs Executive Office on June 12, 2014. This framework incorporates industry standards and best practices and the document has been provided to the California State Auditor. Implementation is in progress.

At the August meeting of the Judicial Council the Strategic and Tactical Plans for Technology for the Judicial Branch recently developed by the Judicial Council Technology Committee (JCTC) are expected to be approved. The AOC ISCF aligns with the plans to be approved by the Judicial Council. The California State Auditor has been sent the letter sent to trial court presiding judges and court executives concerning this under the signatures of the chairs of the JCTC, Trial Court Presiding Judges Advisory Committee, and the Court Executives Advisory Committee. (estimated completion December 2014)

The AOC has moved from a manual process for recording access to the implementation of an automated overlay process (May 2014) to record all access changes.

Audit Services of the AOC continues to contact courts on a periodic basis concerning the status of incomplete items. Audit Services reports status of contact process to the Chief of Staff. (Feb. 2014 and on-going)

California State Auditor's Assessment of 6-Month Status: Partially Implemented

60-Day Agency Response

This is a multi-part recommendation with different target/completion dates.

1. A draft framework for information systems policies for the AOC has been prepared with a draft action plan for implementation at the AOC and encompasses FISCAM, ISO, and other industry guidance relating to best practices. This draft framework will also be utilized in the governance and compliance model for the superior courts and will be submitted for final approval to the Judicial Council at its June 2014 meeting. Dec. 2014

2. The AOC has implemented a manual process whereby anyone who is separated or terminated will be taken off of the network on the day they leave (physically leave not when vacation etc. run out) and a screen copy of the Active Directory (AD) entry is made and placed in file to provide documentation supporting this action. Individuals who change positions will have their access reviewed and approved without copying another individuals access allowances. Feb. 2014

3. Internal Audit Services has contacted courts with audit reports issued during the last two years concerning the status of incomplete items and requested status of incomplete issues for any issues not reported subsequently as completed. Follow up will also be done periodically on each finalized audit report going forward.

Status reports will be prepared every six months and forwarded to the AOCs Executive Office for open issues from audit reports finalized.

January 2014

California State Auditor's Assessment of 60-Day Status: Partially Implemented

Recommendation #6 To: Administrative Office of the Courts

The AOC, the Supreme Court, and the first, second, and fourth districts should implement procedures to ensure that they follow a competitive process for their procurements when required.

Annual Follow-Up Agency Response From November 2016

The Judicial Council of California (Judicial Council) believes its current Judicial Branch Contracting Manual contains adequate procedures to ensure judicial branch entities (JBEs) follow competitive procurement requirements and state law. The State Auditor has concluded that this recommendation is only partially implemented because it believes the Judicial Council should require courts to have competitive bid summaries in all procurement files. The Judicial Council believes its current practices and documentation complies with existing law and is pleased more recent reviews by the State Auditor have yielded only a minimum number of audit findings.

California State Auditor's Assessment of Annual Follow-Up Status: Will Not Implement

1-Year Agency Response

Procurement/contract files contain documentation to support competitive or non-competitive bidding including the determination of fair and reasonable pricing when required. While we follow the JBCM in all areas and see no need for further procedures, consideration will be given to enhancement of procedures in the next update of the JBCM.

Additionally, Judicial Council (formerly the AOC) staff will review the competitive process for procurements, including documenting procurement summaries, with the Supreme Court, and the first, second, and fourth appellate districts during training.

California State Auditor's Assessment of 1-Year Status: Partially Implemented

In the AOC's 60-day response, it indicated that it is was including a competitive bid summary in its procurement files. However, the AOC has yet to implement procedures, such as revising the judicial contracting manual, to ensure that the summary is included with all competitively bid procurements. .

6-Month Agency Response

The procedural changes already implemented and discussed with the California State Auditor will be part of items being considered for the update of the Judicial Branch Contracting Manual (JBCM) later this calendar year and effective Jan. 1, 2015.

California State Auditor's Assessment of 6-Month Status: Partially Implemented

60-Day Agency Response

The AOC has already implemented changes to its procurement process to ensure that each procurement/contract file has either a competitive bid summary or a non-competitive bid form; any amendments that add money to the procurement would require a non-competitive bid form. The non-competitive bid form has been revised to address the fair and reasonable pricing when required by the JBCM, regardless of the value of the procurement. This form was distributed to the referenced judicial branch entities in January 2014.

California State Auditor's Assessment of 60-Day Status: Partially Implemented

The AOC's response indicates that it is now including a competitive bid summary in its procurement files. However, the AOC has yet to implement procedures, such as revising the judicial contracting manual, to ensure that the summary is included with all competitively bid procurements. .

Recommendation #7 To: Court of Appeal, First District

The AOC, the Supreme Court, and the first, second, and fourth districts should implement procedures to ensure that they follow a competitive process for their procurements when required.

6-Month Agency Response

In addition to the measures described in the courts response of February 14, 2014, the court has finalized a checklist to aid court staff procuring goods and services in complying with the Judicial Branch Contracting Manual and the Local Contracting Manual for the Court of Appeal, First District. A copy of the checklist is today being e-mailed to Aaron Fellner of your office.

California State Auditor's Assessment of 6-Month Status: Fully Implemented

60-Day Agency Response

All court staff involved in purchasing have been verbally instructed to review and follow the updated Judicial Branch Contracting Manual (JCBM) and the Local Contracting Manual for the Court of Appeal, First District (LCM), which are comprehensive. When there are questions about the manuals requirements, the AOC Business Services office can be contacted for assistance. There will be additional focus on and review of each procurement/contract for compliance with the JBCM and LCM. A draft checklist is in the process of finalization and will mirror the requirements of the JBCM and LCM.

California State Auditor's Assessment of 60-Day Status: Partially Implemented

Although the first district has taken some steps to implement the recommendation, the draft checklist has yet to be finalized and used by court staff to procure goods and services.

Recommendation #8 To: Court of Appeal, Fourth District

The AOC, the Supreme Court, and the first, second, and fourth districts should implement procedures to ensure that they follow a competitive process for their procurements when required.

60-Day Agency Response

The Fourth District is using a checklist that is included in all procurement files and helps guarantee that the competitive process for procurements is followed.

California State Auditor's Assessment of 60-Day Status: Fully Implemented

Recommendation #9 To: Court of Appeal, Second District

The AOC, the Supreme Court, and the first, second, and fourth districts should implement procedures to ensure that they follow a competitive process for their procurements when required.

60-Day Agency Response

Second Appellate Court response: Have implemented changes to our Local JBCM and checklist to ensure compliance in the future.

California State Auditor's Assessment of 60-Day Status: Fully Implemented

Recommendation #10 To: Supreme Court of California

The AOC, the Supreme Court, and the first, second, and fourth districts should implement procedures to ensure that they follow a competitive process for their procurements when required.

1-Year Agency Response

The Court developed and implemented a comprehensive checklist to ensure the documentation of fair and reasonable pricing, evaluation and selection processes, and justifications and approvals for sole-source purchases are addressed going forward. The checklist was forwarded to the email address below on December 11, 2014.

California State Auditor's Assessment of 1-Year Status: Fully Implemented

6-Month Agency Response

This will be fully implemented at the same time the AOC implements that changes in the procedures.

California State Auditor's Assessment of 6-Month Status: Partially Implemented

60-Day Agency Response

Will ensure going forward that when it is determined that the pricing is fair and reasonable, documentation for such determination is maintained in the appropriate procurement file.

California State Auditor's Assessment of 60-Day Status: Partially Implemented

The Supreme Court has developed a draft checklist to ensure it adheres to procurement requirements. However, the Supreme Court has not yet finalized the document or used it when entering into new contracts.

Recommendation #11 To: Administrative Office of the Courts

The AOC should implement procedures to ensure that agreements it considers LPAs include in their terms and conditions language that expressly allows other judicial entities to use them.

60-Day Agency Response

The AOC has model language for use in agreements it considers LPAs. The model language for use in the terms and conditions for LPAs specifies that the Purchasing Group may use the agreement and the definition of Purchasing Group typically includes all judicial branch entities.

California State Auditor's Assessment of 60-Day Status: Fully Implemented

Recommendation #12 To: Administrative Office of the Courts

The AOC should provide additional training to its staff and the judicial entities on how to conduct procurements in compliance with the judicial contracting manual.

Annual Follow-Up Agency Response From November 2016

The Judicial Council of California provided additional training to court procurement staff in December 2015 and March 2016.

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

1-Year Agency Response

We have discussed the need for training with appellate courts at a recent meeting. In the first quarter of 2015, Business Services will offer training to the appellate courts and Judicial Council (formerly the AOC) staff on compliance with the judicial contracting manual (JCBM).

California State Auditor's Assessment of 1-Year Status: Partially Implemented

6-Month Agency Response

Monthly procurement conference calls continue where all trial courts and other judicial branch entities participate with general and specific topics, and specifically requested topics discussed. (Continuous)

The AOC asked appellate court personnel to review the original training material provided by the AOC and the current JBCM, and to then inform the agency of what sections of the JBCM they would like the training to focus on. Feedback has been received and training needs are being evaluated with an October tentative timing for the training. (October 2014)

In addition, a contract specialist position, which will be dedicated to appellate court procurements, has been approved. The person hired for this position will report to the business services manager to ensure consistent compliance with the JBCM.

California State Auditor's Assessment of 6-Month Status: Partially Implemented

60-Day Agency Response

The AOC asked appellate court personnel to review the original training material provided by the AOC and the current JBCM, and to then inform the agency of what sections of the JBCM theyd like the training to focus on. Feedback has been received and training needs and timing are being evaluated.

In addition, a contract specialist position, which will be dedicated to appellate court procurements, has been approved and is in the process of being recruited. The person hired for this position will report to the business services manager to ensure consistent compliance with the JBCM.

California State Auditor's Assessment of 60-Day Status: Partially Implemented

Recommendation #13 To: Administrative Office of the Courts

The AOC should revise the judicial contracting manual to require judicial entities to maintain documentation on their determinations of fair and reasonable pricing for purchases under $5,000.

Annual Follow-Up Agency Response From November 2016

As noted in our response from December 2014, the Judicial Council of California's (Judicial Council) contracting manual is consistent with both the Public Contract Code and the State Contracting Manual. The Judicial Council respectfully declines to implement this recommendation and is pleased that the State Auditor's more recent reviews have yielded only a minimal number of audit findings.

California State Auditor's Assessment of Annual Follow-Up Status: Will Not Implement

1-Year Agency Response

Public Contract Code (PCC) section 19206 requires the Judicial Council (formerly the AOC) to adopt a judicial contracting manual that is "consistent with" the PCC and "substantially similar" to the State Administrative Manual and the State Contracting Manual (SCM).

The SCM states at volume 1, section 5.90: "Contracts under $5,000 are not required to be competitively bid (PCC section 10335). However, agencies should obtain price quotes if there is a reason to believe a response from a single source is not a fair and reasonable price. Cost justification should be documented."

Chapter 5, section 5.1 of the judicial contracting manual states in part: "JBEs may purchase non-IT goods, non-IT services, or IT goods and services that cost less than $5,000 without conducting a competitive procurement so long as the Buyer determines that the pricing is fair and reasonable. The Buyer should include documentation on fair and reasonable pricing in the procurement file."

Consistent with the PCC, the judicial contracting manual and the SCM require a determination of fair and reasonable pricing for contracts under $5,000. The judicial contracting manual provides that fair and reasonable pricing for goods and services under $5,000 "should be documented," similar to the plain language of the SCM.

The Judicial Council (formerly the AOC) views the judicial contracting manual to be not only consistent with the PCC in this regard but also substantially similar to the SCM and declines to implement the recommendation.

California State Auditor's Assessment of 1-Year Status: Will Not Implement

The SCM provides that when "should" is used, it is a requirement that needs to be followed unless the agency has a good business reason for the variance. However, the judicial contracting manual notes that when the word "should" is used, compliance is not mandatory, but is favored unless there is a good business reason for the variance. While the Judicial Council may deem the language in the judicial contracting manual as "substantially similar" to that in the SCM, our audit revealed that, in practice, judicial entities did not consistently maintain documentation that the price they paid was fair and reasonable for procurements under $5,000. We therefore recommended that the AOC revise the judicial contracting manual to require judicial entities to maintain documentation on their determinations to justify the price paid for the good or service. As we stated in our report, until the judicial contracting manual requires this documentation, the judicial branch risks being unable to demonstrate whether entities are obtaining a fair and reasonable price on procurements under $5,000.

6-Month Agency Response

Consistent with the previous response, the State Auditor's request to revise the Judicial Branch Contracting Manual in section 5.1 from 'should' to 'require' documentation in file for fair and reesonable pricing will be considered in the process for updating the manual this year.

California State Auditor's Assessment of 6-Month Status: Pending

60-Day Agency Response

As a practical and operational matter, procurement files for non-competitive procurements under $5,000 would generally contain substantiation of fair and reasonable pricing and the Evaluation Team would document the evaluation and selection process for competitive procurements. The AOC will nonetheless, and most certainly, address the Auditors recommendation with other judicial entities and the Advisory Committee on Financial Accountability and Efficiency for the Judicial Branch, before a further revision of JBCM is submitted to the Judicial Council for approval consistent with the Auditors recommendation. The next anticipated JBCM revision will be effective 1/1/2015.

The JBCM (version effective 1/1/14) states at section 5.1: The Buyer should include documentation on fair and reasonable pricing in the procurement file. The should indicates reasonable discretion as does the Auditors recommendation.

California State Auditor's Assessment of 60-Day Status: Pending

Although the AOC is planning on addressing this recommendation, it has yet to revise the judicial contracting manual to require judicial entities to maintain documentation on their determinations of fair and reasonable pricing for purchases under $5,000. Further, the AOC's response asserts that the "should" in our recommendation indicates reasonable discretion in whether it needs to revise its judicial contract manual. To clarify, the recommendation should by no means be contused as discretionary. Rather, the recommendation will not be considered fully implemented until the AOC revises the judicial contracting manual to require judicial entities to maintain documentation on their determinations of fair and reasonable pricing for purchases under $5,000.

Recommendation #14 To: Court of Appeal, Fifth District

The first, fifth, and sixth districts should develop procedures to ensure that they consistently maintain documentation of their determinations that the pricing obtained is fair and reasonable for procurements under $5,000.

60-Day Agency Response

The Court has developed and implemented a comprehensive checklist to ensure the documentation of fair and reasonable pricing, evaluation and selection processes, and justifications and approvals for sole-source purchases are addressed going forward.

California State Auditor's Assessment of 60-Day Status: Fully Implemented

Recommendation #15 To: Court of Appeal, First District

The first, fifth, and sixth districts should develop procedures to ensure that they consistently maintain documentation of their determinations that the pricing obtained is fair and reasonable for procurements under $5,000.

6-Month Agency Response

In addition to the measures described in the courts response of February 14, 2014, court staff have created a new filing system that specifically prompts them to include documentation that such pricing is fair and reasonable.

California State Auditor's Assessment of 6-Month Status: Fully Implemented

60-Day Agency Response

All court staff involved in purchasing have been verbally instructed to review and follow the updated Judicial Branch Contracting Manual (JCBM) and the Local Contracting Manual for the Court of Appeal, First District (LCM), which are comprehensive. When there are questions about the manuals requirements, the AOC Business Services office can be contacted for assistance. There will be additional focus on and review of each procurement/contract for compliance with the JBCM and LCM. A draft checklist is in the process of finalization and will mirror the requirements of the JBCM and LCM.

California State Auditor's Assessment of 60-Day Status: Partially Implemented

The first district has indicated that it is developing a checklist that will include steps court staff must take to document fair and reasonable pricing on procurements under $5,000. However, the checklist has yet to be finalized and used by court staff to procure goods and services.

Recommendation #16 To: Court of Appeal, Sixth District

The first, fifth, and sixth districts should develop procedures to ensure that they consistently maintain documentation of their determinations that the pricing obtained is fair and reasonable for procurements under $5,000.

6-Month Agency Response

The Sixth District Court of Appeal is now utilizing a Buyers Checklist for Non-Competitive procurements under $5,000 (attached). Each buyer will use this checklist to document vendors contacted and quotes received for these procurements. The checklist on page two will be where the buyer will document the obtaining of a fair and reasonable price being obtained for these procurements. This checklist is still considered an interim checklist as during the next few months in coordination with the other districts we will complete a more standardized checklist for general use. This and other checklists and logs will be part of the anticipated training on procurements and training to be provided by the AOC later this calendar year.

Additionally, a file will be maintained for all procurements under $5,000 where a control log (attached), the checklists, and any other appropriate documentation will be placed. This will provide a central location / file of documentation for management or any other party to review for these procurements.

California State Auditor's Assessment of 6-Month Status: Fully Implemented

60-Day Agency Response

The Sixth District now maintains a log documenting procing obtained is fair and reasonable for procurements under $5,000.

California State Auditor's Assessment of 60-Day Status: Partially Implemented

Although the sixth district maintains a log of invoices, it does not have any procedure in place to ensure that staff consistently maintain documentation of their determinations that the pricing obtained is fair and reasonable for procurements under $5,000.

Recommendation #17 To: Administrative Office of the Courts

The AOC should revise the judicial contracting manual to require that judicial entities maintain documentation for their evaluation and selection process used for competitive procurements. The AOC should also strengthen its procedures to ensure that bid evaluations are conducted properly and calculated correctly.

Annual Follow-Up Agency Response From November 2016

The Judicial Council of California (Judicial Council) respectfully declines to implement this recommendation for the reasons provided in our December 2014 response. Further, the Judicial Council is pleased that the State Auditor's more recent reviews have yielded only a minimal number of audit findings.

California State Auditor's Assessment of Annual Follow-Up Status: Will Not Implement

1-Year Agency Response

Public Contract Code (PCC) section 19206 requires the Judicial Council (formerly the AOC) to adopt a judicial contracting manual that is "consistent with" the PCC and "substantially similar" to the State Administrative Manual and the State Contracting Manual (SCM).

The judicial contracting manual is consistent with the requirements of the Public Contract Code (PCC) with respect to the evaluation and selection process used for competitive procurements. Chapter 2, section 2.3 of the judicial contracting manual provides guidance on what should be included in the procurement file. Chapters 4A, 4B, and 4C of the judicial contracting manual set forth procedures for conducting solicitations and evaluating bids. Moreover, evaluation and scoring sheets for competitive procurements remain subject to public inspection after the conclusion of the scoring process consistent with the requirements of California Rules of Court, rule 10.500 ("Public access to judicial administrative records").

The Judicial Council (formerly the AOC) declines to revise the judicial contracting manual to further prescribe how bid evaluations are to be calculated. The Judicial Council is, however, working with and will continue to work with staff involved in the evaluation process for Judicial Council competitive procurements to strengthen its procedures to ensure that all bid evaluations are conducted properly and correctly.

California State Auditor's Assessment of 1-Year Status: Will Not Implement

As we discuss in the report, the Judicial Council's contracting manual states that judicial entities should document the evaluation and selection process for every procurement, but does not require them to do so. We continue to believe that the judicial contracting manual should be revised to require that judicial entities maintain this documentation. Further, although the AOC indicated in its 60-day and six-month responses that it has reminded staff to follow established processes to ensure that bid evaluations are conducted properly and calculated correctly, it has not demonstrated that it has strengthened its procedures.

6-Month Agency Response

Section 4.2 of chapter 4 of the Judicial Branch Contracting Manual specifies the requirements concerning solicitation documentation and stresses that the requirements are favored unless there is a good business reason for variance. Maintaining adequate and accurate records is not valid for a variance. This was stressed in the last appellate clerks meeting and on the monthly procurement calls. The procedures in the JBCM continue to be sufficient to provide direction to judicial branch entities concerning competitive procurements and the retaining of documentation concerning bid evaluations and the selection process. Isolated exceptions upon audit review may occur but they are not the norm and enhancement of the procedures are not deemed necessary.

California State Auditor's Assessment of 6-Month Status: Partially Implemented

Although the AOC indicates in its 60-day and six-month responses that it has reminded staff to follow the procedure, it has not revised the judicial contracting manual to require that judicial entities maintain documentation for their evaluation and selection process used for competitive procurements.

60-Day Agency Response

The AOC has reminded all project managers and selection participants to be more attentive to the mathematical calculations in the future and will review them more closely. The AOC believes that the procedures as discussed in the JBCM are sufficient to provide direction to the selection committees to exercise due care.

California State Auditor's Assessment of 60-Day Status: Partially Implemented

The AOC has not revised the judicial contracting manual to require that judicial entities maintain documentation for their evaluation and selection process used for competitive procurements. Further, although the AOC states that it has reminded all project managers and selection partiicipants to be more attentive to mathematical calculations in the future, the AOC's response indicates that it has not taken any further steps to strengthen its procedures to ensure that bid evaluations are conducted properly and calculated correctly.

Recommendation #18 To: Court of Appeal, Fifth District

The first and fifth districts should implement procedures to ensure that they consistently document their evaluation and selection process for procurements.

6-Month Agency Response

The Court developed and implemented a comprehensive checklist to ensure the documentation of fair and reasonable pricing, evaluation and selection processes, and justifications and approvals for sole-source purchases are addressed going forward. The checklist was forwarded to the email address below on April 2, 2014.

California State Auditor's Assessment of 6-Month Status: Fully Implemented

60-Day Agency Response

The Court has developed and implemented a comprehensive checklist to ensure the documentation of fair and reasonable pricing, evaluation and selection processes, and justifications and approvals for sole-source purchases are addressed going forward.

California State Auditor's Assessment of 60-Day Status: Partially Implemented

Although the fifth district has drafted a checklist for competitive procurements, it is not yet finalized or implemented.

Recommendation #19 To: Court of Appeal, First District

The first and fifth districts should implement procedures to ensure that they consistently document their evaluation and selection process for procurements.

6-Month Agency Response

In addition to the measures described in the courts response of February 14, 2014, the court has finalized a checklist and created a filing system to aid court staff procuring goods and services in consistently documenting their evaluation and selection process. A copy of this checklist has today been e-mailed to Aaron Fellner of your office.

California State Auditor's Assessment of 6-Month Status: Fully Implemented

60-Day Agency Response

All court staff involved in purchasing have been verbally instructed to review and follow the updated Judicial Branch Contracting Manual (JCBM) and the Local Contracting Manual for the Court of Appeal, First District (LCM), which are comprehensive. When there are questions about the manuals requirements, the AOC Business Services office can be contacted for assistance. There will be additional focus on and review of each procurement/contract for compliance with the JBCM and LCM. A draft checklist is in the process of finalization and will mirror the requirements of the JBCM and LCM.

California State Auditor's Assessment of 60-Day Status: Partially Implemented

The first district has indicated that it is developing a checklist that will include steps court staff must take to ensure that they consistently document their evaluation and selection process for procurements. However, the checklist has yet to be finalized and used by court staff to procure goods and services.

Recommendation #20 To: Administrative Office of the Courts

The AOC, HCRC, Supreme Court, and fourth and fifth districts should implement procedures to ensure that required noncompetitive procurement processes, such as preparing justifications and obtaining approval for sole-source procurements, are properly documented. Additionally, the AOC should ensure that it prepares the appropriate documentation when it amends a contract that it has competitively solicited and the amendment includes a change that was not evaluated in the original competitive process.

Annual Follow-Up Agency Response From November 2016

The Judicial Council of California (Judicial Council) believes its current contracting manual is consistent with the Public Contract Code and the State Contracting Manual and stands by its December 2014 response, explaining why no further action is necessary. To the extent that courts are found not to be complying with applicable requirements, the Judicial Council will evaluate the need to retrain court procurement personnel as opposed to revising existing policy.

California State Auditor's Assessment of Annual Follow-Up Status: Will Not Implement

1-Year Agency Response

We follow the Judicial Branch Contracting Manual (JBCM) in all areas and see no need for further procedures. The Judicial Council (formerly the AOC) maintains documentation in its procurement files for sole source and other noncompetitive procurements, including the preparation of appropriate documentation when it amends a contract that it has competitively solicited and the amendment includes a change that was not evaluated in the original competitive process.

A form is used to document non competitive procurements. The Judicial Council has reminded its staff as well as staff at the appellate courts of the importance of maintaining this documentation in procurement files, and will continue to address documentation requirements for procurement files in future training sessions.

California State Auditor's Assessment of 1-Year Status: Partially Implemented

Although the AOC states in its 6-month response that it has conducted training that included this compliance requirement, the AOC has not implemented any further procedures to ensure that required noncompetitive procurement processes, such as preparing justifications and obtaining approval for sole-source procurements, are properly documented. Additionally, the AOC has not taken any steps to ensure that it prepares the appropriate documentation when it amends a contract that it has competitively solicited and the amendment includes a change that was not evaluated in the original competitive process.

6-Month Agency Response

Consistent with the prior response, the AOC believes these are isolated compliance exceptions covered by policies and procedures and they do not require policy and procedure changes. Training on monthly procurement calls have covered this and training scheduled for the judicial branch entities scheduled for October 2014 will again cover it.

California State Auditor's Assessment of 6-Month Status: Partially Implemented

60-Day Agency Response

The AOC believes that these are isolated compliance exceptions and that policies and procedures exist to cover these situations. The AOC has reminded all personnel concerning this compliance requirement and will include the compliance requirement in future training programs and monthly procurement calls.

California State Auditor's Assessment of 60-Day Status: Partially Implemented

Although the AOC states that it has reminded all personnel concerning this compliance requirement, the AOC has not implemented any further procedures to ensure that required noncompetitive procurement processes, such as preparing justifications and obtaining approval for sole-source procurements, are properly documented. Additionally, the AOC has not taken any steps to ensure that it prepares the appropriate documentation when it amends a contract that it has competitively solicited and the amendment includes a change that was not evaluated in the original competitive process.

Recommendation #21 To: Court of Appeal, Fifth District

The AOC, HCRC, Supreme Court, and fourth and fifth districts should implement procedures to ensure that required noncompetitive procurement processes, such as preparing justifications and obtaining approval for sole-source procurements, are properly documented.

60-Day Agency Response

The Court has developed and implemented a comprehensive checklist to ensure the documentation of fair and reasonable pricing, evaluation and selection processes, and justifications and approvals for sole-source purchases are addressed going forward.

California State Auditor's Assessment of 60-Day Status: Fully Implemented

Recommendation #22 To: Court of Appeal, Fourth District

The AOC, HCRC, Supreme Court, and fourth and fifth districts should implement procedures to ensure that required noncompetitive procurement processes, such as preparing justifications and obtaining approval for sole-source procurements, are properly documented.

60-Day Agency Response

By utilizing our checklist, we have ensured that only written authorizations will be accepted for procurements and that justifications include all of the elements of the JBCM for sole-source procurements.

California State Auditor's Assessment of 60-Day Status: Fully Implemented

Recommendation #23 To: Habeas Corpus Resource Center

The AOC, HCRC, Supreme Court, and fourth and fifth districts should implement procedures to ensure that required noncompetitive procurement processes, such as preparing justifications and obtaining approval for sole-source procurements, are properly documented.

60-Day Agency Response

The Habeas Corpus Resource Center (HCRC) implemented the following steps in response to the recommendation by the California State Auditor that the HCRC should implement procedures to ensure that required noncompetitive procurement processes, such as preparing justifications and obtaining approval for sole-source procurements, are properly documented.

One, we have retrained staff on the Judicial Branch Contracting Manual (JBCM) rules regarding competitive and sole-source procurement.

Two, we have provided each staff member responsible for purchasing with a checklist on approval and documentation requirements for non-competitive procurements. The checklist was taken from JBCM Chapter 5 and provided as part of the HCRC response in the California State Auditors report. This checklist, along with others documenting requirements for competitive procurements, is reviewed by staff as part of every procurement.

We believe that these steps fully implement the recommendation made by the California State Auditor.

California State Auditor's Assessment of 60-Day Status: Fully Implemented

Recommendation #24 To: Supreme Court of California

The AOC, HCRC, Supreme Court, and fourth and fifth districts should implement procedures to ensure that required noncompetitive procurement processes, such as preparing justifications and obtaining approval for sole-source procurements, are properly documented.

1-Year Agency Response

The Court has developed and implemented a comprehensive checklist to ensure the documentation of fair and reasonable pricing, evaluation and selection processes, and justifications and approvals for sole-source purchases are addressed going forward.

California State Auditor's Assessment of 1-Year Status: Fully Implemented

6-Month Agency Response

We are working on the changes of our porcedures and will be implemented as soon as possible.

California State Auditor's Assessment of 6-Month Status: Partially Implemented

The department does not appear to have implemented the draft checklist it developed for its 60-day response.

60-Day Agency Response

Have developed and implemented a comprehensive check-list to ensure the documentation of fair and reasonable pricing, evaluation and selection process, and justifications and approvals for sole-source purchases are addressed going forward.

California State Auditor's Assessment of 60-Day Status: Partially Implemented

The Supreme Court has developed a draft checklist to ensure its adheres to procurement requirements. However, the Supreme Court has not yet finalized the document or used it when entering into new contracts.

Recommendation #25 To: Administrative Office of the Courts

The AOC should implement procedures to ensure that its internal controls over payments are followed and that procurements are approved before ordering and receiving goods and services.

Annual Follow-Up Agency Response From November 2016

The Judicial Council of California (Judicial Council) believes that its current contracting manual is consistent with the Public Contract Code and the State Contracting Manual. Although the State Auditor's report from 2013 found 1 case out of 60 were there was a minor and isolated exception for roughly $500, the Judicial Council respectfully disagrees that changes to existing policies are needed and will re-evaluate the need for further training based on the results of subsequent audits by the State Auditor. The Judicial Council is pleased that that the State Auditor's more recent reviews have similarly yielded a minimal number of audit findings.

California State Auditor's Assessment of Annual Follow-Up Status: Will Not Implement

1-Year Agency Response

As cited in the report, the State Auditor tested 60 payments for Judicial Council (formerly the AOC) procurements and found 1 isolated case for approximately $500. The one isolated compliance situation cited was done as an exception that was acknowledged as non-compliant but a expediting deemed necessary at the time and subsequently documented as such. Policy and procedure changes would not change the exception and a formal exception process in policy is not being considered.

The Judicial Council (formerly the AOC)believes that its policies and procedures as documented, including the Judicial Branch Contracting Manual (JBCM), are adequate in this area. Specifically, the JBCM establishes that [P]ayments should not be processed or released by a Judicial Branch Entity (JBE) to a Vendor for any goods or services unless the JBE possesses all of the following: 1. A properly authorized contract; 2. Documentation verifying the goods/services were satisfactorily received and/or performed; and 3. An accurate, properly submitted Vendor invoice.

California State Auditor's Assessment of 1-Year Status: No Action Taken

We continue to believe that the presence of this issue indicates that the AOC's internal controls could be strengthened to ensure procurements are approved before ordering and receiving goods and services.

6-Month Agency Response

As cited in the report, the State Auditor tested 60 payments for AOC procurements and found 1 isolated case for approximately $500. The AOC believes that its policies and procedures as documented, including the Judicial Branch Contracting Manual (JBCM), are adequate in this area and that this compliance issue should not reoccur.

Specifically, the JBCM establishes that [P]ayments should not be processed or released by a Judicial Branch Entity (JBE) to a Vendor for any goods or services unless the JBE possesses all of thee follwoing:

1. A properly authorized contract;

2. Documentation verifying the goods/services were satisfactorily received and/or performed; and

3. An accurate, properly submitted Vendor invlice."

While should not is not considered mandatory, reasonable discretion must prevail according to the JBCM with monitoring for compliance and exceptions. An example would encompass the actual receipt of a good while the documentation may not be as discriptive as possible. The individual has the good in hand and does not need the detailed documentation to validate he/she has it in hand. This is the basis of reasonable discretion and reasoning behind why it is not believed additional procedures are necessary.

California State Auditor's Assessment of 6-Month Status: Will Not Implement

We continue to believe that the presence of this issue indicates that the AOC's internal controls could be strengthened to ensure procurements are approved before ordering and receiving goods and services.

60-Day Agency Response

As cited in the report, the Auditor tested 60 payments for AOC procurements and found 1 isolated case for approximately $500. The AOC believes that its policies and procedures are adequate in this area and that this compliance issue should not reoccur.

California State Auditor's Assessment of 60-Day Status: No Action Taken

The AOC believes that its policies and procedures in this area are adequate and that the issue we found should not reoccur. However, the presence of this issue indicates that the AOC's internal controls could be strengthened to ensure procurements are approved before ordering and receiving goods and services. The AOC has not implemented any procedures to better ensure that its internal controls over payments are followed and that procurements are approved before ordering and receiving goods and services.

Recommendation #26 To: Administrative Office of the Courts

The AOC should implement its plan to review sections of the California Public Contract Code, SAM, and SCM applicable to the judicial branch annually, and more often if there are significant changes, and update the judicial contracting manual as needed. Unless the judicial contracting manual removes the requirement, the AOC should also update its local manual to address construction activities for facilities other than trial courts.

60-Day Agency Response

The AOC has implemented its plan to review sections of the PCC, SAM, and SCM applicable to the judicial branch annually and more often if there are significant applicable changes, so that the Judicial Branch Contracting Manual (JBCM) can be updated as appropriate.

The JBCM was revised to remove the requirement that the AOCs local manual address construction activities for facilities other than trial courts.

California State Auditor's Assessment of 60-Day Status: Fully Implemented

All Recommendations in 2013-302/2013-303

Agency responses received are posted verbatim.