2025-051 The February 2025 Bar Exam

July 9, 2026
2025-051

The Governor of California
President pro Tempore of the Senate
Speaker of the Assembly
State Capitol
Sacramento, CA 95814

Dear Governor and Legislative Leaders:

As required by Business and Professions Code section 6145, subdivision (d), my office conducted an audit of the State Bar of California (State Bar) and its administration of the California Bar Examination administered in February 2025 (February 2025 bar exam). In general, we determined that the State Bar poorly implemented certain changes to the bar exam, which had a negative effect on test takers.

In part to reduce costs and address the solvency of its Admissions Fund, the State Bar decided in 2024 to switch from administering the bar exam in person to offering test takers the option to take the bar exam remotely or in person. To make this switch, the State Bar contracted with Kaplan Exam Services, LLC (Kaplan) to develop multiple-choice questions for the bar exam. However, the State Bar did not specify to Kaplan certain legal topics required for the bar exam, and the State Bar asked another of its vendors, ACS Ventures, LLC (ACS Ventures) to develop additional exam questions, which it did using artificial intelligence. Although the State Bar engaged in a content validation process to review, edit, and validate all the newly developed multiple-choice questions, the State Bar’s execution of this process had flaws that hindered its effectiveness.

The State Bar also contracted with ProctorU Inc., dba Meazure Learning (Meazure Learning), to administer the February 2025 bar exam without first defining business requirements or establishing criteria to evaluate the vendor’s performance. It also contracted with Meazure Learning despite knowing that it lacked an important tool to administer the exam’s performance test portion. Further, the State Bar did not ensure that Meazure Learning addressed the causes of computer freezes, crashes, and error messages that test takers faced during a November 2024 pretest. The technical and administrative problems that test takers experienced caused the State Bar to raise its pass rate from about 36 percent of test takers to about 65 percent of test takers of the February 2025 bar exam.

The State Bar did not achieve cost savings in its administration of the February 2025 bar exam, which will cost at least $5.1 million, not including the costs of pending legal matters, and it experienced at least $4 million in lost revenue from the remedies it provided to February 2025 test takers, which included exam refunds and fee waivers.

Respectfully submitted,

GRANT PARKS
California State Auditor

---

Selected Abbreviations Used in This Report

AI	artificial intelligence
CALS	California Accredited Law Schools
IT	information technology
NCBE	National Conference of Bar Examiners
RFI	Request for information

Summary

Key Findings and Recommendations

One of the key responsibilities of the State Bar of California (State Bar) is the biannual administration of the California Bar Examination (bar exam). Before the COVID-19 pandemic, individuals seeking to practice law in California had to take the bar exam in person. However, after offering the bar exam remotely during the pandemic, the State Bar decided in 2024 to use a hybrid model to administer the exam in February 2025 (February 2025 bar exam). This model allows test takers to take the exam either in person or remotely. By offering the remote option, the State Bar hoped to reduce the cost of administering the bar exam and thus address the impending insolvency of its Admissions Fund, which it uses to support activities, such as examinations, pertaining to admission to the practice of law.

Administered over a two-day period, the bar exam is composed of two sections: a written section with essay questions and a performance test question, and a section with multiple-choice questions. Historically, the State Bar used multiple-choice questions from the National Conference of Bar Examiners (NCBE), but the NCBE does not allow for remote administration of its exam materials. To address this issue, the State Bar retained the services of Kaplan Exam Services, LLC (Kaplan) to develop multiple-choice questions. It also contracted with ProctorU Inc., dba Meazure Learning (Meazure Learning), to provide exam administration services and used its preexisting contractual relationship with ACS Ventures, LLC (ACS Ventures) to provide psychometric consulting services and analysis.^{1 2}

Ultimately, a significant portion of the 4,200 individuals who took the February 2025 bar exam experienced problems. To better understand why these problems occurred, the Legislature and Governor of California amended state law to require our office to perform an audit of the State Bar’s efforts to administer the February 2025 bar exam.

In several instances we found that the State Bar had already taken steps to address concerns with the February 2025 bar exam. However, to improve the quality of its multiple-choice questions, we recommend that the State Bar implement a standardized review process for multiple-choice questions. We also make several other recommendations. For instance, we recommend that the State Bar revise its contracting policies to require managerial approval of any requests for vendor actions outside the scope of contracted responsibilities. Also, to reduce the risk and perception of bias during the question development, validation, and scoring processes, we recommend that the State Bar implement controls that require separation of contractor responsibilities across different stages of exam development, including independent review and approval, and formal oversight of contractors that perform multiple roles.

Agency Comments

The State Bar agreed with all of the report’s recommendations and described efforts it has already begun to implement them. It also indicated that the Supreme Court of California (Supreme Court) is currently considering the State Bar’s recommendation to adopt the NCBE’s NextGen Uniform Bar Exam for implementation in July 2028, which if adopted by the Supreme Court, may reduce the applicability of some of our recommendations.

Introduction

Background

The State Bar is a public corporation established by the California Constitution. Its mission is to protect the public by licensing, regulating, and disciplining attorneys. One of its key responsibilities is the administration of the bar exam. State law requires anyone seeking a license to practice law in California to pass the two-day bar exam. The bar exam’s purpose is to ensure that attorneys who pass possess at least the minimum competency to practice law in the State. Licensure exams like the bar exam distinguish candidates who are at least minimally competent from those who are not competent and could harm the public.

Individuals who take the bar exam may study for months before the exam after having invested a considerable amount of time and money in attending law school. Nevertheless, test takers taking the February 2025 bar exam experienced technical and exam administration problems that could have created barriers to their ability to pass the exam and practice law. In fact, even before the exam began, test takers struggled to book their testing sites because the State Bar did not clearly communicate about testing locations. Media reports indicate that during the exam, the testing platform repeatedly crashed; one test taker stated that he had to start the exam more than 30 times. Media reports also note that test takers experienced screen lagging, error messages, and problems with the copy-and-paste tools. The State Bar acknowledged that myriad problems occurred, and as a result, it offered refunds totaling $2 million for the February 2025 bar exams, and the State Bar Board of Trustees (Board) approved $4.9 million in fee waivers for use on the July 2025, February 2026, or July 2026 bar exams.

Because of the problems with the exam, the Legislature and the Governor of California amended state law to require our office to perform an audit of the State Bar’s administration of the February 2025 bar exam. Specifically, state law requires us to evaluate the State Bar’s bidding and contracting process for the February 2025 bar exam, the process it used to administer the exam, the use of AI to create multiple‑choice questions, and the anticipated and actual costs arising from this bar exam.

The State Bar’s and Supreme Court’s Responsibilities Related to the Bar Exam

As Figure 1 illustrates, two entities governed by the Board—the Committee of Bar Examiners (Committee) and the Office of the Executive Director—were responsible for developing, administering, and scoring the February 2025 bar exam. In 2025, the Office of the Executive Director had a budget of $1.8 million with five full-time positions. The Office of Admissions (admissions office)—under the authority of the executive director—had a budget of $28.8 million and about 68 full-time positions.

Figure 1
The Authority Over the Administration of the February 2025 California Bar Exam

A flow chart that outlines hierarchically the entities that had responsibility over the administration of the February 2025 California bar exam. These entities are:

The Supreme Court of California

The Supreme Court oversees the lower courts and judicial branch administration, including the State Bar of California. The Supreme Court has the power to admit individuals to the practice of law in California. It appoints 5 attorney members to the State Bar Board of Trustees and 10 members to the Committee of Bar Examiners. It sets the passing score for the bar exam and reviews and approves changes to the bar exam.

Lower Courts and Judicial Branch Administration

The State Bar of California

The State Bar Board of Trustees

The Board governs the State Bar, approves the execution of contracts, develops guiding policies, and reviews services of third parties preparing bar exam questions.

The Committee of Bar Examiners

The Committee recommends bar exam rules and guidelines for admissions functions. It also determines the format, scope, topics, content, and grading process for the bar exam.

The Office of the Executive Director

The executive director is charged with ensuring that the State Bar executes its mission.

The Office of Admissions

The Office of Admissions oversees all admissions-related functions to practice law in California, including the bar exam.

The Administration and Examinations Unit

This unit develops, delivers, and grades all admissions-related exams.

The Eligibility and Testing Accommodations Unit

This unit determines applicant eligibility for exams.

Source: State law, California Rules of Court, and State Bar website.

Although the State Bar has numerous duties related to the bar exam, the Supreme Court has the ultimate authority to admit individuals to practice law in California. It also appoints some members of both the Board and the Committee and sets the passing score for the bar exam. Significant changes to the bar exam require Supreme Court approval.

Administration of Bar Exams Before February 2025

The State Bar administers the bar exam twice a year, in February and July.³ Each exam consists of two sections: a written section with essay questions and a performance test question, and a section with multiple-choice questions. The text box describes each of these sections in more detail. The State Bar uses different questions for each administration of the bar exam. It then uses a process called equating to ensure that the multiple-choice score on one exam administration demonstrates equivalent proficiency to another exam administration. The State Bar uses another process called scaling to ensure that any variation in the difficulty of the written section across exam administrations does not affect a test taker’s likelihood of passing.

Before the February 2025 bar exam, the State Bar followed a consistent process when selecting or developing the exam questions it used. Specifically, the admissions principal program analyst explained that law professors draft essay questions and performance test questions. The State Bar retains practicing attorneys to develop and grade essay and performance test questions, and the State Bar presents them to the Committee for approval for inclusion in a bar exam. The State Bar pretests essay and performance test questions to identify problems like ambiguity, bias, and the degree of difficulty.

For the multiple-choice section of the bar exam, the State Bar historically used the questions from the NCBE. A nonprofit corporation, the NCBE develops and licenses the Multistate Bar Examination, the purpose of which is to assess the extent to which a test taker can apply fundamental legal principles and legal reasoning to analyze given fact patterns. The NCBE does not allow for remote administration of exams that use its materials, although it made exceptions in 2020 and 2021 because of the pandemic. During that time, the State Bar generally administered its October 2020, February 2021, and July 2021 bar exams remotely.

For these three exams, the State Bar contracted with ExamSoft Worldwide, Inc. (ExamSoft) to provide exam software and proctoring services. ExamSoft recorded test takers as they took the bar exam, and State Bar staff reviewed the recordings of test takers after the exam to identify possible instances of cheating or other violations of exam rules. The chief of the admissions office noted that the State Bar had to review thousands of recordings after those three exams and found that the technology was not adequate to ensure the integrity of the exam. Therefore, the State Bar was not willing to use the technology again in 2025.

The State Bar’s Changes to the February 2025 Bar Exam

For at least five years prior to the February 2025 bar exam, the State Bar considered making changes to the bar exam. To explore potential changes to the content and format of the bar exam, in 2020, the Supreme Court and the Board established the Blue Ribbon Commission on the Future of the Bar Exam (Blue Ribbon Commission) to make recommendations regarding the bar exam’s content and format. The Board received the Blue Ribbon Commission report in May 2023. The report recommended that California develop its own exam rather than relying on the NCBE. The Blue Ribbon Commission’s arguments in favor of this recommendation included providing California with flexibility related to exam delivery and frequency. The Supreme Court adopted, with modifications, most of the report’s recommendations in October 2024.

While the State Bar was exploring changes to the bar exam, its Admissions Fund developed a deficit. The Admissions Fund supports activities, such as examinations, pertaining to admission to the practice of law. In 2023, the State Bar conducted a fee analysis to address the deficit and proposed substantial increases to most fees applicants pay, as well as fees charged to law schools. Accordingly, the Board voted in September 2023 to increase admissions fees; however, even with those adjustments, the State Bar still projected that the Admissions Fund would be insolvent by the end of 2026. In its 2024 budget, the State Bar anticipated that its Admissions Fund would exhaust its reserves by 2026. In March 2024, the State Bar reported that it was exploring the use of a hybrid bar exam, beginning with the February 2025 bar exam, that would allow applicants the option to take the exam remotely and help the State Bar manage its costs. In October 2024, while the Supreme Court was also considering the Blue Ribbon Commission report, the State Bar petitioned the Supreme Court to allow it to administer the bar exam remotely. The State Bar’s primary motivation for this change was financial, however other considerations were also important drivers, including test-taker preference, access and cost-savings for applicants, as well as improved flexibility for both the State Bar and applicants.

In August 2024, the State Bar contracted with Kaplan to develop multiple-choice questions for the February 2025 bar exam, and in November 2024, it contracted with Meazure Learning to provide exam administration services. In addition, the State Bar had an existing contract with ACS Ventures to provide psychometric consulting services and analysis. Psychometrics is the science of measuring psychological attributes, such as intelligence or understanding, and a psychometrician’s work ensures that tests are reliable and results are valid. The 2023 contract authorized ACS Ventures to provide additional research and advice services as requested by the State Bar, a clause which the State Bar used for ACS Ventures to help develop multiple-choice test questions to supplement the questions that Kaplan provided. The text box lists the contractors that the State Bar engaged to assist with the bar exam. We discuss all three contracts and the State Bar’s processes for developing and administering the February 2025 bar exam in greater detail in the sections that follow. Figure 2 shows similarities and differences between the February 2025 bar exam and prior bar exams, which had to comply with NCBE requirements.

Figure 2
The State Bar Made Changes to the February 2025 Bar Exam, Including in Exam Development, Administration, and Scoring

A graphic that shows differences in how the State Bar developed and administered the bar exam prior to February 2025, as well as similarities between prior bar exams and the February 2025 bar exam.

Differences

Multiple-Choice Questions

Before the February 2025 bar exam: The NCBE developed the multiple-choice questions on the multistate bar examination.

For the February 2025 bar exam: Kaplan and ACS Ventures created the questions and repurposed questions from the first-year law exam. State Bar reviewed the questions.

Content

Before the February 2025 bar exam: NCBE created the content topics and topic proportions for multiple-choice questions.

For the February 2025 bar exam: The State Bar and Kaplan created the content topics. State Bar decided the topic proportions for multiple-choice questions.

Administration

Before the February 2025 bar exam: The State Bar proctored the exam in person with exceptions only during the COVID-19 pandemic.

For the February 2025 bar exam: Meazure Learning proctored the exam in person and remotely.

Scores

Before the February 2025 bar exam: The February 2024 bar exam had a 33.9 percent pass rate (within 1 percentage point of the pass rates of the February 2022 and 2023 bar exams).

For the February 2025 bar exam: The February 2025 bar exam had a 65.3 percent pass rate after adjustments and a retest offered in March 2025.

Similarities

Written Section

The State Bar reviewed essay questions that law professors drafted; State Bar retains practicing attorneys to develop and review performance test questions.

Test Length

Two-day exam consisting of 200 multiple-choice questions, five essay questions, and one 90-minute performance test.

Source: State Bar website, NCBE website, and State Bar documentation on exam administration.

Contracting Requirements That Apply to the State Bar

The State Bar must follow the requirements for public contracting set forth in the State Bar Act and in rules the State Bar adopts. The State Bar Act requires the State Bar to comply with certain standards in the California Public Contract Code, such as competitive bidding requirements, when awarding a contract for more than $100,000 for information technology (IT) goods and services or for more than $50,000 for other goods and services. However, if the purpose of such a contract is for the development or administration of licensing or proficiency examinations, state law exempts it from competitive bidding requirements. The State Bar Act also requires the State Bar to establish a request‑for‑proposals procedure in accordance with those general standards. The State Bar has developed its own processes for awarding contracts for less than $100,000 for IT goods and services and less than $50,000 for other goods and services. The State Bar procurement manual defines these processes and specifies which State Bar contracts must follow them.

Audit Results

• The State Bar’s Timeline and Poor Planning Created Challenges With Question Development
• Question Issues Forced the State Bar to Remove More Questions From Scoring Than It Had Intended
• The State Bar Contracted With Meazure Learning Without Fully Verifying Its Ability to Effectively Administer the February 2025 Bar Exam
• The State Bar Fell Short of Preventing the Technical and Administrative Failures That Occurred in the February 2025 Bar Exam
• The State Bar Experienced No Cost Savings on the February 2025 Bar Exam and at Least $4 Million in Revenue Losses

The State Bar’s Timeline and Poor Planning Created Challenges With Question Development

Key Points

• Despite contracting delays, the State Bar of California (State Bar) committed to using its own questions on the February 2025 Bar Examination (February 2025 bar exam). As a result, it had just six months to develop and validate questions for the multiple‑choice section of the exam.
• Under the terms of its contract with Kaplan Exam Services, LLC (Kaplan), the State Bar had the ability to monitor the vendor’s development of multiple‑choice questions. However, when the State Bar identified a content gap in the questions, the State Bar believed Kaplan did not have adequate time to address it because of the State Bar’s rushed timeline.
• To address the content gap it discovered, the State Bar asked ACS Ventures, LLC (ACS Ventures) to develop multiple-choice questions, allowing the use of artificial intelligence (AI). State Bar staff did not inform the State Bar’s executive director, Committee of Bar Examiners (Committee), or Board of Trustees (Board) of the decision to use AI, and the State Bar did not have provisions in its contract with ACS Ventures to ensure the quality of the questions the vendor provided.

The State Bar’s Contract With Kaplan Did Not Provide Adequate Time for the Development of Effective Multiple-Choice Questions

As we discuss in the Introduction, the State Bar could not offer the bar exam remotely if it continued to use multiple-choice questions from the National Conference of Bar Examiners (NCBE). According to an email from the State Bar’s former director of the Office of Admissions (admissions office) to Kaplan, the State Bar and Kaplan—the vendor with which the State Bar eventually contracted—discussed Kaplan’s ability to provide multiple-choice questions for the February 2025 bar exam as early as December 2023. In January 2024—13 months before the February 2025 bar exam—the State Bar issued a Request for Information (RFI) to identify other vendors that could develop multiple-choice questions modeled after the NCBE’s questions that covered the same legal topics.

The State Bar received Kaplan’s response to the RFI on February 9, 2024—more than a year before the February 2025 bar exam. Kaplan submitted the only valid response to the State Bar’s RFI, as the other interested vendor submitted its response three months after the RFI deadline. By May 2024, the State Bar was ready to present its proposed agreement with Kaplan to the Board. In its proposal, the State Bar included a general timeline showing that Kaplan would deliver questions starting in June 2024. However, two days before the May Board meeting, the NCBE sent Kaplan a letter stating that it sought to ensure that Kaplan would not use the NCBE’s copyrighted materials to create multiple-choice questions for California.⁴ On the day of the Board meeting, Kaplan requested the State Bar to defer the discussion of the proposed agreement because of the communication it received from NCBE, and the State Bar withdrew the item from the May 2024 agenda.

The State Bar and Kaplan continued to negotiate after the May 2024 Board meeting and introduced a shared cost provision into their contract, under which the State Bar would cover 50 percent of any potential litigation costs if the NCBE initiated any claims for copyright infringement. Although Kaplan would have preferred to develop questions for the July 2025 bar exam instead of the February exam to ensure its ability to meet or exceed expectations, it ultimately agreed to produce the questions for the February 2025 bar exam. In its July 2024 meeting, the Board authorized the executive director to execute a contract with Kaplan, which both parties signed on August 9, 2024. Instead of beginning question development in June 2024 as the State Bar planned, Kaplan was directed to deliver its first set of questions before September 2024.

From signing the contract with Kaplan in August 2024 to administering the bar exam in February 2025, the State Bar had just six months to complete the entire process of developing and validating questions. In contrast, the NCBE may take up to three years to develop multiple-choice questions for the bar exam, a process that involves reviewing and editing questions multiple times. Further, the State Bar had just three months, from August 9 to November 1, to determine whether it would purchase the NCBE’s materials as a backup to Kaplan’s questions for the February 2025 exam, as Figure 3 shows. According to a letter the NCBE sent to the State Bar in May 2024, the admissions office had until November 1, 2024, to place its order for NCBE materials. After that date, the State Bar would not be able to revert to a contingency plan of returning to an in-person exam and using the NCBE questions for the February 2025 bar exam. Importantly, although the State Bar planned to pretest its questions in an experimental exam in October 2024, its shortened timeline meant that it could not conduct this experiment until November 8, after the deadline to decide to use NCBE questions had already passed.

Figure 3
The Delay in the State Bar’s Contracting With Kaplan Limited the State Bar’s Question Development Timeline

A timeline displaying an overview of State Bar’s procurement and exam development events, including a delay that halted the procurement process. These events are:

January 2024

The State Bar issued a Request for Information that sought information from question development vendors.

May 16

The State Bar was prepared to seek Board approval to contract with Kaplan but withdraws the agenda item. As a result, there is a two-month delay in contracting with Kaplan.

July 18

The Board authorized the State Bar to execute an agreement with Kaplan.

August 9

The State Bar signed a contract with Kaplan. This begins the three month period in which the State Bar could still revert to NCBE questions as a contingency.

November 1

Deadline to use NCBE questions for February 2025.

November 8-9

The State Bar pretests 49 of its new multiple-choice questions during the November 2024 experimental examination.

January 10, 2025

Multiple-choice questions were finalized for the February 2025 bar exam.

Source: State Bar Board meeting minutes, State Bar Board staff reports, the State Bar’s contract with Kaplan, and the NCBE’s letter to the State Bar.

At the time, the State Bar believed that it would be able to create and validate its new questions in time for the February exam despite the contracting delay. However, the current chief of the admissions office stated that, in hindsight, the timeline for question development was rushed and that the office did not allocate adequate time or resources to ensure the effectiveness of the February 2025 multiple-choice section of the bar exam. After the NCBE November 1 deadline, the State Bar agrees that it was committed to using its own questions on the February 2025 bar exam with no established alternative source for questions should problems arise.

The State Bar’s Contract With Kaplan Did Not Ensure That It Had Questions for All Required Exam Topics

The State Bar incorporated into its contract with Kaplan best practices from its procurement manual, some of which the text box lists. For example, the contract specified when Kaplan was required to deliver batches of questions and the number of questions in those batches. Additionally, the contract stated the State Bar’s intention to review Kaplan’s work product and provide it with prompt feedback, which it required Kaplan to address within 10 days. The chief of the admissions office said that Kaplan successfully delivered the required questions and met its contractual obligations.

However, the State Bar did not specify in its contract with Kaplan the topics within each of the seven subjects that the State Bar needed for the February 2025 bar exam, which resulted in a shortage of questions in certain exam topics. The multiple-choice questions in each bar exam cover seven subjects and specific topics under each subject.⁵ Typically, the NCBE releases content outlines that inform test takers on the subjects, topics within subjects, and the proportions of questions allocated to each topic. While the State Bar and Kaplan could not use the NCBE’s resources due to copyright concerns, it also could not alter the bar exam in a way that would require a substantial modification of the training or preparation needed for passage without providing two years’ notice. According to the admissions office’s principal program analyst, the State Bar had to create its own content outlines with similar topics and proportions so test takers could prepare for the 2025 bar exam in the same manner as they would prepare for the exam if the State Bar used the NCBE’s multiple‑choice questions. The State Bar and Kaplan did not finalize its content outlines until late October 2024, over two months after the contract was signed. The State Bar’s contract required Kaplan to produce an approximately equal number of questions in each of the seven subjects but did not specify topics within the subjects, nor did it specify who would determine the proportions for different topics. The contract also allowed Kaplan to draft the questions using questions from the State Bar’s First-Year Law Students’ Exam (first-year exam) that the State Bar provided to Kaplan.⁶ The admissions office’s principal program analyst stated that the admissions office was not thinking of content proportions when selecting the first-year exam questions Kaplan could use, which the State Bar sent to Kaplan in September and October 2024

The State Bar’s late consideration of topics resulted in a content gap once it realized the first-year exam questions did not cover all the topics needed for the bar exam. The first‑year exam includes questions from three of the seven subjects on the bar exam. Because Kaplan was allowed to use the State Bar’s selected first-year exam questions, Kaplan did not produce questions in the subjects covered by the first-year exam. However, the first‑year exam does not fully cover all the topics within those subjects. As Figure 4 shows, by repurposing first-year exam questions, the State Bar did not ensure that questions were created to cover these topics, essentially creating a content gap in several topics. The admissions office did not identify the content gap until October 2024, when the State Bar began considering the question proportions for the February 2025 bar exam. According to the chief of the admissions office, the State Bar would have been more prepared to meet the content proportion requirements had the admissions office recognized and planned for the missing topics when contracting with Kaplan. Instead, the State Bar did not specify the topics in the contract, resulting in the State Bar needing additional questions to cover the overlooked topics.

Figure 4
The State Bar’s First-Year Exam Questions Did Not Fully Cover All Necessary Bar Exam Topics, Resulting in A Content Gap

A diagram listing the subjects that are tested on the first-year exam, the topics within those subjects, and the proportion of the subject allocated to each topic:

Torts

Intentional Torts, 18%

Negligence, 50%

Products Liability, 8%

Strict Liability, 8%

Other Torts, 8%

Other Considerations, 8%

Criminal Law

General Principles, 6%

Crimes against the Person, 12%

Crimes against Property, 12%

Inchoate Offenses, 6%

Parties to Crime, 7%

Defenses, 7%

Fourth Amendment, 9%

Fifth and Sixth Amendments—Statements, Confessions, and Identifications, 9%

Sixth Amendment—Other Rights of the Accused, 8%

Exclusionary Rule, 8%

Post Trial Rights, 8%

Other Considerations, 8%

Contracts

Introductory Principles/Applicable Law, 5%

Formation, 25%

Defenses to Formation or Enforcement, 13%

Gap-Fillers, Interpretation, and the Parol Evidence Rule, 6%

Performance, Breach, and Excuse, 25%

Remedies, 13%

Nonparties to Contract, 13%

The State Bar’s first-year exam questions only partially covered the topic of Negligence within the subject of Torts. The State Bar’s first-year exam questions did not cover the topics of Fourth Amendment, Fifth and Sixth Amendments-Statements, Confessions, and Identifications, Sixth Amendment-Other Rights of the Accused, Exclusionary Rule, Post Trial Rights, or Other Considerations. The State Bar’s first-year exam questions fully covered the topics within the subject of Contracts.

Source: State Bar February 2025 bar exam content outlines and question documentation.

The admissions office’s principal program analyst said that after the admissions office identified the need for additional questions, it assumed that Kaplan would be unable to produce the necessary questions or adjust the questions already in development because of time constraints. He added that the admissions office reached this conclusion because Kaplan was not obligated to create questions within subjects covered by the first-year exam and was already tasked with creating the February 2025 bar exam’s content outlines, which were outside the scope of their original work. Instead of determining how to extend its question development time frame, the admissions office’s staff asked ACS Ventures—the vendor that provided psychometric consulting services—to draft the additional questions.

By Allowing ACS Ventures to Use AI to Develop Exam Questions, the State Bar Increased the Risk of Performance Issues

The State Bar first discussed the possibility of using AI to produce bar exam questions in May 2024. At that time, the former director of the admissions office, special counsel of the admissions office, and ACS Ventures discussed the possibility of drafting questions using AI as a starting point, likely because Kaplan requested the State Bar defer the discussion of its proposal from the May 2024 Board meeting. However, in August 2024, the State Bar executed its contract with Kaplan to develop multiple‑choice questions for the February 2025 bar exam, seemingly rendering the need to use AI moot.

However, a month later in late September 2024, the State Bar asked ACS Ventures to develop multiple-choice questions after realizing it did not have an adequate number of questions from Kaplan for the November 2024 Experimental Examination (November experiment).⁷ The State Bar planned the November experiment to pretest a sample of its new multiple-choice questions. Its contract required Kaplan to deliver 35 questions to the State Bar no later than September 25, 2024, so the State Bar would have sufficient time to validate their quality before testing those questions during the November experiment. However, two days before the State Bar executed its contract with Kaplan in August 2024, ACS Ventures recommended that the State Bar instead test 49 questions during the November experiment. Although the State Bar accepted ACS Ventures’ recommendation, it did not amend its contract with Kaplan to include the additional questions, resulting in a shortage of 14 questions for the November experiment.

The admissions office’s principal program analyst believed that Kaplan could not have developed more questions for the November experiment because Kaplan was already operating under tight deadlines—Figure 5 shows how the condensed content development time frame limited flexibility to address the content shortage. Consequently, the admissions office asked ACS Ventures to develop the missing 14 questions. ACS Ventures drafted 14 questions for the November experiment using AI. Of those questions, the State Bar reports that it chose to carry over 11 that were well-performing to use on the February 2025 bar exam.

Figure 5
The State Bar’s Condensed Content Development Time Frame Left the State Bar and Kaplan With Limited Flexibility to Address Question Deficiencies

A timeline that outlines the State Bar’s and Kaplan’s question development process from the contracting delay in May 2024 to February 2025. The timeline is split into two categories: events and Kaplan’s responsibilities.

Events

May 16-July 18: For two months, the State Bar deferred seeking Board approval to contract with Kaplan.

August 7: The State Bar realized it needed 14 more questions for the November experiment.

August 9: Kaplan and the State Bar signed a contract.

October 16: The State Bar realized it needed more questions within certain topics.

November 1: The Stater Bar’s deadline to commit to using NCBE questions for the February 2025 bar exam.

Kaplan Responsibilities

September 25: Kaplan was required to produce 35 questions for the November experiment.

September 27: Kaplan was required to produce the Faculty Guide.

October 1-October 30: Kaplan agreed to produce content outlines for the State Bar.

October 15: Kaplan was required to submit 107 questions.

October 15-October 24: Kaplan received 35 questions back from the State Bar and was required to incorporate feedback within 10 days.

November 1: Kaplan was required to produce the Student Guide.

November 4: Kaplan received 65 questions back from the State Bar and was required to incorporate feedback within 10 days. Kaplan only reviewed questions that were Kaplan-developed. The State Bar did not send first-year questions back to Kaplan.

November 5: Kaplan was required to submit 142 questions.

December 1: Kaplan was required to submit 142 questions.

December 14-December 23: Kaplan received 100 questions back from the State Bar and was required to incorporate feedback within 10 days. Kaplan only reviewed questions that were Kaplan-developed. The State Bar did not send first-year questions back to Kaplan.

December 24-January 2: Kaplan received 100 questions back from the State Bar and was required to incorporate feedback within 10 days. Kaplan only reviewed questions that were Kaplan-developed. The State Bar did not send first-year questions back to Kaplan.

The timeline shows how Kaplan had a short period of time to meet many deliverables between September 2024 and January 2025, which was further complicated by the need for more questions and the NCBE’s deadline.

Source: State Bar internal item tracking sheet, emails, Kaplan contract, and State Bar staff.

* Kaplan only reviewed questions that were Kaplan developed. The State Bar did not send first-year exam questions back to Kaplan.

Thus, when the State Bar identified a content shortage in the questions that Kaplan was developing for the February 2025 bar exam, it once again contacted ACS Ventures. Specifically, the former program director in the admissions office sent an email to ACS Ventures asking it to produce the additional questions. As Figure 6 illustrates, the State Bar ultimately included on the February 2025 bar exam 29 questions that ACS Ventures developed using open-source generative AI. We discuss the content and quality of these questions in a subsequent section.

Figure 6
Kaplan Did Not Develop All 200 Multiple-Choice Questions on the February 2025 Bar Exam

A pie chart showing the percentage out of 200 total questions, that ACS Ventures, Kaplan, and the State Bar each provided for the February 2025 bar exam. The figure also shows the count of questions from each source.

Kaplan Exam Services

Kaplan developed 59 percent of the multiple-choice questions, or 117 questions, on the February 2025 bar exam.

State Bar’s First-Year Exam

27 percent of the multiple-choice questions, or 54 questions, were repurposed first-year exam questions.

ACS Ventures (AI-Generated)

ACS Ventures created 15 percent of the multiple-choice questions, or 29 questions, using AI.

The percentages do not add up to 100 due to rounding.

Source: State Bar master bar exam question tracking spreadsheet.

Note: Percentages do not add up to 100 due to rounding.

The State Bar did not amend its contract with ACS Ventures to account for the creation of test questions or establish safeguards before directing the vendor to draft questions for the February 2025 bar exam. Although the State Bar included provisions associated with the use of AI in Kaplan’s contract, as the text box shows, the State Bar’s contract with ACS Ventures did not have similar provisions. It executed its contract with ACS Ventures in December 2023 for psychometric services and analysis related to the bar exam and first-year exam and amended it in November 2024 to increase compensation for services. The contracts did not account for the creation of exam question development but included a provision allowing for ACS Ventures to provide other advice services as the State Bar requested. Prior to amending the contract—in September and October 2024—the State Bar asked ACS Ventures to develop questions for the November experiment and for the February 2025 bar exam. Neither the 2023 contract nor its amendment covered the services provided to the State Bar related to question development or ACS Ventures’ use of AI. The State Bar’s procurement manual and the office of general counsel’s contract procedures do not address how staff should use contract provisions that allow for additional services. This lack of guidance limits oversight and increases the risk that State Bar staff will direct contractors to perform services without clearly defining expectations or controls.

The use of AI to develop questions was not inherently problematic. In fact, in October 2024, the Supreme Court issued an administrative order recommending that the State Bar review the availability of new technologies, such as AI, that could improve the reliability and cost‑effectiveness of testing. This direction indicates that the State Bar was able to pursue innovation in exam development. However, it did not implement this change through a transparent, structured process or clearly communicate its decisions to its former executive director and governing bodies, limiting oversight of a significant change to the examination approach.

The admissions office did not escalate the decision directing ACS Ventures to develop exam questions or the decision to allow the use of AI in that process. According to the admissions office’s principal program analyst, nobody currently at the State Bar can identify who made the decision to ask ACS Ventures to generate multiple-choice questions for use on the November experiment or the February 2025 bar exam. However, a former program director in the admissions office sent an email to ACS Ventures on October 30, 2024, asking it to assist with producing multiple-choice questions for use on the February 2025 bar exam. Although the former director of the admissions office was included on that email, and she and the special counsel of the admissions office had discussed the use of AI with ACS Ventures, neither they nor the State Bar’s staff informed the former executive director, the Committee, or the Board of the decision to use ACS Ventures to develop exam questions or that those questions would be generated using AI. The chief of the admissions office stated that, prior to the February 2025 bar exam, the State Bar did not have a documented framework requiring staff to escalate substantive changes to exam development methodology to executive leadership, including the former executive director and other members of the State Bar’s leadership team, or to governance bodies. The State Bar still lacks a policy that defines substantive changes and requires staff to escalate decisions on them to executive leadership. However, the State Bar is working to develop a framework it says will ultimately address this issue by July 2026.

Ultimately, the State Bar allowed ACS Ventures to perform multiple roles across exam development, validation, and post-exam performance analysis, as Figure 7 illustrates. By assigning these responsibilities to a single vendor, the State Bar created the appearance that ACS Ventures influenced question development, evaluation, and scoring. The State Bar acknowledged that this concentration of responsibilities increased the risk of perceived conflicts of interest in the exam development process.

Figure 7
ACS Ventures Involvement in Multiple Stages of the Development and Assessment of the February 2025 Bar Exam Increased the Risk of Perceived Conflicts of Interest

A timeline from June 2024, when the first content validation panel review session occurred, to the post-exam analysis period in February 2025. The timeline highlights ACS Venture’s multiple roles during content validation, question development, and question performance analysis and scoring.

Content Validation

June 2024: ACS Ventures began facilitating content validation panel reviews, starting with the first-year exam questions the State Bar selected for the bar exam.

Question Development

November Experiment

August 2024: ACS Ventures recommended expanding from 35 to 49 multiple-choice questions on the November experiment.

September 2024: State Bar asked ACS Ventures to draft 14 multiple-choice questions for use on the November experiment.

February 2025 Bar Exam

October 2024: State Bar communicated to ACS Ventures that it had identified a content gap in its multiple-choice question bank and asked ACS Ventures to draft multiple-choice questions to close the gap.

November 2024: ACS Ventures used public AI tools to draft February 2025 bar exam questions.

Question Performance

February 2025: ACS Ventures psychometrically evaluated question performance.

February 2025: State Bar and ACS Ventures identified 29 questions to be removed from scoring based on their perceived accuracy and performance. As proposed to and adopted by the Supreme Court, there were 171 scored multiple-choice questions on the February 2025 bar exam.

Source: State Bar documentation, correspondence records, staff confirmations, and the State Bar’s April 2025 petition to the Supreme Court related to scoring adjustments.

Question Issues Forced the State Bar to Remove More Questions From Scoring Than It Had Intended

Key Points

• The State Bar used content validation panels and subject matter expert reviews to ensure that the multiple-choice questions were appropriate for the February 2025 bar exam. However, its implementation of each of these processes had significant weaknesses.
• Following the February 2025 bar exam, ACS Ventures identified that 40 of the 200 multiple-choice questions on the exam had performance issues. While the State Bar intended to remove 25 questions from scoring, it ultimately did not include 29 questions in the test takers’ scores.

The State Bar Did Not Adequately Ensure the Quality of the Multiple-Choice Questions It Included in the February 2025 Bar Exam

The State Bar primarily used three processes to ensure the quality of the multiple-choice questions it used on the February 2025 bar exam: content validation panel review, subject matter expert review, and pretesting. However, the State Bar’s execution of its content validation process had flaws that significantly hindered its effectiveness. First, when the State Bar recruited its content validation panels, it did not follow industry standards for selecting panelists. Second, the State Bar assigned only one subject matter expert and did not ensure that the subject matter expert reviewed all questions. Third, the State Bar lacked enough time to pretest most of its new multiple-choice questions before the February 2025 bar exam. Consequently, the State Bar either did not identify or did not have time to address all problematic questions before administering the exam.

Content Validation Panels

As Figure 8 shows, the State Bar engaged in a content validation process to review and edit the newly developed multiple-choice questions for administration on the February 2025 bar exam. According to the chief of the admissions office, the State Bar relied on ACS Ventures to create that process to validate the appropriateness of questions for use on the February 2025 bar exam. In the process’s first step, ACS Ventures facilitated multiple content validation panels composed primarily of law school faculty and recently licensed attorneys (legal professionals). The admissions office selected the content validation panelists and under the facilitation of ACS Ventures, these panels used the evaluation criteria the text box presents to review the multiple-choice questions over several nonconsecutive days. The State Bar’s legal professionals reviewed all multiple-choice questions without knowing their source.

Figure 8
The State Bar’s Execution of Its Content Validation Process Had Weaknesses in Multiple Stages

A flow chart listing the steps in the State Bar’s content validation process and, for each step, any weaknesses in the State Bar’s execution of the process. The 200 February 2025 bar exam questions went through the following steps during the content validation process:

1. Content Validation Panel review

The State Bar did not follow industry standards when selecting panelists.

2. Subject Matter Expert review and edits

The State Bar assigned only one subject matter expert and did not ensure that he reviewed all questions.

3. State Bar stylistic review

Admissions office checked questions for grammar and style. We did not identify weaknesses in this step.

4. State Bar approval

The State Bar included potentially legally inaccurate questions on the exam.

Note: The State Bar sent questions that Kaplan developed back to Kaplan, which adjusted the questions based on the feedback and returned the questions to the State Bar, and subsequently sent the questions back to the subject matter expert for another review.

Source: State Bar master bar exam question tracking spreadsheet.

Note: The State Bar sent questions that Kaplan developed back to Kaplan, which adjusted the questions based on the feedback and returned the questions to the State Bar, and subsequently sent the questions back to the subject matter expert for another review.

However, the State Bar did not follow established industry standards for selecting content validation panelists. The American Educational Research Association, the American Psychological Association, and the National Council on Measurement in Education provide criteria for the development and evaluation of educational and psychological testing. These standards state that a validation process that relies in part on the opinions or decisions of experts must establish procedures for selecting such experts, including their qualifications and experience. Consequently, the standards recommend that an organization should describe the basis for identifying certain groups of individuals as appropriate experts and document the procedures it uses to select those experts. Nonetheless, according to the admissions office’s principal program analyst, the State Bar did not document eligibility requirements for its panelists nor its process for recruiting and selecting its panelists.⁸ Instead, the admissions office states that it initially selected panelists by contacting individuals they had previously worked with or knew through professional networks and law school contacts and then later began identifying candidates by pulling reports of licensees who had been licensed within the previous five years.

The State Bar did not establish or document minimum qualification criteria or selection procedures for content validation panelists, contrary to testing standards. As a result, the State Bar could not ensure that panelists were selected based on defined expertise or consistent qualifications. Instead, panel composition reflected an inconsistent and informally derived mix of credentials, as Figure 9 illustrates. The former admissions office’s program director acknowledged that the State Bar did not conduct background checks on panelists based on the assumption that the panelists’ employers would already have conducted background checks. The admissions office’s principal program analyst confirmed that neither the Committee nor the Board oversaw the content validation panel recruitment or selection process conducted by admissions office’s staff. Consequently, the State Bar lacked assurance that its panels collectively possessed the qualifications necessary to support reliable and defensible content validation results. At the end of 2025, the Supreme Court approved policies that establish requirements for the eligibility, recruitment, and selection of content validation panelists and address deficiencies in the State Bar’s vetting process.

Figure 9
The State Bar Selected Content Validation Panelists With an Inconsistent Range of Credentials

A diagram grouping the 17 content validation panelists by their credentials to show how the State Bar recruited inconsistently across different legal professional categories. State Bar staff selected content validation panelists by contacting individuals they had previously worked with or knew through professional networks and law school contacts.

One panelist had an inactive license.

Two panelists were legal experts.

Nine panelists were law school professionals.

Six panelists were recently licensed attorneys.

Three panelists were employed by California Accredited Law Schools.

Five panelists were employed by ABA-Accredited Law Schools.

One panelist was employed by an Unaccredited Law School.

Note: Law schools approved by the American Bar Association (ABA-accredited) are nationally recognized by all bar admitting jurisdictions.

Source: State Bar documentation.

Note: Law schools approved by the American Bar Association (ABA-accredited) are nationally recognized by all bar admitting jurisdictions.

Seeking to improve the State Bar’s content validation panel selection process, the Committee adopted a policy in June 2025 that describes the eligibility, recruitment, and selection of content validation panelists. Specifically, the policy states that panelists must have held a license to practice law in at least one state and be in good standing in any state in which they are licensed. They must provide evidence that they have expertise in the knowledge and skills that will be tested and familiarity with the abilities required of an entry-level attorney. In determining whether panelists are well-qualified, the State Bar evaluates factors such as legal practice experience and subject matter expertise. The policy also directs the State Bar to use a broad solicitation process to recruit panelists and to limit recruitment of faculty to those employed by schools approved by the American Bar Association, California Accredited Law Schools (CALS), or law schools registered with the Committee. The Supreme Court approved this policy in December 2025.

Subject Matter Expert Review

As we describe previously, the purpose of the content validation panel review is to ensure that questions are appropriate—that is, that the questions test the correct topics and are free from bias. The NCBE engages in a similar process, in which committees composed of legal experts jointly determine the questions’ appropriateness. As an additional, independent check for bias and legal accuracy, the NCBE also employs two independent subject matter experts per subject, for a total of 14 subject matter experts who specialize in the subject areas of the questions they review.

In contrast to NCBE’s approach, the State Bar employed just one general subject matter expert to review all 200 of its multiple-choice questions before the February 2025 bar exam. While the State Bar’s content validation panels provided only recommendations, the State Bar’s subject matter expert edited the questions according to the instructions in the text box; the State Bar also encouraged the subject matter expert to identify disagreements he had with the panels’ recommendations. By employing only one subject matter expert, the State Bar limited the depth of that expert’s legal review and placed significant time pressure on him to correct issues before the exam, as he had to review eight batches of questions during the question development period. The admissions office’s program analyst could not explain why the State Bar assigned only one subject matter expert to review questions. To address this weakness, at a meeting in June 2025, the Committee approved a plan to recruit 21 subject matter experts—three subject matter experts for each exam subject—for future exam question validation processes.

The State Bar also allowed 23 exam questions that had not been reviewed by its subject matter expert to appear on the February 2025 bar exam. As Figure 10 shows, the questions were originally among those in the November experiment. The subject matter expert did not review them because the State Bar did not believe it had time for such a review. The questions were the top-performing November experiment questions on the February 2025 bar exam and were used as benchmarks to measure the remaining questions’ performance. According to the chief of the admissions office, the State Bar did not review or edit these 23 questions before placing them on the February 2025 bar exam because it wanted to preserve the questions’ performance data. The State Bar later did not score four of the 23 questions, including two questions that the subject matter expert had identified as being potentially legally inaccurate, despite them being top-performing questions on the November experiment.

Figure 10
The State Bar Did Not Ensure That All February 2025 Bar Exam Questions Were Reviewed by Its Subject Matter Expert

A flow chart showing how the subject matter expert did not review 33 of the questions tested on the November experiment, eventually resulting in 23 February 2025 bar exam questions not receiving subject matter expert review.

Initially, 49 questions were tested on the November experiment. Out of the 49 questions, 33 of the November experiment questions did not receive subject matter expert review, while the remaining 16 did. The 35 top performing questions from the November experiment were selected for the February 2025 bar exam, of which 23 questions did not receive subject matter expert review while the other 12 did.  As a result, 23 out of 200 February 2025 bar exam questions did not receive subject matter expert review.

Source: State Bar master bar exam question tracking spreadsheet.

The State Bar knew that, because of time constraints, the subject matter expert was unable to correct all identified problems and errors before it finalized the 200 questions that appeared on the February 2025 bar exam. Twelve days before the February 2025 bar exam, the State Bar tasked its subject matter expert with completing a final review of all 200 exam questions. The subject matter expert determined that 60 questions needed minor wording and grammar corrections and that six additional questions had potential legal inaccuracies. However, the State Bar did not incorporate these corrections or revisions because, according to the admissions office’s principal program analyst, it had already printed exam materials and uploaded the questions to the ProctorU Inc., dba Meazure Learning (Meazure Learning) platform. In its post‑exam psychometric analysis, the State Bar chose to remove the six potentially legally flawed questions from scoring.

Pretesting

The State Bar did not pretest all of the questions for the February 2025 bar exam, contrary to NCBE’s standard practice. According to the NCBE, pretesting is the process of presenting test takers with questions on the bar exam that will not be scored to ensure their effectiveness before they are used as scored questions on a future exam. Pretesting questions allows the NCBE to determine whether those questions demonstrate acceptable statistical characteristics; for example, a question must show evidence that test takers who answered it correctly also tended to obtain high scores on the entire exam. The NCBE’s standard practice is to pretest all multiple‑choice questions. Typically, the NCBE pretests its questions by including 25 of them as unscored questions on its bar exams, alongside 175 scored questions that have already been pretested.

Because the State Bar was developing a completely new exam, it did not have the ability to employ the NCBE’s method for pretesting questions before including them on the February 2025 bar exam. However, the admissions office’s program manager stated that during a Committee meeting stakeholders expressed their desire for the State Bar to pretest questions before they appeared on the February 2025 bar exam. As we described previously, the State Bar tested 49 of its exam questions by including them in the November experiment. The admissions office’s principal program analyst confirmed that the State Bar then used the 35 best‑performing November experiment questions on the February 2025 exam. Ultimately, the State Bar did not pretest the remaining 165 questions before including them on the bar exam. As a result, the State Bar did not have assurance that most of its questions had acceptable statistical characteristics before including them in the exam.

On the February 2025 Bar Exam, 40 of the 200 Multiple-Choice Questions Had Performance Issues and Six Raised Legal Accuracy Concerns

Because it was using new questions, the State Bar relied on post-exam statistical analysis to determine which of the February 2025 bar exam’s multiple-choice questions would count toward test takers’ scores. The State Bar intended to adopt a similar approach to the NCBE and score only 175 of the 200 questions. Unlike the NCBE’s practice of pretesting all exam questions before they are scored on the bar exam, the State Bar had not tested most of the 200 questions on the February 2025 bar exam. As such, the State Bar did not identify the questions it would remove from scoring until after ACS Ventures conducted psychometric analysis on all the questions.

While the content validation process was a qualitative review of the questions’ legal accuracy before the exam, post-exam analysis was quantitative and determined how the questions performed using statistical measurements. ACS Ventures analyzed variables like question difficulty and response options to assess statistical reliability and flagged questions outside target performance ranges.⁹ Figure 11 provides examples of the types of issues that the analysis flagged. According to ACS Ventures’ post-exam analysis, 40 of the 200 multiple-choice questions on the February 2025 bar exam had statistical performance issues.

Figure 11
Post-Exam Analysis Revealed That the February 2025 Bar Exam Questions Had Statistical Performance Issues

A diagram showing the percentage and number of questions on the February 2025 bar exam that had statistical performance issues.

11 percent, or 23 questions, of the 200 questions had low or negative discrimination. Low discrimination indicates that test takers who performed well on the exam overall performed similarly on the question as those who did not perform well overall. Negative discrimination indicates that applicants who performed well on the exam overall did not perform as well on the question as applicants who scored lower on the exam overall.

Six percent, or 12 questions, of the 200 questions had invalid options. Invalid Options indicates that test takers overall chose only one or two answers for a question, meaning the other answer choices were faulty.

Four percent, or 9 questions, of the 200 questions were too easy.

Seven percent, or 14 questions, of the 200 questions were too difficult. Note: The State Bar identified 18 questions that had multiple performance issues.

Source: The State Bar’s multiple-choice question scoring data.

Note: The State Bar identified 18 questions that had multiple performance issues.

The State Bar retained questions with performance issues for scoring when they did not negatively affect exam reliability and were necessary to maintain adequate subject matter coverage. The State Bar ultimately scored—counted for the purposes of determining test takers’ test scores—19 of the 40 questions that ACS Ventures flagged for performance issues. Maintaining subject matter coverage is essential to bar exam design because the exam must assess core legal concepts to measure a test taker’s ability to apply fundamental legal principles and reasoning across each of the tested subjects and the topics within those subjects. In a 2025 petition to the Supreme Court, the State Bar indicated that including some flagged questions would not materially affect overall examination reliability and could inform revisions for future exam administrations. ACS Ventures’ liaison to the State Bar (ACS Ventures liaison) explained that this course of action maximized content coverage and reliability of the multiple-choice exam.

After ACS Ventures selected 25 questions to remove from scoring, the State Bar removed four more questions than it intended because of concerns about their legal accuracy. Keeping in mind the need for subject matter coverage, ACS Ventures initially selected 175 questions to include in test takers’ scores. Like the NCBE, the State Bar intended to score 175 questions and leave the remaining 25 unscored. However, the State Bar determined it needed to remove more than 25 questions from scoring. After the State Bar informed ACS Ventures that its subject matter expert had identified six questions with potential legal accuracy issues, ACS Ventures reviewed those questions’ performance. During its review, ACS Ventures found that it had already removed two of the potentially legally flawed questions from scoring; it then chose to remove the additional four potentially legally flawed questions, resulting in a total of 171 scored questions. According to the ACS Ventures liaison, there were no well-performing questions from the initial 25 unscored questions that could replace the four potentially legally flawed questions—resulting in 29 unscored questions. As Table 1 shows, questions generated using AI were disproportionately more likely to exhibit performance issues and be removed from scoring. Forty-five percent of questions generated by ACS Ventures were flagged for performance issues and 21 percent were removed from scoring. According to ACS Ventures’ analysis, questions generated using AI also produced difficulty scores that indicated they were comparatively easier for test takers.

Post-scoring analysis revealed the potential effects of the State Bar’s flawed implementation of its content validation process, as we identified only limited correlation between the content validation process and whether the questions were removed from scoring. Although the content validation process served multiple purposes, one of its goals was to ensure questions were appropriate to test the minimal competency for an entry-level lawyer. Because this was one of its goals, the content validation process should have identified nearly all questions with issues, such as being too difficult for minimally competent entry-level lawyers. Instead, Figure 12 shows that the content validation panels identified only 59 percent of the 29 unscored questions as having issues. The correlation between the subject matter expert’s review and whether the questions were scored was even weaker. As Figure 12 indicates, the subject matter expert approved all unscored questions after making revisions. Further, the subject matter expert’s final review identified performance issues in only 45 percent of the questions that were eventually removed from scoring for performance issues.

Figure 12
The Content Validation Process Did Not Identify Issues in Questions Later Removed From Scoring

A flow chart displaying the percentage and number of questions removed from scoring after the exam that each step of the content validation process identified as having issues before the exam.

Content Validation Panel

The content validation panel determined that 59 percent, or 17, of the questions had issues. 41 percent, or 12, of the questions were OK.

Subject Matter Expert First Review

During his first review, the subject matter expert determined that 100 percent, or 25, of the questions were OK after his revisions.

Note: The subject matter expert’s first review was out of 25 questions, since the subject matter expert did not review all questions.

State Bar

The State Bar admissions office signed off on 100 percent, or 29, of the questions. The State Bar finalized and uploaded all the questions.

Subject Matter Expert Final Review

During his final review, the subject matter expert determined that 24 percent, or 7 questions, needed minor revisions; 21 percent, or 6 questions, were potentially legally inaccurate; and 55 percent, or 16, of the questions were OK.

Source: State Bar master bar exam question tracking spreadsheet and question performance sheet.

* The subject matter expert’s first review was out of 25 questions, since the subject matter expert did not review all questions.

The State Bar has begun strengthening its oversight of bar exam development, but its actions have not yet resolved all weaknesses we identified in its processes. For example, as Figure 13 shows, the State Bar implemented a generative AI framework and a staff user guide to evaluate and deploy generative AI systems in November 2025. The State Bar’s generative AI framework requires Board approval for the use of AI in the development or drafting of bar exam questions. However, the State Bar has not yet implemented a policy requiring staff to defer decisions on substantive changes to executive leadership or appropriate governing bodies. It expects to address this gap by July 2026. Until then, there is a continued risk that the executive director, chief of admissions, and other senior leaders could again be uninformed about substantive changes during bar exam development. We expected the State Bar to establish a policy requiring staff to defer decisions on substantive changes to executive leadership or appropriate governing bodies before implementation.

Figure 13
State Bar Reforms Addressing Oversight, Governance, and AI Use After the February 2025 Bar Exam

A graphic that outlines reforms that the State Bar has implemented to address oversight, governance and AI use after the February 2025 bar exam and when the reforms began.

AI Reforms

1. February 2025: Established the Office of Operational and Digital Transformation (ODT).

2. November 2025: Created an AI governance framework and generative AI user guide for State Bar staff.

Governance Reforms

3. April 2025: Restructured the office of admissions to separate exam administration and exam development functions.

4. May 2025: Received Committee approval to create the subcommittee on exam administration and subcommittee on exam development.

5. June 2025: Expanded the Committee’s role in exam administration and exam development.

Oversight Reforms

6. February 2025: Created the chief of admissions role to ensure more focused oversight of and engagement with the admissions office.

7. April 2025: Created a director of exam development position to report directly to the chief of admissions.

Source: State Bar documentation.

The State Bar Contracted With Meazure Learning Without Fully Verifying Its Ability to Effectively Administer the February 2025 Bar Exam

Key Points

• The State Bar did not clearly define its exam administration requirements before soliciting vendors for this purpose. Instead, it relied on Meazure Learning’s assurances that it could successfully administer the February 2025 bar exam.
• The State Bar’s data showed that the performance test—which is separate from the exam’s multiple-choice questions—had been the most difficult portion of the bar exam to administer remotely during the COVID-19 pandemic (pandemic). Nonetheless, the State Bar did not act promptly to address Meazure Learning’s known shortcomings related to administering the performance test portion of the exam.
• The State Bar did not communicate to test takers the in-person testing locations until a month before the exam. Because some test takers could not access a California testing location within 50 miles of their ZIP Code, the State Bar had to rush to add additional testing locations, and test takers had to make last‑minute travel plans.

The State Bar Did Not Clearly Define Its Requirements Before Selecting Meazure Learning to Administer the February 2025 Bar Exam

Contracts for the development and administration of licensing or proficiency examinations are exempt from the general requirement of obtaining bids or proposals from multiple vendors. Despite this exemption, the State Bar reported to the Committee and Supreme Court that it evaluated other vendors before selecting Meazure Learning to administer the February 2025 bar exam. As a result, we expected the State Bar to have documented its assessment of each vendor’s abilities against a set of defined business requirements related to the exam’s administration. Such a process is critical to ensuring that a vendor can meet an agency’s needs and that its proposal represents the best value for the State. Specifically, when using a competitive bidding exemption, the State Bar procurement manual requires contract managers to provide a best value memo that describes the process of determining that the selected vendor offers the best value to the State Bar. We discuss this topic in more detail later in the report.

Nonetheless, the State Bar did not establish evaluation criteria or identify business requirements when assessing potential vendors for the administration of the bar exam. According to the chief of the admissions office, who did not work in the admissions office when the State Bar evaluated the vendors who submitted proposals, the document that most closely resembled a formal listing of the State Bar’s business requirements was a term sheet the State Bar developed in the fall of 2024. This term sheet summarized the key contract terms that the State Bar was negotiating with Meazure Learning in October 2024 and later finalized in November 2024. However, it did not identify the State Bar’s business requirements or define the criteria the State Bar used to evaluate potential vendors.

According to the State Bar, it selected Meazure Learning because the other three vendors with which it communicated were not viable options. Figure 14 presents the State Bar’s reasons for concluding that the three other vendors could not meet its requirements. For example, State Bar officials and one vendor—which we refer to as Vendor 1—exchanged emails in October 2023 in which the vendor indicated that it used a dual-camera proctoring model that would have required test takers to use both a webcam and a mobile camera. The exam development program director told us that this proctoring approach was technically complex and that having phones in the exam room was an exam security risk.

Figure 14
The State Bar Determined Three Vendors Could Not Administer the February 2025 Bar Exam

A graphic that shows three anonymized vendors that the State Bar determined could not administer the February 2025 bar exam due to factors like exam security, cost, and capacity.

Vendor 1 could not meet the State Bar’s requirements for exam security, capacity and number of in-person testing centers. Additionally, the vendor’s dual-camera proctoring model would have required test takers to use both a webcam and a mobile camera, which would have been too technically complex and an exam security risk. The State Bar also had concerns about the vendor’s ability to administer an exam of the necessary scale. The vendor also only offered remote testing and did not operate in-person test centers.

Vendor 2 did not meet the State Bar’s exam security needs because the vendor required a longer exam testing window that would have created a risk of question exposure. The State Bar also reported that the vendor did not have the capacity to administer the bar exam and was more expensive than Meazure Learning.

Vendor 3 withdrew its proposal in June 2024 because it no longer had the capacity to meet the State Bar’s needs.

Note: The State Bar did not establish documented business requirements or evaluation criteria for evaluating the proposals it received related to the administration of the February 2025 bar exam.

Source: State Bar staff, email correspondence, and staff reports.

Note: The State Bar did not establish documented business requirements or evaluation criteria for evaluating the proposals it received related to the administration of the February 2025 bar exam.

Although the State Bar did not identify its business requirements or establish a formal assessment process for potential vendors, it did conduct an initial review of Meazure Learning’s online platform and test centers before contracting with the vendor. Specifically, in August 2024, State Bar staff and representatives from the Committee and Supreme Court toured some of Meazure Learning’s testing centers and tested the user experience of its online platform. At that time, some participants reported problems with the online platform and test centers. For example, one participant could not use the copy-and-paste function and was disconnected from the platform when she tried that action. These problems were similar to those that test takers taking the February 2025 bar exam ultimately experienced.

Rather than ensuring that Meazure Learning addressed these problems, the State Bar relied heavily on Meazure Learning’s assurances that it could administer the bar exam effectively.¹⁰ Based on such assurances, the State Bar reported to the Committee and Supreme Court that Meazure Learning had corrected those problems. Further, during a September 2024 Committee meeting, Meazure Learning presented information about its online platform and highlighted its ability to successfully administer exams for organizations like the Association of Medical Colleges and Law School Admissions Council. It asserted that it had administered 25,000 exams over a two-day period at the same time with no issues, and that it had thousands of proctors available to meet the bar exam’s scaling needs. According to the State Bar, these assurances, combined with Meazure Learning’s reputation as a large and established vendor, led it to conclude that Meazure Learning could deliver the exam at the required scale.

The exam development program director explained that to some extent, staff must rely on a vendor’s assurances regarding its software, servers, and technical capabilities. However, he also noted the importance of including contractual terms to ensure that those assurances are realized. If the State Bar had clearly identified its business requirements, it could have taken steps to better ensure that its contract with Meazure Learning fully identified benchmarks Meazure Learning needed to meet and dates by which Meazure Learning needed to demonstrate specific capabilities. As a best practice, the Statewide Information Management Manual recommends that agencies include requirements in contracts that identify project deliverables and milestones required for project completion that are specific, unambiguous, testable, and traceable. For example, the State Bar’s contract with Meazure Learning did not include performance benchmarks or metrics that the State Bar could have used to assess Meazure Learning’s administration of the November experiment. In the absence of identified business requirements, we believe that the State Bar did not recognize the necessity of establishing such contract terms.

In the aftermath of the February 2025 bar exam, the Supreme Court, Board, Committee, and State Bar took steps to strengthen the process for procuring vendors to administer the bar exam. Between May 2025 and September 2025, the State Bar and the Supreme Court made amendments to the California Rules of Court related to administering the bar exam, among other areas. Those changes clarified that the Committee is required to develop standards, subject to Board approval, for assessing a vendor’s ability to administer or proctor the bar exam in any format. In January 2026, the Committee and Board approved business requirements and a rubric for the State Bar to assess vendors’ ability to administer the bar exam using the factors in the text box. By defining its business requirements and establishing a formal vendor-evaluation process, the State Bar has substantially addressed our concerns about how it selected Meazure Learning.

The State Bar Contracted With Meazure Learning Despite Knowing It Lacked an Important Tool to Administer the Performance Test Portion of the Exam

The performance test portion of the bar exam requires test takers to complete a task related to a fictitious case using two separate sets of materials: the file and the library. The text box describes these materials. Administering this portion of the exam remotely presents challenges. Prior to the February 2025 bar exam, test takers would receive paper packets of these materials and scratch paper, and then could complete the performance test task using a laptop.¹¹ However, during the pandemic-era remote exams, 58 to 72 percent of survey respondents reported that it was difficult to view the legal files electronically when they were completing the performance test. The admissions office’s operations program manager confirmed that the performance test was the most difficult portion of the pandemic-era bar exams to administer remotely.

For the February 2025 bar exam, the State Bar and Meazure Learning would provide the performance test materials to test takers on a laptop rather than on paper.¹² The State Bar’s contract with Meazure Learning stated that test takers must be able to use tools such as highlight, strikethrough, copy and paste, and note writing. The contract further stated that Meazure Learning’s platform was fully capable of delivering the needed tools, with the exception of the highlighting function, and that Meazure Learning would identify the external resources that could provide highlighting. In late February 2025, four days before the bar exam, the State Bar and Meazure Learning amended their contract to stipulate that Meazure Learning had contracted with a third-party vendor—Hypothesis—to provide an annotation tool that Meazure Learning could integrate into its platform and that would allow test takers to annotate, highlight, copy and paste, and use the search function for about 40 pages of documents provided by the State Bar.

Despite this amendment, many critical annotation functions worked poorly or not at all during the February 2025 bar exam. In the post-exam survey, more than two-thirds of the nearly 2,700 respondents reported that the copy-and-paste function did not work, and about half reported that annotation tools, such as highlight and strikethrough, did not work.¹³ These functions are important because, for example, many test takers wrote parts of their answers to the performance test in the notepad on their performance test file and library PDF, but were unable to copy and paste those notes to the official response field. Additionally, about 400 survey respondents reported that they could not even open the performance test file and library materials.

The exam development program director asserted that when the State Bar initiated research on external annotation tools in November 2024, it realized there were unexpectedly few options that had the functionalities it would need for the performance test portion of the exam. He added that Meazure Learning subsequently did not begin implementing the Hypothesis tool until January 16, a day before the mock exams took place. Then, during the week before the February 2025 bar exam, the State Bar worked with Hypothesis to test the functionality of the tool. Although it found minor issues with text selection and lag, it deemed those issues acceptable for the bar exam, and Hypothesis shared that it had insufficient time for its engineers to make changes before the exam. The State Bar believes that the problems test takers experienced with the annotation tools were rooted in the Meazure Learning platform rather than in the Hypothesis tool because Meazure Learning had to customize the platform’s functionality in order for individuals to copy text from an external resource and paste it into the exam platform.

However, the State Bar did not provide documentary support for its belief, nor did it know whether further platform development Meazure Learning performed in the days before the bar exam could have caused the problems test takers experienced in the performance test portion of the exam. Nonetheless, the State Bar knew that test takers from pandemic-era bar exams struggled with this portion of the exam and that the nature of this portion of the exam was important and stressful. As a result, the State Bar should have immediately created key deliverables and deadlines for Meazure Learning to identify a supplemental tool, integrate the tool into its testing platform, and test that tool. By starting this process at the end of November 2024, the State Bar gave itself and Meazure Learning a shortened timeline to search for, contract with, integrate, and test an external PDF annotation tool.

The State Bar Did Not Communicate Clearly to Test Takers Trying to Schedule In-Person Bar Exams

The State Bar did not communicate to test takers the in-person testing locations that would be available for the February 2025 bar exam until a month before the exam. In the past, test takers were able to choose their February testing sites as soon as the application period opened in October, leaving test takers several months to arrange travel and accommodations. However, when the registration period opened on October 1, 2024, for the February 2025 bar exam, the test takers did not know where they would be able to take the test. Instead, from November 2024 to the beginning of January 2025, a February 2025 bar exam Frequently Asked Questions document available on the State Bar website indicated simply that Meazure Learning had testing locations throughout California and the United States and provided a link to Meazure Learning’s website, which listed test site states and cities. The document also stated that all test takers had to test on Pacific Standard Time regardless of whether they were taking it remotely or in-person. Without specific information, test takers may have assumed that they could take the exam at one of Meazure Learning’s testing locations. However, on January 17, 2025, the State Bar sent test takers an email stating that only test takers in the Pacific and Mountain time zones would be able to take the bar exam in person, and that all other test takers would need to take the test remotely. Although the email provided step-by-step instructions on how test takers could change the format in which they were taking the exam from in person to remotely, test takers had just three days to make such a change before the deadline on January 20, 2025—all without specifying which in-person Meazure Learning testing locations were available for the February 2025 bar exam.

The State Bar’s January 17 email also provided additional testing site locations that it had not previously communicated but had known about for months. It explained that test takers who lived near one of four large testing sites—located in Chula Vista, Ontario, San Francisco, and Roseville—would be able to select only that site. However, the State Bar had known since late November 2024 that the four large testing sites would be in those regions of the State because they were listed in the Meazure Learning contract. Additionally, that contract specified that a total of 1,350 test takers would take the exam in-person, that around 600 in-person spots would be available at small Meazure Learning test centers, and that an additional 750 in-person test takers would have to take the exam in one of four large testing sites. Informing test takers of these limitations before January could have helped them plan for their travel and accommodation rather than requiring them to make last-minute preparations.

Notwithstanding the State Bar’s delayed communications about where in-person test takers could take the bar exam, test takers were able to book their testing location starting on January 30, 2025—just under a month before the exam. However, problems further hampered test takers’ ability to schedule their bar exam. To track these problems, the State Bar maintained a spreadsheet that showed 73 reported scheduling issues, such as test takers not receiving notice to schedule their exam or test takers receiving a test cancellation without requesting one. The State Bar worked with Meazure Learning to resolve the issues. Additionally, test takers reported not having test centers close to their locations. One test taker reported that they found no testing sites within 100 miles of where they live. Another test taker reported that the closest testing locations available were the four large test centers, which they said were three to seven hours away by car. The State Bar reported at the February 3, 2025 joint meeting of the Board Executive Committee and Contracts Committee that 14 percent of in-state in-person registrants without testing accommodations could not access a testing location in California that was within a 50-mile radius of their ZIP Code.

In the weeks before the exam, the State Bar took steps to mitigate these scheduling problems. First, it responded to the emails it received from test takers, and it worked with Meazure Learning to schedule their test sites. Second, to reduce the number of test takers having to travel more than 50 miles to an exam site, the State Bar added three in-person testing locations two weeks before the exam. It also offered applicants a full refund if they chose to withdraw from the exam. Additionally, the State Bar offered reimbursements for certain travel expenses to test takers who booked nonrefundable lodging but later switched test sites or were reassigned by Meazure Learning.

The State Bar Fell Short of Preventing the Technical and Administrative Failures That Occurred in the February 2025 Bar Exam

Key Points

• The State Bar did not ensure that Meazure Learning adequately addressed the causes of technical problems—such as freezes, crashes, and error messages—that participants faced during the November experiment. As a result, many of the test takers who took the February 2025 bar exam reported experiencing similar problems.
• Following the November experiment, many participants reported that Meazure Learning’s proctors misunderstood or inconsistently applied exam rules. Despite these complaints, the State Bar did not ensure that Meazure Learning adequately improved proctor training before the February 2025 bar exam. Consequently, test takers encountered similar proctoring problems during the bar exam.
• Because of the technical and administrative problems that test takers experienced, the State Bar, in consultation with ACS Ventures, implemented several scoring adjustments for the February 2025 bar exam.

Many Test Takers Reported Technical Problems That Impeded Their Completion of the Bar Exam

The State Bar was aware of potential technical problems related to Meazure Learning’s exam administration as early as November 2024. The State Bar shared with our office roughly 200 emails that it had received that described technical or other problems, such as freezes and error messages, that test takers had experienced during the November experiment, which had about 4,000 participants. In addition to detailing technical problems that persisted during the November experiment, at least two emails specifically stated that the test takers had verified prior to the exam that their computers had met technical requirement specifications but still encountered technical issues. The State Bar also received a letter signed by 11 deans of the California Accredited Law Schools (CALS) association reporting concerns about technical problems, such as crashes, that the experiment participants had experienced. The CALS letter stated that Meazure Learning servers appeared to have bandwidth problems.

The former director of the admissions office forwarded the CALS letter to Meazure Learning and asked that it provide responses to the assertions in the letter. In its reply to that request, Meazure Learning stated that it believed the root causes were related to participants’ preparedness and computer requirements. The exam development program director explained that the State Bar had the ability to view November experiment participants’ individual user session data, which, on an individual user basis, seemed to confirm Meazure Learning’s assertions that the technical problems were caused by participants’ computer system specifications.

As a result, Meazure Learning updated the computer requirements for the February 2025 bar exam to include higher RAM capacity, more CPU cores, and greater network bandwidth. In alignment with Meazure Learning’s changes to test taker computer requirements and to better ensure test taker preparedness, the State Bar informed test takers of the updated technology requirements in its countdown emails.

The State Bar and Meazure Learning also required remote test takers to participate in a mock exam to assess whether their devices met the system requirements and were compatible with the Meazure Learning exam platform. The State Bar held this mock exam in January 2025. Remote test takers who took the mock exam would be allowed to take the February 2025 bar exam remotely; remote test takers who did not take the mock exam would be withdrawn from the bar exam and would not receive a refund. Completing this mock exam would ensure that remote test takers’ devices met the updated minimum system requirements before they took the February 2025 bar exam. For the February 2025 bar exam, roughly 99 percent of remote test takers completed the mock exam.¹⁴

Nonetheless, following the February 2025 bar exam, a significant portion of the 4,200 test takers reported technical issues. About half of the nearly 2,700 post-exam survey respondents reported computer crashes and freezes, and nearly 1,200 respondents reported error messages during the multiple-choice portion of the exam. The text box shows that a significant percentage of survey respondents reported that they experienced similar technical problems during the exam as were reported after the November experiment, which we described earlier. According to an email among State Bar leadership after the February 2025 bar exam, Meazure Learning had asserted that its server provider, Amazon Web Services, had a server crash, which contributed to widespread problems during the multiple-choice portion of the bar exam. The State Bar attempted to verify Meazure Learning’s assertions by reviewing the Amazon Web Services official service health web page in March 2025 and did not find reports of an outage during the February 2025 bar exam. Further, the State Bar indicated that Meazure Learning has provided no evidence of an Amazon Web Services outage and that the matter is currently a subject of ongoing litigation.

According to the exam development program director, the State Bar did not test Meazure Learning’s servers and platform following the November experiment because it relied on provisions in its contract with Meazure Learning related to its technical capabilities. In fact, the contract stated that Meazure Learning understood and agreed that the administration of the State Bar exams required uninterrupted availability of Meazure Learning’s platform to all test takers on the scheduled exam dates and times for both remote and in-person exams. Moreover, the contract stated that Meazure Learning would maintain proactive monitoring, regular performance testing, and documented and tested system backup capabilities to ensure that its platform availability would be 99.982 percent at all times while it was administering the exams. However, these contractual provisions did not prevent the widespread crashes, freezes, and error messages test takers reported experiencing during the February 2025 bar exam. The State Bar’s procurement manual indicates that the State Bar should monitor vendor performance by requiring the vendor to propose and implement plans to cure unsatisfactory performance when contract goals are not met. While the State Bar did implement changes that increased user system requirements, the widespread problems leading up to and during the February 2025 bar exam indicate that additional testing of Meazure Learning’s platform may have been warranted.

The State Bar Did Not Ensure That Meazure Learning Resolved Proctoring Problems First Identified During the November Experiment

Problems with Meazure Learning’s exam proctors also appeared during the November experiment. Exam proctors play a key role in bar exam administration—they monitor test takers and enforce exam rules and policies. For bar exams conducted before February 2025, the State Bar oversaw the hiring and training of proctors; however, for the November experiment and February 2025 bar exam, Meazure Learning agreed to provide proctors whom it would train on exam security, the State Bar’s examination rules and protocols related to exam conduct, prohibited and allowed items, and basic customer service. The State Bar’s contract with Meazure Learning includes provisions with respect to proctor training; specifying that proctors would be trained in exam security and State Bar examination rules and protocols, and that all Meazure Learning personnel would be qualified to perform services with the highest industry standards for similar services.

Despite these provisions, November experiment participants and the CALS letter reported problems with Meazure Learning proctors, as the text box explains. Again, the State Bar shared the CALS letter with Meazure Learning and asked it to respond to these concerns.

Meazure Learning responded to the concerns by stating that it believed the root cause was lack of test taker preparedness, as test takers had only two weeks to schedule exams, familiarize themselves with the new testing environment, and ensure their computers were up to Meazure Learning requirements. It asserted that these problems increased the amount of support needed, which led to a degradation in participants’ experience and a slower response rate than Meazure Learning had expected. Meazure Learning stated that frequent and advanced test taker communication and a mock exam, which would require test takers to download software and connect to a proctor, would address the problems in time for the February 2025 exam.

However, test takers still reported complaints about their proctors for the mock exam in January 2025. The State Bar maintained a complaint tracker that showed concerning complaints from mock exam test takers regarding proctors. One test taker said they notified their proctor about the technical problems they experienced, but the proctor could not answer questions related to technical problems because, according to the test taker, the proctor responded that they did not know anything about the exam. Another test taker reported that their proctor told the test taker, who had signed up for a remote test modality, that they were not allowed to take the test remotely in their home office and then signed off. The State Bar shared with Meazure Learning a list of questions about Meazure Learning’s proctors and mock exam participants’ complaints, including the examples we mentioned above. This communication resulted in a meeting between the State Bar and Meazure Learning, in which, according to State Bar staff attendees’ notes, Meazure Learning went through specific complaints and indicated that the proctor job description would be reiterated to proctors. Additionally, the State Bar gave Meazure Learning a guide for in-person proctors but not until February 5, 2025—less than a month before the February 2025 bar exam. Moreover, that guide was specific to proctors for in-person test centers. After the mock exam took place and the State Bar discussed proctor concerns with Meazure Learning, the State Bar had very limited options and time to ensure adequate proctors, since it was less than a month before the February 2025 bar exam. Ultimately, the State Bar announced in February 2025 that test takers had the opportunity to withdraw from the exam with a full refund or request fee waivers for future bar exams, which we discuss later in the report.

Despite the State Bar’s efforts to resolve problems with proctors through the mock exam—which Meazure Learning stated would address the problems—proctor problems persisted for the February 2025 bar exam. Around 1,000 of the 2,500 survey respondents who took the February 2025 bar exam reported significant problems related to proctoring. Specifically, nearly 1,100 test takers reported that proctors appeared to lack training on the bar exam and almost 900 test takers indicated their proctor had provided unclear or confusing instructions. Many test takers also reported inconsistent enforcement of permitted items policies or being transferred among multiple proctors during the exam, while others reported the exam started up to 90 minutes later than expected. These problems with proctors created unnecessary challenges for test takers, whose ability to pass the bar exam can impact tentative job offers, future earnings, and employment stability.

The State Bar Adjusted Test Takers’ Scores to Remediate for Technological and Administrative Problems During the Bar Exam

To remediate the disruptions test takers experienced during the February 2025 bar exam, the State Bar implemented scoring adjustments. The State Bar petitioned for, and the Supreme Court approved, two scoring adjustments: setting a low raw minimum pass score needed to pass the bar exam, and replacing test takers’ blank multiple-choice and written response questions—which could have happened because of technical and administrative problems—with projected scores through a process called imputation.¹⁵ The text box provides more information on the score imputations made to the February 2025 bar exam. Figure 15 shows the effects these scoring adjustments had on the final pass rate in more detail.

Figure 15
The State Bar’s Four Scoring Adjustments Raised the February 2025 Bar Exam Passing Rate By Almost 30 Percent

A graphic that details the scoring adjustments that the State Bar made for the February 2025 bar exam.

Initial Pass Rate

The psychometrician estimate of the pass rate before remediation was 36 percent.

Adjustment 1, setting the raw minimum pass score and Adjustment 2, score imputations for missing or blank responses combined to cause a 22 percent increase in the pass rate.

Adjustment 3, taking the higher of the two essay read scores caused a 5.4 percent increase in the pass rate.

Adjustment 4, imputing low performance test scores if essay scores were higher than performance test scores caused a 1.9 percent increase in the pass rate.

Final Pass Rate

The pass rate after remediation was 65.3 percent.

Source: State Bar Report to the Supreme Court on the February 2025 California Bar Exam and ACS Ventures psychometric documentation.

After the State Bar released test takers’ scores, the Committee recommended two more scoring adjustments, also due to the technical and administrative issues test takers experienced. First, the Committee decided to change a grading policy for test takers. Graders typically read written answers for essay questions and the performance test once and, depending on their score, test takers are eligible for a second reading by a different set of graders. The test taker’s score is then the average of the two sets of scores for the question, referred to as read scores. Instead, for the February 2025 bar exam, the State Bar used the higher of the two read scores for test takers’ written answers to determine whether a question received a passing score. Following the State Bar’s confirmation that the scoring adjustment fell within the Committee’s authority over grading policy decisions not requiring Supreme Court approval, the Committee approved this scoring adjustment, which resulted in an additional 227 test takers receiving passing scores on the exam overall. Second, the Committee agreed to impute test takers’ performance test scores with the test takers’ essay scores if the essay score was higher than the awarded performance test score. This change was approved by the Supreme Court and resulted in an additional 80 test takers receiving passing scores.

Ultimately, the Supreme Court approved the remediation strategies that were subject to its approval. These strategies culminated in a final pass rate of 65.3 percent, which is almost double the pass rate of the previous four February bar exams. Figure 15 shows the impact of the different remediation efforts on the February 2025 bar exam’s final pass rate.

The State Bar Experienced No Cost Savings on the February 2025 Bar Exam and at Least $4 Million in Revenue Losses

Key Points

• The State Bar spent an average of $7.7 million annually in direct exam costs to administer two bar exams per year from 2022 to 2024. In contrast, the February 2025 bar exam alone has cost it $5.1 million, excluding pending legal matters, and the State Bar experienced a loss of revenue of at least $4 million due to its remediation efforts for test takers as a result of the problems they experienced with that exam.
• The State Bar has taken steps to address concerns regarding the condition of its Admissions Fund, such as increasing certain fees in late 2025. Moreover, the State Bar is in the process of working with its Board, the Committee, and the Supreme Court to assess and identify the future of the bar exam to help ensure that the State Bar avoids problems experienced during the February 2025 bar exam.

The State Bar’s Final Costs for the February 2025 Bar Exam Far Exceeded the Costs of Previous Bar Exams

As we explain in the Introduction, each year the State Bar offers two sessions of the bar exam, the first in February and the second in July. From 2022 to 2024, the State Bar spent an average of $7.7 million annually on direct costs for both bar exam sessions, as Figure 16 shows. The State Bar’s accounting records do not identify a singular explanation for the annual cost differences. Some direct costs, such as those the State Bar incurred to rent rooms and hire exam proctors, decreased when the State Bar offered the bar exams in a remote format in 2021. However, when the State Bar offered the bar exams in person again in 2022, these costs significantly increased. Other factors also likely contributed to rising costs, such as inflation and changes in the number of test takers sitting for the bar exam. For example, in February 2022, a total of 3,451 test takers sat for the bar exam, whereas 4,125 test takers sat for the bar exam in February 2023.

Figure 16
Annual Bar Exams Costs Fluctuated From Fiscal Years 2020 Through 2025 as the State Bar Shifted From Hybrid Exams and In-Person Exams

A bar graph showing the annual cost of administering the bar exam from 2020 through 2025.

In 2020 and 2025, one of the two bar exam administrations occurred in person, and the other exam was a hybrid administration. In 2021, both bar exams were hybrid. And in 2022 through 2024, both exams were administered in person.

Annual Bar Exam-Related Costs:
2020: $5.3 million
2021: $5.3 million
2022: $8.2 million
2023: $7.8 million
2024: $7.1 million
2025: $8.6 million

The three-year average of 2022 through 2024 costs: $7.7 million

Note: The State Bar fiscal year runs from January through December. Amounts above do not include indirect costs. The 2025 costs do not include the $1.4 million in withheld payments to Meazure Learning.

Source: The State Bar financial statements and independent auditor’s reports for fiscal years 2020 through 2025, and State Bar accounting records.

Note: The State Bar fiscal year runs from January through December. Amounts above do not include indirect costs. The 2025 costs do not include the $1.4 million in withheld payments to Meazure Learning.

The State Bar’s rationale for remotely administering the February 2025 bar exam included its expectation that the change would lower its costs. Nonetheless, the State Bar did not develop the first cost estimate for this approach until its contracts were nearly finalized. Specifically, in May 2024, the State Bar estimated costs for four services—exam administration, question development, other development costs, and grading—for the February 2025 and July 2025 bar exams, based on different models that compared in-person and hybrid test administrations. At that time, the State Bar concluded that offering the two exams using a hybrid model would cost about $4.4 million, compared to $8.4 million to offer them only in person. Although these estimates suggest that offering a hybrid model would be significantly less expensive, we are concerned that the State Bar did not develop the estimates until it was close to entering contracts with both Meazure Learning and Kaplan, indicating that its decision to modify the bar exam was not based on documented cost estimates.

Further, the State Bar’s estimates did not reflect contingency costs related to administering the February 2025 bar exam.¹⁶ The February 2025 bar exam has cost the State Bar $5.1 million, as Figure 17 shows. A portion of this amount relates to additional costs the State Bar incurred as the result of problems with the exam. Specifically, the State Bar has not paid $1.4 million to Meazure Learning (withheld payments). The figure also shows that the State Bar may incur up to an additional $5.2 million in remedies it offered to test takers, which we discuss later. The State Bar’s costs also include $500,000 for legal representation—excluding an amount related to pending legal matters, such as the State Bar’s ongoing litigation against Meazure Learning.¹⁷

Figure 17
The State Bar May Spend Up to $5.1 Million on the February 2025 Bar Exam and Remedies May Cost the State Bar up to $5.2 Million in Lost Revenue

Two pie charts showing the State Bar’s February 2025 exam expenditures in the first, and lost revenue in the second.

Exam Expenditures include:

Exam Administration: $1.1 million

New and Existing Contracts: $2.7 million, including $93,000 in startup costs.

Legal Representation, excluding ongoing litigation costs: $500,000

Withheld Payments: $1.4 million

Total: $5.7 million (rounded)

Lost Revenue includes:

Refunds provided to February 2025 Test Takers: $2 million

Value of Used Fee Waivers: $2.1 million

Value of Unused Fee Waivers: $1.2 million. As of May 2025, unused fee waivers will no longer be valid after the July 2026 bar exam.

Total: $5.2 million (rounded)

Note: Discrepancies in total amounts are due to rounding.

Source: State Bar accounting records, invoices, and staff.

Note: Discrepancies in total amounts are due to rounding.

* As of May 2025, unused fee waivers will no longer be valid after the July 2026 bar exam.

^† Includes about $93,000 in startup costs.

The remediation that the State Bar offered to test takers took two main forms. First, on February 13, approximately two weeks before the February 2025 bar exam date, the State Bar sent a message to test takers apologizing for the problems that were occurring. In that message, the State Bar offered applicants the opportunity to receive a full refund if they chose to withdraw from the exam. These refunds amounted to roughly $2 million in lost revenue.¹⁸

Next, the State Bar sent a message to test takers on February 21, 2025, four days before the exam, with a statement from the Board further apologizing for the February 2025 bar exam rollout and offering test takers who failed or withdrew from the February 2025 bar exam the option to apply for a fee waiver to take the July 2025 bar exam. In May 2025, the Board agreed to extend those waivers so that test takers had the option to take the February 2026 bar exam or the July 2026 bar exam. As of March 2026, about 1,000 of the nearly 3,000 test takers to whom the State Bar granted fee waivers had not used them, including about 200 test takers who signed up to take the July 2026 bar exam. The value of fee waivers that test takers have used so far is about $2 million.

Additionally, test takers who were unable to launch the test in Meazure Learning’s platform, submitted fewer than four out of six written responses, or submitted fewer than 100 of the 200 multiple-choice questions were eligible to take a makeup exam. The State Bar chose these thresholds based on the recommendation from ACS Ventures that it could impute or replace scores with statistical reliability if test takers had answered at least four written responses or 113 of the 171 scored multiple‑choice questions. The State Bar also offered these test takers travel reimbursements so they could take the exam in person at the State Bar offices in San Francisco or Los Angeles. Ultimately, 78 test takers participated in either a full or partial makeup exam.

Effective October 1, 2025, the Supreme Court approved revisions to rules that govern the bar exam to include a requirement that the State Bar conduct a cost-benefit analysis of relevant direct and indirect costs for changes to the bar exam that substantially modify either the method of exam administration or the training or preparation required to pass the exam. This change to the rules would help ensure timely and more complete assessments of the cost of substantial changes to exam administration.

The State Bar Continues to Seek Ways to Reduce Solvency Risks in Its Admissions Fund

The State Bar is taking steps to address concerns regarding the condition of its Admissions Fund because of the costs associated with the February 2025 bar exam, including lost revenue and expenses. To address these concerns, the State Bar reviewed admissions fees in late 2025. Through that analysis the State Bar proposed and the Board approved at its November 2025 meeting an increase to certain fees, such as one-time fees for out-of-state lawyers to represent their client in California. The State Bar anticipates these increases will result in the Admissions Fund having a fund balance of $2.2 million to $2.5 million by 2028.

The State Bar is continuing to work with its Board, the Committee, and the Supreme Court to determine the future of the bar exam in California. In April 2026, the State Bar presented a risk-benefit analysis of bar exam options to the Committee, which adopted the analysis and recommended presenting it to the Board. The analysis indicated that the lower-risk, lower-cost, and more operationally efficient option would be for the State Bar to adopt the NCBE’s NextGen Uniform Bar Exam. Although the State Bar must take additional steps to implement such a change, acting now to assess and identify the future of the bar exam will help ensure that the State Bar avoids problems leading up to and during the February 2025 bar exam.

Other Areas We Reviewed

In alignment with state law, we determined whether the State Bar appropriately adhered to its existing laws, regulations, and policies when contracting with Meazure Learning. In addition, to fulfill our statutory obligation, we determined whether the State Bar appropriately evaluated the potential for and protected itself from any potential conflicts of interest that may have existed between relevant State Bar employees and Meazure Learning.

The State Bar Did Not Adhere to Certain of Its Own Procurement Policies When Contracting With Meazure Learning

To administer a hybrid bar exam and contract with Meazure Learning, the State Bar obtained Board, Committee, and Supreme Court approvals. State law requires Board approval for contracts for IT goods and services that exceed $100,000. The Rules of Court that were in effect until October 2025 indicated that the Committee had the responsibility for determining the format of the bar exam, subject to the Supreme Court’s review and approval. Figure 18 is a timeline that shows that the State Bar received these approvals less than six months before the February 2025 bar exam.

Figure 18
The State Bar Did Not Finalize Its Agreements With Meazure Learning Until Four Months Before the February 2025 Bar Exam

Figure 18 is a timeline of events leading up to the State Bar finalizing its statement of work with Meazure Learning, beginning in September 2024 and ending in November 2024.

September 2024, six months until February 2025

September 9: The State Bar submitted a petition to the Supreme Court requesting to modify the bar exam.

September 18: The Supreme Court denied the State Bar’s petition to modify the bar exam without prejudice to a future petition seeking modifications that have been considered and approved by the Committee.

September 19: The Board approved the Meazure Learning contract amounts of $4.1 million to administer the February 2025 bar exam and $151,500 to administer the November experiment, subject to any action that the Committee deemed necessary.

September 30: The Committee approved Meazure Learning as the vendor to administer the February 2025 bar exam.

October 2024, five months until February 2025

October 4: The State Bar submitted to the Supreme Court a revised petition to modify the bar exam.

October 22: The Supreme Court approved the State Bar’s petition to modify the bar exam.

November 2024, four months until February 2025

November 7: The State Bar finalized its master services agreement with Meazure Learning.

November 8: The State Bar finalized the statement of work for Meazure Learning to administer the November experiment on November 8 and 9, 2024.

November 27: The State Bar finalized the statement of work for Meazure Learning to administer the bar exam.

Source: State Bar petitions, Supreme Court notices, Board and Committee meeting minutes, State Bar staff reports and agreements with Meazure Learning.

The State Bar did not obtain necessary approvals for changes to the administration of the bar exam until October 22, 2024, and it consequently did not execute its contract with Meazure Learning until November 2024. Initially, the State Bar interpreted the Rules of Court as not requiring it to obtain the Committee’s approval for changes to the bar exam’s mode of administration; however, the Supreme Court denied the State Bar’s proposed changes in September 2024 and directed it to obtain Committee approval before resubmitting another petition. Despite expressing concerns, Committee members approved the State Bar’s proposed changes on September 30, 2024. The Supreme Court subsequently approved the State Bar’s renewed petition in October 2024, enabling the State Bar to finalize the master services agreement and three statements of work with Meazure Learning in November 2024. Table 2 describes each of these agreements.

Because it obtained these approvals so close to the November experiment, the State Bar expedited the procurement process with Meazure Learning and failed to comply with two requirements from its procurement manual. First, the procurement manual provides an exception from competitive bidding for contracts related to licensing or proficiency testing examinations. However, the manual requires that when using an exception, contract managers must provide a best value memo. This best value memo must justify in writing that the selected vendor provides the best value to the State Bar, generally by describing the vendors the State Bar evaluated, the evaluation criteria it used, and the reasoning for selecting the vendor in question. For the Meazure Learning agreements, State Bar staff submitted to the procurement office four memos in which they requested an exemption from competitive bidding and provided their conclusion that Meazure Learning offered the best value.¹⁹ However, State Bar staff based the content of these memos on an outdated vendor analysis from 2021 that was not specific to the administration of the bar exam—rather, the State Bar conducted this vendor analysis to justify its selection of a vendor that would provide an item-banking solution for the Legal Specialist Exam.

According to the IT program analyst who submitted two of the best value memos, the former director of admissions instructed him to use the 2021 vendor analysis because it remained valid for Meazure Learning and covered key areas such as security, functional capabilities, and general vendor information. However, by approving the contracts associated with these best value memos, the State Bar did not sufficiently evaluate whether Meazure Learning provided the best value to the State Bar for administering the February 2025 bar exam. Despite this deficiency, the State Bar had a limited selection of vendors to choose from that made Meazure Learning the only viable option, as we discussed earlier in the report. Having only one viable vendor makes the best value memo inconsequential as a means for the State Bar to demonstrate that Meazure Learning provided the best value.

Second, the IT Office director did not approve the master services agreement with Meazure Learning in the financial system. The State Bar’s procurement manual requires the procurement file to be routed to various offices at the State Bar for their approval, depending on the nature and amount of the services the contract manager is procuring. The finance director stated that IT Office approval is necessary to check that the requisition request makes technical sense, fits within the State Bar’s overall architecture, and meets security standards. Although he did not approve the master services agreement in the financial system, the chief information officer who oversees the IT Office reviewed and approved specific sections of it pertaining to privacy and security requirements. The chief information officer explained that the IT Office had limited involvement during the contracting process because the Meazure Learning platform did not require integration with any of the State Bar’s existing information systems. He added that the IT Office could have helped to provide a more comprehensive evaluation of the platform by asking questions about the system build, functionality, customizability, and scalability. Nevertheless, in its contracts with the State Bar, Meazure Learning agreed to meet the State Bar’s functionality, customizability, and scalability needs.

The State Bar’s Conflict of Interest Policies and Procedures Align With State Law and Regulations

The State Bar Act requires designated State Bar employees and members of the Board and committees to disqualify themselves from making, participating in the making of, or attempting to influence any decisions of the State Bar when they have a financial interest that may be materially affected by the decision. Similarly, Government Code section 1090 prohibits a public officer or employee, including State Bar employees and Board members, from participating in making government contracts in which the officer or employee has a financial interest. The Political Reform Act of 1974 requires each state and local agency to adopt a conflict of interest code that identifies all officials and employees in the agency who make governmental decisions and requires them to disclose certain financial interests. To help identify potential conflicts of interest, state law requires public officials and employees in designated positions to report their financial interests in a Statement of Economic Interests on an annual basis and when assuming or leaving office. In accordance with these requirements, the State Bar adopted a conflict of interest code mandating that designated employees and members of the Committee and Board disclose their financial interests.²⁰

To determine whether the State Bar had established appropriate protections from and evaluations of potential conflicts of interest with Meazure Learning, we reviewed the Statements of Economic Interests of each member of the Board, Committee, and designated State Bar employees involved in the procurement and contracting process with Meazure Learning. Of the Statements of Economic Interest we reviewed, we did not identify any disclosures of financial interest with Meazure Learning. In addition, we reviewed the State Bar’s agreement with the outside counsel it hired to facilitate contract negotiations with Meazure Learning and the master services agreement between the State Bar and Meazure Learning. We determined that these agreements include evaluations of and protections from potential conflicts with the State Bar. For example, the master services agreement required Meazure Learning to ensure that its personnel complied with all applicable federal, state, and local laws and regulations pertaining to conflicts of interest and to notify the State Bar immediately if it became aware of any facts that might create, or might reasonably be expected to create, a conflict of interest. The State Bar also confirmed that it did not identify any potential or actual conflicts of interest between its staff, Board, or Committee members and Meazure Learning. Finally, we reviewed the State Bar’s conflict of interest policies and guidance and found they align with state law and guidance outlined in the State Contracting Manual.

Recommendations

The State Bar has implemented changes, as we describe throughout this report, that address some of the findings we identified. For example, the Committee adopted a policy in June 2025 that describes the eligibility, recruitment, and selection of content validation panelists. The Committee also approved a plan to recruit more subject matter experts for future exam questions validation purposes. In another example, the Committee and Board approved business requirements and a rubric for the State Bar to assess a vendor’s ability to administer the bar exam. This change was the result of the Supreme Court amending the California Rules of Court related to administering the bar exam. Another revision to those rules requires the State Bar to conduct a cost-benefit analysis of relevant costs for changes to the bar exam under certain circumstances. Nonetheless, we identified additional areas that need improvement, and we make recommendations to the State Bar to further improve its ability to successfully administer the bar exam in the future.

To ensure that multiple-choice questions that the State Bar develops adequately cover all bar exam topics, the State Bar should, by December 2026, implement a policy that requires the admissions office to finalize content needs and proportions before it starts developing questions.

To improve the quality of its multiple-choice questions, the State Bar should, by December 2026, implement a standardized review process for multiple-choice questions, regardless of the source of the questions, that includes a timeline that ensures recommendations or edits to questions occur and are reviewed before the exam is administered.

To ensure the State Bar’s executive leadership is aware of and can make well-informed decisions about how best to utilize contracts related to its bar exam, the State Bar should, by December 2026, revise its contracting policies to require managerial approval of any request for vendor services that do not align with the scope of services and responsibilities under the vendor contract.

To reduce the risk and perception of bias related to question development, validation, and scoring if the State Bar develops its own bar exam, the State Bar should, by December 2026, implement controls that require separation of contractor responsibilities across exam development stages, including independent review and approval, and formal oversight when a contractor performs multiple roles.

To improve oversight of bar exam development, the State Bar should, by January 2027, implement a policy that defines substantive changes to exam development and requires staff to document and escalate decisions on those substantive changes to executive leadership and appropriate governance bodies for review and approval.

To improve test takers’ ability to successfully complete exams, the State Bar should, by December 2026, implement a policy to require the testing of all new or changed components of its bar exam, such as the implementation of new annotation tools or changes to its rules regarding the items test takers may use during the exam. The State Bar should require the completion of this testing early enough to allow adequate time to resolve any issues or problems it identifies.

---

We conducted this performance audit in accordance with generally accepted government auditing standards and under the authority vested in the California State Auditor by Government Code section 8543 et seq. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on the audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Respectfully submitted,

GRANT PARKS
California State Auditor

July 9, 2026

Staff:
Vance Cable, Audit Principal
Joshua Hooper, CFE, Senior Auditor
Ixtzel Duran
Kaleb Knoblauch
Kelsey Meyer
Prabhleen Reen
Savanna Rowe

Legal Counsel:
Natalie Moore

Appendices

Appendix A

Sources of February 2025 Bar Exam Questions, Issues, and Removals From Scoring, by Subject

Table A maps the number of February 2025 multiple-choice questions by subject, source, and outcome in scoring, based on the State Bar’s question tracking data, performance metrics, and post-exam item analysis. After the February 2025 bar exam, the State Bar’s psychometrician, ACS Ventures, evaluated all 200 multiple‑choice questions to assess reliability and content coverage across subjects. ACS Ventures flagged questions with poor statistical performance, including those with item difficulty values outside target ranges, weak discrimination values, or nonfunctioning response options. The State Bar retained questions with performance issues when they did not negatively affect exam reliability and were necessary to maintain adequate subject matter coverage. The State Bar removed questions flagged for performance issues or potential legal inaccuracies from scoring when those issues affected exam reliability. For example, of the 200 questions, 17 had a negative discrimination value and were removed from the scored items for adversely affecting examination reliability. Certain questions removed from scoring had not been flagged for statistical performance issues because they were either removed for potential legal inaccuracies or part of ACS Ventures’ process to maintain content proportions. Removing poorly performing questions from scoring improved the reliability of the multiple-choice exam.

---

Appendix B

Scope and Methodology

We conducted this audit pursuant to the audit requirements contained in the Business and Professions Code section 6145(d). Specifically, state law required our office to conduct a performance audit of the February 2025 bar exam to evaluate the administration of the February 2025 bar exam and how the problems with the exam occurred. Table B lists the objectives and the methods we used to address them. Unless otherwise stated in the table or elsewhere in the report, statements and conclusions about items selected for review should not be projected to the population.

Assessment of Data Reliability

The U.S. Government Accountability Office, whose standards we are statutorily obligated to follow, requires us to assess the sufficiency and appropriateness of computer‑processed information we use to support our findings, conclusions, or recommendations. In performing this audit, we relied on accounting records and invoices that we requested from the State Bar to document expenditures related to the February 2025 bar exam. To assess the reliability of these data, we reviewed existing information about the data, interviewed people knowledgeable about the data, and compared the totals of the accounting records and invoices to audited financial statements. Additionally, we relied on the State Bar’s master tracking sheet to describe the number and characteristics of questions used on the February 2025 bar exam. To assess the reliability of these data, we reviewed existing data and interviewed State Bar staff knowledgeable about the master tracking sheet. In both instances, as a result of our assessment, we found the State Bar’s data to be of undetermined reliability. Although this determination may affect the precision of the numbers we present, there is sufficient evidence in total to support our findings, conclusions, and recommendations.

Response

State Bar of California

SENT VIA EMAIL

June 18, 2026

Grant Parks, California State Auditor
California State Auditor
621 Capitol Mall, Suite 1200
Sacramento, CA 95814

RE: State Bar of California Response to 2025-051 Audit Report

Dear Mr. Parks:

The State Bar of California is in receipt of the final draft report in response to the audit required by California Business and Professions Code section 6145, subdivision (d) relating to the administration of the February 2025 bar examination. We agree with all of the report’s recommendations and have made significant strides to implement several of them already.

The State Bar appreciates the efforts, professionalism, and dedication of your team in conducting the audit and helping us identify areas for further improvement. The State Bar acknowledges the significant and unacceptable experiences that test takers faced in the lead up to and during the February 2025 bar exam administration. We have engaged in extensive efforts to identify lessons learned and to improve processes to prevent such issues from occurring again. The State Bar has also initiated a legal action against the State Bar’s vendor, ProctorU, Inc. d/b/a Meazure Learning (Meazure Learning) to hold it accountable for its failures and false statements to the State Bar. Having your team conduct an analysis of the issues that contributed to the problematic administration of the February 2025 bar exam and provide recommendations provides a valuable perspective to support the State Bar’s efforts.

① While we agree with the recommendations, we disagree with certain findings in the report. The State Bar is in active litigation with Meazure Learning as of the date of this response, the State Bar will not address its concerns with those findings here. But we do note, as acknowledged in a footnote in the report, that your office did not access Meazure Learning’s internal documents or interview any of its employees. 

Once again, we extend our appreciation for your team’s efforts throughout this process. The State Bar remains dedicated to ensuring that the California bar exam is a reliable, secure assessment of minimum competence for the practice of law, and we strive to eliminate barriers that prevent test takers from demonstrating that competence.

State Bar Responses to Audit Recommendations

The report notes that the State Bar has presented a risk-benefit analysis to the California Supreme Court, with a recommendation to adopt the NextGen Uniform Bar Exam (NextGen UBE). To the extent any of your recommendations would become moot if the Supreme Court were to order adoption of the NextGen UBE, we note that in our response below.

1. Recommendation: To ensure that multiple-choice questions that the State Bar develops adequately cover all bar exam topics, the State Bar should, by December 2026, implement a policy that requires the admissions office to finalize content needs and proportions before it starts developing questions.
   • Response: The State Bar agrees with this recommendation. The State Bar is in the process of developing a comprehensive exam development framework, which was discussed with the Committee of Bar Examiners’ Subcommittee on Exam Development on May 13, 2026, and the Committee of Bar Examiners on June 12, 2026. This framework, which will be finalized in the coming months, will set out processes for conducting content validation and subject matter expert review. The framework is designed to serve as a roadmap for how processes are to be implemented and issues are to be escalated. It is intended to ensure consistency and assist staff by setting forth a decision framework. This statement will be incorporated into that framework. We note, however, that the Supreme Court is currently considering a petition from the State Bar recommending the adoption of the NextGen UBE for implementation in July 2028. If the Supreme Court directs adoption of the NextGen UBE without a California component, the State Bar will no longer be developing multiple-choice questions for the bar exam. If the Supreme Court directs the creation of a California component, or a new California bar exam, exam content may or may not be tested through multiple-choice questions.
2. Recommendation: To improve the quality of its multiple-choice questions, the State Bar should, by December 2026, implement a standardized review process for multiple-choice questions, regardless of the source of the questions, that includes a timeline that ensures recommendations or edits to questions occur and are reviewed before the exam is administered.
   • Response: The State Bar agrees with this recommendation. The State Bar is in the process of developing a comprehensive exam development framework, which was discussed with the Committee of Bar Examiners’ Subcommittee on Exam Development on May 13, 2026, and the Committee of Bar Examiners on June 12, 2026. This framework, which will be finalized in the coming months, will set out processes for conducting content validation and subject matter expert review. It is being designed to serve as a roadmap for how processes are to be implemented and issues are to be escalated. It is intended to ensure consistency and assist staff by setting forth a decision framework. The framework is designed to ensure that all appropriate review procedures are concluded before questions are tested on the exam. We note, however, that the Supreme Court is currently considering a petition from the State Bar recommending the adoption of the NextGen UBE for implementation in July 2028. If the Supreme Court directs adoption of the NextGen UBE without a California component, the State Bar will no longer be developing multiple-choice questions for the bar exam. If the Supreme Court directs the creation of a California component, or a new California bar exam, exam content may or may not be tested through multiple-choice questions.
3. Recommendation: To ensure the State Bar’s executive leadership is aware of and can make well-informed decisions about how best to utilize contracts related to its bar exam, the State Bar should, by December 2026, revise its contracting policies to require managerial approval of any request for vendor services that do not align with the scope of services and responsibilities under the vendor contract.
   • Response: The State Bar agrees with this recommendation. State Bar staff has already initiated revisions to the General Procurement Manual (GPM), which will be finalized in the coming months, and will include appropriate revisions in the GPM to reflect this recommendation.
4. Recommendation: To reduce the risk and perception of bias related to question development, validation, and scoring when the State Bar develops its own bar exam, the State Bar should, by December 2026, implement controls that require separation of contractor responsibilities across exam development stages, including independent review and approval and formal oversight when a contractor performs multiple roles.
   • Response: The State Bar agrees with this recommendation and will implement it in the event that the Supreme Court orders the State Bar to develop its own bar exam or a California component of the NextGen UBE.
5. Recommendation: To improve oversight of bar exam development, the State Bar should, by January 2027, implement a policy that defines substantive changes to exam development and requires staff to document and escalate decisions on those substantive changes to executive leadership and appropriate governance bodies for review and approval.
   • Response: The State Bar agrees with this recommendation. The State Bar is in the process of developing a comprehensive exam development framework, which was discussed with the Committee of Bar Examiners’ Subcommittee on Exam Development on May 13, 2026, and the Committee of Bar Examiners on June 12, 2026. This framework, which will be completed in the coming months, will set out processes for conducting content validation and subject matter expert review. It is being designed to serve as a roadmap for how processes are to be implemented and what issues are to be escalated to executive leadership and governance bodies. It is intended to ensure consistency and assist staff by setting forth a decision framework. The State Bar will implement it in the event that the Supreme Court orders the State Bar to develop its own bar exam or a California component of the NextGen UBE.
6. Recommendation: To improve test takers’ ability to successfully complete exams, the State Bar should, by December 2026, implement a policy to require the testing of all new or changed components of its bar exam, such as the implementation of new annotation tools or changes to its rules regarding the items test takers may use during the exam. It should require the completion of this testing early enough to allow adequate time to resolve any issues or problems it identifies.
   • Response: The State Bar agrees with this recommendation. However, if the Supreme Court directs the adoption of the NextGen UBE, the State Bar will not have the ability to modify the software platform or select a different vendor. The NextGen UBE is a computer-based exam using the ITS platform. Hardcopy materials are not provided to test takers absent an approved testing accommodation. The annotation tools, notes fields, the ability to size answer windows, and accessibility features have been tested repeatedly, including through a beta test administered to 1,500 test takers in January 2026 and a small pilot (less than 100 participants) administered at the University of California Berkeley School of Law in April 2026. If the Supreme Court orders the development of a California exam, or a California component to the NextGen UBE, the State Bar will ensure that appropriate testing is performed on any software that test takers will be required to use for the exam.

Thank you again to the audit team that conducted the review. We appreciated the opportunity to work together, and we look forward to implementing the recommendations outlined in this report to further our mission to protect the public.

Sincerely,

José Cisneros
Chair, Board of Trustees

Laura Enderton-Speed
Executive Director

---

Comment

California State Auditor’s Comment on the Response From the State Bar of California

To provide clarity and perspective, we are commenting on the response to our audit report from the State Bar. The number below corresponds with the number we have placed in the margin of the State Bar’s response.

① The State Bar’s response indicates that it disagrees with certain findings in the report and notes that our office did not access Meazure Learning’s internal documents or interview its employees. State law required the State Auditor to conduct an audit of the State Bar and the administration of the February 2025 bar exam. Ultimately, the State Bar is responsible for defining, reviewing, and accepting the deliverables of its own contractor and the State Bar should have all the records necessary for us to evaluate its efforts related to the February 2025 bar exam. Thus, we rightfully focused our audit scope on the State Bar’s actions and not on Meazure Learning’s performance, which is the subject of ongoing litigation.

Footnotes

1. In May 2025, the State Bar filed a lawsuit against Meazure Learning alleging, among other things, that Meazure Learning knowingly misrepresented its remote testing capacity to secure the contract with the State Bar and then breached the contract in several ways, including by failing to provide test takers with an exam platform that allowed them to submit responses without issue. During this audit, we did not evaluate the merits of the parties’ claims and defenses in the lawsuit, which was ongoing as of June 2026. ↩︎
2. Psychometrics is the science of measuring psychological attributes, such as intelligence or understanding, and a psychometrician’s work ensures that tests are reliable and that results are valid. ↩︎
3. The Supreme Court postponed the July 2020 bar exam to October 2020 because of the pandemic. ↩︎
4. Kaplan operates a bar exam preparation business, which now excludes California, that offers previous bar exam questions. The NCBE’s letter stated that Kaplan’s license agreement with the NCBE authorized it to reproduce, distribute, and display only the publicly disclosed multiple-choice questions in connection with Kaplan’s bar exam preparation programs. ↩︎
5. Appendix A shows the number of questions within each subject the State Bar included on the February 2025 bar exam. ↩︎
6. Each year, the State Bar administers the first-year exam in June and October, which is required for only certain law students, including students who attend State Bar-unaccredited registered law schools. The first-year exam consists of 100 multiple-choice questions that cover three subjects: contracts, criminal law, and torts. The bar exam differs from the first‑year exam in that it covers more subjects and includes essay questions and the performance test. ↩︎
7. The State Bar conducted the November experiment on November 8 and 9, 2024. The November experiment field tested 49 multiple-choice questions both remotely and in test centers using the State Bar’s exam administration vendor. The State Bar announced that participants who met the minimum performance threshold of 28 correct questions were eligible to receive a score adjustment of 40 points to the multiple-choice section of the exam. This translates to an actual adjustment of 20 points for qualifying test takers. ↩︎
8. The admissions office’s principal program analyst was the audit team’s primary contact at the State Bar regarding question development. The principal program analyst previously served as the admissions office’s program manager for question development for both the November 2024 experiment and the February 2025 bar exam, and his former managers are no longer with the State Bar. ↩︎
9. Item difficulty measures how many test takers answered a question correctly. Higher values indicate that the question was easier, while lower values indicate it was more difficult. Item discrimination measures the relationship between test taker performance on an individual question and their total exam score. Negative or low discrimination values indicate the question did not effectively differentiate test taker performance and may reduce exam reliability. ACS Ventures also evaluated the answer options to ensure they functioned effectively. For example, effective questions typically have multiple answer choices selected by a meaningful share of test takers; in this case, the goal was for at least three answer choices to be selected by at least 5 percent of test takers. ↩︎
10. We are not assessing or reaching conclusions about any claims or defenses made in the State Bar’s lawsuit against Meazure Learning, including but not limited to the reasonableness of the parties’ actions. We did not audit or interview Meazure Learning. ↩︎
11. Applicants could choose to use a laptop for the bar exam if they completed the laptop registration process via the mock exam before the February 2025 bar exam. ↩︎
12. The State Bar provided hard-copy exam materials to certain applicants, such as an those approved for an accommodation for a disability-related functional limitation. ↩︎
13. The post-exam survey had a total of 2,810 respondents, which is approximately 66 percent of all February 2025 bar exam test takers. Because some survey respondents skipped survey questions, not all questions had a total of 2,810 respondents. Only 2,679 survey respondents responded to the question about experiencing technical problems during the exam. ↩︎
14. Of the 2,730 test takers who took the February 2025 bar exam remotely, 2,692 test takers submitted a mock exam. ↩︎
15. The State Bar’s psychometrician suggested a raw minimum pass score based on historical test taker data. The Committee lowered that pass score because of the pervasive and significant difficulties test takers reported. Thus, the raw minimum pass score the State Bar recommended to the Supreme Court was lower than it would have been otherwise if there had been no technological issues. ↩︎
16. We do not attempt to determine the amount the State Bar may be able to establish as damages in its litigation against Meazure Learning. ↩︎
17. The California Supreme Court ruled in 2016 that the contents of invoices for legal services that reflect work in an active and pending legal matter are privileged. ↩︎
18. Besides the problems with the February 2025 bar exam, the State Bar also refunded test takers for other reasons, such as for the wildfires in Los Angeles County and for medical reasons. ↩︎
19. The State Bar executed a master services agreement and four statements of work with Meazure Learning for exclusive exam administration of online proctoring and test center services. ↩︎
20. Until January 2025, the Political Reform Act of 1974 did not include designated State Bar employees and members of the Board as public officials prohibited from using their official position to influence a governmental decision in which they have a financial interest. Nevertheless, before January 2025, the State Bar adopted a conflict of interest code that deemed designated employees and members of the Board and committees to be public officials for purposes of financial disclosure requirements. ↩︎