Report 2020-104 Recommendation 9 Responses

Report 2020-104: Calbright College: It Must Take Immediate Corrective Action to Accomplish Its Mission to Provide Underserved Californians With Access to Higher Education (Release Date: May 2021)

Recommendation #9 To: Calbright College

To ensure that it is fulfilling its mission to help students obtain positive employment outcomes, including jobs in their field of study, earning gains, and upward mobility, Calbright should do the following:

1. By November 2021, develop and implement a specific plan that describes how it will assist its students in acquiring jobs, earning more income, or being upwardly mobile after graduation; the plan should include a path toward securing job placements for its students.

2. By the same date, also establish a method to collect and review data on student use of its career services, employment outcomes following graduation, and employer satisfaction with Calbright's preparation of its students.

3. By July 2022, Calbright should begin demonstrating that it has been successful at assisting its graduates in obtaining positive employment outcomes, including jobs in their field of study, earning gains, and upward mobility.

Annual Follow-Up Agency Response From October 2023

As agreed upon by the auditor's team in the spring of 2022, Calbright has fully implemented the first two elements of Recommendation 9.

To expand on the College's work supporting positive labor market outcomes for students, Calbright has developed an employer engagement plan and expanded its career services team. In the fall of 2023, the College announced the launch of a data analyst apprenticeship training pipeline in partnership with Bakersfield College. Complementing initiatives underway with workforce boards and labor unions, Calbright is also working with NextGen Policy and Los Angeles County to establish a public sector technology apprenticeship program.

To track labor market outcomes for graduates, Calbright has been added to LaunchBoard, a statewide CCC Chancellor's Office data system. However, meaningful employment data lags and may not be available for several years while information is populated. In addition, as is standard practice in higher education, Calbright conducts alumni surveys to measure labor market outcomes. The current version of the survey launched in the spring of 2022 and reflects a 28% response rate (55 responses) of completers who finished between 3/22 and 7/23 (197 certificates issued during this period). 89% of students who completed the survey answered the following question: "How long did it take you to see a positive impact of your coursework on your employment, after completing your program?" Of the students who responded, 61.2% indicated a positive impact within 12 months of program completion. Students, through the alumni survey, may also provide their income prior to enrollment and after graduation. Of those who did, nearly 48% either moved from unemployed to employed or earned a raise, and nearly 48% kept their current role. This is a rough comparison to the CCC system-wide figure that shows roughly 28% of adult learner graduates move from unemployed to employed after completing their credential.

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

As previously noted, Calbright submitted a plan for providing career services to its students. It also tracks usage of those career services and includes questions about students' use of those services in its career survey. For this update, Calbright provided data on responses to its alumni survey. We shared with Calbright some concerns regarding the specificity of the information that it solicits from its graduates regarding the degree to which the certificate they earned from Calbright has helped them achieve positive employment outcomes, and would suggest that it ask more specific questions about what positive outcomes those graduates enjoyed. However, Calbright's survey data does show that some students reported experiencing a positive employment outcome as a result of Calbright's program.

Annual Follow-Up Agency Response From November 2022

As agreed upon by the auditor's team in the spring of 2022, Calbright has fully implemented the first two elements of Recommendation 9: developing and implementing a specific plan that describes how it will assist students with positive labor market outcomes and establishing a method to collect and review data on student use of support services.

To expand on the College's work supporting positive labor market outcomes for students, Calbright has developed an employer engagement plan which complements the strong partnerships we have established with regional organizations, workforce associations and boards, community based organizations, and government agencies. Through the third quarter of 2022, the plan highlights our focus on statewide outreach and engagement with cross-industry employers to increase visibility of Calbright programming and position our adult learners for optimal labor market outcomes.

To measure students' employment outcomes after graduation, Calbright currently conducts alumni surveys and has been added to LaunchBoard, a statewide data system supported by the CCC Chancellor's Office. However, employment data is a lagging metric and may not be available right away, due to the manner in which LaunchBoard collects it. Calbright is also a new college and does not yet have a large alumni base, so it will take time to accumulate alumni survey data - the College will focus on this throughout the 2022-2023 fiscal year. There are strong anecdotal data of positive labor market outcomes, many of which are noted on the College's "Our Students" webpage, and continued measurement will be a focus as completion numbers accelerate during the 2022-2023 fiscal year.

California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented

1-Year Agency Response

Calbright has fully implemented the first two items under Recommendation 9, and will complete the third by the stated deadline, July 2022.

To expand on the College's work supporting positive labor market outcomes for students, Calbright has developed an employer engagement plan which complements the strong partnerships we have established with regional organizations, workforce associations and boards, community based organizations, and government agencies. Through the third quarter of 2022, the comprehensive plan highlights our focus on statewide outreach and engagement with cross-industry employers to increase visibility of Calbright programming and position our adult learners for optimal labor market outcomes. Building on this, we have also partnered with the Bay Area Council, a leading business association in California, to facilitate information sessions, job site visits, webinars, and multi-industry convenings with priority employers.

As shared in November 2021, Calbright has also developed and implemented a Career Services Plan (CSP) that describes our strategies for helping students and alumni obtain positive employment outcomes. We provide career readiness and career advancement services and resources to students throughout their educational journey, including career coaching, career readiness workshops, and industry events.

To measure students' employment outcomes after graduation, Calbright currently conducts alumni surveys and has been added to LaunchBoard, a statewide data system supported by the CCC Chancellor's Office. However, employment data is a lagging metric and may not be available right away, due to the manner in which LaunchBoard collects it. Calbright also plans to survey employers to track the outcomes of program completers, such as retention after upskilling, earnings gains, and upward mobility.

California State Auditor's Assessment of 1-Year Status: Partially Implemented

As noted in the previous evaluation, we look forward to reviewing Calbright's success in assisting its graduates in obtaining positive employment outcomes in July 2022.

6-Month Agency Response

Calbright has fully implemented the first two items under Recommendation 9, and will complete the third by the stated deadline, July 2022.

Calbright has developed and implemented a Career Services Plan (CSP) that describes our strategies for helping students and alumni obtain positive employment outcomes. We provide career readiness and career advancement services and resources to students throughout their educational journey, including career coaching, career readiness workshops, and industry events. Career Services also collaborates with the Workforce Development team to establish partnerships with employers, industry advisory boards, workforce development boards, and other agencies to provide a path toward securing job placements for students.

To measure students' employment outcomes after graduation, we currently conduct alumni surveys and have taken steps to be added to LaunchBoard, a statewide data system supported by the CCC Chancellor's Office. LaunchBoard provides data on employment and wages by matching student records with the state's Unemployment Insurance wage file. We expect to be included in LaunchBoard in 2022, but employment outcomes data may not be available right away, due to external lags in data collection. Calbright also plans to survey employers to track the outcomes of program completers, such as retention after upskilling, earnings gains, and upward mobility. Using available data, Calbright will demonstrate that it has been successful at assisting its graduates in obtaining positive employment outcomes by July 2022.

Additionally, as discussed in the CSP, Calbright tracks students' use of career services through case notes, post-session surveys, and its alumni survey. To assess employer satisfaction with Calbright's preparation of students, we collect feedback from employer partners, such as through our advisory councils, and plan to conduct surveys of employers that are strategic enrollment partners for particular programs.

California State Auditor's Assessment of 6-Month Status: Partially Implemented

Calbright provided a career services plan that describes career services and the various stages throughout a student's enrollment where those services will be offered to students. The career services that the plan describes include services such as resume building, job searching, and networking. Calbright also noted in the plan a method for tracking student usage of career services. Calbright listed one method for tracking student success in achieving positive employment outcomes - an alumni survey - however, the survey that Calbright provided was a draft.

Although the career services plan briefly describes some work that Calbright has done to establish partnerships to increase employment opportunities for students, specifically with local Workforce Development Boards and a job search organization called Opportunity@Work, it has little description of plans for future partnerships or how those partnerships may lead toward job placements.

We look forward to reviewing Calbright's progress toward demonstrating that it has successfully aided graduates in obtaining positive employment outcomes when its next update is due in July 2022.

60-Day Agency Response

Calbright centers its work on ensuring students obtain positive employment outcomes. This work is shared across pathway development and career services.

Our design research approach to pathway development is centered on employer needs, market research, and user feedback. As referenced in our response to recommendation #8, Calbright's process starts with determining need based on employment supply and demand. We engage with employers, Advisory Councils, Workforce Boards and Unions to understand urgent needs. We conduct Labor Market Research on emerging trends and qualitative interviews to stay abreast of workforce realities. While validating our programs we use qualitative interviews to determine challenges, realities and aspirations of those currently in the workforce. Those insights inform our coursework and guide how we help our graduates to become workforce ready. To support our continuous improvement efforts, Calbright conducts ongoing outreach to industry and employers to stay current with urgent needs and trends and distributes student outreach surveys.

Career Services provides opportunities for our students to supplement their program coursework with connections to the workforce they are studying. Based on the needs that students communicate, Career Services also develops and facilitates workshops for resume writing, interviewing, and other skills development. Through a variety of surveys, including alumni surveys, we discover which students have received job offers as a result of their development with us, and we are able to continue to offer services to ones who are still seeking jobs.

Calbright has begun to document this process and is on schedule for full implementation.

California State Auditor's Assessment of 60-Day Status: Pending

All Recommendations in 2020-104

Agency responses received are posted verbatim.