BLAIRE TOSO: Thank you, Veronica. I welcome everyone, and thank you for joining us today. We are winding down this program year with a focus on labor market data and how it might inform the work that we do. The presentation today will introduce you to some of the key concepts on this topic, and a tool that we've developed that can support and inform pathways development, that aligns workforce development courses to regional labor market. 

And this is so that we can offer learners relevant educational opportunities that lead to living wage jobs, or sets them on a pathway to a living wage occupation. So after this workshop, which will be very informational, we'll be offering two more workshops later this month that dive more deeply into the data, and using that data to inform and to identify transition skill and equity gaps and opportunities. 

I'd like, also, to take a moment to introduce you to the WestEd team, which is myself, senior program manager, Jessica Keach, who is our senior research associate, and Ayanna Smith, our program coordinator, too. And I'd also like to offer a particular shout-out to Allie Bollella, who will be presenting today. And she's really been the driver behind this work that you're learning about today. 

We're very grateful for her time, energy, and expertise in this area of using labor market and course offerings data to inform program and career pathways development. I'd also like to give a shout-out to Alexandria, who is also here on the presentation with us. And she'll be working closely with Allie. And the two of them will be presenting in those next two workshops that will really give you a deep dive. And they're both labor market experts and adult education experts. I'm delighted to have them both joining us here today. 

Next slide, please. I'd also like to acknowledge the Chancellor's Office team-- Mayra Diaz, who's the CAEP program lead, and Lindsay Williams, who is the program manager. And they have supported and informed all of the work that we do. 

And we really appreciate their commitment and engagement on providing the field with updated adult education data, as well as professional development that will help you with your program planning, from the adult education pipeline dashboard perspective. Due to other commitments, they're not able to join us today. But they have asked me to extend a welcome from their office on their behalf. Next slide, please. 

So today, we'll cover the following topics. We're going to introduce you to the background that has informed this work, which includes an Adult Education Course Scan, which we recently refreshed, an example of data collection and management from San Diego, that also strongly informed this work from a field and practice perspective. Allie's going to walk us through a review of the updated Adult Ed Career Education Dashboard, as well as how to use this information as a setting for the next two workshops. 

Next slide, please. And so our goals really are the following, which are providing an overview of this work and the dashboard, demonstrating the value of aligning educational offerings to labor market data, providing some examples of the course alignment that can create transparent career pathways, demonstrating how to identify trends for planning purposes, and then, really, presenting the labor market resources and career exploration tools for regional planning and alignment. 

And so at this point, I'm going to turn it over to my colleague, Allie Bollella, to take us through the rest of the presentation. 

ALLIE BOLLELLA: Thank you so much, Blaire. Thank you for the shout-out. I really appreciate it. And thank you all for joining us today. We're really excited to share this refresh of the adult education career technical education course data that we've collected, and the tool that we've now updated and fully revised for you guys to use. I'm going to start off by talking a little bit about how we started this work, how the tool came to be, and then get into a lot more of the details. 

So we first started this work back in 2019. And, really, the purpose was to understand the full continuum of CTE and career courses offered both by non-credit and adult education practitioners. We wanted to do this so that we could map the relationship between adult education CTE career offerings, credit programs of the colleges, and also regional labor markets. 

We envision this work really as supporting consortia pathway development and conversations around using labor market data, and developing other resources to support CAEP students, as well as other conversations of alignment between adult education and community colleges. And one day, hopefully, integrating this data into statewide data sets that we could use for more rigorous research. 

So in our initial scan of the adult education courses, we looked at 521 institutions across the state. And we really scanned these institutions based on web pages, brochures, course catalogs, knowing that there'd be some errors. But we wanted to get a broad scan of what was out in the field without putting a lot of extra work on the field themselves for collecting this data. 

And we found, out of those 521 institutions, 225 agencies providing CTE and workforce preparation courses. And from that scan, we found over 7,000 course offerings in CTE, in adult education and non-credit offerings. So really there's a lot of amazing work being done in the field. And we were really excited with everything we found, even given that there were some errors, or old course catalogs, or things that hadn't been updated. There were some really promising findings and exciting work being done. 

So from that very large data set, we were able to broadly categorize CTE workforce course types. And this allowed us to think about how those courses align to the labor market more strategically. While all of them are equally important, it's important to understand what level they're at, so that we can align them to specific occupations, or maybe broader sectors, and really understand the relationship between programs in adult education and living wage jobs in a region. 

So the first is the workforce preparation courses, which include workforce readiness, OSHA, ServSafe. These are general skills necessary for success in the workforce. And some adult schools have this embedded into more rigorous job training. Some of them have them as stand-alone courses. But there's a clear workforce readiness, general skills that prepare people for any field and are not sector related in any particular way, necessarily. 

Then we have the occupational skills builders, which are offerings that advance skills related to a specific industry or a specific sector, but wouldn't, themselves, independently allow a student to obtain a specific occupation. So these might be courses in Excel or QuickBooks that are stand-alone. And again, a lot of schools may incorporate these individual skills builders as a sequence into a larger job training program that's more specific. 

But as stand-alones, and there are a lot of them, skills builders are really focused on a specific, discrete skill, sometimes related to a specific industry, that would allow a student to maybe pivot or get promoted in their current career path. And then finally, we have the occupational credential programs, which are really the more rigorous job training programs. These are programs and courses of sufficient duration and intensity that provide skills for an individual to enter, or advance, in a specific occupation. 

So medical assisting is a really common one. Accounting certificates, or a more rigorous accounting job training program-- they're found in all sectors. And the intent is you can link that specific program with a pretty discrete occupation that maybe a student will enter into, or maybe not, but that's really the intent of the program, is to enter into a very specific job. 

And we had some higher-level findings from the work that we did, looking at all these regions, and talking to some schools about how they're offering programs, and what their programs look like. So across all the regions in California, there was a really diverse mix of occupational training courses and skills builders, as well as occupational sectors. Nothing really stood out as one region is offering a ton of health care, and another region isn't. There was a pretty diverse mix, and pretty even across the state. 

However, statewide approximately 60% of the courses were skills builder courses. And we also found that at least a 1/3 of adult education schools did not have any CTE or workforce preparation courses in their catalogs, websites, or associated brochures. And again, that doesn't mean that those schools aren't offering it, but we weren't able to find anything for at least a third of the schools' online resources. 

From the public information, it was also really hard for us to identify which schools were offering integrated or IET programs. And this, as I'll talk about later in the San Diego work, really informed the work we did collecting data with them, and thinking about how best to communicate what a program is to students, and how to display things on a website, and how to communicate that information effectively so students know what a program is. 

And then the emphasis on skills builder courses, we felt was really strongly aligned with the mission of adult education-- help students get skills quickly, and relevant to their needs. And this also, again, later informed conversations about how to communicate effectively to students, what a course is, and what the intent is, and for them to understand how to take those skills into the labor market. 

So I want to pause for any questions we may have about that initial first scan that was in 2019, winter of 2020. 

BLAIRE TOSO: I don't see any questions in the chat, Allie. 

ALLIE BOLLELLA: OK, great. Well I'll keep going. Feel free to drop things in the chat if you have questions or anything comes up. So next, I'm going to talk about how we took this work and implemented it in San Diego. And we really worked with the super-region there to think about collecting their data, their CTE courses, and creating a kind of base management system for them, so that the region could be aligned. 

So our goal of this project with San Diego Imperial region was to organize and classify regional CTE courses to build a career pathway system, and career navigation platform one day, across the SDIC super-region. We started in this project really just establishing common definitions, organizing courses and programs, and coding courses and programs. 

And that really just meant creating a template that would allow adult schools to enter all of the data consistently, regarding their courses, organize them by the different types of courses, either workforce preparation, skills builders, or occupational training programs, and also align them to Standard Occupational Codes-- in other words, specific occupations or specific sectors. 

We then worked with the region to establish processes and a data structure for aggregating and updating courses and program data. So we really worked with them to think about what would be the process for updating this information regularly? Where would they store it? How would it be stored? Who would have access to it? And what kind of meetings or regular collaboration there would be to think about aligning those courses or revising things. 

And then, in the future, they're working to implement an adult learner career exploration portal that captures the regional AE career pathways and system. And that is still ongoing work in the region. And we're excited to see what comes out of that. 

So the structure was, we really worked with the super-region adult schools and college non-credit programs. We provided three Train the Trainer sessions for consortia reps to learn about occupational coding and framework. And these were pretty detailed Train the Trainer sessions, where we talked about Standard Occupational Codes, how to use ONET online resource center to find information about occupations and the skills associated with them, and to think about what the appropriate level of coding courses is. 

So thinking about whether it's an occupational training program, and you want to align it to a very specific occupation, or whether it's a skills builder, and you just want to align it to a broad sector that would just have a two-digit code. And thinking about what the process is for understanding that, and working with faculty at the adult school to understand more about the course, or have them code it themselves. 

And really a lot of this work was then done by the adult schools themselves. We conducted the Train the Trainer sessions. And they went back and coded all the courses, and provided us with data sets. And we then regrouped with those consortia reps and reviewed everything. And basically had a peer-review process to start discussing, why might this medical assistant course have 50 more hours of instructional hours than another course at another school? 

Or thinking about, why is this medical assistant course, or this accounting course, have a different occupational code than another one? So really working on aligning the data itself across the region, and making sure that we're appropriately coding courses, and that there is some type of consistency and logic in coding across the region. And so the next steps of that was really the horizontal alignment of course features across the region, ensuring an equitable understanding. 

And this really led to a lot of our conversations about why a course feature might be slightly different, why it might have a few more instructional hours, or why it may have a very different description in the text even though it's a very similar course. And talking with the school is about how students are getting that information, and whether it's relevant, or important, to align across the region horizontally, so students are getting similar information for similar courses. 

We also had a lot of conversations, in those next steps, to align with community college credit pathways. So we had conversations with the non-credit programs and the adult schools, to start thinking about how their courses could on-ramp to credit, and what some of the challenges and barriers might be to that, and start thinking about strategic ways to overcome those barriers. 

And the final thing we started was exploring the development of an online career platform. And that was beneficial. A lot of our conversations about horizontal alignment was very beneficial for thinking about how the data should be structured for that, and how to communicate with students what's going on in the region, and what's being offered. And that work is still ongoing, but we're excited, again, to see what comes out of it. 

So this is the initial data collection process. And some of you that sent us updated course data may be familiar with this template. We used a very similar one for this new refresh of the dashboard. And this was the template that we developed for San Diego, where all of the schools use the same template and organize data in a very similar way. So we have their course title. We have their unique course ID that they use at their school. 

We have the tier level, which is really just the level that the course is at, workforce preparation or occupational training courses. And then we also have the primary Standard Occupational Codes. So we have the actual code itself, in the primary SOC code. And we have the title of that. And you can see in this example, there are some that have six digits, which would be a very specific occupation. 

And we have some that are just a two digit with four zeros, which would be just a broad sector. And you can see that the ones with two digits are usually these occupational skills builders. And the ones with more discrete, more specific, occupations are the six-digit codes. And this was the basic layout. 

San Diego is currently holding all of their courses in an Excel document like this, and using a very similar template. Although I've seen they've adapted it to start thinking about what would be important for a career online platform. But are using this basic layout to keep their data until they get a more rigorous data system. So it's been, hopefully, really effective and helpful for them, for storing all these courses across all of their schools. 

And so all of this work gave rise to quite a lot more questions, but also a lot of opportunities to work with. So the questions that came out of this work were really how, and to what extent, do we need to align classes? How exact do we need to be in aligning a specific accounting course in one adult school to another accounting course in adult school? Are they different for important reasons, like maybe their student populations are different enough that the courses should be a little bit different? Maybe not. 

And those were some of the questions that started to come out of this. Another question, how do we leverage similar, but different, educational opportunities? How do we measure the value and quality of our courses? So how do we think about what's the most valuable thing for students, whether it's hours, or certificates, or work-based learning? Knowing that we have only a certain amount of time for students, what's the best way to leverage and understand value of those courses? 

And then what's the value of skills builder courses versus occupational training courses? How do we best communicate to students and help them understand what they really need out of adult education. Are they looking for a promotion? Are they looking to pivot to a slightly different career field? And maybe that could be accomplished with learning a couple discrete skills, rather than going through an entire occupational training course. Or maybe they don't have a lot of experience and they're looking for an entirely new job, or looking for a much higher-paid job. 

So those are some really important questions that came out of it. The opportunities we found in doing this work and really thinking deeply about how the data is structured, and how courses are described to students, is refined understanding of different student populations among members. So a lot of our conversations became student-centered, and thinking about what the student population is at this particular adult school or this non-credit college, and how they may differ. And why a delivery of programs maybe importantly different, or maybe not. 

It promoted discussions across all the members. And people started having some really exciting new ideas-- open programming conversations among instructors. So thinking about what's really a relevant mix of courses. It refined marketing messages for students and employers. So a lot of schools told us that they started to revise, and think more deeply about, the course descriptions themselves. And provided some focus program planning, as well. 

And, Blaire, did you want to add anything to this? Because you were very involved in the work, as well. 

BLAIRE TOSO: Thank you. I think it's just exciting that it continues to go on. And I would like to pause just for a moment. I think within the participants, I saw a few people who were actually on the ground doing this work. And I didn't know-- I'm not going to call you out, specifically. So if you want to remain silent, that is completely fine. But if there's anyone who was participating in this process, and continues to participate in this process in San Diego, if you wanted to add a commentary or any thoughts to what's been presented here, we would love to hear them, or even an update. 

KELLY HENWOOD: Blaire, this is Kelly Henwood. I'm trying to see if Suzanne or Uta-- I'm trying to see who's here from the super-region from San Diego. 

BLAIRE TOSO: Yeah, I didn't see either of them. I saw you, Kelly, I have to admit. So thank you for coming. 

KELLY HENWOOD: Crystal's here, too, from Sweetwater. So I haven't been as involved in the meetings, but we do have a community of practice that's been working on it. So the big thing is that we have several K through 12 adult ed providers that have a lot of different CTE courses in our consortium. It's a little bit interesting. And Nate is here from San Diego Unified, he's our consortium member partner. 

They actually don't provide CTE programming in their adult ed programming at San Diego Unified School District. But the process, I think, since we go through in the non-credit side, we go through a lot of work to do curriculum review. And so a lot of the top code pieces and things, we had already gone through. Our challenge is, right now, we're in the repository, is what we're calling this. But we have some missing data. 

So it's basically just Blaire, Greg had done a lot of work of pulling the information out of the state system. But we still have some pieces that we need to put in there. And that's our summer project at San Diego College of Continuing Education. But a lot of work has been done on the K through 12 adult ed side to get their information in, in terms of number of hours, the course outline of record, descriptions, and all sorts of other information. 

So I think it's going to be very helpful. And our dean of our college CTE programs, Cassandra Storey's on the line, although she hasn't been involved in the community of practice. But I think one of the things-- my thoughts about this is the data is really important. And we really need it. I see Crystal just came off her video, too. But I think where we're going to be left with is still how are we making sure that we have processes and procedures to have consistent guidance for students, and those types of things? 

So I think that's the thing that there's been conversation about it, but I think that's the thing that we really need to figure out, is how do we make sure that this information is being shared across the different member agencies? And then, Crystal, I don't know if you have things that-- I know you were a lot more involved in the community of practice. 

CRYSTAL ROBINSON: Yeah, that was a great summary, Kelly. Thank you. Yeah, just that it's been a lot of conversation around what we are actually going to include. Coming from a K-12 adult perspective, the distinction that we went over today, as well, like occupational skills builder, occupational training program, that kind of distinction is almost foreign. 

Everything is just CTE. And it's organized around CTE credential, and those industries. So even discussions around SOC codes and TOP codes, all of that is like learning a new language for the k-12. But at this point, we're at a point where it's a lot of discussion, a lot of just trying to make it make sense for everybody. 

And, well, we will be updating everything by August. So we decided, OK, we'll take twice a year, August and January, to update what we have in there. And every time we update it, we're probably going to be finding new things to discuss, and just refining as we go along. It's really a work in progress, but it's been great. 

BLAIRE TOSO: Thank you, Kelly and Crystal. I really appreciate that you were willing to open up and have a little bit of a conversation about this. Because I think it's one thing to hear us talk about it, and another to talk-- particularly because you're reinforcing the idea that it's a lot of work. I really appreciate the fact that you're talking about that student-centered lens, that these conversations come back to that, which is part of the aligning of the courses. And then the emphasis on the data, and the fact that it's really important. 

It's important to use clean, good data. And that those conversations, you have to have discussions around them, so that when you're trying to create a regional initiative, that those open conversations take place. And we knew that those conversations would take place. But I think what was really interesting was also the conversations that were spurred when people came back after the first effort of doing this within their institution. And saying, wow, we even had alignment conversations within our institution, and across our colleagues and instructors about what does this mean? And how are we all collectively talking about this? 

So thank you so much. And I don't know how many of you on this website, Allie will talk about this a little bit later, but when she was talking about that Excel sheet, when we gathered. We did send out a request for people to update the data from their own schools. Because one of the things that we found out, Allie, who did this incredibly hard work about doing that scan that she talked about up front, is that it really needs to come from the institutions. Because they best-know what is being offered in their institutions, or in their consortia. 

So if anybody is on here and they realize, or do not know whether that data was updated, you can certainly reach out to us and we'll be able to tell you whether we received updated data. And if not, if you would like to update it, it makes the dashboard that we're going to demonstrate and walk through later in this presentation-- it will make it much more relevant and accurate for you. 

And I also wanted to point out that Ed posted in the chat. Thank you, Ed. He asks, how do we measure the value and quality of our courses, exactly? And that the Pell accreditation requires a minimum level of completion, job placement, retention, and licensure rate. And that these metrics keep you focused. 

And then if you take that to the next level, where you're talking about that across your consortium, across institutions, it's just another way of looking, and serving students, and identifying those gaps where there might be opportunities to both engage more, or to bump up those outcomes. Thank you. 

Does anybody else want to comment? Anybody else from San Diego? Anybody within the group at large? All right, I'm going to turn it back over to you, Allie. But please, feel free to continue to pose questions, or just raise your hand, or even unmute and interrupt. This is not being controlled. 

ALLIE BOLLELLA: Yeah, feel free to interrupt if I'm going too fast or if you have any questions moving forward. I'm going to get into some of the more technical stuff, which includes the labor market data, and then a review of the dashboard itself. So feel free to stop me if something doesn't make sense or if you have questions. 

So we recently, this spring, started a revision of the dashboard. We had a previous version of the dashboard that displayed the course information and some labor market data. But we decided to one, update the data itself, and send out a request for course data from you guys, based on the template we did in San Diego, so we could have some more accurate information where possible. And also redesign the dashboard itself, because why not? We learned a lot through this process. And we thought we could probably make a dashboard that works a little better for you guys. 

So the data collection process for this round started this spring, in 2022. And you are welcome to send us more data if you would like, and have not done that already. The data collection is a modified version, as we've discussed, of the template in San Diego. And we hope that it's helpful for you guys to think about your courses, as well, as you're filling it out, and it's not just another thing you have to do. 

And that you would get to then use the data on the dashboard, and see all your courses in different formats. So the dashboard includes data from the most recent round of data collection. And we also decided to include the original round of data collection, only for schools that have not sent us updated data. So if you have sent us updated data, only your updated courses will be in the dashboard. And for those schools which we did not receive updated data for, we have our course information from the 2019-2020 year. 

So for this design of the dashboard, we decided to really put the labor market data first, because we know that's what students are focused on. They're focused on finding a job, finding a slightly higher wage. And the labor market data is organized by Standard Occupational Codes. And these occupations are standardized by BLS. And the discrete occupations are given six-digit codes and placed within one of the major 23 groups. 

So there are 23 sectors that have two-digit codes. And then there's layers of detail in between those six digits and 23 digits. But the dashboard primarily focuses on either the six-digit discrete codes, or thinking about the sectors as a whole. We have three variables with the labor market data. 

We have annual openings, which is a combination of both new jobs and replacement jobs. So jobs that are anticipated to happen, and then jobs where someone may be retiring or leaving an occupation, and the anticipated replacement jobs that will be needed. And that's just divided by the number of years in the time frame. So we've used a 2020 to 2025 data set, with projections to 2025. 

Given COVID and some of the changes that have been happening, we decided to limit projections. The median wage definition-- so we have median wages in there, which is just the halfway point of all workers within that specific occupation. And it's important to note that this value is collected from employers themselves. And then we have the 2020 to 2025 percent change. 

And that is really just the percent change in annual openings of a specific occupation between the base year, 2020, and the end year, 2025. So it's the anticipated growth, in a sense, of that occupation. We've also included, regionally, the MIT living wage. So we will have a link on the dashboard, as well. And as you filter through for the regions, you'll be able to see the living wage for your region. 

And we've done this a little bit differently this time. Because there are some regions that have really high-cost metropolitan areas mixed with some lower-cost areas, instead of averaging the living wages for counties, we've decided to use the county with the highest living wage possible. Primarily because those metropolitan regions tend to have a lot more people. 

They tend to be more populous, so we've decided to use living wage for the highest county we can, so that it's better representative of more of the people living in that area. We also have a set of ONET skills. So ONET online standardizes skill and assigns them to occupational codes. And we've included that in the dashboard just as a reference point to think about what skills are relevant for a particular combination of occupations. 

So I'm going to go through, and this is a screenshot of just the labor market data on the dashboard. So we have, at the very top here, as you select the sectors that you want to investigate, or filter the dashboard for, at the very top there'll be an average of the sectors that you select. So if you select one sector, let's say business, you will have an overview of the annual openings, median wage, and the anticipated change for that particular sector. 

If you select multiple, it will update and do an average of those multiple sectors. So you'll be able to think about the sector as a whole across the state or in a particular region. And then you will have-- each of these rows is a particular six-digit occupational code. And first will be the annual openings, so on average, how many openings there are expected to be for that occupation. We have the median wage next to it. 

And then finally, we have the anticipated change. And some of them will be negative if the projections indicate that there will be fewer occupations in the future. We also have, at the top here, the macro region living wage. And we have a link to the MIT living wage calculator. So if your school is in a county that you don't think is probably represented, or if you want to get more specific county information, you can navigate to that site pretty easily. 

And they have downloadable data for each county in the United States. And then finally, we have skills on the dashboard. And these are the ONET skills. So they are pretty rigorously categorized. And they have a ranking from zero to five, which five being the most relevant for a specific occupation. So the higher the ranking, the more central or important it is to a specific occupation. And these rankings are averaged. 

So as you filter the occupations themselves, the skills will update. And you'll be able to see, for a specific cluster of occupations, what skills are most central to that cluster of occupations. Do we have any questions about that before I continue on? I'm seeing a few things in the chat. 

BLAIRE TOSO: Yep, there was one about-- Adele made a really interesting comment that we'd love to touch base a little bit further about-- is that she says in some of the smaller schools and the rural schools, that they often offer courses, but they're not necessarily listed in a central catalog or website. And so she was asking about her particular consortium, or region. And I just responded that we'll touch base with you. And definitely that that's an excellent point. 

And what we would like is as complete a data set as possible. So if there's nothing in there, or Adele, if it doesn't look like it's updated, we will definitely reach out to you. So that's one. That's that. If you have anything to add-- 

ADELE MCCLAIN: Thank you. No, I appreciate it. And some of us that are newer, we're just trying to figure out-- we have stuff going on, but how do we get seen? 

ALLIE BOLLELLA: Yeah, absolutely. 

ADELE MCCLAIN: How do we become part of the big picture? We want to be there. And we're kind of, probably, even there somewhere, but how do we even find where our data is showing up, other than maybe our WIOA partners know we exist. 

ALEXANDRIA WRIGHT: Perhaps, Adele, did you receive the inquiry that was sent out a couple of months ago, regarding the collection of data for this update? 

BLAIRE TOSO: We sent it to consortium directors, as being the most effective way. So that's why-- when we have people who are representing particular institutions, it's really a great space for us to be able to find out more directly who has, or has not, participated. And that's why we continue to mention this, that if you didn't see something come from us, it may not have passed along. We know people are really, really busy this year. 

We completely understand, with the three-year planning, that this probably did not seem as relevant as some other topics at that time. So we definitely want to hear from people who are unaware as to whether their data actually got updated or not. 

ADELE MCCLAIN: That would have been Martha Mendez. But I think I actually did get something from you, as well. Either that or she showed it to me. I'm not sure which way that went. 

BLAIRE TOSO: Well, we can confirm. Well, "we," that's rather a large use of the word. I'm always reframing this. Allie is really the one who digs into the data. So we'll take a look at it and definitely get back to you. And someone else was asking about the website, the page that Allie's been showing. We'll pop the link to the dashboard, in a minute, into the chat so everybody can access it. 

ADELE MCCLAIN: Thank you so much. 

BLAIRE TOSO: Yeah, absolutely. 

ADELE MCCLAIN: This is really good content. Don't think in any way I'm complaining. I'd feel like it's not even a problem getting us in. You make us feel welcome. I'm just like, it's a lot, when I'm looking at the bigger schools. We're like, whoa-- coming from a school of 500, yeah. 

ALLIE BOLLELLA: Yeah understanding that, as well, we are working to put a filter on the dashboard, so you can filter by consortium. We're not quite there yet, but we hope in the next month or two to have that filter for you guys, so you can just look at your consortium. And Paul, maybe one day county. That will be a longer-term project for me. 

ADELE MCCLAIN: Thank you so much. 

ALLIE BOLLELLA: Yeah, absolutely. So right now the main filter is the macro regions. But we will hopefully have the consortia up there soon. And we just wanted to provide a slide for y'all, so you can look at what counties are in your macro region, and find which one you're at. 

And we also have some documentation, a link to that, in a Google Sheet for you guys to just have until we get the consortium one up there. So if there aren't any questions, I'm going to go through an actual demo of the dashboard. And we can pop the link into the chat real quick. 

BLAIRE TOSO: I think Ayanna just popped it in if you all wanted to look that up. And then she's got another one in there. So if you all want to take a look at those. The first one, I believe, is to the dashboard 

ALLIE BOLLELLA: OK, great. So is everyone seeing the dashboard up here? Well, all right, great. So when you land on the dashboard initially, without any filter selected, it's going to be showing you everything for the state of California in all occupational sectors. And the averages up here for the sector are then averaged for all sectors across all regions. And you'll see all of the data. 

Here we have the occupational data, the skills. We have the adult education job training programs. So these are the programs that are associated with a six-digit occupational code. And we have a bar chart that shows the number of courses, by sector, across the state here. We have a filter for school name. And then we have a table that allows you to dig into a little bit more of the data itself, the school name, the course title and hours of instruction, and when it was updated. 

So you can make sure it's the most up-to-date information if you're going through this dashboard. And then we have the community college programs. So these are also organized by all of the award types. We have everything in here, including non-credit and no award. And this was all pulled from COCI. And it's organized in a similar way of the number of awards, by a bar chart, and by sector. 

And then we also have a detailed table that allows you to sort through the information a little bit more discreetly. And we have a filter so you can select a specific college. And the award type filter. So when you come up here, you can first select, maybe, business. We're going to look at the business sector. And once you have a sector selected, this will now show you everything for the entire state of California, because we don't have a region selected. 

So this is now showing you the average across the state of California for the whole sector. The median wage, the change, and then you can see by the number of available openings. So the occupational data is sorted by the highest level of annual openings. So we can see project management, accountants and auditors, and all of our skills are now aligned to just that business sector. 

We can see that there is 172 job training programs in adult education, and 12,000-ish community college awards. And you can start to sort through, in more detail, what's being offered at what schools, and what the degree pathways are. And then down at the very bottom, we have the skills builder courses. So these are courses that are aligned to the two-digit, or maybe aligned to slightly less discrete occupational codes. 

And we have them separated out so that you can really look at the full description, and review them in more detail separately. And this is a great way to start thinking about whether there is alignment between your courses and the labor market. So if we just look at accountants and auditors, it looks like there's a decent amount of annual openings. It has a high median wage. And it's anticipated to continue growing. 

And if we just really just scan through the job training programs in adult education, we can already see quite a few accounting, tax preparation programs available. And you can download this sheet. You can export it either into a CSV or into Google. So if you're interested in a specific occupation, you can export this whole table or a specific school. 

And then you can look at-- accounting is pretty clear. So we can start to see that there's definitely things happening in accounting. And it's aligned to the fact that there are a ton of accounting occupations. If you're here and you're like, I don't really know what a compliance officer is. How would I even start to align that? You can click on that title, and it'll take you directly to the ONET page for that specific occupation. 

And it'll give you an overview of that occupation and the specific tasks, work activities. And you can dig into what that occupation is like. And then you could look at a specific region. I'm in the Bay Area, so I want to know what's going on in the Bay Area labor market and adult schools. We can see the skills and occupations have changed a little bit, but not hugely. 

The median wage has gone up a little bit in the Bay Area. And then we can see there are 49 job training programs in adult education, 2000 community college programs. And then we have all our data for all of the schools here below. Are there any questions about that process, or does anyone want to look at anything specific on the dashboard? 

BLAIRE TOSO: Janae has a question in the chat. She says, is the audience for this dashboard for consortium members, transition counselors, advisors, or is it also for students? 

ALLIE BOLLELLA: Yeah, that's a really great question. So we had originally designed this dashboard, really, to be for the consortium, and for counselors, and for staff to think about regional alignment, to think about alignment within a school, or thinking about programming, or changing, or modifying, or enhancing programs to align better with the labor market. And potentially for counselors, as well, to think about what offerings are available for transition, or to provide any kind of career counseling. 

But we haven't gotten a lot of user feedback yet on how well a dashboard works for any of those purposes. So your feedback would be really helpful. If you start to use this dashboard and like it, or don't like it, definitely send us an email and let us know how you're using it and how it's helpful. We definitely didn't design it with students in mind. We have a set of opportunity maps that are more static images, or infographics, that have a summary of this data, that we designed specifically to engage students. 

And we're working on including adult education in those because we thought it would be a little bit more friendly, a little bit more accessible and engaging for students. But definitely let us know if you use this with students and like it, and enjoy it. 

BLAIRE TOSO: So a couple other things-- yes. And I wanted to say, giving a plug for the future webinars is we're hoping to use the opportunity maps, and introduce those in the upcoming webinars. And we are excited, because they've always been basically non-credit community college focused. 

And now we have included the adult basic education population, so that they really create a lovely educational pathway for adults that include both the labor market data, but as well as those longer-term educational trajectories that might be available and useful for some of the students who have that opportunity. Just as a quick note, this dashboard is not linked to the launch board. It is similar to the CAEP fact sheets. 

It is on a different website. This one is designed and currently hosted by WestEd. And we'll be talking further with the chancellor's office about maybe moving it over to the CAEP website, and having it be under that purview. So you will need to use that link in order to access this dashboard. Let's see, what else? 

Someone wanted to ask, what if their school is not on the list? If you would like to email us, I'll repost our email into the chat, but definitely just reach out to us and we will do a little bit of digging. And then if we don't have your data then we will definitely send you on one of those sheets that you can either update, or that you may need to just start anew because we did not originally find any information on your school. 

Let's see-- can we post it on our website, on our career page? I don't see why not, but I'm going to ask Allie. 

ALLIE BOLLELLA: Yeah, you absolutely can. 

BLAIRE TOSO: Erin suggests careerzone.org is more student-facing, is the same way as ONET is, as well. And John wants to know, is it missing a lot of adult school data? Updated data, we would say yes. And I'll let Allie answer that more specifically. 

ALLIE BOLLELLA: Yes. I would say for updated data, we're missing quite a few schools. I think we only have a quarter of the schools that we know that have CTE have updated data for the spring. And then, of course, the old scan, I'm sure, has missing data that we were not able to find. I don't know how much missing data we have, other than for the updated. But if you find that your school is not on there, if you find that there is incorrect data, definitely reach out to us. We would love to update that right away. 

BLAIRE TOSO: John, to answer your question, we don't have access to that data set. He asks why we don't pull it from CASAS TOPSpro Enterprise. It's because we don't have access to that data. Also the CASAS TOPSpro is not complete data, because there are other non-credit schools that don't necessarily-- the non-credit community college programs. If they're CAEP funded and not WIOA II funded, they don't necessarily report into TOPSpro. 

They report directly into COMIS. Why no access? I believe it's a data-sharing memorandum that is not available to us. To access other organizations' data sets, there are a lot of different protocols around that. John, it has been up for discussion for many years. So thank you for that. And if you would like to advocate for that, that would be lovely. 

KELLY HENWOOD: Blaire, can I just add something? This Kelly from San Diego. So we're non-credit, but we're also WIOA II provider. And I'm wondering if it's worth, and maybe you've already talked to Dr. Zachary again about this, but right now-- so they're asking for course approval information from all the WIOA II and TE's. And it's the first time we've ever done it. And it's really interesting. 

So I'm trying to find out, we already go through the chancellor's process for certain things. There is a gap, in terms of data sharing. And I'm just wondering, because it sounds like, or it seems like, it looks like CDE is re-emphasizing, because they've brought it up at the last three meetings that we've had about course approval. And I just was poking through the TOPSpro Enterprise system yesterday about it. I don't know if it has TOPS codes, but there are some codes. I don't know what those codes are because I'm not a good TOPSpro person, I mean I'm good TOPS code person. 

But there might be information in there that people are starting to load, that they have to load right now. But we're trying to get clarification, myself and Mira Costa are trying to figure out, do we have to do this A-22 course approval thing? So we've never done it before, but it looks like, in the system, it's been in the system for the last two years. So I don't know. We're a little confused on the community college side. But it might be worth revisiting because it sounds like CDE's going there. 

BLAIRE TOSO: Yeah, thanks for that insight, Kelly. We'll talk among ourselves and see if that is a possibility. And John, you were asking if K-12 is excluded from this? No, this is largely focused on K-12 adult schools. Allie, do you want to just touch base on how you first initiated this project? 

ALLIE BOLLELLA: Yeah, so initially because we didn't have access to pull any of the data for adult schools, anywhere it's kept, we decided to do that online scan. So looking at websites, brochures, course catalogs, and we scraped everything off the internet and created a data set from it, which was flawed, but the best we could do at the time. And then we've done this most-recent refresh, where we've reached out, through CAEP, to collect K-12 adult school data, where available. 

And right now, we have about a quarter of the schools that we know have CTE. And we would really like to have as much accurate data as possible. So please bother us, and email us your courses. And we'd love to update it annually, too. We don't want this to be a huge drain on K-12 adult schools. But we also think it could be valuable. And so we're available, as needed, to help with some technical assistance or help enter courses, and talk to you guys about how to do that, and try to make it as easy as possible. 

BLAIRE TOSO: So Allie, when you did the scan initially, that was of the K-12 adult schools, as well, correct? 

ALLIE BOLLELLA: Yeah. 

BLAIRE TOSO: Yeah, OK. 

ALLIE BOLLELLA: Yeah. Oh, so there's a question about does the LMI data align with what's found in the 2021 fact sheets. These are the fact sheet dashboards in Tableau. Is that correct, Blaire? OK. It should. I think they definitely come from the same data source. I'm sure we used MZ in those. And I'm not sure what the date ranges are for projections of those fact sheets, but it should be, effectively, the same. Alex, you might have a better idea. 

ALEXANDRIA WRIGHT: Yeah, so we did-- thank you, Allie. We did, indeed, use the same database. The times might be varied, because last summer we pulled the data as of last summer for the CAEP fact sheets. And Allie, I believe you pulled data for this dashboard as of spring 2022. 

ALLIE BOLLELLA: Yes, I did it when I updated the other dashboard. So early this spring. 

ALEXANDRIA WRIGHT: Yeah, so I wouldn't imagine a terribly great difference in the LMI, because we were always using five-year projections, and using that same range of 2020 to 2025. So it should be just about right. If there's any anomaly, we'll make sure to look at it. Projections aren't including, just so everybody knows, projections at state and national levels are not including the disruption from COVID. 

So when you look at five-year projections in any type of California State EDD databases, or MZ, in general, economists are not including the COVID discrepancy and disruption because it was such an outlier. We'll see what happens after this, if we have some closures coming back around. And so we may determine that that changes in the future. 

But for right now, those types of disruptions are not included in this data, and will not be included for the foreseeable future, in any type of major economic database. But again, the data should be the same as in the CAEP fact sheets. Thanks. 

ALLIE BOLLELLA: Any other questions before we start to close out? 

BLAIRE TOSO: Janae is asking whether these will be-- that as programs grow, this will be updated annually. Yeah, I think that that's our hope. Of course, it's always dependent on funding and available resources. But one of the reasons we would like to update it annually is because people become accustomed to it. Also, the lift, once you update it, is so much easier. Because what we actually send you out-- if we have data on your institution, we send out an Excel sheet with all the information already populated. 

So it's really a review, and a correction, and an updating with anything that might no longer be being offered, or that you have new offerings. So for some schools, when they sent their sheets back to us there was very little updated. And it was a quick process for them. So, yep, that would be great. 

Let's see, I see Imagina, can you send the email or link to update our sites, please? Yeah, let's-- Allie, do you want to pop your chat, yours in? And I will pop mine in. You can email either of us, both of us, maybe both of us is the easiest way to do that. One of us will definitely pick it up. And John, thank you. 

We do hope, and in the future workshops will really be able to dig into that, so that people can look at their own data and then do that match. But yeah, and if it's missing a good portion of the K-12 provider data-- I think actually the K-12 data set is stronger than the non-credit community colleges data in what we currently have. And so that we had, really, a lot of data to begin with. 

And probably what's missing now is the updated data. And for that we really need support from your institutions throughout California, in order to keep the database most relevant. It's really because institutions don't necessarily have the opportunity to continually update their brochures, or their course catalogs, or have it in different setup, in different ways on the websites. 

It's very difficult for us to systematically get that information and pull that information, beyond the fact that it's an incredibly huge time commitment. That if we don't have the funding for, we're not able to expend those hours. So, really, it's one of the reasons why we're trying to engage people in the field to participate in this way, so that we can share the responsibility of having that data updated. Adele? 

ADELE MCCLAIN: I have another question. I basically did find one of the schools in our consortium on your data set. It's odd. Well, it's Hesperia. I'm Apple Valley. They're Hesperia, a little bit larger. They have ROP, where I only have matriculated to the college courses. But it says the title, like cosmetology, with multiple question marks. 

What does that mean? If you can. You may not know. It just has the two programs, but there's a lot of question marks next to it. Was that something that was put in by whoever submitted the data? 

ALLIE BOLLELLA: Yeah, so it could be that for values that were submitted incorrectly. Yeah. 

ADELE MCCLAIN: It's interesting. I will email you. I will take a look at this, at how other people put their stuff in. And then email the things that I know we have for each of the people. Our consortium lead is out on vacation, but she usually brings me to stuff as a co-lead. So I will also cc her on whatever I send you. Thank you so much for this presentation. You guys have done a lot of hard work. And it's a lot to look at. 

BLAIRE TOSO: Yeah, Allie has really done a huge lift on getting this up and clean. And as she said, when you get those question marks, it's not just a matter of making sure that we receive the data. But then, again, there's always that cleaning and scrubbing of the data. Paul, you said some state alignment would be helpful. Did you want to say a little bit more about that? 

AUDIENCE: Yeah, just the value that there could be achieved by having the top-level leadership have some alignment about these data systems, to address the types of things that John has highlighted. It's really, as you say it's labor-intensive to manually update. And if there could be more of a feed, that certainly would be nice. 

As I know, it would make your lives easier, our lives easier. And it's also an impossible task. So I just was being a little snarky. 

BLAIRE TOSO: Paul, thank you. Snarky or not, we really appreciate that. And yes, I think if there can be some advocacy from the field, as well, it both communicates that tools like these are relevant, whether it's WestEd who holds the contract or someone else is not the issue here. We're very happy to share it with anyone who would like to do this work. 

But yes, sort of the general advocacy of what tools are important, what information, and what data could be shared to make this easier, that would be lovely. 

AUDIENCE: And one thing that may help you, or help the field, to advocate for more aligned solutions, you mentioned the ability to sort by consortium. This tool is a bit opaque for Mother Load Central Valley. There's 21, or there's a lot of, counties in there. 15 counties. And it's hard to identify with that, from the consortium perspective. You look at the numbers, and it's all the way down to Fresno, Bakersfield, for San Joaquin and the North. 

And we have the surf regions now, that are a little bit more aligned to labor market approximations. So I don't know. I think that would help people just like look at it and validate it, and say, oh, yeah, this is-- we are in here. We're not in here. But if we did the effort to go inside of this, it's in a frame of analysis that aligns to our three-year planning, and aligns to our consortium work. 

So that might help people gravitate to this and advocate for some more systematic changes. But thanks, it's a great tool if we can get all these systemic issues fixed up. This is what we need. It's just getting it into that realm. 

BLAIRE TOSO: Duly noted, and duly agreed. Thank you. There are a couple other questions. Can you show how to access the micro region again, Allie. 

ALLIE BOLLELLA: Janae, do you mean in the filter, or the list of counties by macro region? Oh, yeah, the filter. So it's just this one. And you could select-- so it's up at the top here. These are the main two filters that will filter all of the tables in the dashboard. And you can select one area only, or you could select multiple if you wanted to look at that. 

And you could also select multiple sectors and look at a comparison. And you'll see that the bar charts pop up with a comparison of the sectors that you chose. No, it does not go to the county level now. That is for future work. There's some technical issues with that, in terms of identifying all of the schools, what county they sit in, as well as getting the labor market at the county level. But our next step will be consortium, and then hopefully county. 

ALEXANDRIA WRIGHT: I know somebody asked about the updating on the LMI on a regular basis. As much as we can update this dashboard with programs on an annual basis, we will update the LMI, too. Understanding that our economy is so complex and dynamic in the 21st century that we certainly need to be attuned to what's happening, particularly with emerging technologies and emerging occupations. And I know you guys want to be aware of all of that, as well. So thank you. 

I would encourage you guys, if any information is missing, summertime is a great time to get this work done. So please, please do reach out to Allie or Blaire. Let them know. We have a nice little spreadsheet, easy breezy access. You can enter all your programs. And tell us what's upcoming, too. If you have some funding asks out there, and you have some new ideas for IETs or CTE programming that is on the horizon, go ahead and include that in there. And just put a little note that says upcoming. 

And that way we know to be aware of it, and we can track you guys as well. So we appreciate your time and effort with this, to pull this in. And yes, any advocacy with state groups, as far as sharing of data, would be super appreciated by Ms. Bollella. Appreciate especially, thanks. 

BLAIRE TOSO: John, that's a great idea. Yeah. And I do think that that is exactly what San Diego is working on for their super-region. Theirs would be student-facing. And it would both allow learners to, if they moved, to be able to continue their studies without a gap, because they could easily find something being offered by another school that might be closer to them. 

So that is exactly-- you're reading their intention. I think if it's-- go ahead and move to the next slide. And I think we've done that. 

ALLIE BOLLELLA: OK, yeah. We definitely have. 

BLAIRE TOSO: I'm going to hand it over to my colleague Ayanna, who's going to bring us home. 

AYANNA SMITH: OK. Thank you all for hanging in with us throughout this presentation. We do have, as Blaire mentioned at the start, two more upcoming webinars that will go even deeper into this dashboard. So if you want to learn more, please come and join us for these upcoming sessions June 14 and June 28. These are two-part webinars. So if you attend the first one, we really want you to come back for the second one, as well. And we'll have Allie and Alexandria both presenting during those. 

So they're going to go a lot deeper and provide a lot more information for you all on this dashboard. And you can register for these at the same place you registered for today's workshop. 

BLAIRE TOSO: Can I make one correction? I apologize. It's June 15, and not the 14th. I missed that. We had to change the date. So it's June 15, and not the 14th. We'll correct it if it-- but when you go to register, it will say the 15th. 

AYANNA SMITH: Thank you for that, Blaire. 

BLAIRE TOSO: Super. We wanted to thank you all very, very much for joining us. And really, these conversations that we have, and your input, and your insights are really helpful for us in thinking about this work. And it goes beyond just this dashboard. But really, as we talk about system-wide ideas, and thoughts, and topics that it's a broader conversation that we know that you all wrestle with in the field, and we do as well. 

So thanks for participating and both informing this particular topic, but also for the way we think about our work. Our webinars and the feedback we get is important. And we've really applied it to development and professional development, as well as our data tools. 

Any last thoughts? Otherwise, I'm going to turn it back to SCOE, who I'd like to thank, also. They always seamlessly set up, manage registration and host our webinars. So usually I say it up front, and I forgot, that they are really wonderful colleagues and collaborators in being able to do the work that we do. Thank you. 

VERONICA PARKER: No worries at all, Blaire. Thank you all very much for participating in this afternoon's webinar. There was a lot of great conversation, as well as feedback that was recorded. So we'll have that information to be able to pass along. But thank you all very much for joining us. Holly has graciously added some links in the chat. Please, please, please complete the evaluation. It should only take you about two minutes. 

We want to get your feedback on this particular session, as well as any other professional development needs that you may have at this time. So please take a couple of minutes to complete that. We also have posted a link to the two upcoming workshops that WestEd will host on this particular topic, on June 15 and June-- is it 28? 

BLAIRE TOSO: I was going to say 28th, but I don't know if we should rely on my memory these days. 

VERONICA PARKER: Yeah, it's the 28th. So please register for those upcoming workshops and any of the additional workshops that we have posted on the website. We'll be hosting one on a research brief that was completed by our partners with High Road Alliance, on adult dual enrollment. And we also will have a information session on the distance learning cooperative, the Canvas LMS system. So register for that if you would like more information. 

Otherwise, we'll go with our closing. Again, thank you all very much for your time and your participation. And we'll see you next time. Have a great afternoon, everyone.