VERONICA PARKER: We're on. All right, so next slide, please. So today's agenda, so we'll be covering the purpose of the three-year plan, providing a high-level overview of the purpose, taking every opportunity we can to reiterate that, and then providing a high-level overview of the three-year plan guidance and the template. 

We did have a kickoff webinar back in August when the three-year planning guidance and template were released, but we'll just cover just a high-level overview and mainly honing in on the mandatory metrics section of those two documents. And then after that, we will move into the CAEP data for mandatory metrics. 

And that's where I will turn the presentation over to our colleagues at WestEd and CASAS, and they will go over how to use CAEP data and TE as well as LaunchBoard and other resources in order to set those targets for the mandatory metrics. And the CAEP funds projections, we will have a webinar next week in NOVA-- excuse me, on the three-year plan in NOVA. 

And at that time, we'll cover those funds projections because you utilize your NOVA information in order to set those. And then, we'll have a Q&A section of this webinar and then we'll talk about some upcoming professional learning, networking, and other resources that are available to you for your planning of your work through the three-year plan process. 

So next section, please, or next slide. So the purpose of the CAEP three-year plan, here is the purpose statement that is listed in the documents. I will not go over that word for word, but just to highlight a few key areas. So the plan, as you all may know, will let the readers know what's happening in your consortium and what the consortium's future plans are to address the needs within the region. 

The plan will highlight the three-year plan, the consortium-- excuse me, past three years' successes, areas of improvement, and the changing demographics. In the region, the plan will identify educational workforce needs. The plan will define and develop strategies and activities to meet goals over the next three years. 

And then the plan is an opportunity to have a collaborative engagement across the consortium, so all members and partners working on this plan together. The purpose of the three-year plan, next slide, please. And then we go into the Ed Code. So the Ed Code is listed on this slide. Again, I will not read it word for word but just point out a few of the highlights. 

So part of the Ed Code, the plan will evaluate need within the region. It will identify the current levels and types of education and workforce services provided to adults. And it will evaluate the funds available to consortium members. The plan will have actions needed to address educational needs. And then the plan will hope to improve member effectiveness as well as integration of services and transition to postsecondary and the workforce. 

Next slide, please. So now, we're going to go to the actual guidance document as well as the template and talk about a few sections in there. Again, this is just to provide the high-level overview and kind of the reminder or a review of what was covered in that August webinar-- that kickoff webinar-- with Dr. Carolyn Zachary, Gary Adams, and Neil Kelley. 

So we'll transition over to the guidance document and display that. And while we are transitioning to the guidance document, I will post a link to the actual guidance document. By this time, most of you, if not all of you have access to this document, but just in case. We want to provide the link to it now. 

And Mandilee, if you can add that. Thank you. All right, so this is a three-year plan guidance document. I have posted a link to the guidance document in the chat. So if you do not have access to it, there's your access to it. The guidance document, it was released in August of 2022. 

And it's a 30-page document that outlines the three-year plan starting with an overview of the three-year plan and different agencies' role as well as partners', et cetera, included in the three-year planning process. And if we scroll down a little bit, it also covers the actual three-year planning process, including the Ed Code associated with it. 

The next portion of it goes through each of the sections and provide instructions for each of the sections, so the sections of the three-year plan, an overview of the three-year plan The assessment section, metrics, objectives, activities and outcomes, and the funds evaluation section. 

And more information about the different types of data that you can use to support the three-year plan and the targets. And if you need technical assistance, definitely contact TAP. 

And then it goes through-- it talks about the submission format and timeline. So for the first time ever, the three-year plan will be submitted in NOVA, and again, we'll cover what that looks like next week on December 8. And so we haven't registered for that webinar. We'll post a link so that you can register for that webinar next week December 8 beginning at 12 o'clock PM. 

And the three-year plan, so it's not going to be available in NOVA until the CAEP office uploads to 2022-2023 allocations in NOVA. And then that will populate the three-year plan for you to begin inputting your information. And that will be done by February 28, 2022. 

However, the template, you will not be able to submit the three-year year plan in NOVA until after the May 2nd CFAD certification. And so what's going to happen is once the CFAD is certified and all member representatives have signed off on the C CFAD, then it will populate whom the signers will be for the actual three-year year plan. 

And so after that May 2nd deadline is when you'll be able to submit and approve the three-year plan in NOVA. And you will have until June 20, 2022, to submit your three-year year plan. And then it talks about some events that had taken place. And then it goes through the objectives. 

And so we'll scroll to the metrics section of this document because, again, we are here to talk about metrics. And so within that section, in the section 3, the first link that is included in this particular section is a link to definitions. So each of the metrics are defined. And when you click on that link, it'll take you to the bottom of this document, which is page 27. 

But we'll just go to actual section 3. So keep going, and it is coming up. Keep going. OK, here we are. So this is section 3 of the metrics section. Again, there is a document that's included as a part of this overall guidance document. It starts on page 27, and it'll give you definitions and resources for the metrics. 

And then, it'll go into each of the types of metrics that you'll include including your three-year plan, starting with the CAEP barriers and metrics. And for each of these sections, you'll have a description of the metrics, what the metrics are, and some associating data that you can use. 

And then the next section are guiding questions and resources. So for this particular section-- I believe there's only one guiding question. But it actually-- which barriers and metrics should we track to best measure the impact of our consortium's adult education services? So that question is provided to start the thought process of the different barriers and metrics that you should use to best track. 

And then the next column over provides the data sources and other resources that are available to you-- to help you begin to set your targets for the three-year plan. And if you scroll down to the next page, it goes over the consortium-level metric targets, and what each of them are, and the associating data, and then the member district-level targets and what each of them are and associated data. 

So we will go to the template. We will go into more detail about these different metrics. So if you can pull up the template-- and the template was again released in August. And it's a Word document. If you haven't accessed it already, it's a Word document that can be utilized to actually start inputting your information if you are ready to do so at this time. 

But again, you will input everything into NOVA instead of uploading it like we had done in prior years. So if you want to scroll to section 3, Mandilee, that will be wonderful. So in this particular document, especially for the metric section, one thing that we want to call out is that in Nova-- and we'll look at this next week when we go through the Nova process-- but you have dropdown menus that will have each of the metrics listed. 

But for here, we just have them listed in a table format, since we aren't able to do drop-downs inward, or at least we don't know how to up until this point. So section 3, again, provides you with a little bit of overview of what the section will be and the associated data. And then if you scroll down to the actual table, it outlines each of the metrics that you are to include in your three-year plan. 

So starting with the consortium-level metrics. So there are two consortium-level metrics that are to be included. One being the Number of Adults Served, so that metric includes adult [AUDIO OUT] obstruction, or receive services, and then the barriers that you are to identify. 

And so CAEP have already identified four barriers that are associated with the funding for CAEP. And so those are ELL, low literacy, low income, and long-term employment. And so the consortium will pick at least one of the four metrics to include in the three-year plan. However, the consortium has the opportunity to pick up to four. 

So if you want to address two, three, or even all four, you are able to. But you have to address at least one in your three-year plan. And then in NOVA, you will see the percentage of students in this category for the last three years, and you will project or set targets of what the consortium will look like for the next three years. 

And so the next set of metrics that the consortium are to include are at the member level. So these are the member district metrics, and the first two being mandatory. One is the percent of funds spent, so that's active CAEP funds only to be included. And the NOVA system will show the percentage of active funds spent in NOVA. 

And then the member will project the percentage of active funds spent down over the next three years. So all active CAEP funds and only CAEP funds. And the next metric that the member district will address is a mandatory metric. That's adult served who became participants, so those adults who meet the 12-hour mark and are a part of the outcome pool. 

And then there are 10 additional optional metrics at the member level. Each member have the opportunity to address any of these 10, and so those are participants who are in a high school diploma or equivalency, participants with EFL gains, ABE, or participants with EFL gains, ASE, participants with EFL gains, ESL, so on and so forth. 

And if we scroll to the next couple of pages, you'll see all of the optional metrics. There are notes associated with each of these metrics that will describe the metrics, so descriptions are provided, and then supporting data that can be used for these particular metrics in order for you to set your targets. 

All that I have for this particular section, again, it was just to be a high-level overview or review of what was presented back in August. And so I will now turn it over to Blaire with WestEd who will get started on the WestEd piece of this presentation. Blaire. 

BLAIRE TOSO: Hi, welcome, everyone. We're delighted to have you here. And we are going to focus on the primary resources where we will help you find and identify the metrics that you are looking at and determining which ones you're going to use and then also ways to analyze and apply some thinking around how you might want to identify, then go about conducting the needs assessment. 

So next slide, please. So first, I'd like to welcome my colleagues on here Randy and Jessica. You know Randy and me. We've been on several of these webinars. Jessica is new to our team and we're really delighted to have her here. I'm going to have her just come on camera, say a few words. 

I think she'll be very useful in-- because she knows the system from the inside out. She's had experience with consortia data, TOPSpro, enterprise data, MIS data, and she has also served as a researcher for community college. Jessica. 

JESSICA KEACH: Thanks, Blaire. Hi, everyone. My name is Jessica Keach, formerly Jessica Noel. I was with Southwestern Community College District before I joined WestEd a few weeks ago. So I'm really excited to be here. Like Blaire said, I had experience with our Adult Education Consortium as well as MIS data. So I'm excited to be here, and I look forward to working with all of you. 

BLAIRE TOSO: Thanks, Jessica. And I just want to note that I'm going to do the primary part of this presentation, but then I have to leave to go to another presentation. But Jessica and Randy will both be on in order to answer questions and dig into work or the questions that you might have. Next slide, please. 

So just to reiterate what Veronica was talking about is and highlight the key components that were covered about, the purpose of the three-year planning, it's inclusive and collaborative process among your consortium members. It's a guide for over the three years. So when you're looking at setting it your targets, I know you get to go back and revisit them each year, but thinking long-term as well as just in the immediate that you're conducting an assessment-- needs assessment. 

To determine both what you're currently offering and what you might want to offer in order to provide education and workforce services for adults. And then to identify some actions that you're going to take. And so we hope that we have created some resources that will support you in this process. Next slide, please. 

So the first thing that you do-- how do you set your targets? And where do you start? And what do you want to do? And so like all of these questions. Do we want to increase the number of serve? Do we want to highlight identified populations? And one of the ways we're going to talk about is, how you want to go about this. 

And we suggest that you look at your data and then make a decision as opposed to coming at it with an already formed question, so going sort of take that perspective of it. And if I'm fast, then we'll get to the data equity walk, which will help guide that process. Next slide, please. 

So as I said, I think that we are all determined to use the data to inform our decisions. And we always think about data in three different ways at WestEd. There's the Interesting data, which helps us to notice that something's going on. There's Useful which really highlights structural issues and trends. And then there's Actionable data, which really helps to guide our decisions. It tends to help us answer the what and the why. 

So we really reside in that interesting and useful, which will help provide some context and begin to answer some of those questions. But it's important that when we're talking about the data and the resources that we're using, you'll see where are-- we talk about the data and then we say and now push it a little bit further, and we'll provide you with some strategies for doing that so that you can look at what's happening. 

So here are some of the highlighted and primary tools and resources that we have developed for this three-year process and ongoing and across the years. Today, we're really going to talk about the first three, which is the adult education pipeline dashboard. We'll show you the guide to using the adult education pipeline data, and then the CAEP fact sheet, which is also accompanied by a how-to guide. 

The other three, the regional education workforce dashboard, the low-income policy map, and the ACS county commuting patterns, we'll cover in a deeper webinar, but they are important to be able to dig into those pieces. Up front, we're going-- this is a really quick overview. We have roughly 20 minutes given to us. 

And just know that we are preparing longer webinars, and we'll also be developing some extended professional development, where we can really begin to walk through some of these resources and people can get their hands dirty and do some activities. So here is what we lay out sort of as a basic four-step process to guide your thinking as you begin to refine your targets and design the-- and make some decisions around what you want to offer or what you want to put in your three-year plan. 

And that's the document. What's going on? Who are you serving? That's like, what's currently happening? What's needed is looking to the future and identifying successes and challenges, I mean, probably those gaps where you might want to go by asking, what's actually going on in our consortium? 

And then, there's the gauge like, what are your community needs, what are your partner needs, bringing in that broader community of stakeholders. And then lastly, identifying the goals and then determining what you want to do, so the analysis to identify that. Slide. So we're going to get started with adult education pipeline, which offers a comprehensive look at your consortium data. 

I think most of you already know what the adult education pipeline is. It really is key to offering you all the metrics you need for your three-year plan. And we also have designed it to provide drill-downs that allow you to explore aspects of your data, such as equity by gender, by age, by race, ethnicity, and also by program. 

In other of our webinars, we would do a walk-through, but we're just going to just call that out and then identify a few feature resources that you can use to dig in. Next slide, please. So Veronica went over through those metrics, the four that you all need to address. We have created this resource, which is-- it's a very simple, straightforward. 

It tells you the metrics that you need to address, and then where exactly you find them on the dashboard. I will pop a link into the chat so that you can access that. So that will list out exactly where you find those metrics on adult education pipeline as well as give you some strategies and some thinking questions to guide you through your three-year planning. Next slide. 

The other thing that we think is really important is that the adult education pipeline allows you to really drill down into some of the-- to refine your thinking about the people who you're serving and what their outcomes are. And so we put together an infographic that explains how to use the filters on the adult education pipeline for the different metrics and how to begin to refine that. 

So you're not just looking at a collective group, but you can drill down into the gender, race, ethnicity, as I said, and then student type and program type. And this really helps you be able to think more deeply and refine your thinking as opposed to just saying, oh, we're serving the Latinos really well. 

Well, are you serving all Latinos? Or are you serving a subset of that? What is your-- what is the purpose? And who are you really serving, and through which programs are you serving them? And I will pop that link into the chat as well-- hang on, one second-- so that those resources are accessible. 

And I'd like to mention that these are all on the adult education resource library through CAEP. And there are a host of other resources that we'll include how-to guides if you're looking for coding, if you want to have an overarching orientation to the adult education pipeline that aren't just looking at equity or they aren't just focused on the metrics. There are a lot of resources in that library. 

I would take a moment to browse in through there. They're all downloadable and accessible and straightforward. So they can help guide you in other aspects of planning or orienting to some of the aspects of the adult education pipeline. So then, let's talk about the CAEP fact sheets, which is something that we have revised. 

There's new updated data, and we've added some additional components to that dashboard. And most of you should be familiar with it. I will go ahead and pop that link into the chat as well. Well, it'll take me a minute. I must have dropped it out. While I'm talking-- all right, I'll pop it in a minute when I'm not trying to negotiate presentation at the same time. I apologize. 

I thought I had it up and ready to go. But the CAEP fact sheets are-- we expanded them. And we'll walk through these for the four tabs and tell you a little bit about them. I did want to call your attention to that we have developed a how-to guide, which you can access through that question mark, as you can see within the red box. 

If you click on that, it'll be dropped down and the how-to guide will be right up front there. And the how-to guide will provide you with the background information. So for example, I have a screenshot there that we'll talk about the key elements and what you can expect to find on the different tabs. 

And there's some general information about how to use that, how to use each of the tabs, and what the information is, how to use the filters, as well as it'll provide you with some case scenarios, which you'll really be able to see how those are-- the contacts, there are contacts-different scenarios so that you can help analyze and make decisions based on the data that you have, that you're able to access through these pieces. 

So the walk-through on the CAEP fact sheet, so we have the first tab is one that you all are familiar with, although the look is a little bit different. Tab 1 is your consortium demographics, and this is on the ACS data. I would like to say that currently it is the '15 through '19 ACS data. 

We will be updating this in December as soon as the census data is released. So it will be updated into the newest-- into the most up-to-date demographic information. And as I said, that's supposed to be released in December. We're keeping our eye on it. And we have an analyst and our Tableau person ready to work through that. 

Each of these pages are filterable by your consortium, so you can drill down. We're showing the statewide data, but you can look for your own consortium data for each of those. And really, if you take the top at a glance, these are your different population characteristics, which will-- once you click on them, you can filter to align to the target metric. 

So for example, this one that's showing here is the overall population. But you can also look at and click on Adults With Disabilities, Foreign-Born, Limited English, Near Poverty, High School Attainment, Unemployed, and Disconnected Youth. These are key, and they align with some of the key metrics that you need to address. Once you click on one of those, the data refreshes and will show you just the information for that demographic, which will really help you identify areas of sub-service. 

Tab 2 offers you a snapshot to quickly view your key demographic points on your consortium as compared to who you serve in your consortium. So on one side, we have that ACS data. The other is the data that pulls directly out of the adult education pipeline. And you can, again, filter that using those bars on the right-hand side, so you can look at some particular groups within your adult education pipeline data. 

I will say that while this is new and we think it's a great add, it also is one that you'll want to go ahead and once you've identified some of these pieces, you'll really want to dig into it on the adult education pipeline, as I said, using some of those equity drill-downs that you can really think, have a more refined look at what's going on in your adult education programs. 

So one of the reasons why we put these up here is because using these two data sets, you can ask questions, such as these that we've got on the slide to set your targets or hone in on some of your targets like, who are we reaching, who aren't we reaching, and then compare, are those learners representative of the most in-need of our consortium demographics. 

And then-- and maybe you are, but how well are they doing, so you look at the in-need population and you look at the outcomes. And then from there, you can choose what kind of metrics that might be available, that you think would be important for your consortium to address. Next slide. 

One of the things that we know that everybody has been working on is on transitions. And so we pulled out transitions to postsecondary here. And this is tab 3. Again, you can also use the filters there to look at more distinctly. But we wanted to help you see what kinds of transitions are being made and who is making them. 

And this is a side by side view for a quick comparison. You can see the categories on the screen. And this will help you to more easily identify gaps and successes. And I did want to be clear that when we're talking about identifying targets, that will be-- they will be-- sorry, I totally got distracted by the question that popped up in the chat. 

Yes, we will send an email once the CAEP fact sheets are updated, so thank you for that. That's a great question. So we'll get that posted probably into the CAEP newsletter. What this transitions will help you do is, look at them, dig into them, and then prompt some questions as you look at them as a full set of data. 

And this is informed from the data that's on the adult education pipeline. So what we've done is drawn those in so that people can begin to see the consortium demo information next to your adult education um outcomes. Next slide. Lastly, to focus in on those workforce needs as part of your plan, we have an LMI Data tab. 

And this will help you not only just identify what your community's needs are, but we also in the how-to guide prompt you how to think about-- how do you want to identify your targeted services, how do you want to create possible pathways that will help support those transitions either into employment or thinking about higher levels of postsecondary education. 

And you can. The filter's up on here on the right-hand side will allow you to look at educational level, what experience is needed, and then whether there's on-the-job training. 

And this is really to refine that. The lower piece really helps you think about the cost of living and living wage jobs so that you're not simply-- if you're thinking about one of those isn't necessarily a living wage job, this will hopefully prompt you to think about how those pathways will be able to be structured because you do have the ability to long-term plan across these three years 

Any questions thus far? Super. All right, so next slide, please. Can we go back just one more? Sorry, Jessica, I didn't notice that it would already switch. You're faster than my brain. So this, we have not-- this was a board that one of our analysts created, where you can really dig into some more refined comparisons by consortium, by a select demographic, or a subgroup. 

It is not part of our CAEP work, but we think it's important to share. And there are different ways that you can look at educational attainment. And there's really, for example, here are the barriers, which we know is one of the mandatory metrics. You can really begin to look at what's going on there. 

And we particularly hope that it will offer both a side by side as well as a way to think about different populations and categories within those populations and to refine your thinking about, when you're setting a target who do you really want to target and be specific about those strategies that you're going to be able to come up with. 

RYAN WHETSTONE: Blaire. 

BLAIRE TOSO: Yes, sir? 

RYAN WHETSTONE: Was that one in LaunchBoard? 

BLAIRE TOSO: No, that one is not in LaunchBoard. That is work that WestEd did, not funded by CAEP. It was just when we were looking at some of the data and preparing both the LaunchBoard and particularly the CAEP Fact sheets, we thought there were some really interesting trends. But it wasn't part of the work that we were contracted to do. So that is really currently-- it's a set-aside. 

RYAN WHETSTONE: Do you have a link to that? 

BLAIRE TOSO: But it's public. It's available to everyone to look at, but it's really quite interesting. 

RYAN WHETSTONE: You have a link to that data set? 

BLAIRE TOSO: It is embedded in that AEP data story. Let me see if I can pull it up really quickly. 

RYAN WHETSTONE: OK. OK. 

BLAIRE TOSO: Ryan, how about this? When I hand it over to Jay, I will post that link in there. 

RYAN WHETSTONE: All good. All good. 

BLAIRE TOSO: OK, Thank you. 

RYAN WHETSTONE: I just know. I wasn't familiar with that one. 

BLAIRE TOSO: No, it is brand new. And we have not really discussed it in any of our webinars. As we do something a little bit more about digging into it, we'll go into it further because there are some really interesting ways to use it. It's not necessarily the most intuitive, which is one of the reasons why we're just doing the brief overview. 

But I really think it provides some important information, which is why it's plugged in into this webinar. But I will get that as soon as I get off screen and do that. 

RYAN WHETSTONE: Thank you. 

BLAIRE TOSO: I just have few more slides. Yeah, absolutely. Yeah, it is interesting work. So I wanted to just talk a little bit about-- we talked about an approach. And so-- and coming at it instead of with an already set question is to use a data equity walk. And this will be something that we'll do more interactively in one of our longer professional developments. 

But I did want to overview it for people who may not be able to attend that, have the time. But it's just a way to think about it. And this is really out of the work from the Education Trust-- West. And we have included a link to their toolkit if you want to explore it further. But it's really relevant for focusing on individual and collective exploration. 

These are activities you can do within your consortium with your members, even if you're just working with your staff in an institution to explore things this way. And it's really relevant for exploring and discussing equity gaps, disparities. So I'm going to take you through it really fast. I think I'm coming to the end of my time, so let's go ahead and move through this. 

So for example, what you might want to do is start with just looking at your demographics on the CAEP fact sheet on that first sign-- the first ACS page. And you would start looking at your data, for example. Just like looking at this and thinking, oh, look at that data, and who could benefit from adult education services. And Lori, I'll touch base on your question in just a sec. 

Next slide, please. And then, you think, oh, well, look at that. There was a really high level of No High School Diplomas. So let's check on that. And when you click on that, as I said, it will refresh. And then you get the data on those who have no high school diploma. And so looking at this, what would you say is needed if you take a moment to look at this? 

You can see that there are several opportunities available to you. If I look at it, I'm like, OK, look, there's two ends of that. What's going on in the 16 to 20? What's the opportunity on that? You probably wanted to dig into it and find out how many of those are out-of-school youth. 

You can look at the English language Ability and wonder, where are we targeting that? And so you choose-- you look at it and you discuss, what are the opportunities? Without making judgment, just a thinking activity. Next slide, please. 

JODEE SLYTER: Blaire, you just said 16 to 20 on that previous slide? 

BLAIRE TOSO: Yeah, well, it could be-- I have to admit. Very small on my screen. I believe we did the '16 to '20 because I know a lot of the WIOA programs include those. And so we wanted to make sure that was inclusive and it's also the opportunity youth as far as the-- 

JODEE SLYTER: Well, it says 19 or less. We're not talking about potential high school students or K-12, we're talking about only adults. 

BLAIRE TOSO: It should be-- it would be-- I believe it's-- now, you're challenging-- I have to go back and look. I believe that we did think about it as 16 to 19 because we wanted to be able to capture that population that might appear. But I would have to go back. I apologize. I will verify that. 

But I believe our thinking was we wanted to include that. So yes, you would need to go back and see who actually is in school and who is out of school within that demographic. 

JODEE SLYTER: OK, because that's really confusing when you're looking at what the qualifier is, what the definition is there, 19 or less. So it's really not under 16-year-olds. 

BLAIRE TOSO: No, it is not. It is definitely not under 16-year-olds. So that aspect of-- yes, CAEP is 18 and older WIOA II is 16 plus. Again, trying to find that balance between what the field needs and how they want to look at their data. So thank you. It is an excellent question, and that was where we went with that. 

And if we get a lot of feedback that the field would really only like 18, starting at 18, that would be great to know. And then, we can adjust that when we update the ACS data. We'll rerun those numbers. 

JESSICA KEACH: Blaire, I have a question just real quick on that page, just a clarification . When you select a consortium, so let's say adult education pathways, and then overall population 36,000 pops up, that's for our county, correct? Not-- 

BLAIRE TOSO: That would be for your consortium. 

JESSICA KEACH: Consortium. OK. 

BLAIRE TOSO: It is for your consortium and there is a whole process that our data analyst did in order to set those. And if you are interested in knowing, email me and I will dig up the language and send that on to you. 

JESSICA KEACH: OK, thank you. I just needed that clarification. 

BLAIRE TOSO: Yeah, and thanks, Paul. Yes, we'll clean up. We can look at that. This is lovely. This is the nice thing about refreshing all the data is that we found another error and we will update that. And then also we will be able to clean up some of those. So I made a note on that too, so we can be more specific like put 16 to 19. 

All right, perfect. Thank you. So just quickly, then we go on. You can compare it to who you're serving and what you're doing by using the adult education pipeline data and then compare those. And then, you can dig into them. So for example, if you've chosen that you want to look at high school attainment, then you can dig into that piece and look at how you're doing, who do we want to target, what would indicate progress. 

And you can compare. For example, here are this might be. This one looks at the race ethnicity. You can also then look at age. And when you're looking at those different ages in that population you're surveying that the percentage, do you want to dig in? And look at that opportunity, school youth. 

We're kind of focused on that because we really think that that's going to be an area of need given COVID and what happened with educational attainment during that time and leave-taking from school. And so we were thinking that might be an interesting space to explore. 

And so just returning to your setting of your targets and looking up but this four-step process and the resources that you can go through and help support that process. And I just wanted to address one of the questions about where the data on the adult education pipeline comes from. It comes from MIS data and the TOPSpro enterprise data. 

What we do is we put together calculations that make these-- the different metrics equitable and use and then those get uploaded through to the dashboard. 

VERONICA PARKER: All right. Thank you, Blaire, so much. And thank you all very much for engaging in this portion of the webinar. We will now transition over to Jay who will talk about CASAS data, and then we'll have time for more questions and answers. Jay. 

JAY WRIGHT: OK, hello, everybody. I just moved over to my PowerPoint. I guess if somebody has-- somebody can respond kind of one way or the other, can you see my PowerPoint or is it still something else that you see on the screen that's not my PowerPoint? 

VERONICA PARKER: I can see it. We can see it. 

JAY WRIGHT: So wonderful. Everybody can see it. Trying to make a long story short here. There is an abridged version of this PowerPoint, and there is an unabridged version of this PowerPoint. I guess the good news for now is, today, we'll be covering the abridged version right here. So I'll go through this today. 

A lot of you were at the meeting we had earlier this morning. So we did a few little bits and pieces of this one and the other one. And basically, we'll post some of this on our website here on Monday, I think, both the abridged and unabridged version. The difference, I'll just say, is this one kind of a-- we have limited time so there are some extra slides in the other one that provide a few more examples, provide a few more screen shots. 

There is a lot of how to get there from here that I kind of leave out because we don't really have time to go step by step, but we are really just focusing on what these metrics are and then just trying-- for not all of them, but for most of them-- just give a concrete spot and see where you can find it and what on that report you could use to measure the different metrics. 

So I'll start with this one. This one has been out there for a while. This is basically what Veronica just presented. There's the mandatory consortium metrics. There's the mandatory member metrics. And then there is that list of 10 optional metrics. So I'm starting by listing those out. We'll go in this order, cover what some of the metrics are. 

There will be a few like percentage of funds spent that we'll obviously skip. But we'll try to go through at least some of these and point out here is at least what I think the report is and TE that you can use. I'll point out now before jumping in, as there's obviously probably multiple options for a lot of these metrics. 

So if you know good and well, and looking at the participants list, I know some of you do know good and well that there's different ways of doing this. I'll just kind of stick to one that I think is pretty firm for each of these. So here's the two consortium-level metrics. Veronica brought this up. There's the four different student barriers. 

You can choose one. You can choose all four. You can choose a number in between 1 and 4, if you prefer, and then looking at those with the one or more hours of instruction. So we'll start with barriers. You can look at barriers and aggregate. You can look at each barrier individually. There's two general reports in TE you could use. 

There's an NRS, one that relates it to federal reporting. And then there's a CAEP Barriers to Employment. The format and basic information that these two reports, way closer to the same, that it is different, but it just looks at the different population. CAEP, for the state reporting group, obviously. NRS, limited just to VABEAS, ESL that are included in NRS. 

The other key thing in TE to consider for barriers is the ad hoc NRS cross tab report. We talked a little bit about this one in the meeting we had earlier this morning. So this one, we suggest if you really want to focus in detail on one of those barriers and/or if you want to look at barriers in terms of outcomes, I would say these first two are mostly-- they are just to give you a bean counting. 

You know what date. You know how many students with each barrier you're presently serving. If you want to look more specifically at how well individuals with barriers, they're doing on specific outcomes, you can potentially use this ad hoc and NRS cross tab report. Pretty much, any possible metric you might measure in TE beyond just what's in CAEP, beyond just what's in NRS. 

You name it, it's pretty much there. You can just basically select what you want to serve as the x-axis. Select what you want to serve as the y-axis. And it's kind of a do-it-yourself report to look at individual categories. So again, this is an example we've been using in particular for if you want to focus on barriers in terms of how well they do with outcomes and not just the number of individuals with barriers you happen to be serving. 

The next one we have is that enrollment criteria. That is we're looking at how many make it to a program or looking at how many get in to the one or more hours of instruction. So we're saying there's a relatively new report called TE Enrollees by Hours. I'll just say in the unabridged version, we get into this report a lot more. I'll keep it limited here. 

Some of you know about this. I'll just say earlier this morning-- though, not as many of you know about this maybe as I thought. I'll just see there's two relatively new reports that measure attendance hours in TE. This one I'm showing on my screen, it's called TE Enrollees by Hours. There's another one called TE Service Enrollees by Hours. 

These basically look at those three buckets that Veronica, Neil, and others have been talking about a lot here over the last several months. That is it's looking at those different areas we already have on the CAEP summary. And then, the Enrollees by Hours is not a consortium-level report the CAEP summary is. 

So this is getting more to student-level agency data with this particular report right here. We're looking at maybe making that a consortium-level report. This one relies a lot on drill-down. So I've got to say that's a little bit of an "if" because we keep agency-level data out of that consortium-level level login for a lot of reasons, but we are looking at it. 

In any case, this report basically looks at those three sections of the CAEP summary and then just kind of digs a little deeper into those three buckets. So it looks at outcomes and then itemized out how many have 12 or more, how many have 1 to 11, how many have 0 hours. It goes into the Services section and does the same thing, dividing it into those three buckets. 

So that the answer here is, you can look at those right-hand two columns in the Services section to basically determine how many of those that are in services have at least one hour of instruction. We highlighted the columns you could potentially use. There is a slide that has math and, in effect, makes it a little bit better. 

But you can see we've got the two columns. If you add those two columns, we've got 609 with 12 or more, 98 with the 1 to 11. So if you add those together and divide it into the total number, that shows that right now at this level of reporting, we have 82% of our students with at least one hour of instruction. So again, that's the TE Enrollees by Hours for this. 

Those are the first two in that first category. Here's the member-level metrics. Percentage of funds, obviously, I'll skip this one. And then we're kind of taking that Enrollees by Hours to the next level at the member level to look at how many have the 12 or more hours of instruction and actually qualify for official CAEP reporting. 

So we're going back to the same report called TE Enrollees by Hours. We're looking at a slightly different section of this report. That is, we're looking at the middle Outcome section of the report instead of the right-hand Services section. And so now, we're just saying, OK, how many stay long enough to get that 12 hours of instruction. 

So we're looking at the same thing here. We're looking at that 580, looking at how many you get, so you can see we've got 92% of that 580 right now have at least 12 hours of instruction in this exam. So again, if you're not familiar with this report, I'll refer you again. 

There's an unabridged version that has close to double the number of slides with a lot more detail orientation on how to generate these reports, drill-down options, different ways of looking at it, trying to compare the different reports that you can figure out how they relate to each other, a lot more of the software detail that we don't really have enough time for in this presentation today. 

So here's the longer list of optional metrics. We won't go over all of these, but we'll go over five or six of them. The short answer is, for most of these, you can simply use a CAEP summary. There's a couple others that I want to point out. A couple others, when we're looking at adding a couple of things to make it easier to highlight the exact numbers. So I'll try to point out that too. 

I'll say this will be posted Monday morning. I might say, hey, we haven't gotten it posted on our website yet. Our web team would say, hey, that's because he hasn't sent us anything yet. And they'd be right. So I'll be sending all this here at the end of the day. By Monday morning, you should have this and the unabridged version posted. 

I'll just say, Jenée, you're asking, so I know you were at the meeting earlier this morning. So we have that page where we post a lot of our hand, the monthly-- WIOA PowerPoint and a lot of those documents we use the monthly network meetings, we'll have it there with all that monthly networking information that you already know and love. 

So here are the optional metrics. I won't read them off. Veronica already did that at the beginning of the presentation. We'll look at a lot of these. We'll start with HSE and HS diploma. That's column H on the CAEP summary. Start with one that should be pretty easy to figure out, I think. 

We're looking at this Outcome section that includes all the different outcomes you can achieve outside of pre- and post-testing. Column H is the one that has a high school diploma and HSE results. We're sticking simple. We're going to get a little more in the weeds than just this. But we're going to keep it all above ground for the moment. 

The next one is postsecondary. That's another one of the optional metrics, that simply one column over to the right in column I of the CAEP summary, again, that same section. Our postsecondary outcomes are included in column I. Again, the same read that we just talked about. 

Sticking with simplicity here, there's also an optional metric for transitions. The Transitions outcome is over here in column L on the CAEP summary. Just like the other ones we've talked about, it just itemized it out by program has the unduplicated count and so on. So the only difference here really is you can see that little white arrow is bouncing around just to show you which column it is. 

I'm repeating and repeating while giving you the same answer for a lot of these. Because the simple answer is, yeah, you can use the same report for a lot of this. Last but not least, we've got that immigrant integration indicators or I-3. So I-3 appears as column F. Again, same report, same section of the report, same format. 

Column F as your different numbers for those that pass the COAAP and pass the I-3 out. This is a generic one. We'll just stick with I-3 because I have the example readily made. But you can look at outcomes the same way, comparing to the total numbers. So here's just an example where we used column E on the CAEP summary, which is the total number of enrollees that qualify for outcomes with the 12 or more hours of instruction. 

And we're just taking a look. Again, it's flexible, I think, in terms of whether you want to use these for a specific program or whether you want to run it across all programs. In this particular example, we're sticking with the numbers here for ESL with the idea that that would be the most logical program to evaluate I-3. 

You can use all programs, though, if you thought that was more appropriate for sure. So we're saying, hey, out of our ESL population, we've got about 46% that have passed at least one I-3 outcome looking at those that pass I-3 and simply dividing it against the number of enrollees in column 3. 

The I-3 assessment, so first off, I'll just repeat, to be clear, I-3 stands for Immigrant Integration Indicators. It's the outcome and the issue that was highlighted by AB 2098 there a few years ago. I think Paul Downs is here. He led a discussion group a couple of years ago, looking at how we can measure progress for immigrant students. 

We've had the EL Civics program for quite a while and WIOA II. From our point of view, we have many different EL Civic's COAAPs that we would say have been actively measuring immigrant integration now for 20 years. We would, obviously, say immigrant integration is anything but new in California Adult Education. 

We feel like that's what we've been using EL Civics for, for quite a while. I'll just say the focus group that looked at this agreed. So now, we're using EL Civics additional assessments as ways to measure immigrant integration. A little bit longer answer without getting carried away is we work with LLIs few years ago, I'd say, to our mutual benefit. 

They really thought EL Civics was a good way to move forward with their metrics. We thought this was the right way to go. So we've done a lot of work with LLIs over the years. And I've got to say the work we've done pretty strongly and resoundingly, I believe, you're hearing from the CASAS guy, so go figure. 

But pretty resoundingly, those immigrant integration metrics and the COAAPs you already know and love had a very, very high correlation. Thanks for jumping in, Paul. That really helps. Great. So good question, Edward. OK, so moving along here, so I don't run too far over. I think I've got five minutes. 

There's another one you can run. It's in the same setup window as the CAEP summary. Again, I'm not going to dwell on. TE Navigation here, but there's a more detailed report called CAEP Outcomes if you don't want the generic number for transitions, or you don't want the generic number for ASE. 

You can get in the weeds if you want. We've got CAEP outcomes. It appears as this nifty grid. So you can see it's looking at I-3, ASE, postsecondary, and so on. It's taking all of those different categories that I referred to there in the Outcome section of the CAEP summary itemizing out exactly what individual outcome or checkbox or bubble, whichever way you want to look at it, and then by program, exactly how many students, by program, got each individual outcome. 

We are going to make some updates. I'll stick with an easy one that you don't need, but I think this is the easiest example to point out what I mean. So there is a little bit of guidance in using this report that we agree we probably need to do. For the ASE outcomes, you could look at diploma as a standalone outcome. 

You could look at HSE as a standalone outcome. Or you could do it in aggregate. So we'll start parsing this out, so you know exactly which of these light items pertain to which side of it. Transitions is another one that we noticed where there's a transition to CTE and a transition for college. 

So we need to sort out exactly which is which to make it easier. So we'll probably add a couple rows here, so you can get into the weeds without having to do any math, I guess, kind of cutting to the chase the best way. Yeah, that's what we're talking about is the COAAPs are actually what create the I-3 outcomes, just so you know. 

I might be stealing Diana's thunder, so correct me whatever you need to do, Diana. But that just to cut to the chase, Edward. That's what we're talking about these I-3 outcomes are students that pass the EL Civics COAAPs. OK, so the next outcome relates to EFL Gains pre- and post-testing. That's the left-hand section of that same CAEP summary report. 

So you can see here's a little snippet with the enrollees showing you how many completed that pre/post pair in column C showing you how many made the gain and achieved the EFL Gain in column D. 

So you can go ahead and compare the columns C, the number with pre/post if you want to look at overall learner persistence. You can go ahead and compare column D to that column B if you want to compare the percentage that make the gain. I think I'm just going to go to the next slide, where I think we have some math to make this a little bit better. 

So here is the math. If you go ahead and look at column C divided by column B, that fits your persistence rate, by persistence that is, what percentage of learner of learners in each program stuck around long enough or quote unquote "persisted" long enough to at least complete a pre and post. That's what we're talking about. 70% is a good rule of thumb for that. 

And then column D divided by column B, those are the ones that made enough gain to achieve that EFL Gains, so you just do column D divided by column B. And that will give you the performance rate. You can also look at some of these metrics using the DIR. This is just one example. 

Some have said, yeah, we use the DIR more than the CAEP summary. So yes, as long as you stick with it and you stay consistent with the metrics-- so here's one where you could look at 12 hours, 0 or empty hours, 1 to 11 hours on the DIR and use that. You can basically organize your metrics similar to what I showed you on the CAEP summary. 

If you're a WIOA II agency and you choose to use pre/post or EFL, you could use the CASAS data portal. It is limited just to WIOA has used. So it will be useful for some of you, but not all. But I'll say if you really want to dig in to pre/post data, the user friendliness on the data portal is probably the best and a lot of you have used it. You can dig in. You can go all the way back to '04, '05. 

Another option is back to that ad hoc NRS cross tab. That's a good one to look at goals for special populations. You can look at barriers. You can look at any demographics category you can possibly think of. So this is just an example where we're looking at-- we just cherry-picked a couple barriers. 

Nothing to it other than what we picked, and just said, OK, we want to focus on some of these areas. And so who's earned an MSG, who hasn't. So this is just some examples if you want to focus on that sort of stuff. And that's it. I'm showing that I'm a minute over, so close to perfect timing. 

But I'll just say, any questions? And I'll just say, obviously, I skipped a ton here that went exactly 20 minutes as far as I can tell, but again, that unabridged version, if you want to get into [INAUDIBLE] steps. And we'll probably have a more detailed data dive here in January, early February, where we'll kind of get into how you can really dig in and look at it and drill-down and stuff like that. So I'll stop sharing and turn this back over to Veronica. Thank you very much. 

VERONICA PARKER: Thank you, Jay. And thank you to everyone for your participation up until this point. We are now at the point where we will take any questions that you may have. We still have CASAS on the line as well as WestEd on the line. So if you have any questions to ask of them, now is the opportunity to do so. 

OK so we have one question from Lori. "What's the formula for persistence again on the CAEP summary?" Jay, do you want-- 

JAY WRIGHT: Whoops, I violated my mute rule, didn't I? I muted myself. Sorry, Yeah, it's column C divided by column B. That is the number who have pre/posts divided by the number of enrollees either by total or by program. Thank you, Tricia. 

VERONICA PARKER: All right, any addition? 

LORI HAWKYARD: Jay question. I'm sorry. I thought you said that was the performance rate. 

JAY WRIGHT: No, that's D. So column B is enrollees. Column C is those who complete a pre/post. Column D as in "delta" is the one that shows the one that has the actual gain. 

LORI HAWKYARD: Thank you. 

VERONICA PARKER: So Crystal says she saw eight optional metrics from Jay's slide but heard 10 from earlier. Did I miss something? No, you did not miss anything. In the template, all 10 of the optional metrics are listed. And I'll provide the link to the template again so that you'll be able to see all 10. They are on different pages, but you'll be able to see all 10 of them. One second as I pull the link again. 

MANDILEE GONZALES: I can add it for you. 

VERONICA PARKER: OK, thanks. So you'll be on pages-- the optional metrics in the template start on page 7 and go through page 9. So pages 7 through 9, again, on three different pages are the 10 optional metrics. 

JAY WRIGHT: All right, Thanks. You saved me a trip having to bean count on my slide there. 

VERONICA PARKER: No worries. And another question from Eric. "Where/what is the best resource to determine if students who have graduated or earning a living wage?" A question from the template. 

RANDAL TILLERY: So the LaunchBoard actually tracks this as a metric in the employment and earnings part of the pipeline. The caveat there, of course, it's done based on a UI wage match. So students who don't have Social Security numbers are not counted as a part of that denominator. But it will tell you, for at least those students what we can make the match to EDD, which ones are making a living. 

VERONICA PARKER: All right. And then Annabelle's question, "what do these metrics look like in LaunchBoard or somewhere else for community colleges not using TE/CASAS?" 

RANDAL TILLERY: We have metrics in the LaunchBoard aligned. And these are all-- a lot of the data is being pulled in NOVA comes from the LaunchBoard. So if you need support in terms of finding out where in the LaunchBoard, if you want to dig deeper into any of those metrics for looking further, you should reach out to us, and we can help provide pointers for you. And also Blaire identified the resources. I think there's some information in the resources as well that can help with that. 

VERONICA PARKER: OK, and then Jenée asks, "the numbers that are found on the fact sheet are taken from our consortium numbers. That's the question. "An example, the number of people without a high school diploma are calculated from the number of students in our consortium, not from the total population of people in the counties are consortium serves." 

RANDAL TILLERY: Is that a question or a statement? 

VERONICA PARKER: The first part of the question, [AUDIO OUT] "is the numbers taken from our consortium numbers?" She's confirming. 

RANDAL TILLERY: Oh, so the number of adults without a high school diploma in the demographic parts of the fact sheets are for the area you're serving, not just your consortium. 

VERONICA PARKER: All right, I'm not seeing any other questions at the moment. Here's one. "So agencies who do not collect Social Security numbers cannot utilize the LaunchBoard to determine the graduates/living wage data?" 

RANDAL TILLERY: Currently, no. This is actually something-- Jay, do you know? Are you collecting information that would actually identify whether they're making a living wage in your survey? So for WIOA Title II agencies, CASAS conducts a survey because it's one of the supplemental collection methods. And so if we have that, it would be there. We're also going to start displaying CTE outcome survey data, which will also capture some of that. Jay? 

JAY WRIGHT: Yeah, it doesn't really do living wage. It asks the student whether they make a pay raise to respond to that item on NRS table 5. The living wage, though, I got to say, it's not really part of that. The median wage is what they use. So we end up having to calculate that kind of after the fact. We clean it. [INAUDIBLE]. 

RANDAL TILLERY: Just in a general sense, the challenge with employment and wage data is there's no absolute way to collect it where you actually get comprehensive information. So you have the wage matched to EDD, which gives you just data for students with socials. 

CASAS administers their survey, and they get a certain response rate, but not necessarily everybody. And then the CTOS survey also captures like 30% to 40% of the students who were surveyed. But also, it's not completely summative. 

VERONICA PARKER: All right, thank you, Jay and Randy. And Jenée, I see you have your hand raised. You want to unmute? 

JENEE CRAYNE: Yes, please. Thank you so much. So I guess this is for Randal, and I'm not sure if Blaire is still on the call. I feel like I've misunderstood what Blaire said. I thought that I heard her say for the fact sheets that those numbers were taken from our consortium data. And so I just wanted to clarify. 

But the numbers that are found on the fact sheets, like, what you're saying for a number of people who don't have high school diplomas and for age, demographic numbers, those actually just come from the counties that we serve. It's not necessarily the students that are in our consortium. 

RANDAL TILLERY: Yeah, it's actually-- it's more refined than that. It's not at the county level. So we actually did calculations to identify your specific service area. But it is the population. Now, I'm not as deep in the fact sheet, so that's something I'm going to talk to Blaire about a little bit more. 

But we actually don't have, I think, precise data that we receive that identifies how many adults don't have a high school diploma necessarily in your consortium. So I'm pretty sure it's the area you're serving. I mean, it's hard because there are views and the fact sheets that are based on your data. 

And then there are views based on the community because we're trying to get people to really compare who are we serving, what are their characteristics, and what's the characteristics of our community, and how do those line up, and how can doing that comparison help you get more specific or intentional about identifying populations that you want to actually go after and recruit. But we'll follow up and double check. 

VERONICA PARKER: And JoDee, you have your hand raised. 

JODEE SLYTER: Yeah, just to follow up on that. Randy, looking at the fact sheet, this is the population that's within your consortium, but the label on the age group-- because that's pretty significant for 19 or less really is 16 to 19. 

RANDAL TILLERY: 16 to 19. Yeah, and that was identified earlier. We've already made a note of that. So we're going to be repopulating the fact sheets with the new census data. And when we do that, we'll make that correction. 

JODEE SLYTER: All right, and that would be great. But my point was also going to be, I understand that the alignment is intended to be with WIOA, but if we're looking at in CAEP, who we can serve with CAEP, and we're talking about 16 to 18, that really doesn't tell us what to do with our CAEP funds. 

I appreciate the alignment with WIOA, but from a consortium member and lead standpoint, I would rather it were 18 and up, not 19. Anybody else want to weigh in on that? 

TODD LIVINGSTONE: Yeah, I would agree with that. We were funded for 18 and older. So that makes a lot of sense to me JoDee. 

RANDAL TILLERY: So we can look at that as well, JoDee. So part of the push-- originally, the LaunchBoard itself was only focused on 18 and up, and then the push was because a lot of adult schools do serve 16- and 17-year-olds. And the disconnected or opportunity youth are served 16 to 24. 

The people like to know those numbers as well. So we'll discuss that internally and see if we could do something, maybe just with the labeling, so you can really identify-- get more refined if you're looking at what you can do with WIOA as opposed to what you can do with CAEP. 

JODEE SLYTER: Thank you, would appreciate that. Because if those are the numbers we're basing our plans on, that's really critical. 

VERONICA PARKER: OK, thank you, JoDee and everyone else who chimed in on that and said 18 and over. I'm seeing if there are other questions. So Jenée, I think this is an additional question. 

She asks, "beyond the one mandatory barrier, do all of our strategies that deal with the 10 CAEP student barriers and metrics need to be listed in the Metric section of the three-year plan?" I believe you're referring to the four primary barriers that are listed in the three-year plan, Jenée? You want to unmute yourself? 

JENEE CRAYNE: Yes, sure. Thank you so much. Yeah, so I guess my question is, with the four, if we are just choosing one-- like let's just say, ESL, ELL learners, because that is a metric that all five of our agencies can track. 

Then therefore, if we're obviously going to have different strategies and objectives further down in the plan, do we have to have all of our strategies aligned with the metrics or barriers that we choose? Or can we have other strategies that then are not reflected in those metrics and barriers that we choose in that section? I'm not sure if that's clear. 

VERONICA PARKER: Not totally. 

JENEE CRAYNE: So if we choose to-- that we're not going to track everything because we're not able to track them across all of our institutions. And so we want to know, can we, then, create strategies that we're not tracking? I mean, obviously, we're tracking them in a way, but it's not tracked in the Metrics section. 

VERONICA PARKER: Yeah, and I think Jay just came off mute. 

JAY WRIGHT: I did. Could you hear my packing? 

VERONICA PARKER: Yes. No, I saw your mic. 

JAY WRIGHT: Oh, OK, well, I'll just say this came up a little bit in the meeting I did this morning. So I can't really give the official answer, but I will paraphrase some things that Neil said when these sort of questions came up. So what he basically said is you can use the data system of your choice. 

You can use COMIS. You can use LaunchBoard. You can use TE. You can use your third party SIS, et cetera, as long as you're consistent with using the same SIS every step of the way, so it's apples to apples. Different data systems might have different ways of handling some of these metrics. 

But you can't go completely contrarian with it. You can't do two quarters after exit, for example, and just do the number, who entered military or got a better job or whatever. You have to stick with the method. You have to keep the same methodology. 

I don't know if that makes sense, but you need to use the metrics as intended, so all the data systems are kind of measuring it the same way. You can use which data system you want, but you kind of have to follow the metrics as they are. 

VERONICA PARKER: OK, and Jenée says thank you. 

JAY WRIGHT: If I get contradicted, you can safely assume I'm wrong. That's just my interpretation of the way it came out earlier this morning. 

VERONICA PARKER: OK, thank you, Jay, for that clarification. I believe this is an additional question from you, Jenée. "Is it possible to have the label reflect where the data is coming from?" Are you referring to the data on LaunchBoard, or the data on the fact sheets? 

JENEE CRAYNE: I was just referring, sorry, to the labels that Randal was talking about for the fact sheets so that we know which ones are coming that are actually-- that are consortia data and what one is our county data. So when we're comparing them, we know which one is which. 

VERONICA PARKER: OK, Randy, did you have an answer for that? 

RANDAL TILLERY: Sorry, I muted myself. Yeah, so the data sources should be referenced there somewhere. But what we'll do is we'll look at how they're identified in ways to make it clearer when you're looking at if there's an issue with their interpreter. 

VERONICA PARKER: OK. And she says thank you. And then another question from Ed, "is a central region in the Northern or Southern Central Valley region?" 

RANDAL TILLERY: So the Central Valley Motherlode region is pretty much everything from Kern up to San Joaquin. And I and I think most of the corresponding the mountain counties are actually a part of that region. 

VERONICA PARKER: OK, does that answer your question, Ed? And Sherry has shared a link for Ed and Lori. Additional questions? OK, and Jessica has shared another link for the-- 

RANDAL TILLERY: --symmetric definition dictionary. 

VERONICA PARKER: OK, thank you. All right, I'm not seeing any additional questions at this time. We have posted several links in the chat that are related but unrelated to this particular webinar. One being the link for next week's webinar. So if you have not registered for that webinar on Wednesday, December 8, definitely take the opportunity to do so. 

We have also posted a link for our previous webinars on the three-year plan. So we hosted a number of webinars back in September that focused on different components of the three-year plan as well as the directors event. And so those links-- we posted the links in the website, to the website, in the chat, so you'll have access to those previous webinar recordings, just in case you need to go back for future reference. 

And I see something came in-- OK, so Emma says, "when we review the metrics in NOVA, it may become clearer on how we select them in the dropdown menu." Yes, absolutely. And I just lost my train of thought. Oh, we also posted the link to where the template in the guidance document are both located, as well as that initial kickoff webinar where each of the sections, we were going through, a little bit in more detail. 

So if you need to reference that for any reason, you have access to that. And we have about five more minutes left. So if you have any last minute questions, definitely feel free to post them in the chat. Otherwise, we will prepare to close. But before we do, we are looking at some additional opportunities for us to come together to work through the three-year plan process. 

And one being a three-year plan peer learning circle, where we would ask the field to identify questions that you have regarding the three-year plan process. And then as a group, will come together to solely work through those questions and other topics that may come up in the chat if you-- or you can raise your hand. 

But in the chat, if you can just state whether or not you will be interested in something like that, please let us know. We will be hosting them on a monthly basis, January through March of 2022. And so we are just trying to identify if the field-- this is something that the field would find useful. And so I'm seeing some justice come through. 

So thank you all very much for saying that you will be interested in that. We are in the planning phases of that. But before the winter break, we will release information about those upcoming PLCs. So outside of that, I don't have anything else, and I do not see any additional questions, so I will prepare to close this monthly-- 

RYAN WHETSTONE: Veronica. Quick question. 

VERONICA PARKER: Go ahead, Ryan. 

RYAN WHETSTONE: Did you say that the consortium-level metric target data, is that going to be prepopulated? 

VERONICA PARKER: Yes. 

RYAN WHETSTONE: The three years of barrier metric data? 

VERONICA PARKER: Yes, that will be prepopulated into NOVA. 

RYAN WHETSTONE: OK. 

VERONICA PARKER: So the number of adults served and the barriers. Yes. 

RYAN WHETSTONE: And then we enter our targets? 

VERONICA PARKER: Mm-hmm. Yep. 

RYAN WHETSTONE: OK. 

VERONICA PARKER: All right. So yes, that is all that I have. If anyone does not have any additional questions, I will definitely leave the room open. One of the things I have been seeing in our evaluation data is that we close the room a little too abruptly. So I want to give everybody the opportunity to grab any links or resources that were shared in the chat so that you will have access to it. 

You can use the three communication dots to save the chat to your device so that you'll have it for future reference. And it will hold all of the links as well as the conversation that has taken place during the webinar. We'll give everybody a chance to do that. But again, if you have any additional questions of CAEP TAP, the CAEP office, CASAS or WestEd, definitely feel free to reach out. 

You can submit a support request via TAP as well as email us tap@caladulted.org. And we will work with our respective colleagues to get you the answers to your questions. And they have also shared their email addresses if you want to contact them directly. And so if there is nothing else, than we will see you all next week. Hopefully, we'll see you next Wednesday. 

JENEE CRAYNE: Veronica, just piggybacking on what Ryan just asked, so in the guidance, when it says "will be populated," will all of then those will be prepopulated by into NOVA, not just the one that Ryan brought up but anywhere that it says populated? 

VERONICA PARKER: Right. 

JENEE CRAYNE: OK, fantastic. OK, thank you so much. 

VERONICA PARKER: You're welcome. All right, thank you, all. Everyone have a great weekend. Thank you.