Jay Wright: OK. What I did for starters is I just sent the PowerPoint through the chat. So if you want to have a copy, I just sent it. We'll probably send it again here once we get more late admits. coming through. But I just sent the PowerPoint, so everybody should have that now. So you can go ahead and download it through the chat. And I'm going to go ahead and move my stuff over here.

I think I am sharing. Can everybody see my screen? I am going to just go ahead and blow it up. I guess before I get too far, I'll just quickly sanity check everybody hear me OK, everybody see the PowerPoint, everybody have the opportunity to download the PowerPoint who wants to, et cetera. It sounds like everybody's more or less okey dokey on things like that. That's my story, and I'm sticking to it. So I'll go ahead and move over here to the next slide.

Actually before I get into the agenda, I will bring a caveat to this. My very first one is, hey, good to have you all. Everybody's welcome. Please, everybody stay. You know everybody is welcome here on hanging house. But in any case, everybody's welcome. But I will point out that when we targeted this date between CASAS and tab, I will point out that the official target audience for this webinar are the far northern agencies.

We did a statewide webinar in September and another one I think mid-December that was definitely for everybody. We opened this up to everybody because that's the nature of online registration. And hey, if you're really eager, it's great to have you. But I'll just say again, the reason why we're doing this is because we wanted to have something for the far North.

And I think everybody kind of agreed doing that region in person just is a lot more difficult than doing Bay Area or Sacramento or Central Valley in person. It's just a little too far flung. So for this region, we had to admit, hey, this one just has to be online, whether everybody likes it or not. So again, everybody's welcome, but we are looking to get input from far northern agencies.

Mostly this is the same accountability everybody else is hearing, but there are a couple slides where we're looking for input. And in particular, there were a couple things where we got some input from some of you in the far North. And we also know that there's a lot of issues that you have a little bit because you're far flung, a little bit because you're rural, smaller, all kinds of issues where there's just things that come up typically with some of the more far flung rural agencies in the far North that if you're in LA or Sacramento or San Diego or whatever really isn't as big of an issue.

So I'll just say, hey, San Jose, Bakersfield, Oxnard, you're welcome. But just know we're going to talk about kind of what happens at smaller rural agencies, and we're going to go out of our way to do that. Just saying that may not apply to you, but that's kind of an on purpose here for this session. I'll stop talking about that. Back to the agenda.

This agenda, the same agenda as our other statewide webinars, same agenda as some of the in-person. I'll also note a few of you I know were in some of those in-person trainings we've been doing here lately. Good to have you back. I do think a lot of the ones we've done have been really successful. It really has been good to hear from everybody.

My big victory declaration with the in-person trainings I think mostly relates to we had a little list of CASAS where we kind of thought these were problems, we kind of thought there were some other things that were going well until COVID hit and then they became more like problems. The more we hear from agencies, the more it seems like a lot of the concerns agencies are having are not exactly the same but definitely very similar to what we thought.

A lot of you have come to me and talked about problems with CAEP reporting. Some of that relates to things like workforce prep, some of it relates to things like short term services, but a lot of things that CAEP accountability. I think a little more so than WIOA too, maybe more of a boots on the ground element. I'll just say the same thing I've said in all the in-person sessions is I kind of feel like statewide COVID data collection-wise, when you look at the WIOA side, everybody was still attuned to payment points.

Everybody miraculously still managed to do a really good job with pre and post testing, even though it was in the middle of a pandemic. I'm sure some of you don't want to detail how you did it, but just about all of you did get it done. Again, things like payment points, EL Civics, everybody did pretty well and remarkably well, all things considered.

But once we get a little more detail-oriented with things like bubble boy slides, occupational skills gain, short term services, that's where that other shoe kind of dropped. And the superstar is not the ones that were lagging, the best of the best are the ones that came up and said, yeah, there were definitely some challenges with this. So on some level, we're kind of hitting the reset button and going over basics, on some level, we're really trying to figure out what are some of the issues, what do we need to tighten up.

A couple of you brought up in emails to me about LAO. A lot of that discussion has been happening. That really has triggered a lot of really good discussion on hey, we need to look at this probably more earnestly. I would love to hear what you have to say about this. You might want to hear some of these issues though before chiming in.

So we'll do a programs update. This will be where we make up time. I'll talk about the same sort of things as the in-person trainings, but it will be consolidated. But some real basic overview things to get everybody's foot on the rail as well as a little bit of a reset button, some things we were talking a lot about when we were doing the road show in '18, '19, some things that have come up here more recently, more kind of an update on current events. We'll try to be brief.

That's what we also mean by reviewing key trading issues. Some of you were around when we did a lot of those webinars back in 2018, '19. If you were, you remember the same questions always came up. So pointing out a few of them that still seem relevant and still seem kind of problematic for some, we'll talk about that. And then we'll do an overview of outcomes and services.

We definitely don't have time to look at every single outcome in every single service, so don't worry, we'll spare you on that. But we will give you the overall overview of that structure. We will point out some of the highlights. Again, not pointing out all, but picking out a few of the outcomes and a few of the services that tend to generate questions the most, try to point out why they're confusing to a lot of people or what some of the trouble spots are, hopefully give you a chance to bring that up as well.

Do the same thing with services. Got to say, services is an area that people have been mindful about, but reporting was really difficult during COVID. Sometimes I think you are doing them, but recording it is difficult. Some of you are more like, hey, give me a break, it was a pandemic. So some of those counseling situations just didn't really happen during COVID like they would in a normal time. So we'll go over that. I encourage you to tee up and come up with all the problems or tales of woe you can think of. We really want to bring as much of that up as possible.

And then it will really be flummoxing to those of you that are new or bringing the brand new stuff up in the middle. Some love that, some don't. But I kind of feel like getting the overview and getting everybody's feet wet with CAEP kind of is helpful. And then OK, now we've got CAEP and we all are really the same. When we were doing the road show four or five years ago, that wasn't true.

But that is kind of another one of those funny things happen on the way to the forum issues is, hey, we're out of COVID, and all of a sudden a lot of these things we've talked about and WIOA II accountability, really 99.9% the same for CAEP accountability. So we'll show what those are, talk about some brass tacks with things like entry and update record, quarterly data, and that sort of thing.

This is not a TE training, but we will spend a few minutes at the end at least going over a couple of those biggies, making sure you at least a couple of those report titles. And then that last bullet is solely for your viewing pleasure. There's about 20 goal setting slides. There's no way on God's green Earth we'd make it if we do anything other than nothing about that. But hey, you've got it in the PowerPoint.

And I'll just point out, it feels like eons ago, but in the spring when we were doing CAEP training, we spent just about all of our time talking about goal setting. Hopefully, all of you remember and you remember that you have goals, and that everybody, as far as I know, is required to set goals at the agency and consortium level on or before June 20th. And April, May, it felt like a pretty big stinking deal. Now that it's the following January, it feels a little closer to forgotten.

But hey, we will show you how to set goals, ways to use TE to establish good measure goals, lots of good ways to verify progress, all kinds of good stuff for your viewing pleasure. So I'm sure that table-setting was way too long, but I will just kind of stop and say, hey, hopefully that sort of relates to what you were expected to get yourselves into here this afternoon.

It looks like the number of participants has stayed solid. It doesn't look like anybody's run to the hills yet, so that's good news. Number one, I guess. It looks like it just upticked from 58 to 59. Anybody with any comments? Everybody's staying eerily quiet here. Uh-oh, that doesn't sound good. All right, I will keep talking.

So we'll start with a real basic overview again, some basic level things that we really feel we need to kind of talk about. So this was really a big deal, I would say, more when we were still a BG, it still applies with CAEP. That is the basic statewide structure. We have 71 regional consortia. The consortia are based on community college boundaries. It's done that way because the legislature says so.

We've got the 71 consortia. Again, the college boundary is typically a lot larger geographically than K12 boundaries. I don't think there's any K12 districts that have multiple colleges, but your typical college boundary usually has multiple K12s. So for those sort of practical considerations, it's based on community college boundaries. And the way we have it set up in CAEP land is we have what we call members and what we call partners.

If you're a member, congratulations. You're cordially invited to the consortia whether you like it or not. That means you're a college district, K12 district, county office of ed, a few ROPs I think fit in that formula as well. That is if you're a college or K12 district, you're part of your own regional consortia whether you like it or not. You're a member. You need to participate. Congratulations.

Every consortia also has the option to bring in partners, and a lot of you have done that. And you bring in your Title I agencies, some libraries, jails, CBOs. It's really up to you. A lot of you in your communities have had regional organizations that feel like they should have a seat at the table, the way the legislature is written is it's up to you the consortium to decide this. If you want to step to your college and K12 districts and leave it at that, that's OK. If you want to reach out and bring in a bunch of partners, that's also OK.

Again, the legislation just stipulates if you're K12 or community college, you're there like it or not. And then you have the option to bring in partners. Here's the nifty little map that kind of shows how that kinda shows how that is outlined. Looking at program structure from a completely different lens, here is the structure based on instructional programs.

So here we have five programs. Some of you that know the system really well start scratching your head about now and wondering why it says five instead of seven. I'll just say it's more of a superstar question than a new person question, but there is a little bit of a difference. This is how it's set up in Nova. Nova-- or CAEP land is set up to roll up the programs. It's really set up to have these three super programs represented at the top of this graphic, ABE, ESL, and career tech ed.

I'd argue the other two that you see here, adults with disabilities, parents/K12 success, nobody could really quite slot them into these three big areas, so they're kind of floating on their own. But know their structure to streamline it to the maximum extent possible to roll into these three programs. Just to try to explain what I'm talking about here, here's an example of how it's rolled up. I'll start in the middle, ESL. Not really any difference. ESL is ESL.

Over to the left though I'll point out it's called ABE. A lot of people would kind of understandably, I'd say rightfully, argue that hey, it's not just one program, it's actually two. Or really more appropriately, three separate programs, that is, it includes ABE, it includes high school diploma, and it includes high school equivalency. I'll just say at least in TE land, we would definitely say that's three separate programs not one. But in CAEP land, they really make an effort to roll that up into one single program called ABE, and it includes these three that you can see on the slide.

Career tech ed over to the right has a very similar process. Not quite as straightforward, I guess you might say as ABE. But nonetheless, it's rolled up. You can see we've got CTE. In short term CTE, we also have pre-apprenticeship, we also have workforce preparation. I'll just point out, in TE land, those bottom two are completely separate programs.

Just again to step out a little bit more because people are starting to wonder what the heck the difference is, again, these are not official definitions, but just to try to let you get your foot on the rail on how you might distinguish them. Obviously, CTE and short term CTE, one of them is obviously long term and one of them short term. They're both occupation-specific. There's not really an official CAEP definition of where you separate short term and long term. Everybody always ask.

The one number I've given is 48 hours. That's actually a college number. The only system I know that gives a number is the colleges. So that's the number I use is 48 hours, but that's unofficial. But again, it includes short term and long term CTE. Pre-apprenticeship like CTE is also long term and specific to an occupation. Obviously, the delivery model completely different. Workforce prep, on the other hand, is obviously workforce-related, but it's short term and it's not occupation-specific.

If you're doing things like resume writing, job interview skills, workplace safety, et cetera, that is instruction that relates to the workforce, but not necessarily a specific occupation. That's what we would call workforce preparation. So just to kind of bring it all to fruition here, this is the way TE would define the structure. And again, some of the superstars are kind of wondering why there's a difference, so we're trying to point out what it is here.

So in TE land, again, we've got ABE, ESL, and-- what is TE land? OK, sorry, I'm getting a little tune on basic. Just hold on for a minute. But again, we've got the three at the top, we've got adults with disabilities and parents/K12 success. I'll just say added pre-apprenticeship and workforce prep to this diagram to show in TE. TE stands for TOPSpro Enterprise.

I made a really big deal about, hey, we have newbies after COVID, so you're keeping me honest. There's a lot of you that haven't heard any of this crazy stuff before. I'm going on saying TE land understood, you have no idea what the heck I'm talking about. So again, TE, TOPSpro Enterprise. I'll just say for my own edification, when JoDee typed out the acronym in the chat, does that help you or does that just make you say a follow up question like, hey, what the heck is TOPSpro Enterprise? Inquiring minds need to know.

So when we say TOPSpro Enterprise, you do know what that is. OK, good. So TE is the acronym. In CASAS land, and hopefully that's more familiar to you than TE land, we use the acronym TE all the time. We probably assume everybody knows that acronym as well as we do. Obviously, no one does. Thank you for catching me there. OK. I'll stop there. For some of you that have been added, that should be helpful. If you're new, in context with the, hey, what the heck do you mean by TE land, it might be semi helpful, semi not. Bear with me though. I'm trying to bring it back over the last few years.

Sanity check, everybody hanging in there because we've got a mixed audience in so many different ways. I ask that all the time anyway, I'm going to be a little bit more obnoxious about that than even I am normally because we're the epitome of mixed group. So I'll just say thank you a couple of you for chiming in. I'm going to take it and run.

So here's a few of these. I tried to consolidate a little bit, keep it from running amok. But here are some topical issues. Again, I'll say and CAEP land, these are some things that we've been talking about for a while. Some of you have heard this constantly, some of you not so much, but we'll start with goal setting. I would argue the first half of 2022, everything I talked about CAEP-wise is related to goal setting because that was the big evolution last year.

Like I said, it feels kind of like it went with a thud because I hear nothing about it anymore. But nonetheless, we all have goals. We all had to set goals at the agency and consortium level in June. I kind of do feel like there were a lot of folks and their bunkers for a while. With the budget out now and the LAO report now, it feels like everybody's come out of their bunkers.

I do kind of feel like now that everybody's coming out of their bunkers, that we'll be hearing a lot more about those goals that we all dutifully set in Nova last June. So that's one big issue. Adult served. That's something we talked about for the first time in 2019. We brought it up as the three buckets that really didn't make any sense to anybody then. Hopefully it makes a little bit more sense now.

But by those three categories or three buckets, we're referring to hours of instruction. I'll just say, I've talked about it a lot over the last year because CAEP has-- that is we're having a lot of new tracking related to how many students have zero hours of instruction, how many students have one to 11 hours of instruction, versus how many students are participants, that is how many have 12 or more hours of instruction.

When I bring it up that way, does that make sense to you? Yes or no? And again, I'm going to continue to be obnoxious this way. I got to say that what on Earth do you mean by TE was a good wake up call for me. I want to make sure I'm not just talking in acronyms or whatever. Everybody know what I mean by the three buckets when we're looking at those different areas of hours of instruction?

Again, the mandatory goals relate to that quite a bit. We'll have more on this throughout, but that's a big issue that we have been talking about the last few years. Another related issue is hours by program. This isn't the three buckets. But before COVID, we had a couple of fire drills I remember kind of that year leading into it. By fire drill, that is there was a big state need to figure out the total number of hours your students were accruing.

So in TOPS Enterprise, there's a report called hours by program that just spits out the total number of instructional hours at your agency by program. That is how many total hours do all of your ABE students collectively have, how many in ESL, how many in HSC, whatever. That is you're sending your hours by program so we can see the total number of hours you're serving at your agency and your consortium.

The big issue is what if you've got multiple programs, that is, you have a class that has more than one program assigned. How are you going to figure that out? The default answer in TE is it will basically treat every single class 50/50. That is TE assumes everything is evenly split. So just to give an obvious example, if you've got an IET class that maybe you're targeting for both CTE and ESL at the same time, TE is always assuming that it's 50/50. So it will credit happy hours to ESL and half the hours to CTE in my example.

If you know good and well the right answer is really more, 75% ESL and 25% CTE, that's fine, but then that means you need to create two CASAS, not just one, and more carefully allocate those hours so all the ESL hours get credited to the ESL class and all the CTE hours get credited to the CTE class. That seems like so 2019 because it is 2019. But I bring that up because it was a really big deal kind of the last time we had these trainings.

And I will point out, related to a lot of the things we've been discussing over the last two or three months or so, in my opinion, this is probably going to make a comeback. It wouldn't be surprising if we have a couple of these fire drills here in the next year or so. So just remember, that's hours by program. A similar issue is service hours. And I'm waxing nostalgic, but the big issue of program year 2018/19 in CAEP land, sorry all, I'm going to keep using those goofy cliches, is whether services count as hours of instruction.

At that time, half the state said yes, half the state said no. For about a year when we were doing trainings, we just shrugged and said, no, do whatever you want. We gave those kind of answers for about a year. I remember it well Summer Institute 2019, by the way, in Garden Grove, California where we'll be in 2023. And I did a training and we brought this up. It was a little bit of acrimony I remember at that session.

But we did our homework and we found out pretty clearly that the answer is no, service hours don't count looking at federal regs. When they say instructional hours, it doesn't necessarily have to be in the classroom. COVID obviously taught us that our instruction need not be in the classroom, that's one thing we definitely know now, but it does need to relate to that instructional program. If you're doing counseling, that's wonderful, but it's not part of the instruction. So yes, you should track that you're doing that and it's great that you are, but it's not technically part of one's hours of instruction.

Another one coming down the pike I think that we've talked a ton about, WIOA II. We really haven't talked about it as much in CAEP. I don't think it's collaboration with your workforce partners. That, of course, continues to be a top priority in WIOA land. But again, in CAEP, there's some reasons why I think it will start becoming a bigger deal. Caroline yesterday mentioned that governor 130 million, that is. There's some funding for health care, in IT, for English language learners, there's a new funding coming out with that.

It does really look like that's going to involve career tech ed. If it does, that definitely means that we'll probably be more of a CAEP collaboration issue much more so than WIOA II because of course, CAEP includes career tech ed and WIOA II generally doesn't. So I do think there'll be a lot more of that collaboration addressed by CAEP, not just WIOA II, moving forward.

All right. So I'm picking my spots here. I'm not following up on all of these issues. But here is a little bit on goal setting. I'll point out at the end of the presentation. I won't rattle them all off, but there is a slide that lists all of these if you want the official list. But just to summarize, these are how we're setting up goal setting for CAEP.

So there's two mandatory metrics at the consortium level. There's one involving four barriers. And I think it's what low-income English language learners, low literacy. And there's one I'm forgetting. Maybe somebody can kick my head in and remember the fourth one. But there's four barriers. I know the number is four. I'm remembering three out of the four but I'm drawing a blank.

But there's one involving four different barriers. It's just a simple one to show that you're serving learners with those four barriers. And then by enrolled adults, that refers to those three buckets. At the consortium level, the requirement is to be able to show you're moving as many students as possible from that zero hours bucket and moving them into the one to 11 hours bucket. That is so you can at least call them enrolled adults or adult served. If they have zero, you can't really call them that. But once they have one or more hours, that is a little more tangible. So if you can get them to that one to 11, that is better.

Than the mandatory member level metrics, there's a funding metric. I won't get into that. That number of participants is the other one related to the three buckets. That's the next step forward. So at the agency level, the mandatory metric is to show how many students you're moving from that one to 11 hours bucket to the participants bucket, which is the 12 or more hours. That is for CAEP outcomes, for pre and post testing.

For WIOA II federal reporting, that bottom line requirement is for the 12 hours of instruction that is you need to have your student become a participant. That's the one that's required at the agency level, and for many reasons. Obviously, the target for all students is to get that 12 hours of instruction so they qualify for outcomes and they get on all those reports. And when we show all the wonderful things we're doing to the legislature, that 12 or more hours is what gets them on the map so we can show all those great things.

And then the optional level, my understanding is every agency needs to choose at least one. Again, there's a list later that you can look at if you want, but I tend to feel like all the things on the list are pretty much as expected. Pre post gain, high school diploma, high school equivalency, occupational licensure, get a job. All the things that you're tracking are the things that are on the list for these goals.

And then here's a little bit more on what I'm calling the three buckets. Again, we brought this up. This is a slide from 2019. Nobody really understood this then. I think everybody kind of does now. Number one, we say service only students. That's our term for zero hours of instruction. Again, they're students that you're serving, but they're getting services, not instructional hours. Then bucket number 2, they've enrolled in the class, they've accrued hours, but they're still less than 12. And then those that have continued long enough to qualify for CAEP reporting and qualify for federal reporting, that is they get the 12 or more hours.

Here is the same concept from a TE point of view. This report is the CAEP enrolees by hours. I'm calling it new, but it's not really new anymore. I think we released this one in mid-2021. But again, you can see the three buckets here where we're looking at that CAEP summary. We're dividing each of those areas into the three buckets. You can see those two sets of three arrows, obviously red, blue, and green for each of those three buckets.

And this is the report you can use to show and evaluate how many of those students have 12 or more hours, how many have one to 11, how many have zero. This should be the quote unquote, easy way to measure your progress on that mandatory consortium and agency level match. I'm in a little bit of a transition here. Let me just stop. Again, I'm going to be obnoxious this way. Sanity check for everybody, is this making sense? Is this getting eerily like 15 or 20 minutes you'll never get back, all those usual cliches? Where are we here with that. Sorry, inquiring minds have to know.

OK, thank you Elle. You've been very responsive, Elle. Thank you. Anybody else but Elle still hanging in there? I know Elle is hanging in there, not sure about the other 59. OK, there's somebody chiming. Better from the last meeting. And I'll just say I don't know who brought that up. I won't go back and find you or whatever. But yes, CAEP accountability suspiciously similar to WIOA II accountability.

That is anybody tells me they got it all down after one sitting, there's 125% chance I don't believe you. Just about everybody takes four or five sittings of this craziness before it really starts to germinate. I'd say your average person-- forget about average person. Your superstar person needs four or five sittings before any of this has a chance of germinating. It's the ultimate drinking from the fire hose. So yeah, little by little, hopefully it gets a little bit better.

Thank you for humoring me on that. Five or more. So I'm underestimating it. You need way more than five. All right, I'll take your word for it, Elena. Thank you. Anyway, this is a slide we were talking about a lot at the very start when we were still AB 86. So I'll start by getting everybody's foot on that rail by saying around that time, it was 2013 or 2014-- JoDee probably knows this better than I do. I think it was 2013, but JoDee, bet you can confirm or deny. JoDee will indicate and grade me on whether I'm right or wrong.

Anyway, the legislature passed AB 104. That is they specifically identified the fact that we have this funding for adult ed and took it a step further. These are the specific outcomes that adult ed is going to be achieving. You might say, hey, this is why we're giving you the money, because we're expecting you to show that your students are achieving outcomes in these areas.

I'll just say over there to the right, there's those six areas that everybody knows and love. And again, they're the areas that we know and love because the California legislature says so. So what I also like to bring up here, and this was also part of the state legislation, is it also required us to align all of these outcomes and all of this reporting to the WIOA federal. OK. Thank you, JoDee. Yeah, that's great. I knew you'd know all that.

Anyway, we basically required us to align to WIOA, which, as most of you know, is the current federal reporting system. They were real clear. Hey, we've got this framework. The feds in their infinite wisdom have wind out this very detailed system. No, we're not going to reinvent the wheel. We're just going to align to all that legwork the feds have already done. That's what this slide tries to show.

So what I like to say is when you look at the federal WIOA system, we have what the feds like to call performance indicators and what the feds call measurable skill gains. That's what we mean by MSGs, by the way, measurable skills gains. Anyway, the feds have those two types of outcomes. Performance indicators are what I like to call kind of the tangible outcomes, or I also like to call them the outcomes you can kind of hang your hat on.

Real easy to identify and measure. Things like get a job, get your occupational license, complete your high school diploma and/or college. All of those things that are easy to measure, hang your hat on type of outcomes. I like to say there's good news and bad news about performance indicators. The good news I'd say is pretty much everybody agrees these performance indicators are wonderful outcomes.

Nobody really tries to argue that getting a job isn't a good outcome. Everybody agrees entering college is great, getting your GED is also good, getting a wage increase as good. Everything on the performance indicators list is no less than 125% positive. However, the bad news about performance indicators is if we rely solely on performance indicators, we're highly likely to show that the overwhelming majority of our students are doing a lousy job, which of course, we know nothing could be further from the truth.

But again, if we rely on performance indicators, that is kind of what it looks like. It takes a while to get a job, it takes a while to finish up the diploma or GED or to be ready to enter college or be ready for workforce training or completing your certificate or whatever. All of those things take time. So we're going to leave a lot of students out in the cold if all we do is measure ourselves by performance indicators.

So that's where the bottom measurable skill gains comes into play. You're listening to the CASAS piece here. So of course, we're going to say by far, and I mean overwhelmingly, the most obvious example here is pre and post testing. That is, hey, they're not ready to get a job, they are not ready to finish up their GED quite yet. But they did a bang up job on their CASAS math or CASAS reading. So we can at least show that literacy gain that level gain to verify the fact that our students are still doing a terrific job.

So going back over here, we're aligning these indicators and measurable skill gains in CAEP land. Some of it when we were looking at-- this was easy. Obviously, the CASAS guy is going to say, hey ABE, ESL, mission accomplished. We have CASAS testing. We already have an outstanding system in place to measure ABE, ASE, and ESL without requiring that final prize outcome.

So we can just implement that for those students and do the same wonderful things we've already been doing in federal reporting but for some of the other programs, workforce prep, and CTE. Obviously, it's not quite so easy. We don't have any built-in way to show progress without relying on that final prize. So the word I've used over the years is retrofitting. We have lots of federal outcomes for WIOA reporting. Some relate to us as Title II adult ed agencies, others relate to workforce agencies.

So what we've done is we've retrofitted some of those outcomes and basically identified them as priority areas for these six areas of AB 104. I better get moving here. We're not doing badly, but I do need to move along. So anyway, we talked about four areas of AB 104. Not so coincidentally, here are six areas of AB 104. Not so coincidentally, 1, 2, 3, 4, 5, 6 color-coded boxes resembling those six areas.

So these are the six areas. And this slide just lists all of the very specific outcomes a student can achieve, and which of those six areas of AB 104 those different outcomes fall under. Chances are lots of you have encountered these over time. So this is our current system. Most of you would say it's a little bit overwhelming. More on that here in a minute. But just for now, these are the six areas that we're required to report, these are the different outcomes that fall under the different categories.

Here is the infamous bubble boy slide. This is just the same information as the previous slide but in much gorier, much more annoying detail. We called it the bubble boy slide because hey, it's our bubble form. This is a screenshot of our closest date record. Nobody uses these bubble forms anymore, but we still like them for training. They're a really, really good way to present things from a training point of view. Here's one of them.

So again, six areas of AB 104, same non-coincidence here. Six different color-coded bubbles at the bottom. We've color-coded these bubbles to show exactly which outcomes relate to CAEP. So just to give you an easy example, looking at the upper left, we've got that blue bubble, got a job. Surprise, surprise, that's blue. All of the ones we've marked in blue relate to employment. Nobody's really going to be that surprised that got a job is one of the three employment related outcomes for CAEP. You can use the same logic for all these others. Again, six different areas, six different color codings.

So when you're looking at what you need to report in Enterprise, what you need to work on with your students, here is a cheat sheet to make sure you're reporting the right things. One more of these. Same exact information, but this is from the perspective of TE software, letter codes instead of color codes, but the same six areas of AB 104. The exact same outcomes, but this is just showing where you can find them inside the TE software. If it makes more sense to be able to figure out checkboxes rather than bubbles, here's a way to get that same information in a slightly different way.

OK. So I'll stop for a minute. I hear what you're saying about the update record. Totally would be nice. We've heard that. I don't think there's a final answer. I got to say, TMI, we probably will look at that but back to we're looking at how we might need to streamline this, things like LAO report. There's a lot of things we know we might need to tweak about this or perhaps even totally redo. So we'll probably look at that first before getting all the infrastructure in place, I got to say. But yes, I'll just say to Shell, you're in a very big club of people that make that suggestion at the very least.

So I'm not going to go over all the outcomes. This is going to sound nuts. I'm taking probably the hardest to understand outcome and bringing that up first. Some people don't like that. But just to explain myself, I'm sort of picking and choosing here. I'm just picking a few of these outcomes and I'm purposefully selecting ones that typically garner the most questions.

There are some outcomes like pre and post testing and get a job, and we don't need to explain that one because everybody kind of understands how those work, but a lot of these other ones maybe not so much. So two good examples, the ones I'd say Neil and I have explained more than any others, are these two literacy gains called occupational skills gain and workforce prep outcome.

So for starters, the title is part of the problem. We've looked at it and we know the title is part of the problem, but we're kind of leaving as it is. We're calling them literacy gains, but yet they really refer more to CTE and workforce prep. That gets us started on the wrong foot, and lots of other things that confuse people about these too. But anyway, we've got these two literacy gains.

Going back to that federal slide I showed there a couple of minutes ago, just so you can make sense of this, this is what I mean by retrofitting. For ABE students, for ESL students we can get gains in bite-sized morsels by doing pre and post testing. Most people would say, for CTE, that's not really appropriate. We need to evaluate their vocational skills, not necessarily their literacy skills. So that's the retrofitting we're talking about.

So when we looked at this, we came up with a couple of ways of doing it called occupational skills gain and workforce prep outcome. Obviously, lots of confusion between the two. So what we've said over the years is occupational skills gain is what you mark when a student accomplishes a portion of something longer term whereas workforce prep milestone represents full completion of something shorter term.

The occupational skills gain example, we're just making it up. We'll say we've got a student in welding. We're saying, it's five semesters or five modules long. The student basically does a great job in module 1. They pass some sort of exam at the end. Maybe that's CASAS testing, maybe it's a written test you made up, maybe it's just a functional skills check you made up. It can be any of that. But they pass some kind of exam that suggests they did a great job in module 1, and now they're moving on to module 2. That's an example of occupational skills gain.

Workforce prep outcome, the example is, hey, we've got a 15-hour course on job search strategies that's kind of open and shut case. It's a real short course, just focusing on that one skill. The student sticks around for all 15 hours. They're still alive at the end of it, congratulations. That's an opportunity where you could mark workforce prep outcome to show that they completed that shorter term program as a stepping stone usually to bigger and brighter things. But that's our example of workforce prep milestone.

I don't see anything else. I thought I saw a couple of new chats, but no, I don't. So I'll just say any questions about that? Sorry, I'm doing multitasking a little too much here maybe. Does everybody understand that distinction? I know I'm just sort of throwing PhD-level issues when we're trying to complete our stinking GED, for crying out loud. So I admit guilty as charged. That said, the people feel like they got this-- again, I'm picking on what people ask. At least somebody's thumbs upping me. I'll take the money and run. OK, thank you.

So here's another, again, higher level question, but this is what everybody always asks, so this is what we're addressing here. So we've got that workforce prep milestone and occupational skills gain. We also have the question of hey, we understand that, but when is it higher level? When can we mark that as a post-secondary outcome and not just, quote unquote, literacy gain?

Well, you can see over here to the left, we've got those two CTE literacy gains at the bottom. We've got those CTE related outcomes at the post-secondary level up top. That's just the list for you. So what we say is, hey, we get a literacy gain for either completing something shorter term or partially completing something longer term. That outcome rises to the level of post-secondary when the student achieves full completion of something longer term.

So sticking with my example, once that same student completes module 5 of 5 in that welding program, then you can give them a post-secondary outcome, which sometimes is a credential, sometimes licensure. Depends on the specific occupation. It's a higher level post secondary outcome. One more thing on post-secondary. This hasn't been a long standing issue, but it's kind of been one more recently, that is in the post-secondary area, why do we have so many stinking checkboxes and bubbles for post-secondary?

This is one area where the list is understandably way too overwhelming. We will look at maybe paring it down because that's kind of a new thing that's come up that I got to say has really sunk in at least in this nugget a lot. Something we really need to think about, make it a little easier. Got to say, I think we were really worried five years ago about-- I kind of remember pulling that bandwagon into town back in 2016. It was an empty bandwagon. Ain't nobody touching that bandwagon with a 10-foot pole.

So I kind of remember we would talk about we really need to set it up so we can be there and say, hey, bring us your tired, your hungry, your weak. This bandwagon accepts everybody. So it doesn't matter whether you're marking occupational licensure occupational certificate. Either way, we want you on our bandwagon. That was kind of what we did in our forming years of 2016, 2017. Forming, storming, norming, performing.

Got to say we're past the forming stages, now we've had all kinds of storming. Now we kind of need to look more at norming and performing, and that might mean fewer bubbles. But anyway, we've got a lot of bubbles because post-secondary-- back to the wonky federal reporting explanation is post-secondary is the term the feds use to represent everything that's directly above the high school level.