Holly Clark: Jay Wright. Jay.

Jay Wright: OK, just like that. All right, so we'll blow it up. And I think you have the PPT. So you can send when it seems advisable to do so.

So welcome, everybody. I'll backpedal immediately and just say, hey, I'm looking at the audience that doesn't look like a basic-level audience to me. It looks pretty gosh darn close to the exact opposite of that.

But I will say this one really is intended to be a basic-level accountability. It's about an hour in length. It's obviously an adaptation of the one we did for the face to face last year with a few little updates for '23, '24, of course. But it was intended to be an online just basic accountability.

We don't really have the big stuff going until CAEP Summit. That seemed a little too late to do heel dragging. So we did decide to do a couple real basic-level sessions with CAEP TAP here online here in early September with the idea that, hey, that timing might be a little bit better. So this will be that for the final summary of backpedaling.

This will for, I think, all. But a few of you, this will be 90% to 95% stuff that you've not just heard before, but probably heard many, many, many, many, many, many times. It'll be whatever the exact opposite is of new. But, hey, always good to get a little fresh face on stuff you've heard before. So here it goes.

So here's the agenda. This should reinforce what I just said, I think, real basic-level agenda items. We're going to define the CAEP programs. There are a few little changes in NOVA that are worth mentioning. So, hey, there'll be a little bit-- there'll be at least 90 seconds of this hour and a half that might be worth it for some advanced people.

We'll go over some outcomes and services. We'll stick to the basics on what the outcomes are. We'll stick to the basics on what the current services are. A lot of you have been at a lot of these meetings in recent years. So you know yahoos like me have spent a lot of time talking about how we need to revamp this.

We've had a lot of activities starting to get at that and move that direction. But as is, we haven't moved one iota. So as is, we're still looking at the same bubble boy slides and the same set of outcomes and the same set of services for now. I think for '23, '24, probably safe to say that's what we'll be using. But we'll be continuing to look at new ways of looking at it and what might work better.

Basic data collection requirements, that really reinforces the basic-level training. Again, a lot of you that have been to these over and over again will recognize, hey, wait a minute. These are slides that I don't recognize from CAEP circa 2017. These look more like Title II circa 2003. That's because a couple of them are-- and they're updated-- more updated than that.

But, hey, lots of real basic-level stuff on entry, update, record, timeline, deliverables, CASAS California accountability page, stuff like that with basic data collection requirements, timeline, deliverables, and resources. And then we'll finish out on CAEP reports in TE, not really a deep dive from a TE point of view. But from our perspective, obviously, it's hard to talk about CAEP accountability without at least covering the TE report. So we'll show what some of those basic reports are.

Some of the key considerations for each, and that will be it. I'll just say, to what Holly said, because this was intended to be basic, it really was built in to try to have some Q&A and so on. So most of this is a lot of Holly stuff just doing some housekeeping. Great. Thank you, Holly.

So I think it's mostly Holly stuff for now. But I'll take a look, but lots of chances for Q&A. My hunch is unless we have a lot of Q&A during the slides, which is certainly possible, we'll have time left over. So if you have a lot of questions, my hunch is that there might be questions on things not really related to the topic at hand. Kind of OK with that, too, because I have a funny feeling we'll have time.

So starting with program structure-- I know a lot of you have seen this one before, mostly just an oldie but goodie. I like to start with this one. There are at least a few of you I see that are names that are not super familiar. So that does make me a little hopeful. There's at least maybe four or five of you here in the audience that really are new to all this CAEP accountability madness and really do need a basic-level understanding of the issues we're talking about.

So for your benefit, here's a good square one where we've got, starting with the map of California. That is, the way the structure works for California Adult Education Program is it's aligned according to community college boundaries. 10 years, ago when we were AB 86, there was an effort to align the community college districts and the K-12 districts into separate adult education regions.

Bottom line is they were aligned according to community college boundaries simply because they're bigger than K-12 boundaries. So, well, then 72, now 71 community college districts and California all had their own region. If you were a K-12 district within that community college boundary, by definition, you were with that community college. So the college and all of the K-12 districts within that college district comprised those consortia.

So we have 71 regional adult ed consortia, including members, which are basically those from the K-12 districts and the community college districts. Congratulations, college people and K-12 people. You're in the consortium. Whether you like it or not, by definition in the legislation, the college districts and the K-12 districts are inside the 10. If you don't want to be, well, you're inside the 10 anyway because the legislation says so.

Then if you're not from a college and you're not from a K-12, you may be a partner. If you're a partner, then you were accepted by the K-12s and college district in your region. You are allowed in the club.

If you're a partner, the tables turn. You're allowed in if your members want you to be there. If your members don't want you to be there, you're not necessarily part of the club. So you're, like it or not, is-- you may be allowed, or you may have to be outside looking in.

But we have members that are the ones the legislation requires to be part of the consortia and then partners such as libraries, CBOs, one stops, and so on that you may want to invite in if your region chooses to do so. That's been the structure we've had since about the 2015, '16 year where we have the regional consortia that manage themselves locally based on how your regional structure determined was the best way for your local region.

Looking at structure from an entirely different vantage point, these are the instructional programs. So we're not looking at it in terms of geography. We're looking at it in terms of schools and colleges and programs and students. This is how we set it up in TE.

Anecdotally, the way NOVA has it set up now is a lot closer to the way we have it in TE. So I think it's a little easier now to explain. There used to be more of a disconnect between NOVA and TE.

But in TE, we have seven programs. We've got the three primary ones. They're right up at the top-- ABE, ESL, and career tech ed. Under career tech ed, we also have workforce prep and preapprenticeship.

Then we have adults with disabilities. And we also have parent supporting K-12 success, not the way they probably want to be characterized. But I do say those two tend to be a little bit flapping in the wind because they don't really connect with the other programs. But the legislature at the start said we needed to include those programs. So we definitely make sure we include all seven programs when we're reporting our CAEP data.

Looking at the primary programs in a little better detail, again, we tend to roll it up according to these big three. I think most people agree with that, not everybody. But we've got ABE/ASE. We've got ESL. And we have career tech ed.

I used to drum this up to say, hey, the way things are in CAEP land are a little different than the way they have it in TE land, quite frankly, which makes CAEP land a little different than the other 98%, 99% of the world as well. Again, it feels like NOVA is a little bit more like the rest of the world now, and that it spread it out and change the programs. I'm not positive of that. I'm not the NOVA person. But I've seen screenshots. It looks more like what we have in TE.

But, anyway, just to explain, we have ABE and ASE. That's in a bucket some people get confused by that in TE and most other reporting areas. ABE is one program. Adult secondary is another program.

In a lot of real-world scenarios, ASE is actually two programs, that is, high school equivalency and high school diploma. So, again, some would call this one program. Some would call it two. Some would call it three. Either way, all of those programs that you now and love are in that ABE bucket for CAEP reporting.

ESL doesn't generate much confusion or questions at all. ESL is ESL. Kind of think the way we do it for CAEP is just like the rest of the world there. CTE is where some of the confusion kicks in, is that it rolls into CTE sometimes.

But within CTE, it's broken up into those four bullets. You can see in CAEP land, there's a lot of effort to distinguish CTE and short-term CTE. I'm not the decider on that. I believe there's a 48-hours issue within the college system.

If you know the college system, you probably know that better than I do. I think that's the cutoff point. But I'm not sure of myself there. But because there's a lot of distinction in the college system between CTE and short-term CTE, that's a distinction we make in CAEP land.

Then we also have preapprenticeship, which is a workforce-related program, but definitely not the same as CTE workforce preparation. You might say the same thing entirely different from those others, but, again, not the same thing as CTE.

OK, now the chat's empty. I guess the question were about PowerPoints and registrations. When I was talking about easy stuff, it felt like the chat was off the hook. Now I'm talking about this convoluted stuff, and there's no questions. Go figure.

I will at least stop. Is this resonating? I was kind of expecting there to be questions now because it feels like we're getting convoluted about now. No, nothing, everybody? All right, I'll take no news to mean good news, I reckon.

OK, so let's just go to the next slide. So here's that same yellow box for CTE. Again, these are not official. I don't think you're going to go on Google and find this. You're not going to find any CAEP documents that say this is the official clarification. But I do think this helps to figure out what's going on with these different subprograms.

So CTE-- obviously, CTE, hopefully, everybody knows what that is. But in general, that's workforce-related CTE is, usually, a longer-term program that focuses on a specific occupation. Short-term CTE, of course, just like CTE, focuses on a specific occupation.

But who's buried in Grant's tomb, hey, short-term CTE is short term. Regular CTE isn't. Again, I think the cutoff is 48 hours of instruction.

Preapprenticeship gets folded into CTE in some areas because it is workforce related. Like CTE, it's long term. Like CTE, it focuses on a particular occupation. But very unlike CTE, it has obviously a completely different format for the student. So it's considered a completely different instructional program.

Workforce preparation maybe look similar to CTE but is focused totally differently. It's something for the short term. And I think more importantly, workforce prep is not occupation specific. That's really probably the single most important distinction of this slide is people always get confused between CTE and workforce prep, not foolproof, I guess.

But in general, CTE is always focused on a particular occupation-- HVAC, welding, nursing, et cetera. Workforce prep isn't. It's workforce-related instruction. But it's usually more generic-- workplace, safety, resume, writing, filling out job applications, et cetera, et cetera. It focuses on work. But it's not related to a specific occupation like CTE.

I'll just say nobody's responding. I'll just have to assume no news is good news. I think they're freezing the screen with those 35. I think I've probably lost all of you. It's just my screen kind of lying to me saying there's still 35 people around.

But, anyway, looking at it completely, differently, totally-- all right, thank you-- totally different angle, this is summarizing a bunch of slides I cut out because it's like so 2019. But just a reminder that we're aligned to WIOA. WIOA is the federal structure. More on that over the next few slides, getting more gory detail-wise. But I do like to point out that WIOA exists.

Here's the slide, again, another one that a lot of you have seen, triple-digit number of times over the years, explaining these four title numbers under WIOA. We're Title II or adult ed and literacy. Title I is workforce. They're the ones that have the lion's share of authority, lion's share of responsibility because the federal legislation says so. Title III, short-term workforce; Title IV VOC, rehab. Those are our WIOA partners.

I make a big deal about this because it's written into the legislation there 10 years ago that our consortia are required to align with WIOA. That means we're aligning and collaborating with our WIOA partners. It's a big part of the process. When you look at the data-reporting structure, we're really looking to align to that federal system to the maximum extent possible.

I'll also just say there's kind of a spirit of CAEP, spirit of WIOA that I'll always say was on purpose. I'm not sure if it feels that way anymore. But, hey, in 2016, it did anyway, where there was a lot of movement toward more and more collaboration after many years of not really doing all that much of that.

So we were collaborating our brains out there for a few years that felt like the federal legislation required us to do so for WIOA. We were exploring all these new partners we didn't know. Hey, what do you know in CAEP land, that AB 86 land? We were doing exactly the same thing, a lot more collaboration. So just know we need to be looking at our WIOA partners, just like we're looking at our CAEP partners.

I think in most regions, maybe you differ from this. I'm not sure. But it feels like this overlaps a lot. And in the high-performing regions, probably feels like it's the same collaboration, not separate and different. Anyway, this is the WIOA information.

So now we're moving into CAEP land. It started with AB 104. That was legislation from 2013 10 years ago, again, California legislation. I'll admit if you type in AB 104 in Google. You'll get this as a hit. But you'll get a bunch of newer AB 104s. So I admit from that standpoint, it's confusing.

But the AB 104 in 2013 basically required California Adult Ed to align to the federal system, align to WIOA. So this slide is what I've used forever to show how we align the data in that fashion. So TMI here, but for federal reporting, the feds have what they call performance indicators. That's in the upper left. There's more, but that's what we have aligned.

And within performance indicators, there's a subset called Measurable Skill Gains, or MSGs, you can see those MSGs listed. So what the state of California said is our adult ed program is going to align to that federal system they came up with these six areas of AB 104.

I used the slide to show not so coincidentally, the state of California looked at those federal performance indicators and MSGs and came up with basically those six pillars of what we're required to do. In California Adult Ed, you can see not so coincidentally, the exact same things we already had in the federal system.

The one thing I'll point out that's a little bit different is I'd argue those top five begged, borrowed, and stolen from the federal system, not a lick of difference. But they did add number six. That's why it's in italics. That is transitions.

We've been recording transitions in CAEP the last 10 years ever since. It's not quite so obvious where transition fits in the federal system. So we'll just say, yeah, in this case, the California legislature went out of its way to say, hey, we need to align to the federal system. And in addition to that, we also need to go out of our way to record student transitions.

So that brought us to this slide. Another one that I know a lot of you have seen many times-- again, we just talked about the six areas of AB 104. Not so coincidentally, here's 1, 2, 3, 4, 5, 6 pretty little colored boxes corresponding exactly to the six areas of AB 104.

So this is showing how we did, it number one, and providing some of the details, number two, again, literacy gains, secondary, postsecondary, employment, wages, transition. Not everybody agrees, but I think most people agree those are six pretty basic areas to be reporting. Most people agree, yeah, these are probably the areas that are really good to be reporting and showing that our students are making accomplishments in these areas.

So the detailed alignment from about five or six years ago represented in this slide where we had WIOA outcomes, we didn't necessarily choose all of them. But we came up with some specific outcomes, again, aligned to the federal WIOA system. We did what we called retrofitting, aligning some of these outcomes, some of them like GED and high school diploma, not really much retrofitting what you're marking in TE or whatever exactly the same thing as the outcome we're looking for CAEP reporting.

Others-- segueing here-- are more like the bubble slide where we have some of these WIOA outcomes that were required to collect for TE because the feds say so. When we looked at this, they're five, six years ago. For state level reporting, we knew we wanted to collect a lot of the same outcomes as required for the federal WIOA system.

But, hey, we're adult schools. We're community colleges. We're not exactly the same thing as one stops or AJCCs or whatever. So we need to make it a little different. We need to retrofit it so it really fits a little better to what we're doing in adult ed. So that's what we came up with this monstrosity.

This is what we've called the bubble boy slide. This basically aligns the bubbles on the CASAS update record, the boxes that you can check for student outcomes in TE with the official outcomes that we identified for AEBG and CAEP reporting six or six years ago.

So just to make sense of this monstrosity, I'll point out, again, 1, 2, 3, 4, 5, 6 pretty little color-coded legend at the bottom aligned to those six areas we've talked about. So for example, got a job up there in the upper left. It's color coded and blue. You look at the legend at the bottom.

Surprise, surprise, blue is one of those that we recognize for employment. So surprise, surprise, when you get a job, that corresponds to a student outcome in the CAEP area of employment. Why? Because this bubble boy slide says so.

I'm being a little cutesy about it, I guess, probably just because, hey, we've talked about this so many times. That's my story, and I'm sticking to it. Hey, does everybody understand this?

Some of you, if you don't know by now, I guess you ain't never going to. But there's at least a few of you I should stop being too cutesy. There's some of you really hearing this for the first time. So, hey, those of you that are, is it making sense, yes or no? For you, it may not be so flippant. I'll take that as good news and move on.

So next slide, this is exactly the same information as the previous slide. But it's using the screenshot directly from TE rather than the CASAS update record, looking at checkboxes rather than bubbles and aligning it by letter codes rather than color codes, really just looking at it from the direct and TE perspective. I believe all the checkboxes are the exact same checkboxes in this slide as the bubbles, as we just saw previously.

OK, this I'm using as a segue where we just looked at the bubble boy slide. I don't really want to get into all the bubble-related details. I do want to make sure everybody has it for reference. But yeah, it gets a little nutty going bubble by bubble.

That said, this slide sure contradicts that, doesn't it? But I like using it just because this is the area that I waxed philosophic about when we talk about outcomes. A lot of you know this. This is the area for my two cents that tends to generate the most questions and so forth.

So I'm going to read this one off, even though it contradicts just about everything I've said to date with the training. So we'll start with prepost gains. When we say literacy gains in WIOA federal reporting land, a lot of you know not all, but most of the time, we're really just talking about pre and posttesting.

What we're talking about in CAEP land, probably safe to say not all the time. But most of the time, it's still pre and posttesting too. Just like a lot of you know and love-- I'll say 90% of you know this, I think-- we do our classes pre and posttesting. We're not marking anything because the test results speak for themselves.

We don't need no stinking bubbles. We don't need no stinking checkboxes. We just do the testing and e-tests. And the results work out automatically based on those test results. And We will get into this in more detail. But those pre/postgains you know and love are one of those outcomes for CAEP. That definitely does count and is a vital part of CAEP just like WIOA II.

Because it's here on this slide, I'll mention Carnegie units. It's not my top priority item. So nobody thinks I'm dodging. There's those high school credits, outcomes. I'm going out of my way to not show that slide lately, because we need to come up with something better for that. But we haven't come up with something better yet, but it's not going away.

If you've got a detailed way to account for Carnegie units or high school credits, that's another way to show a literacy gain. If you've got students in high school diploma that are earning high school credits, potentially, you can get a gain if you can show that high school diploma student has moved from the 9th or 10th grade level to the 11th or 12th grade level within a year. That's also a literacy game.

CDCP certificate is another one I'm not that interested in talking about. But, again, so I'm not dodging. I'll point out the very first year of CAEP, that AEBG reporting, hey, you're getting TMI here. But some people like it. That very first year, everybody reported in TE all the CAEP community colleges and all the CAEP K-12 districts.

For one year, anyway, we were one big happy family. But that only lasted one year. After, that the colleges peeled off and used their community college MIS. So this is a leftover. It's not really one that's a big part. It's a big thing for colleges.

But if you're marking it for colleges, you would be marking it in your community college MIS, not TE. So for the most part, got to say CDC certificate, 0% my problem now, 125% community college, MIS's problem. But, hey, if you mark one of these bubbles, it will still show up as a literacy gain in a no harm, no foul sort of way.

The reason why I make a big deal about these literacy gains, of course, are these bottom two-- occupational skills gain and workforce prep milestone. Those are the ones that we retrofitted for CTE and workforce prep. What I like to say here is, again, we're going all out on TMI. So why do it any different?

But what I like to say is for programs like ESL and ABE, this whole issue with literacy gains and being able to show that incremental progress, mission accomplished. Hey, you're listening to the CASAS guy. So the CASAS guy is automatically going to call that a mission accomplished. We don't need to worry about how to show incremental gains for those students.

That's why we have CASAS testing. We already have the ideal way to show and record exactly that. So that's how we're going to do it for ABE, ASE, and ESL.

However, we got to admit, for workforce prep and for CTE, that's not what pre and posttesting is for. There's probably different better ways we really need to look at and use for those programs, not really CASAS testing. So that's where these came in.

We were looking at ways to show progress for CTE and workforce prep, just like we do for the WIOA II programs without relying on the final prize. And I didn't really set that one up well. But back to this slide-- sorry, I'm pulling the rug out from under you-- is with this slide, I like to point out the performance indicators MSGs for my two sets performance indicators are those outcomes that are a final prize. Get a job. Enter college.

OK, I'll get to that in a minute. Enter college. That is those outcomes you can hang your hat on. Measurable skill gains is looking at it to show that incremental progress.

The way I like to frame it is this. When people look at performance indicators, which also includes things like enter college and get your occupational license, not just employment and wages, everybody agrees all of them are really, really valuable, wonderful outcomes. Nobody quibbles with that.

But people also agree if all we do is performance indicators, then we're setting most of our students up for failure because it takes a long time to get the diploma, earn that college degree, get the pay raise, get the job, whatever. That takes time. Most students don't do that within a year.

That's why we have MSGs. That's why we have wonderful things like CASAS testing. So we can still show beyond a shadow of a doubt that our students are, in fact, doing a wonderful job, like most of us already know.

So back to the retrofit, for ABE and ESL, mission accomplished. But for CTE-related programs, not mission accomplished. That's what we have here. Go ahead, Holly. I've been babbling too much. So I better just stop and hear the question.

Holly Clark: No, it's OK. We do have a couple of questions in the chat. Tashelle asked, "Can CTE use pre and posttests to show literacy gains?"

Jay Wright: If you're talking about this, yeah, keep in mind you're asking the CASAS guy. So you're letting the fox rule the henhouse here. But yeah, as long as the fox rules the henhouse, that answer is obviously going to be yes.

And we have said that if you want to test your CTE or workforce prep students, obviously, CASAS testing is one option you could use. We admit a lot of CTE types would say, hey, we have mo' better ways. That's what we like to do. If so, we understand, and that's fine.

But yeah, if it's just easier to do CASAS testing for them, why sure, if they make gains. And they show progress, we'd still certainly say that's a good way to do it.

Holly Clark: Perfect. And then Carla asked, "What's an example of met work-based projects?

Jay Wright: OK. So yeah, I saw that one and went off on a tangent and [inaudible] about it.

Holly Clark: OK.

Jay Wright: OK, I think, really, you're asking this from the, hey, what does it really mean not what does it mean on the goofy retrofit slide? So you're going to get way more TMI than you bargained for. Work-based project is a bubble that's been there since the late '90s. It started in WIOA.

I can't remember why. But it was a WIOA thing from literally '99, 2000 where it was in the old VABE and VESL where there were VABE and VESL students that did things the way a lot of students do it now, where they were doing it in blocks. My recollection is it was like 30-hour increments where you might have a CTE student in ABE or in ESL for that very fixed period of time.

Here's where the crazy TMI is that you weren't bargaining for is for whatever reason, that would be a drop reason under WIOA. So we went out of our way to have the bubble. There were WIOA funds that served work-based projects students. But I think it was because they were vocational or whatever.

Back then, that meant no NRS outcomes, thereby no payment points. So it was a weird drop reason. We don't really use it as much now under WIOA. I haven't seen anything about that since WIOA.

So, hey, back to the retrofit, that was a first round draft choice for retrofitting because we knew from a WIOA point of view, we probably weren't going to really need it anymore for federal reporting. How about that? Talk about getting way more than you bargained for. I think that's like what you were actually asking. If not, I guess you'll let me know.

Holly Clark: OK, perfect. And then Teresa asks, "If a student transfers from our HSE, HSD, ESL program to a noncredit college course or even career courses, where would we mark that?"

Jay Wright: I think you can look in the bubble boy. But I believe there's a transfer to credit and transfer to noncredit. Those are two bubble boy options. So in your example, obviously, you'd want to transition to noncredit.

Holly Clark: OK, great. Thank you.

Jay Wright: But the short answer is you can verify. That's why we put out the bubble boy slide. Obviously, it's hard to look at every bubble and prescribe bubble by bubble by bubble. But we do go out of our way to add that crazy slide just so you have that reference, so you can look it up.

Holly Clark: Great. And then--

Jay Wright: I don't think there's a transition from a WD to something. But, hey, you can send me that. That seems like an obvious suggestion. Some of you might have heard that before. If you did, I'd be interested.

But I'm going to say that seems like a really stinking obvious suggestion. But I'll go on a limb and say I don't think I've heard anybody suggest that to me before, as obvious and straightforward as that one sounds. I'll admit that it does sound obvious and straightforward.

And at surface glance, it sounds like a good idea. But I don't know of anything right now where we're really tracking it. So I'm going to have to say no, I don't think so. But I like it as an idea, not that that's what you're hoping or looking for. But for what it's worth, I do like it as an idea a lot. Better than nothing, I guess.

But yeah. And I'll add on, as you know, we kind of are thin-- our bench is thin in adults with disabilities. We could probably use a few like that in adults with disability. That's a small little roster. We could probably use a few more bodies on that team, so to speak. OK, anything else? I'll just say, hey, Holly, anything else I missed seeing as how you're doing a good job interjecting here?

Holly Clark: No, don't think so Carla mentioned the definition sounds like the 243 COAAP. Is that how you say this?

Jay Wright: OK. All right, so occupational skills gain, workforce prep outcome, I'm sure I've done more than enough setting it up. So we'll talk about the outcomes. Again, this is how we set it up.

My setup is you can use occupational skills gain for when students accomplish a portion of something longer term. Use workforce prep milestone when the student completes something shorter term. So you can see the examples.

Example for occupational skills gain, we'll just say a five-semester long welding program. They complete that first semester, do a great job. Maybe they CASAS test because they're working for Tashelle. Maybe they're doing a skills check. Maybe you're doing your own agency homemade written test. All those suggestions are fine.

But you should give some kind of exam, whether it's skills check, written or otherwise, that proves that that student's doing a great job. If so, you can record occupational skills gain that shows they're moving in this example from module 1 to module 2.

Workforce prep outcome, the example is, hey, they're in a 15-hour course on job search strategies. In this case, the formalization is very low. Hey, this one could be that certificate that the secretary prints out of her computer or whatever, but something that shows that the student did all 15 hours. You can then mark a workforce prep milestone is another way to show that your student is making progress toward those other things. OK, thank you there.

OK, taking it the extra step, again, this is looking back to what we talked a lot about in training and previous years. But one of the big confusion points of CAEP accountability training has been that ability to distinguish those CTE literacy gains. The next step out of that is then looking at those CTE literacy gains and being able to distinguish those with what we already have on the table for postsecondary.

So back to that measurable skill gain versus final prize performance indicators, you might say the ones we have for postsecondary are more like the final prize. Some of them are college. Some of them are workforce training because we're talking about CTE here. We're looking at the side of postsecondary related to workforce training where we've got that occupational licensure and occupational certificate.

So back to the slide, when you've got that CTE student that's progressing within an occupation or something like that, you can mark workforce prep milestone. You can mark occupational skills gain. So, again, those literacy gains are partial completion of something longer term or full completion of something shorter term that rises to the level of a postsecondary outcome upon full completion of that longer-term program.

So back to my example, if the welding student is finishing module 5 of 5 and getting that occupational certificate, then you're not marking occupational skills gain anymore than you're marking that postsecondary outcome to show that they achieved that final prize. OK, hopefully, this is making sense here. I'll have to say no news is good news, like usual.

So here's a little more on postsecondary. This is just a laundry list, a lower volume problem, I'll say. But, hey, with postsecondary, it's a really big bucket. I do think sometimes the fact that it's such a big bucket confuses people. The fact that it really is two buckets in one represented by the slide also confuses people.

The two buckets I like to say is we've got this bucket full of workforce training outcomes. At the top, we've got similar but separate bucket of outcomes that relate to college here at the bottom for federal reporting that all fills into that very large bucket called postsecondary, another obnoxious explanation about federal reporting here.

But in federal reporting land, the feds use the term "postsecondary" to refer to all those student-level outcomes that can be obtained after secondary. Go figure. You have secondary, and then you have postsecondary, surprise, surprise, on that one. Sorry, I'm being a little cutesy.

But secondary, of course, relates to those high school-level outcomes. That refers to high school diploma and high school to equivalency. Once you get that high school diploma or high school equivalency, a little bit-- but because the feds say so, but a lot of bit, because in this case, the feds are just using common sense. Once you finish high school, you move on to postsecondary.

For some students, that means move on to college. For other students, that means move on to workforce training. The feds use postsecondary to refer to that whole layer that's one layer up from secondary. I don't know if that makes sense or not. Maybe it does. Maybe it doesn't.

But sticking with more information about postsecondary, admittedly, here's where I'm being a little bit of a sadist. I'm getting deep in the weeds after saying I'm staying out. But this is just a reference for you because a lot of times, it's hard to decide what's really postsecondary.

I'll elbow you all in the ribs and say where this has probably come up the most is more on the WIOA side when we're looking at IET and doing that IELCE report or whatever. And you've got all those indecipherable gray lines between what really counts for IET and what doesn't. This is a slide we've used for that. I've backed it into CAEP because I think it helps determine what's postsecondary in what ain't.

So this is straight from the federal website. So I know it's correct, by the way, except for that dotted line that I drew to show there's the run of the mill certificate. That is, you're doing food handler. You're doing CPR first aid. You're doing that one-shot deal that's kind of independent of everything else. That's what we just call plain old certificate. That doesn't really rise to the level of postsecondary.

Over here to the right, we have certification, that is, some occupations. You get that occupational certification. All the way over to the right, we have license. That's what we mean by occupational licensure. That's another subset of that workforce training that does rise to that level.

And then in between, I don't know why they did it in this order. But that degree column refers to, again, we're sticking with postsecondary. But, obviously, you can see from the sort of things they say under degree. That's college stuff, not occupational stuff.

I still look for that guru out there that can answer the question, why do some of these use government? Why do some of these use the business and trade association? It's super duper clear to me that all the ones that the business and trades themselves manage on their own, those are occupational certificates.

The ones that have the mighty arm of government intervention, those are licensures. That's a distinction that I feel like I understand exceedingly well. I've asked a million people. I haven't had anybody yet that knows this answer.

I don't think there is one as far as why do some occupations get managed by themselves. Why do other occupations get managed by the government? Nobody seems to be able to answer that question.

So I'm just going to say, hey, we'll see it-- well, I guess good question, Thoibi. But I'll just say the last three minutes of crazy I don't know talk is exactly-- I don't know why some are certified, and others are licensed.

There is something there where some require government intervention, and others don't. That I can answer. The ones that require government intervention are the ones they call occupational licensure. The ones that the business or industry manages on its own those are called occupational certificate.

The why to which ones are certificates versus which ones are licenses, I admit I don't know what the rhyme or reason is to that. But the ones you're referring to, Thoibi, are on that same list. But you'll have to go through the list to figure out whether it's a license or certificate.

I've spent quite a bit of time on it, as you can tell. And I don't think there's an answer. I think you just got to look it up and figure it out. I'll just say, hey, did I miss a question that might have-- no, that got silent.

But that's not the answer you're looking for. But I really do think that's it. Right. Certificate or license, it's one or the other. But yeah, you're going to have to look it up. I haven't figured out anything that shows how you cover that other than that you just got to look it up and know which way it goes. You just got to know whether it requires the government yes or no.

I haven't found any rule of thumb with that. You just got to go look it up. Right. Same thing with that second question. Sorry. But you just got to look it up. I have, and I haven't found anything that really gives you a nice easy guideline that lets you know that automatically.

You just got to know which ones are involved with the government and which aren't. You just got to look it up online or check with the people that are providing that certificate or license and find that out. I haven't found anything that makes that easy where, hey, if you just know these little tidbits, you can easily decide which one to mark.

Sorry. That's the best I can do there. Hopefully, there's something there. But yeah, this chart is as detailed as I have. It gives examples. There is nothing that I've seen that gives the master list of all the things that are licensed and all the things that are certified, gotta say.

I've looked for that too. And I don't think there's anything like that either. Again, back to the-- yeah, you just got to look that one up. I don't have any better answer.

OK, moving on to transition, this is another one that gets questions. I always like this nifty little diagram. I'm not sure everybody else does, but I do. So this is what I like to show.

That is, we've got these transitions. I think this diagram shows the official ones for CAEP. So the blue boxes to the left are the starting points. The red boxes to the right are the endpoints. So the starting points to get an official CAEP transition are either K-12 adult ed, ABE, ASE, or ESL or noncredit community college ABE, ASE, ESL.

To successfully transition to CTE, you're going to end in either community college CTE or K-12 adult ed CTE. You can see those two red boxes. If you're transitioning to college, your transitioning going to for credit community college.

I will go out of my way to say only for credit, as you can see, there's a lot of these things in CAEP land where specificity isn't exactly a strong suit. You got to figure it out in a lot of areas. Here's where CAEP tends to contradict itself.

One area where CAEP is super specific is for it to be a transition to college. It's got to be for credit. If it's a transition to noncredit, that's considered more like a, quote unquote, "lateral transition."

So if it's to noncredit, then no. That's not a CAEP transition. It's got to be the for credit college, not noncredit college. So you can see by the diagram the different way in which you can record transition to CTE and the ways in which you can record transition to college.

OK, what about noncredit CTE? I'm just going to say that's a good question for CCTAP or for the Chancellor's Office. As far as I know, that doesn't count. But that would be a question for them. I know that there is a data match that they do within COMIS. So probably somebody from West End-- or maybe it's the COMIS person-- can clarify exactly where they draw that line on the sand and what gets in and what gets out.

But if you ask me, it's noncredit. It's got to be for credit to be transitioned. So as far as I know, it's the same. But I'm not quite sure what the difference is between credit and noncredit CTE. So there may be some differences there.

"Enrolled in secondary." I guess, Carl, I need you to elaborate that one a little bit. I guess you need to talk that one out rather than chat.

Speaker: Yeah, in some of the charts and bubble boy graphics that you showed us, enrolled in secondary triggers column L, a transition to postsecondary.

Jay Wright: All right, that's the newer model.

Speaker: It's very confusing when we're working on outcomes. We're all kind of baffled why that's listed there.

Jay Wright: All right, and I'll just say the short what's not the answer. It's like the answers I gave Thoibi. But the answer is we have that now on Table 11, NRS Table 11. So it was a back door thing where, hey, we've got it for Table 11.

There were some reasons for WIOA where we really felt like it needed to be aligned to Table 11. Well, hey, wait a minute. We're lining it to the NRS Table. So we ought to align it for CAEP.

So I'm interpreting this as less about the outcomes themselves and more of a simple bubble boy alignment issue where, hey, maybe it was a dumb bubble boy alignment. If so, then point well taken. And it probably was a dumb bubble boy alignment that we need to change. But we probably won't do any changes until we are ready to officially bubble boy align.