Thank you. Hopefully everybody can hear me just like you can hear Veronica. I guess I'll do what she did just cause last week I remember some people having trouble, or I had trouble. I'm using the exact same setup I did a week ago. So there's a little bit of concern, but it's also the exact same setup I've used many times over the last few months.

My guess is a lot of you were here a week ago. A week ago we tried to stop and smell the coffee a little bit. We did some basic level CAEP accountability. Partied like it was 2017 all over again with some of those original CAEP accountability trainings we did there three or four years ago.

That was probably the bigger request, but there was also a request to dig in a little bit, so I'll call it a data dive. I admit, calling it a data dive is a little marginal, I admit, because we're not in the lab, we're not really on TV, we're not really diving into anything, but we're trying to make this a little bit more advanced than the one we did last week. It's also more advanced than the broader accountability we did there at the beginning of August.

So again, we can't really talk about anything without briefly running through COVID-19 so this one's no exception here. I have programs update. So stopping to smell the coffee last week, we went back to the original training identified the original ABG programs moved it up to date to the CAEP programs. So we won't go all the way back to 2017, we'll just turn the clock back to 2019 for this one, and we'll go back to a lot of the changes.

You might remember the last road show we did for this last year. There were several very big changes that we talked about, pretty much ad nauseum, last summer and fall. So since COVID, I think, a lot of them have been forgotten, so this one, because we're in a more advanced group, will really focus on those changes over the last year or so with the idea that there's probably a lot of forgetfulness given that things like COVID have dominated everything.

CAEP outcomes and services just piggy backs where we were last time. We really went and did every single category gave the basic definitions, last time. We'll do a lot more like we do in data dives, so we'll cover the frequently asked questions this time. Wanted to also point out that new immigrant integration indicator or I3. That's the one that at least I am perceiving is the what's new for 2020, 2021.

And then key training issues, again that's just highlighting the ones that always come up every training. We'll get a little bit into CAEP reports again. Probably a future session where we can figure out how to deal with labs and things like that to really dig into the reports, but we will get into the reports this time. Last week, I remember really avoiding them with a 10 foot pole. We will not do that this time. We would like to identify them.

And then we'll have what I'm calling-- sorry, same old goofy terms as usual but I'll call it qualitative and quantitative troubleshooting. The qualitative review relates to the chat question. I'll get to it next slide. Quantitative review again, we're not going to deal with a data dive, but I wanted to rehash some of the questions we've been talking about.

And in particular, really get in to how COVID-19 affects the questions. Maybe it'll be this way all year, maybe it won't, but wanted to look at the same questions we've been going over quite a bit over the last year, but really look at it more from a fresh plan. Here are some reasons why maybe the questions aren't as relevant as we're in COVID-19. Here's some other ways whether every bit is relevant as ever. Here's how COVID-19 really affects the question, and makes it a different question. Here is other ways where it really doesn't affect it at all. It is the same question with or without COVID-19.

So moving on, Veronica mentioned this, but these are the Chat questions because we've been hashing and rehashing and we have lots of individuals so you still-- you can't hear me again. See that's why I'm making a big fuss. Can you hear any better now? Any better now? Any better now? I'm not really sure what to do. I'm right in it. I'm not sure what on earth to do here.

Jay, it's sounding better. I don't know--

I mean I'm just lift-- I'm not really doing much. Is it still better now?

Yes.

OK, I'm not really sure. I'm doing very little. In any case, I guess if it's like this again I'm just warning. I might just have to go back to phone because the cause is using laptop instead of phone the problems are always when I'm on my laptop, and never when I'm on my phone.

OK, so these are the questions that are what is the background related to the outcome. So what I've heard from a lot of people is you feel like you understand the data. You understand all these muffled voice slides. You understand what it is you need to market TE or on the update form, or whatever. But where there is still a good bit of confusion is more at real time when you're actually working with students trying to connect the dots between things, and stuff, and things that your students are doing and what that means in the wacky--

I've done nothing. I don't know what on earth has happened. I've done zero on my computer. I haven't budged. I haven't budged there are no changes between a minute ago when you could hear me fine and now. There's literally no reason why it should be different at all. I'm not sure what to do because I've done nothing. It's the exact same. Nothing's even budged. We have [interposing voices]

Jay, I think it may be a connection issue. So-- are you--

I've got four bars so I'm not sure what it is there either. Gremlins, yeah, I guess. Can you hear me all right now?

Yes, we can.

We're just going to have to because I don't know what to do. But I probably will go on my phone next time because it's clear it's my laptop. Anyway so we've got these two questions that is, what are some outcomes that you might be experiencing, either at your agency or your consortium, overall where you've had trouble deciding exactly what to market? Exactly what to record?

When we get toward the end I'll give a couple examples that some folks have given me either over the last couple days. But I'm really interested in what sort of might be tricky. If you give me a lot, obviously, I won't be able to address it now, but we really are looking at this as a potential future advance data accountability session here some time in the near future. So we're looking to plant the seeds here so you can give me some examples. Give us some examples of outcomes that cause problems.

I know pretty much all of you have certain student scenarios and/or certain CAEP outcomes that consistently cause trouble and so in particular what are scenarios? Whether it's workforce prep, whether it's DTE, whether it's deciding whether it's workforce prep or whether it's a short term service. Maybe it's figuring out EL Civics, maybe it's differentiating CAEP with WIOA II maybe you've got co enrollment issues with WIOA Title I all sorts of possibilities.

But I'm looking if you can identify scenarios that occur at your agency or consortium that you know every time you're marketing something, but you're never really that sure that you're marketing the right thing. You're guessing, you're asking me the same questions every time. You're never that happy with the answer you're getting. All those kind of things, I'd really like to hear what sort of issues you might have. And again, I really think this might be the seeds for another session because I do get a lot of questions like this. So we're really looking to roll up our sleeves.

OK, so enough on that one. I see a few of you have put input already. I'm really encouraging you, as much as possible, at your leisure, in this chat-- Veronica will save it. I'll take a look at it.

I see the names here. Just about all these names, I think I know your email or phone number anyway. But if you're not sure I have it, put it in. I probably won't address a lot of this today, but I will try to follow up the work of, I may do a future training next month or the month.

So COVID-19, same blanket statements. Here is the OCTAE website we have those several memos. I'm not going to spend too much time on that specifically. But just to say, we're pretending like COVID-19 doesn't exist for some of these things.

We're requiring all these things at enrollment. We're requiring all this testing. We're requiring hours between tasks and hours of instruction. We're pretending like all that's not really a big problem, but we know it is. So here's some blanket statements about COVID-19.

Here's a little bit of review from last week to get our foot on the rail. These are our five CAEP programs. We talked a little bit about last week-- about how seven ABG programs worked in to 5K programs. We have what we consider the three biggies at the top. ABEASE is one, ESL is the second, CTE is the third, and then we also have Adults with Disabilities and Parents K12 Success.

We also talked a lot about connecting the dots with these programs. I really feel that's a big issue that we never really covered adequately last year or any other time is how, for K purposes, we fold these programs into categories to some extent. The easy one is over at ABE, where ABE, high school equivalency, and high school diploma, or CAEP and NOVA reporting, are considered one uniform program. In most of the rest of the real world, those are three separate programs.

The other one is CTE. In this one, CAEP acknowledged is that it's CAEP just being crazy CAEP here. But we're basically folding four different things that go into the CTE umbrella.

CTE is the obvious one. Short-term CTE is second. And we've also moved Pre-Apprenticeship and Workforce Preparation into that CTE umbrella. So if you know how we have that umbrella set up, it's easy to figure out how we've set up our K programs.

So now, moving forward, so getting in to the advanced part-- you thought I'd never make it. So here's our program structure. This is literally rehashing what we said a year ago. If you're tired of it, I'll just apologize in advance. But anecdotally, with crazy stuff like COVID-19 a lot of this stuff has been forgotten. So this is what was a really big deal a year ago. Workforce Reentry was renamed Workforce Preparation.

We have a new concept, not really anything new overall, but new specific to CAEP reporting, called Adults Served. We divided it into categories. Here later on, you'll have some options, in the CAEP summary, that will address this issue as well. More on this in a minute.

Tracking hours by program for learners in multiple programs. This is one that totally got forgotten but it's been-- at least brought back on my radar screen literally the last week or two. I think what's happened is some of you have had to do NOVA program hours again. You're comparing hours in '18, '19 to what you have in '19, '20. Finding some discrepancies, this third bullet is one reason why-- that is we are tracking hours for learners in multiple programs.

This fourth bullet is even bigger reason why you might look different at '18, '19 to '19, '20. It's back to the start of last year we're no longer tracking service hours. That was the one we probably talked the most about a year ago, but we'll rehash it here again. Student barriers, everybody is doing that right, but that was a formal change a year ago.

And then that WIOA I issue parlayed into a CAEP reporting issue where at the federal level for workforce programs they had a new requirement that the feds called, Passage of Exam. We felt like that covered that skills progression, measurable skill gain at the federal level. We used that skills progression as the basis for those chief outcomes that we've called Occupational Skills Gain and Workforce Prep Outcome. So because we related those CTE outcomes so directly to that federal outcome we really felt like if the feds were changing this, we really needed to change those CTE related measurable skills outcome and gain. So again, I'm just rehashing what we talked about, seemingly ad nauseam, a year ago but again, anecdotally, it has been a little forgotten.

So the first one relates to these CAEP programs. I talked about how we folded four programs and one that we're calling CTE. This again has not been this way the whole time. This was a change we did a year ago. But one thing we did this CTE is short term CTE, to be exceedingly clear at least in TE land, you do not have any requirement to ever need to distinguish CTE, or short term CTE.

But I believe, I'm not positive of this but I believe, when it gets pulled up to the state they're using that 48 hour cut off like they use at colleges for regular and short term CTE. Again, if you have something less than 48, if you really don't see a short term at all, that's not a problem. If you've got something that's 100 hours and you categorize it as short term, that's also not a problem. But there is an issue at the state level to distinguish CTE from short term CTE.

And then we have that free apprenticeship that was its own program before. It's now folded under CTE. And then that Workforce Preparation before 2019 we called it Workforce Reentry at the start of the '19 '20 year we changed it from Workforce Reentry to Workforce Preparation because we no longer really considered it related to any age range like we did in the ABG days.

And so just to explain this slide. We talked about this a little bit last week, where everybody got on board with the fact that CTE is writing under this larger umbrella now. But there was a lot of confusion between what these four specific components are. What the heck is the difference among them.

So again, as a memory tool if nothing else, you don't if it's too much trouble don't worry about it, but CTE we're generally saying it's a long term program. It's occupation specific, by that we mean it's covering a particular trade or occupation. It's focused on welding. It's focused on HVAC. It's focused on nursing, et cetera, as opposed to workforce preparation that's not occupation-specific. And it's focused on general workforce issues-- safety in the workplace, writing your resume, sitting in job interviews-- that is, workforce skills that apply to any occupation, not just something specific.

Short-term CTE is like workforce prep, in that it's supposed to be a quick course. But like CTE, it focuses on a particular occupation. And then the pre-apprenticeship, that's just like CTE long-term and occupation-specific. But of course, the manner in which that student is receiving that training is a lot different.

I see a lot of you have put things in chat. I went on a huge tangent there. Hopefully everybody's hanging in there.

Because I see a lot of chat because I realize I went down a rabbit hole there. Is everybody hanging in there? For that matter, can everybody hear me as well?

OK, thank you. Is that a yes on both? Thank you. All right, great.

OK, so moving on to the next concept-- again this was a big deal a year ago. It's been dropped, but now to rehash it.

So we have this concept called Adults Served. So Adults Served is our generic term that refers to anybody covered under CAEP at all. You might say it is the term that we're using to replace the infamous term previously known as Touched by AVG. For a lot of reasons, we don't use the term Touched by AVG anymore. But we still have a need to have a generic term that refers to everybody that has some activity related to CAEP in some way, shape, or form.

In some way, shape, or form-- yeah, I'm not really sure what to do about the static. It is right there. It's right-- I mean my mouth is right on it. I don't know what that there is to do about it at all, sorry. I've done everything I can do. Next time I'll just go on my phone and make sure we don't have this problem.

Anyway, we've got the CAEP reports called, Adult Served. That's what I'm doing, all right I've done that a million times in the last five minutes. So anyway, we've got Adults Served. We've got Service only students. We've got those one to 11 hours, and we've got those with 12 or more hours. So Adults Served is the term we use to refer to folks in all three categories. The generic term that refers to everybody related to CAEP.

And then we've got three subcategories. The one we're looking for, of course, is number three, that is, those with 12 or more hours of instruction. That is, they qualify for federal reporting. They qualify for CAEP outcomes. They're official CAEP participants. Then we have those that participated in instruction but didn't get 12 hours, that's number two. And then working backwards, we have those Service only students. They did participate in your CAEP [inaudible] and your CAEP activities, but they weren't in one of your CAEP instructional programs they just received services.

Next one is hours by program, and again that refers to the report you do in NOVA. It also refers to that report in TE that gives you the total agency hours by program that you can use for that NOVA reporting requirement. So that started, I believe, in February, of 2018 where we started working with everybody needing to report total hours as data the legislature needed so we could see the degree of participation globally across all CAEP agencies and all CAEP programs.

So we report those hours by program. When we started that with that initial fire drill in February, 2018 everybody rose to the occasion but a couple of problems with our reports and everybody else's reports arose. So shortly after we made some changes. Long story short, it is a lot of you got odd ball numbers when you first did this because you would have hours duplicate. One major source of that duplication was what we have on the slide where sometimes you have classes that you assign to more than one program.

So the easy example would be like an IET course that you might have designated for ESL and simultaneously also designated for CTE. When we did this the very first time, out of lack of any other way to do it, TE and other programs, would put the hours in CTE and also put the hours in ESL, in my example. Quite frankly, most if you loved that because it gave you an inflated hours total. As the hours totals were wrong, but it always would give you too many not too few, so obviously most people loved that getting more hours than you actually recorded.

Anyway, but we obviously need to get it correct. So this is what we did to correct the problem. Where you designated two different programs to one class the default answer was we would just split it 50-50. So in general, if you know that 50-50 is right, in my example, half the time goes to CTE and half the time goes to ESL. You can do nothing just mark it for both programs.

You know sometimes, that may not necessarily be the correct answer. If so, then you just need to record two different classes. So if in my example, the right answer is really 70% ESL and 30% CTE, that's fine, but in that example you'd need to create two separate classes. One that you mark for ESL, and one that you mark for CTE, and then you can carefully allot the hours recorded accordingly.

I'm reading your question, Rick. Sorry. That's why I'm hemming and hawing.

I don't think I really can show it to you now, based on time and where I am. But the short answer is, if you're talking about that report in TE, there are a couple radio buttons in the middle of the setup window. I guess I'll just say, email me, and I can send you a screenshot, if that helps.

But there are some radio buttons in that CAEP program hours setup window. There is a couple of them related to ABEASE and a couple related to CTE. The default view will be the NOVA view. That is, it will give you a super simple view with the minimal amount of information necessary to complete that NOVA requirement. But if you want to stop and smell the coffee, you can do the expanded view. If you're saying you want it collapsed, again the answer is the same, it just means the change you make in the setup up window is the exact opposite. But there are three or four different ways you can run that report in TE based on those radio buttons and that TE CAEP program hours report set up. Hopefully that makes some sense.

I'll move right along, again this is advanced, so I'm talking fast and furious here knowing I've got an advanced audience. So another issue is with hours. This was a biggie a year ago where in 2018, '19 there were a lot of questions related to recording hours for short term services. So in '18, '19 just about everywhere I went and I got asked this question, and we did lots of trainings, and we got asked this question. So for about a year we would just shrug when this question was asked because we really didn't have any answer.

Then fast forward to summer 2019 we had to really discuss it at length, and figure it out. So at the end of the day, the decision was, no, service hours do not count. So last year when we did these trainings we went way out of our way to say, no, service hours do not count. So if you're doing hours oe doing services, that's great. Mark the service. But the service doesn't count under hours of instruction.

The issue that we talked a lot about a year ago and is exceedingly obvious now in the age of COVID-19 is that policy has nothing to do with the fact that you have to do things in a regular classroom. Obviously, nothing could be further from the truth, but the issue is all the hours you record do need to relate to that individual's instructional program. If you're giving the student counseling, that's fantastic, but that time in counseling doesn't necessarily relate to that individual's instructional program. So thereby, those hours do not count as part of that individual's instructional hours. So again, only count those hours related to that person's class, whether it's in the classroom, or whether it's distance learning, or whatever format you want in the instructional hours.

So my bird walk with this one is I've gotten a lot of questions here in the last two or three weeks or so because a lot of you have started running this report again preparing for that next NOVA requirement that's coming up here soon. And so a lot of you have been showing me your reports, and showing me that the long story short is our 2018, '19 hours are way higher than '19, '20. I thought '19, '20 would be lower because of COVID-19, but this is getting a little "ri-dicki-licki-lickilous" with the degree of difference.

So I can't answer everybody. But one big reason-- and I really think this is a big reason, with a lot of you, especially if you're one of those agencies that does third-party import-- because a lot of those systems were relying on this prior to summer of 2019-- is, your '18, '19 included some of those hours from services, whereas your '19, '20 didn't So making that long story short, what that means is the lower number in your '19, '20 report is probably really good and dead on, whereas the error is really more with what you did in '18, '19, not '19, '20.

That's not everybody, but I've seen a lot of them in the last couple of weeks. And I'm convinced that that's the number one reason why a lot of you are seeing big differences in terms of number of hours. Because you're not including those hours from services like you might have done in '18, '19.

I'm going to sanity check back because that was another ridiculous bird walk. So a little bit of a sanity check. Everybody hanging in there or is everybody running for the hills already? Has anybody left yet? I guess if you did, you wouldn't be still here, would you? All right, so moving along.

So here's the rationale, I won't read this it's crazy. But anyway, this is the rationale I won't read the whole thing. But here's the link to the federal regs if you need the proof in the pudding again the definition does not require regular classroom but it absolutely does require that the hours relate to that individual's progress.

This one should be review for everybody. Everybody records barriers. We won't make a big fuss about that. To be clear, the barriers are automatic as far as if you put English language learner will automatically be recorded for ESL and low levels of literacy will automatically be recorded for ABE. I think where I probably explained it wrong, and everybody heard it wrong, whatever. We'll just say it was all me but either way that still is 100% true but not necessarily from a mechanical point of view.

So a lot of you said, hey, wait a minute didn't you say, oh, ESL are going to be English language learner? The answer is yes. The answer is, I stand here right now continues to be 125%, yes, but there's nothing mechanical that does that it's more at the state level. If you report 500 ESL learners then by definition on the back end we know that you have 500 learners at your agency with that English language learner barrier to employment.

So be clear, we report that to the feds, and we report that to the state legislature. So if that's what you're worried about just know that reporting is definitely taken care of for you. But if you're worried about seeing it in a mechanical barriers to employment report, or on your program outcomes menu, or some of those mechanical reports related to entry update information then you'll still want to mark it. Again, that's just if you need to see it on some report but just know in terms of state reporting and federal reporting that's still true.

So I'm just making that distinction because some of you, I know, do like to troubleshoot with the reports. If so, you might still want to check that check box or bubble that bubble. The big change with barriers relates to that workforce prep issue we talked about a few minutes ago. Again for several years, we called that program Workforce Reentry that's because it was initially conceived to be a program to facilitate reentry into the workforce for specific barriers to employment, as well as individuals 55 years or older.

Long story short, is when in the AB 86 days there were a lot of people very upset that AB 86 didn't address older adults. The legislature said OK, OK, we'll address it, but we're not going to address it how you want it. We're going to do it the way we think is more appropriate which is to reenter the workforce. So they said, yes, we'll have a program for older adults, but it will be to reenter the workforce rather than the way we had it set up in the old pre-2008 system.

So for a couple of years, workforce reentry was supposed to relate only to those 55 years or older or in specific areas, categories, that were reentering the workforce. A lot of you were very vocal about the fact that that seemed really silly and unnecessary. So much to the point that we had to agree, yeah, that seems silly and unnecessary to tie it to barriers and tie it to an age range.

So last year we said OK, OK, uncle. We agree with you. It's silly to do all that.

So we untied it to any barrier, and we untied it to that age range, and said, yes, you can offer this for all individuals who might want to reenter the workforce. Because it wasn't really, quote, unquote, "reentry" anymore. We went ahead and now call it Workforce Preparation, which, quite frankly, is probably a better title and relates more to a lot of what we do in other walks of life anyway. So again, Workforce Preparation not tied to barriers and not tied to 55-plus years of age.

And then the final change here is a change on the WIOA I side, and we're calling it Passage of an Exam. So on the WIOA I side, they have one measurable skill gain called Training Milestone and another called Skills Progression. The short answer is those are federal level measurable skill gains that are among the five fundamental MSGs WIOA F and nationwide but those are also the ones that apply to WIOA I. They're not really reported for Title II. They're not under that national reporting system or NRS.

If you're WIOA II, you know what I'm talking about. NRS is the way all WIOA IIs are required to report. There are three of the five measurable skill gains included in NRS reporting but two of them again, Training Milestone and Skills Progression, are not included in NRS. We did include those at CAEP even though they're not in NRS however for the simple reason that CAEP includes CTE NRS and WIOA Title II does not. So we added them here.

So back to a year ago in Title I they made a big change to that Skills Progression Measurable Skill Gain they renamed it to Passage of an Exam because starting a year ago in order to report skills progression you had to have some evidence that the student passed an exam. They don't really care what that exam is. It can be a skills demonstration. It could be something standardized like COSAS testing. It could be a made up exam that you did at your own agency. It can be something from that educational software that forces the student to take exams. Whatever it might be, you can decide what that Passage of an Exam looks like. But bottom line is to really get the measurable skill gain you have to pass an exam.

So winding it to CAEP-land, those two CTE measurable skill gains are called occupational skills gain and workforce prep milestone. So now technically, when you record those in TE when you report them for CAEP reporting you still have the same outcomes but somehow, some way that should reflect the fact that the student has some sort of an exam.

I'll rewind a year ago, and also rewind to how this prompted all sorts of goofy jokes about the CAEP data police. So we admitted quite blank is this something we really feel like the CAEP data police are going to be able to enforce? We admitted quite strongly the answer is probably not. However, you're under order to have this relate to some sort of Passage of an Exam that is when individuals are doing CTE they're demonstrating skills They are passing a written task, whatever, that shows that they really are picking up the skills that you're reporting they're picking up.

So that one is a huge mouthful so I'm going to give another sanity check to that one because good luck understanding a lick of what I just said the last couple of minutes, I suppose. Some of you are quick to chime in. Is everybody still with me or just one or two of you, and everybody else has checked out? Sorry that was such a crazy-- that was such nuttiness that I really am unsure. OK, a few of you, anyway, are admitting to save us. All right. We'll move on.

So transitioning here, we're actually pretty good in terms of time. But this is the slide we did show a week ago how at the beginning of AB 86 we basically aligned WIOA to CAEP, in particular. We used what we call AB 104 outcomes. That is the legislature. In California, you passed AB-104 and basically said, for Adult Ed, the outcomes must fit in one of these six categories that we will legislate decree. So these were the six categories that they decreed, over there to the right.

And what I always like to point out is, not so coincidentally, they are very similar to-- almost word-for-word the same as-- what the feds have had in WIOA since 2016. In the federal reporting, they have what they call performance indicators and measurable skill gains a couple of our CAEP outcomes are what the feds would call performance indicators. Most of them are what the feds would call measurable skills gains, or MSGs, either way pretty much all of the things that we record in our CAEP data are directly begged, borrowed, and stolen from the federal WIOA system as illustrated by this diagram.

Moving along, here are those six categories of AB 104. And here is the detailed laundry list of all the outcomes that fit under each category. This was a big part of what we did in training a week ago. Where we turn back the clock and identify these six areas and try to give a dictionary definition last week of all of these different areas and what they meant in terms of reporting.

So here are the literacy gains. I think we've covered this as well. This is just showing the measurable skill gain area of CAEP reporting. We call it literacy gains. These are the five ways in which a student can achieve a literacy gain. The most prominent way, of course, is through pre and post testing. We have Carnegie units also known as high school credits. We have the CDCP certificate on the college side.

And then there are the two CTE-related outcomes with the corresponding things you can mark to show them. To parrot what we said last week, which just parrots what we said a lot, a few years ago is, when we started identifying all of these CAEP outcomes, what we talked a lot about then was-- the word we used was "retrofit." We had all of these bubbles and all these checkboxes already set up. We had these terms that the feds had already required us to use for WIOA.

So we talked about how we couldn't really-- to some extent, we couldn't, and to some extent we didn't want to. You might say both were pretty emphatic. But either way, we did not change the terms, because we had the terms the way we had them because the feds said so. So we didn't really feel like we could change.

But the feds required us to name all the outcomes. But we did feel like the ones we had related quite closely to the ones we were identifying for CAEP reporting. So we retrofitted the CAEP outcomes to these specific bubbles or checkboxes on the TE entry and update record.

For the most part, that retrofitting makes sense, but I will openly admit, especially now in hindsight, some of that retrofitting does not make sense at all. The command decision continues to be. Changing it right now would only do hurt. We've disseminated these PowerPoints and these Bubble Boy diagrams all over the place, and then some, since 2017, so I'm not sure we're doing favors by changing it. So mostly I just admit there's some retrofitting that looks a little goofy now, but we're just leaving it the same because we feel changing it would be worse, not better.

But this shows how you can record these different outcomes again pre post. We're not recording anything we're just entering our pre post results. High school credits, it's got a special way that's the next slide. And then the other areas of literacy gains you simply bubble or check it when the student accomplishes the outcome. Here is that high school credits. We talk about it every time but there is inevitably somebody that wants to know this. So here is the slide for how to record high school credits.

To be clear, this is something the NRS acknowledges so it's part of Fed tables. It's part of WIOA II payment points and it's also, of course, part of CAEP reporting. But what it says is it's mostly for your high school diploma students that is if your students in your high school diploma program start the year at the ninth or tenth grade level and they finish the year at the 11th or 12th grade level that's a Carnegie units or high school credits gain. This slide shows how to record it in TE.

Long story short, is there is a place to record high school credits on TE, but all districts tend to be a little bit different from each other. So we know that every district is a little different. As long as districts are a little different, using high school credits obviously will not work. So we use self-reported level report this outcome.

You would mark ASE low upon enrollment. ASE low is the federal level that represents ninth and 10th grade level. And then some time later in the year you would go back, create a second record from this screen, and the second time you would mark ASE high. ASE high at the federal level corresponds to the 11th and 12th grade level. So you would mark that later in the year to show that student progressed in high school credits.

This one, I don't think I ever do a training without mentioning this, and this training will be no different. What the heck is the difference between occupational skill gain and workforce prep outcome? So both of them relate to CTE. Both of them are measurable skill gains. The difference is there's no absolute here, but in general, mark occupational skills gain whenever the student accomplishes a portion of a longer term program. Mark workforce prep outcome for full completion of a shorter term program.

So the occupational skills gain example, we have a welding student. We're just making it up. We'll say it's five modules, five semesters long, so the student completes the first semester module, does a great job. Ace's the written test at the end of that first semester module is ready to move forward and start module in semester number two, so that would be an example of occupational skills gained. They competed the first big portion and are ready to move on to module number two.

Workforce prep outcome, the example is a 15 hour course on job search strategies. So they have some kind of documentation that they are-- it's an informal certificate, or whatever. But you have some kind of way to document it for the passage of an exam part. It's a short term local course so at the end of that-- Hello?

[interposing voices]

Hello? I think that's background noise. Anyway, so they are they're still around at the end, so that's when you can record workforce prep outcome.

Another one is how do we relate this to post secondary? So we have those six categories, one called literacy gain another is called post secondary. In federal reporting land, in general, post secondary is a higher level outcome that is post secondary is represented to suggest completion of something. Whereas a literacy gain or MSG is a way to measure the fact that the student progressed but in program. And show that they're making progress even though they're not through yet. So we consider post secondary to be a quote unquote, "higher level outcome."

So we talked about the two literacy gains are partial completion of a longer term program or full completion of a shorter term program. So it rises to the level of post secondary when we have full completion of that longer term program. So in that 15 hour course you're never really going to have a post secondary outcome for that. All we're really going to get out of that is a workforce prep milestone.

But if it's a longer term program, like welding, or nursing, or something like that at the end of the day and my example they finish all five modules they get that official licensure that certifies them as a welder in state of California. Then you would mark post secondary outcome. You would mark one of those outcomes you can see in the upper left here where they attained a credential and they get occupational licensure.

Passing one section could potentially be a literacy gain, I reckon. We're really just looking for other things there. But I'm glad you have it in the chat. You can bookmark yourself if you want that. That's it.

Let's see. I'm not sure what you mean by STV. I don't know what STV means, so I'm not sure about that. Connie, just bookmark it. I'll see it in the chat.

Potentially it would, I mean given that it's 120 hours I would say, yes. I would say it sure sounds like post secondary to me. But I know there's issues that the colleges have with hours with which I know absolutely nothing. So put an asterisk by my answer because I really don't know what those hours mean, in terms of long-term and short-term, when it comes to college CTE reporting. So as long as there's nothing related to that that gets in the way, I can attest that there's nothing in CAEP-wide is it gets in the way from recording it is as post secondary.

Good questions, just to keep beating it in. These are good suggestions. I'll definitely be reviewing this for all the reasons I gave you at the beginning. But let me move on. Believe it or not, we're actually making pretty good time. So here is that issue with Passage of an Exam. I won't repeat myself, but I put this here just to reiterate myself and explain that these are the actual bubbles you're marking, just to make it a little more clear.

Though again this is a retrofit anomaly for whatever reason when we decided what we wanted to do for occupational skills gain. We decided that there should be two different bubbles for that outcome. When we just went to mark for work preparation outcome we only decided that there should be one bubble. There was no good reason and that's what we decided. That is a question that comes up sometimes, so I'm making a big stink of it. It's not really a big deal. But for what it's worth we chose two bubbles for one and only one for the other.

Transition is another area that gets a lot of the question-- we've three different transitions ASE to CTE in college. ASE is the one under the hood. You don't need to mark anything, but for what it's worth, it is something we track that is learners that start in either ESL or APE and they end up in either HSE or high school diploma, technically that's called transition to ASE.

The other two you do mark. You've got a few things you can mark for transition to CTE. A couple other things you can mark for transition to college. This diagram, anything but new. But this is just showing how you can figure it out is the difference between college and CTE, everybody gets that easily. But once we get into these exact X's and O's, that's where a lot of people tend to get confused.

So to reiterate, the person starts in either K12 adult education or non credit community college, ABE, ASE or ESL. If they transition to one of these over in the upper right, one of these red boxes on the upper right, that suggests transition to CTE. That is, if they fit either community college CTE or a K12 adult education CTE, that's the correct end point.

If you look down below, if they end up in for credit community college that's where they need end up to transition to college. What I'll point out is we go out of our way to specify for credit here on the low far right. If the person transitions from K12 Adult Ed non-credit community college, we would consider that more of a lateral transition. For it to really count as a CAEP transition, they need to transition into for credit, not non-credit.

It's the transition to post secondary is picked up by what you mark. So back up here, good question. So ASE is under the hood. You might say that we decide-- I mean it may have been smart. It may have been not so smart, but we decided to keep all that under the hood just because we know everybody in ABP, ESL, and ASE. You're required to mark, so we felt like that would be easily accounted for under the hood because everybody in ASE, ABE, and ESL would definitely be there.

Once we get into for-profit college and some things in CTE, though, way less sure whether all that is really going to be recorded in TE, so we thought that would be better set for self-report. That's the explanation, Dana, I still think that makes sense, but I admit it's hardly an absolute answer.

Yes, so you will see it on the report Jane. But just so you know, under the hood, once it gets to launch board, once we report to the legislature, there's an under the hood outcome that will do that automatically for ASE. There's no data match that I know of. You may want to email those watch board posts. There is a data matched that we use with the TE data that gets data matched with Chancellor's office data before it gets put into launch board.

My understanding is it does get checked for a lot of those transition to post secondary. That's another way where we boost that number when it gets on watch board and on legislature reports and so on. So there is a data match that we do with the Chancellor's office and TE data. It does look for transition to college. So yes, there is a data match, but I'm not sure it will necessarily specify that specific transaction.