HOLLY CLARK: Jay Wright. Jay. 

JAY WRIGHT: OK, just like that. All right, so we'll blow it up. And I think you have the PPT. So you can send when it seems advisable to do so. 

So welcome, everybody. I'll backpedal immediately and just say, hey, I'm looking at the audience that doesn't look like a basic-level audience to me. It looks pretty gosh darn close to the exact opposite of that. 

But I will say this one really is intended to be a basic-level accountability. It's about an hour in length. It's obviously an adaptation of the one we did for the face to face last year with a few little updates for '23, '24, of course. But it was intended to be an online just basic accountability. 

We don't really have the big stuff going until CAEP Summit. That seemed a little too late to do heel dragging. So we did decide to do a couple real basic-level sessions with CAEP TAP here online here in early September with the idea that, hey, that timing might be a little bit better. So this will be that for the final summary of backpedaling. 

This will for, I think, all. But a few of you, this will be 90% to 95% stuff that you've not just heard before, but probably heard many, many, many, many, many, many times. It'll be whatever the exact opposite is of new. But, hey, always good to get a little fresh face on stuff you've heard before. So here it goes. 

So here's the agenda. This should reinforce what I just said, I think, real basic-level agenda items. We're going to define the CAEP programs. There are a few little changes in NOVA that are worth mentioning. So, hey, there'll be a little bit-- there'll be at least 90 seconds of this hour and a half that might be worth it for some advanced people. 

We'll go over some outcomes and services. We'll stick to the basics on what the outcomes are. We'll stick to the basics on what the current services are. A lot of you have been at a lot of these meetings in recent years. So you know yahoos like me have spent a lot of time talking about how we need to revamp this. 

We've had a lot of activities starting to get at that and move that direction. But as is, we haven't moved one iota. So as is, we're still looking at the same bubble boy slides and the same set of outcomes and the same set of services for now. I think for '23, '24, probably safe to say that's what we'll be using. But we'll be continuing to look at new ways of looking at it and what might work better. 

Basic data collection requirements, that really reinforces the basic-level training. Again, a lot of you that have been to these over and over again will recognize, hey, wait a minute. These are slides that I don't recognize from CAEP circa 2017. These look more like Title II circa 2003. That's because a couple of them are-- and they're updated-- more updated than that. 

But, hey, lots of real basic-level stuff on entry, update, record, timeline, deliverables, CASAS California accountability page, stuff like that with basic data collection requirements, timeline, deliverables, and resources. And then we'll finish out on CAEP reports in TE, not really a deep dive from a TE point of view. But from our perspective, obviously, it's hard to talk about CAEP accountability without at least covering the TE report. So we'll show what some of those basic reports are. 

Some of the key considerations for each, and that will be it. I'll just say, to what Holly said, because this was intended to be basic, it really was built in to try to have some Q&A and so on. So most of this is a lot of Holly stuff just doing some housekeeping. Great. Thank you, Holly. 

So I think it's mostly Holly stuff for now. But I'll take a look, but lots of chances for Q&A. My hunch is unless we have a lot of Q&A during the slides, which is certainly possible, we'll have time left over. So if you have a lot of questions, my hunch is that there might be questions on things not really related to the topic at hand. Kind of OK with that, too, because I have a funny feeling we'll have time. 

So starting with program structure-- I know a lot of you have seen this one before, mostly just an oldie but goodie. I like to start with this one. There are at least a few of you I see that are names that are not super familiar. So that does make me a little hopeful. There's at least maybe four or five of you here in the audience that really are new to all this CAEP accountability madness and really do need a basic-level understanding of the issues we're talking about. 

So for your benefit, here's a good square one where we've got, starting with the map of California. That is, the way the structure works for California Adult Education Program is it's aligned according to community college boundaries. 10 years, ago when we were AB 86, there was an effort to align the community college districts and the K-12 districts into separate adult education regions. 

Bottom line is they were aligned according to community college boundaries simply because they're bigger than K-12 boundaries. So, well, then 72, now 71 community college districts and California all had their own region. If you were a K-12 district within that community college boundary, by definition, you were with that community college. So the college and all of the K-12 districts within that college district comprised those consortia. 

So we have 71 regional adult ed consortia, including members, which are basically those from the K-12 districts and the community college districts. Congratulations, college people and K-12 people. You're in the consortium. Whether you like it or not, by definition in the legislation, the college districts and the K-12 districts are inside the 10. If you don't want to be, well, you're inside the 10 anyway because the legislation says so. 

Then if you're not from a college and you're not from a K-12, you may be a partner. If you're a partner, then you were accepted by the K-12s and college district in your region. You are allowed in the club. 

If you're a partner, the tables turn. You're allowed in if your members want you to be there. If your members don't want you to be there, you're not necessarily part of the club. So you're, like it or not, is-- you may be allowed, or you may have to be outside looking in. 

But we have members that are the ones the legislation requires to be part of the consortia and then partners such as libraries, CBOs, one stops, and so on that you may want to invite in if your region chooses to do so. That's been the structure we've had since about the 2015, '16 year where we have the regional consortia that manage themselves locally based on how your regional structure determined was the best way for your local region. 

Looking at structure from an entirely different vantage point, these are the instructional programs. So we're not looking at it in terms of geography. We're looking at it in terms of schools and colleges and programs and students. This is how we set it up in TE. 

Anecdotally, the way NOVA has it set up now is a lot closer to the way we have it in TE. So I think it's a little easier now to explain. There used to be more of a disconnect between NOVA and TE. 

But in TE, we have seven programs. We've got the three primary ones. They're right up at the top-- ABE, ESL, and career tech ed. Under career tech ed, we also have workforce prep and preapprenticeship. 

Then we have adults with disabilities. And we also have parent supporting K-12 success, not the way they probably want to be characterized. But I do say those two tend to be a little bit flapping in the wind because they don't really connect with the other programs. But the legislature at the start said we needed to include those programs. So we definitely make sure we include all seven programs when we're reporting our CAEP data. 

Looking at the primary programs in a little better detail, again, we tend to roll it up according to these big three. I think most people agree with that, not everybody. But we've got ABE/ASE. We've got ESL. And we have career tech ed. 

I used to drum this up to say, hey, the way things are in CAEP land are a little different than the way they have it in TE land, quite frankly, which makes CAEP land a little different than the other 98%, 99% of the world as well. Again, it feels like NOVA is a little bit more like the rest of the world now, and that it spread it out and change the programs. I'm not positive of that. I'm not the NOVA person. But I've seen screenshots. It looks more like what we have in TE. 

But, anyway, just to explain, we have ABE and ASE. That's in a bucket some people get confused by that in TE and most other reporting areas. ABE is one program. Adult secondary is another program. 

In a lot of real-world scenarios, ASE is actually two programs, that is, high school equivalency and high school diploma. So, again, some would call this one program. Some would call it two. Some would call it three. Either way, all of those programs that you now and love are in that ABE bucket for CAEP reporting. 

ESL doesn't generate much confusion or questions at all. ESL is ESL. Kind of think the way we do it for CAEP is just like the rest of the world there. CTE is where some of the confusion kicks in, is that it rolls into CTE sometimes. 

But within CTE, it's broken up into those four bullets. You can see in CAEP land, there's a lot of effort to distinguish CTE and short-term CTE. I'm not the decider on that. I believe there's a 48-hours issue within the college system. 

If you know the college system, you probably know that better than I do. I think that's the cutoff point. But I'm not sure of myself there. But because there's a lot of distinction in the college system between CTE and short-term CTE, that's a distinction we make in CAEP land. 

Then we also have preapprenticeship, which is a workforce-related program, but definitely not the same as CTE workforce preparation. You might say the same thing entirely different from those others, but, again, not the same thing as CTE. 

OK, now the chat's empty. I guess the question were about PowerPoints and registrations. When I was talking about easy stuff, it felt like the chat was off the hook. Now I'm talking about this convoluted stuff, and there's no questions. Go figure. 

I will at least stop. Is this resonating? I was kind of expecting there to be questions now because it feels like we're getting convoluted about now. No, nothing, everybody? All right, I'll take no news to mean good news, I reckon. 

OK, so let's just go to the next slide. So here's that same yellow box for CTE. Again, these are not official. I don't think you're going to go on Google and find this. You're not going to find any CAEP documents that say this is the official clarification. But I do think this helps to figure out what's going on with these different subprograms. 

So CTE-- obviously, CTE, hopefully, everybody knows what that is. But in general, that's workforce-related CTE is, usually, a longer-term program that focuses on a specific occupation. Short-term CTE, of course, just like CTE, focuses on a specific occupation. 

But who's buried in Grant's tomb, hey, short-term CTE is short term. Regular CTE isn't. Again, I think the cutoff is 48 hours of instruction. 

Preapprenticeship gets folded into CTE in some areas because it is workforce related. Like CTE, it's long term. Like CTE, it focuses on a particular occupation. But very unlike CTE, it has obviously a completely different format for the student. So it's considered a completely different instructional program. 

Workforce preparation maybe look similar to CTE but is focused totally differently. It's something for the short term. And I think more importantly, workforce prep is not occupation specific. That's really probably the single most important distinction of this slide is people always get confused between CTE and workforce prep, not foolproof, I guess. 

But in general, CTE is always focused on a particular occupation-- HVAC, welding, nursing, et cetera. Workforce prep isn't. It's workforce-related instruction. But it's usually more generic-- workplace, safety, resume, writing, filling out job applications, et cetera, et cetera. It focuses on work. But it's not related to a specific occupation like CTE. 

I'll just say nobody's responding. I'll just have to assume no news is good news. I think they're freezing the screen with those 35. I think I've probably lost all of you. It's just my screen kind of lying to me saying there's still 35 people around. 

But, anyway, looking at it completely, differently, totally-- all right, thank you-- totally different angle, this is summarizing a bunch of slides I cut out because it's like so 2019. But just a reminder that we're aligned to WIOA. WIOA is the federal structure. More on that over the next few slides, getting more gory detail-wise. But I do like to point out that WIOA exists. 

Here's the slide, again, another one that a lot of you have seen, triple-digit number of times over the years, explaining these four title numbers under WIOA. We're Title II or adult ed and literacy. Title I is workforce. They're the ones that have the lion's share of authority, lion's share of responsibility because the federal legislation says so. Title III, short-term workforce; Title IV VOC, rehab. Those are our WIOA partners. 

I make a big deal about this because it's written into the legislation there 10 years ago that our consortia are required to align with WIOA. That means we're aligning and collaborating with our WIOA partners. It's a big part of the process. When you look at the data-reporting structure, we're really looking to align to that federal system to the maximum extent possible. 

I'll also just say there's kind of a spirit of CAEP, spirit of WIOA that I'll always say was on purpose. I'm not sure if it feels that way anymore. But, hey, in 2016, it did anyway, where there was a lot of movement toward more and more collaboration after many years of not really doing all that much of that. 

So we were collaborating our brains out there for a few years that felt like the federal legislation required us to do so for WIOA. We were exploring all these new partners we didn't know. Hey, what do you know in CAEP land, that AB 86 land? We were doing exactly the same thing, a lot more collaboration. So just know we need to be looking at our WIOA partners, just like we're looking at our CAEP partners. 

I think in most regions, maybe you differ from this. I'm not sure. But it feels like this overlaps a lot. And in the high-performing regions, probably feels like it's the same collaboration, not separate and different. Anyway, this is the WIOA information. 

So now we're moving into CAEP land. It started with AB 104. That was legislation from 2013 10 years ago, again, California legislation. I'll admit if you type in AB 104 in Google. You'll get this as a hit. But you'll get a bunch of newer AB 104s. So I admit from that standpoint, it's confusing. 

But the AB 104 in 2013 basically required California Adult Ed to align to the federal system, align to WIOA. So this slide is what I've used forever to show how we align the data in that fashion. So TMI here, but for federal reporting, the feds have what they call performance indicators. That's in the upper left. There's more, but that's what we have aligned. 

And within performance indicators, there's a subset called Measurable Skill Gains, or MSGs, you can see those MSGs listed. So what the state of California said is our adult ed program is going to align to that federal system they came up with these six areas of AB 104. 

I used the slide to show not so coincidentally, the state of California looked at those federal performance indicators and MSGs and came up with basically those six pillars of what we're required to do. In California Adult Ed, you can see not so coincidentally, the exact same things we already had in the federal system. 

The one thing I'll point out that's a little bit different is I'd argue those top five begged, borrowed, and stolen from the federal system, not a lick of difference. But they did add number six. That's why it's in italics. That is transitions. 

We've been recording transitions in CAEP the last 10 years ever since. It's not quite so obvious where transition fits in the federal system. So we'll just say, yeah, in this case, the California legislature went out of its way to say, hey, we need to align to the federal system. And in addition to that, we also need to go out of our way to record student transitions. 

So that brought us to this slide. Another one that I know a lot of you have seen many times-- again, we just talked about the six areas of AB 104. Not so coincidentally, here's 1, 2, 3, 4, 5, 6 pretty little colored boxes corresponding exactly to the six areas of AB 104. 

So this is showing how we did, it number one, and providing some of the details, number two, again, literacy gains, secondary, postsecondary, employment, wages, transition. Not everybody agrees, but I think most people agree those are six pretty basic areas to be reporting. Most people agree, yeah, these are probably the areas that are really good to be reporting and showing that our students are making accomplishments in these areas. 

So the detailed alignment from about five or six years ago represented in this slide where we had WIOA outcomes, we didn't necessarily choose all of them. But we came up with some specific outcomes, again, aligned to the federal WIOA system. We did what we called retrofitting, aligning some of these outcomes, some of them like GED and high school diploma, not really much retrofitting what you're marking in TE or whatever exactly the same thing as the outcome we're looking for CAEP reporting. 

Others-- segueing here-- are more like the bubble slide where we have some of these WIOA outcomes that were required to collect for TE because the feds say so. When we looked at this, they're five, six years ago. For state level reporting, we knew we wanted to collect a lot of the same outcomes as required for the federal WIOA system. 

But, hey, we're adult schools. We're community colleges. We're not exactly the same thing as one stops or AJCCs or whatever. So we need to make it a little different. We need to retrofit it so it really fits a little better to what we're doing in adult ed. So that's what we came up with this monstrosity. 

This is what we've called the bubble boy slide. This basically aligns the bubbles on the CASAS update record, the boxes that you can check for student outcomes in TE with the official outcomes that we identified for AEBG and CAEP reporting six or six years ago. 

So just to make sense of this monstrosity, I'll point out, again, 1, 2, 3, 4, 5, 6 pretty little color-coded legend at the bottom aligned to those six areas we've talked about. So for example, got a job up there in the upper left. It's color coded and blue. You look at the legend at the bottom. 

Surprise, surprise, blue is one of those that we recognize for employment. So surprise, surprise, when you get a job, that corresponds to a student outcome in the CAEP area of employment. Why? Because this bubble boy slide says so. 

I'm being a little cutesy about it, I guess, probably just because, hey, we've talked about this so many times. That's my story, and I'm sticking to it. Hey, does everybody understand this? 

Some of you, if you don't know by now, I guess you ain't never going to. But there's at least a few of you I should stop being too cutesy. There's some of you really hearing this for the first time. So, hey, those of you that are, is it making sense, yes or no? For you, it may not be so flippant. I'll take that as good news and move on. 

So next slide, this is exactly the same information as the previous slide. But it's using the screenshot directly from TE rather than the CASAS update record, looking at checkboxes rather than bubbles and aligning it by letter codes rather than color codes, really just looking at it from the direct and TE perspective. I believe all the checkboxes are the exact same checkboxes in this slide as the bubbles, as we just saw previously. 

OK, this I'm using as a segue where we just looked at the bubble boy slide. I don't really want to get into all the bubble-related details. I do want to make sure everybody has it for reference. But yeah, it gets a little nutty going bubble by bubble. 

That said, this slide sure contradicts that, doesn't it? But I like using it just because this is the area that I waxed philosophic about when we talk about outcomes. A lot of you know this. This is the area for my two cents that tends to generate the most questions and so forth. 

So I'm going to read this one off, even though it contradicts just about everything I've said to date with the training. So we'll start with prepost gains. When we say literacy gains in WIOA federal reporting land, a lot of you know not all, but most of the time, we're really just talking about pre and posttesting. 

What we're talking about in CAEP land, probably safe to say not all the time. But most of the time, it's still pre and posttesting too. Just like a lot of you know and love-- I'll say 90% of you know this, I think-- we do our classes pre and posttesting. We're not marking anything because the test results speak for themselves. 

We don't need no stinking bubbles. We don't need no stinking checkboxes. We just do the testing and e-tests. And the results work out automatically based on those test results. And We will get into this in more detail. But those pre/postgains you know and love are one of those outcomes for CAEP. That definitely does count and is a vital part of CAEP just like WIOA II. 

Because it's here on this slide, I'll mention Carnegie units. It's not my top priority item. So nobody thinks I'm dodging. There's those high school credits, outcomes. I'm going out of my way to not show that slide lately, because we need to come up with something better for that. But we haven't come up with something better yet, but it's not going away. 

If you've got a detailed way to account for Carnegie units or high school credits, that's another way to show a literacy gain. If you've got students in high school diploma that are earning high school credits, potentially, you can get a gain if you can show that high school diploma student has moved from the 9th or 10th grade level to the 11th or 12th grade level within a year. That's also a literacy game. 

CDCP certificate is another one I'm not that interested in talking about. But, again, so I'm not dodging. I'll point out the very first year of CAEP, that AEBG reporting, hey, you're getting TMI here. But some people like it. That very first year, everybody reported in TE all the CAEP community colleges and all the CAEP K-12 districts. 

For one year, anyway, we were one big happy family. But that only lasted one year. After, that the colleges peeled off and used their community college MIS. So this is a leftover. It's not really one that's a big part. It's a big thing for colleges. 

But if you're marking it for colleges, you would be marking it in your community college MIS, not TE. So for the most part, got to say CDC certificate, 0% my problem now, 125% community college, MIS's problem. But, hey, if you mark one of these bubbles, it will still show up as a literacy gain in a no harm, no foul sort of way. 

The reason why I make a big deal about these literacy gains, of course, are these bottom two-- occupational skills gain and workforce prep milestone. Those are the ones that we retrofitted for CTE and workforce prep. What I like to say here is, again, we're going all out on TMI. So why do it any different? 

But what I like to say is for programs like ESL and ABE, this whole issue with literacy gains and being able to show that incremental progress, mission accomplished. Hey, you're listening to the CASAS guy. So the CASAS guy is automatically going to call that a mission accomplished. We don't need to worry about how to show incremental gains for those students. 

That's why we have CASAS testing. We already have the ideal way to show and record exactly that. So that's how we're going to do it for ABE, ASE, and ESL. 

However, we got to admit, for workforce prep and for CTE, that's not what pre and posttesting is for. There's probably different better ways we really need to look at and use for those programs, not really CASAS testing. So that's where these came in. 

We were looking at ways to show progress for CTE and workforce prep, just like we do for the WIOA II programs without relying on the final prize. And I didn't really set that one up well. But back to this slide-- sorry, I'm pulling the rug out from under you-- is with this slide, I like to point out the performance indicators MSGs for my two sets performance indicators are those outcomes that are a final prize. Get a job. Enter college. 

OK, I'll get to that in a minute. Enter college. That is those outcomes you can hang your hat on. Measurable skill gains is looking at it to show that incremental progress. 

The way I like to frame it is this. When people look at performance indicators, which also includes things like enter college and get your occupational license, not just employment and wages, everybody agrees all of them are really, really valuable, wonderful outcomes. Nobody quibbles with that. 

But people also agree if all we do is performance indicators, then we're setting most of our students up for failure because it takes a long time to get the diploma, earn that college degree, get the pay raise, get the job, whatever. That takes time. Most students don't do that within a year. 

That's why we have MSGs. That's why we have wonderful things like CASAS testing. So we can still show beyond a shadow of a doubt that our students are, in fact, doing a wonderful job, like most of us already know. 

So back to the retrofit, for ABE and ESL, mission accomplished. But for CTE-related programs, not mission accomplished. That's what we have here. Go ahead, Holly. I've been babbling too much. So I better just stop and hear the question. 

HOLLY CLARK: No, it's OK. We do have a couple of questions in the chat. Tashelle asked, "Can CTE use pre and posttests to show literacy gains?" 

JAY WRIGHT: If you're talking about this, yeah, keep in mind you're asking the CASAS guy. So you're letting the fox rule the henhouse here. But yeah, as long as the fox rules the henhouse, that answer is obviously going to be yes. 

And we have said that if you want to test your CTE or workforce prep students, obviously, CASAS testing is one option you could use. We admit a lot of CTE types would say, hey, we have mo' better ways. That's what we like to do. If so, we understand, and that's fine. 

But yeah, if it's just easier to do CASAS testing for them, why sure, if they make gains. And they show progress, we'd still certainly say that's a good way to do it. 

HOLLY CLARK: Perfect. And then Carla asked, "What's an example of met work-based projects? 

JAY WRIGHT: OK. So yeah, I saw that one and went off on a tangent and [INAUDIBLE] about it. 

HOLLY CLARK: OK. 

JAY WRIGHT: OK, I think, really, you're asking this from the, hey, what does it really mean not what does it mean on the goofy retrofit slide? So you're going to get way more TMI than you bargained for. Work-based project is a bubble that's been there since the late '90s. It started in WIOA. 

I can't remember why. But it was a WIOA thing from literally '99, 2000 where it was in the old VABE and VESL where there were VABE and VESL students that did things the way a lot of students do it now, where they were doing it in blocks. My recollection is it was like 30-hour increments where you might have a CTE student in ABE or in ESL for that very fixed period of time. 

Here's where the crazy TMI is that you weren't bargaining for is for whatever reason, that would be a drop reason under WIOA. So we went out of our way to have the bubble. There were WIOA funds that served work-based projects students. But I think it was because they were vocational or whatever. 

Back then, that meant no NRS outcomes, thereby no payment points. So it was a weird drop reason. We don't really use it as much now under WIOA. I haven't seen anything about that since WIOA. 

So, hey, back to the retrofit, that was a first round draft choice for retrofitting because we knew from a WIOA point of view, we probably weren't going to really need it anymore for federal reporting. How about that? Talk about getting way more than you bargained for. I think that's like what you were actually asking. If not, I guess you'll let me know. 

HOLLY CLARK: OK, perfect. And then Teresa asks, "If a student transfers from our HSE, HSD, ESL program to a noncredit college course or even career courses, where would we mark that?" 

JAY WRIGHT: I think you can look in the bubble boy. But I believe there's a transfer to credit and transfer to noncredit. Those are two bubble boy options. So in your example, obviously, you'd want to transition to noncredit. 

HOLLY CLARK: OK, great. Thank you. 

JAY WRIGHT: But the short answer is you can verify. That's why we put out the bubble boy slide. Obviously, it's hard to look at every bubble and prescribe bubble by bubble by bubble. But we do go out of our way to add that crazy slide just so you have that reference, so you can look it up. 

HOLLY CLARK: Great. And then-- 

JAY WRIGHT: I don't think there's a transition from a WD to something. But, hey, you can send me that. That seems like an obvious suggestion. Some of you might have heard that before. If you did, I'd be interested. 

But I'm going to say that seems like a really stinking obvious suggestion. But I'll go on a limb and say I don't think I've heard anybody suggest that to me before, as obvious and straightforward as that one sounds. I'll admit that it does sound obvious and straightforward. 

And at surface glance, it sounds like a good idea. But I don't know of anything right now where we're really tracking it. So I'm going to have to say no, I don't think so. But I like it as an idea, not that that's what you're hoping or looking for. But for what it's worth, I do like it as an idea a lot. Better than nothing, I guess. 

But yeah. And I'll add on, as you know, we kind of are thin-- our bench is thin in adults with disabilities. We could probably use a few like that in adults with disability. That's a small little roster. We could probably use a few more bodies on that team, so to speak. OK, anything else? I'll just say, hey, Holly, anything else I missed seeing as how you're doing a good job interjecting here? 

HOLLY CLARK: No, don't think so Carla mentioned the definition sounds like the 243 COAAP. Is that how you say this? 

JAY WRIGHT: OK. All right, so occupational skills gain, workforce prep outcome, I'm sure I've done more than enough setting it up. So we'll talk about the outcomes. Again, this is how we set it up. 

My setup is you can use occupational skills gain for when students accomplish a portion of something longer term. Use workforce prep milestone when the student completes something shorter term. So you can see the examples. 

Example for occupational skills gain, we'll just say a five-semester long welding program. They complete that first semester, do a great job. Maybe they CASAS test because they're working for Tashelle. Maybe they're doing a skills check. Maybe you're doing your own agency homemade written test. All those suggestions are fine. 

But you should give some kind of exam, whether it's skills check, written or otherwise, that proves that that student's doing a great job. If so, you can record occupational skills gain that shows they're moving in this example from module 1 to module 2. 

Workforce prep outcome, the example is, hey, they're in a 15-hour course on job search strategies. In this case, the formalization is very low. Hey, this one could be that certificate that the secretary prints out of her computer or whatever, but something that shows that the student did all 15 hours. You can then mark a workforce prep milestone is another way to show that your student is making progress toward those other things. OK, thank you there. 

OK, taking it the extra step, again, this is looking back to what we talked a lot about in training and previous years. But one of the big confusion points of CAEP accountability training has been that ability to distinguish those CTE literacy gains. The next step out of that is then looking at those CTE literacy gains and being able to distinguish those with what we already have on the table for postsecondary. 

So back to that measurable skill gain versus final prize performance indicators, you might say the ones we have for postsecondary are more like the final prize. Some of them are college. Some of them are workforce training because we're talking about CTE here. We're looking at the side of postsecondary related to workforce training where we've got that occupational licensure and occupational certificate. 

So back to the slide, when you've got that CTE student that's progressing within an occupation or something like that, you can mark workforce prep milestone. You can mark occupational skills gain. So, again, those literacy gains are partial completion of something longer term or full completion of something shorter term that rises to the level of a postsecondary outcome upon full completion of that longer-term program. 

So back to my example, if the welding student is finishing module 5 of 5 and getting that occupational certificate, then you're not marking occupational skills gain anymore than you're marking that postsecondary outcome to show that they achieved that final prize. OK, hopefully, this is making sense here. I'll have to say no news is good news, like usual. 

So here's a little more on postsecondary. This is just a laundry list, a lower volume problem, I'll say. But, hey, with postsecondary, it's a really big bucket. I do think sometimes the fact that it's such a big bucket confuses people. The fact that it really is two buckets in one represented by the slide also confuses people. 

The two buckets I like to say is we've got this bucket full of workforce training outcomes. At the top, we've got similar but separate bucket of outcomes that relate to college here at the bottom for federal reporting that all fills into that very large bucket called postsecondary, another obnoxious explanation about federal reporting here. 

But in federal reporting land, the feds use the term "postsecondary" to refer to all those student-level outcomes that can be obtained after secondary. Go figure. You have secondary, and then you have postsecondary, surprise, surprise, on that one. Sorry, I'm being a little cutesy. 

But secondary, of course, relates to those high school-level outcomes. That refers to high school diploma and high school to equivalency. Once you get that high school diploma or high school equivalency, a little bit-- but because the feds say so, but a lot of bit, because in this case, the feds are just using common sense. Once you finish high school, you move on to postsecondary. 

For some students, that means move on to college. For other students, that means move on to workforce training. The feds use postsecondary to refer to that whole layer that's one layer up from secondary. I don't know if that makes sense or not. Maybe it does. Maybe it doesn't. 

But sticking with more information about postsecondary, admittedly, here's where I'm being a little bit of a sadist. I'm getting deep in the weeds after saying I'm staying out. But this is just a reference for you because a lot of times, it's hard to decide what's really postsecondary. 

I'll elbow you all in the ribs and say where this has probably come up the most is more on the WIOA side when we're looking at IET and doing that IELCE report or whatever. And you've got all those indecipherable gray lines between what really counts for IET and what doesn't. This is a slide we've used for that. I've backed it into CAEP because I think it helps determine what's postsecondary in what ain't. 

So this is straight from the federal website. So I know it's correct, by the way, except for that dotted line that I drew to show there's the run of the mill certificate. That is, you're doing food handler. You're doing CPR first aid. You're doing that one-shot deal that's kind of independent of everything else. That's what we just call plain old certificate. That doesn't really rise to the level of postsecondary. 

Over here to the right, we have certification, that is, some occupations. You get that occupational certification. All the way over to the right, we have license. That's what we mean by occupational licensure. That's another subset of that workforce training that does rise to that level. 

And then in between, I don't know why they did it in this order. But that degree column refers to, again, we're sticking with postsecondary. But, obviously, you can see from the sort of things they say under degree. That's college stuff, not occupational stuff. 

I still look for that guru out there that can answer the question, why do some of these use government? Why do some of these use the business and trade association? It's super duper clear to me that all the ones that the business and trades themselves manage on their own, those are occupational certificates. 

The ones that have the mighty arm of government intervention, those are licensures. That's a distinction that I feel like I understand exceedingly well. I've asked a million people. I haven't had anybody yet that knows this answer. 

I don't think there is one as far as why do some occupations get managed by themselves. Why do other occupations get managed by the government? Nobody seems to be able to answer that question. 

So I'm just going to say, hey, we'll see it-- well, I guess good question, Thoibi. But I'll just say the last three minutes of crazy I don't know talk is exactly-- I don't know why some are certified, and others are licensed. 

There is something there where some require government intervention, and others don't. That I can answer. The ones that require government intervention are the ones they call occupational licensure. The ones that the business or industry manages on its own those are called occupational certificate. 

The why to which ones are certificates versus which ones are licenses, I admit I don't know what the rhyme or reason is to that. But the ones you're referring to, Thoibi, are on that same list. But you'll have to go through the list to figure out whether it's a license or certificate. 

I've spent quite a bit of time on it, as you can tell. And I don't think there's an answer. I think you just got to look it up and figure it out. I'll just say, hey, did I miss a question that might have-- no, that got silent. 

But that's not the answer you're looking for. But I really do think that's it. Right. Certificate or license, it's one or the other. But yeah, you're going to have to look it up. I haven't figured out anything that shows how you cover that other than that you just got to look it up and know which way it goes. You just got to know whether it requires the government yes or no. 

I haven't found any rule of thumb with that. You just got to go look it up. Right. Same thing with that second question. Sorry. But you just got to look it up. I have, and I haven't found anything that really gives you a nice easy guideline that lets you know that automatically. 

You just got to know which ones are involved with the government and which aren't. You just got to look it up online or check with the people that are providing that certificate or license and find that out. I haven't found anything that makes that easy where, hey, if you just know these little tidbits, you can easily decide which one to mark. 

Sorry. That's the best I can do there. Hopefully, there's something there. But yeah, this chart is as detailed as I have. It gives examples. There is nothing that I've seen that gives the master list of all the things that are licensed and all the things that are certified, gotta say. 

I've looked for that too. And I don't think there's anything like that either. Again, back to the-- yeah, you just got to look that one up. I don't have any better answer. 

OK, moving on to transition, this is another one that gets questions. I always like this nifty little diagram. I'm not sure everybody else does, but I do. So this is what I like to show. 

That is, we've got these transitions. I think this diagram shows the official ones for CAEP. So the blue boxes to the left are the starting points. The red boxes to the right are the endpoints. So the starting points to get an official CAEP transition are either K-12 adult ed, ABE, ASE, or ESL or noncredit community college ABE, ASE, ESL. 

To successfully transition to CTE, you're going to end in either community college CTE or K-12 adult ed CTE. You can see those two red boxes. If you're transitioning to college, your transitioning going to for credit community college. 

I will go out of my way to say only for credit, as you can see, there's a lot of these things in CAEP land where specificity isn't exactly a strong suit. You got to figure it out in a lot of areas. Here's where CAEP tends to contradict itself. 

One area where CAEP is super specific is for it to be a transition to college. It's got to be for credit. If it's a transition to noncredit, that's considered more like a, quote unquote, "lateral transition." 

So if it's to noncredit, then no. That's not a CAEP transition. It's got to be the for credit college, not noncredit college. So you can see by the diagram the different way in which you can record transition to CTE and the ways in which you can record transition to college. 

OK, what about noncredit CTE? I'm just going to say that's a good question for CCTAP or for the Chancellor's Office. As far as I know, that doesn't count. But that would be a question for them. I know that there is a data match that they do within COMIS. So probably somebody from West End-- or maybe it's the COMIS person-- can clarify exactly where they draw that line on the sand and what gets in and what gets out. 

But if you ask me, it's noncredit. It's got to be for credit to be transitioned. So as far as I know, it's the same. But I'm not quite sure what the difference is between credit and noncredit CTE. So there may be some differences there. 

"Enrolled in secondary." I guess, Carl, I need you to elaborate that one a little bit. I guess you need to talk that one out rather than chat. 

SPEAKER: Yeah, in some of the charts and bubble boy graphics that you showed us, enrolled in secondary triggers column L, a transition to postsecondary. 

JAY WRIGHT: All right, that's the newer model. 

SPEAKER: It's very confusing when we're working on outcomes. We're all kind of baffled why that's listed there. 

JAY WRIGHT: All right, and I'll just say the short what's not the answer. It's like the answers I gave Thoibi. But the answer is we have that now on Table 11, NRS Table 11. So it was a back door thing where, hey, we've got it for Table 11. 

There were some reasons for WIOA where we really felt like it needed to be aligned to Table 11. Well, hey, wait a minute. We're lining it to the NRS Table. So we ought to align it for CAEP. 

So I'm interpreting this as less about the outcomes themselves and more of a simple bubble boy alignment issue where, hey, maybe it was a dumb bubble boy alignment. If so, then point well taken. And it probably was a dumb bubble boy alignment that we need to change. But we probably won't do any changes until we are ready to officially bubble boy align. 

I know you've heard about that forever. And you probably don't think it's ever going to happen. Maybe I don't, either. But, hey, it'll be something that we would probably incorporate in that. Not unilaterally, I think that makes sense. 

OK, all right. Thank you, Holly. Sorry, I'm babbling a lot longer. Well, then I would just say yeah, ultimately, that's a question for you to ask COMIS people, not me, because that's definitely something that would be more like something they would do in their data match so they would be able to make that distinction better. 

I'm not really sure whether they make a big deal about in the college side or not. But I got to say a better question probably for them. 

OK, let's see here. So I'll move on from transitions. Good questions. OK, I'm going to mention I-3. I made a big deal of this last year because nobody knew what it was. I think people at least know what it is now. 

But I made a big deal with it because, hey, COVID, people really forgot about this. We had a lot of momentum before COVID. Got to say that this is one of those areas that COVID really did take a lot of wind out of its sails with this one. 

So we've been trying to bring it back up as an issue. That is, we've got those Immigrant Integration Indicators. At this point, as far as I know, this is what we have in CAEP land. 

So there is sometimes pushback because I think sometimes the advocates here look at it and say, yeah, COAAPs are great. But it's just a small part of the immigrant integration pie. It's not the whole dang pie. 

But for now, as far as I know in CAEP land, let's face it. It's the whole dang pie. So, hey, something is better than nothing. So I'll go back to say AB 28 requires us as a state to have something instead of nothing related to immigrant integration. 

There were supposed to be a bunch of things related to this, one of which was what we brought up at CASAS along. That is, hey, we've been doing immigrant integration for 20 years. Only we never called it in immigrant integration. We've always called it EL Civics. 

So, hey, we've got tons of stuff that's perfect for immigrant integration. We just have to agree with to that and align it to CAEP structure rather than some of this crazy, convoluted EL Civics structure we've been doing for years. So everybody agrees we have done exactly that. 

So now we can use not all, but most of those COAAPs. For immigrant integration reporting in CAEP, when students pass a COAAP, that counts as an I-3 indicator on the CAEP summary report that will get your student an official outcome for immigrant integration. I've taken some of this out. It's really just more of a reminder. 

But you can see the goofy little screenshot not that helpful. But I'll use that as a reminder. For my two cents, what you get from those I-3 reports, really, a lot better than what you get for the regular EL Civics reports, in my opinion only. 

But the difference is for I-3, you don't get a lot. But you at least get a little bit of information about what those COAAPs entail about what instructional-related areas. That suggests as areas of strength or areas of need that reports we have for EL Civics help a lot for accountability, but really don't do nothing in terms of instruction. 

I'll just say I-3, not perfect. But it does give you some of that COAAP content-related information. Thereby, in my opinion, I think they are a little bit more helpful as standalone reports. 

OK, I'm not going to read all of these. But I did leave this slide in there as just a time a little bit for Q&A. It sounds like you didn't need this slide. You've all done just fine asking whatever questions you want. 

Nobody needed any of this goofy prompting. There was a little bit of that by keeping it here. I'll just use this as just to say most of you are in a lot of those trainings last year, maybe in way too many of them, whatever, whatever. But we had a lot of these discussions. 

Some of them were just areas in which COVID exposed some warts. Others were areas where, hey, they didn't really need COVID to expose anything. There were warts exposed with or without COVID. 

There were some that really weren't warts at all. But just people had good questions, things that we needed to out in a little bit more better detail. So we talked a lot about this in most of the sessions. We spent at least half of the training talking about this slide, answering some of these questions. 

Just use it to say, admittedly, we haven't answered them all. We've had a lot of good discussion, but they're not answered yet. So I'm leaving it here to say, no, not mission accomplished. We're still sorting it out. I added this one just because some of you were involved in some of the work we did at the California consortium this past June. 

We did that activity, the previous California consortium. Not nearly as K-8 focused, but we started talking about some of these sort of issues a couple of years ago as well. We've had a lot of times now where we've discussed that. I feel like we've made progress, but we're not there yet. 

So I'll just say more to follow on a lot of the discussions we've already had with trying to winnow it down and trying to get it to where it's a little bit easier to decipher and a little bit easier to retrofit. So I'll just start there, talk about a mouthful and nothing on that one. I'm just going to stop. People aren't leaving. So that's good. I'll just start, though, seriously. 

Whenever I feel like I'm getting into extreme gobbledygook and not making any sense, for whatever reason, that's when the chat dries up. When I feel like I'm talking about perfectly straightforward, easy to decipher stuff, that's when all the questions come up. And then I get into this gobbledygook. And then, hey, wait a minute, so everybody understands that. 

All right, so, anyway, we're moving on to services. Maybe there'll be momentum, just saying. If there was a lot of momentum on services, is that wonderful? Or is that oh, no, that's ugh. Anybody willing to say which one that is? Is that a no, boy, oh, boy? Or is that muttering expletives under your breath? 

Oh, you love services. Yeah, so you love giving them, or you love receiving them. [LAUGHS] Sorry. I'm losing. "Not sure. It depends. They're very clear." Yes. "Not sure what am I getting myself into." Hey, look, at all this fine print about APR. Hey, I'm not signing that darn agreement. OK, everybody loves services, of course. 

So here's our screenshot. This is it. Hey, if you're loving services, hey, I'm thinking about a little bit of a guided tour of attachments on Friday. So I know some of you are of that ilk. So I'll just say I might even look at that attachment as an attachment X because I realize now that there's a gazillion attachments, and nobody knows nothing about them. 

So that might be how we kill an hour and a half Friday is biting the bullet and pointing out what some of those attachments actually do. I think by Friday, I'll get cold feet and change my mind. But for now, that is actually what we're doing. So if you want to unenroll and make sure you're conveniently doing something else Friday morning, there's your official warning on Wednesday to make sure you've got plenty of time to reschedule yourself out of that. 

Anyway, I'll point out Attachment S for short-term services is one of those myriad of attachments that in my mind of minds really gives you definitions. They're not perfect, but it gives you something to go by. So when you're looking at these checkboxes that mean nothing, download Attachment S, and that will give you some context on what all this craziness means. 

So supportive services are the services. Here's the screenshot. There's other things, of course, you could be doing. But these are the ones we have in TE. Basically, the ones we have in TE are the ones that are aligned to WIOA. 

I'll admit that WIOA alignment is a little sketchy because services have never been part of Title II. They've always been part of Title I. So there's a lot of retrofitting with these, not a little bit of retrofitting because there is official WIOA stuff. 

I think that's like Oreo. That's like one F, not two, when we're talking about WIOA I stuf, S-T-U-F, not two Fs here, like double stuff, double stuf with only one F. Sorry, I'm getting into alphabet soup. 

But, anyway, these are the ones for WIOA I that we have here in TE. So the alignment, admittedly, is super duper loosey-goosey. But these are the ones we have. So we use them for CAEP. 

I think overall, they cover all the areas. But we've done a lot of retrofitting for these, even more so than outcomes. You might say, where can you find a little bit more better information? Look at Attachment S for short-term services. 

But back to the slide, supportive services are the ones focused on the student's personal needs-- legal counseling, fiscal counseling, child care, transportation, and so on. Training services is more related to the student's instruction and what the student is doing with you at your college or at your school that is maybe short-term little trainings. You're doing tutoring, maybe mentorship, et cetera. 

Transition services, in my opinion, again, talk about acting, talking, walking like a duck. To me, when you use that goofy way of looking at it, transition services sure look an awful lot. Like training services, to me, the list really does seem the same for my two sets. But the difference when you get into the WIOA stuff is the transition services are very specific to either transition to employment or transition to college. 

It really seems like it's a lot of the same training stuff like we have in training services. But it has that very narrow lens for whatever reason. So, again, supportive, training, and transition services, for now, that seems to be working pretty well. That's how we're working it. 

I'll add it was just a five-minute copy and paste, so not exactly something to hang your hat on. But I will say some of these slides that we talked about last year that I dare say a lot of you really, really liked and asked a lot of good questions about, we've kept them here. We've folded them into Attachment S. But these are just some ways to look at these services a little bit more finer for you because, again, the definitions say a limited amount. 

So these are some ways to try to get you in the car, so to speak, with a little bit more context, some suggestions of how you might relate some of these WIOA one services to fit a little bit better with what you're doing at the adult school and college and so on. So here's some key supportive services with some extra guidance. Here's the same thing for training. Here's the same thing for transition services. 

You mean like things, like, what sort of document you might provide. I think that's probably a better CDE question there. I mean, I would just say I'll treat it as if you're doing like third-party import where, hey, as long as you've got it back up, I'll just say three years backing that up like everything else. 

That's what I say is a generic answer to those that are not doing the entry update scan forms anymore, which, of course, is like 90%, 95% of you. So I'll just give you that same answer I would for other sort of backup info, just to make sure you can access it for at least three years electronically, just like you've been doing for however many years that you haven't done the entry update scan forms. 

That one's a loosey-goosey answer. But that one, I think, is a little bit more based in reality that we've been looking at that limited backup for at least 10 years, probably more like 15 since we've split from people not doing paper scan forms and everybody doing everything in the cloud electronically. Hopefully, that makes sense. 

OK, so I'll just say, hey, does that make sense? I'll just say major transition here too. We're OK for time. It seems like I'm babbling and heel dragging. I'll just say I wanted a lot of time for Q&A. 

But, again, you're not being shy. To be clear, by not being shy, you're doing exactly what I would hope you'd do, which is ask lots and lots of pointed questions. I was really genuinely hoping that would happen today. If it didn't, it wouldn't be much fun for any of us. But what do you know? You've been asking very pointed questions. 

So in all sincerity, that's made me happy. That said, I make a big deal about it, and nobody has any questions. So here we go, super basic stuff here with entry update record. 

So, again, with entry records-- actually let me stop because there's a few of you that haven't been doing this for a million years. So does everybody at least know what the entry update records are? I know a few of you do, good grief. But some of you, I know, are not names I've seen all over. 

So if you're really new to this, I could see some of you not really knowing what I mean by these goofy shots. So yeah, I know you know what it is. I was looking for some of these newbies that haven't heard this for a million-- OK, so true to form, all the people that I already knew are answering the ones that I didn't-- or hiding and not answering. 

But, anyway, we're using our old scan forms. That is, hey, if we were in 2006, we'd all know what these are because we'd all be using them. That is bubble forms that you'd scan in all the data. 

Now, a few are using them. Whenever I ask, I always get more people saying yes than I realize. So I know a lot more of you are doing this than I realize. Some of you are just doing manual entry. Some are doing third-party import. Some have something more highfalutin than that, maybe. 

But whatever, you're getting that information in an enrollment. You're getting information in an update. You're getting information in a follow-up. So the entry record is what you're used to record enrollment in a CAEP program. 

With the entry record or enrollment, we've always been really rigid. That is, there's only one time to do enrollment. That's as soon as the student sets foot in time inside your doorway. That's day one, square one. 

As soon as you see the whites of your student's eyes, whatever cliche you want, we've always been rigid about this. We've always given you zero flexibility. You need to record enrollment as soon as you first start serving that student. 

OK, I'm not sure what you mean by your question at all, Cynthia. I guess I'll let you-- I don't even know what you mean by that question. I'm sorry. 

AUDIENCE: Cynthia, would you be willing to come off mute, and you can ask it directly? 

JAY WRIGHT: Right. I mean if you don't, I guess you can try to-- yeah, and that makes me more confused, not less. I guess I'll let you reformulate it. But yeah, I'm sorry. I don't know what you mean by-- I don't know what you're talking about with that. 

AUDIENCE: They're talking about the intake demographic collection. 

AUDIENCE: I think I'm here now. 

JAY WRIGHT: So what's the question? 

AUDIENCE: The question is these questions that are up on the screen are the ones that show up before they take the CASAS test. And so whatever-- 

JAY WRIGHT: What questions are on-- what do you mean questions on the screen? 

AUDIENCE: Labor Force Status. 

JAY WRIGHT: Oh, you mean just the screenshots? 

AUDIENCE: Yes. 

JAY WRIGHT: OK. 

AUDIENCE: Yes. 

JAY WRIGHT: OK, so what about them? 

AUDIENCE: So the question is these are self-reported by the student who sometimes click through the screens without answering correctly. That's my question. Are we supposed to-- 

JAY WRIGHT: So the question there-- 

AUDIENCE: --go back and clear up-- 

JAY WRIGHT: So the question is what you didn't ask, which is, what the heck are you supposed to do about it? 

AUDIENCE: Yes. 

JAY WRIGHT: OK. And that is yes, you should be following up on that. I mean, to give you the answer that's definitely 30 or 40 minutes we don't have, because yes, some things are more important than others. Again, 30 or 40 minutes, we definitely do not have at the moment. 

But yes, most of these are required. There's reports you can use, such as the data integrity report, lots of other little tools, things like the monitor report, the summary audit report. There's lots of reports to follow up to find students with missing information, lots of things related to the things like CAEP, payment points, NRS where you can get it based on what's required for state and federal reporting, et cetera, et cetera. 

But the short answer is yeah, if your students provide incomplete data, the bitter pill to swallow is an emphatic yes. You absolutely should be following up. If you don't, yes, your bottom line will suffer. Yes, it will be all your fault because you didn't follow up on things that you should have followed up on. 

I think that's the answer you're really looking for. All the draconian answers are true. All the soft answers are false. Yes, that is your problem. Yes, you should be following up on those issues. How's that for an answer? Because that is the honest correct answer. All the things you-- 

AUDIENCE: That's what I needed to hear. 

JAY WRIGHT: OK. So there you go. So, anyway, with the entry record, we're recording enrollment in a WIOA II or a CAEP-funded program. Please do collect demographics, gender, race and ethnicity, date of birth, all required. 

Here's where it gets looser is that little bullet. Yeah, if you miss that, you're missing NRS. You're missing payment points, 125% your fault for not following up on easy stuff. 

These down below are a little more nuanced. They're all required, but they're not as direct hits as things like date of birth or gender. But yes, please do collect labor force status. Please do collect education level. Please do collect barriers to employment. 

Those probably won't directly impact your bottom line like the demographics will. But if you don't collect them, yes, your bottom line will be affected. Just like the other question, yes, it is 125% your fault for not following up on things you should be following up on, et cetera, et cetera, et cetera. So please do collect that information and intake. 

I guess probably what I'm hearing is I'm sure you're doing a great job, Cynthia, at following up on stuff. I'm giving you a hard time you don't deserve. But it sure sounds like you probably want to bite the bullet and stop doing that goofy practice 

at e-tests. What you're describing sounds like lots and lots and lots and lots and lots and lots and lots of other agencies where you rely on your students to do that on e-tests, and, hey, what do you know? You end up with incomplete information. What do you know? You end up spending more time fixing all the mistakes than you would if you just went ahead and did something like the entry record or some enrollment form or whatever in the first place. 

Sorry, there's my unsolicited advice. Sorry, you got way more than you bargained for, I realize. 

Anyway, please do select the program for WIOA II. That means ABE, ASE, ESL for CAEP. That means the seven programs we talked about at the beginning of the session. We talked about barriers. 

And then this bottom one is one we've made a bigger deal about lately. If you have anybody that's coenrolled with one of your WIOA partners, please mark it. We've made a big deal of that in CAEP and WIOA training lately because there's a lot of interest at the state level. 

There's also a lot of interest at the federal level in that. And our data doesn't show that much. We know we've got a lot more than we have to show. 

Yes, if you don't put their program in there, that's a deeper problem because yeah, if you don't mark ABE, you don't mark ESL, then yeah, they might get weeded out of the report entirely because yeah, the report is conditioned to look at just those that are involved in CAEP for the CAEP DILR or just those that are involved in WIOA II for the NRS DILR. So yes, that can potentially present problems. 

OK, another entry record issue-- this was a big deal last year, not such a big deal now. But last year, we went back to requiring primary and secondary goals. For 10 years, it was on the form, like you say. Just about all of you were doing it anyway. 

Even though it wasn't required last year, we, once again, made it required. And, hey, there's a nifty little attachment on that if you need some help figuring out some good ways to record student primary and secondary goals. 

OK, so moving away from the enrollment or entry side and moving-- there's no hard and fast-- I would say no more than two makes sense to me. But there's nothing in TE or on the federal level that strikes you down as being wrong. So you can mark as many as apply. 

The no more than two, I think, is what people have said for 20 or more years. But yeah, if you have more than that, it gets confusing and silly. But I'll just say I don't think there's a state or federal policy that puts a CAEP on it if you just go crazy at your agency, and you like giving your students more goals. 

So we're moving away from the entry side and into the update side. So there's a goofy line. I've liked the line for whatever reason. I'm a party of one. But this bullet above the line is what we've talked about for 25 years-related updates. The stuff below the line, not really new anymore, but new as of the last five years or so. 

Above the line relates to what we're talking about with enrollment. Sorry, I'm clicking too many things at once. I wanted to show this one, where what I made a really big deal about on this slide is for every year since '99, we've said we're rigid about entries. Do entries as soon as you see the student. Day one, square one of that enrollment, it's always been very rigid and inflexible. 

For updates, we've been a lot looser. We've always said, hey, it's up to you when you do the update, hey, a substantial block of instruction. Maybe that's semester, trimester, monthly, quarterly, your decision, not our decision on how often you do it. Just align it to what makes sense for you. That's been our long-time instruction. 

When WIOA kicked in, we have now this special clause for attendance. I won't get into periods of participation here. Hey, you can go to WIOA training for that one. But for WIOA federal reporting, we use attendance to determine who exits. That's a big part of the process that still fits in for CAEP because we're doing that employment and earnings survey. 

Because we still need to govern PoPs we're saying do attendance at least once a month. Because of PoPs, we have that 90-day rule. I know at least a few of you know it. That is, for all learners, TE should know for everybody, whether more than 90 days or less than 90 days has transpired. 

For a lot of you, that's difficult. So we say once a month, if we just keep it loosey-goosey, we're setting you up for lots of failure. So we make you do it once a month. That's not perfect. But you might say, in this case, literally, good enough for government work, I suppose. 

OK, here is just a blown-up version. I always question, why do I even put this slide in there? I think this is just showing you proof pudding that attendance hours, technically, are an update record issue. 

For some of you, that's straightforward. But some of you scratch your head. Why are you talking about attendance when we're talking about updates? That's just showing you that attendance is on the update. That's why I consider it that way. 

Anyway, long story short on the update record, all of that issue related to 90 days, doing attendance at least once a month, getting into TE at least once a month, etc, is the last couple years' updates have been a big-ticket item, much to my chagrin. It feels like a lot of this is like so 2009, '10. 

But people keep coming out of the woodwork saying they're having their teachers fill out the whole update. If so, probably a thousand things that are a better use of your time and the teachers' time. We don't need that update status or update progress. You're just wasting everybody's time if that's what your teachers are doing. Leave all that blank. 

However, that's not to say having them do the update in general is a waste of time. Having them do the update in general is a really, really good use of anybody's time. But it's really just focused on marking these outcomes. We're not really worried about all that other stuff with status and progress and all that, like what everybody was doing 10 or 15 years ago. End of sermon there, I guess. 

So I'm moving on. I better pick it up. I really thought I was going to have trouble. But, hey, great. You've really saved me. Thank you. But that said, I better pick up the pace a little bit. 

So here are some resources. We've got the '23-'24 CAEP Data Dictionary posted. It was better late than never. About the 1st of August, about a month late, but it is there. As an aside, WIOA II dictionary right there in the same spot. 

There's a summary of changes that a lot of you like. If you're just wondering what's different, you can look at that. And it points out what's different in '23-'24 versus what's different from '22-'23. 

For my two cents and my two cents only, the big one this year that probably has the changes is not as much the data dictionary, but the California Assessment policy, which is Attachment A. I like to bring that up because it has those new CASAS test series, of course, STEPS for ESL and Math GOALS 2. 

In California, we went way out of our way to do a second federal fire drill this year. Hey, must be pretty gosh darn important if you're inviting a federal fire drill that you're not required to do. But that is what we did. 

We submitted an updated assessment policy a second time there in March, April so we could get a second assessment policy approved by the feds, so we could thereby include listening STEPS, reading STEPS, and Math GOALS 2. That's the one we have posted as Attachment A. And then, hey, I'll just point out we have Attachments A through X there too. 

OK, hey, I guess I covered this slide, and the previous one did. And what do you know? I bird walked an inordinate amount on the previous slide. So I don't need to talk about this one as much. 

Again, we updated it to include the new series. It has all of the guidance related to how you need to use CASAS tests for state and federal reporting. I'll point out it also has that Attachment B Local Assessment Policy. 

We talked about that a little bit Tuesday, probably a little bit Friday too. That is, hey, 'tis the season. Now is the time to update that local assessment policy. 

And, hey, this year, most agencies, not everybody, but your average agency this year, I think, will have more local assessment policy updating to do than usual because your average agency is at least trying to move forward a little bit with Math GOALS 2 and STEPS. If so, there's 125% chance you need to update your agency's local assessment policy. 

OK, and then here, hey, I'm just being clairvoyant and doing everything a slight in advance. Gee, everybody wants at least four or five more hours of that probably. But, hey, here's what I just got through talking about related to local assessment policy. 

A couple other resources-- you do need to do the Quarterly Data Submission. This, I think, is a little bit out of order, because I should have put this one here, sorry, out of order here. But here's our data submission calendar. I think most of us know this. But we're submitting quarterly for CAEP just like we're doing for WIOA II. 

Here's our timeline with our four quarterly due date. Sorry, I should have shown that one before this one. So for the last three years, I think this has been pretty easy for agencies. This has been in good shape for most of us. 

That is, we've got that TE Quarterly Data Submission. That is, you can do your CAEP data, CAEP DIR, WIOA data, and WIOA DIR all in one fell swoop. Run it every quarter. If you've got issues with TE, here's a link to that help document so you can follow the steps with the wizard. 

Similarly, we do the employment and earnings survey for CAEP, just like we do for WIOA II. That's also not new. Last year, I think it was a rude wake-up call post-COVID this year. I feel like most people that needed the wake-up call have gotten it. 

So I think for the most part, we're in good shape for this now. But we still have this help document with the several different help documents available to help you with that employment and earnings survey wizard. Bottom line is you do that wizard in TE following the exact same timeline as for the data submission wizard. 

But in this case, hey, it might be different, depending on what quarter it is. So we have multiple help documents to give you the step by step in TE. Your average agency probably needs help with this wizard more often than it needs help with the data submission. 

OK, and then, hey, just you want to see it one more time, here's the calendar. To keep it super basic, it's by fiscal year, July 1 to June 30. Divide the fiscal year into quarters. One month from the last day of the quarter is the due date. 

OK, CAEP reports-- obviously, limited time, but that was a little bit on purpose. This is the CAEP Summary. This is what I like to call the hub of the CAEP universe. This has all of the basic reporting issues we have for CAEP. 

I like to call this one three reports in one. We've got simply left-hand side, middle side, and right-hand side. I like to say it's three separate bars at three different levels-- high bar to the left, middle bar in the middle, low bar to the right. 

Starting to the right with the services, we set the bar very low. You can see that's where we're recording Services Received. We set that bar very low for services on purpose. That is for that. 

And this is back to your question. I think it was Cynthia, but there might have been others. If I'm picking on you, Cynthia, I'm sorry. But a couple of you are asking. 

For services, the requirements are low. In fact, they're basically nonexistent program, not required, demographics, not required, pre and posttesting, not required, instructional hours, not required. For services, you just got to mark the services. We're going way out of our way to not require any of those extra things. That's why I say low bar. 

So good news is you don't need to get through any of those extra hoops. Bad news is a lot of you get clutter in this section. Because it's so loosey-goosey, it literally filters out nothing. For now, we're leaving it that way. A lot of you have good reasons for wanting no requirements here. But as long as it is, you're going to get a lot of clutter deposited in that section. But we don't require anything for that. 

Moving in the middle, CAEP outcomes-- for that we, require those demographics. I guess I should step out and say, we've got literacy gains to the left, CAEP outcomes in the middle. Quite frankly, and a lot of you know this, pre/postliteracy gains are an example of a CAEP outcome. 

But we put literacy gains separate. Number one, just for programming, we bring it from NRS Table 4 because we're doing all that work under the hood anyway. But bottom line is we've got that special section for pre and posttesting because it includes pre and posttesting. 

The middle CAEP outcome section includes all the other CAEP outcomes except for, of course, pre and posttesting. So get ready for another who's buried in Grant's tomb explanation. On the left-hand side, it's all about pre and posttesting results. 

So, naturally, we require pre and posttesting. The middle section shows all the outcomes except for pre and posttesting. So, naturally, we don't require pre and posttesting. 

Everything else about the left-hand section and the middle section are the same. That is, we require what I like to call the usual suspects. Program is required. 12 or more hours of instruction is required. All of those demographics are required. 

That is, you need to get through all those hoops to get a CAEP outcome. Obviously, you also need it if you want to get a pre/postgain. 

We have our CAEP Data Integrity. Again, it was a flippant smart-alecky answer. But this really is the answer to your question on, hey, what do I do about all these fields that are missing? Hey, the students are not being thorough enough, or the teachers are not being thorough enough, or I'm not being thorough enough or whatever, whatever, whatever. 

This is the long-winded answer to, hey, you could look at that CAEP DIR or the NRS DIR or the payment points DIR. It gives you a lot of these data elements with an item count and percentage of what might be missing. This is something you can use to follow up on all of that missing data you mentioned. We have these hours reports. I really thought was going to be a big stick and deal a couple years ago. Got to say it hasn't really turned in to be that big of a stick and deal. But I'll still pretend like it is. That is, we're looking at moving people out of 0 hours and into 1 to 11. We're looking to moving people out of 1 to 11 and making them a participant with 12 or more hours. 

This is getting in the weeds a little bit, looking at those that don't have 12 hours and trying to figure out how students are progressing. We know that it's very important to maximize the number of students that have 12 or more hours. 

This is just blowing it up a little bit so you can see how the reports are working and how you can look and see what your item counts are and how you can drill down on those right two columns to try to move people out of those right two columns and into the left-hand column. 

We are this is just a different variation. This is a little weedy here. But this is looking at it in terms of services, where if you are looking at that area where you've got students with less than 12 hours, students, missing demographics, and so on, you can dig into that services. 

OK, that you go and run CAEP tables. There's a TE menu item called CAEP tables. I think this is what you're asking. And yeah, I know this can be confusing. I think this is what you're asking. I think you're getting in the TE weeds with the question. If not, I guess you'll just correct me. 

But, here, I'm typing it in the chat. I am doing something. It just doesn't look like it. But you go to Report, State Reports, CAEP-- sorry, it's CAEP tables, not CAEP reports. I'm thinking of too many things at once. It's CAEP tables, not CAEP reports. Sorry about that. 

But CAEP Tables is that option in the TE menu that has basically everything CAEP. There is a selection you need to check called Report Selection that has literally 20 checkboxes where you can select and deselect the specific CAEP reports you want to generate versus those you want to refrain from generating. OK, thank you. That is what you are asking. 

Anyway, Services Enrollees by Hours, this just allows you to look at all that clutter and figure out, well, how much of this are really students receiving services versus other stuff that might not be at that at all? What's this clutter I need to fix? 

So here's what I skipped. You can read this. But, again, we're looking at hours by program. If you've got classes covering more than one program, we're saying, yeah, and the hours are not 50/50. Create multiple class records. Just know the TE behavior is always to split 50/50 by program. If you know that's not true, then you need to create extra class records. 

Another big issue, not as much anymore, is when we're recording hours, we're recording hours for classroom instruction. Services are great, but they're not part of instructional hours. Here's that I-3 again. This is just showing you an example. 

Again, I made a big sermon for these reports. I kind of think the I-3 report-- well, not kind of. I do think the I-3 reports are better than EL Civics. 

We're preaching to the choir for the most part here. So I know a lot of you are consortium managers. If so, hopefully, you've got consortium-level access that allows you to get higher-level information. 

You can look at all the agencies in your consortium. But you cannot drill down. But a lot of you like being able to compare your different agencies and see how the results are working. 

This is just showing you what you get where, hey, if you've got a lot of agencies giving you a convoluted answer, example, on purpose, where here, you can see it's listing by column each agency in your consortium. And you can look agency by agency and, in this example, compare how well they're doing with highest diploma earned. 

Here is the key. Again, we've got a high percentage of choir members here. So it is a worthwhile slide. When you're running those consortium-level reports, I always make a big deal. It's a really teeny-tiny checkbox. 

We probably need to just make it bigger, because for consortium-level reporting, in my mind, this checkbox means everything. I can never remember what it is by default. But bottom line is whenever this box is checked, you're going to get one piece of paper aggregating all the agencies in your consortium once-- OK, thank you, Carla. 

You're going to get one piece of paper with one set of results with all the agencies in your consortium. If it's unchecked, then you're going to get page by page, agency by agency. In my mind, that's the ultimate you say tomato, I say tomato issue. There's my little favorite cliche for you just so you can get your little drop of knowledge. 

So you say tomato. I say tomato. It really is up to you whether you want one piece of paper with an aggregated or multiple pieces of paper with each agency separate. That literally is just what wild hair you happen to have at the time, so make sure you check it. 

What's coming up? Hey, we've got plenty of choir members. So I'm going to be doing a presentation on September 15 at that Director's-- hey, paid program announcement. I think I'm doing something on September 22 as well. 

Way more than me, they are lots of better stuff than me. But, hey, that's what I know. So that's what I'm putting on the slide. Nicole is going to do a CAEP DIR on the 27th. Same exact format and frame is this one. 

We'll be doing everything you want to know and more at the CAEP Summit. We've been kind of lazy. But at the CAEP Summit, I do hope by then, we'll have some discussions. And we can include some of the information from some of the discussions, start talking about. And some of you had questions, sort of where do we go from here stuff. 

I went out of my way to exclude all of that here today. But, hopefully, in the summit, we'll have a little better chances to talk about that. And then, hopefully, if that's true, then we can have a small number of face-to-face trainings. 

Here in late October, November, I guess it's going to just come around full annual circle. Right around the time we had those CAEP trainings last year is when I'm hoping we can do a couple a year. That is November. And that's it. I think I ran over maybe obnoxiously so. Oh, yeah. 

HOLLY CLARK: No, never obnoxious. 

JAY WRIGHT: So obnoxiously so that Holly had to run, and Mandilee had to cover, huh? 

HOLLY CLARK: No, I'm still here. 

JAY WRIGHT: Well, no, people did evacuate. 

MANDILEE GONZALES: No, Holly is still here. 

JAY WRIGHT: We're down to 30. A lot of people did. Anyway, thanks for hanging in there. Most of you actually bore with it. So, hey, great to have you. Thank you much. I'm sure I'll see you sooner rather than later. 

HOLLY CLARK: Jay, thank you so much for coming today and presenting. Just the fact that everybody stayed-- you went over a few minutes, and nobody left. So that's-- 

JAY WRIGHT: It was pretty obnoxious, too, wasn't it? It was awful. 

HOLLY CLARK: No, it wasn't. Jay showed you the dates for the upcoming events. We did link those in the chat for you for the summit and for the upcoming trainings that are going on. 

We also dropped the evaluation link a couple times. Please take a moment to give us your feedback. It does form our professional development moving forward. 

So for everyone who gave up your time to be with us, we appreciate you. Jay, of course, we couldn't do it without you. So we are extremely grateful. And we are excited for the next session with you. 

JAY WRIGHT: All righty, thank you very much. 

HOLLY CLARK: All right. Thanks, everyone. Have a great rest of your day. Bye-bye. 

MANDILEE GONZALES: Bye, everyone.