JAY WRIGHT: Kind of looks more or less like what people are expecting to get themselves into. I'll just check, is this what everybody indeed was expecting to get themselves into? Hopefully it was but this is going over data and accountability. So a couple quick qualifiers here, just looking at the audience. 

This is the CAEP Summit obviously. So this accountability presentation focuses on CAEP. That is California Adult Ed Accountability not WIOA II federal accountability like we do in our quote unquote, "regular" classes virtual presentations that I know a lot of you have attended over the years. We're focusing on the CAEP side of the house. 

Number two, we have started doing some regional trainings on this. Sounds like a likely story. But a lot of you have been coming to me in recent months. Since mid-august or so, I've had several come to me talk about issues related to CAEP data collection, CAEP outcomes, CAEP services. 

Got to say that quote unquote, "popular" demand really is what kind of move to do in another round of regional trainings. I'll give you a couple less schedule dates here. Short answer is if you're in the valley, apologies ahead of time. I've gone to Fresno and Bakersfield. I went there a couple of weeks ago, the 12th and 13th. Got some stuff in Northern California early November. 

Southern California mid-November, we'll probably do another round here in February at similar but slightly different locations. So some of that is just saying, yeah, we're trying to get the band back together and talk about this stuff again. 

The other part I'll say though is the regional training is 2 and 1/2 hours. This is kind of a synopsis of that trying to cover highlights. So I will point out the number of slides you have here is not just greater but overwhelmingly greater than we'd ever conceivably have time to talk about. 

So I am going to cherry pick a little bit. Some of this stuff is purposefully just some things for your reading enjoyment. Some of it, you might have to skip because we're in an hour. So programs update. I want to start by defining the structure. That's been one getting reacquainted question. We'll talk about some of the key training issues we were talking about back in 2019. 

We're not going to belabor that but a lot of things that were a really big deal there three years ago kind of fallen off the map for a few years but come back into vogue. So we'll talk a little bit about that, how it's happened. 

We'll redefine those outcomes and services. There was a workshop I did yesterday where Neil was in the audience. Traded some banter about hey, we're really looking at remodeling and updating this to some extent. That's been another reason to bring this up is to find out what people have been doing with all this the last couple of years. So for now we'll redefine what those outcomes and services are. 

Probably won't have enough time to iron out all the questions that came up a lot a few years ago, but will at least lay it on the table, point out some things that were confusion areas and probably still are. We will make sure that we cover these basic issues. That's where the focus has been in regional. We'll be sure to do this here. 

We'll talk about the basic deliverables timeline and resources. That's the low hanging easy stuff but that's also been a big theme the last month or two that a lot of you are new to this, a lot of you didn't do CAEP before, and now suddenly you do. A lot of people have forgotten about some of the CAEP stuff, so we'll make sure we go through the basics with that. And then, time permitting, we'll talk about some of those reports in TE. 

What we definitely won't cover but I'm including for your reading enjoyment are goal setting examples. Some of you probably attended workshops that we did back in the spring in March, April. That goal setting issue was a very big issue of concern. 

I did a few workshops with CAEP TAP. I did a couple regionally where we did a deep dive and talked about goal setting. Not enough time to set that up today, but we will definitely keep those slides in. Don't feel like it does any harm to allow everybody to be able to have that information. 

So we'll start with the CAEP program structure, just defining what CAEP is number one. And then also we're looking at a question that's really come up a lot here lately, which is why does TE look different from NOVA the basic definitions of what's included in CAEP a little bit different in the two areas. 

So we'll start with the official CAEP program structure which is, of course, the version that's in NOVA. That is the program structure is really set up to emphasize those three basic programs that you all know and love there at the top, ABE, ESL and career tech. I'll point out it's five because adults with disabilities, parents serving K-12 success. Couldn't really figure out a way to slot either of those in the other three big areas, so those are kind of flapping around as programs number four and five. 

When you look at NOVA, everything attempts to roll up into one of those five programs and in particular, the big three that you see at the top. So here is spreading it out the way it kind of rolls into these three programs. This is getting at the question you might say of why are they different. So in CAEP, it really tries to roll it up into these three programs. 

A lot of times the other 99% of the world would probably look at it differently but NOVA rolls it up. So for starters, we'll look at this ABE block here, where ABE includes the ABE-- here we go. I'm wondering maybe Marjorie might have disabled that just because they're going to post. I'm not sure. I'm not allowed to-- I don't have a way to send it from chat normal. Maybe Marjorie if you can chat I'm not sure if I can send it or not. I'm happy to do-- 

MARJORIE: You should be able to, but hung on. 

JAY WRIGHT: OK. I'm happy to share. OK, here we go. It's giving me an extra step it doesn't give me maybe. I think that's what it is actually. So yeah, hang on. Sorry I'm interrupting, but I do think yeah, your question is a good one, Jamie. And if it's no problem then yeah, let me just quickly do that. 

Sorry I'm fumbling around here finding the right folder. But I really do think that's a good point. I think it would help so I'm going to-- So here it comes. OK, good. So here it is. It looks like everybody has it. There it is. Sorry about all the hemming involved. 

All right, great. So in any case ABE is a good example. I think most people would say this includes at least two different programs. ABE number one, ASE number two. Probably most people would say it's actually three programs and that's kind of the way TE would classify it. ABE being program number one, high school diploma number two, high school equivalency that's GED and HiSET number three. 

But in CAEP land being a little bit contrarian, ABE is rolled up into one uniform program. It's not two different programs or three different programs like most of the real world acknowledges. Middle one ESL not much to say. I think CAEP land handles that the same as everybody else. Career tech ed, a little less familiar to most of you, but that one divides it into CTE and short term CTE. 

And then at least in TE, those bottom two bullets are definitely career tech ed only oriented, but TE considers pre apprenticeship and workforce preparation to be separate programs. And making a big deal about this back to the original question, why does TE and NOVA report programs a little bit differently? 

Just to spread it out, here is the different NOVA programs under CTE. Again, it's CTE and short term CTE. The question that always comes up is what's really the difference between the two? Quite frankly, like a lot of things, there's no formal distinction. The number of hours I give is 48 because that's the one number I found, but I don't think that's an official answer. That's just the only number I found but anyway CTE and short term CTE. 

There's another program called pre apprenticeship. It's long term and occupation specific just like CTE, but it's obviously a different delivery model. Workforce prep is also folded into CTE. It is workforce related but it's short term as compared to CTE and it's also not occupation specific. 

So things like Microsoft Office skills or resume writing skills or workplace safety or those kind of things would be examples of things that would be workforce preparation that is workforce related but not related to actual occupational skills like a CTE plan. 

Here is the CTE model just so you can kind of see the difference. Again, we've got those big three up at the top out of CTE. When we say seven programs from a TE point of view, again, pre apprenticeship and workforce prep, we consider to be separate programs. At least in so far as when you work instructional program inside TE software, it's definitely a separate instructional program you have to mark from career tech ed. 

I'll just stop right here. That was a little bit abrupt. I suppose everybody hanging in there even though I kind of hit you upside the head with a frying pan to start Did that smart? Did that keep everybody in? Did that make everybody tune out? Not really sure. OK. Some of the faithful are saying yes anyway. 

OK. Thank you. All right, so anyway sticking with structure again, really going fast, haphazard whatever but these are what to say the last few years in current events headline news fashion have been some of the big topics on the table. We've been talking about some of these are in the last year. 

Some of these quite frankly, are things that we talked about when we were doing the road show back in 2018 2019. But again, what people keep telling me is gee, with WIOA II, payment points, pre and post testing, EL Civics. Everybody's hung in there for the most part, when it gets to some of these CAEP things like bubble boy slide, short term services, occupational skills gain, all that crazy stuff we were talking about in CAEP land, that really seems to be the stuff that fell by the wayside for a lot of folks. 

So again, we've really been looking at rebuilding and repurposing this whole darn shooting match. So these are in no particular order. Some big issues. Goal set it starting with goal setting just to get everybody's foot on the rail as everybody here aware of the fact that in CAEP land for the last year or so there's been lots of discussion about goal setting and everybody here technically should have some formally submitted CAEP goals for which both your agency and your CAEP consortium is working. 

Is everybody familiar with that? Yes or no. All right, everybody's saying so. Great. So we've got goal setting that's been a big issue with me. We'll talk about that a little more. But not a lot a bit, but that's a big issue. 

Adult served. This was laid out when we fell off the grid in 1920. We started this as a big part of CAEP accountability in 2019 to get everybody ready. That is those three categories or I call it the three buckets. That is the three buckets related to hours of instruction with the zero hours being one bucket. One to 11 hours being another bucket. And the 12 or more hours, that is the participants being the third bucket when I bring it up in that rudimentary way. 

Sorry you're supposed to be quizzing me not the other way around but is everybody familiar with that where we've got the different buckets, three different buckets related to hours of instruction? Got to say we made a really big stink and deal about that in 2019. Once the goals were laid out a year ago, I remember people like Neil bringing it up a year ago. 

That's been cemented by things like TE reports and a lot of the goal setting talk here lately. With that, we had a lot of discussion of hours by program. That continues to be a big issue. It would seem that that will be a bigger and bigger issue moving forward, but we have that CAEP hours by program report. 

We looked a lot at things like hey, measuring how many hours each instructional program aggregate of all students had? That is using the quote unquote, "ABA model" to look at hours and so on. We've had a lot of those fire drills lately. The big issue that came up when we had training is what if you've got learners in multiple programs? Short answer is TE splits it 50/50. 

Another big issue with hours of service hours. That's what I call the big CAEP question of 2018/19. Half the state felt like services should count as hours of instruction. Half the state did it. We did a lot of research they are back in summer of 2019. I remember final answer, it was really clear. Service hours do not get included with instructional hours. 

We look through the federal regs, felt like we had a pretty conclusive answer. That's what we started talking about a few years ago. That's what we continue to say. I'll get less into this but we had some wonky issues where the Feds restructured the way Title I deals with its outcomes. We pass that level on with our CTE literacy gains more on that later. If I talk about that now, I think it'll be confusing. 

We've also talked a lot about more collaboration with workforce partners. Definitely a lot of initiatives coming down the pike where that will get more and more and more important. I bring this up because I feel like we've talked about this collaboration issue ad nauseam on the WIOA side, but the CAEP side, not so much. 

Got to say some of these initiatives probably way more likely to involve your CAEP programs than your WIOA programs. So thinking about this for both WIOA II and CAEP I think will be an important concern moving forward. 

So again, in the interest of time, I'm not going to go crazy on all these individual issues. But here are the goals that were laid out about a year ago. That is mandatory consortium level metrics, mandatory agency level metrics. And then also at the agency level, you had an optional list. The optional list was intended to cover all the different reporting areas. 

You can target employment or wages or pre post testing or immigrant integration or transition or several other high school diploma. All sorts of different outcomes that you could target as possible metrics to show progress. 

Again, I'm going rapid fire. Here is that issue with adult served. This is what we laid out there three or four years ago that we were going to start looking at this related to categories. Number one, service only students. We've gotten into that lately. Not always service only students, but by number one, we're talking about the students that appear on your CAEP reports but have zero hours of instruction. In a perfect world, those would be service only students. 

You've got another bucket of students that came into class but got less than 12 hours. That's the one to 11 bucket. And then again, we're maximizing the number who are participants. That is those that get 12 or more hours of instruction. Those that qualify for federal reporting. Those that qualify for CAEP outcomes and love. 

Here is the TE report. I say it's new but it's not really new anymore. It's been around for a year, a year and a half or so now. I think the report illustrates the buckets a little better than the slide previously does. You can see there's clearly laid out those three buckets or three categories that I'm talking about here. 

We've got the CAEP enrollees by hours and service enrollees by hours. There's different ways where you can monitor for each program how many students have 12 or more? How many have one to 11? How many have zero? Obviously in CAEP land and looking back at that goal setting, the name of the game is to maximize those with 12 or more and minimize the number with zero. 

OK. I'm going quick. Again, we've got an hour and we've got a lot to cover. It's 10:50 already for goodness sakes, so I'm moving in to move in to outcomes. This is what I'm using as transition. Some of you I'm sure fondly remember the slide for more like six years ago. Is what we were talking about this when a big exercise we needed to do when we were still ABG. 

This might even be more like AB 86 here, where we looked at what we had for federal reporting related to WIOA. And we had the word I've been using over the years as retrofit. That is the exercises. We have this new state level adult ed program. What the legislature did is it came up with AB 104 legislation which number one specifically required CAEP then ABG to align to WIOA federal reporting. 

And then outlined these six very specific areas. The slide is showing five of them are directly beg borrowed and stolen from WIOA. The sixth one was specified by the legislature transition. That was one they thought was very important and added. 

I'm doing the setup after the laundry listing I guess. But what we've talked about over the years is in federal reporting and WIOA land, the Feds have defined what they call performance indicators and also what they call measurable skill gains. Measurable skill gains by the way, is what I mean by MSGs in the lower left hand corner of the slide. 

In short by performance indicators, the Feds are laying outcomes. I like to call them outcomes that you can hang your hat on. That is very easy to measure. Things like getting a job, increasing wages, getting a college degree, high school diploma, getting your occupational licensure, et cetera. That is final price outcomes that are easy to see, easy to measure. 

The good news with performance indicators you might say is everyone agrees. These are great, great outcomes. Nobody disputes the fact that getting a job or getting an occupational license or getting your GED is a great outcome. Everybody agrees with that. 

The problem with relying on performance indicators is hey, that leaves a lot of students out in the cold because getting a job, getting an occupational license, getting a high school diploma, college degree, et cetera, all of those things take lots of time. If we rely on those or only those then we're going to have a big problem. 

It's going to look like a few students are doing well but most are. And as we well know, nothing could be further from the truth. So over time the Feds have used what they call measurable skill gains or MSGs. You're hearing from the course guys, so of course, I'll say the most obvious one is pre and post testing. 

That was a big reason why pre and post testing was integrated and popularized was the idea that, hey, we can't rely on GED or college degree or that sort of stuff. We need to be able to have better ways to show that most of our students are doing a great job. That is hey, we only have this many that got the diploma or a GED, but we have that many that have done a great job on their courses pre and post testing thereby we know for certainty that our students are doing a great job. So I'll stop here. 

I've been ambling and rambling everybody hanging in there. I've been going a little overboard on this one, I think. But the reason why this is a big deal to ABG CAEP land is we had these other programs for ABE and ESL kind of good to go. We have pre and post testing to measure those programs. But now we have CTE, workforce prep. 

These other programs that we also need to measure so that retrofitting related to coming up with outcomes that related to these other programs and would show progress the same way pre and post testing does for ABE and ESL. So we were using this concept to develop the official CAEP outcomes. 

So here's another slide that I'm sure a lot of you have seen. Again, one, two, three, four, five, six areas of AB 104. Not so coincidentally six different color coded outcomes categories on the slide. We're matching the six categories that the legislature stipulated and we started folding under the outcomes into each of these six different categories. 

I'm going to go back to this one. I want to talk about-- let me go back to this one. Here's some key areas. I don't want to dwell on these, but they are here-- sorry I'm getting into complicated stuff too quickly. But here is an example of ones that we've been talking about where we have the literacy gains for CTE. 

Sorry I'm going from zero to 60 way too fast. But in any case, an example of this is what we call occupational skills gain and workforce prep outcome. To be clear this is just a small portion of it, but we'll stick with this and dig into it as an example of what I'm talking about. 

So with CTE, the usual thing you're looking at is that certification or licensure, which as most of you know takes a while. So we added a one called occupational skills gain. So what we said is just throw in a random example. It may be right. It may be wrong. But we'll say we've got a welding student. We're just making it up. We're saying it's a five module or five semester long program. 

So what we're saying, hey, the student goes through the program. They complete that first semester. Complete a skills check or written test. They do a great job. They move into module number two. That's what we're calling an occupational skills gain. That is the student successfully achieved a portion of a longer term program. 

Another one we did. This is just another big area of confusion. If this seems like gibberish don't worry about it for now. This is just giving you a taste. But another one we've talked about over the years is one called workforce prep outcome. 

That's a similar CTE short term outcome but this is completion of something shorter term. That is a student enrolls in a workforce prep class or program. The example we're giving is hey, a 15 hour module on job search strategies that hey, they complete that class, they get an informal certificate, or whatever. That's where you could mark workforce prep outcome. 

So again, I'm not trying to make it all about this. This is just an example of some of the things we've carved out. This is also a really, really, really good example of something that caused a lot of confusion and generated lots of questions four or five years ago when we were presenting this training more often and so on. 

Another one relates to postsecondary. For some of you, this will be great. For some of you, this will really sound crazy, but just to throw another question out there. OK, we understand what these CTE literacy gains quote unquote, "mean," but how does that relate to this that has a lot of those CTE outcomes? 

We know we should be marking, so this is outlining another way to differentiate some of the outcomes where for post secondary that's what we're using when the student completes that long term program. That is they achieve that final prize I'm talking about. 

So you can see some post-secondary CTE outcomes in the upper left. Some CTE related literacy gains in the lower left. To the right showing how we differentiate it where we have the literacy gains that represent completing something short term or partially completing something long term whereas the post-secondary is when they complete that longer term program. 

OK, here's another issue related to post-secondary. Again, just outlining some of the concerns that we're looking at and soliciting input for. But another big issue is this huge umbrella that the Feds call post-secondary. They do that on purpose because they consider this in theory the next big layer of program or education or whatever you want to call it. 

For everybody once they complete that secondary or high school level, for some students that means move to college, for other students that means move to workforce training. Either way, that means they're enrolled and looking at completing what the Feds would call post-secondary. That is the Feds generic term for what happens after high school. 

Half of that relates to occupational. Half of that relates to education. We've got this huge laundry list of outcomes you can mark in TE for this. About half are occupational, half relate to college. This is just showing which are in which category. 

Another issue is transition. Again, just still giving you a taste here. But we've got official transition outcomes. Making sense of this sometimes is tough. This first one is just showing you what we have available in TE to document when a student transitions to CTE in that middle row or transitions to college. Those are the ones in the bottom row. 

This is the graphic. I think tons of you have seen before that we've had in the training for a long time. This is just showing you exactly how a student makes the transition. That is regardless of what transition the student is making, they're starting out in either K-12 adult ed or non credit community college, ABE, ASC or ESL. 

The red boxes in the upper right are where they go to transition to CTE. That is there an adult ed CTE or a community college CTE. Down there in the lower right is what happens if they make a transition to college. That is they end up in four credit community college. Again, they're in four credit college not non credit college. If they go from K-12 adult Ed to non credit community college, that would be more like a quote unquote lateral transition. Thank you very much, Marjorie. 

OK. One more thing on outcomes. This is coming a little more out of left field, but I've made an effort to really talk about I-3 who's buried in grant's tomb. But what does I-3 stand for? Have you heard about I-3? Yes or no? What does I-3 stand for? Who's buried in grant's tomb? OK. Anybody want to rattle off I-3? 

Some people are saying no. Some honest engines here. Why do I say who's buried in grant's tomb? That's right, immigrant integration indicators, what do you know? It's right there on the slide. All right, thank you for humoring me a few of you. I make a big stink and deal with this one because I've done a few regional trainings and a lot of discussions. 

This is one of those where not everybody did but the overwhelming majority of the state clearly missed this memo. Not really anybody's fault. I think it was June of 2020 when we released all this stuff in TE. That is it came out at the very worst possible time when everybody clearly had bigger fish to fry. But that is when we released it for a little history. 

That was that AB 2098 legislation that was passed in 2018. That is it didn't require that we put I-3 outcomes in TE. But it did require the state to come up with some way to manage progress related to the area of immigrant integration. That was a workgroup that convened there in 2018, 2019. They did a lot of things more than just this. 

But in CASAS land, they answered what our assertion always was. That is our CASAS assertion has been we've been addressing immigrant integration in California for 20 or more years. Only we never called it immigrant integration. We called it EL Civics. 

The workgroup agreed with us. They also agreed with us the I-3 areas that the state was looking for very closely matched a lot of the things we were doing with EL Civics COAAPs for years and years. They allowed us to align those immigrant integration indicators to COAAPs. 

So that is those I-3 outcomes you see on those TE reports are when students complete EL Civics COAAPs and other civics assessments. That is they relate to those areas of immigrant integration. 

So I'll just say the news to you where it might fit is a lot of you have been saying recently boy, it sure would be good if we could use things like COAAPs for students in ABE, ASC, career tech ed and so on. Why is it always restricted only to ESL? So here is your opportunity. 

When you do this for I-3 and CAEP, it's not restricted to ESL. You can use it for all CAEP programs. We've gone out of our way on this with the idea that all immigrant students are in all programs, not just ESL. So admittedly this doesn't do anything for payment points. That's on the WIOA II side not CAEP. But this is an opportunity to implement this for other students. 

If you're not interested in that you should at least know where those I-3 outcomes are coming from. Short answer is a lot of you saw big numbers there even though you didn't know what that column meant. It was because you were doing a good job in a EL Civics. If you're doing a great job in EL Civics, you'll get good numbers in that I-3 column. 

All right, a little bit of a bird walk, but that's another outcome I wanted to impress. Here is the infamous bubble boy slide. This is just taking stock of all the outcomes. This is mostly for reference but I'll just point out again, the six different color codings at the bottom. Again, those six different areas of AB 104. 

This is the CASAS update record just here to show which outcomes relate to all those different things you can report for CAEP reporting. So just to give you an easy example. You can see we're marking it all the employment related outcomes in blue. Very unsurprisingly in the upper left. 

Got a job is in blue. That shows you that's one of the employment categories. Keeping it simple. You can see, get a job, retain a job. Entered military are the three we have in blue. Those are the three that officially relate to employment outcomes for CAEP reporting. 

This is the exact same information, but it's showing you the information from the vantage point of TE software rather than that CASAS update record. This is directly from TE. It's giving letter designations rather than color coded designations. 

But the same six categories the same methodology just showing you which of these different outcomes in TE relate to different areas. You can see some of them are not marked. If they're not marked, that doesn't mean they're not a great outcome but it just means they're not officially related to anything for CAEP reporting. 

OK. I'm going to stop here. That was frenetic. That was crazy, but we only have so much time. When we have 2 and 1/2 hours in regional training, we'll kind of stop and smell the coffee more. But just to give everybody a flavor, any questions? Any comments, or concerns? If not, we're going to transition to services. I'll pause for a minute. I'm not seeing anything. No dissent at least so I'm just going to move on. 

So we've also been talking a lot about services. When we talked about all this years ago, I'd say we spent a lot more time on outcomes than services, but I've been bringing up services more lately just because this is one of the concern areas you might say. 

Number one, some of you are saying this fell by the wayside. This is an area that the best of the best. Again, I'm not talking about the ones with the head in the sand. I'm talking about the people I consider to be CAEP superstars are saying, yeah, this is an area where we really fell off the grid during COVID. 

In some cases, it's just hey, it's COVID. It's a pandemic. A lot more difficult to provide short term services, especially a lot of those community related services. When COVID was going on, they just weren't happening. Others of you are still doing it, but just the nature of recording it was more difficult. So some of you knew what you were doing, but probably not recording it as well during COVID as you were before that or now or whatever. 

Others of you have been saying gee, even before COVID, this was a loosey goosey area. What I've been talking a lot about is when we did regional trainings, this is a good example. That outcome bubble boy another good example. You'd ask us questions about what you needed mark. 

We'd shrug, we'd laugh, and say, oh, my gosh, the CAEP data points are probably going to get you. We'd explain hey, we're a one person CAEP data police, so probably never going to get you. So as long as you're systematic and deliberate about this stuff, you're probably going to keep out of trouble. 

Just to make sure everybody's listening to me, does anybody remember that illustrious narrative from three or four years ago? Just to make sure everybody understands. That's been a big issue in CAEP land. Didn't hear about it for a while, but hearing a lot about it here the last few months. 

So I make a big stink and deal about this because what a lot of people are saying, maybe you agree with this, maybe you disagree. But people are saying, gee, we want you to stop making all those stupid CAEP data police jokes and start tighten it up or shift. 

That is what you're saying is, hey, I'm looking at these reports, I'm looking at it from the vantage point of Bakersfield. I want to make sure that all these CAEP out comes and all these CAEP services that I see on the report in Bakersfield are the exact same outcomes and services that they see in Fresno, that they see in San Diego, that they see in Sacramento whatever. 

They want there to be consistency so they know those numbers on the TE reports really mean something and that you can kind of compare across regions. And really, I'll use the same cliche, hang your hat on some of the numbers you see on these reports. With all the goofy CAEP data police jokes a lot of people feel like those numbers don't really mean anything because the way you're doing it in Bakersfield is probably completely different than the way you're doing it in San Diego, for example. 

So I'll just say, looking at this has been-- it's not ready for prime time but something we're looking at. Back to feature presentation for services, we have these three categories, supportive training and transition. Here's the screenshot from TE with all the different ones. 

In short, supportive services are ones that focus on the students personal issues with the idea that the student has personal needs. If you don't address them, they're not going to be whole. If they're not whole, they're way less likely to Excel in the classroom. So things like child care transportation, fiscal counseling, and so on. 

Training services typically relate more specific to employment and training in the local region specifically workforce training in the local region. So sometimes the student wants to do employment or employment training. There's very specific qualifications. The student may not have training services are there to try to fill those gaps to get the student up to speed so they're able to qualify for said training. 

Transition services. I'd say a lot like training services. They're specifically there to help students transition to employment, transition to college or both. So what I'll point out, I'm not going to go over this. This is 20 minutes we don't have. 

But what we've also included here is a good start for ways to connect the dots, relate some of these services we have in TE to real time activities you're more likely to be doing at the adult ed program. Picking a few giving you some specific examples. 

I'm not going to go blow by blow but for your reading enjoyment, here are some that we've looked at. That's what I mean by looking at these a little more carefully, more thoroughly defining some of these. This isn't the final product but this gives you some ideas on where we might be going and might give you some ideas to better relate services for right now but you can look and see what you think. 

I think that covers what I really want to say on services. Before I move on though, anybody with any comments either on the three categories of services on what I said about your ability or inability to record them during COVID, or any of this retrofitting gobbledygook I've been talking about. Any comments, concerns, questions at all? 

NANCY: Yes, have a question about our Family Resource centers. So we have a family Resource Center that provides a lot of those different types of services that you just outlined. It's a bit of a pain to register everybody that walks in the door because they had some help writing a resume. Is that something we should be trying to capture better? 

JAY WRIGHT: That's a great question. That's probably something we need to address for sure. But I'll say that's actually one where we always gave a firm answer in the good old days rather than a wishy-washy answer. So I'll use that one. That is you're not required to do that. 

Sorry, a little history is when we first started making a big deal about services there in 16, 17, nobody believes me, but it was a by popular demand request. Kind of like you're saying, hey, these are very important things we do in adult ed. We want to make sure it gets documented and recorded, so we added them. 

What shortly followed that was the idea that you're saying, hey, we're doing family support services. The example I remember is things like CPR training. Or yeah, we'll do CPR training or record that we're doing it for these students. But come on, give me a break. We're not going to pass out an entry record to everybody. 

So the TE reports purposefully go out of their way to not require demographics program enrollment hours of instruction for those services students. As long as you record the service, you're OK. Back to that, yeah, there's a groundswell of support to more broadly defined services. So I can see how that might need more information about the student. But for now, that's optional. It's definitely not required. I see a question Connie which I think is a lot like what Nancy outlined. 

So I'll just say everybody, but especially Nancy and Connie, does that sound like an OK answer? I give that answer though admitting that that's a good example of something that we're looking at. And once we come up with hopefully a better system for this, yeah, I could see that might require us to move the goalposts a little bit. 

OK. What I want to do and again, because another big issue that's come up a lot the last few months. Again, all those concerns about recording outcomes and services and making it a little more streamlined, firming it up, having it reflect what we do a little bit better is one big concern or one big bucket of concern. 

Another one I think is a little more straightforward. Gee, we've had turnover. People have unwittingly inherited duties they didn't really want to inherit. But gosh, darn it, they're stuck with inheriting it anyway so a lot of people need real basic level. What I'll add here anybody here from CAEP only agencies? That is you're at an agency that's funded for CAEP but not federal WIOA II. 

Those are the agencies that in particular have come out of the woodwork and say, yeah, we need some basic level instruction on what we need to do to just keep the trains running on time, so to speak. So here's a few of these from WIOA accountability on the basics that's presenting it from the lens of the courses, entry and update record, but it should give you what you need moving forward. 

So here is some enrollment or entry record requirements. Do you need to fill out the courses form? Absolutely not. Lots of you have different ways of recording enrollment. Maybe you do it manually. Maybe you have your own agency local system. Maybe you have some other way of doing it. It doesn't matter your method, but regardless of your method, you need to record enrollment in a CAEP or WIOA II program. 

For years and years, going back to 1999, we've been super firm on when you record enrollment, that is day one square one. For CAEP, it's exactly the same policy as WIOA II, day one, square one. That is as soon as the student sets foot inside your doorway, that's when you should record enrollment or complete an entry record on that individual. 

When you do so, please do collect demographics. Be sure to collect gender, race and ethnicity, date of birth, and so on. Please do record education level. Please do record labor force status. Please do record barriers to employment. 

Sticking with enrollment, here's some things that were new. Not really new anymore, but new five years ago. Barriers was a real big one. Everybody's done great with that. But yes, we caught as many barriers as apply. Yeah, that's still something you would do in updates. Yeah, I think if you're talking about Carnegie units, yes. 

The other one is if you have people who transition or who are a co enrolled with your WIOA partners, I'll say these top or I'll say that bottom one really is the one I mean, co enrolled with Title I for year four. The two big ones that came up for WIOA were those bottom two bullets. 

The first one barriers has been a big success story. The bottom one is if they're co enrolled. We haven't done as well with that. You can't record what you don't know. But if they're co enrolled, please do record that in personal status. 

To your question Nancy, I'm going to give it short shrift, but the short answer is what it means is if you were a quote unquote, "housewife" for a really long time. Maybe it's not as relevant now as it was years ago. But hey, you've been a homemaker all your life. Now you're divorced or you're a widow or you're a widower and now all of a sudden you have to go in that big bad world out there. That's considered a barrier to employment. 

It's not that you haven't been working. You've been working your tail off but not in a formal work environment. That's been a federal category for years and years. It still is. I don't know if that answer is it or not, but that's my-- the short answer is, yes. Sorry, way too long. I'm not going to do that long answer. That's 10 minutes. We don't have. Thank you, Nancy. 

So here's the big new one this has been a big deal in WIOA land. The policy is the same for CAEP relating to enrollment. That is record primary and secondary goals. If you're like most, you've been doing this anyway, so it's no big deal. But for WIOA and CAEP, that's now required at the state level. Not going to dwell on it too much. But the short answer is when it was required in 2012 and earlier, it was required because it directly related to the NRS federal tables. 

It still continues to not relate to NRS federal reporting in any way. That's still completely detached. But because we're doing things like CAEP goal setting. We're working on the SIP or WIOA reporting. Yahoos like me have talked about goal setting a lot. It was thought, hey, we really need to require goals at the student level because we've been talking so much about goals. 

OK, moving on to update. That is sometime later. Short answer is for entry, we've always been super duper firm. Day one square one. You really shouldn't deviate from that. It's very prescriptive about when you do it. For updates, we've been intentionally loosey goosey. That bullet at the top is what we've said every year since 1999. 

For updates, you can align it to your academic calendar or course schedule. Do it by semester, trimester monthly, quarterly, however you want. Five years ago when we shifted to WIOA, we got more prescriptive when it related to attendance. 

Anybody want to give me the one or the three word or the one acronym answer for why we're so persnickety about attendance hours now? Anybody know where I'm grazing, just wondering? OK. Great. Thank you, alright Yeah, that's right POP is the one acronym answer I'm looking for. That is POP or TE is automatically determining your exited students through attendance, through that 90 day rule. Don't have time to dwell on that either. 

But because we're doing that automatically in TE, you need to record attendance at least once a month. For most of you, that's probably not a quote unquote, "Update record issue." But update record is one of the main ways you can record attendance and get it in TE. So whether you're doing attendance with the update record directly into TE or through the import, export for a third party system, either way, you need to make sure you do that scanning or manual entry or import export at least once a month. 

Here's a screenshot of updates. This has been a really big issue the last six or seven months. I kind of unwittingly thought everybody had moved out of this, but was wrong. It came to a boil about five or six months ago. So we're saying it more explicitly now on that form, there's fields called status and progress that used to be important. Please leave those blank. 

If you're doing attendance elsewhere and not completing that, then what on Earth does the update mean? It means those outcomes. Really that's all it means anymore. But I will add because of CAEP and because of WIOA, these outcomes and field 9 more important than ever. 

I think you mean, if they get through the 90 days and our eggs that are-- Yeah, sorry that one's a big time. Too bad. What I'd say to that, sorry about that, but I've got to say that's come up a lot. For now, I'm not sure exactly, but I got to say, in that new RFA that CDE is doing they're going way out of their way to tell you, don't have any summer break more than 90 days. Going way out of their way to say if that's what you're doing it's 125% on you and 0% on us. 

Got to say that means you need to whack your district upside the head and have them come up with a better schedule. Yeah, that just means you've got to follow up on all of them. Yeah, we've gone through that in the last few years. 

Sorry, but there's a very firm answer. It's talk to your district. Correct your district because again, we're feeling like the district needs to be corrected. We're never going to correct the feds. Is unmovable as your district is. It's a lot more maneuverable than the feds are. So bad news for you is you better move your district or else. 

Sorry, very firm on that one. I've heard Carolyn give that answer. I know that's the answer. We're not going to do any weird funny business. That just means you follow up on those students. Get your district in line. 

Anyway, here's just a blown up version of what we just talked about related to attendance. And here's some resources. Sorry, I knew that I'd run out of time. Again, try consolidating 2.5 into one hour, but I'll point out here is that. So let me get everybody's foot on the rail because I'm talking about too many things at once. We've got five minutes, so we're kind of OK. 

Yeah, that's been said a couple of times. All you're doing is admitting you're late because we've sent it a few times already. Maybe Marjorie will graciously send it again. Anyway, here is some basic resources. To get everybody's foot on the rail, has everybody here been to www.casas.org, yes or no? I'm asking an purposefully obtuse easy question. Of course you have. 

So less obvious, but is everybody been to what I call our CASAS California Accountability page? That's what we're showing on the slide. That's where we post the CAEP data dictionary, WIOA data dictionary. Gazillions and gazillions and gazillions of attachments. So this is just showing you how to get to that page. Thank you, Marjorie. 

I don't know, it's annoying thing about Zoom. You post it and anybody who comes in late doesn't get it, so you have to continue to post it. I've got to say it's my least favorite Zoom feature. Anyway, here is where you can get the CAEP data dictionary. There is an updated one for '22, '23. Not much different from the previous year but it has been updated. It was posted mid to late July. 

For your quarterly data. Due date right around the corner next Monday. If you're new to this, we've got this Data Submission Wizard. I think most people like it, but it makes it, streamlines it. It allows you to submit your WIOA data, WIOA DIR CAEP data. And CAEP DIR all in one fell swoop. It's the one. It's in the wizard. It's in the tools menu of TE. You just run that Wizard and take care of all your data submission issues. Here is a link to our website with some very detailed steps to get through the Wizard and TE if you're new to this. 

We also do the employment and earnings survey. That's been a CAEP requirement now for the last year or so in addition to WIOA II. It's also been streamlined, so you can take care of the WIOA II side and CAEP side in one fell swoop. 

It is a different wizard than the data submission wizard. However, here's a link to the instructions for the employment and earnings survey. I admit a few years ago, this was a big stink and mess. I always point out Nicole Jordan from our office. If you're here, you can lap up more credit. But what she figured out is most of the confusion had to do with exit quarter which looking at it from a federal point of view is beyond confusing. 

It's a joke. It's so darn confusing. So a good way to make it easier is just create four different documents for each quarter. So you're not keeping track of that crazy federal system. Yes, if you use Social Security number, that gets you out of a lot of this employment and earnings survey related activity. 

You still need to run the wizard, but most likely you won't have-- if you're using SSN, you run the wizard but you won't have to follow up on any of the students. Here's the calendar. Again, the due date is one month after the last day of the quarter. The exception is that end of year. It's a little bit earlier. The due date is very, very unpopular. Everybody hates it. 

I'm here to tell you it's here to stay. Nobody believes me, but in CAEP land, we moved it up from August 1 to July 15. Again, by popular demand simply because everybody said, they don't like the early due date, but better to have everything on the early due date than try to come up with two stinking due dates, so CAEP moved it to July 15 just like WIOA. 

Yes, if you're using SSN for all learners. I don't know if you are or aren't. You still need to run that wizard every quarter. But yes, if you're using SSN, the wizard will just tell you you've got SSN so there's no students for follow up. No, you don't need to do any of the grunt work for following up, but you still want to run the wizard to show well, then, yeah, you're going to have some students to follow up with. That's the issue there. 

The Feds have set it up to where data match is what they want us to do. For a year that's what we did. The data was beyond miserable because most agencies, most students either won't give SSN or don't have it. So our data is inherently terrible if we rely on SSN. So we came up with this employment and earnings survey as an alternative way. 

So if you don't use SSN, then yeah, you run the same wizard like Ann as asking. But in your case, obviously you'll have lots of students that need follow up. If Ann is doing what she suggests and using SSN and runs the same wizard, but when she runs it, she won't have to follow up on anybody because she is using Social Security number. 

If you're half and half like you suggest, Elsa, yes those that have SSN won't appear on that ENE survey wizard. Those that don't have SSN of course, will and those will be the ones that get followed up. Hopefully so I am talking a mile a minute. Elsa and hopefully that answer is it. 

Upcoming training. Like I said, I did the Valley a couple of weeks ago doing Mt Diablo and Sacramento County Office of Ed November 9th and 10th. Doing LAUSD LAREC region region on the 17th. CCAE Palm Springs on the 18th. More sessions for January and February forthcoming. That is the 2 and 1/2 hour face to face session. 

And I think that's about where we want to leave. I think we're done at 11:30. It's 11:31. That puts us on a transition. So all kinds of stuff related to TE Reports. Obviously we need to cut it off. But lots of information in your PowerPoint related to reports. And then if that's not enough even crazier stuff here related to using TE reports for goal setting for your regional enjoyment. 

I'd say use this last slide. Contact me by email or contact CAEP TAP by email to get that started and we'll take a look. Thanks for that offer. All right, I'll turn it over to Marjorie or I'll never make it or Marjorie is going to kick me off stage here very soon. So thank you very much everybody. I'll move it so Marjorie can get going. 

MARJORIE: All right. Thanks, Jay. Everyone, I did put the evaluation link in the chat. And just reminder that the posting of this recording. I know a lot of you are asking for it is dependent upon review of the evaluation so please make sure to provide feedback there. 

Also it's time for lunch. So the exhibitor sponsored lunch today is being presented by National Geographic, so check that out. Have a great lunch, and we'll see you back for the next sessions at 12:30. Take care everyone. Have a good afternoon.