HOLLY CLARK: Luke. 

SPEAKER 2: Thank you so much, I'll go ahead and quickly share the steps to access. So hopefully, everybody knows how and has logged into TE and they understand all that, and this is all-- but if you're brand new or if you're just wondering where I'm going, very quickly, you can always go to the CASAS home page, type in Rolling Hills, you'll pull up a search if you have Google Ads because I don't disable them. Just scroll past them, you'll see steps to access the simulation server. 

And then it's going to take you to where to go, right? So, hey, make sure you put Rolling Hills if you're going there, or if you're logging in, make sure you select the Rolling Hills, 4908, Administrator10@rhas passwords, admin for admin, coordinator coordinator, proctor proctor, teacher teacher, right? Just simple stuff. 

I'll go ahead and start sharing TE now. And let me share. So I logged in here. Let me disconnect so you guys can see what that would look like. Sorry about that. So Rolling Hills, 4908, administrator, and admin is the password. I'm going to try and go over a few things fairly quickly. We'll go over the Program Nova hours, and then I'll go over some of the reports for the CAEP stuff, and then I'll go over some of the transitions. That way, we'll try and leave the most time for you guys to ask questions. I'll go ahead and pull up the chat so I can see if anything's going on. 

So the first one is if you go Reports, you can go to State Reports, California, and you'll see the CAEP Program Hours, right? Like your Nova hours, stuff like that. So when you click it, you're going to have a lot of general settings, like on all the reports under General Settings. When you scroll down, you're going to notice you have, do you want to see all of them, do you want them broken down, a ASC equals high school diploma and HSC or ABE/ASE, where you merge all three, or CTE merges the workplace and the pre-apprenticeship, and the CTE. Or do you want them all broken down in all their instructional programs, OK. 

So you can kind of change how you want that to be. You want to leave the Nova, so if you don't change anything-- excuse me, and if you just generate it, you're going to get the default settings, and it's going to-- you'll get your hours, and it won't say preliminary. 

Now some of you might want it broken down a little differently. And I think we've updated it where it does break it down properly in the different ways. But you can, the one thing you do need to be worried about when you do generate this is, sometimes people will say, oh, I'll come back here and I just want to run my basic skills hours, right? And you'll run your basic skills hours. And then you'll say, oh, want to just run my ESL hours, and I just run my hours, and so forth. The one thing you need to be aware of, when you do that, you're now changing how things work. 

When you use those Nova's hours, say you have a CTE class and an ESL class, and it's like a 243 or something, and they're doing that extra stuff, and so those hours have to get split, right? So if you look over here on your instructional programs, I don't know if we have one in necessarily, but we're just going to-- because it's the Rolling Hills database, we can just create our own scenarios, right? 

When you go in here, if you do have a scenario where you have two instructional programs selected, you want to make sure when you kind of run those final reports, though, you do leave everything enabled, right? The reason, if I go back to here, if you do it one at a time, you're going to be doubling up those hours, if that makes sense, right? Because now you're getting the total hours. Whereas if you were to leave everything defaulted off and selected properly, then what's going to happen is it's going to split properly, right? It'll split those hours because half will go CTE, half will go ESL. But if you run it one at a time, you're going to lose some hours. 

The other thing is, when you mark all, I think we fixed it, though, where when you mark the basic skills, it should still split it. But I just know we recommend when you're doing your final numbers to run it. The other thing you're going to be asked is, I think, sometimes you need to pull specific classes out and all that. That's also perfectly fine. You can do that. There is no issue with that. 

We just say when you're going into your class instances, when you're pulling selected things, when you're filtering properly, just know what your filters are, right? So you can do that. It's designed to do that. Just be aware of when you're doing this, you know exactly what classes it is, and then you're making notes. 

The other thing you can do is on the right-hand side, you have the criteria information, I always recommend leaving this. And the reason being is when you scroll down, it tells you exactly what classes, what filtering, what settings, things like that. And so that's-- to me, that makes things easier. 

Is there any-- I know I'm moving a little quick, but any weird questions? Or does that kind of make sense when you know, hey, I want the hours, you pull it up you, if you need to go filter, you can filter on the left-hand side, if you don't need the filter leave, it default? You're going way too fast. All right, Cecilia. I apologize for that. 

So again, to slowly kind of go over it, when you're going in to the program Nova hours, Reports, State Reports California, Program Hours. It's also under your CAEP tables, but this is just specific. We'll have a thing in there, stuff posted. You can filter, just like anything else. So if you need to filter by class instances on the left, you can. If you want to change how it's broken down, you can here, next to the CAEP program areas. 

You can filter by specific programs, just make sure you know which ones you're selecting when you do it. And there's a way to aggregate hours for the consortium. If you're a consortium manager, when you go into reports, this is only when you log in at that like C30, C19, C4, C1, whatever consortium level, there will be a consortium-level manager reports. And those are going to break them down that way. 

If you're at the agency level, you cannot do it at all. And vice versa, if you log in at the consortium level, you cannot log in and run agency-specific reports to get specific information that way. You can only run the consortium-level side. That might be inside ballpark some people, but I hope that kind of makes sense, as I'm trying to answer questions I might be missing. But yeah if, you run the consortium you can do the consortiums. But in the industry, at the agency, you can't run consortium, and vice versa. 

OK, and then-- excuse me-- I'm a little sick, obviously. So that's program hours. Same thing, if you go reports, or any questions on that? Makes sense? Perfect. I know I'm going a little quick. The call, they was saying it is going to be recorded. But basic, it's ways to generate hours, essentially. And you can filter and change things if you need to, is the gist of it. 

All right, perfect, perfect, Connie. The next one we can say and do is the big one. When everyone says, oh, I got to-- what's the tables? Is that just one table? No. No, it's not. Same thing, when it populates, again, this is the CAEP stuff. So we're getting all of our CAEP program areas. You can kind of do the same thing to help you out with the program, the Nova hours. And you're wondering why that's there. And the same thing. If you need to break it down by specific classes, you can. 

Or if you want to break it down by program, you can still do that. All that is the exact same. So when you go ahead and click Generate-- well, hopefully, that muted, and you guys didn't hear me cough. Perfect. You have the CAEP summary first gets generated. And so, this is going to show you their literacy gains, the CAEP outcomes, the services. And it's going to break it down by those program areas in a nice, clean little one-page summary. 

On the left-hand side, though, this is where it's kind of saying those tables. This is where, OK, summary, that's a great little thing, it's perfect. But then, if you click outcomes on the left, now you're like, whoa, what is all this great stuff? And so, now we're going to little deeper, right? So now if you tilt your head, OK, post-secondary, wages, transition. OK, you start getting all this stuff. 

And then, now we got, across the board, our numbers. And like, oh, that's cool. That's all it is, right, just that? And then you say, no, there's some I-3 stuff. I think Martha's going to be going over some of this stuff. And then, you've got CAEP services on the left-hand side. So now we've got more, all the services. So all those different options, they're going to get broken down by program as well, pretty quick. 

Yes. There, I guess we could-- well, on the left-hand side, you got program hours, so that's the same as the Nova hours, same exact thing that's how you can edit it. Where you click here, there's an info. Here, let's go back to Martha was saying. So if you click Info, you can kind of see the things where it's going to pull all the stuff. Is that what you're saying for the CAEP summary and how it breaks it down? 

MARTHA PEREZ: Yeah, exactly. That's correct. 

SPEAKER 2: So if you guys need it. 

MARTHA PEREZ: It's just a little tip for everyone to know that is there, and it's kind of like a cheat sheet. 

SPEAKER 2: Built-in cheat sheets are awesome. 

MARTHA PEREZ: Thank you, Luke. 

SPEAKER 2: OK, perfect. Yeah, so I guess you don't need the PowerPoint. It's all here, at least, yeah. Then, when we go back-- thank you both for that, Martha. I forgot about that one. We updated that. Program hours, so now we've got all the outcomes, the services, but now you've got the hours, the learners, if they have no hours, the enrollees with, without. And it kind of breaks it down a little more detail. 

CAEP Service Enrollees by hours, because obviously, you have with CAEP, it's a little different, right? You have your people that you're just making contact with that you want to collect, hey, we provided those services. They didn't necessarily, maybe, didn't attend, but we put resources toward that, right? And so every agency, I'm going to say, is different. And I'm obviously not going to argue how you guys collect your services and information, and things like that. 

But there may be a debate about, hey, if you did make contact, if you did put resources, if you did help somebody out, maybe you do want to try and collect that information, right? And then, obviously, if they attend for some hours and you provide more, you have that stuff. And then, obviously, if they have 12-plus hours and you provide resources, that's also great and wonderful. But that's why you have those here, it's because it is going to be that scenario. 

And oh, yeah, I guess we got a couple extra hours maybe. Career, OK. It's a good one, under the CAEP Services Enrollees. Hey, that's a good one. Good catch. We'll make sure we fix that. Butchering career and transition. So when we go into the transition we'll kind of show that there's-- how do I explain it? There's transition services received, and the career services received is, I believe, the path they're going. 

So when they're exiting out-- and Janice, thank you very much posting it. It's the padding they're going through, if I'm not mistaken. Because, kind of like the 243, if you're going that way, they can do the workplace or they can do like the CTE class way. And so I don't want to say it's like on the wages and things like that, but like Martha showed, when we go over the transitions, it's going to show up if you mark it how they get marked that way, if that makes sense. There's kind of career selections, employment, things like that. 

And then zip codes that we can break down where your populations are coming from. Obviously, I might be missing questions. I apologize. Or we'll try and make sure we get through everything and get back to those if I do miss something. And then zip codes, very plain, very simple, can help you identify where your populations are coming from. 

Your races and populations, your demographics, you can pull all this awesome information. It gets a little small, so you can zoom in and figure that out. Then outcomes by hours as well, just like the other outcomes, except this time it's breaking down by those hours. Because again, those hours are going to be important, especially how you break it down and how you contact those people. Hours are everything. Make sure you import them. Make sure you track it. Make sure you put the instructional programs. 

So this is the CAEP tables. It is a lot. It is one report with many reports within it. It's going to be confusing, it's going to be messy, but you do want to run it. You do want to go through them. Are there any generalized or specific or something that doesn't make sense? Even though I went over that very rapidly, but it is a lot of stuff. But the tables are very amazing, like Lydia's saying here. 

Does anyone at least get the gist of when they say the tables, or if they say, hey, the services, things like that? Anytime they say a report with CAEP, think of it as it's probably going to just be in the CAEP tables and it's going to be on the left-hand side. 

The I-3 stuff is-- Martha, if I'm not mistaken, is going to go over a little more detail. But it's essentially those CAEP things. And the feds, I think, approved it to make them I-3 now. So it goes across the country. Yeah, Martha gave me the thumbs up. So she's going to do some more detail with that and kind of explain and do those breakdowns for you. 

So the CAEP tables are great. If someone's talking about something, and when you go to Reports. Or if you see it-- so you'll notice the codes and stuff-- always, if there's anybody who shows a report, there's always a code in the top right. So just make sure you always-- if anybody hands you a piece of paper, and you wonder what that report is, there's always a code right here. 

And as long as there's that code-- OK, it's in TE-- we can regenerate. It if they hand you something and you don't see that code, they made it themselves, but that's OK. Then when you go to Reports, State Reports, California again-- and so if you don't see it here, and you're like, well, they said it was a CAEP table-- it was probably just under the Tables, all right? 

Then you've got your barriers to employment. So same exact thing. When you're pulling the barrier to employment, you have all your program areas. You have your basic things. You can do break it down on the reporting levels. 

You can also go to the class instance, break it down on classes, things like this. Every report is going to have these ways to filter. Nothing's really going to change. Under the general settings, things will change a little bit. They'll always be slight changes based on the report itself, but generally you can always kind of do the same filtering and then with barriers. 

Now, you get to see all those things that you tell your students make sure you mark. Hey, it does matter, right? You can see there's some Cultural, Person with Disability, Displaced Homemaker. And it's going to get broken down by the CAEP programs, right? 

And so there will be some duplication, obviously. And then you're going to show ones with how many, right? So we had three with seven employment barriers. And so it's a simple report, but it's a good report. 

It's going to show, hey, here's our students. Hey, they do have these barriers. We're helping them get through these barriers and overcome them. And so it's just helping you build up that defense, showing you where you're improving. 

Or you might see, hey, actually, we've got a lot of language as a big barrier or low levels. And then you can apply resources and try and counteract those things, right? Because that's the whole idea. 

Where's their problems? Where's their issues? And then apply resources and services to make it a little easier for the students. So this will kind of help you out a little bit here. And obviously, it's also broken down by program. 

Any questions? Barriers to employment, pretty self-explanatory. Anything specific here? Oh, everyone's asleep. OK. 

Show the drill down. We're going to show the drill down, Connie. We're going to show the drill down. So, simple thing. So this is where we're going to have fun. Connie already knows I'm going to have fun. She knows me too well. 

The Cape Data Integrity. Yes, my friends. Yes. The most magnificent report to solve all issues you will ever have is the DIR. Now I will say one simple thing-- if you're worried about testing, don't worry about the KPIR on your testing. 

The reason being is you're going to have those extra programs that do not require you to have testing, right? So if you've got those CTEs, or if you've got those workforce, if you've got things like that, careful with the call pass. They're not going to-- 

I'm just operating the chat. They're not going to work because it's going to be inflated, right? So when you come down here, the numbers are inflated. So we recommend just running the regular NRS/DIR to compare your testing because that's just going to run the WEOA, just the ABEE cell, HSC and HSD. 

But for missing birth dates, you've kind of got the situation where you've got five missing birth dates. Now, if you left click, boom, it's going to pull up the five students. And you can see that there's no birth date. 

And obviously, you can double click on the student, and then you can select the student up in the top because now we're at that highest demographics level. You can edit the student, right? And now we're literally in the students database and saying, hey, you need a date of birth. And that's your date of birth. And we can save that, right? 

And so we can fix these students by doing that. That's one way to do it. And that's how we would kind of recommend. You just click it and fix it. 

The other thing is if you're trying to figure out a specific-- mostly with testing and other things. Like Connie was saying, if you right click, it gives you a lot of options. One of them is called the drill down to NRS monitor, drill down to the outcomes. 

And so you can kind of do all those same reports, but the big one is drill down to NRS Monitor. And you can do this on any report with any group. And it's going to give you those federal tables monitor. 

If you've ever ran it, it's going to show you their pre, their post, their start date, end date, their drop reasons on the far right. Obviously, these are-- we're only looking at missing birth dates, so that's why it's all one here. But then also, if they had a level gain, things like that. 

So this is going to be a case where, hey, we do want to solve these because we could earn some money, or we can show progress better, or can help out our federal tables, things like this. Yeah, so first person. So I hope you saw how I cleared it out. So, yeah, for item 1, you want to just left click and click through them. And then you can always do the drill down to see exactly what's going on. 

When you're running some of the federal reports-- so if we regenerate it, we're now going to have only four because we fixed that one, right? I'm trying to keep up. Some people are direct messaging me. That's probably making less sense. Sorry about that. 

So essentially, you have to regenerate it to show that, hey, when we solve that one, it now does come down to four, OK? So because we clicked it, opened up the student, added the date of birth. 

Now, when it says 18 to 110, that literally means their age range, right? So when you go in there, you might notice, oh, this student is 18, right? But what's going to happen is there are two factors of the 18th birthday. 

We reconfigured it to look at program enrollments. So we'll go ahead and open up another student. So you can left click, click a student, click a student again. Here's their name. 

So when you click on the name, it's now going to take us to-- notice how we have more things on the left hand side of the Navigator? We're now at the very highest level. We can't click anymore. This is like essentially the demographics level. So we're the way. So again, you can click Edit. And this is 10/6/2006. So it's out of range necessarily. 

So the other thing you can look at is Program Enrollments. So with Program Enrollments, you're going to be looking at, say, this is-- say they just turned 18 like right now, right? Like in 12/7. 

But say you have your e-testing set up where you know how you can choose test date or program your start date? So this might be like an accidental 7/1/2022 or '23 because it's this year. And so because they have a program enrollment essentially on 7/1, they're saying, hey, they entered your program on July 1 when they were 17, right? 

Even though you're saying, no, no, no, they turned 18, then they entered the program, you want to check the program enrollment dates to see if that date was before their birth date, right? And then the other thing is to see if you guys enrolled them in classes. And so this is where we're going to say make sure when you're enrolling them that you're keeping accurate dates, right? 

Because if they are enrolled before they are 18, or maybe they were signed up, but they didn't start until thereafter, it's one of those things like, oh, we accidentally enrolled them too soon, even though, technically, they didn't even show up until after they were 18, right? 

And so it's one of those things where you want to make sure you're very specific. And when it's not like, oh, yeah, yeah, we're starting that class. They all start on this day. But it's like, well we had to hold this person back a couple days. 

So that's where I'm not going to argue schematics. I would say please talk to your-- like Jay or Barbara or whoever your consultant is. I'm not going to say there's exceptions or not exceptions or rules. Every agency is different. Everything is different. Every grant is different. 

What I am saying is I would not-- clean accurate data is clean accurate data. And if somebody gets dropped off, somebody gets dropped off. But if you need to help somebody from your agency, I would contact them. And I don't know if there's a list or something. 

I know years ago, there was kind of an exception thing where you could sign off your DAR and things like that. I don't know if things like that have changed necessarily. But would contact your consultant. 

I'm not going to say there is or isn't. Every single year, it changes, everything like that. I'm just saying you always want to have clean data. Don't change it. You're an agency. Things happen, services have to be given. We kind of-- we understand that respect. 

So when you're talking about less than 12 hours and the different hours, so the big thing is you have one to 11 hours-- I need to speed up a little bit-- and you have zero hours. So zero hours normally means you made a mistake in your class setup. 

So when we go back to-- hopefully, I still have it. I don't, obviously. So if you have zero hours, that means the student is getting zero hours. You're most likely just missing a class instance, instructional program under the class instances. 

So here you would just want to go in and make sure you add like if it's high school diploma class. And then those hours in that class would start getting picked up properly. So that's the big difference. 

Whereas, when it's one through 11, that means, hey, they actually are attending. That means they are getting some hours. But most likely, they're just one through 11, meaning they're missing a few hours, and they're not at 12 yet; or they're a brand new student and they're getting the hours, then hopefully in a few weeks or next week, they'll have more than 12 hours. So that's kind of the big difference there. 

No gender, same thing. You can go through that. Race/ethnicity, the same situation. Labor force, same exact thing. You can go in and fix it. 

The whole thing with the pre-testing, again, we would prefer just running the DIRs or changing the program areas because this is going to include those extra programs that don't require testing, right? The X's in any area, that just means you've got like a pre and a post. 

Shall students with zero attendance not imported? Again, that's where you can have students with no hours if and maybe you're collecting those services, right? So remember when we ran those CAEP tables, Eve was saying like, hey, should we not import students with no hours? And so that's where it's kind of how you guys set up your system. 

Do you set it up where you give-- maybe they're receiving some services, but they don't have any instructional hours. And so hey, we've got 50 students this year where they didn't necessarily come in 10, but we gave them some counseling. We gave them some of these services. We helped them out with some of these things. 

And so we're technically giving them services, but they're not going to be participants. They're not going to have any hours. They're not going to show up in those other things. But we provided services to these students. And we want to track that, hey, these students came in through our doors. We provided services to them. And we tried to help them out that way. 

And so some agencies have it set up where they try and track those. Even though they're not necessarily going to get hours, they're going to track those students coming through their doors. So that's kind of what you want to be doing. 

If you're in a situation where you did like an accidental rollover-- you're saying, like, hey, you did some proxies and they never showed up-- that's a little different. They never technically showed up this year. But if you've got people coming through the doors and people you're seeing and the people that are testing, you want to be tracking those things or people were enrolled and things like that. 

Now, if they never showed up, if they never did that, that's a different kind of thing. But you do want to be tracking those people to show, hey, they came through our doors. We provided things for them. That's the whole idea of it. Even if they maybe didn't get hours, you still help them out in some way. 

Same thing-- so they're achieving their functional gains, high school credits, ninth, 10th to 11th, 12th. They're earning things, the HSC, high school diplomas, the gaps. These are those POPs. Remember, if they have missed 90 days, and they create a new POP. 

The IET thing was the vessels. Earned IET, no primary goal, no secondary goal. At least one barrier-- the barriers we went over. And so all of these are essentially the-- what's a POP? So a POP is a Period of Participation. 

So that's where we were talking about how if they have-- missing 90 days, they're going to get dropped, right? So we'll do a drill down like Connie likes. Right click NRS Drill Down Monitor. It's going to pull the federal tables monitor I need to hurry up, probably. 

So they get a start date, and they get an end date, right? And remaining are separated. So if there's a 90-day gap, they'll get dropped off. This is not a great example. Let's see if we can find a great example for you. 

I know I'm going a little outside. You'll have the drill down option when you right click, yes. So here's a good example. So 8/31/2023 dropped off on 9/21, 120 days missed. But then they came back on 1/8/24. 

Obviously, this is fake data. And so now they're creating that second POP. So instead of it just being one POP, one POP, one POP, they've got one POP in a year, and now they've got two POPS in this year. 

And so they can have three, right? They could drop off again, have another 90-day gap come back, have another POP. Or vice versa, you could have somebody who attended five years ago constantly had attendance or a test, because that's what's considered activity test, and hours. So attendance and test is considered activity. And so it could be 2020 all the way till now, just one continuous POP. And so that's kind of how the federals do that. 

So any quest-- so I know data integrity, there's a lot of things. It's very good. [INAUDIBLE] the reports this year. Yeah. So the other thing you can do is-- so, again, as you can see, more things qualifying for the CAEP, qualifying for that, the SSN things. The ITN we added if they're missing a phone number. 

I'm not going to say it gets less and less important. It's just information, right so if you're concerned, like, oh, they didn't sign it, but they did sign it, it's just more information. Or if they're missing a phone number, they don't have a number, they don't have an email, it's going to help you with NE obviously, so we're going to recommend, hey, try and track them down. 

But any specific questions with DIR before I move on to some of the static reports since we're going to be adding some more of those as well? I know I answered a couple at the top. Mostly, these top ones are very important because these are going to affect you because these are going to be drop reasons. And then these ones are going to kind of mostly help you out with making sure you're checking your accounts. Hey, this is how many grads we have; this is how many reporting, things like that, making sure all your numbers match. 

So some of the static reports are when you go into your state reports, you have static statewide reports. And so this way, we can see-- you can double-- or I guess download or open. I guess I should just opened it since we're here. 

I think you guys-- so if you click Open, you can't see this. So let me new share here. I'll just do Share Screen 2. I don't want to do Share Screen 2, but I will because I'm nice. So does everybody see this? 

So essentially, instead of seeing your agency, you can see the entire state if that makes sense, right? So 9,000 people are missing birthdates. So we got to say outside of the range or outside the range. Maybe there's 17 or maybe they're 111. You don't know. 

But yeah, work on those, obviously. And then same thing with-- so we can close that. And you can open up the NRS one. And then we're going to add more reports this year. This year, we'll be adding more reports. So keep an eye out on the static reports because we will be adding more. Thank you, Nicole. 

All right, and now, just to make life easy for everybody, let's hopefully-- I will quickly go through outcome and services. So since I'm-- well, we're doing good. So WIOA core, you've got Title I, Title II, Title III, Title IV. 

Same thing. Most of you, job careers counseling, job search, adult education, high school diploma, experience, job seekers including getting unemployment benefits, Title IV, disabilities, employability, integration-- all those wonderful things doesn't really matter. But it kind of does matter, right? 

Adult education, dislocated worker, Wagner-Peyser/one-stops, vocational rehabilitation. Those are the different breakdowns in the WIOA programs. They can also mark it. So make sure students are still marking those. So if you're like an adult school, and they're going to those other places to get those extra resources, they're still marking that just like in the transitions. 

Now, when we go down to WIOA Alignment, you have your indicators-- employment, wages; MSGSs, literacy gains, post-test, secondary, training milestones, skills progression. I'm talking way too fast. 

And then if some of you might have remembered the old AB 104 before it was the CAEP stuff back in the day, they had the different outcomes, right? And so what I think most of you guys are really looking for is these. 

As Martha showed you when you go in there and you click the Info, it shows you the exact breakdown. This is also going to show you the exact breakdown, meaning when you mark these on your update records, this is now going to count as literacy gain. When you mark this, this is going to count here. 

When you mark these, this is going to count for post-secondary. When you mark these, it's going to count as transition. When you mark this, increased wage. Interim employment's going to-- when you mark these, right? These are all going to be the triggers. 

And so when you see here, here's the triggers on your update record, right? Under your student records for your results, here's all the triggers. So I'm not going to go over these. I'm not going to kind of go through them. 

But the idea is you can kind of print this out or look at these and say, hey, do these go with any of our programs? Are we making sure we're asking these questions maybe during the update time? Are we making sure we're getting everything updated correctly, right? 

And so here's another thing. And so this is what you saw when you clicked Info, how it kind of-- and this looks like it in TE with the work results and all the different results on the TE side, how when you mark it, these are the triggers for those things. Perfect. Yeah, exactly, Clarence. 

Hey, print them out. Make sure everybody understands them, make sure they know the triggers because this is what's important, right? Don't necessarily be guessing. We kind of have that plan laid out for you for your agency, right? 

Maybe you have some of these programs. Like, oh, we have a program, or maybe you have another college on-site where they go through and they get signed up, and you have that enrollment. Or you can contact your secondary positions where they're going to and getting lists and matching them, seeing, hey, did they pass GD, things like that? You're kind of making sure you're not missing anything. 

Same thing. So again, just breaking it down the different ways and the different courses. Some tip sheets. Actually, I think we're good. Yep. Post versus literacy gains. And so it's going to break down with post-secondary? Attained credential, occupational licensure, occupational thing, versus longer term program versus longer term completion of a shorter term. 

And then so when you break down the occupational, you have these educational outcomes, you're talking about these sides. And so you can kind of to-- this is kind of your cheat sheet if that makes sense. If you ever were unsure or were debating, go back to this-- I think we posted this. It'll be posted again. But use this as a cheat sheet. 

And I've used way too much time. Any questions as I went through this before I pass it off to Martha? And hopefully, we'll have some time for obviously more questions. Sorry for the voice. Martha, take it away. 

MARTHA PEREZ: Thank you so much, Luke. Thank you so much for all that interesting very important information. So that's amazing to know how much we can learn and how much we can report in TOPSpro Enterprise. So bear with me. I'm just sharing my screen. 

And so we're going to continue with the-- let's see, with the second part, which is part of the employment and earnings, employment and earnings part. So we're going to start with doing a quick overview of the employment and earnings survey. 

I'm going to do a quick overview on how to send those employment and earnings, and then show you all the reports that we have available, of course, under the Reports menu. So we would start with-- under the Reports menu under State Reports, we want to generate the Core Performance Population report. This Core Performance Population report will give us the exact information on how many students were selected as exiters for a particular time period. 

So the report by default will show the current program year. So what we want to do is always go back to two quarters back, which is the quarter where we're going to select the exiters from. So we want to see the ones for the '23-'24 program year. For right now, we are in-- physically, we are in quarter two of the '23-'24. 

But I want to see who needs to be surveyed. So we want to go to the '22-'23 program year. We want to go back to quarters. So that would be in quarter four of the '22-'23 program year. So I'm going to select that program year and click on the Generate button. 

So once we click on the Generate button, that should be showing all the information for the '22-'23 program year. It's just taking a few seconds. And then this will present us with all the exiters from exit quarter one. 

If you want to only reduce that to quarter 4 learners, then we can scroll down to the bottom section. It will show us which quarters we want to include. Well, I didn't unselect any of the others. That's why it took a little longer to populate. 

Remember, you can always unpin this navigator on the left so we can have a better view. So it will give us from exit quarter, and it will include if you're also NRS-funded, like WIOA-funded. It will also include a list of the exeters, and it will separate them. 

It will give you the period of participation, when they exited, the exact exit date, and also include a Y if the student has a Social Security Number. This doesn't mean that these students that have the Social Security number will need to survey because, obviously, they won't. 

They have already a Social Security number. And CASAS will do the data match for these particular students. So it is just telling you that they were selected as exiters. The total amount of students, for example, on the quarter 1 was 554 on the NRS side. 

And then 23 of them have Social Security numbers, so they won't need to be surveyed. As you can see, there's a little note in here. 23 students have SSN that they won't need to be surveyed. 

So then we will have another list for CAEP students. You may not see the NRS if you're only CAEP-funded. So this is also included in here, as well the same legend for the students who have Social Security number which doesn't need to be surveyed. 

So once we review this report, which you can print and export if you like, now we know for sure exactly which ones are the exiters. So once we know that, we will be running the Core Performance survey, which is under the Tools menu. 

In the Tools menu, we'll click on Core Performance Wizard. And once in this wizard, we also want to make sure to retrieve back the '22-'23 program year and click on the Next button. For right now, of course, we're using the Rolling Hills, but the survey has already been generated for all the quarters. 

Anyway, once you go through this wizard, you will be seeing the exact same list of exiters. So I'll go ahead and cancel this. And once you have generated your Core Performance Wizard, then you want to go to the Records menu and click on Core Performance Students. 

This is where you're going to find your list of students that got selected as exiters. And you may notice that this already has an filter embedded by default to go back to the two quarters behind. So two quarters back, this is going to be filtering for those students who exited two quarters back. 

So as you can see here, there's quarter 4 exiters from the '22-'23 program year, which are the ones that are targeted to be sent out all the surveys. So from this point of view, you can start sending the survey invitations. 

You may want to-- I don't know. That's up to you, whether you want to sort by English or by native language. You can send them in native languages. We have about nine different native languages translated into these surveys. 

So I would recommend starting on the very first language on the top. And that way you can filter and send out all the survey invitations by selecting them all and going through this Survey Invitation Wizard. That would be the first step on how to send out all the surveys. 

If you like, you can send them all in English by just doing Control A; and that way, you'll be selecting every single student. And you can send the survey invitation all in the English language. 

So this is a task that every agency must go through every quarter. I usually recommend to put a note in your calendar and a reminder to be able to send that a survey like at the beginning of the quarter or as soon as you can at the beginning of the quarter, because, that way, you will have more time to be able to follow up with all the students. And then how you would follow up with all these students once you have sent out all their invitations, you would go to the Records menu and then click on Survey Responses. 

And again, I'm just doing this quick overview of the survey invitations and the Employment and Earning surveys. So once you have sent all your surveys for the current quarter, again, there is a filter embedded by default, which is set to the current quarter, which is where they need to take the survey, which is in the current. For people who exited in the fourth quarter, they need to be exited in the second quarter, which is the current one. 

You may want to pay a special attention on this when it's the end of the quarter. For example, on December 31, we'll go into January 1 of next year, and that is when the time physically changes. And you want to pay special attention in that time after January 1 because it's not going to be in the current quarter, but in the previous quarter. So that's where you can change your filters and then go through all the other-- the previous quarter exiters, which is also going to be a reportable for the second-quarter deadline, which is on January 31. OK. 

So once you go through the Surveys Lister, why you would want to filter for the people who have responded. So I would click on-- OK, I have a question. All right, Molly, give me just one second. 

So I want to filter for the people who have not responded to our surveys and then click OK. And once I get this list of 66 people, I will go ahead and send them out a second time. I will send them out a second time. And like Molly said, do you notice that the delivery status says Pending or Delivered? So the delivered is pretty self-explanatory. That survey has been delivered. 

But for the pending surveys or the pending delivery status, it means that it's probably still going through, like, getting out of TOPSpro Enterprise, and sending it out into every customer or every user or student that has exited the program. So if you see this pending, it's probably because the California server might be getting a little traffic through sending out the surveys. 

But if you notice this on the next day that you send out the surveys, and it's still showing as pending, please give us a ring or send us a message to techsupport@casas.org so we can ask our programming team about it. Sometimes like these, there's many agencies sending out surveys at the same time, so they have to do like a mass push out of all those surveys. 

Now once you have a-- you can resend surveys up to three times from TOPSpro Enterprise. As you can see, the recent Invite button will be enabled, and you'll have to go over through the wizard again. You don't have to make any overrides or anything because if you already did some customizations, they will stay in there until you change them. So you can resend the invitation. 

And up to three times, you can send them SMS messages or emails from TOPSpro Enterprise. And after that, you will need to follow up with them via phone call. And you must also make sure that before you send the first survey invitation, you want to gather all the emails possible or all the phone numbers you can so you can send them into your users or your students, all right? 

So once we go through all this information-- this is about sending the surveys and following up with them-- then here comes the other fun part, which is tracking them in here in the reports. And for that, we have several reports under the Reports menu, State Reports, California. And then we have the Cape Employment and Earnings Survey report. 

So this report-- oh, this is a report-- OK, question from Rebecca. "How do you manually enter responses from the phone calls?" That is a very good question. So let me go back here into the Records menu and then Survey Responses. 

Let's say I am on the phone with a customer that has not responded, so I'm calling them right now. And they get my-- they answer my question. I'm going to go through the one that it's been delivered. I would go ahead and double click on this response. 

And once I have the user or the customer on the line, I would click on this Answer Survey button. As you see, the questions are here, but we can't really manually enter any responses in here because we don't have an Edit button. This would be very easy to manipulate, and then we would have probably not good responses. So that's why we have to click on this Answer Survey button. 

And that will-- well, we have this little message. But this should show us report in-- this will open up on our default web browser. Could it either be Chrome or Edge or whatever your default browser is. And you would be able to enter all the responses as if you were the student. 

So this will connect you. Yeah, for some reason right now, it's not working. But this will connect you into your web browser and say, hello. And then it's going to say, Candido Stephen Cordova. This is going to say, "hello, Candido, are you-- have you been into-- are you attending school in a different area? And that way, you will be communicating that to the student. 

And then if they say yes or no, then you would apply that response manually for them and go over to the next question. What school? And then that way, you'll be entering the responses as if you were the student. 

Once you finish that, this has been answered. And then it will show as filled in by staff. It will show, Yes. OK. 

"And then what is generation date and time?" That is a really good question. I always like to populate-- you can customize your columns however you like. But I always like to include this generation date time because I want to see when this survey actually went out to the students. So you can right click in any of the gray areas on the top of the columns and then say, show generation date time or show other options that you would like to see in your lister. 

And then once you have that and you want to save it-- you always want to see those columns in there-- you can click on More and then click on Schema and then save the schema as your default. That's fine. You can just save it. And that way, every time you come to the Surveys Lister, you'll have that generation date time. 

So when you send the survey invitations for the first time to a group of students who have exited, you will send out two survey invitations. You will send out their first survey. And then after six months, another survey will automatically go out to them just to double check because everybody will need to get surveyed twice. 

So that is why this is-- I mean, this date is in the future because that's how we have the Rolling Hills database. It's always populated with fictional data. But you can always go back to other months as well, depending on the quarter that you are checking it for, OK? 

So let's see. "Click Get Item Responses in order for the data entries to show on study records." That is correct. That is correct, Molly. 

So then we will go through-- oh, yeah, the dates throw you? Yeah, I know, it throws off a lot of people, the dates. But you always want to refer to our documentation on our website to send out those survey invitations. And that way, you can follow through. Or again, you can send email to techsupport@casas.org, and will help you out. 

And now, another part, which is the reporting part of it that you can rely on in order to track all these students. We have these great reports that once they were created, a lot of people has made life a lot easier, which they are under the CAEP, the State Reports California Cape Employment and Earnings survey, the survey summary, the quarterly status. And I'm going to go through each of them, as well as some other reports for the Immigrant Integration Indicators, I-3. 

So let's generate this first Core Employment and Earnings Summary report you can always generate it as is. So we can see what programs or what surveys that students have received. And then this will show you all the responses, how many surveys per quarter we have sent out. 

It will give you the breakdown in every quarter, how many they were delivered, how many failed. They wouldn't fail unless you would try to send them outside of the quarter that corresponds to those exeters. It'll tell you how many of them have access and what percentage of that it is; how many have not responded; and how many have responded. 

So you want to take a look at this percentage here. This is the total percent of answers per each quarter. So seven did not respond by-- and set 6, for example, out of 30 people, six have been responded. 

And as well, anywhere where you can see the cell highlights in red, you can drill down to see who these people are, who they had to wait for a call for them to respond, or how many are the first responders by the student directly. You can always see all that information and then the total percentage that has responded into-- then there's another drill down. 

Like, which language they were sent in, what method, as well as the delivered invitations for them. As you can see, there's 30 students in here, and then how many pending invitations they would have. There's "I always forget how to navigate to the Employment and Earnings Survey. Can you share the link on how to access it on CASA's website for this next quarter?" 

Absolutely. I have a link in here for documentation on our website that will provide you in just a second. Here is a link. And you can download that document. Oh, thank you, Luke. Perfect, excellent. 

All right, so now this will give you all the drill down for all the quarters in the program year. So it's same thing, but just different quarter and how many people in total. Very, very important-- I always emphasize when talking about employment and earnings and sending-- or first running the Core Performance Wizard, is never to mark your students as retained. 

In their program enrollment status, if you go to Records, Programs, Enrollments, please do not mark them as retained because the retained in program status will not acknowledge your student as an exiter. Let's say if a student exits in quarter 1, this Retained in Program will override their POP, their Period of Participation; and it will leave them until the last quarter. 

They will still show up as an exiter, but until the fourth quarter and not in the exact quarter they left. So this is a good reminder to never mark students as Retained in Program in the Program Enrollment status, OK? That way, you'll have your exact count and appropriate numbers in your tables. 

So I have another. Judy asks, "My CAEP Employment and Earnings surveys are grayed out. Why? I'm the one who sends the survey invitations out. I've never been able to look at those reports." OK, you may just want to ask your data manager to enable those on your account, and then you should be able to see them. 

So again, Reports, State Reports, California, Employment and Earnings Survey. And then let's take a look at the Employment and Earnings Survey summary. I'll go ahead and click on Generate for that. 

You may also want to track back to the prior program year as well because as you know, these are-- we always go back to quarters. So if I go back to the general settings, you may want to do it for the previous program year as well, or really depending on what you want to see. Again, our Rolling Hills database has a pre-populated fictional-- with fictional data database that will always give us accurate information even in the future. 

So in this report, we will see the exit quarter on the top right area and then also the program year that we're doing it for. Let me unpin that. And it will show you your agency information and how many have exited with invitation; how many total invitations you've sent; as well as how many surveys have been accessed, not responded, or responded by student and by staff. 

So this is just like a summary information. And again, you can drill down on any of these cells that highlight in red. And then we have eight pages for that. So that means that you have a couple for each quarter per se. 

So now, once we have reviewed on these reports, I would highly encourage you that you generate those among your agency. And if you don't have access to that, ask your data manager for permissions to receive or to review these reports. They're very handy, especially if you work directly with sending out the Employment and Earning Surveys. 

Now, I'm going to go through another report under State Reports, California, CAEP Quarterly Survey Results. Let's go ahead and generate this for the '22-'23 program year. I'll go ahead and generate that. 

And another thing that you can generate for if you want to drill down like for-- or you want to break it down to a particular class per se. I don't know if anyone would like to-- well, I don't think that this can be done. 

But we have Employment Earnings Survey. We have Student Survey or Core Performance students. So the report selection is the CAEP quarterly survey results. I would just select-- keep it to the defaults. Just manage the different program years that you want to generate it for. 

You can always change the report style option and, like Luke said previously, just try to include all the criteria, and just generate it as is with the default settings. And then so we can see what information it's brought up here. Well, the quarterly survey results will give us all the California WIOA Title II exit students, the ones without an SSN. 

It's just giving you a general roundown, as well as the average wage that every student that responded to the surveys have. And then I know that we have been recommending to use-- if you have them on the phone, and they don't want to say how much they earn because they consider like it's a privacy issue, I would enter-- which is the recommendation from CDE. I would also expand on that, which is use the general wage or the minimum wage for that area where the student lives in if you have access to. 

You can Google up the zip code, and that way, be able to use the minimum wage-- or the state. Yes, thank you, Kim. The state minimum wage. Perfect. And that way, you'll have all these elements entered in here, and they will reflect on your report. 

All right, and then another report. All right, thank you for joining, Doi. And then another report that we have under the California State Reports-- we have the El Civics Immigrant Integration Indicators Report, I-3. 

Before we go over that, I want to share a slide on the PowerPoint presentation, which talks about the legislature the AB 2098 which requires the CAEP to include updates on immigrant integration in annual reports to the legislature. So for that, we have this new-- well, not very new. They've been there for a little while. 

But the Immigrant Integration Indicator reports, they track the El Civics APS outcomes and relate them to immigrant integration goals. So that way, we'll know when the student is passing the COAAP assessment that relates to the I-3 goals and indicators. So in order to generate these, you can find them under Reports, State Reports, California, EL Civics Immigrant Integration Indicators. And then we have the summary, which we'll also go through these. 

All right. Now, we'll go ahead and click on the Generate button so you can just see what they are looking like. And that way, we will explain this report. So we have the number of-- we have the names of the student IDs and which immigrant integration goal areas civic objectives they have passed. And they are also related to citizenship tests as well. 

So as you can see, this student, Lilia, she did two immigrant integration goal areas, which was health and well-being, as well as digital literacy. And this is what they are able to demonstrate knowledge on health and safety precautions, as well as use online tools. 

So this is a great report. So you can see all the immigrant integration areas your students are involved in and that have passed any of the COAAPs that they have received. So again, this report is under the State Report. 

So let's go over another report, which is the EL Civics I-3 summary. So the I-3 summary is a summary version of this prior report that we saw. Let me click on Generate. 

And this one, you can-- these ones, you can filter by a particular class instance. So from the Navigator on your left before you click on Generate, if you want to generate it for only a specific class, you can uncheck them all. And of course, make sure that is an ESL class that has the focus area. 

I always like to generate or populate the focus area for civic participation, whatever the focus it is. Then you would want to generate it for that particular class by unchecking them all and just checking that particular class if you want to just see that particular class report. 

But I'm just doing it for all the classes. This is a summary for all the classes. And it includes these credentials and residency immigrant integration goals areas. And then it tells you how many people have attempted that and what percentage they successfully passed. And then you have the health and being immigration integration goal area, as well as digital literacy and how many people have responded on that and the percentage success rate on this report. 

This is a very summarized version, but, again, you can always drill down. You can right click as well and then drill down to I-3 detail. This drill down to I-3 detail, I'll get it from right clicking in that red cell. And then when we go to this I-3 detail report, it will tell me-- it'll give us the rundown on each specific student and which objectives they've passed or immigrant integration goal areas that they have received. 

So let's see, this also includes the citizenship interview tests as well. So let me go back to my other report Immigrant Integration. Here we go. Let's see, Use Online, Demonstrate. All right, just going back. 

Here, passed the government and history test, passed the citizenship interview test. So these are also included in here. If you have a citizenship program that teaches the citizenship interview test and how to pass it as well-- they receive some government history instruction-- then you can-- and you have those COAAPs added in there or have them take those assessments, then they will also show up on these reports. 

Any questions on any of these reports that we have shared today? Or any questions in general? Thank you so much, Luke, for taking care of, responding to those questions in the chat. 

OK. You can unmute yourselves as you like if you like to contribute or have any questions. Or you can also post them on the chat. Remember that these reports can be easily found under the Reports menu, under the State Reports, California, CAEP Tables. Under the CAEP tables, you'll find the Core Performance Population, all the ones that I've gone through, as well as the Immigrant Integration Indicators. 

If you ever forget where to find these, then you can always come to the Report Locator Wizard, which can show you. If you just see I-3, if you type in I-3, then you will add a dash 3. And then you will get the reports and here. 

If you want to just see the summary, then you can click on that. And you can say, add to my reports if you like. The Report Locator Wizard will perform as like your favorites in like a list of songs or something. And you can just add it to your reports. And that way, you will have it into your favorites. 

And you can also launch it from here by clicking on the Launch button. And this will automatically launch it in the background. Then we can close this locator wizard and will automatically launch it for you. And then under Reports menu again, you can find under your reports. And you will see the ones that you have as your favorites. 

So I already had a couple in there, but then the El Civics I-3 summary is now your new favorite. And you can add as many reports as you want from the Reports Locator Wizard, which will help you find any possible reports that we have in TOPSpro Enterprise. 

And you can always say something like employment. And you can just type in like key words or key letters on how to find. Then you will find EMP as a keyword. Then you will find all the other reports that are related to employment. So then here, you will find employment earning service, a summary, as well as to the barriers to employment and all of these good reports that we have available for you. 

Any other questions on any of these reports or any of the information shared in today's meeting? We also shared on how you can log in to the simulation server, which is something that I always heavily recommend on all users. So that way, you don't mess up your own data. 

You can use the Rolling Hills simulation server so you can retrieve and review all these reports that we have available for you. And that way, you won't mess up with your own data. 

So, any questions? I don't see anything. I hope this was clear, or clear as mud in today's meeting. 

HOLLY CLARK: Yes. I don't see any questions either coming into the chat. I'm trying to start my camera. I'm not sure why it hasn't restarted here. I'm going to go ahead and drop the evaluation link. If you can please take a moment, provide us your feedback. 

Martha and Luke, thank you so much for this presentation. Part 2 was just as amazing as part 1. You guys are absolutely amazing in your knowledge of TOPSpro. And I know that we all have our little nuances that we struggle with. 

This will be recorded, everyone. I'm sorry, this has been recorded, everyone. So again, we will send it out for 508 compliance remediation. As soon as it's ready, we will post it on our website, as well as follow up with an email to everyone who registered sessions 1 and 2 to let you know that they are available, and provide the link as to where to find them. 

So I'm still seeing the chat. I don't see any other questions come in. I think that wraps us up for today. Again, Martha, Luke, thank you so much. To all the attendees, thank you for spending your afternoon with us. 

Today, we gave-- yesterday, we gave you 28 minutes. Today, we're giving you back 15. So have a great rest of your week and enjoy the holiday season that is approaching. Thank you. Bye bye.