Speaker: OK. Good morning, everyone. I'd love to introduce these two fine educators, Thoibi Rublaitus and Jay Wright, who are here to present Fly a Kite-- oh, exciting-- On the Less Than 12-Hour Range Students. So without further ado, I think we know the drill by now, welcome, and I'm going to pass it on over. Thank you for coming.

Jay Wright: Thank you, Karen. So this cover slide is really to make sure everybody knows they're in the right place, that was the title. So this should be self-explanatory that everybody is in the right place. I'll move-- here are introductions. I'll let Thoibi introduce herself, and then I'll introduce myself. And I'll probably be talking for the most of the first half of the presentation, and Thoibi will be guiding through most of the second half. We'll be interjecting, of course. But Thoibi, go ahead and introduce yourself if you don't mind.

Thoibi Rublaitus: Thank you, Jay. Thank you, everybody, for being here this Friday morning. It is pretty early, and it is admirable that you are all here to join us. And Jay and I are happy to have you here because I'll be sharing a little bit of some of the action and research work that we've been doing, I've been doing at my school. I am the principal at Corona-Norco Adult School.

And I've been an adult educator for all of my adult life because I started teaching English as a second language by fluke sometime in 1994 in New Delhi, and then I came to the US and I still-- that was a good time, it change my career when I came to the US, but I couldn't get anything better, so my heart is still in teaching so I ended up teaching at Corona-Norco Adult School as a part time adult there teacher at night and day time ESL teacher at the community college. But eventually, I'm lucky to be part of the Corona-Norco family. And this is my fifth year as an administrator.

Jay Wright: All right, thank you. And I'm Jay Wright from CASAS. I'm the accountability manager, do a lot of trainings all over. By the looks of the participants, I've probably-- I worked with you and your region some time in the past. So moving to the next slide, these are the main things we're going to go over, is we're going to talk about the issue related to 12 hours of instruction. Oversimplified, long story short, in the first half I'll talk about a few TE reports. Surprise, surprise, surprise.

And we'll also look at some of the data. Some of it is updated. We did recently update end-of-year DIR, so we'll look at some of the data. I think the long story short there is the 12 hours of instruction data looks like that same checkmark, that NRS performance and persistence and CAEP summary data looks like.

But we will show the data, and then second half, we'll turn it over to Thoibi who's done a lot of really good research with her team at Corona-Norco one this issue.

The other thing I wanted to bring up is I do realize now, I like serials. That is, S-E-R-I-A-L, not C-E-R-E-A-L, but serials, and that we have installments. That seems to be the name of the game. This is another one where I think this is the fourth installment of this that Thoibi and I have done together. We've advanced it a little more.

And I gotta say, on her end, there's, a lot of data results that we took a teeny-tiny peek at the last time we did this at the CASAS Summer Institute in June, but it was literally hot off the presses, so there wasn't enough time to really get into it, but there's more time to get into it now. It's not quite so hot off the presses.

So that's really, I think, the buy into why it's really worthwhile, is there's going to be some great data that Thoibi has that I don't think anybody's really been able to sink their teeth into yet.

The last thing, and we tested this-- we had a wonderful cutesy little Charlie Brown video we were going to show, but we couldn't figure out a way to do it without blasting you with a bunch of ads. So we decided that would be bad, but we call it Fly a Kite.

If you know Charlie Brown, you know there is that-- actually more the Springtime Charlie Brown's Special. The fall special is my more favorite analogy related to Lucy with the football, but in the spring, there's always that little shtick where Charlie Brown's out-- that's right. Good Grief, Charlie Brown. He's trying to fly a kite. It always ends up in failure. It always ends up looking something closer to this.

Of course, thank you, Carla. Perfect segue. So the reason why we like that analogy-- or at least I do, I'll say, is that that's how we look at the 12 hours of instruction. When you're flying a kite, it's really, really, really hard to get it off the ground, it seems like it ought to be easy, but it's always about 50 times more difficult than you think it ought to be.

Once you get it up in the air, it's not so bad. It usually stays flying pretty well. But getting it off the ground is difficult. That's the point we're making here, is that is you really need to focus on those first 12 hours of instruction. With a lot of students, that makes or breaks their time in adult ed.

If you get them off the ground past that 12 hours, not always, but usually the result is a really good partnership. If they don't get past that 12 hours, then you probably won't. That's kind of our bottom line. We have the video to try to illustrate it. I think that description, not as good as the video, but hopefully better than nothing.

What that, we'll move on. I do want to start with some TE reports. This is the CAEP Summit. So I gotta say, there are definitely some official CAEP activities that relate to this issue. So it really seems remiss if we don't at least point it out.

We spent a lot of time in the last year talking about CAEP goal-setting. That is at the consortium level. Like it or not everybody has a goal to monitor how many students move from the 0 hours bucket to the 1 to 11 hours.

At the agency level, everybody has, like it or not, how many students move from the 1 to 11 hours bucket to the 12 or more hours bucket. That is, if they're in the 12 or more hours bucket, they're officially participants. When they're officially participants, they qualify for the CAEP Summary, they qualify for CAEP outcomes.

Oh, by the way, that also means they qualify for payment points and NRS federal reporting outcomes you might say not so coincidentally. Also not so coincidentally, we're presenting this as a new report. It's not really that new. We've had it in TE for about a year, but we have a couple of new hours reports.

One is the CAEP Enrollees by Hours. The Enrollees by Hours is laid out a lot like the CAEP Summary. The same three categories you know and love from the CAEP Summary with a little more detail related to hours. Again, those three buckets.

Within each of those categories, how many students have 12 or more hours, how many have 1 to 11, versus how many have 0. We've got those nifty little arrows to hopefully point out what I'm talking about.

Here's just a blown-up version so you can see the different buckets, the number of enrollees that appear in the CAEP Summary in that particular area, and then a breakdown of the different hours categories. That is 609 plus 98 plus 148 hopefully equals 855 just for a quick example.

There's another-- again, not really new, but semi-new report called Services by Hours. The purpose of this, the same as the one I just showed you, but looking at it in an inside-out way, it's not directly related to the CAEP Summary like the other one, but it breaks it down into those service categories.

And so it's a little bit more-- I'm sorry, I'm playing musical slides, but if you look at this right-hand section, that is the Services section of this report, same as the services. Section of the CAEP summary, that is where people go that don't have any hours.

As a lot of you people know who really dig into this, a lot of times you'll get big numbers in that right-hand Services section. Sometimes that's because you really are providing a lot of short-term services to your students and a lot of people are there because that's where they belong. Other times that shows up there because what I call clutter, students without hours, import/export issues, duplicate students, orphan records, et cetera, et cetera.

So the Service Enrollees by Hours allows you to look at that Services section and be able to see how many of the students actually have the different kinds of services--

Short, oversimplified answer here being if you have big numbers here in these different services, then that suggests, yeah, you've got a lot of people there because you're providing services. If you've got real low numbers on this report, then you can probably more safely assume a lot of that-- a lot of those numbers and services are more related to just those that don't have hours or demographics or whatever.

There is a newer report, this one's not brand new but more like three months old instead of an hour old-- or a year old-- I don't know why I said hour, but Outcome Enrollees by Hours, this is just another kind of inside-out way to look at it.

This one is breaking it down into the Services categories. This one is breaking it down into those different Outcomes categories where you can look and say, OK, of those that made literacy gains, of those that made I3 gains, et cetera, et cetera, where do the students break down in those different hours categories?

So overall, this is a segue way-- I'll just stop-- actually, I will stop here. You could do that-- and I'm going to give it short shrift, but I'll answer it. You can do that in the Setup window the same way you would do on that CAEP Program Hours because I know you're asking it, Carla.

You know what I'm talking about. I'll just apologize, I'm not sure everyone does. But there's that Nova format. So this is just showing you that default Nova format like they have in Nova where they shovel it all in.

I think you can do it if-- you can't drill down, but you can make the same changes where you can break down A, B, E, A, S, C. I think it's a three-way split where you can do it the way they do it in Nova, you can pause out A, B, E from A, S, E. You can also parse it out so it's A, B, H, S, C, N, H, S, diploma. I think there's a three-way split there, but it's all governed from the Setup window.

I'll just say what we're showing now is the default view because it is the way it is in Nova. We set the default view to the way they have it in Nova. Thank you. Great question. We know people are paying attention. That definitely did way more than enough to give me the warm and cozy and moved to the next slide.

So this is just a recap of a lot of things we've been talking about. I won't get into it too much. I've had a couple other sessions this week that got into this more, but hey, you look at this data, you check out, in this case, hours information. We've been talking about what do you do for the next steps. Quantitative using data reports.

I like to use that Sesame Street analogy. Find areas of particularly strong or not very strong performance. Focus on the neediest areas. Focus on some of those that look better than expected as well. See which of those is not like the others. What are they doing different that might represent something a little better or worse than everybody else?

Make those adjustments accordingly. Qualitatively follow up from your-- follow up with your students. Follow up with your staff. Find out what they say are the areas of strength and areas of weakness and so on.

I'm using that as a segue way to get into this data dive section. That is a little deeper with these reports. We've talked about goals. Here is a way to look at that first consortium-level goals. That is how many are moving from that 0 hours bucket to the 1 to 11 hours bucket. You can look at this report to see those numbers and track how many are in each one.

This is a deeper screenshot on that agency-level goal where you're looking at moving from 1 to 11 to 12 hours. This is a little more case-specific slide to show you which columns and which cells you might review to track your performance in that particular goal.

Changing gears a little bit, looking at the DIR, I'm changing gears a little bit because there is an updated DIR Document. The updated document focuses more on NRS than CAEP, but I really do think the pattern of the numbers is the same.

I'm being a little loosey-goosey here, but generally if you tack on like a plane or two to the NRS numbers, that's basically what the CAEP numbers are. The patterns with NRS and CAEP tend to be the same.

Overall, it's the same data patterns, but because it's CAEP, because it's not quite as punitive, some of those cluttered numbers like 12 hours in demographics on the CAEP side are typically a point or two higher than they are in NRS, but again, the patterns are the same.

There is a link to where you can download a new document that includes end-of-year data from 21 22 where you can look at all the DIR items and compare and contrast by region, you can compare and contrast what you're doing at your own agency to the statewide numbers.

Here is a little bit of a deeper dive on that. This one is, I gotta say, hot off the presses from just about an hour and a half ago. How's that for a good little true confessions? So you can't get any newer than that. OK, I see what you're talking about. Sorry, it's taking me a minute to do-- or it might take a while.

This is a heavy PowerPoint, by the way, so it'll take a couple of minutes to show up. But here is the updated DIR data. Hey, by default, you're going to get a nifty update on missing birth date. Believe me or don't believe me, but the slide and all these screenshots really did look better with that extra row, so I just left it in there.

But you can see, here's four years' worth of less than 12 hours of instruction. Statewide data going back to 2018-19. You can see that same kind of, you might say, reverse checkmark that we've been talking about with lots of our other data when we're talking about table 4 persistence. You name it.

Lately, our data tends to look like a reverse checkmark, meaning we were starting to look pretty good there around 18-19. Things took a little dip for obvious reasons in 19-20. Took an even bigger dip for the same obvious reasons in 20-21. 21-22, we got better.

We're still not as good as we were three or four years ago, but we are a lot better than we were one year ago. You can see all these numbers follow those same patterns like we've talked about with lots of other data performance issues.

Here is a little more specifically how we did last year. This is breaking it down a little bit into those extra buckets we have on the DIR. Again, not enough time to dig into some of these basic issues, but most of you, I think, have a pretty good handle on this.

On the DIR, we have categories that pretty much match those different buckets we talked about a few minutes ago. For CAEP reporting, that is we have the overall number of qualified-- or the overall number of learners that have less than 12 hours, and then we break it down into those extra buckets, of those that don't have the 12 hours, how many of them are there because they're 0 or empty versus how many of them are there because they have hours but just not quite 12?

You can see 0 hours tends to be the bigger issue. That's true this past year. That's true pretty much every year. That really hasn't changed, although I will say in the COVID years, the 0 hours was the more fluctuating factor that drove our problems up more so than the 1 to 11.

I'll just say, hey, here's a little comparison where we've got our most recent year, the breakdown by those different buckets here. Below that is a little comparison from where we stood in 20-21. I'm going fast and furious here. I'm doing a lot of overgeneralization.

But we've had some discussions over the last year where we've tried to look at this in a little bit more detail. I've got to say, the most recent year, for the most part, just reinforced these basic points we've been talking about over the last year or two.

That is, when you're looking at these hours, that 0 hours bucket tends to be the more dominant factor. It's more dominant in that it tends to govern the overall percentage we have with less than 12.

It tends to also be more dominant in that the ebb and flow of good data performance and bad data performance at both the local and state level tends to be more manipulable-- I'm not sure if I'm phrasing it right-- if you govern the 0 hours bucket than it is 1 to 11. 1 to 11, you can control, but not as well usually as you can with 0-- with the 0 hours bucket.

And here's just repeating what I just said. The fluctuation tends to be governed by 0 hours, not 1 to 11, not so coincidentally. When those COVID years, when we had more problems with this, COVID tended to create a lot more with 0 hours.

Some would say that's a little counterintuitive with the idea that students show up and then COVID hits and then they don't, but really, I think the COVID issues at the agency level related to how well agencies are collecting this data, quite frankly, had a bigger impact on our COVID numbers than students appearing and disappearing.

The other thing-- and this, I think, is explained in other issues, but the 0 hours tends to be governed more by data reporting and clean-up issues. 1 to 11 tends to be affected more by actual student ebb and flow. But again, by cleaning up your data and focusing on that, that typically minimizes the number with 0, which, back to the first bullet, tends to keep the less than 12 hours numbers under control.

And then finally, the 1 to 11 hours really tended to stay pretty constant. It, of course, did bump up during the COVID years, but in much less-- much less noticeable than the 0 hours. And I'll just say again, 1 to 11 can be improved by data collection and cleanup, but gotta say, that 0 hours is what really gets helped when you do that extra cleanup.

So you're saying-- yeah, it could well be, yeah, that we're not sure which bucket that would affect more, probably the 0, but yes, I agree, Carla. OK, I'll say any questions or comments there? I am moving quickly because I really do want to turn it over to Thoibi here.

Actually, so I have one more thing I did want to bring up here, sorry, is that I wanted to relate it to some things we've been talking about at the state level. And this will be a good segue to Kobe's re-- or to Thoibi's research. I don't know why I said Kobe. Sorry about that.

Anyway, we've been talking about this concept called Student Pain Points. This is an NRS or federal-level term. This is the term the feds have been using to refer to issues at a typical school, whether it's in the recruitment phase or the registration phase or the enrollment or when they're in class or when they finish instruction.

That is, all the different ways in which students are touched by your school and looking at it every step of the way and trying to find out which are the steps that are painful for students. Sometimes it's just missing the bus and not knowing adult ed exists. Sometimes it's an overly cumbersome enrollment process. Sometimes it's an orientation. Sometimes all that's fine, but the problem is more once they're in class.

But a lot of the-- it varies from agency to agency, but no matter what, what is a common denominator all over is the need to focus on those first 12 hours. That is, when you're determining program class and level placement, when they're meeting their teacher and meeting the other students for the very first time, are they getting access to the teacher or a counselor?

Relating those goals to placement and so on. Those first 12 hours really tend to be the make-or-break time. Back to that fly a kite, if you can really double down a little bit in the first 12 hours, then maybe you can scale back after that a little bit once they're up and running. That is, once Charlie Brown has that kite flying up in the air.

So I'll use that as a good time for me to squelch myself a little bit and turn it over to Thoibi so she can take over and talk about the research.

Thoibi Rublaitus: There's some good chat going on, and I'm trying to respond to them as well. And yes, Jay, first of all, I wanted to say thank you for adding the hours of instruction and also having a bucket for the 0 hours because it's critical that the 0 hours are separated from the 1 to 11 hours.

And like Carla says, of course, there are times we have students enrolled and they're waiting to start a class. So they will still be on our list with the 0 hours, but they will not get any hours if they are not attending a class.

So thanks again, Jay, for a wonderful start with the data and how we get all that data and the good latest goal-setting CAEP data in comparison with the last three years also-- last four years also that really shows that persistence and performance go hand-in-hand.

But there was a slide in-between where Jay ends and I start with the cutesy little Charlie Brown video, but we don't-- we actually got rid of it.

So I wanted to ask everybody in the audience, before I get to my part, let's do a waterfall chat so that we get a little bit of engagement going, which is, what is performance and how is it measured in your own words? Just take a minute to write it in the chat so later we can all save the chat and also learn from each other.

Think about it. What is performance and how is it measured at your agencies? Ready to put it in the chat? Ready, set, go! Wonderful. It's all coming in. Thank you, thank you. Measured and level gains. Performance can be measured and measurable skills gains.

Jay Wright: Wow.

Thoibi Rublaitus: Progress in pre and post. CASAS testing. It is measured by goals. Students create their success.

Jay Wright: This really is a gathering of the faithful here. All these CASAS, these sort of answers.

Thoibi Rublaitus: Uh huh. Love it, right?

[laughter]

Is the result of various elements working together. I love that, Carla. All right. Thank you, everybody, for indulging a little bit on that. And so when we talk about performance, we're talking about setting goals and getting the students to meet their goals, et cetera, et cetera.

But can we have enough performance if we don't have persistence? That's the question, right? So persistence is important. We may be enrolling a lot of students, but if those students do not stay, they continue to leave because they are Charlie Brown, they are struggling to stay, or keep their kite up or keep their game up, then we keep losing, and it's also the reason why it's important to look at the 12 hours of instruction.

In addition to that, I have these-- enrollment is really important right now. After the COVID, all of us are struggling to increase our enrollment and that's our biggest objective. We enroll the students, but we cannot keep them. First, to get-- to enroll them is our goal. But then if we cannot keep the enrolled people, we're losing out again.

Because low enrollment means low payment points. But not just that. If you cannot keep the ones who came to us and we fail to retain them, we are losing trust in them-- their trust in us. It's important that we maintain their trust in us.

They came to us with a goal. They already thought, hey, this year I'm going to get my GED or I'm going to finish this, I'm going to finish that, or I'm going to get my high school diploma. They come to us, but if we cannot keep them, then somehow or the other, something that we did did not match their needs.

And so they have gone from us, and that is when we lose trust-- their trust in us. And so it also means it's a cost, because we have put an effort to bring them into our doors. They have somehow come to us, but they did not persist and they left.

And the reasons for persistence could be many. There could be lots of research we can do. We all have our hypothesis on why our students leave, and we still don't know what is the magic potion to give them so that they would stay once they come with us.

And ultimately, of course, we all as agencies, if we do not have those students to persist, we do not have the performance, and that also means a loss on the payment points for us to be able to continue to serve more people. This is for NRS, but even CAEP.

CAEP can also start looking to say, hey, let's see which schools are-- or which way we can get more students to complete what they came to us for. So these are big-picture things and this is why it's so important and critical for us to look into those students that come to our doors and leave, and I'll share more reasons why how many of them leaving and why so much are leaving.

I don't know all the reasons, but we're doing some research and some things, and so I just wanted to share that. Jay, the next slide, please. And the slides are a cue for me to remember what I have to talk about.

So I was very fortunate to have worked at Corona-Norco for many, many years as a part-time teacher, and while I was a part-time teacher, I got to do my admin credential. During my credential, I did a study on our school, the master's part where I did integration of technology in adult schools.

And what I learned from that little study I did was often, we have all these support systems with us, but we fail to communicate those support systems to all students that comes to our door. So what happens is they don't know what we have and they end up not taking advantage of the support systems that we provide them.

After I did that, I also wanted to study a little bit more on, do teachers and the teacher characteristics enhance learner persistence? Because persistence is such a big deal.

I had this hypothesis that there are certain teachers who are able to keep students longer in their classes because as I was teaching those 10 years, I noticed that some of my colleagues are better off at keeping the class full throughout the school year, where some of us were losing students like crazy. And then I wanted to see what type of teachers keep students longer in their classes because that persistence is so important.

But for my doctorate studies, I decided I would do something on that same thing, and that's not because I wanted to do it, because I was in a culture where we looked at numbers. I had a wonderful mentor and a director that really looked at numbers all the time.

And then my colleague, who was also working with me as an admin, we all continued to look at numbers and why students leave or why some classes are keeping all the 40 students we give to enroll in their classes where some classes are losing students in there. We give them 40 to begin with. At the end, with 12 or 15. And so we wanted to look at that.

But for my doctorate, I could not study our own school, so I ended up studying these six schools in the Inland Empire region. And in these six schools, these are a number of students in those classes. So the study I did was not just 1,000 students here and there. It's about 20,000 students in this region. So to the next slide, please.

So to learn and to look at persistence and the persistence score in these classes, what I did was I looked at-- the data was actually not from TE, but it was from ASAP, and all of us-- most of us use ASAP. And to collect the raw data, what I got was the Master's Report that I got from ASAP. That is called Class Enrollment and Total Attendance Hours and Home School by Time Period.

On my very last slide at the end, you will see exactly how to get to that, and you are welcome to look at that yourself because we look at that very often. It is very insightful.

So what I did was I looked at the total class-- student class days. So for example, we know, we are adult school, there is open entry, open exit, so not all students are coming in at the same time. All of our students come at different times of the year.

So, what we do is the first day the student comes to class-- and then so here, we are not looking at the ones who are 0 hours because if they are 0 hours, they won't even show up on that list. So we're looking at students the first day of attendance to the last day of the class, not the student, because some students come in the middle of the semester-- or middle of the session.

And used available class dates for the student. So if I look at the number of classes that are available from the start date to the end of the session of that course and how many days they stayed, that is what the persistence score-- this is how I looked at it.

The next slide will show us, like you said, originally-- let's hold off explaining this slide for a second here. Originally, my study was to look at the teachers' persistence scores and compare it to the teachers' personalities or their characteristics, and to see what type of teachers kept the students longer.

However, the biggest finding that I got is not truly about the teachers and the students, but more about the attrition rate in our schools. In those six schools that I studied, what I noticed is this chart that was just shocking.

And so if you see that, in the chart, you see from 0.0 to the 0.10-- or, say, 1 to 10, say, if a student were to stay 10 hours, the first hour, or at the 0 mark, you see that the bar is the tallest at the left end. That means that's the highest number of times when the students dropped.

So the students came into class within. The first few hours of class is when students already dropped out. And that's so high in the beginning. And that's when I started saying, looking at the first few hours of students and making sure that students don't drop out within that.

And if we can help them to sustain that first few hours of instruction, they will continue to stay later, and that whole Charlie Brown activity thing of, once-- getting the flat kite off the ground is the biggest challenge. Once they're up there, maybe-- some will drop for sure, we are adult schools. Adults have lives, things happen, and they leave, but that will make the biggest difference is my understanding out of this doctorate studies that I did.

And I also put a copy of it in the very end page. Feel free to read it. It's a lot of it, but don't worry about reading all that theory that came before the study, but feel free to read the last chapter and also the poster that I put there. poster gives you an entire picture of what-- my dissertation in a poster page.

So when I learned this, I thought, I could just keep it to myself or share it with the field so that all of us learn from this, and that is why this has become my big thing with Jay all the time. The 12-hour instruction, 12-hour instruction. Why 12-hour instruction? But that is because if they have persisted two weeks with us, we know that they will persist.

Now the next point is to study why do they drop out in the first week-- first two weeks more? And what can we do to help them to sustain and persist the first two weeks and more?

And so all of us-- I hear a lot of my colleagues, a few that I presented with the first two days ago with Jay-- thanks to Jay, I get introduced to all these lovely people who do a lot of wonderful work at their schools. And hearing things that they are doing at their school really encourages me to do more.

And then whatever we learn as a group together, all of us can share. This is not a matter of just keeping the school and the field afloat, it is to serve the community who really, really needs us.

And we all know as adults, I myself have gone through it, if you have to go back to school after being an adult, it is a huge challenge. You're just changing your mindset of that learning. Getting into a great mindset where you feel like, oh, this is tough, but I'm going to ride this out. Jay, can we go to the next slide, please?

So to me, persistence and performance is such a huge deal, and there, we need to have a lot of more deeper discussions on this. What happens when we have students leave us when they have less than 12 hours? Because sometimes-- this is just my thought, it's not theory or it's not-- it's just a theory, it's not a study. We haven't done anything on it.

But these are questions that we have to all work together to help, because the ones who persist, they already know they want something and they're there, but the ones who don't persist are the ones who need us most.

Can we gain performance from students who do not persist? I've already asked this before. What systems are in place at your agencies to reduce the attrition? That's the question. And how can analyzing the reasons why students drop us-- drop out and getting support accordingly.

And some of us are really doing good things at this. And I've heard a lot of good things. But there's one little study that we did in our school, and I just wanted to share the findings from that. Next slide, please, Jay. That's just a case study.

As you have heard, my school is not that big. It's a medium-sized school. We serve about-- last year we served about 1,500. Before the pandemic, about 2,800 to 3,000 students. The study that we did, on the next slide, you will see is--

So looking at all the reports that Jay showed us, we have done something like looking at the 13 hours of student-- hours students, less than 12-hour students, and the zero-hour students. And when we look at that, we see, if they have 13 or more hours, great. We have to still continue to help the students and serve them well.

If they are less than 12 hours, we give them-- if they dropped out, we give them the opportunity to re-enroll. If they did not drop out but they are still less than 12, I send out messages directly to the students. Sometimes we teachers call the students to check into them.

And if it is a zero-hour, it's an opportunity to re-enroll them. Like Carla said, though, of course, we're not teachers. And staffing, that's another issue. That's a big other discussion. Next slide, please.

Jay Wright: What I will bring up is Carla has a good question. Was there any breakdown in this related to instructional program?

Thoibi Rublaitus: Yes, yes. So we did a little market study. So the first step we did, we looked at it-- we got all the students with less than 12 hours' attendance, and then we took the students of 0 hours, separated from 1 to 11 hours.

We removed students with more than 12 hours in a different program and set a target to complete the survey with at least 30% of students from each category, so we separated them by the program as well.

The goal was to have a representative study of students with less than 12 hours across all programs. Next slide, please. And I was very fortunate-- or, well, Corona-Norco is very fortunate to have diverse talents. And one of our instructors and also our CTE coordinator has a marketing background, so she helped us put this little survey together. So we were going to do this market study.

But then, of course, for a study to be unbiased and very, very objective, we had to have a script. And that's when she helped us create the script, and the script which goes something like this. Hello, I'm so and so. And first we had to have those guidelines so that what we're doing is not biased and very, very clear objective. So we had created a script. Next page, please.

So we also learned-- and this is from another presentation I did last year around the same time, and I should say, that this is also a sequel to the presentation that I did-- Jay and I did last year on shine a spotlight on the 12 hours or more student-- less than 12-hour students.

And so from that discussion, from the crowdsource input we got, everybody's been saying, use your teachers, use your teachers. Teachers are the best interviewers. So we used some of our teachers, and using interviewing techniques for unbiased, qualitative information, we surveyed those 210 students. And the finding is on the next page-- slide, please.

So, the biggest finding is out of those 215 students we surveyed, 29% of them had changed or stopped coming because of work schedule. 20% of them had personal reasons. 13% had family obligations because around the time of COVID, so they were taking care of family members, et cetera. 10% had class availability or scheduling issues. And the rest, 28%, had other reasons.

So I will talk more about this and we will get to it. But just to give you a sense, those 28% of them are the areas where we never thought would be a problem. Where students are voicing their opinion. That's the part that is the most important part, because the others, we already know, these students have work schedules, so we can have a plan for it.

Personal reasons may come up. OK, that we cannot deal with so much. We're going to leave that aside. Family obligations are there, but also, how can we support our students? And we know a lot of our students, they have culture-- they come from cultures where education is not a big deal, especially as an adult. Why would you go to school? It's your children's time right now, it's not your time.

So those kind of things that we know we already-- so what we learned more was from those 28%. So Jay, if you don't mind hitting the link-- no, on the same page. Go back. The findings. It's a hyperlink. It should go to a Google Slide. Google Doc, I mean. So probably you are going to see it as a new page.

Jay Wright: Can you see the new page or do I need to stop sharing and go out? I'm not quite sure if you can see it or not or if it's set to the PowerPoint.

Thoibi Rublaitus: It's set to the PowerPoint. So maybe if you unshare, I can share my--

Jay Wright: It's all right. I can go to it. Here it is. I'll just switch. I think it should be showing now.

Thoibi Rublaitus: OK. So this page has more details. Feel free to read them more later. So I talked about the 221 students. We scheduled-- we called. And you can scroll, Jay. And then I already shared about this. So here are the reasons here.

The third column is so important. So those are some things that we uncovered. We had incorrect attendance recording. And so that is a school agency problem. So our problems are different, your problems may be different.

We also learned that some. Students were inappropriately early placed in that class. And there was no class information. So we got those-- and so this is a data cleaning up issue as well. Completers with 12 hours or less. Lack of teacher follow-through.

So there are some things that we can do something about, but there are some things we cannot do anything about. It's just a matter of data collection. And then there's the lack of teacher follow-up there. One of the students also said something about, we learned that it's a technical issue problem.

And then by technical issue, it's because the student was put in one class and then they were transferred to another class, and then when the data moved over, they still stayed in this bucket. And the class environment issues were there. Transportation, errors. The students were not fully registered, but it showed like they were registered or somehow something happened. So those were some things we found.

And in general, if you scroll down below, Jay, from these findings, what we learned was that 24 out of the 61 students, they were unable to attend due to the change in their work schedule. And so we know that we have morning classes and evening classes, but something that we learned also is that those students did not know that they could switch.

And then because it was returning after COVID, this was a study we did last year. So less than 1% of the students mentioned that the day care was a suggestion for improvement, and we have taken that into heart, and we started providing child care at two centers that we have now.

Almost a third, 28 of them, were non-persistent students are classified under the variety of issues. This is the same thing I've already explained in the third-- or last column in the previous table above. So we can scroll down. So I think I've shared all of this here already, I've discussed this.

We also learned that online, Edgenuity, was also an issue with some students. Class placement. Category, 33% was mostly from the ESL students expressing a need for more basic level, and we've already taken this to heart, and this year we are providing more lower-level classes that is level.

Partner locations have multiple-level classes-- multi-level classes, and that was also an issue with some students. And so there were a few students in the computer classes also who talked about their levels were either too high or too low, their placement was wrong, which also helped us to correct how we do our computer basic classes this year.

Lack of teacher follow-up. 13% students working remotely, self-paced, and unable to contact teachers. The lack of understanding of what to do next. So teachers were not following up on time. So this year, we've already put in an action in place where every week, students will check in with students-- or teachers will check in with the students who do not come to class, or make sure that every week, every student has a touchpoint in some way or the other.

So some action items we've taken are those. Then no class information. 10% includes students who were told staff would follow up and provide them virtual class information, but the students never received the information.

So there were gaps here and there that we found out, and all of these, we are trying to rectify some of those this year by putting some of these actions in place, by revamping our orientation to share common frequently asked questions and concerns.

So now we have all of our three different programs. All the orientation has the same look and feel. They all start with the same way of discussing the important items, and then we diversify a little bit with the program-specific information.

And we have all of these on our website as well for students to go and access it after the orientation date, because what happens is most of the time, we learned that our orientation was you come in, get this fire hose of information, and then the rest of the semester we expect them to remember what we said, which is not the case with any one of us.