(DESCRIPTION)
Text, Recording C A E P attendance hours. November 10, 2020. Jay Wright.

(SPEECH)
Everybody-- hopefully, everybody can hear me OK. I guess I'll hope that somebody can flag and say, yes, you can. I'm looking at the group. It's good to have everybody here. I will say it's a slightly different group than I was expecting. Although, maybe I'm reading you wrong.

So to be clear, we're going over how to record hours. And we're going to be talking about recording hours related to distance learning instruction, and so on. A lot of you have been to these. I'm sure a lot of these slides are going to look suspiciously familiar. And yes, that's on purpose, not an accident, I think.

But what I will say is this workshop started based on the need to provide this information related to hours for community colleges. So just curious, how many of you are from colleges? Maybe there's a head count.

I'm not sure if this is an Amanda Lee question, or just having the group through chat saying, yeah, I'm in a college. No, I'm in an adult school, or whatever. I'll just look. Most of you look like you're from adult schools, but it might be more mixed than I can tell from.

But anyway, there was some information that was big about a month ago on the college side, where there were new ways that you were going to be required to record attendance. Basically, you weren't going to be required to report things in a daily fashion anymore. That is if students attend, you don't have to record two hours on Monday, two hours on Wednesday, two hours on Friday for that 20 weeks. You just have to take a census a few times a year, and based on that census you do sporadically, that would determine your attendance for the whole year.

So there were a lot of big concerns that came out of that related to WIOA II reporting, and also CAEP reporting. So we had some focus groups. We did a couple focus group meetings there early to mid-October, so a little over, or about a month ago, I guess. And what we found out is the methods you are already working a lot on the college side did seem to fit pretty well with a lot-- or did seem to at least be very, very similar to the methods we've been talking about since March 13 related to distance learning. That is you're not recording daily, but you're using an account of your instruction and measuring students to come up with ballpark figures. I'm using my terminology loosely. Thank you, Amanda Lee and Holly for that. I see that. Thank you.

And we found it to be very similar. So that's where the idea of doing this webinar was hatched is I would kind of bring up, more or less, the same recording attendance hours for distance learning that I'll just say on my end I feel like I've done many, many, many, many times since the middle of March.

But basically, present the same information, but in the manner that, yeah, we may not be using it just for distance learning overall anymore. It might also specifically be a way for colleges to record attendance. So it looks like we've got maybe a third from colleges for that reason. 2/3 of you, I think, are here just because you want to recap on how to record DL hours. Either way, that's fine.

I don't want to scare anybody away. But I just wanted to give the etymology of this, and also just kind of see what I was working with. So all that. I guess there's not really anything to ask you other than is everybody ready to get started. Or out of all that, were there any concerns or questions that might have come up that I need to address before just diving in?

I'm not seeing anything, so I think that probably means get on with it, and dive in. Let's dive in. There you go. Taking a page right out of that playbook there. So I'll blow it up here. I can leave the chat up like that.

So here's the cover slide. The agenda, like I said, should really look kind of familiar to everybody.

(DESCRIPTION)
C A E P attendance hours. Covid 19's effect on adult education and end of year data reporting. Review of distance learning considerations. Creating classes. Recording instructional hours. C A E P outcomes. Resources. Attendance hours examples.

(SPEECH)
Again, here's the CAEP attendance hours agenda. Recap for some of you. I'm sure this will be the umpteenth time you've heard me recap quote COVID-19's effect on adult education. Surprise, surprise there. And then we'll kind of review some of those basic concepts.

So yeah, if you did a lot of those OTAN classes, webinars there March, April, May, a lot of that information will sound familiar. We'll talk about creating classes. A lot of what we address has been the same there, but we have kind of started talking about it from an inside-out point of view. More on that later.

We'll get into details on how to record instructional hours. Mostly that, too, will be a recap of what we talked a lot in the spring, a few twists, a few new things have come up, but mostly the same. And then, again, because it's a CAEP training, not just the usual distance learning.

There was also some interest in kind of bringing up outcomes for this webinar. So there's going to be about 10 minutes where I kind of zag a little bit and start talking about CAEP outcomes, and also start talking about CAEP resources. And looking at it, it sure kind of seems like we're just sort of zagging for no particular reason. I've got to admit.

So my backpedaling defense is that that was one of the specific requests out of those focus groups. The number one request was present distant recording instructional hours for distance learning, because that will help colleges record hours and stay in compliance for CAEP and stay in compliance for WIOA II without having to dramatically change anything they do.

Another request was let's recap some of these CAEP outcomes so we can incorporate that. In particular, I'll talk about some of the things we covered in the data dive trainings in June and July when we were looking at different ways to record outcomes, knowing that our students are in distance learning, and perhaps struggling with remote testing. So we were looking at alternative ways to record outcomes, basically so we can show all the great things we're doing in distance learning. A lot of you have said that's been difficult.

So anyway that third to the last bullet and second to the last bullet come out of the blue, I admit, but that was a request, so we're keeping it in. And then at the end I want to go over examples some of those might be relevant to colleges. Some of it may not. Most of it, I think, will be generic and equally relevant for everybody. So I kind of left it generic that way on purpose, so that should be good news, given most of us are not from colleges.

But that last bullet, I daresay, might be the most useful part of this, because it gets into brass tacks and provides real world examples of different DL-related scenarios and providing kind of a mish mash of some of the methods we bring up in this training, just different applications of these different methods.

A lot of you have said, yeah, you understand those NRS methods for recording distance learning hours, and that's all fine and dandy. But it really doesn't sink in to how you need to work with it with your students until you hear some concrete examples. So that's what we'll focus on on that last bullet.

So sorry I'm talking all over the map here, but that said, I really-- I want to know now more than in a typical training. Does that sound more or less like what you were expecting to get yourselves into this afternoon, yes or no? Nobody's answering. Uh-oh, that could be ominous there.

Belatedly, we've got a bunch of yeses. I don't see any nos yet, so I'll just hear what I want to hear, and move on, I guess, on that one.

(DESCRIPTION)
Covid 19's effect on C E A P agencies. With most agencies closed for Covid 19, many agencies are quickly adapting to implementing distance learning options. O Tan has a resource page that provides help to agencies responding to Covid 19. O T A N dot U S slash resources slash covid dash 19 dash field dash support. For E O Y reporting C A E P agencies should continue to follow the C A E P beginning the year letter for P Y 2019 2020.

(SPEECH)
So again, COVID-19's effect on CAEP agencies. This slide everybody's seen a million times, I think. Long story short, the long-- the short answer is yes. COVID-19 has had a dramatic effect on adult education. It has at least since March 13.

So I always like to make a plug for that OTAN COVID-19 web page. We've been talking about it lots, and lots, and lots since March 13. My guess is everybody's been there. Everybody's been there many times. It's definitely been the hub of the adult ed universe since March 13.

This last one, we've got a new begin-- sorry, this one, I should have updated. There's a new beginning of year letter for 2020-2021. It's the same feedback, though, is there's not really anything in terms of deliverables or whatever that has changed. A lot of the way you do business has, but from a bottom line deliverables [INTERPOSING VOICES].

Oh, I didn't bring the power cord.

You still have the same quarterly and end of year data. To say, all of those requirements are basically unchanged.

(DESCRIPTION)
Distance learning, O C T A E. On march 27, O C T A E disseminated Memorandum 20 3. Posted on Casas website, casas dot O R G slash social dash media dash newsroom slash 2020 slash 03 slash 27 slash casas dash testing dash during dash the dash covid dash 19 dash pandemic. Updated memos 20 4 april 17 2020 and 20 5 May 29 2020, www 2 dot E D dot G O V slash policy slash adulted slash G U I D slash memoranda dot H T M L.

(SPEECH)
Backing up, a little bit more on how COVID-19 has had a big effect. We've had some federal memos. We made a really big deal about this in the spring, because they came up with not one, not two, but three federal memos pretty much right away. One in March, one in April, one in May when we started this COVID-19 craziness.

I've got to say, since the end of May, they have not come up with any updates. I really kind of feel like they're laying low. They're not really going to weigh in on anything until we're a lot closer to the end of the year is my guess, but in any case, there's those three memos that they had right away. Nothing really to update since May 20 at the federal level.

(DESCRIPTION)
Distance learning, O C T A E. Summary of memos. Statewide performance reporting to the N R S is stil required. For P Y 2019 2020 all states must report force majeure. States can now decide whether or not to allow remote testing. each state that opts to allow remote testing submits its own remote testing plan for O C T A E. Memo 20 5, 5.29, participants can self report into a federal E F L without a pretest. Memo 20 5 5.29, a pretests is still required to achieve a pre post test M S G.

(SPEECH)
So just to recap exactly what that was in the spring, one thing that's worth bringing up is remote testing. One story short is remote testing was just flat out not allowed before all this COVID-19 stuff hit. The feds had always required that all pre- and post-testing happened in person, proctor present. That was the big thing that these OCTAE memos changed, as you might say they, quote, unquote, "legalized remote testing." So for all of us, CASAS included, everything related to remote testing is 125% new.

Another big thing from last year is it had that force majeure, which means unable to perform or unable to test. To be clear, that was a big deal in 2019-2020. It matters like not at all for 2020-2021.

I know a lot of people are kind of hoping that we have another force majeure this year. It's possible that we might, but I can say with 100% certainty there is definitely no federal guidelines for any sort of force majeure this year. For this year, all the enrollment and hours requirements are business as usual. All the pre- and post-test requirements, business as usual. All of those sorts of things are pretty much the same requirements as always.

I will link into that last federal memo that came out at the end of May where they allowed for self-reported placement. That is you can place students for federal reporting without doing that pre-test So the feds are kind of acknowledging that getting everybody pre-tested right away and getting all that done in an organized fashion is pretty close to impossible. So what they're saying in that memo is, yes, pre- and post-testing, absolutely still required. But we know that it's going to be very difficult to do it right away like you normally do, so you can use informal means of placement so you can get the student in the appropriate program and the appropriate class and so on. We understand it might take a while before you can actually get that person with a valid pre-test.

(DESCRIPTION)
Distance learning, N R S. Distance education, formal learning activity where students and instructors are separated by geography, time, or both for the majority of the instructional period. Distance learning materials are delivered through a variety of media including but not limited to print, audio recording, videotape, broadcasts, computer software, web-based programs, and other online technology. Teachers support distance learners through communication by mail, telephone, email or online technologies and software. N R S web dot O R G slash sites slash default slash files slash N R S dash T A dash Guide 82019 dot P D F.

(SPEECH)
So I'm kind of out of the COVID-19 section. Everybody hanging in there. Are we still all right here, hopefully? Sorry. Transitioning out. Thank you. Great, great, great.

So here's another slide that I'm betting a lot of you have seen a bunch of times, the official NRS definition of distance learning. That is formal learning activity where students and instructors are separated by geography, separated by time, or both. Lots of fine print, lots of-- can everybody hear me? Hopefully. I got one of those internet connection things.

I'm a little worried. I'm all plugged in. It shouldn't be doing that at all. Anyway, so with distance learning, here's all the fine print. There's obviously lots of ways in which you can deliver distance learning.

Bottom line, if students and instructors are separated by geography, time, or both, well, then by definition it's distance learning. The federal definition for this is very broad. If you need proof in the pudding, there is a link to the federal website that gives you the page number and the link for exactly where that's listed.

(DESCRIPTION)
Individiuals in distance learning programs have the same data collection requirements as other W I OA title 2 learners. N R S policy prescribes the 50% rule. For learners with more than 50%, agencies should mark special programs equals distance learning. A new feature is now in T E that tracks students with P Y 2019 2020 activities affected by Covid 19, force majeure.

(SPEECH)
So long story short, when you're looking at NRS federal reporting, you know what the feds have said all along is individuals in distance learning have the same requirements as everybody else, the same enrollment requirements, the same hours of instruction requirements, the same pre- and post-testing requirements, and so on.

The little wrinkle they've had for about 10 years for distance learning students is what I like to call the 50% rule. That is if the student is more than 50% distance learning, you need to mark them as such in your data for federal reporting. If they're less than 50%, then you need to do nothing.

So just a little more on that, because it's come up more now that we've been talking about the 50% rule for a while, is this came out of a lot of federal monitoring 10 or 12 years ago, around-- sorry, too much history, but I'll just say around '06, '07, or so, distance learning was becoming very popular. It was the next big thing. Everybody was doing it.

But in most cases, everybody was doing it where it was maybe 10% or 20% of what the student was doing. But everybody was still being marked distance learning. That kind of rubbed the feds the wrong way. They didn't really want to could everybody that was just doing 10%, 15%, 20%. They really intended that to be just for the people who were doing all distance learning, or at least mostly distance learning.

So they came up with the 50% rule. They kept it simple. More than 50% means they are distance learning. Less than 50% means they're not distance learning. The way we've done that in TE all along is there is a field in special programs called Distance Learning that you can mark at either the class or the individual student level to mark that they're more than 50% distance learning.

That new feature is not really there now. But in '19-'20, we had that force majeure to track students that might have been affected. Some of you are saying you're using that informally now. To be clear, if you use that informally, that's fine.

But just know if you mark that, there's 125% chance that will accomplish nothing. There's nothing we're doing at the state or federal level with that. So if you do have reasons to mark it and you are looking to flag students that might be having some kind of COVID-19 related troubles, so they're not meeting certain requirements because of that, by all means, that might be a way to informally mark it. You can check the box. You can use the TE Listers to do some of that bean counting and so on.

But to be clear, it's not helping you for state and federal reporting, because that's like so 2019-20. Force majeure, at least for the moment in this program year, absolutely does not exist. But again, mark distance learning.

The other thing I'll bring up is that I'm not going to dig in deep into it in this workshop, but there are lots of newer features in TE where you can now mark all the distance learning hours as distance learning, and all the regular hours as regular hours. There's lots of PowerPoints and guidance documents on how to go into TE and make that work correctly for you. But you can now record that to the n-th degree if you want, where you can work all the distance learning hours one way, and mark all the, quote, unquote, "regular hours" another. Your choice on that.

(DESCRIPTION)
Distance learning, class strategies. Calculating this at the class level for each agency depends on how your agency has structured classes in response to Covid 19. 1, pick up where you left off. 2, start new classes specific to D L.

(SPEECH)
So moving into some of the meat here, the three big things we talked a lot about in the spring. And again, it's not going to be word for word the same, but generally, it's going to be a lot like we talked about in the spring. The three big issues that came up right away-- class creation, recording instructional hours, and assessment. We'll be talking about those first two a lot. Assessment has really changed a lot since we did those March, April workshops, but those were the big three areas where everybody was having trouble.

So we'll start with classes, where this is a blast from the past. We talked about this ad nauseum in March and April, where, what do we do now that we're in COVID-19? What do we do with our classes now that the schools have closed and we've changed everything around? So in March, April that, of course, meant everything closed. So there were no in-person classes. 100% was moved to distance learning.

So what do we do now that we don't have anything in person and everything's distance learning? We basically gave you two solutions. We made these names up. One is called "pick up where you left off." The other is called "start brand new classes specific to distance learning."

(DESCRIPTION)
Pick up where you left off. If your agency is continuing classes as scheduled but just changing the delivery method to a distance learning context because of Covid 19, then calculate whether or not 50% plus of the instruction overall is in a distance learning format.

(SPEECH)
So "pick up where you left off" is suggesting-- and it was a little bit more for there in March, April when we were transitioning to COVID, but I do think it might be relevant here as we get into 2021, where if you're keeping the classes the same-- so last March, April, that meant, hey, between, say, August 2019 to March 2020, you are obviously offering all of your instruction in person, so thereby you just created classes that usual way.

But after March 13, everything hit the fan. Everything moved to distance learning. Suddenly everybody was distance learning. So this is the suggestion we gave you for if you're keeping your basic structure in place, where you're still meeting with ESL intermediate low Tuesday, Thursday mornings at 10:00 AM. You're still meeting for two hours a day on Tuesday and Thursday. Title is the same. Teacher is the same. Students are the same.

The only difference is they're basically having everybody come in and meet on Zoom, rather than the regular classroom. That's a good example of "pick up where you left off." Most things are exactly the same. So we're basically just going to keep everything and record hours, but we just need to look at it carefully and figure out what date it was that that changed from regular classroom to distance learning. Really, no muss, no fuss, no changes to our classes at all. That's the good news.

But the bad news then is that we need to put more effort into making sure which students were more than 50% and which were less than 50% so we can mark everybody accurately in terms of that rule for distance learning.

(DESCRIPTION)
Start new classes specific to D L. If you are now offering distance learning and starting this activity as new classes, with a new group of students, then create the new class in T E with the new start date. You may want to optionally label these with a specific title to distinguish them from other classes. For these newly created classes, mark special programs distance learning. For the classes that existed prior to Covid 19, leave them as regular not distance learning classes.

(SPEECH)
"Start new classes specific to distance learning," we brought that one up to say, hey, yes, we moved everything to distance learning. Some of you said that was just way too big of a seismic shift to simply pick up where you left off. The teachers are different. The groups of students are different. Some of the titles change, whatever. So it's a lot better just to create classes new.

So the bad news, of course, is then you need to go into TE, create a new class instance record. If you're using some other system, create the new classes in that some other system. But it did mean you needed to create new classes.

It made it easier for calculating the 50% rule, because this way you could just mark the old classes that were in the regular classroom as not distance learning. All the new stuff you can mark exclusively as distance learning.

Now that we're in 2021, my guess is that you're about 90% something, maybe 99% this method rather than the old one, because my guess is just about all of you are all distance learning. I guess I'll just say. Any of you at all, just curious-- have any of you been offering any sort of in-person instruction yet in the 2021 program? Has that happened with any of you at all? One class. Mostly no. A few one classes.

I guess for now we're just looking at hours of instruction here to bring it up for-- yeah, I know a lot of you have done-- gotten some people in for that. I guess for this purpose, I'll say no, but not that big a deal.

Straw poll says you're about 80% no. 20% of you say yes. Of those of you that say yes, half of you are saying special cases like testing. Half of you, it sounds like, really and truly have done something that's genuinely in person.

So for most of you, I would say that means start brand new classes is where you want to go here, because you don't really have anything that's not DL yet. So that is you've created new classes there in July, August. And you've been 100% distance learning ever since. So it's kind of irrelevant.

So again, if it's all DL, you can just create a new class. Give it a different title. And then, of course, because it's a new class that's now all DL, for all of those, you would mark special programs, distance learning.

Again, back in the old-- from last spring, you might have had some that were all in person that you went out of your way to mark not distance learning. What I'll just say is maybe for a few of you this is relevant, or if you started July 1, and obviously then we were pretty much all 100% DL, if some of you have shifted some classes out of DL back into regular classroom, you might say this decision is the same decision, but just an inside out decision where you still have the same decision.

But if it's "pick up where you left off," it's the opposite, where for starters you're 100% distance learning. Now you're looking at some that might be doing in person and doing it a little bit differently. So you have that same issue where you need to do a little extra work, calculating the 50% rule. But instead of itemizing out the distance learning, you're kind of itemizing out the, quote, unquote, "regular instruction" in this case, because the regular instruction is probably going to be the exception now, not the rule.

So I've been babbling pretty much nonstop for 10 or 20 minutes. So I feel compelled I need to be quiet for a minute. Is everybody following this? Is this not making any sense at all? Sanity check is for everybody, but especially me. Is this what you were expecting to hear, hopefully? OK. I'm getting all yeses so far. So at worst, everybody's still feeling compelled to humor me. OK. Great.

(DESCRIPTION)
Distance learning N R S. 3 models of measuring distance learning instruction. 1, clock time, assigns contact hours based on the elapsed time that a participant is connected to or engaged in an online or stand alone software program that tracks time. 2, teacher verification. Assigns a fixed number of hours for each assignment based on teacher determination of the extent to which a participant engaged in the assignment. 3, learner mastery, assigns a fixed number of hours of credit based on the participant completing content of each lesson.

(SPEECH)
So that's the class creation issue. So the other big one is recording instructional hours. So here's another slide most of you have seen many times. That is the three methods of measuring distance learning instruction from a federal NRS point of view-- clock time, teacher verification, learner mastery. Clock time just means an hour of distance learning is exactly equal to an hour of face-to-face instruction. That is your recording hours the same way you would the clock. That is recording hours the same way you would for any sort of in-person instruction.

So tying it to that community college issue, some of you on the college side are saying you're going to need to do this for in-person instruction. So again, this might be the way you do it, but you might have other ways of angling it.

We also have teacher verification and learner mastery. That's looking at recording DL instruction a little bit differently, where teacher verification is basically having teachers watch students while they're doing distance learning, and use their professional judgment to assign hours. Learner mastery presumes having the teacher watch everybody is going to be impossible, so you kind of need to set it up front and say, here is the instruction we're providing via distance learning. Here are what our expectations are for these students and these particular classes. And here's how many hours we think this work is worth.

So you're basically putting all that DL activity in writing, and then recording it as an agency in writing so it's all on the up and up. And you can apply it the same to all students. And then when students go through distance learning and they complete all of the required instruction, you can then give them a fixed amount of hours of instruction based on the fact that they completed the lesson, they completed the requirements, and so on.

So Dale's question, going back to class creation, that's what it sounds like, I think. Sorry, I'm going to get out of presentation mode. I'm hoping I've got a couple ex-- I guess these are all in the examples at the end.

But in the examples at the end, I do have some of these examples that will hopefully make it clear. But yes, I would say so. Sorry I'm sounding like a broken record. It'll be a little different from what we talked about last March and April and so on, where then everything was all in person, and now all of a sudden where we've gone from 100% in person to 100% distance learning. So we just need to be aware of that. The same sort of mindfulness is required, but, of course, in your example, Dale, we'll be starting with 100% DL and moving the other way around. We're just sort of going in the opposite direction, as we were in '19-'20. Hopefully, that makes sense.

So if you have both in-person and Zoom classes, you'll have different instructional hours regular and DL. I have a couple slides of this. I don't have time to get in the weeds. Quite frankly, I want to avoid getting in the weeds, but I do have a few slides that I want to show you where if you want, you can record hours in TE. You can record DL hours as DL hours and regular hours as regular hours. That's not required.

What's required, again, is simply to calculate the 50% rule. That said, more and more are looking at recording distance learning hours, because that does seem to help.

Yeah. I would say that if you actually-- I mean, it would be picked up where you left off. Again, let's just say you started a DL class on August 1. We'll call it ESL intermediate low. We'll just-- I don't see it. I'm looking to see. I don't see a date, Dale. So I'm making the date up, but I'll just say January 1. You're now taking the same class as you did now, but now on January 1 you're going to have the same class, but it's going to be offered at your school, rather than online.

So if it's the same title, same group of students, same teacher, and so on, still meeting Tuesday and Thursday morning, 10:00 AM, but the only difference is now in January you'll be meeting back at the school again instead of online, like you've done here in the fall, then, yes, it's take up where you left off.

And so, Kathleen, I think your question is pretty similar as Dale, but you're just giving a more specific example. Thank you, Dale.

Kathleen, are you kind of the same as Dale? Or maybe I'm reading you incorrectly. I don't see anything there, so I guess I'll just pull up and move on. But these are the three ways to measure instructional hours using distance learning.

So a little bit more about what we do.

(DESCRIPTION)
California has historically used the proxy learner mastery model to document hours in distance learning programs. The student is credited hours of instruction for completing an assignment such as an instructional video or educational software lesson rather than a set block of clock time. Agencies can vary their approach to hours based on specific delivery model.

(SPEECH)
In California, we've historically used the proxy learner mastery model. That is we set it up. We figure out how many hours our distance learning instruction entails. We put it in writing at the agency level so the student knows upfront how many hours that entails. And then once the student completes the required instruction, we then record that number of hours for each student.

We've always preferred that model in California, because mostly we've been using means that isn't really conducive to clock time. We've always preferred learner mastery to teacher verification, because teacher verification requires that that teacher watches every student while the student is completing the DL instruction. In most cases, we think that's probably in the ridiculously unrealistic category that puts way too much burden on the teacher. So for that reason, we've always favored learner mastery.

To be clear, you can use any one of those three models. To be clear, you could use any combination of those two models. The answer is the same. And we'll get more into these examples in the end. Obviously, I'd better make time for these examples now. I've referenced a bunch already.

But what I'm trying to point out is you just need to clearly mark what it is they're doing. And then once they complete that, record that number of hours for each student. What we've said is if you use some wrinkle, you've got your own system that works better than any of these cookie cutter models, we're pretty flexible in saying, sure, use any combination you like, as long as you document that in your local agency, or your local agency assessment policy. And then kind of follow through and make sure you implement that with all students in the same way.

That's what I mean by that third bullet. You can vary your approach based on delivery model. You can use all kinds of combinations. The trick is you just need to be very mindful and upfront about what it is you're doing. Document it. Put it in writing, et cetera. And then apply what you put in writing for each student. It's very flexible as long as you're very upfront on how you're recording it.

The issue is-- and I'm sure none of you are doing this anyway, but the issue is if you don't document it, if it's not put in writing anywhere, it does definitely appear to be a kind of a mystery science. We don't want mystery science in any way, shape, or form. We want it to be all on the up and up. Everything needs to be clear on what you're doing. If it looks mysterious, well, by definition, it also looks 125% suspicious.

So you can use any combination, but you just have to do a good job documenting it so there's no way anybody will perceive it as looking at all suspicious. Again, I hope that makes sense as well.

(DESCRIPTION)
Distance learning, hours of instruction. Facilitated online instruction can be recorded the same as regular instruction, clock time. For educational software applications the recommendation is to pre-assign a number of hours to each lesson or module, learner mastery. There are 3 general approaches useful for determining proxy hours from educational software.

(SPEECH)
So some general considerations here. This is what we've been saying a lot here over the last six months. If you're doing facilitated online instruction, which is basically what you also might call synchronous online instruction, then we're generally recommending clock time. That is if it's facilitated or synchronous, we feel like that's the same as in-person instruction.

So again, by synchronous, that means that it's distance learning. So I should have put this little grid here. Sorry for that. But there has been some confusion between distance learning and regular instruction and comparing that to synchronous and asynchronous.

So I'm going to step out here for a minute. When I use those two terms, synchronous and asynchronous, does everybody generally know what I'm talking about, yes or no?

Sorry. I'm catching myself here and realizing that I'm getting into a lot of jargon that I'm not defining. So you're saying you know it.

But anyway, both of those can be considered distance learning. So that is a big part of my point. I've heard of lots of people mix that up into thinking everything that's asynchronous is DL and everything that's synchronous, that's not DL. That's completely wrong. Both synchronous and asynchronous instruction can be DL.

So synchronous is facilitated online instruction. So that would be simply meeting Tuesday morning at 10:00 AM via Zoom. The teacher and all the students are all meeting on Zoom at the same time, but nonetheless, it's still distance learning, because you're still separated by geography.

So synchronous you might say is, yeah, you're separated by geography, but you're not separated by time. Asynchronous would be accessing that recording some time later, or getting on at Burlington at a different time from the teacher or other students. That would be asynchronous. That would be when you're separated by both time and geography.

So if it's facilitated or synchronous, everybody's there at the same time. So there's really no reason to use anything other than clock time. You can use the other methods if you want, but you're just-- in my opinion, you're creating a lot of extra work for yourself. It's much easier to use clock time.

If it's asynchronous, though, you're using video checkout. You're using educational software. The other example I gave you where, yeah, when you do Zoom, inevitably a few students are absent.

So they have to go back and access that recording some time later. So in that example, obviously, some students are synchronous, because they were there with the teacher when the instruction was delivered. Some are asynchronous, because they had to go back later to catch up on what they missed. So use clock time for synchronous, learner mastery for asynchronous.

And then we've detailed this a little bit more. For educational software, we generally say learner mastery, but there is a little bit of a mixture with that.

(DESCRIPTION)
What if the facilitated instruction received online is recorded and a student watches the recording instead of the live presentation? Agencies can decide this as a team whether or not to allow this as hours of instruction. If allowed then assign the time that equates with the time assigned to the live presentation. Mark distance learning in T E for those over 50% D L. Whatever the agency decision record this policy in detail in the local assessment policy.

(SPEECH)
Sorry. I should have brought that one up after I brought this one up. This was a big question. I think everybody figured this out.

What if the facilitated instruction was recorded. So first off, I think most of you now are counting the time the student spends accessing recordings as hours of instruction, but I'll bring up some of you say pretty resolutely that that's a bad idea. If so, that's fine. You can not record those hours if you don't think it's appropriate. But if you do allow it, then just make sure that the time you give the student is equal to the amount of time the students did with the facilitated session.

So the example is let's just say you did a two-hour ESL class Tuesday morning at 10:00. We'll just say everybody made it but one student. The one student, as is, took three hours to get through that recording because of interruptions and so on. What we're saying with learner mastery is we want to be sure that that student is recorded with two hours, even though it took the student three. That is we're not going to wrongfully give that student an extra hour of instruction just because they accessed it via Zoom.

If that student took three hours when everybody else took two, that suggests that the student was just interrupted with stuff. They might have bird walked on the internet, whatever, whatever, whatever. Not a big problem. But again, we need to up front suggest that we're giving that time that the recording took, and not the amount of time that the student took. That's part of what we're talking about by learner mastery, where we're just putting that in writing ahead of time so it's clear to us, it's clear to our teachers, and it's clear to our students exactly what we want to do in all those different situations.

(DESCRIPTION)
How do we record hours of instruction accurately when students complete lessons using educational software, examples Odysseyware, Aztec, and Burlington english. The most common technique to document instructional hours is to tabulate proxy hours assigned for learner completion of each module using learner mastery. Contact your software publisher for recommendations and best practices.

(SPEECH)
And then here is some information about software. Sorry. I had a couple of slides out of order there.

So if you're using Odysseyware, Aztec, Burlington, et cetera, the primary way is to do that learner mastery, again, just like we were talking about with Zoom recordings, where we're looking at how long it takes a typical student to complete that unit of Aztec, or that module of Burlington, or whatever. So you're looking at saying, chapter 1 in this as Aztec module takes an hour. Chapter 2 takes 1 and 1/2 hours. Chapter 3 also takes 1 and 1/2 hours. Whatever. Put it in writing, and just equate that number that you indicated in your policy. And just give all students that uniform number when they complete that unit or module.

The other thing we always say is always contact your software publisher, whether they have things built in that help you, or they don't. You know we're saying always contact them, because they should know better than anyone else exactly what methods you should use, or give you some tips maybe on how many hours to grant, and that sort of stuff.

I think, Leanne, you're talking about-- yes, you are. Sorry, I'm sure I have it, but I wasn't sure if it's the next slide. So conveniently enough, Leanne, here's the slide that I did want to bring up.

(DESCRIPTION)
Distance learning, 3 approaches to proxy hours. 1, with software applications that include a time pre-programmed for each lesson or module use the hours recorded from the software. 2, for applications not linked to a specific time frame, but provide a recommended time for student completion, follow the educational software publisher's recommendation. 3, if the software publisher does not provide recommended times for completion, meet as an instructional management team, determine the amount of time students spend on each module and document that in your agency's local assessment policy.

(SPEECH)
I think what you're talking about is number one on this slide is it gets some software applications, I like to say, are, quote, unquote, "higher end," where they might recording hours built into the software. Other high end features might be they might have some lessons or quizzes built in at the end for the students to pass before they can move to the next module.

If they have that, we strongly encourage you to use both features. Again, the publisher knows better than everybody else what it really takes to get through their modules. So definitely put their work to good use, and make it easier on yourself.

So what I would say is, technically, yeah, I would call that clock time. I think there's a couple examples at the end as well where sometimes they automatically record it. So if the student gets on the module and the software automatically records 45 minutes for that student, take what the software gives you. Record 45 minutes, because the software says so. By all means, if your application is doing that for you, definitely just use clock time and record that.

I don't think there's a good import/export for that sort of stuff, so I'll add you do just pretty much need to manually record those numbers into TE so you get the hours right. I don't think there is an import/export with that, but absolutely it still nonetheless saves you from having to do all that extra work in your assessment policy.

So the next one might be not linked to any kind of programming. So in this second example, maybe it's not higher end. It's not built into that software programming, but sometimes if you look at the instruction manual or go to the software publisher website, or whatever, there will be some recommendations there. So that's the second easiest way, where it's not built into programming. But at least you don't need to get together as a team and figure it out yourself.

Just use what your software publisher suggests. And that way you don't need to put in that extra work to determine it. A lot of times when you get the software, it'll just say, hey, you're ready to start chapter number 3 or whatever. And a lot of times they'll give you a recommended time at the beginning of that lesson. If so, just use what they give you there as well.

Then number three, that's if they don't give you any of this stuff. That's when you've got to figure it out on your own. So that's when we're saying, yeah, that's when you need to meet as an instructional management team, determine the amount of time it takes a, quote, unquote, "average student" to complete each module, and then document that in your Local Assessment Policy.

(DESCRIPTION)
General guidelines for D L hours. Contact your instructional software publisher for guidance on hours. If the application does not embed hours related information into the software the vendor may have information that may guide you in determining these totals. The recommended hours you record for each student should reflect an average number of hours that the instructional team agrees on and should not be determined by only 1 person.

(SPEECH)
So similar guidelines, but not directly related to software. A bunch of things here, well, this first one does, I guess. Even if you don't have any of this information in your software, we still think it's a really good idea to contact them. A lot of times they might be able to give you recommendations, even though they don't publish it on their website. Lots of good little nuggets they can probably give you. Before you get busy and burden yourself with all that extra work, definitely good to check in with them and see if they can maybe give you a little boost and let you cut down on some of that work you do for this.

The second bullet relates to-- a lot of you rightfully brought up, good grief, how on Earth are we going to get a management team together? That sounds like crazy talk. And it probably is.

So what we mean by that is we know you're not going to get a big team, but do try-- do get it to where it's more than one person. What we mean by team is avoid letting one person unilaterally determine this. At least have it be a two-person team as a sanity check so you're not having one person determine everything. So do at least have people kind of sanity check each other before coming up with the final answer.

(DESCRIPTION)
If your agency prefers to assign hours on an individual student basis and not use the proxy hours method that is permissible. Explicitly state that your agency is using this method in your local assessment policy. To assign hours by student more vigilant observation from instructional staff is needed to ensure an accurate amount of instruction is assigned to each student and reflects the level of effort each student gives during each instructional session.More detailed description of this process is needed in the agency's local assessment policy.

(SPEECH)
The next bullet is if you want to use teacher verification rather than learner mastery. Like I said earlier, that's 100% A-OK. But be sure to state that in your local policy, that you're using that second method instead of the third method. The second bullet on this slide is why we prefer learner mastery.

But if you're doing teacher verification, that definitely means more vigilant observation is required from the teacher. It's suggesting that the teacher has observed every single student doing that DL instruction and coming up with that determination and assigning hours. Again, we think that's pretty darn unrealistic.

If you're able to do it, fine, do it. That's great. But again, in most cases, we don't think that teacher is going to be able to watch everybody doing all these activities. We think you're going to need to do learner mastery, because it's way too much work for the teacher to observe everybody.

And then, again, describe the process in your Local Assessment Policy. That's the part that definitely is covered with those examples. I did just do a copy and paste from that Local Assessment Policy workshop I've done a few times that just gives you some details on how to document it. Again, I've built it up way too much to get out of it now, so I'll definitely make time.

(DESCRIPTION)
Recording in T E. T E now has the capability to record exactly which hours are received via distance learning versus which hours are received in a regular classroom. This enables agencies to track and specify distance learning hours versus hours from the regular classroom. Instructions for recording distance learning hours in T E are posted on the same webpage as the recent F A Q.

(SPEECH)
So here's a little bit on the TE. So actually, I'm shifting gears here a little bit. Sanity check. Everybody hanging in there? Everybody feeling like they're hearing what they expected, and so on? Sorry. It seems like it's time for a sanity check. OK. Thank you, Jackie. All right, Dean. OK, it looks like everybody else rests the case. Still saying great. All right. Thank you.

So here's a little TE. Again, we don't have time to dig in the weeds. This tends to be a little bit different based on whether you're bringing it from another system or doing it directly in TE anyway.

But I just want to point out there is the ability in TE to record exactly which hours are DL hours versus which ones are regular classroom. So far this is not a big deal, but once we get into early 2021 and some of you are transitioning back, this might be kind of a big deal again. So again, you're just specifying distance learning versus regular classroom.

There's a bunch of instructions. We've got instructions on our TE Help page, instructions on that OTAN COVID-19 website, and so on.

(DESCRIPTION)
Recording in T E. Tops pro enterprise, T E, has the capability to record exactly number of hours received via distance learning versus hours received face to face in a regular classroom. And if you pre-define the default number of minutes per day for D L instruction T E will preload the attendance input screen. Pick a class instance, edit, intensity. 1, D L interface setting, yes no default. 2, distance learning duration in minutes, fill in the value.

(SPEECH)
So here again this is a little bit more in the weeds here where you're just redefining that default number of minutes or hours in the day.

Maybe it's, hey, we've got Monday and Friday. We do 1 hour distance learning and 1 hour regular on both of those days. Maybe it's that Monday is the regular classroom day and Friday is the distance learning day. I'm just making this up.

But there's lots of ways to just predefine that in your TE class instance record to where by predefining it, TE will divvy it up accordingly. So if you predefine all the Mondays are regular classroom and all the Fridays are DL, whenever that student shows positive attendance, all of the Monday hours will show up as regular classroom, and all of the Friday hours will show up as distance learning in that example.

If you're doing an hour of each each day, then whether they attend on Monday or Friday, in my example, it will give the student an hour of regular classroom and an hour of distance learning, regardless of day. It won't worry about what day it is, but it will split it up based on DL and regular.

So here's some TE directions where you go into the class instance record and you edit the intensity field, where you can basically look at the interface setting. And you can fill in how many minutes are DL, and so on.

(DESCRIPTION)
Screenshot of T E class instances list. Text, let's setup class 8101 for easier data entry of D L attendance.

(SPEECH)
Here's another screenshot where you can go ahead and open up that class instance. You're basically just finding-- in this case, we're finding a class that we already created, and we're setting it up so we can record DL attendance.

Here's that collect DL attendance, where, by default, it will be no.

(DESCRIPTION)
Screenshot options page, collect distance learning attendance, yes. Distance learning duration in minutes, 120. Text, specify the default minutes of D L instruction, and set the D L attendance flag to yes. click save.

(SPEECH)
You just need to change it to yes. And then you need to put those minutes in there that are total for distance learning. Again, say yes. Click Save. Input those minutes. These are the simple directions.

(DESCRIPTION)
Attendance, source daily attendance. Row, Distance learning attendance sorted by minutes per day.

(SPEECH)
So now here's where you're going to input attendance, where here we're basically going to these class records. So here is a record that we set up when we started talking about this late March, early April, where we're basically specifying which days of the week. So in this example, we're saying it's a Tuesday, Thursday class. It's 120 minutes on Tuesday and 120 minutes on Thursday. And we're inputting the percentage by class or by day of the week.

We can get as granular as we want where we're inputting the minutes. In this case, we're saying it's all distance learning. So all 120 on Tuesday are going to be distance learning. All 120 on Thursday are also going to be distance learning. So you can split it up.

To Dale and Kathleen's question, if it was starting this January. And it was going to be a 50-50 split, maybe we've got 120 hours of instructional attendance, but only 60 of those will be distance learning. So again, you need to know the correct ratio before getting into this. But as long as you do, you can preset those classes. And then they'll record those hours by regular DL recording.

So if you've got regular students-- sorry.

I was just going to read the question to you if you'd like.

OK. At the same time-- I guess what I would say is if the ultimate percentage is the same, then that probably doesn't matter. So let's just go to my example, and say it's the class on Tuesday, March 31 is 120 minutes, but in this example, we'll say it's 60 minutes of regular and 60 minutes of DL.

If at the end of the day, everybody receives the same amount, or maybe it's what I like to call a port and starboard routine-- don't worry what that is-- but where half the students are in DL the first hour, and then you just flip-flop it so the other half are in DL the second hour, but at the end of the day, everybody is 50% DL and 50% regular classroom in that example, it probably doesn't matter that you're doing half and half, and flip-flop.

If, on the other hand, the total amount of DL time and the total amount of regular classroom time is different, then you probably need to do it at the student level. And I'll just show you you can do student by student here.

So the easy way would be to mark it for the whole class there at the top where you know it's 50% DL and 50% regular, and just mark it in those top rows at the class level so all the students that attend that class will be treated the same way. You just click that Fill button, and that will fill everybody up the same way.

If student 1 test is doing things a little bit differently than student 2 test is, though, and at the end of the day some of the students are 50% distance learning, then you might want to record that student by student. Then you would just need to go to each student. And you would need to click the button at the student level rather than the class level.

So if every student is different, then you probably need-- you follow the same protocol here, but you're going to need to do clicking for each individual student with the idea that the ultimate number of DL hours and for face-to-face hours is going to vary from student to student. If that varies, you're just going to necessarily record it at the student level.

I don't see anything else on that.

(DESCRIPTION)
Outcomes and resources.

(SPEECH)
So any other questions?

So here's the section that, in my opinion, really comes out of left field. So it really-- I've got to admit it doesn't relate to some of these other things quite as much, but it was requested, so we're keeping it. So here's CAEP outcomes. It's a CAEP presentation.

(DESCRIPTION)
C A E P outcomes. Literacy gains. Pre post level completion, Carnegie units, H S credits, C D C P certificate, occupational skills gain, workforce preparation. H S E H S diploma, High school diploma, passed G E D, H S E T and T A S C. Post-secondary, college degree, A A, A S, B A, B S, graduate studies, training credential, occupational licensure certificate, apprenticeship. Enter employment, get a job, retain a job, enter military. Increase wages, increase wages, get a better job. Transition, transition to A S E, post-secondary C T E, post-secondary college.

(SPEECH)
So yeah, I know we've just spent an hour talking about DL. And now we're showing bubble boy slides, what's up with that? It doesn't make any sense. So hopefully by explaining it, it's not so crazy.

But these are the areas of AB104. We've been talking for three or four years now, the six different areas. We've got all the authorized CAEP outcomes. They all fall in one of these six areas. Or actually, they don't now. I guess I've got to update this for I3, don't I?

But either way, we've got these six categories here-- literacy gains, secondary, post-secondary employment wages, and transition. We've got lots, and lots, and lots of different outcomes. What we did there a few years ago was we slotted 100% of them into one of these six categories. So we have kind of a system here that's logical across all of those categories.

So I'm not going to go into all of these outcomes, like CAEP accountability, but kind of adding to the confusion, you might say, is the issue here, is I'm not going to be talking about all the outcomes.

(DESCRIPTION)
Literacy gains, C T E related outcomes. Occupational skills gain. Usually suggests accomplishment of a portion of a longer term program. For example, a student enrolls in a long term welding program in C T E which is 5 semesters and modules long. the student passes a skills check written test that indicates the student is ready to finish module 1 and enroll in module 2. Workforce prep outcome. Usually suggests completion of a shorter term program. For example, a student enrolls and completes a 15 hour instructional module on job search strategies. The student earns documentation such as an informal certificate at the end of the instructional module.

(SPEECH)
But I am going to be digging in a little bit on some of these outcomes that have been talked about in these data dives that might be additional things that you might want to consider recording now that we're doing mostly distance learning.

So to set that up, I'm looking at the literacy gain side, and looking at those outcomes we have for a career in technical education. So the ones that reply to that are occupational skills gained and workforce prep milestone. A lot of you have heard about that.

So just as a sanity check, have you all heard about these outcomes before, yes or no? I just want to know how far I need to dig into this. So have you all heard about these CTE related literacy gains? So most of you are saying yeah. All right, great. Good for you.

So you've all heard about it. So again, occupational skills gained. A couple nos. Some of you may not be CAEP, so I'm not sure you need to know it.

So anyway, occupational-- the bottom line is they're both literacy gains. We're looking at ways to measure student progress within their cores of instruction. If you're doing that for ABE or ESL, we would say, heck, that's easy. That's what we're doing pre- and post-testing for.

So it's all built-in. There's an obviously answer through pre- and post-testing to accomplish that for ABE, ESL. We're already mission accomplished. But for workforce prep students and CTE students, it's not as intuitive, not as easy.

So we created these new outcomes called occupational skills gain and workforce prep milestone. When we first brought it up, it caused lots, and lots, and lots, and lots, and lots of confusion.

So what we say is use occupational skills gain. If you're recording that, that suggests partial accomplishment, or partial accomplishment of a long-term program, whereas workforce prep milestones suggest full completion of a shorter-term program.

So the example for occupational skills gain, we'll just say they're in a welding program that's five semesters or five modules long. We'll just say, hey, they finished module 1. They're doing a great job. At the end of module 1, they passed some sort of informal skills check, or written tests to say they're good to go. So they're ready to move on to module 2. That would be an example of occupational skills gain.

The workforce prep example is-- we'll just say it's a 15-hour course on job search strategies. That's really short-term, but you can record it. If they finish it up, if there's some sort of documentation, an informal certificate, or whatever. Hey, there's still alive. At the end of those 15 hours, you could give them a workforce prep milestone for that.

By measuring gain, my suspicion is you're asking about payment points. And I'll just say this is CAEP. So payment points are 100% irrelevant to this training. I'll just say if you're looking at payment points, the short answer is you can't. You've just got to get GED, or high school diploma. But if you really are talking about CAEP, you could put them in workforce preparation. And this is basically your answer.

To be clear what I'm talking about here does not have anything whatsoever to do with WIOA II payment points. And I'm not talking any further about it, because it's a CAEP training. But short answer is you're just stuck in that situation. You're just looking at getting GEDs or EL Civics. The longer answer if you're really and truly asking about CAEP, and my suspicion is you're not, then you can use some of these outcomes we're talking about right now.

(DESCRIPTION)
M S G's for C T E. The passage of exam measurable skills gain for W I O A 1 aligns with C A E P occupational skills gain. When a student achieves an occupational skills gain that now entails that the student passes an exam such as work skills demonstration. Workforce preparation outcomes should include some documentation of work skills progression or attainment.

(SPEECH)
So moving on here, here is just some screenshots. These are the specific bubbles or checkboxes in TE that correspond to occupational skills gain and workforce prep milestone accordingly.

(DESCRIPTION)
Literacy gains outcome. Mark when C A E P learners achieve outcomes in distance learning classes. Record achievements already attained before closures. Use passage of exam process for C T E M S G's as a guide for recording student achievement. Student passes informal exam such as a virtual class assessment. Completes skills demonstration in workforce preparation activities.

(SPEECH)
So here's kind of why we're including this in the training is in the trainings we did June, July, we started talking about the fact that, yeah, we're in distance learning. It's sure looking like distance learning is here to stay. COVID-19 is here to stay. So we need to look at some additional ways to record all of these marvelous achievements that students have been accomplishing in distance learning.

So I bring up that passage of an exam that is whether we're talking about occupational skills gain or workforce prep milestone. You need to demonstrate that they passed some kind of an exam. It doesn't really matter what that exam is, but you should have something that documents the fact that they really and truly did show that they know their stuff.

So what we're doing is kind of combining that passage of an exam approach with some of these activities you're doing with students in distance learning. So some of them might be informal exam, like some of those informal assessments we're doing for distance learning. We've been giving more written assignments, because we're doing distance learning, more oral interview stuff.

So if the student is testing those informal activities, potentially you could use those informal exams that they're doing as a way to record those short-term literacy gains. Again, if they complete-- so to get some of that question, sort of like what Anna asked for, if we're looking to at least be able to record something instead of nothing.

Let me-- so Rhonda, your question. If they completed that program-- so I kind of weeded out, because I didn't want to get too much into outcomes. But I think you're on this slide, Rhonda.

(DESCRIPTION)
Literacy gains, C T E related outcomes. Occupational skills gain. Usually suggests accomplishment of a portion of a longer term program. For example welding. Workforce prep outcome usually suggest completion of a shorter term program. For example a student and rolls and complete a 15 hour instruction on module on job search strategies. The student earns documentation such as an informal certificate at the end of the instructional module

(SPEECH)
So in this example, again, partial completion of a long-term mark occupational skills gain.

If this student completed module 5 of 5, that is they completed where they're completely done with welding and they're now certified to be a welder in the state of California, then that wouldn't be a literacy gain anymore. Then you might say it rises to the level of post-secondary. So you wouldn't mark a literacy gain anymore. You would mark earned occupational certificate, or earned occupational license, or that is, in that case, it would be a post-secondary outcome, rather than literacy gain. Good question.

(DESCRIPTION)
Literacy gains outcomes. Passes virtual class assignments, written assignment, oral interview. Learner mastery, educational suffer, local program milestones.

(SPEECH)
So back to this. These are some examples. So one thing you could do-- again, this is really specific to distance learning, where some of you have been giving students take-home assignments. So if they can show the assignment, that would be one way to show an informal exam. If you gave them an oral interview or written assignment, that would be another way to do it.

Another thing you could do is within learner mastery. So we talked about using learner mastery as a way to record hours. So some of you now are not really sure whether the student is really completing that video checkout or educational software or not. So one way to be extra sure that's happening is give that informal quiz to the student. If they pass that quiz with x% proficiency, that would be another way to potentially document passage of an exam.

In the educational software, some of the higher end software will give built-in quizzes. So things like Aztec or Burlington I'm pretty sure do that. When you're doing chapter number 1, you complete that hour of the educational software. Before the software even lets the student move out of module 1 and into module 2, it'll give that student a short little quiz that they have to pass to do that.

So that would be another way to leverage what your educational software is doing for you. Hey, they passed that Aztec module or the Burlington unit. So that's another way to potentially get a workforce prep milestone. Some of you have been doing similar things at the local level now that we're in distance learning.

(DESCRIPTION)
Complete skills demonstration in the workforce preparation activities. E L civics slash I E T, transitions, E L co-enrollment.

(SPEECH)
The other example we gave, our skills demonstration and workforce prep. So that relates a lot to what you might be doing in a EL Civics. I'm leaving it on here because I know a lot of you are doing this. Some of you may be doing it separately from immigrant integration, but I would just say now with the I3 outcomes, this is probably built-in.

But again, a lot of you are doing those COAAPs. Some of you might be doing less formal workforce preparation activities in conjunction with IET or IELCE. Some of you have been doing formal activities to document transition to WIOA Title I or other workforce programs.

Some of you have been doing things like EL Co-Enrollment. So if you're using-- if you're working with some of those other grants where you're looking to facilitate co-enrollment activities for EL learners, these would be all good ways to document some of that skills demonstration and some of the specific ways in which learners are earning informal outcomes that might be documentable in distance learning.

So again, you're not required to do any of this. But we've been presenting this, that if you're really stuck, you're not really able to move forward on remote testing, or maybe you are, but you're just not there yet and getting everybody done, or you just have some-- maybe it's like Anna with her higher level students that aren't really going to make a level gain using a class's testing. These are all kinds of different ways where you still have something instead of nothing to document these kind of activities.

(DESCRIPTION)
T E I 3 reports. T E has new I 3 reports that relate results from E L civics C O A AP's to A B 2098 immigrant integration indicators. There will also be a new column on the T E C A E P summary that displays these new outcomes.

(SPEECH)
So I summarized before I was done. I guess this is it. Sorry. I keep forgetting this is animated here. But we also have the I3 reports. This relates to that last slide, where now we do have those I3 outcomes.

This is a new column on that CAEP summary. So making a long story short, we've been working on this for a couple of years, where we're relating immigrant integration indicators. That is there's AB 2098 legislation that requires us as a state to have some way to measure outcomes in the area of immigrant integration.

So there's other things that might come up as well, but one of the big outcomes of that research or whatever, what relates to EL Civics, where a lot of us, CASAS included, would say something like, we've been doing immigrant integration already for many, many, many, many years. And it's called EL Civics.

So the committee that looked at this agreed with that, that we're already doing a lot of this work in EL Civics. It just has slightly different terminology. So what we did is we worked together to relate those EL Civics COAAPs to the eight different areas of immigrant integration.

We programmed that into a few different TE reports. So the reports in TE, you can see by the screenshot are a lot like those CASAS competency and content standard reports, whereas just making a basic instructional proficiency statement and the competency report, it would just state that CASAS competency. And these I3 reports stating that EL Civics COAAP instead of CASAS competency.

But at the end of the day, it's the same sort of information at the student class or agency level. You can relate performance to these immigrant integration indicators. The difference is for a competency report, it would be the percentage that got the item right that include that competency. For this purpose, it would be the percentage of students that passed that particular EL Civics COAAP.

(DESCRIPTION)
Remote testing information. Remote testing 1 on 1 demo on video, You tu dot B E slash U L O A W dash B H O dash D. Registering test stations for remote testing on video, You tu dot B E slash 1 I P R E 4 dash 8 T I W. Remote testing overview documents. www.casas dot O R G slash product dash overviews slash remote dash testing. Agency remote testing agreement for california must be completed annually and copy sent to C D E consultant and Casas program specialist.

(SPEECH)
A little bit. Here's the other part. Remote testing, not going to get into these gory details either. This is just paraphrasing some of the videos we have on our CASAS remote testing website. My guess is most of you have been there. Lots of YouTube videos, PowerPoint documents, and so on.

(DESCRIPTION)
Remote testing for E L civics. Go to the Casas website and go to california remote testing page. Guidelines for remote assessment for C A E L civics additional assessments and the Casas citizenship interview test. webinar recordings and frequently asked questions.

(SPEECH)
We've got a lot of information for doing remote testing for EL Civics as well.

(DESCRIPTION)
Remote testing considerations. Parking lot testing, Novi adult education, testing in cars. Youtube video of the webinar. Remote operations with on site proctor. On site testing with social distancing. Updates on current remote testing options.

(SPEECH)
What's happened here in more recent months, what we mean by considerations is a lot of you have now, like you answered in that question, some of you have been able to open up for testing. You haven't been able to open up for instruction.

But you have opened up for testing. Some of you have done the parking lot testing. Some of you have done what I like to call hybrid testing, where maybe the staff are on site, but the students are doing remote testing. There's been a lot of different configurations that have come up.

So at the end of the day, we're encouraging you to really be creative with this. If you are, we're not really willing to just to approve you on the spot, but you certainly can contact us. And we'll set up a meeting. We'll review what you have done, and try to give you pointers so it can kind of fit in with our requirements. And we can kind of move you forward. But again, lots of information on agencies that have done things like hybrid testing and parking lot testing.

(DESCRIPTION)
Remote testing: preparation. Take time upfront to make sure students are aware of what will happen and what to expect before the remote testing session. Offer a digital Boot Camp for students and staff to bring them up to speed for distance-learning and Remote testing. Designate technology navigators to work with students before testing.

(SPEECH)
Some things on that. Again, make sure you work with your students. Some of you have been offering digital boot camp. Some of you have been designating staff to really be working full-time on this.

(DESCRIPTION)
Involve a variety of stuff not only certified E tests proctors. Use native language before testing begins to support lower level E S L learners. Use the Casa E tests sampler to take practice items before taking the test.

(SPEECH)
Make sure you involve a variety of staff. If you just resort to certified proctors only, you're going to put lots and lots of strain on those poor proctors. Use the native language before the testing to get them going on all those digital literacy issues.

And then use that eTest sampler. The eTest sampler, of course, was designed for regular eTests, but it should get the students the feel of doing the testing online, or this remote testing, just like it will for, quote, unquote, "regular tests."

(DESCRIPTION)
Revoked testing our L O I. Provide in for my class, program, and level placement for students. Accessible to students via cell phone. Does not require a proctor. The reading level indicator is 4 6 0 1 R. View results in the T E test Lister.

(SPEECH)
And then another feature a little bit more popular in August, September than now, but we have that new instrument called the RLI. RLI is not a pre- or post-test, but it provides informal placement. So if you are not worried about pre- and post-testing, but you just want to make sure you get the class and program placement right, you might want to use the RLI. It can be done on cell phone. It doesn't require a proctor. Those are the two really big advantages.

So what happens is you use the same functionality as ENE survey we have. That is you contacting students via text, via email, or both. You're using that same methodology where you're using the TE student portal to reach out and access those students. That's where the student will receive the invitation. They just receive the invitation, and then they can automatically just jump right in and complete the RLI on their cell phone.

When they finish, they just click the final click. It will automatically send it right back to TE. And then once they complete, you can just simply view the results in the TE Test Lister. That's what that screenshot is trying to convey.

(DESCRIPTION)
Distance learning examples.

(SPEECH)
So you might say now, back to our feature programming. I think we're OK with timing. It's 2:15. We're here till 2:30, so I think we're right on time.

So I'm going to take a breath here, because I kind of zagged. And now I'm going to go back to zigging again. Is everybody hanging in there? Any questions before we go back to any of that zigging?

Sanity check. OK. Thank you. Everybody is hanging in. Okey dokey, thank you.

So these are just going to be rapid fire, a bunch of examples related to those hours issue we talked about. I don't-- I'm going to start on this one here, I think.

(DESCRIPTION)
Clock time model example 1.

(SPEECH)
So we're going to start with example 1. Here's a nice easy clock time model example.

So in this example, we have 20 students enrolled in an online ESL class. Everybody attends for one hour, one hour on Tuesday, one hour on Thursday via Zoom. That would be clock time model. That would be synchronous.

So here's an example of how we might write it.

(DESCRIPTION)
The classes listed below me two hours a week online via zoom. Agency uses clock time model record hours all instructional hours are code via zoom. Students are required to enter name in chat upon arrival to document participation. I’ll zoom sessions are recorded for teacher to verify hours and participation for each student

(SPEECH)
We just list the classes below, say how many hours a week they attend. We're using clock time. The hours are recorded via Zoom. We list a little process for how we can identify the fact that the student really and truly attended via Zoom.

We're going to record the session so we can verify hours. We're going to document it in chat. Lots of different ways you can do that. This is just an example, but this is the sort of information you want to put in your policy.

What model are you using? Why are you using it? And then just a little bit on the security side to show how you really and truly can be sure that that student really did indeed participate in that distance learning instruction.

(DESCRIPTION)
Clock time model example to. 20 students are using educational software that automatically record the correct number of hours for each student. Local assessment policy example. Students and classes listed below complete lessons in software app ask for up to six hours a week. Agency uses clock time model hours are automatically record it in the software. Recorded time for each module sister below. Students are required to login using unique user name and password to document participation. Students to complete all exercises as required by the software to demonstrate mastery.

(SPEECH)
So here's another clock time example, where in this case we're using instructional software. I want to say this might have been Leanne's question about the software. I'm not sure if I have the right person.

But anyway, in this example, we've gotten the same 20 students, but this time they're using educational software. In this case, they're using the high end software that automatically records the correct number of hours for each student.

So again, if it automatically records it, then it would be using clock time. So here is a way that you could write it in your policy. The students in this class complete lessons, and whatever. Software application, you write how many hours a week they're completing it.

We're using clock time, because hours are automatically recorded in this particular software. The recorded time for each module is below. That would be one thing that would be helpful is if you know the software does it. Put in writing how many hours each module gives you in that software. You don't need to apply any of those techniques or figure it out yourself.

But putting in writing what the software gives you is a good idea. Students log in with a unique username and password. Again, just put some of that ticky tack information on how is it that you're sure all the students really did require this. How is it you can be sure that it really and truly was that one particular student.

There's no limit. I mean, again, if you've got that higher number of hours, fine. I would suggest please do document it. So you can see in my example I said they're participating up to six hours a week. If you're saying, yeah, that's appropriate, but they're participating for up to 20 hours a week, not merely six, then I would say, fine. That's A-OK. Just document the fact that they're logging that much time in any week in that software.

I guess I would just say set up-- you can put the extra hours. There's a little bit where, hey, set up the class to allow for up to 20. In general, it's better to set up the class with that larger number, and just know that most students are going to get a smaller number than have to give more than what you've already recorded. So I would say probably in this example, you want to set up the class for 20 hours a week rather than just 15. But overall, the short answer is yes.

Does that make sense, hopefully, Leanne?

OK. Thank you. All right. So next example.

(DESCRIPTION)
Teacher verification example one. 20 students are using educational suffer that does not record hours and does not include additional exercises at the end of each lesson. Local assessment policy example. Classes listed below complete lessons in software up ask for up to six hours a week. Students are required to login using unique username and password to document participation. Agency uses teacher verification model. Students complete all activities as required by the software for each lesson. Teacher notes the progress for each student and assign 0 1 or two hours of instruction for each lesson.

(SPEECH)
So here's a teacher verification example.

So in this example, we're using educational software, but in this example, the software does not record hours. And it does not include any additional activities. So in this example-- still we're requiring the teacher to observe everybody. So in this example, we're listing which classes are completing lessons in this specific software.

Again, we're stipulating how students are logging in so we know the student really and truly did this. So we're point blank saying we're using that teacher verification model. This is just an example. It doesn't have to be that complicated.

But in this case, we're saying the completed activities required by the software. We're specifying that the teacher is noting that progress for each student. In this case, we're saying we're observing the student. We're noting how vigorous each student is participating. In this case, we're saying we're assigning 0, 1, or 2 hours based on how the teacher observes each individual student.

There's more to follow.

(DESCRIPTION)
Learner mastery example to. 20 students are in rolled in an online ESL class for one hour Tuesday/Thursday via zoom. Most attend as scheduled but some students who cannot attend access the recorded session. Local assessment policy example. The class is listed below me two hours a week online via zoom. Sessions are recorded for teacher to verify hours and participation for each student. Students that missed the live session can access the recording any time before the next scheduled class. Learn his login to the recording just as they do the live session, and must complete a short quiz at the end of the session to get credited with the instructional hours.

(SPEECH)
So again, none of these are what you're required to use. Hopefully everybody's getting this. We're just providing multiple examples. There's no one size fits all. You're not required to do any of this. We're just trying to give you lots of different examples covering lots of different scenarios, to hopefully where everybody can kind of find one that at least more or less relates to the sort of activities you're doing at your agency.

So here's a learner mastery example. In this case, we're saying we've got 20 students enrolled in an online ESL class Tuesday, Thursday via Zoom, similar to that first example. Most students attend to schedule, but some students can't attend, so we direct them to access the recording some time after the scheduled session. So most students are synchronous, but a very small number might be asynchronous, because they missed the class and have to access the recording.

So here's the example where we're listing that they're meeting two hours a week via Zoom. Sessions are recorded to verify hours. And we're stipulating those that miss the live section can access the recording. If you allow this, we generally say give a timeline so they access it before the next class, or something like that.

A lot of you have kind of rightfully noted that can be a slippery slope. It can look a little bit like Mickey Mouse. So hard to say exactly what the right answer is. But if you allow them to access the recording, we definitely think it's a good idea to put a timeline on that.

So in this example, we're just saying they can access it any time before the next scheduled class. They log in using a username and password the same way they would the live section. And then in this case, we're saying, yeah, we're going to give them a short quiz at the end, because they had to access the recording. So we're going to give them a short quiz at the end. And by passing the quiz, then and only then are we going to credit the hours, because we want to make sure they accessed the recording and really listened.

Again, that quiz is not required. Some of you really like to make sure the students are doing it the right way. Some of you see that as extra busy work. That choice is up to you, but here's an example where you just stipulate you're doing that quiz at the end to make sure they get credited with those hours.

(DESCRIPTION)
Learner Mastery example 3. 10 students are in rolled in independent study classes and watching a saint YouTube videos as part of required instruction. Local assessment policy example. The class is listed below meet two hours a week online via zoom And also complete three lessons in video series why one hour per video. Agency uses learn a mastery model for the online videos. Once a student complete all three videos, the student must achieve a score of at least 80% on the quiz that incorporates a video content to achieve mastery receive the three hours of instruction.

(SPEECH)
Here's another learner mastery example. We're giving a slightly different one here, where in this case, we've got 10 students enrolled in independent study. In this example, they're watching assigned YouTube videos as part of the required instruction.

So lots of ways you can set it up. Here's one of many different ways. So we're just saying we're meeting for these two hours a week. So we're recording that using the other methods, but part of it includes completing three different lessons in video series whatever. Each video is an hour long. For these videos, we're using learner mastery.

Once the student completes watching all three videos, we're then giving a quiz. They've got to get at least 80%. So once they pass that one quiz, that gives them all three hours of instruction. That's just a made-up example.

The different ways in which you do it, you can do one hour at a time. You can do all three hours at once. You can make an 85% proficiency instead of 80%. As long as you put it in writing and do it the same way for each student, the details don't matter so much. That's up to you.

But yes, you should stipulate what they need to score on that quiz. Yes, you should stipulate exactly how you're recording the hours, and exactly what passing that quiz does for you. Putting those sort of details are important.

(DESCRIPTION)
Example one using next delivery bet it is. 20 students are using educational song for that does not record hours or include additional exercises at the end of each lesson. Most participate in a live session with the teacher but somehow missed the live session complete activities on their own. Students meet two hours a week online via zoom. Sessions are recorded for teacher to verify hours and participation for each student. Students that missed the live session can access the recording anytime before the next scheduled class. Learners login to the recording just as they do the live session, and must complete a short quiz at the end of the session to get credited with instructional hours.

(SPEECH)
Here's an example with mixed delivery methods-- I'm running out of time here-- where they're using software that doesn't record hours or do those exercises. Most do a live session with the teacher, but some have to do it on their own. So this is an example where those who are doing the software with the teacher live, then you might use teacher verification, because for those students, the teacher can watch the student.

But you've got a small handful that have to come back through learner mastery. So those you might do learner mastery and do that short quiz, like you do for other learner mastery students.

(DESCRIPTION)
Example 2 using next delivery methods. Local assessment policy example. Student complete lessons in Southport up ask for up to six hours a week. Students to login using unique username and password to document participation. For students to participate during scheduled sessions, use teacher verification model by facilitating online activities and prompt and participants with questions. For students to complete activities on their own use learner mastery, and must pass the quiz with at least 80% to be credited with the instructional hours.

(SPEECH)
Here's another mixed method where they're completing software application x for a certain amount of time. Students log in for teachers who participate during scheduled sessions use teacher verification. For those that attend later, then that means they do learner mastery and get at least 80% on that quiz.

(DESCRIPTION)
Example describing learning mastery process. Example, 12 module educational software program X. 12 modules, 24 hours total. Module one intro to work one hour. Module to career exploration 2.5 hours. Module three making of your choice is 1.5 hours. Follow the same process listing all 12 modules. List their number of hours assigned to each module. Record hours and T E according to the specifications.

(SPEECH)
So here's the longer-winded example, last example, where if you're just doing it for one full unit, you're doing a unit of Burlington or a unit of Odysseyware, whatever. You write out all 12 modules. Assign a number of hours for each module. List all 12. Give a number of hours.

This is where your instructional team is going to figure out how much is each module worth. Assign a set number of hours to each module. Put it in writing. Once you do it once, it should be A-OK. So this would just be the roadmap that you would be using every time a student completes that particular set of modules in the educational software.

So sorry. Oh, I guess I'm not as bad as I thought. I thought I saw 2:31. No, it's 2:27, so I've got a couple minutes left. I guess I'll just stop here for questions, and kind of turn it over to Holly and Mandilee.

All right. Thank you, Jay. If anyone has any questions, we do have a few more minutes. If you'd like to type them into the chat, he can answer them for you. Otherwise, as a reminder, this webinar is being recorded and will be available on our CAEP website at CalAdultEd.org.

And I don't see any questions. Thank you. So with that, I would like to say thank you, Jay, so much for presenting with us today. And attendees, thank you so much for joining in.

I have posted in the chat the website where you can access the recording if you wish. As another reminder, we do have some PD opportunities coming up beginning on November the 12th. We have a three part series on equity. If you are interested in that, please go ahead and register for that. And again, I'm going to go ahead and post that link here in the chat.

And I think that does it for us today, unless there's anybody else that has questions. It doesn't look like it. OK. Thank you all. And have a great afternoon.

Thank you very much.