Anthony Burik: This is Anthony from OTAN welcoming you to this webinar, Considerations When Implementing Distance Learning. And if we are ready to go, then I'm going to turn it over to Jay. Go ahead, Jay.

Jay Wright: OK. Hi, how are you? So here's the overview of the overview, very much the same as the other sessions. Again, I think about half of you probably have done this session before, but obviously, COVID-19 has had a huge effect on California, a huge effect on adult ed.

There's an OCTAE memo. We've been covering it a lot. It's a Memo 20-3. We'll talk about that. That's the big directive from the feds. They sent out a big directive at the end of March. They very recently, I think last Friday, sent a follow up to that memo nationwide, so we'll talk a little bit about that. And then there's three big issues when we're talking about considerations for distance learning.

Issue number one is how to deal with classes, so we've definitely not come up with anything scientific here, but we have come up with what we think are the two bottom line basic approaches to how to create classes. So we'll identify and define those two different ways. The other big issue that's come up is how to account for hours. There was a session that Penny did yesterday. Probably a lot of you attended it. She got in depth with a lot of the different models and how it applies in the classroom.

What she was just using are the three models the feds stipulate of how to do hours, so we'll look at a couple examples and talk about how to do hours. And then we'll talk about assessment. So there's another three-headed monster within the three-headed monster there. There's pre and post testing, placement testing, and EL Civics.

The short answer for all three is we're still saying an in-person proctor is required for all three, but we've talked quite a bit about we're moving fast and furious on remote options. We started a big field study yesterday, so we've got progress on all three fronts-- pre-post, placement, and EL Civics. More about that later. Then there is a recording distance learning and enterprise. There's one Shana Sparrow has done a couple times. My guest says a lot of you have been on that one too, but we'll talk briefly about some of the TE features.

There's one that Janice does that really gets deep in the weeds on how to use TE. We don't really have time to get deep in the TE weeds here, but we will at least show a couple of those features with the short answer being you can record exactly what showers occur in distance learning versus exactly what showers occur in the quote unquote "regular classroom." And then a little bit of a preview. This is the temperance between the slides last time and this one, as we have put together what we're simply calling distance learning Part II.

There's been lots and lots and lots and lots and lots and lots and lots and lots of questions that have come up about this, lots of different areas in which the questions have arisen, but I think maybe the number one area is what the local assessment policy that is throughout this one. There's numerous times when the short answer is you can do what you want, but you need to carefully define what you're doing. And your own agency's local assessment policy.

That's naturally generated a lot of questions to the effect of, well, hey, that's great. We're glad we have the flexibility. We definitely have that policy, but what exactly do we need to do to record it correctly of said policy. So we'll look at a lot of the models I've been talking about, a lot of the models that Penny really detailed a lot yesterday, and try to come up with examples. We're not going to get every single one, but we've come up with six or eight different examples that characterize six or eight different ways of providing distance learning, some examples with all three models and how we might document that in the policy. That's what will be going on with Part 2.

We'll probably be looking-- this is tentative here-- we've got a few I's to dot and T's to cross before it's official, but potentially, we'll be looking at delivering Part 2 on Tuesday, May 5, and then we'll be doing some stuff on Thursday May 7 as well. OK, moving on, obviously, COVID-19 has had a huge effect on all agencies.

Schools have closed, colleges have closed, and so on. We've talked a lot about that OTAN COVID-19 page. Here is the link on the slide. Lots of information from CDE, from OTAN, from Cal Pro, from CASAS. We have our FAQs on the State and federal accountability. There's a TE help document. There's official letter and policy statement from CDE. So anything you wanted to look at and more is on that COVID-19 page. Next slide.

The other big reference here is OCTAE Memo 20-3, so it was mid March when we started transitioning to this COVID-19 quarantine mode, you might say. Once everything was headed in that direction, we were eagerly awaiting the feds to weigh in on what will be and what won't be. They weighed in with OCTAE memo 20-3. To be clear, for those of you who haven't heard these a lot lately, OCTAE stands for Office of Career Technical an Adult Education. That is the component of the US Department of Education that pertains to us.

So they came up with Memo 20-3 that had all of the updated federal guidelines now in response to COVID-19. Basically said, yeah, we know that we've changed a lot, but we're still collecting data. We're still requiring all States to report to the feds. We added extra emphasis that there are non-testing MSGs that can be collected. It brought up a checkbox that I'll talk a little bit about later. There's now a checkbox in TE that we'll use to record everybody that was affected by COVID-19. We're still looking at these procedures for how they might affect us in California.

I think as far as how to record classes and hours, we don't really have any more investigation we need to do. We've just had to dust off and clarify a lot of existing policies where the review fits in though, of course, is related to remote testing. A lot of that Memo 20-3 is about remote testing. So CDE and CASAS are still working on more issues related to that. More on that later. Sorry, I should add before I move to the next slide. There is an updated memo that the feds sent out. Can you just back up? Sorry, I missed the one in bold. Thank you. The updated memo April 17.

So the feds did send out an updated memo last Friday. It basically reiterates most of the things they stipulate and Memo 20-3. However, it gives States a little bit more flexibility to explain why they may not necessarily be doing remote testing. In late March, things had just started to kind of hit the fan, you might say, and the feds-- well, I think we're really sort of figuring that everybody would need to scurry and do remote testing as soon as possible.

Now, it's starting to realize that that's easier said than done, and there's not going to be an expeditious process, and it's also determining that some States are eager to get started on remote testing yesterday. Others are not as eager, so it's provided I would characterize it as more cautious language, where it's still looking at ways to move forward on remote testing, but it's looking a lot more on States just explaining why it may not be able to do remote testing and more emphasis on identifying those students that weren't able to test. So anyway, I'll move on to the next slide, just probably more than enough on that.

So basic requirements for DL students, the short answer is individuals and distance learning have the exact same requirements as everybody else. Those who are in distance learning should mark Special Programs Distance Learning. You can mark it at either the class or student level. And here within the last week, we've added a new checkbox now that didn't exist before, but again, back to that OCTAE memo, what required all States to report a head count of all students affected by COVID-19. So we added in TE, a checkbox that we're calling Force Majeure, which is the term the feds have been using to characterize the students that were forced out because of the quarantine and so on.

So in any case-- force, there you go. Somebody wrote it. Great. So anyway, we have a check box now in TE. If you want some screenshots, you can get it from me, from Janice, or from tech support. It's been in TE now for almost a week, so a lot of you, I'm sure have seen it. So the check box is nothing special, nothing really mechanically notable about it, but I will say we are suggesting that you just leave it blank for now.

We did want to get out in front and get the programming done sooner rather than later. But we still have more to discuss with CDE related to exactly which circumstances require you to check that box. Some are suggesting, well, hey, COVID-19 affects everybody, so it should be everybody. Others are saying it should just be those who can't test or just those who got closed out of class or whatever, so lots of possible options. No answer is right or wrong, but we want to come up with something uniform so you're all marking that the same way. Next slide.

What if the students in blended learning-- and this will be a lot of your students who are in blended learning, that is if you look at it across the year or across the class, there will be some of that time in the regular classroom and some of that time in distance learning. The NRS, or feds, have always had or not always, but for the last 10 or 12 years have had the 50% rule. That is if more than 50% of the instruction is outside the regular classroom, marked distance learning, if less than 50% is outside the regular classroom, don't mark distance learning. So it's just simply more or less than 50%. If it's more than 50%, obviously, mark distance learning and special programs.

You don't need to specify exactly how many hours for DL versus how many were not DL. All you really are required for federal reporting is to account for the 50% rule, but I will add there is now a feature in TE where you can get to that level of detail if you choose to do so. That's a lot of what Janice presented in that TE-related session.

We're also pointing out a field in the class instance record called Instructional Setting. Instructional Setting is an old, antiquated field that we used in the old cohort system five years ago, so it's not required. It hasn't been relevant for at least five years, but we are pointing that out as an optional field to check. A little bit of that is for you because you're trying to track classes that you might have switched over midstream. You might have had a regular class through March.

Now, we're in COVID-19 mode, so it's all DL. It's still the same class. It's way less than 50% distance learning. So you're refraining from marking special programs, but you know that a funny thing happened on the way to the forum there on the ides of March. You know it's not like any other class, so we're saying that's an informal way that you can show that that classroom is affected in an informal way to show that distance learning is occurring, if perhaps you decide you don't want to earmark every single hour.

So that's an optional way that you can mark it. And there's a lot of information in that federal memo that requires all States to provide narratives on all of the different ways in which we're affected by COVID-19. That's a way where we can potentially provide the anecdotal information. That certainly can't be anything official, but if we can say, yeah, we anecdotally have this large number of classes that were not marked officially for DL, but were marked unofficially as less than 50%, but where you did have to address some of these issues at some point in time, we're thinking having some numbers to that effect would probably be a good way to report that informally with some of the extra things we'll need to do.

OK, next slide, please. If it's less than 50%, again, you need to do nothing. That's where I would really hone in on that classroom plus distance learning where you know it's less than 50%, so you're not marking distance learning. You don't really have enough information to get in the details and record that hour by hour. But again, you know a funny thing happened on the way to the forum. You know it's not like any other class. You know that some of it is distance learning. That bottom bullet is a good way to show that you made some of these changes without necessarily rising to the level of recording distance learning based on federal rules. Next slide, please.

So the first big topic is creating classes or addressing classes, or actually, sorry. So we're looking at those two methods. In doing that though, we have two key considerations we know we need to follow. We need to make sure that we follow the 50% rule. So we can do that by recording the hours granular or we can just look at each student or each class and mark it. Bottom line though for what's required is you must calculate that 50% rule for every single student.

So that's the part that is non-negotiable. You really need to figure out the 50% rule. Normally, it's a small issue, but obviously, this year it's a big one. The second one is totally optional. Janice and others have shown some of these sessions where you can now show exactly what showers our distance learning versus exactly what showers a regular classroom. If it's best or most promising or easiest for you to do it that way, of course, I'm going to say really, really great idea. But I'll point out you're not required to stipulate every single hour if some of that is easy to track and some of that not so much you're not required to do so, but it is available if you wish. Next slide, please.

So I'm just going to go to this one in bold here. We've talked about more than 50% or less than 50%. Bottom line is calculating this at the class level depends on how you structure classes in response to COVID-19. So I took a one time to build up to that bullet, but we'll go to the next slide, please.

So again, here it's repeated again, but not in bold, calculating this at the class level depends on how you resume classes and how you structured that activity in response. So again, not scientific. I can't find any official terms anywhere for this, so we made them up. One we're calling pick up where you left off, the second we're saying is start brand new classes specific to distance learning. Next slide.

So pick up where you left off. So what we're suggesting with pick up where you left off is you're leaving all the classes as scheduled. So maybe you started them at the beginning of the program year. Maybe you started them in January for the beginning of the winter/spring semester or whatever, but the thought is you have lots of classes in progress. Yes, we probably had to suspend the classes. Yes, we definitely converted them all out of the regular classroom and into distance learning, but we're still keeping it basically the same, the teachers the same.

The students are the same. We're still meeting on Tuesday, Thursday mornings, or whatever. The only difference is we're meeting Tuesday, Thursday morning at 10:00 AM via Zoom, rather than in the regular classroom. So we're just keeping it all the same. We're just meeting the same time, but doing it a little differently. So in that case, we just keep the class record as is, but we do need to worry about that 50% rule. Next slide.

So here's an example. So we have an ESL class that started in August and was scheduled to run through the end of May. We're just using a sample situation where we kept that regular classroom instruction going till March 13. We closed for a week or so as we were in COVID-19 mode, but a week later, on the 23rd, we reopen the class, same teacher, same students, but now we're doing it in distance learning.

Here's an example where we would not mark distance learning at the class or student level because less than 50% of the instruction was delivered outside the regular classroom. Again, we went early August through early March in the regular classroom. We've just got the final month or so, distance learning, so we would not mark distance learning in this example. Next slide, please.

Again, pick up where you left off. Same approach, but in this example, the class started the 1st of March. Again, scheduled to run through the end of May. The same interruption time, we kept the regular classroom through March 13, reopened it via distance learning on the 23rd, same general set of circumstances, but here we would mark distance learning because, again, here the majority of the instruction is via distance learning. So in this example, we would mark it because it was more than 50%. Next slide, please.

So in either example, whether it's more or less than 50%, you can specify that in TE. Just saying that's the session that Janice got into, where, again, we're calculating 50%. We're not required to mark this level of detail, but there is a way to do it if you really want to make sure you capture exactly what's held at distance learning versus what's in the regular classroom. Next.

Next approach is start completely new classes specific to distance learning. So we're basically under the same conditions, of course. This suggests that up until the 1st of March we were doing everything in the regular classroom. Since mid to late March or so, we switched 100% to distance learning just like everybody else.

But in this approach, we're suggesting that what COVID-19 hit, we had to do everything completely differently. We had to close down all the classes. Now that COVID-19 has hit and we have distance learning, the teacher assignments are a little different, the student enrollment is a little bit different. Because of that we're now offering at different times. The new world doesn't really resemble the old world at all, so we're deciding to create completely new classes specific to distance learning.

So in this case, we're closing out all the existing classes in the regular classroom where perhaps we're giving new class records, new ideas, most likely giving completely new titles to those classes. So we've got completely new classes now for COVID-19. So the simple answer there is we close out all those old classes. The old classes were in the regular classroom, so obviously, we're not marking distance learning.

The new classes all are completely distance learning. So for 100% of those new class records we're creating, we're marking them as distance learning, obviously. Next slide.

So first example, again, here's an example where we started everything up in early August. We were scheduled to run that class through the end of May. The COVID-19 timeline, we're using the same as those other examples. We closed the existing class on March 13. We reopened new classes via distance learning on March 23. In this example, we had the class started in August. It will be non-DL because, of course, that was entirely not distance learning, and then we just create a second class that we're marking for distance learning effective March 23 because that's when we started that brand new class in distance learning. Next slide, please.

The second one, again, this one we started the 1st of March. Again, it was scheduled to run through the end of May. Again, the same interruption due to COVID-19. We suspended the regular class on the 13th. We resumed the distance learning class on the 23rd. So same basic approach and answer here. We close out the non-distance learning class, give it an end date of March 13, create a brand new class, give it a start date of March 23. The new class, of course, is exclusively distance learning, so mark that as distance learning at the class and/or student level. Next slide, please.

And then again, either way, you know what, whether you're marking it, you have that option of marking all the distance learning activity. You know what, in TE, perhaps this is a little less necessary for this approach, probably for the pick up where you left off, I would probably encourage you to consider this for this other option where one class is 100% not DL and the other when 100% is DL. You may not need to really record this as much given that situation. Next.

So moving out of class creation and into the second big issue that's submerged that is recording instructional hours. So just like we did with the class issue, we'll start at the top with the feds. In the federal implementation guidelines, they report three different models of measuring distance learning, clock time, teacher verification, learner mastery.

We'll start at the top with clock time. Clock time is the easiest and most straightforward. It's simply using the clock. That is, you know what, basically, it's suggesting we're using the same method or approach to record distance learning as we would in the regular classroom. That is in the regular classroom, if the students and teacher meet for one hour, the default answer is you would record one hour of instruction for all the students that met for that class. Clock time model and distance learning is doing the same approach, that is one hour of regular classroom is the same as one hour of distance learning classroom via Zoom, for example.

Teacher verification requires the teacher to observe the students and assign a certain amount of hours based on how the teacher observes the student to complete. So there might be a set amount of instruction for which you're meeting with the students. The teacher is observing the students. Some students might be hard at work, so you might record the entire amount of time for those students.

Other students may not be hard at work, so you may not record as much time for those students as the ones who are hard at work. Teacher verification is doing that and allotting different totals to each student based on the teacher's observation of different students and those putting some students to complete all the hours of instruction and other students to perhaps not be completing all the hours of instruction. That's teacher verification.

Learner mastery, on the other hand, again, not using clock time, but it's assigning a fixed amount of time based on observation ahead of time, recording it ahead of each lesson, and then recording that amount of time once the student completes each lesson. So learner mastery suggests the same basic scenario as teacher verification, that is some students are going to be hard at work, some students are going to motor through it because they're working very hard, other students either due to not being hard at work or perhaps just being interrupted by many, many things take much longer to complete the same amount of work.

Learner mastery assigns the same amount of time to that amount of work for all students and basically, records that set amount of time for each student upon completion with the idea that some students might complete that hour of work in an hour. Others on the other hand, might complete it in a few hours based on interruptions, so you use learner mastery and define the amount of instruction for that work prior to the students that engaging in it to record the amount that you defined ahead of time and thereby, record that same amount of time for each student equally. Next slide, please.

Of the three, you can probably detect from my manner of presentation I spent the most time describing learner mastery, not so coincidentally. That's the one that's explicitly stated in our California statewide policy, that is the proxy learner mastery model to record distance learning. To be clear-- and I think Penny covered this in really good detail yesterday-- you're allowed to use any of these three models in California. The State goes out of its way to allow all three. It basically allows all three, but it does require you the agency to formally choose which model or models you're using, and it does very much require you to explicitly state that in your local assessment policy number one and also describe in detail how you're employing each of those models in your local assessment policy number two.

So again, you have the flexibility to use which model you want. You can use multiple models if you choose to do so. However, once you employ that flexibility, that does also mean you have the responsibility to describe in detail how you're employing each of those models if you're using more. But again, learner mastery is the one that we use the most in California, and I'll say it's not just a culture issue. I think I've been presenting it as a yeah, yeah, that's what we use the most, which is true, but beyond the fact that it's the one we use the most, it's also the one that CDE has stipulated in the statewide assessment policy as the preferred model, and I believe it's done so for at least 10 years.

So I will note it's not just, oh, yeah, by the way, that's what most people do, it's what most people do because it is the one that's been specified in our statewide assessment policy for quite a long time. But again, either way, you have that flexibility, but whichever one you use, you need to record that in your local assessment policy. And again, you can vary your approach, depending on the model you're using and obviously, depending on the distance learning format in which you're providing the instruction as well. Next slide, please.

So we've got some examples here. We're looking at facilitated online instruction. Facilitated, meaning synchronous. We have another example we're using for educational software. There is a synchronous approach to educational software and an asynchronous approach. We've got a couple examples there. And then we've looked at that educational software and boiled it down to three practical situations, three practical examples determining those proxy hours, depending on the exact scenario kind of suggests using a different model or get into a few of these starting now. Next slide.

My audio is breaking up. Sorry, I'll try to speak more clearly. Hopefully, this is better. So anyway, the first one we're talking about is facilitated instruction received online. Sorry, I might have had my hand on my mask. Sorry about that. So how do we do facilitated synchronous instruction? We're using clock time. So this is just the example I gave you earlier, where we're suggesting that we're meeting with the same teacher, same student, same days to the week, same time of the day. All 20 students are meeting Tuesday morning at 10:00 AM, but they're just meeting via Zoom instead of the regular classroom. That's a synchronous facilitated example.

So in that example, we'd simply use the clock time model. We've recorded an hour of instruction for all those students on Zoom the exact same way we record that one hour of instruction for the students that attended the regular classroom, pretty much no muss, no fuss here. Use clock time. Again, an hour of online facilitated equals an hour of regular instruction, and that if that happens to be more than 50%, mark distance learning and special program.

What if it's a recorded session? So we're using the same example where we're just doing the teacher talking to the students via Zoom. In this example, we'll say we had 20 students. 15 of them attended Tuesday at 10:00 AM, but five of them missed class, so they had to make it up via sitting through the recording. So if you do the recording, that's fine. We suggest that you do stipulate that process if you want students to not make it through the recording.

I'll add, you, the agency, have the power to decide this for yourselves. Some of you would say, well, yeah, we gotta make sure we give students that option. We've got to give students that flexibility. Others, if you are suggesting, that's a slippery slope and you really don't want to allow that at all. I'll just say it's up to you whether to allow it or disallow it. Just make sure you stipulate it clearly in your local assessment policy. If you do allow it, if you do want to provide that option for your students, obviously, be sure to document it.

For that scenario, we would say that's more of an asynchronous example, not an asynchronous, non-facilitated example. As such, that would be an example where you might need to use learner mastery instead of clock time, that is you just have the students complete that hour, but instead of relying or trusting the student and giving the full hour, you might have a series of exercises or a quiz or something that you have the student complete to show that that student really and truly did watch that recorded session.

You would thereby have the student complete whatever it is you needed. Sometimes that's assessment. Sometimes that's exercises. Sometimes that's an oral interview, whatever. But whatever it is you wish to do you would stipulated and define it very clearly in the local policy. Once the student completes it, you document that same instruction, and again, just record it in detail in your local policy.

What if you're using educational software? This is, obviously, the really common example. We're saying, not always, but usually, we're saying that's learner mastery. It would be a lot like what I just described for the recording. Again, some students might complete in an hour because they're hard at work. Others might get interrupted and take a lot longer.

So what you would do is just figure out, hey, this is an hour of instruction, so we know this lesson should be an hour. So for all students, you record an hour. You can't really monitor the students, so you might, again, give a quiz. You might give a test. You might give an oral interview or just completed exercises. Once the student completes that, you would record that hour of instruction for the student.

Again, we're using learner mastery with the idea that the students that were there at the session completed that hour via clock time. They were all there in class and engaged for the hour. So for the students that made it up, you would want to make sure they're treated equally and recorded an hour of instruction just like those that completed the class.

So with software, we've sort of broken this down into three different approaches based on the software. Some software applications are pre-programmed for hours of instruction, that is some have it set where the student completes an hour on the educational software. That software automatically records one hour of instruction. If the software has that advanced feature programmed in, obviously, our suggestion is use that feature as implemented by the educational software. Just use what they give you. Record the amount of time the educational software records in the enterprise for official recording. Again, use that fancy feature if it's provided for you.

The second one isn't fancy, but relies on the publisher, that is the publisher doesn't program the hours of instruction into the software, but does provide some sort of software publisher best practice recommendation or whatever. If so, simply use that recommendation that the software publisher provides you. Much better to use their professional recommendation, save yourself some time so you don't need to do it yourself.

The third one is if they provide you nothing, that is you have the software, but there's nothing in the software, nothing in this instruction manual, nothing anywhere that provides you that guidance. So that's where you need to meet. It's that management team. Assign a set amount of instruction for each lesson, each module, each activity. Your mileage will vary on exactly how you wanted to find the increments, but you need to meet as a team, define that instruction, and provide a set amount of instruction for the students based on how long it takes each student to complete that activity.

So example 1, the software records the amount of hours or amount of instruction into the software, again, simply provided that total that's recorded by the educational software. So we're just saying we've got a 12 module, you know what software. The student completes less than 1 of 12. The software records one hour for completing Module 1 because that's how long they've already determined Module 1 takes. So the software has recorded that into the software already. So it simply records what that educational software has recorded for you and mark that in TE as the student's instruction allow.

Example 2, here's where they don't program it, but they do provide that test publisher's recommendation. Then again, you don't have anything automatically recorded for you, but again, use that recommendation. Whenever the student completes that activity, just record that amount of time. In that case, piggybacking a little more here, a couple of assets, I would generally say it could be either way, depending on the dynamic of the software. But usually this would entail learner mastery where like the situation where they don't provide you anything, you're defining what activities they're completing. They're completing Module 1 or Lesson 2, or whatever it might happen to be.

But instead of you the professional team at your agency needing to do the extra work to figure out time increments for each module, you're simply doing it the easy way, where you're just peeling off the amount that the publisher tells you and just recording that in your local assessment policy. Obviously, way less work for you. Obviously, the best way to do it, the assumption being that publisher is going to know better than anybody, how long each of their own, exercises is going to take each student. Next example.

So here's where you're using the software, but they don't provide anything. Here's where you need to meet as a team and define it locally. So there's no policy for exactly who needs to be included on this team, but we are definitely saying make it a team, not just one teacher, one administrator. We want to have a team so we're not putting anyone in the position of unilaterally determining this. It needs to be an agreed upon amount by a team, not one person.

So again, same example. Observe students, come up with an average, make sure that it's an agreed upon average, and then simply document what that amount of time is in your agency's local assessment policy. You don't have to write a book, you don't have to write anything scientific, but we are definitely suggesting you should give at least a little information about how you observed it, why you came up with the answer you came up with. Next slide, please.

Here's just an example where we're just listing off the modules, providing an exact amount of time for each module under sometimes that might be the same an hour for all 12 modules. A lot of times, it might be a different amount of time, depending on the specific module or specific exercises the student is completing. If you have a different amount of time for each, simply list it all in your assessment policy, specify the hours for each module, and then record those hours in TE.

You know what, in Penny's session yesterday, it talks a bit about providing some kind of assessment or exercises. What I'll add here is, in general, that's going to be for these software activities where you're just expecting the student to go through it. For videos, I'd say that's really what you're going to need to do because you don't really have activities. I'll add in these software modules sometimes you can just use what the software gives you.

Again, a lot of those software educational software that has hours also has built-in assessment and built-in activities. That is, the software won't allow the student to advance to the next module until that student completes all of those checkup activities to move on. If the student does that, you can use the checkup activities that the software provides instead of making up your own. Just simply define that you're forcing the student to complete those activities, and those activities that the publisher provides you is what you're using to check the student and approve the student to move on to the next level.

So general guidelines, contact your publisher for guidance on hours. Sometimes they provide a lot of information. Sometimes they provide nothing. If they provide nothing, we're still saying contact your publisher. There's a good chance they can help you out any way. Again, we talked a little bit about making sure it's a team approach and an average, not just the decree of one person at your agency.

If you want to do it on an individual-by-individual basis, again, that would be using teacher verification. Penny, Janice, Jay, we've all gone out of our way to say, that's fine. It's allowed. If you think that works better, 100%, A-OK, but we're saying it does require more work for you. Number one, it's more work on the teacher because that's suggesting that the teacher has a way to dutifully monitor all students.

Through distance learning, obviously, that's difficult. So there is a way in which you're videotaping the student while they're doing the distance learning so the teacher can observe, or perhaps you've got really, really detailed assessment or exercises that you're having the student do. Again, the teacher has some very detailed ways of monitoring what the student is or is not doing.

Bottom line is if you're doing teacher verification, it assumes that that exists somewhere and that it also requires you to document what sort of scenarios might require you to document high or low hours totals based on what the teacher observes. So it puts more on the teacher to be very, very vigilant. It requires more on the management team to document that detailed process in your policy.

Again, you've got to document that hour of instruction case-by-case basis. That's why the state and why we suggest that learner mastery might work a little more easily because you're not generally observing the students in that much detail when you're doing distance learning. Easier for you to define that ahead of time and just check up at the end whether the student did or did not complete the requirements and document it that way knowing that you're unlikely to be able to have that high level of observation.

Either way, document the process and the local assessment policy. Next slide. I think we're switching gears here. So back up a slide. I'm sorry. I thought we were, but I couldn't see. Sorry about that.

But in any case, the one last thing I'll say is this is where Part 2 comes in. We reference Part 2 a couple of weeks from now, May 5, May 7. This is what we're getting into. We didn't just make it up or please ourselves by this. We're definitely putting together Part 2 related to this. Document everything in the local assessment policy because that's definitely been where everybody's questions have come in.

So we've got at least one new session and potentially two. Of course, I'm biting off way more than I can chew, as usual, so that's a little bit TBD. But here's where we'll get into Part 2 and try to get into super concrete examples of what it looks like at the agency level, what types of instruction might relate to each module, kind of what Penny has done already to some extent, but taking it that next step of how does it look based on classroom instruction.

How does it look based on distance learning approach, and then based on that, how do you take that next step to accurately document that approach and that accounting for hours and putting it in your local assessment policy. That's what everybody has been asking. It doesn't seem like a lot, but there's been lots and lots and lots and lots and lots of new material that keeps coming up.

So anyway, a little bit on testing. It's the last section here. The short answer to tie it all neatly in a bow is that, again, back to that OCTAE memo, it clearly leaves it to the States to make decisions. The California State decision is still in progress. It clearly leaves it to test publishers like CASAS to get the show on the road with this. I'll implore you to believe that we definitely have, there's a huge field test for remote testing that we started just yesterday. We have it-- I'll just informally characterize it as phase 1, phase 2, that is we've got a process at CASAS that we've already started that basically does the pilot testing for remote testing.

We've got a phase 1, phase 2, phase 3. Phase 1 started yesterday. We're keeping it really, really cautious and conservative. That is that it will include only five or six agencies nationwide. It'll include just one-on-one test administration. Again, we're starting very cautious, keeping the low-hanging fruit approach. We'll do that for two weeks. Complete phase 1 at the end of next week. Around May 4, start phase 2 and open up to a larger number of agencies and open it up to a larger number of distance learning options, where we'll try to do a couple of agencies with group administration, not restricting the one on one.

We're a little unsure how long that will take, but hopefully, it will be short and sweet like phase 1. And then as we get into phase 3, we'll kind of get more into simulating the way we do it in real time, do a little piloting kind of match in the way we do it in real time for a certain amount of time, and then unveiling it to the public. But we're starting phase 1. We'll be doing a webinar that, hopefully, will include CASAS and CDE there around that first week of May as we start phase 2 of this pilot test.

We've had some discussions with CDE on the ups and downs of what we should and what we should not allow related to payment points. So when we roll out this approach for remote testing, everybody will be 100% clear as to whether this is something that will be playing for keeps, so to speak, where, yes, you will need to do this, and you should keep payment points in mind as you do this versus, yes, we've got all these wonderful options for remote testing and remote EL Civics, but it's not going to be for payment points. It's more like a fire drill here. We're closing the books on payment points.

I'm not saying I know which way CDE is going to decide, but we admit that no matter what we do, that webinar will have a decidedly different tone. One way versus the other way, we really need to get that decision 100% rectified before we can realistically do that webinar and expect it to not be completely sabotaged by that one question, obviously. OK, next slide, please. So that's what I just described, you can move to the next one.

So for now, we're saying for pre/post testing, for placement testing, and for EL Civics, an in-person proctor must be present. For pre and post testing, we won't really have that changing until we complete those couple phases of the pilot. I'll add though for EL Civics, for a few select COAPPs, as well as our citizenship preparation, citizenship interview test, we should have some more tangible guidelines here in a couple of weeks. With that webinar, we've figured out some things kind of in the weeds where that really requires CDE to weigh in before we can really disseminate any of this.

Just to be convoluted with the typical Jay Wright example, we know that EL Civics is different than pre and post testing. We know that theoretically, EL Civics, is a little bit lower bar, lower hanging fruit to do remotely than pre and post testing. However, if we're going to unveil that as a latest and greatest solution for collecting payment points. Given everything with remote instruction and remote testing, we realize that as is, it's a terrible way to solve the problem because any EL Civics COAPP payment point requires a pre/post pair to earn the payment point.

So obviously, if we're coming something to do that's a lot easier than doing pre and post testing in a remote format, there's a 100% chance of failure if we allow it for EL Civics if we don't make any additional policy decisions So everywhere we turn those, kind of things rear their ugly head, so we're hoping to be in lockstep between CDE and CASAS before proceeding. So some of that relates to policy CDE needs to make at the State level on whether we're doing any of this for payment points, and some of that is on us at CASAS to get a little bit further ahead on remote testing and remote EL Civics. So OK, let's move next slide. I'm not seeing anybody kick it to a curb.

So again, for now it's still present, and in a couple of weeks, we'll be doing a webinar for placement. Same thing, we'll be covering this as well. But again, I'll just reiterate for now, you can do things like oral interviews and writing samples and all those other things in lieu of CASAS locators, CASAS appraisals. Next.