MELINDA HOLT: Our first presenter Doctor Carolyn Zachary from the adult education office at CDE. 

CAROLYN ZACHERY: All right, thank you very much, Melinda. Welcome to our webinar. I hope that you're ready because we are ready to start remote testing. And I am happy to be here with the CASAS team. We'll be giving you all of the details. I'm giving you a real high-level view you right now, and that is the fact that we are going to be instituting a remote testing policy. 

If you are going to be doing this, you will need to amend your local assessment policy. And that local assessment policy is going to need to identify what tests you are planning to administer in a remote setting. Included in that will be an assurance that your agency is going to follow all of the remote testing requirements and an assurance that all of your staff are trained to administer those assessments. This is following OCTAE guidelines, which are part of our NRS reporting requirements. 

So this will include that you have some way or process for identifying the student to ensure that your proctors are completely trained for procuring the test remotely, that you are also ensuring test security, and that you are providing live proctoring. A templating guidance will be available for agencies to use by the end of the week you may notice that in TOPSpro Enterprise, known as TE, that there is an optional box for students that says force majeure, not able to test. This is what we will be using as we provide you the guidance-- so don't do it now-- but as we provide you the guidance for what you do if you are not able to do remote testing with students. 

The reason we're asking you to do this is because OCTAE is very interested in the data that will be coming from states with those numbers of individuals that weren't able to test. We'll be including this information in our narrative report to OCTAE. 

OK, I know these two words cause a lot of anxiety for everyone. So I want you to all breathe. And I want you to be aware and know that the adult education office knows that you are very concerned about your payment points and that you're concerned that COVID-19 is going to dramatically impact the work that you're able to do with your students and that results of all of the hard work, known as payment points. 

We will be evaluating all of the data that comes out of this after it is certified by your agency. And that doesn't happen until August. Redo your certification. So we will not be evaluating this data until the fall. I want you to understand that the data from the 1920 year is used for your 2021-22 grant award year. So it's not used for your grant award that's coming up here in July. We really want to evaluate the data very carefully and ensure that agencies are not dramatically impacted by COVID-19 and the inability to test students. Because we know that this is an equity issue across the state. 

What we really want you to focus on right now is on your students and using testing to help your students see their growth as well as your teachers so that your teachers can see the growth that the students are making in a remote setting. So we encourage you to use it for those purposes, relax about it for payment points. We will be focusing on those in the fall when we have the data, which is what we really need to have to be able to look at all of this. 

I would also encourage you, those of you that have issues with connectivity because of the remoteness of where your agencies are located, to look into the RACHEL device. This could help with your connectivity. And I've asked Penny to put information about this in the chat. And it is from the World Possible Organization. And this may help you in your remote setting. And with that, you're welcome to email me any questions that you have. So with that, I am happy to hand this over to Pat Rickard, president of CASAS. 

PAT RICKARD: Thank you very much, Carolyn, and thank you all for joining us today to learn more about remote testing for the NRS-approved CASAS pre and post test. We're very excited to share with you what we've learned, what we're doing, and what we plan to continue to do to support you. And we really thank those of you that have willingly agreed to pilot test and have given us some really great feedback. 

Many of you joined us last week on Thursday to learn more about remote testing for the COAPPs and CIT. And I want to put this session today in the context of the other two. I'm looking at it as three different levels of remote testing. A first level would be our performance-based COAPPs. And those are developed by the local agency, and you have in place trained staff. And the security on those-- definitely, we want security, but it's something that is within your agency and not across the state or national. 

The second level would be our performance-based citizenship interview test. It is standardized for use across all agencies in California that are offering English language citizenship preparation, civics citizenship preparation. And we have trained set assessors throughout the state in your agency, and we talked about simply providing some additional training for them in order to administer the CIT. 

The third level, which we're going to be talking about today and Linda's going to be covering, is our nationally approved NRS-processed pre and post testing, which is multiple choice. We will also cover the EL civics government and history test, which is multiple choice. Each of these three levels, the COAPPs level one, the performance-based CIT, level two and the NRS pre and post multiple choice level three each have separate guidelines and each have separate go-remote instructions. So I just wanted to clarify for those of you that joined us last Thursday that what Linda will be covering today will be the pre and post testing. And Linda, with that, I'll turn it over to you. 

LINDA TAYLOR: Thank you, Pat. Hello, everyone. Glad to be able to present with you today, and thank you for taking time to come and hear about this. Next slide, please. Yes, here's our agenda. And as Pat mentioned, we are focusing today on the multiple choice tests, the NRS-approved pre and post tests, and government and history for citizenship. 

We're going to give you an overview, some background about remote testing and how CASAS is approaching it. We're going to tell you about our clinical tryout and the pilot phase that happened mostly in April and then get it to what's happening starting today, May 4th, where we're moving into phase two, still piloting but also rolling out four agencies that have been piloting and rolling out different approaches that I'm going to describe. 

Then also, we have a phase three, which is, you'll see. Then, we will tell you about next steps, where to find all of the documents and supports that we're going to be sharing with you today. And then at the end-- and actually, I'll be stopping along the way and pausing to see if there are any burning questions that haven't been able to be answered in the chat box. As there are so many people on the call, we may not be able to get to all of your questions in real time today, but we will compile them at the end and post that on the website. So I hope to answer everyone's questions at some point, if not today then afterwards. 

We'll get into an overview now that is going into a little bit more detail about what Carolyn introduced in terms of the Department of Ed. The Office of Career Technical and Adult Education, OCTAE. they put out a memo on April 17 that stated very clearly that states have flexibility at this time with this disruption caused by COVID-19. States may allow local programs to exempt students from pre and post testing, those that are still enrolled doing distance education. 

And also, OCTAE, the federal government wants to know what the time frame is for that in your programs and who or the extent of those exemptions. They know you're not going to be able to test everyone under the circumstances, and they would like to have an idea of what happened. 

As Carolyn mentioned, there are certain guidelines that they put out that are to be followed when doing remote testing whether with CASAS or with another test publisher, and these are in these very broad areas of making sure that the student identification is authenticated, making sure the test security protocols are in place, and making sure that proctors that are administering remote testing have been trained and know what to do. 

On April 9, OCTAE met with all of the test publishers, and CASAS, of course, was there. And then about a week and a half later, CASAS met with OCTAE, presented them with a plan to roll out remote testing. They have given us permission to do that, and we were happy to hear that they were quite impressed with our remote testing plan and with the speed at which we were able to pull that together. So that is something that has given us-- made us feel that we are supported by OCTAE, and we are looking forward to rolling this out with you. 

We did conduct a webinar on the 20th of April with state directors in 30 states and told them what our plans are. We have actually had pilots going on in the last few weeks in probably 15 to 20 states, including California. So it's really being tested by programs like yours all over the country. We're hearing how it's going, and we're making adjustments in very real time as we go. We've been updating the guidelines pretty much daily or even sometimes more than that to really make it as streamlined as possible. 

So what are some of the benefits of doing remote testing? One certainly is that you're able to continue providing services in terms of assessment. You're able to post-test. Now of course we know that you're not going to be able to test all of your students. We really want to make it clear that that is not the expectation. 

But during this time, we hope that you will take advantage of it to try out remote testing. Post-test perhaps selects certain students that are priority or that you would like to see how they do with remote testing. And you'll also, of course, be able to see how they did in their coursework. 

If you have incoming students, and we know you do have a lot that are recently unemployed, therefore might have more time than they had before to study and taking advantage of that, we know you would love to get baseline scores for those students. And so if you choose to do remote testing, you would be able to get those using a test remotely as well as use these assessments to guide where to place these students in distance learning. 

We really want to emphasize that this time also gives you the chance to develop your agency's capacity to learn how to test distance learners remotely. And when all of this is back to at least some semblance of normalcy, that will add a lot of flexibility to your assessment program so that what we envision is that all of our assessment programs will consist of both a face-to-face component as well as a remote testing component so that you'll be able to test students that have difficulty coming in to test at your test centers. 

So it looks like remote testing is the wave of the future so that we can do distance learning as robustly as we've now learned how to do or are still learning how to do. And we actually are very much in a learning mode but feel that we're improving day by day in terms of understanding what's possible, how to do it more efficiently, more cost effectively, and with additional types of technology. So we do know that we're learning together and look forward to that. 

Now challenges, I'm sure you all could fill the chat box with all the challenges that you've already encountered with distance learning, getting that going. One of them, first and foremost, I'm sure is the students' access to technology, the equity concerns related to that, the digital divide that has come into such sharp focus these days. When you're going to consider doing remote testing, you'll need to have certain devices for proctors and students, and I'll be giving you details for that shortly. 

Smartphones are too small, as you, I'm sure, can understand for the test items that we have in our NRS-approved approved tests, that they would not be able to see them well on a screen, and that would mean that, in turn, they would not be able to perform well on the test. So unfortunately, although most of our students do have smartphones, that is not a possible device for the pre and post testing. It can be used-- if you attended our session on Thursday, it can be used for or the co-op assessment deliveries. It can be used for the CIT, but not for pre and post testing. 

In terms of internet bandwidth, that can be a challenge as well, with Wi-Fi being much in demand in a given household, there could be kids trying to use it at the same time as they're adults working from home trying to use it. So we know that our students may have a challenge getting access to the bandwidth that they need, so that's something that will have to be planned well ahead and, hopefully, dealt with in a way that will work out. 

Also, in the students test environment at home, they may not be able to get a private place to test or it may be challenging, many distractions with others in the household. So here, again, there are many challenges that we're well aware of. And I will say now and I'll keep repeating to really reassure you that all of these challenges are things that we have considered as we are writing our guidelines. 

So whether it's for CIT, or COAPP, or these pre and post tests, the remote testing guidelines that we are going to be posting that will require you actually to study very carefully before you embark on remote testing, they have very detailed instructions-- step by step, procedures, scripts, here's what you do, here's what it looks like. We're going to be able to tell you more later, but we're planning to do some follow-up sessions with videos of what settings in certain types of platforms. So we really are aware that you are going to need that kind of very specific help, and that is provided in these guidelines. 

In this webinar today, we don't have time to go into all those details. I'm going to get into some level of detail, but please be assured that these guideline documents have very, very specific instructions. So it is going to be somewhat of a challenge for you and your program staff to learn and implement these new processes but, based on our pilot testing, with a little help from CASAS and a little practice, we're sure you're going to be able to do a great job. 

We know that your doctors have been trained to give eTests face to face, but they have not been trained to do that in a remote manner. So we are relying on you to ensure that they have read the guidelines carefully, that they take advantage of the training that will be posted as recordings on the CASAS website, and that they are ready to go when it's time to go. 

Test security is extremely important. And we're not going to be able to make it airtight, but we hope that we can close up as many loopholes as possible, that we really are going to count on the goodwill of our students to some extent, but we are putting in place as many precautions as we can without making it too cumbersome for you or the student. Equity concerns have already been mentioned to some extent with the technology, but there may be others. 

So as we researched all of this, we looked at some third-party vendors. There are lots of them out there, and we met with many of them. What we found is they're expensive, little bit steep for our adult ed budget. It's $15 to $20 an hour to have a live proctor and a third-party vendor. So that's not what we went to first. 

We may reach out to them and perhaps have-- be able to crack a deal that is affordable at some point, but at this point in time, we are not offering that kind of solution, nor an artificial intelligence solution at this time. And we looked at a variety of video platforms. 

We talked to state directors and their staff. We talked to a lot of folks who are familiar with distance learning to learn what we could from their experience with that. weighed multiple factors. Cost, which, in terms of the technology to use for remote testing, we think is fairly minimal. 

However, there is a cost in terms of staff time. And that is because, with remote testing and the need to be able to observe the student in real time, it's necessary to limit the number of students that can be tested at one time. And so it's between one and five we are allowing at this point. If we find we can allow more, great, but we understand that that is going to be labor intensive and, therefore, costly for your programs, depending on how many students you decide to include. 

Again, though, I want to emphasize that this is a time when you're trying this out, and you are not obliged to do this remote testing. But if you do want to try it, it will be more labor intensive because of the high ratio of proctor to-- just one proctor to only a maximum of five students. So we understand that. There's really, I'm sorry, nothing we can do about it, but that is the situation at this time. 

We know that there may be technology constraints, both for proctors and students, which perhaps you've already been working to alleviate when you've done distance learning now by loaning out devices. So perhaps that's a way that you can work this out as well. Test security is something that we'll talk about. It takes somewhat different forms with remote testing. 

Ease of implementation, we've tried to make this as easy as possible, though it does take a bit of doing in terms of understanding how to set up the video platform and et cetera, all the different steps, but it's possible and doable. And we know that many of you are very eager to get started, so we've tried to address that urgency by making this available on a very short timeline compared to how we normally develop new types of assessment. 

In terms of the delivery, it will be live proctoring, so that's in real time, synchronous. There's something called a "remote testing agreement," where we'll want to have one per agency. And that will cover all of the different remote assessments that are possible. There is an agreement for proctors that you will have them fill out and keep on file at your local program. And then students, this is not something we've had to do before but because they're at a distance and not under our watchful eye next to us, we will ask them to verbally agree to certain test security promises. 

In terms of delivery. it is on eTests, so we need you to use certified eTest proctors who've already been-- ideally, who already have experience using eTest, so they must be certified proctors. By the way, there's no certification for the eTest administrators, just the proctors. And these proctors will need additional remote test training as we just said. 

We had originally talked about videotaping, but we have gone away from that. After thinking it through a bit more and getting input about it, we have decided not to require videotaping of these remote assessments. It's just a little too cumbersome and hard to figure out how to store them, lots of reasons. So that'll make it easier for you. It also could be a privacy issue. So no videotaping required. 

Also, just to be very clear, the CASAS remote testing cannot be delivered with paper tests. If you're doing remote testing, it can only be with CASAS eTests for the pre and post tests. Now in terms of COAPP or CIT, you may the conditions are different, and you still need to be able to see the student live, but there can be some paper involved, but we'll explain that-- that is explained clearly in the previous webinar, and we can clarify that. It's all laid out in the guidelines for COAPP and the CIT as well. But as far as the pre and post tests, no paper tests. 

Also, we know that, in some programs, they have figured out ways to do face-to-face testing, even during these times of the COVID-19 crisis. That could be in corrections programs. We know that some programs have got creative and have found ways to have students take the test in their car, getting a box that they come and pick up, and it's never-- they never touch anyone on the staff's hands. 

Anyway, there are ways, I guess, to do it, where you can observe social distancing and still have a student be doing it in person. So any situation like that, where you're giving the test face to face, essentially, the basic, normal rules still apply that a proctor must be physically present. So if you are doing that, the usual rules apply, and you need a proctor. 

So which tests can you use for remote testing with multiple choice tests? It's the same ones you've been using-- the GOALS series, Reading GOALS and Math GOALS for ABE and adult secondary. For ESL, it's life and work reading and the life and work listening, 980. And also, though it's not a pretest but it's a post-test really, the government and history for citizenship test, a certification test that is given to ESL citizenship students is also a multiple choice test and can be used with remote testing following the very same guidelines that we're going to tell you today for the pre and post tests. 

Now this slide shows the phases. And the one we have just completed is phase one, where we did clinical trials and piloting. And we restricted that to one on one, one proctor, one test taker, and pretty much used only Zoom because that made it a little easier to see what was going on. So that was, I would say, quite successful. We got a lot of great input, and we incorporated those suggestions into our guidelines. 

Starting today, we are moving into phase two, where we are going to be having-- if you're just starting up, we're going to ask you to pilot, that is not go full scale right away but start small-- pilot and then roll out, whether Zoom or another platform like GoToMeeting, Webex, or other ones. One on one is one option. 

Another option is practicing with up to five test takers, and that can be with various devices. I'm going to explain the details soon. And that will soon include Chromebooks. It does not include Chromebooks at this moment in time. That's something that we hope to be able to have available within a week or two. So we're talking weeks and not months. Very shortly, we hope to be able to make it possible with Chromebooks. But starting now, you may test up to five test takers with these other platforms like Zoom and GoToMeeting, for example. 

And then we are adding on the ability to do remote testing with the listening 980 series as well as the government and history for citizenship test. In the future, we are still interacting and exploring with third-party vendors, if we can possibly work something out to make that work. There are other options we're looking into that, if they can make this a more efficient and less costly proposition, we will certainly look into them and let you know as soon as we've worked anything out. 

All right, let's go on to the next slide then. So now I'm going to tell you in some more detail about the three approaches that we have come up with so far and that we're making available. The first one is one on one. And in this approach, the test takers have a variety of devices that they can use. That's one of the advantages of this approach. They can use a PC, they can use an iPad, or they can use a Mac. 

And in this scenario, the proctor needs to register their computer, and it has to be Windows 10 because the proctor is going to run eTests on their computer and then remotely share their screen with the test taker through the web conferencing platform. So that's why this can only be one on one because the Proctor is loading one test onto the Proctor's computer and then sharing that with one student. The Proctor and the student are connected and be able to see each other real time, using a mutually agreed upon web conferencing platform with a webcam. And they need an internet connection that is 2 MBPs or higher. OK, Let's go on to the next solution. 

In this one, it is possible to test up to five students; however, the, I would say, challenge here is that the test takers have to have Windows 10 PCs. So that may be a high bar. We know that. But we do want to make it available in case there is a chance that you can provide students with these or if some students do have a PC with Windows 10, then they could and you could also take advantage of testing more students. This is necessary because they have to be able to run eTests on their computer. 

So another aspect of this approach is that the test takers' computer has to be registered. And just want to let you know that for the [AUDIO OUT] remote testing, we are allowing there to be only one person, that is the proctor, to register the student's computer. Normally, we require two certified staff to do that, but for remote testing, we are waiving that requirement. And so the proctor would be able to register the students' computer. 

Then using the web conferencing platform that you select and the webcam, the proctor would be monitoring these one to five students. And I believe this is not a very fast internet connection. But again, given that it may be in competition with others at home using the connection, it might be challenging. 

So part of, again, these detailed guidelines, step-by-step instructions you're going to get, there is a link, and all the instructions for how to test the proctors and the students' internet connection speed. There are a number of web sites that do that, and that's something that you'll have to do on the day of testing. You can do it before to double check but also-- I mean, to see if it's even close, but also on the day of testing, and this is in the guidelines, you'll be checking their internet speed to ensure that they're going to be able to have passed enough strong enough internet connection to do the job. 

The third solution is coming soon, and we know it's eagerly anticipated, and that is with chrome books. Because we know that many of our students either have a Chromebook, or their students were given a Chromebook, or you can give them a Chromebook because they're so affordable, and so we are moving as quickly as we can to make the Chromebook solution available. And in this solution, the test taker would register the Chromebook on their Chromebook that they have at home. You would help them set it up, register it, again, just only you and not two people. 

Then, like in solution 2, the proctor would monitor using a live web conferencing platform, and you can do up to five students, but the difference here is that with Chromebook, you can only run up one application at a time. So in this scenario with Chromebooks, the test taker would have to use their smartphone with a video cam. And they would set it up on the side so that the proctor would be able to see their screen and to see the test environment while they're taking the test. So that would be how we do a workaround with Chromebooks because you can't have the web conferencing platform working on Chromebooks at the same time as eTests. 

By the way, one thing that is, again, in these very detailed step-by-step instructions is that we asked the proctor to do a sweep of the testing environment. And this is something we explained to the student beforehand. We encourage you to have a session before the testing day or before the test session to explain to the student so that they can be aware and anticipate all of these situations that would be new to them, including this requirement that the proctor will need to scan the room. The student will have to scan the room so the proctor can see. 

This is to make sure there are no people there who could be giving answers to the student, no inappropriate documents, papers, aids that might help the student answer questions. So it's a test security measure. We know it could make students slightly uncomfortable, but we hope that you'll have a way to explain to the students that this is necessary for test security, that it's only for that purpose. We're not looking at anything else in the room, just looking for that. And this would be done before and after the test session. Or if the proctor senses that there is something amiss, the proctor can also ask the students to do that any time if they're concerned. 

And there's also instructions in the directions that if a proctor sees a reason to be concerned, then the proctor can interrupt the test at any time and try to see if there is a real concern, in which case they would stop the test. And then, as I think you all know, you were able to start the test up again if appropriate or in the test and in the session. 

So this is a mock-up version of these guidelines, just to let you know they do exist. They're quite detailed, about 20 pages, with examples and scripts, et cetera. So there is something called an "agency remote testing agreement." This is a form that will ask every agency that intends to do remote testing to fill out one per agency. It will cover all of the possible tests that you can use for remote-- for EL civics or pre and post. That would be-- the CASAS Center, NRS-Approved pre and post tests, the COAPP assessment deliveries, the CIT, or the government in history for citizenship. 

We will ask you to tell us how you're going to verify the student identity. And in that, we're going to ask you to use the procedures and the ways that you normally do that in your local program. So nothing will change there, but you may have to figure out some ways to do that remotely that are different than you normally do it perhaps. And then another aspect and very important aspect of this agreement is that you attest to having read these remote testing guidelines thoroughly and that you agree to abide by them. 

So now, we're going to go out of PowerPoint and into the actual documents so you can this agreement, and I'll show you a little bit more detail that's in this remote test center agreement. So OK, yes, so first, it says he read the guidelines, then the second paragraph down there is that we would like you to stay informed because we're releasing the version 1.0, but it's very possible, likely I would say, that we will be updating that with additional refinements and improvements. So we will let you know when we release new versions, and they'll be on the CASAS website. We're going to have one page dedicated to California remote testing on the CASAS website, and that's where you'll find all these documents, all these resources and the trainings, including any updated versions of these guidelines. 

And again, to reiterate, we're not asking you to record, in fact, we would like you not to record because we would not like these recordings to get into the wrong hands, so we thought they was really possibly too much danger in recording because you would need to then ensure that the privacy requirements would be maintained also for test security. 

All right, so then here, we would have you list which test series you're going to use by checking off which ones you plan to use. And we're aware that when you fill this out, you may perhaps decide to use certain of these tests and not others. So if that changes over time, that's fine. You can work that out with your program specialist and your CDE regional consultant to see if you want to revise this, but just please do the best you can at the time when you want to begin. We know it may change. 

Then, we want you to indicate which of these approaches, it might be more than one, that you're going to be using in your agency. So are you going to have the proctor have you test on their computer, their Windows 10 and have students have more options for the devices? That's number one, one on one. Are you going to opt for a solution where you can test more students at a time but they need a Windows 10? And if we can get that changed at some point in time, we will let you know. It would be nice. We know. But at the moment, that is the requirement for this option number two. 

And then option number three, which if you're going to fill this out this week, you would not be able to check this one because it's not quite available yet. As soon as this Chromebooks option is available, we will definitely let you know, and then you'll be able to use it. So just to reiterate, Chromebooks is not currently an option as of today, but in the next week or two, we expect to be able to have tested out all of that whole process and have good, clear guidelines for you to use Chromebooks as well. 

All right, then just briefly to share with you about the COAPP assessment deliveries and how this is also in this agreement, so if you're going to venture into delivering the COAPP assessments remotely, we want you to list which ones. We want you to briefly describe how you're going to adapt them, if you need to, for remote testing. 

And then on the right, we've listed from low to high tech some options for the technology. You may think of others. So you can certainly add them to this document when you submit it. But this will give an idea of what you're planning in terms of the assessment deliveries of your civic objectives. 

And then, for CIT, if you're going to do remote testing, again, it's optional, but if you decide to, you can use-- if you happen to have the printed CASAS paper set test booklets, either you brought them home or your agency can get them to you, that's perfect, and then really nothing changes, as you would normally-- it's a consumable test booklet, so as you're giving the test, as you know, you write the 012 rubric responses right in the test booklet, and that's how you administer the test. 

If you don't have access to one of those paper booklets, we have created, and this is new, a fillable PDF version of the form 973 and 974. They're not quite yet available, but they will be available by next Monday, the 11th of May. That's when all of these things will be on the CASAS website. So please have a little more patience. Were almost ready to roll these out. But when that time comes, those booklets will be available for you to use if you don't have the paper booklets. You might decide to use both, depending on what you're CIT administrators have in hand. 

And then for CIT remote testing, there's just two options for the technology and platforms, and that's smartphone with the video camera or computer with the webcam phone. Then we already talked about the identity requirements and then when you're going to start, estimated, is what we want to know. 

OK, back to the PowerPoint. So then the proctor remote testing agreement is something that will also be on the CASAS website. It will basically have each proctor attest to the fact that they've read the guidelines, that they will ensure that the student verbally promises to also abide by the these agreements, not to cheat or-- basically to follow all the test security rules. And this agreement, if you have proctors that do more than one assessment, they just have to do one of these forms, and it's something that you keep on file locally with your local assessment policy. 

Now we're going to get into the remote testing version of the Going Remote! Checklist. We've had this type of thing for eTests onboarding called Going Live! with eTests, but this is the going remote with eTest process. So the first thing, and extremely important, is that your agency has already implemented CASAS eTests. And if you haven't already, but you do want to try this out, you can, during this time, take advantage of the time to go through the certifications and get all your staff certified and get set up in eTests. That's something that our tech support and other staff if available to help you with, even during these times. So please feel free to seize the opportunity to do that now if you're not already on eTests. 

You'll need to complete that agreement we just talked about, the agency-level agreement. You'll need to make sure that the proctor has the equipment necessary to implement the scenario, the approach that you decide to use, or approaches and that, ideally, that they're experienced in eTests so that they will have a little bit of an easier time doing this. You'll have them complete that proctor remote testing agreement. 

Then, you'll make sure they know how to access the training materials on the CASAS website. They'll be the recorded trainings. There'll be some YouTube-type how-tos. They'll be FAQs, there'll be the detailed guidelines step by step, there'll be also a quick start, like a short version of that, that once they get good at using the detailed script, then we'll have, like we do for eTests, the steps for testing day, kind of document where they can have a shorthand version of that that they can refer to easily. Now the training is not required, but it is very much recommended. So it's optional, but we do highly recommend that you have your staff do this training. 

Then, as far as your test takers having the technology they need or if you need to help them by providing them with some of the technology in order to make this happen, registering the test stations is something that the proctor needs to do with the one-on-one solution where they share their screen or that the proctor needs to do for the student in the second and third approaches, where the student takes eTests on their own computer. 

Then, you're going to review and select the appropriate test sessions. And you really want to be ready with those at the time of testing so you're not holding up the process as you go to find those. Very important, step 9 is to conduct a trial run or more than one trial run with colleagues, not with students, but to try it out and make sure that you're comfortable with the process, you really have thought through what might happen and how you're going to guide the students when you actually work with students. And finally, once you're comfortable with that, then number 10 is go remote. 

So now, we'd like to go out and just show you the slightly more detailed version of this to see that it does-- and we are going to be adding to this-- you can see on the first one, we've added the link to the Going Live checklist for getting started with the eTest itself. So we'll be adding links to this document so that you'll be able to easily find those documents on the CASAS website. That would include the second one, the agency agreement, the proctor agreement, and number three, if you scroll down a bit, you'll see that there's a little bit more detail about the requirements for the proctor. They need a webcam, of course. 

Then, the training again, will have a link to that and to the various possibilities for training. Step 5 is list all of the different variables and alternatives for test takers. So there are a lot of different ways to go, and you'll have to see which ones will work. Again, I want to apologize. 

I know it's frustrating that it's limited to only five students or, depending on what kind of technology you have, just one on one. And we know that you're not going to be testing your usual 25 students in a remote proctoring mode. It's simply not possible. But we do hope that you'll take advantage of this opportunity to learn how to do this and prioritize the students that you think would most benefit from doing it now or that, for whatever reason, I think you'll have to talk internally about which students you decide to do this with and which teachers, which classes. 

Register the testing stations, I think we've talked quite a bit about. And I think the other steps we've already covered. Again, the training tools, these guidelines documents are very detailed. FAQs will be forthcoming and will be ongoing. We will keep adding to them as we get additional questions from the field. 

And in terms of the training webinars, there will be an initial one that will be up on the website no later than next Monday, and we will also be offering the subsequent mini-webinars on different topics. Those will be rolling out starting next week to follow up on things like, how do you do security settings? What does that look like? Or, how do you do the sharing part? Or, what should you look for when you're monitoring and proctoring? 

So there are quite a few topics that we know you're going to want a little bit more guidance on, so we're going to be creating short, how-to videos and short mini webinars to help you get more comfortable in different modalities for how to do these remote tests. And there will be these not only for the pre and post testing but also for the COAPP assessment deliveries. I'm sure we're going to have a lot of really creative ideas for how to adapt the assessment deliveries for the civics objectives for remote testing. 

And then also, we'll have a whole set of materials also for the CIT. Just to kind of re-integrate that in terms of the preparation, this is so important that before you're actually giving the test, we'd like you to meet with this test taker. It could be the day before. It could be the same day. Confirm with them that they have the technology they need. 

Make sure that they're aware of the test procedures so they're not surprised by anything. Answer any kinds of questions to make them comfortable with the process. Especially help them understand that there will be somebody watching them, and that can be uncomfortable, but if they're aware of it, then hopefully they'll at least be prepared for that situation. 

And then on the testing day, although you've done things like actually beforehand check their ID and however you do that locally and then an environment scan of the room just so they understand what that's like. You still do need to redo that on the day of the test because, of course, that could be different. Also, in terms of checking the speed of the internet, all those things you have to do again on the day of the test unless you do that preparation just before giving the test, which is possible-- easier to schedule but maybe not quite as comfortable for the students. So that's up to you to see how you're going to schedule that, but you do need to be sure that when you're actually delivering the tests that you confirm everything that you had practiced and explained about before. 

OK, then this is a screenshot that shows the proctors computing using the one-on-one approach, where the proctor has eTests on the practice computer and then is sharing their screen with the student. You can see the webcam, where the proctor can see the student, you can see the Proctor. This is also something I wanted to mention, that we have the eTest sampler, which is on the CASAS website. It's free. I hope you're all aware of that. But if not, please check it out. 

It's practice items in all the modalities for all the test series. It's called the eTest sampler, and it has all of these tests in the eTest format. And these are the practice items for reading in the eTest format. So this is something else you could do in the practice session with the student is to familiarize them. If it's the first time they're taking a test with eTests, show them the eTest sampler, show them how it works, how you navigate through it. Because they're going to be taking the test on their computer. They're going to be controlling the arrows and progressing from one item to the next. 

When the screen is shared by the proctor, it's actually the student taking the test on their own computer. Even in this one-on-one mode where the proctor has the test on their computer, the student is taking it on their computer or on their iPad. So especially, I want to say that if they're taking a math test, it would be really helpful for the student to see, if they haven't already taken a math test, how its done any tests, including the online calculator and other types of things. 

I do want to mention that one of the recent decisions that we made is that we are going to allow for math only. We're going to allow the student to use scratch paper and to use a handheld calculator if they prefer. They can use the online calculator, but they could also use a handheld one. We're going to restrict this, but we realized it's just too difficult to take a math test without scratch paper. 

So the requirement is that you'll have to make sure that that's a blank piece of paper when you start the test and see what that student has there at hand. And at the end, you'll have to make sure that the student tears up that paper in little bits in front of your eyes to make sure that it doesn't go anywhere else after the test. So these are some of the things that are-- this is a good example, I would say, of the kinds of things that are listed in great detail in the guidelines. I see a question here about, what does it look like when you're delivering the test with more than one student. 

And I don't think today we can go through that in detail, but what I can say is that in the guidelines, it does go through step to step how you work it out in Zoom, for example. They have what are called breakout rooms in other platforms. I think they call them private rooms, where you can have the students altogether. Then, you can take the students one by one into a breakout room to check their ID and work it out in that way. So we have that all detailed out in the guidelines. And, yes, that's how it would work. All right, next slide, please. 

So one of the things that is coming soon is that we are going to be able to-- we need to be able to identify which tests are given remotely and which tests are given face to face, and we will very soon find a way to embed that in the eTest software so that you'll be able to indicate that. That will enable us to compare student performance on remote as compared to in-person testing. We certainly hope that with good preparation of the student, that they'll be relaxed, and we'll be as well as relaxed as you can be taking a standardized multiple-choice test, but that they will be able to do their best and have comparable performance as they would on an in-person test. 

I mean, they'll be in the comfort of their home. Perhaps, who knows, maybe they'll do better. But we do want to see. And also, we as we always do, we have ways of reviewing test security issues in the back end, where we can see, for example, if a test was taken very quickly. That can be an indication of guessing. There are other data-type indicators of test security problems. So we will continue to do those sorts of checks as we roll out and ongoing as we do remote testing. 

As was mentioned in the beginning, there is a field, a new one, TOPS for enterprise called "Unable to Test due to Force Majeure." This is where you will be able to indicate students that you were not able to test, because we know that there will be many of those. And maybe I'll ask Carolyn to pop in right here and just comment on this because I want her to be able to tell you the guidance for this. 

CAROLYN ZACHERY: So the guidance for using the Unable to Test due to Force Majeure will be coming out later this week coming from both the CDE and CASAS related to remote testing. We do have to work on a policy for this that's in development, but we need to-- we'll need to get that out to you later this week. 

LINDA TAYLOR: Thank you. OK, now we can move on to some next steps, just to make it clear where, again, kind of-- I think we've talked about this throughout, but I wanted to pull it all together by showing you by when and where you'll be able to get all of this, where is on the CASAS website. 

There's going to be a page that's devoted to California remote testing on the CASAS website. And by next Monday, May 11, we intend to have the following materials there. One is the remote testing guidelines. There would be the document for pre and post testing and government and history of citizenship. There will be other documents, other remote testing guidelines documents for assessment delivery of civics objectives for COAPP and another remote testing guidelines for the CIT. 

Through the Going Remote checklists, there will be one for multiple-choice tests for COAPPs and CIT as well. So there'll be different ones. The one we saw today is for the multiple-choice tests. Those will be there. The agency remote testing agreement will be there as well as the proctor agreement that you'll keep locally on file. The FAQs will be there in, and we'll be addressing FAQs for all the different assessments. And then the resources for training will also be on this website. 

As far as today's webinar, the recording, the PowerPoint, and the Q&A responses will be on the OTAN website. And in just a minute, going to turn it over to OTAN so you can see exactly where that will be, which I think you are familiar with the COVID-19 page. And also, on the CASAS this website at this link is where we will have the recording of this webinar. 

If you have technical questions, you can, as always, contact tech support at CASAS.org. If you are interested in pilot testing with remote testing, we would very much welcome that. And if you would like to do that for the pre and post testing, or for the government in history, or for COAPP, or the CIT, please contact your program specialist, and you can talk about it with them, see if it's something you really want to do. And if so, we can go forward and help you out to get started. 

Well, in that case, I'll ask Pat if she would like to make some closing comments. 

PAT RICKARD: Thanks, Linda. I just wanted to reaffirm that, at this time with the COVID-19, trying to do remote pre and post testing using the CASAS assessments is not required. This is a time when you might take the opportunity to get staff trained, to actually try out and practice within the agency, and to build capacity. And so we really encourage you to look at the long-term outcome, which is building capacity of staff in order to, on an ongoing basis, be able to serve students, both in-class students and remotely with following up with COAPPs, with the CIT, and with the pre and post testing. 

And we are all very much aware at this time that the students lacked-- many of our students lacked the technology in order to do this, and I think Carolyn may talk about and has talked about the CARES Act and the ability to apply for funding for the technology. So we really encourage you to think about this as a longer-term capacity-building opportunity. And we're more than willing to share with you what the pilot sites are finding out. 

We're going to be sharing best practices going forward. We're going to be putting recordings up for the training that are available 7/24 that your staff can access and learn more about the remote testing. So we really thank you very much for-- on this very interesting journey with us, and we are going to be working with you now and into the future so that we can really make this work for all of our students. So thank you very, very much for your participation today.