Speaker 1: Dr. Carolyn Zachry, the director of adult education at CDE.

Carolyn Zachry: Good afternoon, and welcome to my remote outside workspace. It's a lovely day here in northern California. So I opted to move my workspace outside. I'm battling a bit of the pollen, but it's nice to be outside. So next slide, please.

I think you're ready as we move into this era of remote testing. A couple of the areas we want you to make sure that you understand is that we are following all of the NRS guidelines that they have laid out as it relates to testing, as well as OCTAE's guidelines and their requirements. If you do decide to move in to remote testing this year, you're going to need to amend your local assessment policy.

And that's going to need to include the use of a remote testing agreement form that will identify the tests and the COAPPs that you're going to administer remotely. This would also include an assurance that you will follow all of the remote testing requirements, and that all staff are trained to administer remote assessments. A template will be available for agencies to use by the end of next week.

Now, we do know that many of you may not be able to do some of the remote testing. And so in TE, you'll see that there is an area that can be checked called unable to test. Don't check that yet. This is going to be used probably at the end of this grant year.

And we'll be developing a policy as to how that is going to be used for all of our agencies. We know that OCTAE is very interested in this data, and we'll be including those numbers within our annual report narrative that is sent to OCTAE. So that they can see how the impact was on California. Next slide, please.

All right, payment points. I know everyone's interested in payment points. So the first thing I want you to do is breathe.

Take a deep breath and relax. We know at CDE that this is a big concern. We don't have the right answer just yet because we haven't looked at the data, or know the true impact of what COVID-19 is having on our adult agencies in California. We will have a better idea of that in the early fall.

Once you have submitted your data to CASAS and it's been certified, we'll be able to sit down as an office and look at the data. We'll look at past data from every agency. We'll look at trends, and then we will make a decision. Our goal is to not have COVID-19 negatively impact any agency because you weren't able to have students be tested.

However, as we move into this idea of remote testing, for those of you that go down this road with us, I really want you to focus on your students and how they're succeeding in this new learning environment. And use the remote COAPPs and the CIT test. And then on Monday, we'll learn more about pre and post-testing.

Use that to help your students see their growth. And so that teachers can also see the growth so that they know that their students are learning in this new remote environment. Next slide, please.

I do you want to remind you that on Monday, we do have two webinars similar to today that will be focusing on CASAS pre and post-testing. And also, I do know that in our remote areas, doing online school in a distance learning format is challenging. And so Penny is going to be putting into the chat information on RACHEL devices that can help with connectivity. And you can use these with the COAPPs.

We're not sure yet about pre and post-testing. But this could be way for those of you in a remote area to be able to have students complete their COAPPs. So look for that information in the chat.

And certainly, send me questions that you might have. I'll be helping to monitor the Q&A section as well. And now I'd like to turn it over to Pat Rickard, president of CASAS. Thank you, Pat.

Pat Rickard: Thank you, Carolyn. We want to thank you all for joining us this afternoon to learn more about remote testing. We're very excited to share with you what we have learned, what we are doing, and what we will continue to do to support you in the implementation of remote testing. We do have more to learn as we go down this pathway to remote testing.

We learn new things every day, and we're learning a lot from you. We really want to thank those of you who very willingly agreed to pilot test and provide feedback for us. And Linda will tell you more about additional pilot testing if you're willing to do it.

I do want to say that this effort is not just for COVID-19 and that will all go back to quote "normal" in the new year. I think that the remote learning, remote instruction, along with remote assessment, is the new normal. I think that what we're doing right now is we're building capacity for now, but also in the future. We will have the capacity, then, to serve more students and serve those that may not be able to come to our programs.

So this effort during the last quarter of this year is not just an effort between now and June 30th. If you really think about it, it's an opportunity to build capacity at the agency level, the program level, and with staff to be able to provide instruction remotely, and also to test remotely. So I really wanted to emphasize the capacity building opportunity that we have.

With that, I'll turn it over to Linda. And Linda's going to give you more detail for EL Civics, both with the COAPPs and the CIT. Over to you, Linda.

Linda Taylor: OK, thank you Pat. Hello, everyone. It's nice to have this opportunity to share with you what we're doing with remote testing at CASAS today. Let's go to the next slide, Neda.

So this is an agenda that shows what we're going to cover. Today, we're going to focus on EL Civics, the COAPPs, and CIT. Monday, we're going to get into the pre and post-testing and EL Civics.

So that is how we've divided it up. And I know you all are very eager to hear about the pre and post-testing, so I hope you can be patient on that. And there may be other things that we need to cover on Monday as well that come up in the chat box today, so we can see. But today, we're focusing on COAPPs and CIT.

We'll have a chance to answer your questions. We'll try to keep up with the Q&A box. But as sometimes, the questions can come fast and furious, we don't know if we'll get to all of them.

But we assure you that we will review all of the questions that came in. And whatever we get, we will create a document where we compile them all and answer them-- both the ones we've answered and the ones that we haven't. So please be patient with us if we can't get to everyone today. We're over 200 people on the call already.

So first, we're going to do an overview. And I want to back out a little bit to get a national view of this situation. And I think the most important thing to understand about all of this is that, in a way, we're all in this together, including the US Department of Ed Office of Career, Technical, and Adult Ed-- that's OCTAE. They realize that this is an unusual situation.

And they have very openly and clearly stated that they're allowing testing flexibility to the states. So they know that you're not going to be able to test every single student by any means. And so that is something they made clear in a memo on April 17, where they said that states may allow programs to exempt students from pre and post-testing.

And that's what Carolyn was talking about earlier, and that there would be a specified time frame for that. So this is happening as you well know. It's all over the country, and they're aware that you're not able to do what you normally do.

They also made some provisions and some recommendations for everyone to be aware. This was on their March 27 memo, that if remote testing is to occur with the NRS approved tests, they of course are standardized and must be respected as such. So there are areas that it's very important for those who choose to do the pre and post-testing once that's available to make sure that you follow the protocols for the student ID-- identifying and authenticating the student ID, anything related to test security. And that takes a lot of forms with this, but we'll be covering some of them.

And then training, because doing testing in a remote mode does require additional know-how. And that's something that is required for-- even though proctors may already be certified in our e-tests, or whether it's in COAPPs or whatever they may be certified for, they're not yet certified in how to do this remotely. And so we need to provide them additional training and support.

So more of this will be covered on Monday. But just wanted to give you this sense-- a feeling, really, of the national picture. And if we can go to the next slide.

On April 9, OCTAE met with all of the test publishers and briefed all of us about what they had in mind, which is some of what I just shared with you. And then following that-- not very long after, about a week and a half later, just CASAS met with OCTAE. And during that time, we were able to frame and develop a plan to provide remote testing.

And when we presented that to OCTAE, they gave us permission to roll it out. And actually, they were quite impressed with our plan and our timeline. So we were pleased about that.

And I would say there's been a considerable amount of work that we've done. It's been a lot to learn in a short period of time. And it's an opportunity to, I think, do something that we've all wanted to do, but we felt that it would be too challenging. But now we're put in a position where we really want to make it available. And we're trying to balance how we can do it quickly, and yet keeping the precautions that are necessary to protect the security of our tests.

On April 20, we pulled together the state directors from 30 states, actually, where they use CASAS. And we briefed them on our plan. And it is in phases, and we were able to explain to them.

And we have now, I would say, about 15 states or so that have been involved already in piloting. So they are eager and we're very excited that there's so much responsiveness to this. Let's go to the next slide.

So what are the benefits of remote testing? And here, we're going to hone in more on EL Civics. And some of these points were already mentioned by Dr. Zachry and by Pat.

But I'll just go over some of them in a little bit more detail. And it's basically good for students and good for teachers, because when they're doing distance learning, they still want to be able to see their progress and check the progress of learners. So to get started with remote testing is an opportunity to assess students who recently completed a significant chunk of learning. It gives them feedback. Both students and teachers get feedback.

And keep students engaged. They know they can know that at the end of their 30-hour block of COAPP instruction, that they're going to be tested. And I think often, they're very motivated to demonstrate that.

So as Pat mentioned, we really hope that you can very productively use this time to build capacity at the local level by establishing procedures for remote testing, by training staff, just thinking through how it can be done. I'm sure as you get into it, you'll come up with really great ideas about how to do it, and eventually even get more and more efficient at it. So now is the time to explore.

And just as you, I'm sure, learned a lot from distance learning and grown about that in the last eight weeks or so, this is now a time to put attention on how to assess. And I also imagine that many of you have been putting your toes in the water of assessment already informally with distance learning, whether with the materials you're using, or just teachers figuring out ways to monitor progress informally. So I'm sure that there are going to be a lot of great ideas already about remote testing. So this is a time to get prepared for the future and to really in a way redefine our programs as blended, and doing as much distance learning as we can as a regular part of the program's offerings.

We've been very pleased that we've had a lot of interest from agencies that want to pilot. And so that's really helped us to develop and frame the guidelines that we have been creating. And just to say that I don't think this is going away.

Remote testing is the wave of the future for distance learning. And any learning has always been a basic CASAS principal that it takes curriculum, assessment, and instruction altogether to make it work. So we now have an opportunity to build in the assessment part of that triangle.

So what are some of the challenges of remote testing? I think we've all come to realize more acutely even than before, as Carolyn was saying, that equity concerns are there. And our students, many of them don't have access to technology, or very limited. That relates to what devices they have or what kind of bandwidth.

And now with lockdown all over the country, in any home environment, it could be that there are many people competing for that limited bandwidth, whether it's people working from home or kids doing distance learning as part of their schoolwork. So our students are also competing for that. And if they have to, set up a time to do an assessment, then that's a time when they're going to hopefully want to have as much bandwidth as they can get a hold of. So you might need to work with them on when that's the best time.

Their home environment is definitely going to be a challenge for some. They may not have privacy. It may be very busy and full of distractions. So that's something we'll learn more and more about.

Again, you've probably learned something about that from their distance learning efforts. But this is where, for testing, they're going to want to be able to focus in and we need them to not have things available to help. So that's something we're going to have to be keenly aware of, and probably again come up with some creative solutions.

With remote testing, we have these new procedures for training our assessors. As Carolyn mentioned, we are working on a remote test agency agreement, where we'll ask you to have read the guidelines, and agree to protect the security of the tests and the privacy of our students, and other things. This will be an agreement that's for all of the different tests that we're going to be making available for remote testing.

And then this is something that we don't normally have to do, because we normally have the students face-to-face with us. But in this new environment, where they're at a distance, we will need to ask them to agree not to share test items, not to share the assessment information. And that's something that we are planning to ask them to do orally as part of the assessment upfront.

Then test security is, of course, so important. You can imagine It's important for us. But it's, of course, important to you as well, because you've done so much work to develop these COAPPs and you don't want them compromised. So especially for the CIT, I would say, though, because it's a standardized test, there are only two forms. And so we need to be, really, very cognizant of the kind of test security that we are requiring, and making sure that it's adequate to meet the needs.

So how did we go about this? We talked to CDE, of course. We talked to consortium leaders. We talked to agencies. We interviewed practitioners who are familiar with the COAPPs, with CIT, and with distance learning.

We weighed multiple factors, some of which I just talked about, but really a few additional ones. One is how easy will it be for you to adapt, especially the COAPPs, for remote testing? That is the assessment delivery part of the COAPPs.

The test security concerns, how easy will it be to implement? You don't have much time to the end of the year. And so there are questions about how easy it will be also in terms of the devices, etc., and the processes. We know there's some urgency to start up quickly. So we are being as responsive as we possibly can to get these guidelines and supports out to you quickly.

And of course, there's also the question of access to technology as one of the things as to how we can frame the guidelines. Test security measures, in a way-- not in a way. We really have always relied on local agencies to be responsible for test security. When you agree to do any kind of testing, you are also agreeing to keep all the test materials secure, and to make sure that the testing is proctored appropriately.

So with remote testing, it's no different. In fact, it's really heightened because there are additional concerns and there are additional challenges. So we do rely on you. We're counting on you to be responsible about this, and to take all appropriate measures to ensure test security.

These remote testing agreements, the agency one is just one per agency, and that will cover all of the different assessments, as I said. There will be, for the CIT, a separate one for test administrators to sign that has some specific agreements about test security and privacy with remote testing, also for the COAPPs. And then for students, as I mentioned, we'll be asking them to agree to some things at the time of testing.

So how will this be delivered and what's the process? Basically, we are going to ask you-- if you want to participate. Now, this is an optional thing. This remote testing is not something that's required, but it's available as something you can explore and try out.

And I want to just say about COAPPs that we are not going to ask you to revise your plan. The P of COAPPs is for plan, as you all know. So that is not part of this process. The plan will stay intact.

But what we are going to ask you to look at, and adapt if needed, is the assessment procedures or the assessment delivery in those additional assessments. So you need to look at the ones that you had said you were going to do this year. Decide which ones are those that are actively being used, where the students are getting, or likely to get, the 30 hours they need.

Decide how practical feasible it may be for each one of them. And then decide which ones you think you might tackle, one or more, to do remote testing. And then decide about the technology that you think would be appropriate for those.

And the good thing, and the reason we're able to even this quickly roll out the COAPPs and CIT, is that they can be delivered in low-tech format. So that's something that is actually quite fortunate, and that enables us to go forward right now. And we will want you to be using your existing assessors. But you'll need, as we said, to give them additional training in this remote mode.

Now, again, a little more about the test environment. You're going to want, the best of your ability, to ensure that the student's alone are not getting help. And I'm sure that you all do explain clearly to your students what is the purpose of the assessment. And if they do understand that, well, I think most of our students are not that likely to want to cheat. They know that they're trying to show what they can do and that there's not really much benefit in it for them.

But I think we do have to take these precautions. But again, I think if, as always, you explain the purpose of testing, they're not very likely to go astray. Certainly for the CIT, and we ask you also for the COAPPs whenever possible, depending on the technology you're using, you might ask the student to sweep the room with a video camera, which would probably be their smartphone, before and after the test session to see if there are any other people there or any items that shouldn't be there related to this testing. So that's the recommended precaution to take to assure test security.

And then accommodations, those that have been in place will remain. But in addition, we are thinking that there may be some new ones that emerge. As students are in their home environment, there may be some additional kinds of accommodations that come up. And so you can-- if you have questions about what's allowable, you can always talk to your program specialist and get some guidance. But that's something that we'll be seeing as this unfolds.

So what about cost? We don't think this is going to be a heavy lift for you. We think it'll be a minimal cost for a local program staff to review your COAPPs-- that is, I'm going to refer to COAPPs. But I'm actually talking about the assessment delivery part of them, and to decide which ones you can deliver remotely and adapt them.

And see if you can reasonably train your COAPP assessors. As far as the CIT goes, we think again it'll be minimal cost. You'll need to use your certified CIT test administrators, so the existing ones that you already have in your program. But then you'll need to make sure they read the guidelines and set them up for remote testing. And I'm going to get into a lot more about that in the second part of this webinar.

So this is a description of the different phases that we are working in. It's simple. There are just too. But just to be clear, right now, we're in phase 1. And that's where we're doing clinical trials and pilots of these remote COAPPs. And with the CIT.

So we have involved about 15 agencies or so so far. It's been extremely helpful to get their feedback. We're incorporating their comments as we learn from them.

And also now, during this timeframe, we are figuring out what this remote test center agreement should look like and finalizing that. So I just want to be clear that that agency remote test center testing agreement is not available yet. We will make every effort to get that out to you as soon as we can.

But it probably won't be till late next week, until it's all finalized and ready to go. It needs to apply to all the different modes of testing remotely. So please be patient for that. But as soon as it's available, we will let you know. And then that will enable you to signal your intention to do remote testing.

We are planning, on May 11 to roll out, or to allow you to roll out, remote testing both for COAPPs and the CIT. And so before that, we're working as fast as we can to get these support materials ready, including training, which I'll be talking about in a minute. So that is the projected timeline.

So in terms of that agency remote testing agreement, one per agency, we would ask you to submit it to your program specialist and to your CDE regional consultant. It would cover all of the different tests that are going to be available for remote testing. And then it includes all these agreements for test security and privacy. And it'll be something that you would just keep on file, along with your local assessment policy. In that, we'll also ask you to give us what is your estimated start date to do this.

All right, so now we're going to delve into COAPPs in particular. So the first thing to say-- and hopefully this comes as good news, is that we think that most of your agency's assessment deliveries in your COAPPs is going to be possible to use possibly as-is or modified slightly for remote testing. And that would be with computers, with smartphones, or even lower tech than that. So we think that it's going to be something that's quite doable.

The thing that is very important is that you should be able to interact in real time-- that is, live, with the student or students. And I will say that, with COAPPs, you may be able to assess more than one student at a time. We are going to ask you to start one-on-one and get good at it, and then move into what would be a little more challenging to proctor more than one student.

But some of the assessment modes, particularly if it's a written response, you may be able to effectively monitor more than one at a time. But I would say in general, you should not have the expectation with remote testing that you're going to do a whole class at a time. That is just not possible, because you do need to be able to observe the student. And so there are limitations to the number of students you can work with at a time.

And of course, these COAPP assessments are applied performance. They're performance assessment. And so that's the beauty of them. That's how they're so relevant to what you're teaching. And it is, then, required for you to be able to observe, and really see them demonstrate their skills.

Now, this is perhaps a little overwhelming. But I'm going to walk you through this chart. Across the top, we're looking at what are the types of technology. That is, which devices and which platforms can be used for COAPPs.

And so you can see we tried to order this from low to high tech. Not extremely high, but at least in terms of our student populations. So the lowest would be a phone or just a phone. So that is possible.

I think we would prefer a smartphone, because then there can be the video interaction. So hopefully, that is what you can manage to have the student have is a smartphone, where you can see them. But a phone is possible with some of the oral-only.

On the left-hand side, you can see the different modalities. So if the assessment delivery of a COAPP is just oral, then you may be able to simply have a phone conversation. And again, preferably a smartphone, where you can see the person.

If it's a written assessment delivery, then paper and email. Again, we want the camera because we want to be able to see the student writing. But making sure that they didn't get it passed to them by somebody else, or that they're writing in real time. But that's certainly something you could observe and that they could share with you. So a text or email as well, that could be even more real time, where they're actually using what's happened texting the answers as you ask them.

We're going to talk about the platforms in a minute. A smartphone with a camera where you can see them. With camera and zoom would be another level up. If they have an iPad tablet, chromebook, or computer, OK, great, with Zoom. And then a higher level would be a Windows 10, but not needed at all for these.

And then the modalities could be a combination of oral and written. Role play is something you can do with them remotely. Then portfolios, we feel you can do them.

You will need to be very particular and careful about how they submit what they do and where. But that is potentially something you can work out the details for and pull it off. And of course, listening and written is another possibility.

So we don't think this list is exhaustive. You may come up with other permutations and combination of these devices and platforms. But we just wanted to give you an idea of how it all fits together with the COAPPs. And we gave some sample COAPP assessment deliveries there, where you can see how they might play out with these different modes of technology.

So this is just to let you know that there is a document that's called Remote Testing Guidelines for COAPPs. It's still in the works, but it's quite far along. So we really do hope to get it out to you next week. Next slide.

Now here, I know most of you are probably familiar with going live with eTests, which is our onboarding process for eTests. This is the comparable process for going remote with the COAPP. So we've tried to lay out some of these steps. And some of them we've introduced already, but I want to go into a little more detail here.

So first of all-- and this is when you roll out, you'll want to consider which of the assessment deliveries of COAPPs you want to select for remote testing, and if necessary adapt them. So you'll need to very carefully review the COAPP guidelines for remote testing and see, according to those guidelines, what's possible, doable. Then you'll get into revising the COAPP assessment delivery as needed.

Again, not the plan, but the assessment delivery. Decide what technology might be required. And that could perhaps eliminate some, or make you modify how you deliver it.

Determine how you're going to verify student identity if, in some cases I imagine, it's not the teacher who's the assessor. If that's the case, then you'd need to have a process for that. And we are going to basically leave it to your local agency's process that you normally use to verify student identity. And it does vary across the state. So whatever you're using now, you'll need to figure out how to adapt that process for the remote context.

Then there may be also some things you need to consider about documenting the results. If you've used paper in the past and file folders, you'll have to think about that there's some kind of electronic document that now has to be sent somewhere and stored somewhere. So those are things that you're going to have to decide on and then make it very clear to all the assessors. Then we'll ask you, as we've already mentioned, to submit the agency remote testing agreement to both your program specialist and your regional consultant. There, we'll ask you to specifically list the ones that you're planning to deliver remotely and describe how you're going to revise, if you are, the assessment delivery for each of them.

Then it's time for training and looking into rolling out. So we'll be asking you to train the assessors in the remote testing procedures. We're going to be providing you with support for that. The program specialists are working on training webinars to guide you and give suggestions for that. So that's in the works as well right now.

We would ask you to do trial runs just between assessors. So before you launch it with students, try it with just assessors role playing. Work out any kinks.

And then, again, don't go full scale until you've done a pilot with one or maybe two classes. Just a limited number of students to just take it for a test drive and see if you still need to make any other adjustments. And then when everything's ready, roll it out.

I mean, these steps, you could do them very quickly. And I know many of you are eager to get going. But we would encourage you to do all these steps, and not rush ahead without them.

Oh, I see. I'm sorry. There's a comment here, which I will reinforce, if not everyone has seen it, that the teacher is not supposed to be the assessor whenever possible. So yes, please-- then you will need to be looking into the verification of student identity. It may be as simple as just having the student hold up their ID, or however you do verify their identity, so that can be authenticated.

We will ask you to have all assessors that are going to be doing remote testing with COAPPs sign this simple form that you should keep on file. That will assure that they have agreed to all the privacy, security, all of these remote testing guidelines. So that is something just as another measure to ensure that the test security guidelines are being followed. Next slide, please.

And we're not going to leave you high and dry. We're going to give you the guidelines, which we hope will be as robust as they can be, to give you initial guidance. The program specialists are, as we speak, working on webinars to guide you. And again, I'm not sure exactly when we'll be able to release those, but possibly late next week, or definitely by May 11.

And then we are already compiling FAQs about remote testing. And I'm sure that those are going to be a very dynamic and living document, where we'll be adding as we gather more questions. Next slide, please.

So here are some tips, just to put a little bit of some examples to this and make it a little bit more concrete. In terms of if you have COAPP assessment deliveries that are written responses, that could be done on paper, as we were saying, while the assessor is watching. Then the student could hold up the paper they've written on with their response and show it to the assessor. The assessor could take a photo of it, or just read it and confirm. There are some considerations, and you may have already experienced some of these constraints, if a student is using something as small as a smartphone that there's not a lot of screen real estate.

And it may be necessary for them to use line paper or to use something darker than a pencil. So you might have to advise them to use a medium tip dark pen. In other words, something that will be darker and easier for you to read. So if the student has a computer and a cell phone, then they could even be texting in real time while you observe that happening, where you're going to be getting the texts.

Another tip is that things like forms, perhaps a census form, or in the area of health, there are a number of forms, you can make sure that the form has clear numbering of the lines. And that will help you to guide the student to a specific location on that document. And then the student can use those reference numbers in their answers. So we can go to the next slide, and there's an example of that.

From the left, you can see this document that's adapted for the census questions. And then on the right is a fake student, who has dutifully answered all these-- listed all the line numbers, and then put their answers. So this is just an idea of one way of going about it with COAPP 54.

In terms of oral responses, again, it may be possible to use the assessment delivery in your current oral assessments using the same rubrics. You'll have to see, depending on how you have written them in your program, that it may be possible just to be one-on-one with the student, or more than one if that's how you've set it up, and recording the responses using your rubrics. So that's something you'll have to see. We, again, would really like you to be able to view the student if that's possible with the devices and the technology that they are able to get access to.

And you can use any kind of video app. WhatsApp isn't there, but that's another one that's not even listed. So FaceTime, Skype-- so many of them, just as long as both the student and the assessor have access to the same platform.

If you're using illustrations or pictures as a prompt-- this could be for oral. It could also be for written or other kinds of prompts. You need to make sure that they are going to display clearly on a phone, if that's the medium that you're going to be using.

We all know that those phone screens are quite small. And so if you have an assessment delivery in a COAPP that has multiple illustrations, that might not work. You might have to break it up into just one at a time that's not too detailed. So all of these things, you'll need to-- and this is the sort of thing that, when you do your one-on-one with assessors, you'll be able to detect whether it's visible. And when you do those trials together, you will want to use the technology that your students have, and make sure that what you're planning is going to work.

Another consideration is font size. So again, although the screen is small, you still need to make sure that the fonts are visible. And so you'll need to experiment with that, and make sure that they're readable on the devices that you'll be using.

Here's an example of some enlarged font size, so you can see. I'm sure you can have ways of working that out. All right, next slide, please.

So just summarize about COAPPs, this is new for all of us. We know we're going to learn a lot together as we look into this. The COAPP-- these additional assessments are already so creative and innovative. And I know many of you are very proud of what you've developed at your programs. And I think now is an opportunity to find some additional innovative ways to tweak them.

And sometimes, it may be just by adding one more sentence to the instructions, about hold it up to the camera so that the assessor can see, that sort of thing. So you'll be able to, hopefully without too much effort, make these work. And we really would like you to be sharing your experiences.

Program specialists are going to be quite involved, and we'll be pulling you together to share anything we can share with you. Also to learn from each other and share statewide as well. So if you would like to be part of this pilot between now and, basically, the next week-and-a-half or so, please let your program specialist know. And then if you do decide to do that, we definitely appreciate the help you're going to be giving us.

Janice Fera: And it's Janice. Agencies are asking questions about brainstorming ways using Google Forms or answering. One of the questions came up about recording. Is there any requirement for recording the students' responses, whether it's audio or video?

Linda Taylor: CASAS does not have a requirement to record. And the reason-- actually we initially explored that. But we backed off of it because we found that it was really challenging to figure out what to do with those recordings. And we also got a little bit concerned, because we realized that although recording would document what happened, it also can be challenging, we think, to safeguard the recordings. And we might be opening up the assessments to more danger than they might otherwise be.

So basically, we thought about it, and we actually decided against it. And so for those reasons, you really maybe ought to think twice before if you think that you want to do it. So I don't know that-- and, I mean, Pat, you might want to weigh in here. But I don't know that we're necessarily prohibiting it. But we would really caution you strongly about some of the dangers of it.

Pat Rickard: This is Pat. And I agree with you, Linda. I think as agencies look at their COAPPs and look at what might be possible, they may decide that they want to record the first few so that they can comply with the requirement that at least one pass and one fail is stored, so that when CDE does their FPM, and comes out to their agency, they can actually see the full assessment of what is past and what has failed. But I think it is a logistical problem to record all of them

Linda Taylor: Thank you. Any other questions? I just want to mention that we will be also-- when we meet on Monday and talk about the pre and post-testing, we'll also be addressing recording. And just as a little preview, we have decided against recording for the pre/post as well. So that is a change from what we had originally thought.

But as I mentioned, it's a steep learning curve, and we are considering new angles all the time. So although we had originally thought about asking the pre and post-test to be recorded, we have backed off of that. And so now we are not wanting them to be recorded.

So now I'll move into the CIT. I don't know if all of you are familiar with the CIT, the Citizenship Interview Test. It is a very different kind of test than COAPPs because it is a secure standardized test that there are two forms-- 973 and 974. It's a one-on-one oral interview that simulates the USCIS naturalization test.

It's not a pre and post-test. It's a pass/fail test that is given at the end of citizenship prep instruction to determine if a student is ready to pass the USCIS interview portion. Not the knowledge part of 100 questions, but whether they're able to handle the oral part of that interview. And it's after the 30 hours, they can try if you think they're ready. And they can retry and take the alternate form if they don't pass.

So I just want to be very clear that the things we're going to talk about for the CIT are much more stringent, the requirements. And we're talking about them today because they are specific to EL Civics. But they are very different. There are different requirements for the CIT than the COAPPs. Next slide, please.

So for the CIT, as I said, we think it's quite doable. The two possibilities are a phone with a video cam, or a computer or tablet with a webcam and audio. Now, you will see that only a phone-- even a smartphone.

Well, sorry, just a phone without a smartphone is not an option for the CIT. So a smartphone with a video camera can be. That is allowable.

But a phone that does not have any video capability is not possible to use with the CIT. So if there are students that want to take it, maybe they can borrow a phone, or somehow work it out. Or you can perhaps provide them with the technology to do this. But we cannot allow the CIT to be given without some real-time video observation.

The platforms that are possible are open, actually, but must be video platforms. So it could be Zoom, WhatsApp, Google Duo Skype, or FaceTime, or any other platform that both the student and the CIT administrator have in common. So that's something for you to determine. The platforms are more open, but the possible devices, there are restrictions. Next slide, please.

So here's the GoingRemote process for the CIT. We are piloting right now. I just want to give a shoutout to Jennifer Gagliardi. Thank you so much for your help in this first phase.

We've met twice with Jennifer. And she's been, as you can well imagine, incredibly helpful in helping us think of things we hadn't thought of before. And she's been piloting with them already. So thank you so much.

So the first part is to look at these new CASAS CIT Guidelines for Remote Testing. I will explain that these are guidelines that are in addition to the CIT test administration directions. So those of you who are certified CIT administrators are masters of the CIT test administration directions. But now you will need to carefully study the CIT guidelines for remote testing and be very, very familiar with them.

One of the interesting aspects of rolling out the CIT remotely is that there's the matter of the paper test booklet, which has always been paper. It's something that's consumable. And the test administrator for every student fills it out, it goes through the test, asking questions, and marking down the score according to the rubric-- 0, 1, or 2.

So I don't know. Some of you who are CIT administrators may have left your school with copies of the 973 and 974, in which case maybe you're all set. But we imagine there are a lot of CIT administrators who didn't do that.

And so we have created-- and are still fine tuning. But we have created a fillable PDF version of these form 973 and 974 test booklets. So just as you would use the paper, it would be an electronic form that the CIT administrator would have on their computer. And as you were asking the questions, you'll be marking the score as you go and then saving it. And we will be giving you guidelines for how to save it, what to put in the file name.

And then in this third bullet, how do you enter that score in TE? How do you send it along? What do you do with that fillable PDF? How do you store it? So this is things that the agency will have to figure out in terms of procedures for remote testing of the CIT to make sure that all of this is secure and very orderly and clear.