[music playing]

Narrator: OTAN, Outreach and Technical Assistance Network.

Christina Hyatt: We're from Corona Norco Adult School. And today, we're going to share our journey into developing a successful remote program at our school for testing, and CASAS, and civics. So I'm Christina Hyatt and I teach adult basic education math. And under the direction of our Dean of Students, I run the remote testing for the CASAS schools for adults secondary education students.

Marci England: Hi, everyone. I'm Marci England. And I am like the DJ today. So I just need to double check. Can you see-- I'm sharing my screen, right? I know it's like nobody likes hearing that at my screen. But I wish there was some way we could just ask Zoom-- am I sharing it, without asking everyone. OK, and then I'm also going to manage the chat. And I will watch the questions and just make sure that this presentation continues to flow.

So that's my role, a little bit. But also, I'm here to share the EL Civics side of remote assessments. I'm Marci England. Let's see, I'm the ESL program coordinator at Corona Norco Adult School. I'm excited to talk to you about remote testing and what we've been doing. So based on that, I'm going to go to our next slide. Maybe, there we go. All right.

So it's always fun to start a presentation with some fun facts. So did you know that in 2020, puzzles sales increased by 300% to 370% over the same time in 2019? Ravensburger, which is like the gold standard of puzzles, it's the German puzzle company, they were selling puzzles at a rate of 20 puzzles per minute. Penny Gold, a pediatric neurologist said, when you are working on a puzzle, you can't worry about anything else. You can't let other things intrude, other thoughts intrude. It's all about meditation for me. Another puzzle enthusiast said-- it's my chill time after working all day. It's something I look forward to.

So the puzzle fad of 2020 is something that I think we can all relate to in some way, shape, or form. And I know Christina, she completed well over 20 puzzles last year, you guys. And she's even sent me pictures of her completed puzzles. And I have to say that I have contributed to a few or several assists, although I cannot brag and say I completed any. But I am the one that walks by and says-- maybe this piece would go really well right there.

So when Christina and I were talking about implementing remote testing, we really thought that this had a good connection to solving a puzzle. And we love an analogy in how looking at things from a different perspective helps you connect pieces. And so we're really going to hang onto this puzzle analogy and talk about how remote testing has been an enjoyable experience of focus and meditate, maybe not as far as meditation, but it's been an enjoyable process as we've solved problems and created a better picture.

So today, we're going to discuss the critical first steps we took to implementation. We're going to talk about administration of the assessment. We'll also discuss ways that we've worked to maximize learner outcomes, and different ways that we measure learner outcomes, and measure our successes. We're looking at things differently than just testing scores. We're looking at a lot of other positive things. We're also going to give you some troubleshooting ideas. And we're going to share our pitfalls. We will tell you the things that you can do to avoid, like, we've been through some bumps and so maybe if we tell you where they are, you will not hit them as well.

So the format, Christina is going to guide us through remote CASAS testing first. And then we'll have Q&A with Christina. And then I'll guide you through remote Civics testing and we'll have Q&A for me or in general. So that's what it's going to look like today.

So just to give you a bit of background on our school. Let me move this out of the way. OK, so to give you a bit of background on our school, just so you know where we're coming from. In 2019-2020, we served over about 3,600 students. 28 had significant literacy gains based on CASAS testing. So we were successful and we had a robust CASAS testing program. Our programs include ABE, GED, high school diploma, ESL, we had 1,300 students, CTE, concurrently enrolled, and fee-based classes.

Our technology accessibility, to start off with, in 2019-2020, we didn't have a loaning program, we just had laptops on school sites available for students to use, but not really a program where students took laptops home. Thankfully, due to grant funding and our consortium, we are now currently operating under a loaner program where we have enough laptops and Wi-Fi to provide students, anyone that needs it with a laptop and Wi-Fi. So understand that that's where we're coming from. Our etesting history, in 2019-2020, only adult secondary education was doing etesting. ESL and CTE did paper and pencil for Civics and CASAS. All right, Christina.

Christina Hyatt: So when you're getting ready to do a puzzle, everybody has their first steps. The first things they do. Like for example, do you dump all the pieces and sort them first? Maybe you sort them by shape or by color. Maybe you're one of the people that turns them all face up. I personally like to start with the frame. Everybody's got a different way of getting started.

But there's some critical first steps to getting going. And so likewise with our remote testing puzzle, we looked at what were these first critical steps, and we called them the corners. The most important parts of the puzzle is finding those corners and building around them. So the first corner we looked at was the who. Who is this going to affect when we test in a remote situation? Who needs to be trained? How do we-- who do we need to get on board with this program?

For us, we needed to train our staff and increase their understanding of why we give the CASAS test even in the middle of a pandemic. Right? So we worked to shift the mindset from pay points to student growth. Our CASAS test trigger digital badges. They trigger promotions. And we use them for recognition. In our meetings and in our PLCs, we shared about the importance of CASAS test for showing the student growth. And once teachers began to understand this, they became our partners and making sure every student got tested.

And I do remember, when I first heard about remote testing, I think it was back in May, I thought-- you want me to ask students to do what? And what are we offering them in return for this? I needed a mindset shift too. So by shifting my mindset to student growth, I don't even think about it now. Testing from home, that's just what we do.

The second corner piece of our puzzle for testing was the training. We trained staff on the testing protocol. We got them the necessary certifications. Both EL Civics and CASAS have requirements to implement remote testing. So we worked closely with CASAS staff to make sure we were within their framework. CASAS has videos, literature, trainings, and helps to get your remote testing program going. My favorite thing is they have a remote testing checklist. And who doesn't love a good checklist, right?

We also train students. Some of our students have never used Zoom. Some of them needed to learn how to use a computer. We wanted to make sure that the students felt confident, that they had the necessary technology skills to take the test in a remote environment.

A third corner piece was that technology. Getting the right type of device has been very challenging. And in fact, one of our pitfalls at times. But luckily, as Marci mentioned before, we were able to get Chromebooks and Wi-Fi hotspots into the hands of students who needed them. And then we also worked with the district IT department to make sure that those Chromebooks could work in kiosk mode so the students could take their tests online.

The students need the right type of technology. And they need to know how to use it. And they need to feel confident when they go into a testing situation that the technology is not going to be what trips them up. So we continue to work with the staff to get them trained on helping the students on making the students feel comfortable with those pieces.

And another important part of this corner of technology is making sure that the staff has the right technology and the right abilities. We all were nudism a year ago. And so learning what Zoom can and can't do, learning to make Zoom work for you when you're a proctor was really important for me. And I'll talk about some of those things later. Another thing is when our campus went completely remote and we were only testing students from home, I needed enough computers and enough bandwidth to be able to do that from home. So these are all things that we had to consider as our critical first steps in developing our program.

And then of course, the fourth piece is the support system. So when I do puzzles with my family, I have a sister who she likes to start with the water. It kind of blows my mind. Like, why? But she always starts with the water. And I have a brother who likes to look at the shape instead of the color. He's a little color blind. So it makes more sense to look for the piece with that little arm or something. Whereas my brother-in-law, he cannot stop looking for that missing piece in an almost finished section. That bothers him to no end. And for me, I have to start with the structure, the house, or the boat, whatever it be.

So when working on your system, you want to look for a team that has different strengths and different points of view, much like my family does when we approach the puzzle. Having that strong support system helped us work our way through a lot of the wrinkles that came our way. As questions came up, we had people we could go to for answers. Sometimes the pieces weren't fitting the way we thought they should, but getting a new perspective, or having somebody else look at it, or looking at it from a different angle, we're always-- that we were able to come up with solutions. So consider who's going to be on your team. Would it be your CASAS rep, your tops coordinator, the principal, your consortium, the experienced teachers on the campus, the techie people on campus, the district IT department. And for us, it was every single one of them.

Another important thing to consider is the cost. So I kind of think of the cost as like the frame of the puzzle. It's going to determine the size of your testing program. And remote testing can be costly. I'm not going to sugarcoat that. But we looked at ways that we could minimize the cost and maximize our resources. For example, consider what do you have.

And for us, we had lots of computers sitting in remote locations that we were able to bring back and implement in our testing program. You can also look at who do you have on campus. We had paraeducators and substitute teachers who were willing to step up and be part of the team. Look for the emergent leaders in your campus, those who can, and will, and are able to make that strong team to complete the puzzle.

And last of all, look at how it could be better. It's easy to just look at a system. And go-- that's it. But we really need to constantly be thinking of improving the process and perfecting it as a way of reducing the cost and maximizing what you have available. And so for me, recently-- I started looking at I realized I was spending a lot more time on communication than I normally do.

And I had to step back and think about not only what had changed so I could reverse that change, but go even further and cut into what was being done before. How can I make it so I communicate less frequently and yet still get the message out to the people who need it? And Marci, you want to talk about equity? And you're on mute. Yeah, yep.

Marci England: What's a Zoom meeting without someone talking on mute? OK. Glad I could do that for us all. So at our school, equity means everyone has access to learning. We work to ensure that the most vulnerable students are supported. This means bringing students on campus for testing when we can, but also offering remote testing for students that are not comfortable or cannot come to campus. It means creating systems that support all of our learners.

For us, it means getting our Chromebook books to work and having staff to support technology problems. It means troubleshooting different technology types. Equity is listening to teachers as they advocate for individual students and preparing for student needs. Specifically with a remote assessments, equity means access, accommodation, and flexibility within the testing parameters.

Moderator: Oh, Christina, you're on mute.

Christina Hyatt: It's my turn. Thank you.

Moderator: I checked some problems.

Christina Hyatt: Well, I wanted to be like Marci. She is my bestie. OK, so I'm going to start by going over the CASAS goals in life and work program and how that was developed. And I'll stop for questions at the end of that. And then Marci will jump in and share what she did with Civics.

So as we developed the plan, it really-- I think about it is now getting all the pieces to fit together. You've got this 500,000 piece puzzle and now we've got to figure out how the pieces are going to fit. And so here's this key questions that we looked at. What needs to be done? When does it need to be done? Who needs to know about it? Who's going to do it? I honestly didn't think I was going to be the one doing it, but here I am. Because that decision was made somewhere along the line. And what training, technology, skills are required? And what are the CASAS policies that need to be followed. And right here on this slide, for those of you who want later, I'm going to be providing a template for developing your own plan. So that link is there when you want it later.

Marci England: And everything-- we have our slides in a Google slide presentation. No, in a Google folder in Google Slides. So we'll be sharing the folder with you that has access. So any link you see in our presentation, any template, whatever, it's all inside that folder. And I'll put it in the chat here in just a little bit.

Christina Hyatt: Perfect. All right, so on the next slide, I'm going to go over what our school did, or what we do for our testing program. I'm not give you every single detail because we'd be here for four hours and that would be boring. But I'm going to give you some of the highlights.

So for me, one week before, I start e-mailing students to confirm that they have the required technology and they know how to use Zoom. In this email, I even includes some Zoom tutorials. Because the number of times you've had a student come on and this is their first time using Zoom, and they don't know how to unmute, or they don't know how to mute, were all things that we needed to deal with.

So the trickiest part of the process is getting students to show up with the right kind of technology. And so one of the things I do to combat that is first I send them an email. And I tell them-- here's the kind of technology you need. But then I created a survey because it's not enough to send an email and expect them to read it. They're busy. It's more than two lines long. And so Marci, if you'll show the survey.

I asked them instead of just reading, I asked them to do something, to respond. This way I know that they read and they understand. And it's a very short survey. It doesn't take them very long. That I want to know exactly what kind of device they're bringing to the testing session. And I want to know if they understand that they have to be able to have a webcam and a microphone. I want to make sure they understand the amount of time it's going to take and the testing environment that they need to be in.

So this survey has been fabulous. Because one, I now know that if they're showing up with an iPad, and I can send them the tutorial so that they can download the app and get ready. So the night they come, they're like-- yep, I've got it. It's all on screen. And we're not dealing with-- oh, let me send you that. Let me walk you through the process.

And then the other thing, Marci, if you'll go back. I got lost. But give me just a minute. OK. So when they-- oh, and then the other thing is there's no surprises when they show up. Whatever technology they have, we're kind of expecting it and we're prepared for it. Because different kinds of technology require different conditions.

So then the day before the test, I send that Zoom link. I don't send it in the first email. I'm waiting for them to respond to my email. And there's a reason because I need them to think I have to do something before I get this invitation. When I send the Zoom link, I assign them a time, and I stagger those times. The first time I did it, I had all five students show up at the same time. And it was not a good experience. Because I couldn't help five students on board at the same time, especially when each one had a different technology ability and a different technology platform. So now, we give 10 to 15 minutes so we can work with the student one on one and bring them in individually.

So then on the day of the test, the first thing I do, is I open a breakout room. And I have one room for each student that's going to be testing. Immediately after I open those breakout rooms, and this is way before I expect the students to come online, I start bringing on computers in my office. And I bring on one computer for each breakout room I intend on running. So that I can see every breakout room all at once. I'm not having to scroll through and look at each breakout room individually. I can see all students. And if one student is struggling with something or needs help, I can hear and see them immediately.

One of the things, though, that we run into, and I'm sure all of you can relate to this, whether you're running a program or not, is the internet. Does your internet over go out? It's just me? So a couple things that I have a suggestion, these are a couple tips I have. When you do the breakout rooms, there's a setting that says automatically move participants into their breakout room. And there's another setting that says manually at the students.

So I manually add the students and the devices that are in my possession into the breakout room. But if the internet goes down, which it often does, it's lovely because as soon as it comes back up, Zoom knows to automatically move them back into the breakout room. And I'm not scrambling or having to move everyone back where they were. Zoom automatically just puts them back where they were. And a lot of times students don't even realize it ever happened.

And then my second tip that revolves around the internet is because I'm using sometimes six computers to do this, I need more bandwidth. And one of the things I now do is I turn on a hotspot for my main computer, the one that I'm the host on. And that one runs on one Wi-Fi connection, and then the rest of them will run on the second one. So I just use my phone usually, I'm fortunate enough to have enough hotspot on my phone that I can do that. And it's made all the difference. Because if one drops the connection, I still have the other one up and running.

So then the last couple of things or tips is after the test, sometimes students have a lot of questions. And maybe you're in a hurry, or maybe you have a couple other students, but make sure you give them the time and answer their questions and provide them the resources. Direct them where they're going next. I made this mistake early on. I said-- great, thank you so much for taking the test. And we'll get back to you.

And that was not right. There was a process in place that we needed to make sure students knew what to do next and how to do it. And usually, in that end of the conversation after the test, I will tell them, three times, the next step. And later on, I'll show you another thing we've done to make that even smoother.

Marci, if you'll go two slide-- two slide up, I think. Sorry. So this is my workflow. Please don't look at this and think-- oh, my goodness. Right? I'm kind of OCD when I write my steps, when I work on stuff. And I write things like turn on the computer in here. It's pretty intensive.

But I wrote this down because I'm training my staff, or my team, I should say. My team to do this without me. I don't want this system to be my system. I want it to be the school system. And by writing it down, also I could look at the places where I can perfect it or make it better. So I'm not going to give you a copy of this. Some of you are probably saying-- but why, but why?

And the reason, of course, is because this was our puzzle. This is what works for our schools. So instead, I'm going to help you. You guys have your own puzzle, your own way you're going to need to look at things. And I'm going to give you a document that's going to help you develop those skills. You can see, I've got all of the timing set up in here. And I've added even more to it than this picture shows. I've given you some of the tips, some of the resources we're using.

They may not work for your school. We have Zoom in Spanish because we have a lot of Spanish speakers. But maybe you need Zoom in another language. But I've provided some questions or some things for you to consider as you go through the process. Who needs to be involved? Where is it going to happen? How is it going to happen? How is the student going to know about these things? How do we make sure we're staying within the parameters provided by CASAS? So these are all-- this tool is designed as questions to get you talking and get you thinking about how to get your program going.

All right, and next slide, Marci. Oh, yeah, you're there. I'm not. Thank you. OK. All right, communication. So for me, communication is like assembling the vast blue sky of a puzzle. It's difficult. And you usually spend a lot of time on it. And you're constantly trying different pieces that are all the same color. This is where that brother who's colorblind comes in really handy.

So communication is something you have to constantly work at. And it's where I spend most of my time when it comes to the testing process. You just need to make sure that whatever your message is, that it's clear, it's concise, it's consistent, it's continuous, and cohesive. We recently had a student that went through the testing process and I still chuckle to this day because Marci calls her a good little communicator, which is exactly what she was.

When she got your email, then you got three to four emails a day, detailing exactly where she was, and what she needed to know, and how you were going to be able to help her. And it's easy to think about-- ugh, that student. But I'm actually really grateful for students like that because they allow me to look at what's missing. What do we need to relook at, right? How can we deliver the message even better? Because while this student is communicating her needs, there were lots of students who probably weren't going to have the same questions.

Oh, go back, one more time. I was just going to share my quick tip. So a couple more things. So the quick tip. One of the things we run into is we thought we were running a remote testing program, and our first couple go rounds, we had tons of students show up on campus. And we realized we couldn't use the phrase from home, stay at home, do not come to campus, and thus. So we looked-- we went back and it turns out our message for students were signing up was conflicting with the message we were trying to give them.

So we were able to look at fixing that message, making sure the message was consistent throughout anything that the student was seeing. And making sure that every email I sent over and over again told them-- you will connect online. You will not come to campus. We will see you on Zoom. And so my quick tip is make sure when you're communicating to students that they know exactly how this process is going to go.

Communication and miscommunication is one of the biggest opportunities for pitfalls. So when you're developing a good remote testing system, make sure you look carefully at what the message is, who needs to get the message, and how it's going to be delivered.

So one of the ways we did that is by creating a Google document or a Google site, excuse me. And we outline the steps that students need for getting enrolled. It has the necessary steps, the forms, the tutorials, the contact information for each step of the process. And right there on step 6 is our CASAS assessment team. Just this little thing of telling them-- hey, you're going to get a message from Christina Hyatt, has really made a difference in the number of calls that go to the office. Because now, instead of the student calling and saying-- I never heard from the administrator. They know first to search for an email from Christina Hyatt. Inevitably, it is there.

All right. So the learner outcomes. So we talked about earlier that we weren't focused on pay points. We weren't focused on how this was going to look for our school. We were focused on how we can show the student growth. But there was a happy side note that we saw the pay points. Even though we saw fewer enrollments this year, we saw a higher percentage of students with paired scores. We saw higher percentages of students leveling up.

And I also love that now, with a remote system in place, we can capture students who weren't available to come to campus. Even before the pandemic, there were always those students who-- oh, I'm in Mexico. Well, guess what? We tested somebody in Mexico. We got that post-test because of this remote system. And so it just allows the flexibility to test students where they're at and meet them if they don't feel like they can come to campus.

We also are giving students digital literacy skills. For many students, these tests online are maybe the first test they've taken online. But now, HiSETs being administered, online, GEDs being online, all from home. And now, they've got this opportunity to practice. As they enter college or into a job environment, they're going to need to know how to navigate some of these online experiences. And by having this program, we're giving them the first opportunity to try it. And when they come around the second time, it's a lot easier. And so we can see that they are increasing in those skills.

So the process was slow. We started right when CASAS allowed us to begin back in May. But we weren't that great at it back in May. And we're just getting better, and better, and better the more we practice. It's not that it's easy. That communicating with students, or that moment when you're trying to onboard a student with technology, it's still nerve racking. My team and I, we always talk about that being the most nerve wracking part of the process. But each time we put more pieces of the puzzle in, it gets easier and easier to put the puzzle together. So I am going to pause right here and answer some questions.

Marci England: So Christina, we have a question from Peg, she says-- how do you make sure the students read and understand the Google site or other information about your program/testing?

Christina Hyatt: So the first thing a student does when they're getting enrolled is they have an orientation with our Dean of Students. And he takes some time to go over the steps and the process, and he also gives them the website so that they have it. And everybody on campus knows, if a student's lost in the process, here's the site to give them.

And it became a one stop shop of-- if a student calls and says-- hey, I haven't heard from the testing coordinator. They could go-- well, it looks like you haven't done step five. Or you still need to do step four. And so it comes from him, but at each step in the process, the office knows about it, the Dean of Students knows about it, the testing team knows about it.

And when we finish testing with the student, we don't just tell them, we show them. OK, here's where you're at in the process. Have you completed these five steps? And there's just a couple more steps after they meet with us. And we tell them, at least three times in the phone-- in the Zoom session at the end, and then we show them.

Marci England: Thank you. OK. Oh, good question. Kathleen asks-- is there any translation?

Christina Hyatt: Can you be more specific Kathleen?

Marci England: I believe, I'm thinking a translation on the Google site.

Christina Hyatt: Oh, I think that Google does translate. Right? Am I wrong about that?

Marci England: I know for sure the forms, you have the translation option on the forms. And then Google translation really is just if the student is tech savvy enough to have that set up in their Chrome browser. But that's a good point, is that we could have a little mini tutorial, like click here to translate this site.

Christina Hyatt: Yeah, that is great. See? Another piece of the puzzle in. Thank you.

Marci England: Diana says-- how did you find the students who need tech equipment? And then she says maybe Chrome extension for the translation. I'm sure she's talking about. So how do you find the students who need tech equipment?

Christina Hyatt: So usually in the orientation, our Dean of Students does go over-- if you're going to be testing from home, you're going to need some technology. But usually, it's that first email I send a week out. Here's what you need. Here's the options you have for testing. And if they don't have it, there's an other option on that quiz. And I did provide you guys a link to that quiz as well, that you can duplicate it make it your own if you wanted.

And so when that other comes back, then we work with our team to either check out the device then, or maybe find another device. And we were lucky enough that not just our Chromebooks, but every Chromebook issued by the district, can now run the eTest. And it's just built right into the Chromebooks. So a lot of times that solution is-- hey, do you have a K-12 student in your house with a Chromebook? And then we're able to do that.

Marci England: So that means, Christina, that you would use the K-12 device before they're issued a device through our school, right?

Christina Hyatt: Yes. And we've also enrolled students, issued a device, and then tested after as well.

Marci England: OK, good. So do CASAS test students with different devices in the same session?

Christina Hyatt: So I know that CASAS, in their training, they recommended not doing that. And so I tried it the first time. But when you're only testing five students in a remote session, and this was early on, I just felt like it was too hard to find five students who all had a laptop, or five students. And it's actually easier for me if I don't have five Macbooks at the same time. Because that's a whole new dimension of working through the testing.

So for me, I give them the options. My staff and I are well trained on what the options are and what they can do, and how to get the student on board with those options. And that's all part of our plan. I have all the devices written out and the specific steps that need to be performed for any particular device.

Marci England: Let me show that really quick. So that's here on the right. Right, Christina? Where you have if it's a Windows, a Mac, a Chromebook, an iPad, and what to do there. OK, so we have-- Kathleen says-- with all the materials with remote testing steps, can we use native language up to begin the test? I think that probably means like to explain the test, like up to that point.

Christina Hyatt: I'm going to refer you back to your CASAS rep on that question. Because I don't want to say what's wrong. But my understanding, I could be wrong, was that yes, you can right up until the beginning of the test. But I could be wrong. So make sure you check with your CASAS rep on that.

Marci England: And then here's a question from Peg. Do students ever lie about which device they have? If they're scheduled for a test or daytime-- so two questions. Do they ever lie about what device they have, or tell you the wrong device? And if they're scheduled for a test and they don't show up, what happens?

Christina Hyatt: So we have had students that tell us the wrong thing, and especially before I created the survey. Because they go-- well, I have a Lenovo. And then they showed up with a Lenovo Chromebook. And then I learned, another puzzle in the piece-- another puzzle piece placed, that a lot of people make Chromebooks, not just Google. I don't know, mind blown, or whatever.

So lots of people make Chromebooks. Students don't always understand what they have. But they've never intentionally lied about it. So sometimes students come thinking they're bringing one device and bringing another, we work with them. And again, that staggered time to give us the 15 minutes to work with them to see if they can figure out a device that works within our parameters.

If not, luckily our district was allowing us to bring small cohorts of groups in the beginning. And then when we went completely remote, we still would occasionally bring one or two students, usually one student in a room to do a test. And through using that, if a student absolutely did not have the technology, or the technology skills, that was our workaround.

Marci England: So there's actually a lot of questions popping up here, which is great. We have time built in for this. Is there a specific-- oh, hang on, I missed one. So the question-- do you mean the school will provide a Chromebook to the students? So yes, our school does. We have the ability to provide Chromebooks to the students. Is there specific training for proctoring the CASAS test virtually?

Christina Hyatt: There absolutely is. And somewhere in here, I think, back two slides maybe or one, Marci.

Marci England: Well, I'm managing the chat. Let's see if I can--

Christina Hyatt: Oh, I'm sorry. That's OK.

Marci England: Where do you want me to be?

Christina Hyatt: That one. That's the link. Right there is the link to the page where you're going to find that checklist I talked about, going remote checklist. And it also goes through all the training. And I don't know how recently they redid their training thing, but it's really lovely now because it says if this is what you're doing, then you should do these three trainings. If this is what you're doing, you should do these two trainings. And it's very specific and laid out. The training is not and very difficult. It doesn't take too long to become an eTest proctor. I think it takes about 30 minutes.

And then we went-- we were part of a special group. And so we did a little bit more in depth on the remote training process. So I can't exactly say how long that is because I haven't done it. I did the more extensive one. But I can imagine it would take too much more time than the rest of them.

Audience: Great, thank you.

Marci England: OK, so I'm rounding the end of this. How are we able to get a link to your testing document? I'll put that in the chat towards the end of our presentation. And then Christina, where do they all leave after we're done presenting?

Christina Hyatt: I don't think they--

Moderator: OK. So I'll just make sure I get that in the chat sooner than later. And then I think there might be someone here from CASAS that if we want that can answer the question about using native language up to the beginning of the test.

Barbara Lehman: This is Barbara Lehman and I'm a Program Specialist for CASAS. And yeah, you can use native language up till the test in terms of getting them started and getting them practicing on the sample, the two sample questions at first. But in an eTest, you can get-- you can use native language to get them set up.

Marci England: Perfect. Thank you for being here and for letting us call on you.

Barbara Lehman: Thank you for sharing. These are-- you guys have done a really nice job. And I know you were part of the pilot program. And so I really appreciate your willingness to share all of this. Because it is daunting for many agencies.

Marci England: Yes, yes. It's a big puzzle. There's a lot of pieces in this box but the word of the picture at the end. And I don't know, Christina, I'm wondering if while I'm presenting, you could put the link to our slides and everything in the chat.

Christina Hyatt: Have you put him in the Google? I don't know that you shared it with me.

Marci England: They're on slide one-- on the first slide and the last slide.

Narrator: OK, perfect, yes. Then I'll take care of it.

Marci England: OK. So thank you for those questions. And yes, Diana, we have really supportive admin and that is a big piece of the successes. We have the opportunity to make mistakes, to try things out, pilot. OK, so now, I'm going to talk about implementation of Civics. And I'll review the resources that were provided, the key steps. First, I'm going to talk about implementation and key steps, which include reviewing all the resources provided by CASAS for California remote testing, determining your testing team, and then identifying the COAAP. So I'm going to hop into that section and then we'll move on to my next section.

So really, the essential first step of implementation for CASAS remote testing is understanding the CASAS guidelines. So make sure that you've reviewed the parameters of these guidelines and how you see that working at your school with the systems that you're creating. You need to check that constantly to make sure that you're still staying in those parameters.

I feel like I'm constantly just making sure-- OK, am I still on track? Am I still on track? Because you want to stay in the lanes that you're provided with. And I appreciate the guidance, but sometimes I start getting a little farther and farther off track. So check that constantly. It was also helpful for me once I reviewed the instructions and went through it all, to make like a cheat sheet for teachers and staff that don't need every single detail, but they need to know the parameters as well. And those that are administering the assessment for quick reference. And again, I'm talking about Civics assessments.

There we go. So the next step in implementation is really this step of taking inventory. This is about really looking at the school resources and identifying who can help. Is it teachers, classified staff, paraeducators? This is a step where you take the self-assessment of your school to identify what you have to work with. Also, you need to look at what students have to work with at this step. What devices do they have at home?

For Civics, are they using-- are they doing most of their Zooms on their cell phone? Do they need to come by and pick up a laptop? What's going on? Are you using Zoom? Are you using Google Meet? What platforms technology wise are students familiar with and using? So this is about taking inventory and getting people on board.

Some examples of questions you would ask at this point are-- do I have a technology team? Or do we have technology support? What will students be using for the test? Computer lab hours, could we increase lab hours to open accessibility for students that have poor Wi-Fi or distractions at home that make testing difficult? Classified staff with flexible hours can be great at supporting this system.

There's always that teacher that's eager, willing to innovate. And Christina said look for emergent leaders. I love that term and I love the people that are just-- oh, how can I help? The key is to look at the pieces you have and identify what you need. So just really kind of taking inventory.

Then the next step is to identify the COAAPs and to teach and assess, to reflect on the needs assessment, students' needs, the school resources, and ask is this doable. So one of the key things I'm going to say here is don't get tricked into picking COAAPs before you have done step one and step two. You'll end up with a COAAP that you're using and that doesn't fit in with the guidelines or the parameters for remote testing, or doesn't fit into the guidelines or the parameters of the resources that you're working with at your school. And that really varies from school to school. So make sure you do these steps one and two first. This will-- if you don't, it'll lead you down a path where you'll end up committed to a COAAP that you can't actually implement.

So the third step of implementation is choosing the COAAP. Check back frequently again to the CASAS website and look at the COAAPs because they're constantly-- right now, they're reviewing them and they're updating them with that lens on remote testing. And they're adding bits of language that include orally or electronically. And they're updating the COAAPs so that they fit better within the parameters of remote testing.

The second-- the first you need to review-- when you get to this step of identifying the COAAP you need to look back at the needs assessment. What did your students say they were interested and need to learn about? And make sure that the COAAP you picked are also relevant to student needs. Then look at the school's resources to make sure that the curriculum that you have supports the COAAP that you're teaching. And that you have those resources available, whether it's BurlingtonEnglish, which BurlingtonEnglish made a wonderful crosswalk between the Civics COAAPs and their lessons that they have. So if you're using BurlingtonEnglish, ask your rep to get that for you. It's really useful. I use mine all the time.

And then you need to ask yourself-- is this doable? So it's important to look and visualize it. And the reason why I have a little snippet of-- this is COAAP 47.2, internet safety, is because I just wanted to point out looking at task three and task four. What I do is I look at these tasks and I say-- I notice that it says here on task three, students can respond orally or in writing. This gives you some freedom. If there's a problem with the online testing and a student quickly needs to move to oral and meet in a Zoom one-on-one with a teacher, it gives you that freedom or that ability to be able to do this.

And then in four, it says-- you'll see this task says-- optimally, the writing task will be created on a computer and sent to the assessor electronically. So before, when we were really heavily paper and pencil, we almost kind of scoffed at this because we were like-- oh, our students don't have computers. This was maybe two years ago. Oh, they don't have computers. Or they don't have time to be online. And I think we were really underestimating our students. And I think this forward thinking was there from CASAS from the beginning. And now, I'm so glad it's there because it's working better for us and better for your students.

So for us, we did find that role playing was difficult because it's very time consuming and it requires breakout rooms. So it's like one on one or appointments, one on one with the assessor and the student. So that's something that we learned at our school was difficult. It can be done, but it's time consuming. And then we also found that portfolio for us, we've shied away from portfolio because it's difficult for us to verify the work is student work, and not like family work, or group work. We want that portfolio to represent everything that the student created which would happen in a classroom but maybe not when students are working at home.