(DESCRIPTION) Text, CAEP TAP. California Adult Education Program. Data and Accountability for 2020 through 21 and More. Dr. Carolyn Zachry and Javier Romero, CAEP Office. OTAN, West ED, and CASAS. August 5, 2020. A photo of a woman surrounded by computer monitors. A photo of two men wearing hard hats looking at a clipboard. A photo of a woman in scrubs with stethoscope around her neck holding a clipboard. (SPEECH) Good afternoon, everyone, and thank you for joining today's webinar. It's titled Data and Accountability for 2020 to 2021 and More. My name is Veronica Parker with CAEP Technical Assistance Project. And today's presenters are Dr. Carolyn Zachry with VCAP Office. Javier Romero was supposed to join us today, but unfortunately, he was unable to. And then we also have Neda Anasseri as well as Penny Pearson from OTAN, Randy Tillery from WestEd, and Jay Wright from CASAS. They will be our presenters for this afternoon. But prior to getting started with today's webinar, I'd definitely like to go over a few housekeeping items with everyone just to ensure that everyone has a great webinar experience this afternoon. (DESCRIPTION) Title, Zoom room controls. (SPEECH) So first, everyone was muted upon entry into the Zoom room. However, if for some reason you find yourself unmuted, please go ahead and mute yourself. We want to make sure that there aren't any disruptions during today's webinar. We are expecting a lot of educators on today's webinar. So we want to make sure that everyone has a good webinar experience this afternoon. If for some reason we do notice that you are muted, we'll-- unmuted, excuse me-- we'll go ahead and mute you ourselves. You are welcome to use your camera by turning it on or off. It's not mandatory. It's totally your discretion. You do have the ability to raise your hand if you have a question. We will reserve time at the end of the webinar to address questions. And then we'll also address them in the chat. So definitely feel free to type your questions in the chat. And someone will get to them whenever there's an organic pause in the presentation, or they'll answer you in the chat. This webinar is being recorded and will be available on the California Adult Education website by tomorrow. (DESCRIPTION) C A L adult ed dot org (SPEECH) And along with the recording, we'll also post the PowerPoint presentation as well as any other supporting documents that go along with today's webinar. Prior to today's webinar, you should have received a copy of the PowerPoint presentation. And we'll go ahead and update the chat with the PowerPoint presentation just in case you did not receive it or was unable to access it at that time. And again, use the chat to ask any questions, share resources, communicate with one another. And again, we will be answering questions in the chat. So definitely feel free to use that. If you experience any technical difficulties during this webinar, let us know in the chat, and we will address you via a private chat. And how that would look is our name, so CAEP TAP. And it'll have "private" in parentheses in red. So that's our way of communicating with you without disrupting the webinar to address any technical assistance needs you may have. We are taking attendance for today's webinar. So if you have logged on and you are using a name other than what you registered with, definitely be sure to let us know who you are in the chat so that we can capture your name appropriately in our attendance records. Again, if you've used an acronym, you've logged in with your consortium name, or you've used your agency name or something like that, definitely please let us know who you are so that we can make sure that we capture your information correctly in our attendance record. So that is all that I have for now. And I will turn it over to Dr. Carolyn Zachry, who will get us started with today's webinar. Are you there? I am. Well, good afternoon, everyone. And welcome to the webinar. I have the pleasure of going over a couple slides with you related to, first of all, the agenda. And today on our webinar we're going to talk about COVID-19 effects on adult education. We're going to do a CAEP overview for the 2021 academic year. We'll talk about distance learning strategies. We'll also have-- with OTAN, we'll have Randy Tillery talk about the colleges' MIS reporting. And then Jay Wright will follow that, talking about TE reporting, and-- for the 2021 year. And then we'll follow-- we'll end with some questions. So we can go on to the next slide. (DESCRIPTION) Title, COVID-19's Effect on CAEP Agencies (SPEECH) And so next slide. Oh, I'm looking at two different screens. So I should look at this screen. So we know that-- based on our survey data that most of the agencies in the state closed during COVID and really moved to distance learning options. I just want to thank you all for how fast you all pivoted to really ensure that your students are continuing their learning. And whether it was through online options or through packets, you all really stepped up to the plate to make sure that students continued on their trajectory to their goal. And so I really want to thank all of you for that. And I know that it's looking like that's what's going to happen in this next fall, here, starting with school mostly being online. We are going to be sending out a survey in the next couple of weeks. It will be a Google survey. The link will be posted on the CAEP website, and also on the OTAN website for WIOA agencies-- really asking what your agency's startup plans are for this school year so we can gather some of that information. Next slide, please. (DESCRIPTION) Title, OCTAE Memorandums. (SPEECH) So I want to remind you-- these have been posted before on various webinars. But we have really been-- we've been following the directives as OCTAE sends them out through different memorandums, 20-3, 20-4, and 20-5. (DESCRIPTION) HTTPS colon slash slash WWW 2 dot ed dot gov slash policy slash adult ed slash guid slash memoranda dot html (SPEECH) And that has really helped us to determine how we are dealing with assessment during COVID. And as you know, CASAS was able to find ways to have us do online assessment. While it's not mass assessment like you do now, 30 students in a classroom, you are able to do those post-tests on students. And in the last memo from OCTAE, 20-5, they talk about the fact that you may not be able to do a pre-assessment right now, but you can certainly enroll a student and do any type of a leveling-- a level determination that you can to place them in a program. And then, when you're able to do a pre-test, please do a pre-test as soon as possible. But that was in that last memo. And you can find the link to all of those from-- on the website. (DESCRIPTION) HTTPS colon slash slash WWW dot casas dot org slash social dash media dash newsroom slash 2020 slash 03 slash 27 slash cases dash testing dash during dash the dash covid dash 19 dash pandemic. (SPEECH) Next slide, please. (DESCRIPTION) Text, OCTAE Memorandums. Summary of OCTAE Memos. (SPEECH) Well, I just said all of this. (DESCRIPTION) Text, Statewide performance reporting to the NRS is still required. (SPEECH) So really-- so what those memos talked about was, one, they're still holding us accountable. So there's no change in performance reporting for this year, that states are using a-- what's called force majeure to say that while this student was still enrolled in our program, we weren't able to do a post-test on them. And OCTAE's gathering that information so they get a sense of how many participating students we have across the country that were still enrolled, but because of COVID were not able to get a post-test. Obviously, states can decide whether they want to allow or not allow remote testing. California has allowed remote testing. And we encourage you to find ways to do that. I've heard of some creative ones, such as testing in a parking lot with everyone in their own car. And that-- we had to submit that we were going to allow for remote testing to OCTAE. And, let's see. 20-5 talks about what I was just saying about participants can self-report into a federal education functioning level without that pre-test, but that in order to get a measurable skill gain you will have to have the matched pre and post-test. So you will have to, at some point, get that pre-test in and then move to-- and then-- then on to a post-test. Next slide, please. (DESCRIPTION) Title, P Y 2020-21 CAEP Program Structure. (SPEECH) So that's the-- those are pieces that really are governing how we're doing assessment in California. And when we look at CAEP, I just want to remind everyone of the program structure for CAEP. We have five main program areas-- adult basic and secondary education, English as a second language, career technical education, adults with disabilities, and parents/K-12 student success. So those are those five main areas. Next slide, please. (DESCRIPTION) Title, P Y 2020-21 CAEP Program Structure. Five CAEP Main Program Areas. 1, A B E slash A S E, 2, ESL, 3, CTE, 4, adults with disabilities, 5, parents slash k12 student success. (SPEECH) So same information. Go on to the next slide. (DESCRIPTION) Title, P Y 2020-21 CAEP Program Structure: Primary Programs. (SPEECH) So in these areas I want you to look at the fact that we have-- in the adult basic and secondary education we have basic skills, which is the Adult Basic Education, so ABE. And then we also have the high school equivalency path that students may opt for. And that is-- in California, the students have the option of the HiSET or the GED, as well as agencies have an option of which one of those they'll be supporting. And then we also have the high school diploma route. Then we have English as a second language. And then CTE, which is short-term CTE, pre-apprenticeship, and workforce preparation. Next slide. And I am going to see if Javier has made it on. He had sent me a-- a link, or a text, that said he was having a hard time getting on. And I don't want to be taking all of his slides if he's here. Javier? Are you here? Well, not hearing that. So you can go on to the next slide, because this seems like it's-- those are redundant. Great. Thank you. (DESCRIPTION) Title, P Y 2020-21 CAEP Program Structure: AB104 Outcomes. (SPEECH) So when we look at those program structures and the primary programs that we offer, we also want to then look at outcomes for our students and how we are going to help our students to be successful in reaching their goals. And so we want to look at outcomes in literacy gains, whether a student earns a high school equivalency or a high school diploma, how they-- if they transition to post-secondary education on the credit-bearing side, if they enter employment-- they didn't have a job and now they have a job, if they've had increased wages, or they transition to post-secondary. Next slide. (DESCRIPTION) Title, P Y 2020-21 CAEP Program Structure (SPEECH) So when we look at our students, our adult students, we want to look at what we call our students that are served. And they fall into three different categories. And those are those students that only received a service, then those students that receive one to 11 instructional contact hours. And then, once they hit that 12-hour mark they are-- in a single program year, they are considered a participant. And when we look at this data and when you're looking at your local data I think it's very important to be looking at those students that fall into that number 2 category, students receiving one to 11 instructional contact hours, who don't make it to that participant level. And it's important to look at that to find out why. Did they come to an orientation and then were not able to enroll in classes that they wanted to because of the time that the classes were being offered or because they have child care issues or transportation issues? So I think it's important for us to explore, as you can, why those students don't make it into that participant level, because if they don't make it into that participant level then you're not going to be able to have that outcome data on those students. Next slide, please. (DESCRIPTION) Title, P Y 2020-21 CAEP Program Structure. (SPEECH) So when we look at enrollment and instructional hours for those agencies that are K12, adult, or county office of education, you really want to look at the-- if a class is identified as integrated, that the hours will be divided equally between programs designed for that record. And if it's not integrated or if the hours are split unevenly, the agency can create two classes. And Jay, you're really going to have to jump in on this slide. Oh, OK. For this one, this is talking a little bit about that hours by program report that we've had to do a few times. Thank you. So this was a big issue last year, where a lot of classes are designated for more than one program. So the short answer is, if you mark two programs for one class, by default it will just split 50-50. So if you mark a class for ESL and workforce prep, for example, half the hours go to ESL and half the hours go to workforce preparation. If for whatever reason you know that in reality that class should be 80% ESL and 20% workforce preparation, that's fine. But then you'll want to create two separate classes and just carefully allot those hours accordingly. Thank you, Jay. Sure. So that has to do with those classes that are overlapped in those areas that are-- yeah, workforce prep, for one. That's really the big one for me, is that workforce prep-- that a student might be working on their high school diploma and also be considered workforce prep. And so you have to split that up. Great. Next slide. And for the colleges, they have to do a little more exploration on how the colleges are coding their classes. (DESCRIPTION) Enrollment and Instructional Hours (SPEECH) So when we talk about what an instructional hour is, in order to meet the guidelines it needs to be associated with an instructional program. And so therefore, service hours can't become mingled with instructional hours. And moving forward, we are not going to be tracking service hours. We're only going to be reporting instructional hours for NOVA program area reporting. Can I-- I'll just add something here if you don't mind. Yeah, add on it. Is that this is a repeat from last year. A year ago, this was a really big deal. This year, probably not. But I'll just say it was such a big issue coming forward at the beginning of last year that we kept the slide in just to keep it going, that this is the same as what we were talking about a year ago. Yes. Thank you. And then finally, from my section, the next slide, we're going to talk about contact hours. (DESCRIPTION) Contact Hours Definition (SPEECH) And these contact hours-- and this is really important, especially when we're thinking about distance learning. So hours of instruction or instructional activity that the participant receives from the program. And so that instructional activity could be program-sponsored activity that is to promote learning in the curriculum, such as classroom instruction or online instruction, assessment, tutoring, or participation in a learning lab. And so you can look for more information on the NRS website on that. (DESCRIPTION) HTTPS colon slash slash WWW dot nrs web dot org slash sites slash default slash files slash nrs t a guide dot pdf. (SPEECH) But I think it's important to note that the time that's used to simply administer tests such as GED tests cannot be counted as an instructional activity. And so now I'm going to hand it off to OTAN to talk about distance learning. (DESCRIPTION) Title, Distance Learning. Distance Education - Formal learning activity where students and instructors are separated by geography, time, or both for the majority of the instructional period. (SPEECH) Thank you, Carolyn. This is Penny Pearson. I'm a coordinator for distance learning projects here at OTAN. And I'll just speak for a few minutes about distance learning. And just to recall and keep in your head that we are here to help you with these types-- this type of program. And the slide here is delivering the NRS Technical Assistance Guide definition of distance education. And I think it's just very important to see that first sentence of, this is really a matter of, we're separated from our learners. So it can be by geography, time, both. And this is for whatever those instructional periods are. And the method of delivery-- I want everybody to keep in mind that you have a lot of flexibility here. You can do this through media. It could be print. I mean, I will say it-- the good old-fashioned paper packets still work. You can have your learn-- teachers do audio recordings. You can do video recordings online. You can have them working in computer software programs. So there's lots of options for allowing learners to continue toward their educational success. (DESCRIPTION) Text, page 59 NRS Technical Assistance Guide, HTTPS colon slash slash nrs web dot org slash sites slash default slash files slash nrs dash T A dash Guide 8 2 0 1 9 dot pdf. (SPEECH) So teachers have the tools necessary. And if they're a little uncomfortable, there are resources available to help them to learn about these new tools of communicating and engaging with learners at a distance. And these are the most common ones that are listed on the page, here, in terms of mail-- still works good, snail mail and email. And telephones, as well. And of course, we have all of the online tools. And we have a lot of teachers that use resources like WhatsApp and many others-- Remind and those ways to engaging with learners. So this is the definition. It's really that matter of being separated by space and/or time. So if we go to the next slide, we'll talk a little bit about some of those educational strategies that can be brought to bear. (DESCRIPTION) Title, DL Strategies. (SPEECH) And again, this is also defined in the Technical Assistance Guide. And there's links on the side. And I don't know if maybe Veronica has a moment to drop those into the chat so folks can get to it. But there are-- the definitions within the NRS does have a differentiation between blended and distance. And it's looking at distance as anything over 50% of the instructional time. And that can fall into several different categories of, how do you determine these hours? And these are these different models that it talks about in the Technical Assistance Guide-- clock time, teacher verification, and learner mastery. And so I'll give you just the short versions of these real quick. But it-- they're wonderful opportunities that-- and flexibility, I need to say-- flexibility of how agencies can track their learners. So the clock time models is just, what are those contact hours based on elapsed time that a participant is connected to or engaged in some kind of online or standalone software program that typically tracks time? Many of you may already be using those types of programs online where you can go into the back-end database of your learners and you can see exactly how much time they spent on the learning module, the lesson, the project, whatever the case may be. The pros to this, of course, is that the system captures that time. The data can be typically uploaded into your reporting software. The cons of that is teachers may not always have a full understanding of how their learners are progressing. And that's typically needing more training. Are they really learning the materials, or are they just hanging out in the software? Most software programs have contingencies like if you're inactive for a certain period of time, it automatically logs you out. So it does provide ways that teachers can keep a closer eye on that clock time model of how their learners are remaining in the program. The teacher verification model is really about how a teacher is assigning a certain amount of time or credit or contact hours based on the teacher's determination of the learner's engagement in that material and the completion of that material and the actual comprehension of that material. And the teacher makes that determination because they're typically the ones that are reviewing the curriculum. They're writing the curriculum. And they're delivering the curriculum. So the last model listed here is the learner mastery model. Now, this is something that I-- is my personal favorite. So it's my opinion-- is a way of a teacher being able to, because they're the teacher-- they know their materials. They have an assignment of a fixed number of hours of credit based upon the learner passing some type of cumulative assessment on the content of the lesson, the course, the unit, the module, something. And this can be done by-- provided by vendors, where they will say, yes, we have tested with our learners, and it typically takes a learner three hours to get through this lesson in order to achieve a learning mastery of 80% or better. Some teachers will put that mastery level higher. And that's entirely at the discretion of the teacher. It's just, how are you going to report those hours? So this is something that allows the learner greater flexibility at a distance, because they are-- typically, they're going to do their studies from 10:00 PM to 2:00 AM. And teachers are going to deliver direct instruction via another means or method such as online conferencing, like Zoom, or perhaps asynchronously, when they deliver-- working with discussion boards online, in an online environment. So these types of strategies are not only covered in the Technical Assistance Guide that's linked there, but also, we're providing a link to what's called the Ideal Handbook. This is a partnership that we have with World Education. And they've worked very diligently on creating literally a handbook-- how to do DL, Distance Learning. And that particular document provides information on, how do you set the stage for a distance learning program? How do you work on recruitment of learners? How do you screen learners so you ensure that they are successful in this endeavor? Providing really good orientations to online distance remote learners. And then, what does that instruction look like online? What does the assessment look like online? And then some other considerations that administrators may have for creating and/or monitoring and growing a program like this for distance and blended learning. So if you have questions in these areas, you can always reach out to OTAN and myself. My colleague Neda Anasseri will talk in just a moment about some other resources that we have available to help teachers not only understand the technologies available, but also, we can help with if you're looking to grow, start, or even refine your distance learning programs. We're definitely here to give you a hand. So I'll let us go to the next slide and allow Neda Anasseri to talk about some of these other resources. (DESCRIPTION) Title, Resources and Tools (SPEECH) Thank you, Penny. Neda Anasseri here from OTAN, technology projects coordinator. And I wanted to say that Penny briefly discussed the-- both-- all three models. And she actually did a complete webinar on some scenarios and some other guidance. So you can find that on our website if you need a little bit more information, the webinar is recorded. And there is a slide deck attached to it. All these links on our slides today are live. So I noticed that Veronica added the presentation. So you can definitely download it. And then you will be able to access all these links. So the OTAN website will be your home to find everything-- resources and tools to help support you when you're building your distance learning programs. So I want to bring your attention to the COVID-19 Field Support page, where you'll find additional resources, along with links to help you guide you to the webinars and slides that we've done over the last several months that are recorded and have presentation slides attached to them. The Teaching With Technology resource is additional assistance and resources and tools to incorporate into your courses, your distance learning courses. And the Curriculum Offers page-- that is a page where you can access some of the free resources we found in open educational resource land, CK-12 and USA Learns, and some other helpful resources for not only your students, for your teachers. So if you're looking to build up your teacher skills in blended and distance learning there is a course through Essential Education along with the ISTE SkillRise initiative that assists and helps you-- guides you with a vision and what it means to develop more of a distance learning or blended learning course. For your students, CK-12 addresses some of your ASE/ABE needs, especially if you're looking for additional resources to assist your students in high school equivalency. And of course, everybody knows USA Learns and the Learning Upgrade is also a pilot that you can potentially use. They have a specific digital literacy course that maybe your teachers might benefit from, especially your students will benefit from. I definitely want to remind you that the membership for OTAN is free. So if you're not already a member, definitely sign up to become a member, because with that, you'll be notified immediately when our webinar series-- and when we're offering webinars, at what time they are so that you can attend them live and/or tune into the recordings, along with the YouTube channel, which will have all of our recording. And if you haven't already followed us on any of our social media platforms, please do, because in addition to getting an email to remind you of all of our upcoming webinars, our social media will also be reminding you. So I will look at the track and see if we have any questions. But next slide. (DESCRIPTION) Text, CAEP M I S Reporting for Colleges 2020 through 21 (SPEECH) Thank you, Neda and Penny, as well as Dr. Zachry and Jay, for your presentations thus far. Javier Romero has joined us. I wanted to allow him the opportunity to come on, to greet everyone. Javier, are you there? I am. I apologize to everyone for joining you late. I was having technical difficulties on my end. I think between the internet and my aging computer, I think that would prevent me from getting on time. So I want to thank you all for staying engaged, attending the seminar-- this webinar, and actually staying engaged and helping us continuously improve on our data and accountability system. And I want to thank everyone that's our presenters today for all the great work bringing us to where we are today. I hope you have seen the improvements over the years. And please stay engaged. And I guess I want to assure you all Neil's doing well. He's recovering. So we miss him deeply-- more than you'll ever know. But I want to thank you all and turn it back to our presenters. Thank you, Javier. And next up we'll have Randy Tillery with WestEd, who will be covering MIS reporting. Thank you, Veronica. Hope everyone's doing well today. So I'm going to be talking about two issues at a fairly high level. So first we'll be-- I'm going to be going over issues related to MIS reporting for the colleges and issues relating to colleges who are reporting both into MIS and to Tops Pro Enterprise if they're WIOA Title II funded. And then we'll be pivoting after that into just a brief overview of the LaunchBoard. I'm not going to talk about it extensively-- mostly what's new or different or what's changing for this year, as well as some of what people can expect to see happen during the coming year. If you are new to CAEP, like you're a new consortium director or agency lead, new researcher or data person, just be aware that this-- some of this may be-- seem like that Jay and I are speaking in code. We have discussed the possibility of doing a deeper dive webinar on data and accountability issues just for new directors and new agency folks at some point in the near future. So look for something like that, I think probably within the next couple of months. So next slide. (DESCRIPTION) Text, California Community Colleges. CAEP M I S Reporting for Colleges 2020 through 21. (SPEECH) So this is fairly well-trodden territory at this point. But just to be aware-- and this has been the case for two or three years now-- for the purposes of CAEP reporting and your CAEP data, college districts are required to enter and collect all of their-- collect and enter all of their data regarding required CAEP adult learner, student enrollment, and other data into their management information system at the college. Now, that's just the data system into which all of your student data goes, whether it's a credit or a non-credit student. Virtually 99% of all of the fields and data elements you need to capture information about your students are in MIS. And then what we get are more validated data records when you upload that data to the Chancellor's Office MIS system, or COMIS. The data uploads-- it's not a separate reporting process for MIS data collection. So basically, you're just enrolling your students, you're entering their data. There are some specialized data fields that you should pay attention to. And I'll note a couple of those as we get further downstream. But your college uploads all of their student enrollment and outcome data to COMIS every year at the Chancellor's Office. And that's the data we use to inform the LaunchBoard. There's no separate timelines. There's no separate reporting processes. I'll talk about the timeline for data uploads in a second, because it's probably important to at least know those. But you're not saying, oh my god, I've got to report my data now, at some sort of specific point in time. It doesn't work on a quarterly basis the way it does with TOPSpro Enterprise. If you are a college who receives WIOA Title II funds for ABE, ASE, or ESL students, you are still reporting using TOPSpro Enterprise as required by CDE. Nothing changes. Now, I know that means you're reporting the same student data for some of the same students into two systems. But it's just like any other time where you have a blended funding situation and you have separate reporting requirements. So WIOA title II has its own specialized reporting requirements. And you still have to report that data on a quarterly basis, just like K12 adult schools do. So if you're a non-WIOA title II college, we do have a couple colleges who use TOPSpro Enterprise to report students who do not have a record in MIS. I'll talk about that edge case in a minute. But you can you use TOPSpro Enterprise to report that data. And we will find those students. So the LaunchBoard, which is where all the data goes into, which we update every year, pulls data from MIS or COMIS and from an extract of the TOPSpro Enterprise data set for all of the CAEP consortia in the state. And we use both data sets. We do duplicate them, where the same student show up more than one time. But there are some outcomes we do pull from CASAS, particularly where we get data we don't get through MIS, like EFL gains, high school diplomas and equivalencies that wouldn't show up in MIS by themselves. But just to remember, it's that MIS is the primary validated data source for students, courses, enrollments, and outcome data for the LaunchBoard for the purposes of CAEP recording-- reporting. Next. (DESCRIPTION) Text, California Community Colleges. CAEP M I S Reporting for Colleges 2020 through 21. (SPEECH) Just a couple idiosyncrasies about MIS. So if you're wondering when your colleges submit their MIS data, they submit enrollment files approximately 30 days after the end of every term, whether that's quarters or semesters. It works pretty much the same. The files that your college submits includes data on all of your credit and non-credit students. And we look at that whole data set because we do match non-credit and K12 adult education student records against the credit records to see if students have transitioned into the community colleges for post-secondary employment-- or, for post-secondary education. Certain MIS records, like for awards or things like that, do not show up until after the end of the program year. Oops. Hello? Ron, I think you're getting excited there. So just be aware that not everything goes in after every term, which is one of the things-- one of the things that impacts the development of the LaunchBoard and building of the new site, is when we get the final records from MIS, as well as the data extracts from CASAS. So again, there's no separate reporting processes. (DESCRIPTION) Text, There are no separate processes or deadlines for college programs to report their data. (SPEECH) And you should probably get to know your researchers. Or if you're researchers, get to know your program people and sit down and review the data that you're entering at least once a term to see that you're capturing everything you should. Next. (DESCRIPTION) Text, California Community Colleges. CAEP M I S Reporting for Colleges 2020 through 21. Edge Case. (SPEECH) So this is the edge case. So we have a certain number of colleges or consortia who are using third-party providers to provide educational services. And-- like the-- but those students are not captured in the colleges' MIS systems. So they're not registered or enrolled at the college. So we do have some consortia reporting some students into TOPSpro Enterprise and we do capture those students. So just be aware of that. If you're paying a third-party provider to provide educational offerings and you're struggling with this issue, you should probably go ahead and have them report into TE. That is the simplest and most effective way to deal with that. We also have some limited cases where colleges appear to be using community education as a mechanism for delivering CAEP programs. I will not speak to whether that's allowed or not. My understanding through Neil was that that is not an allowable use of CAEP funds, or for community education courses. But if you're doing that and those students are not in your MIS records you want them reflected, you should be reported them into TOPSpro Enterprise. And then the CAEP office will be dealing with the community education issue later. Generally what we recommend, like when I talk to colleges about this, you can start out by doing something as not-for-credit or community education. But you should be ideally transitioning it into non-credit and running it through the curriculum process so that basically you're using apportionment. And then you could-- that frees up more of your CAEP funds for program development or other uses. Next. (DESCRIPTION) Title, Important M I S Elements - Students, Programs, and Services. (SPEECH) So for your MIS reporting-- and I know that this PowerPoint is available. I'm not asking-- I'm not going to go over all of these data elements. And some of you who came to our non-credit coding training sessions during the last year have seen this slide. But these are some of the most important MIS elements related to students and instructional programs, and then for student services. So TE has, like, 37 different fields for coding services for students. What we use are the non-credit student success elements, SS16 through 20. So you want to pay particular attention to those, because you're prob-- you may or may not be using those to record services for students that-- who are-- that you're actually providing services for. So these are very important data elements. And this slide will be available to you to review. We always encourage you, also, to reach out to me or my team if you have questions about these. Or you can go to the MIS Data Element Dictionary, which is at the Chancellor's Office website. It's actually back up there now. [AUDIO FEEDBACK] (DESCRIPTION) Title, Student Barriers to Employment (SPEECH) Here other-- another place where there really are specialized MIS elements. And one again, we're not going to go over-- some of these were actually created specifically for CAEP. And this is so that we're able to mirror the WIOA barriers to employment, that it was important to the field, when we led the process with K12 and community college practitioners three years ago to actually develop the data and accountability system and definitions, that we use the WIOA barriers to employment. And so these are just the data elements related to those. (DESCRIPTION) Text, Reportable individuals who had less then 1 instructional contact hour or received support services in the selected year are broken up into two categories. 1, Ever flagged as having barriers to employment at any time up to and including the selected year - cultural barriers SG 18, English Language Learner enrolled in ESL, Ex-Offender SG 15, Foster Youth SG 03, Low Income SG 14, Low Literacy SG 20. 2, flagged as having barriers to employment only in the selected year - displaced homemaker SV 05, homeless SG 16, long term unemployed SG 17, migrant farmworker SV 09, seasonal farmworker SG 19, exhausting T A N F within 2 years SC 18, single parent SV 04. (SPEECH) Now, be aware when you're looking at these, if you go spend any time with these, the way they're handled in the LaunchBoard-- for some of them, if you select it at any time, if they've ever been flagged as having a cultural barrier or as an ex-offender, then they're always flagged that way. So you don't have to re-flag them every year. And some of them are flagged only within the selected year, which means you would want to go back in the next year and review your students and re-flag them. Like so if you have someone who's homeless in one year and they're homeless again the next year, you would want to do that update. (DESCRIPTION) Title, M I S Reporting Issues - Educational Functioning Levels (SPEECH) A couple basic MIS reporting issues that come up very often-- we had an issue related to educational functioning levels. When we did the original data and accountability work three years ago we identified three ways that colleges could capture an EFL gain for a student, one of which was CB21 course progressions. So you have math, English, or ESL non-credit courses who have CB21 codes, which was-- formerly was the course level below college level course, which is a tongue twister. But if they went from a course that was coded CB21 equals C to CB21 equals B, that would count as an EFL gain. (DESCRIPTION) SA 07 M I S Data Element - Allows college to enter the student EFL level based on pre and post testing using an NRS approved testing instrument. Fix for logic check with SA 01, enter y y y y when prompted. (SPEECH) The second methodology was a new data element that we created called SA07 which we created, we sent off into the wilderness, and then found out was broken. And it took a while to get it fixed. So there was an issue when you're trying to code an EFL level and they're all-- so you can basically pre-test a student, note their EFL level using SO07. Then, when you post-test them, if they test at a higher level then you can enter a second time the higher level, and you'll get an EFL gain for that. The Challenge was, it was asking you, when you tried to enter anything into there, to actually identify which of the Chancellor's Office approved placement tests you were using, which didn't really apply. And so we had to get MIS to do a fix. So when it asks you for that logic check, you have to answer YYYY. And then it allows you to go ahead and enter your level. (DESCRIPTION) Text, Tops pro Enterprise - A E Pipeline uses EFL data from T E for WIOA Title II colleges using CASAS to report EFL data to CDE. (SPEECH) The third way is obviously-- and this primarily applies to WIOA Title II colleges who are pre and post-testing using CASAS and reporting into TE for WIOA Title II reporting to CDE. So we do capture EFL gains there. And I will say it's about 30% of the EFL gains in the LaunchBoard right now-- I'm totally making up that number, but it's about that-- are coming from the CB21 course progression. So if you're wondering if that methodology works, it actually does. We only saw two colleges last year reporting using SA07, I think because everyone had heard that it was broken. But I have been seeing some chatter related to whether or not colleges are using it and how they might be using it, maybe, this year. So we're thinking we might see some movement on that this year. Next. (DESCRIPTION) Title, CB 21 EFL Alignment. (SPEECH) And this is just if you're wondering about the relationship between CB21 and the educational functioning levels. When we did the original cross walking CB21 with the EFLs three years ago, they were totally different rubrics. English actually aligned pretty well. ESL was wonky. And math was kind of a complete nightmare. But what happened is, with the passage of AB 705 in California, Kathy Booth and I worked with the academic senate to lead a process with community college faculty and with adult education faculty to align CB21 to the EFL. So now there are six levels to CB21 for math, English, and English language proficiency, just as there are in the EFLs. The rubrics for CB21 are more expansive and more textured because there are-- we use the common assessment initiative rubrics that were developed quite a while ago, that had a lot more detail in them to support each level. So if you're-- even if you're working in a K12 adult education environment it's worth having a copy of the CB21 rubrics so you can really see the additional level of detail that's in there and how it can inform conversations you can have with your college about transition to post-secondary education and how you can leverage that work. Waiting for the slide to populate. (DESCRIPTION) Title, M I S Reporting Issues - Adults Served and Participants. (SPEECH) This is fairly well-known information, but there were a couple kinks that came up this year when we were comparing the CASAS data to the LaunchBoard data and we were coming with different numbers for a while. But just a couple of distinctions about how we think about adult education students-- so we have a category called adults served. That's what the metric is called in the LaunchBoard. And that is basically any student with one or more instructional hours in a CAEP program or who receives a service. So they don't have to be enrolled in a program. If they come and they can get some services from you, we still count that as an adult education student in California. This roughly corresponds to the WIOA definition of reportable individual. It's just a little-- made a little more concrete for the context of how we're handling this in California. And then, as Carolyn talked about earlier, a participant is any student who receives 12 or more instructional contact hours of instruction across any CAEP program in the same program year. So it doesn't have to be in ESL or all in ABE or all in CTE. If they did three hours of instruction in four different program areas and it adds up to 12 at the end of the year, then they're a participant. The importance of the distinction is that when we look-- all the outcomes that are reported in the LaunchBoard are reported for participants. And so if a student has three instructional hours, they just received a service, we aren't currently reporting outcomes for those students. And the important thing to know is that it's important that the data be complete. So sometimes we have found discrepancies in the data sets where we have a last name, or we don't have a date of birth, or we don't have the gender. So we only report them in the LaunchBoard if we have a first name, last name, date of birth, and gender. So just be aware of that when you're collecting data from your students, that that's the way-- that's the way we count these things. We ran into some issues-- so CASAS provides very useful reports to the agencies who are reporting into TOPSpro Enterprise. But they-- I think the definition of enrollment in that report includes students with less than one instructional contact hour. And my first understanding is that it's helpful for local agencies to vet against their own records. We do not count those students in the LaunchBoard. So if you see a discrepancy between that enrollment number and the adults served in CAEP, that's-- in the LaunchBoard, that's one of the reasons why. Next. (DESCRIPTION) Title, M I S Reporting Issues - Special Admit Adult Education Students (SPEECH) So this is a new MIS reporting element. So SB554, which I think was passed last year-- oh, Carolyn, I didn't put the year in here. I'm bad. We were having a conversation about, we put these numbers in here, forgetting that they reuse the numbers every year. And so people think of different legislation. So SB544, as it was passed by the legislature, was passed to support dual enrollment and special admit status, which is a-- like, if you have a high school student who's also taking college classes, this is called a special admit student at the college. And then there's flexibility, then, in terms of those students. And the colleges are-- they're able to be admitted to two systems at the same time. So this strengthens dual enrollment and co-enrollment strategies for the systems. There were two MIS data element changes associated with this. SB11, the Student Basic Data Elements, created a new flag for adult education special admit student. (DESCRIPTION) SB 11 dash 21 0 0 0 (SPEECH) One of the most important things this does is that it creates a change in SB15 so that the student is not identified as a first-time student. So it maintains their eligibility for College Promise should they enroll as a first-time full-time student in the future, or if leave adult education then take 9 or 12 units at the college. (DESCRIPTION) SB 15 dash y y y y (SPEECH) There are other benefits, as well. And I'm sure, if Matt's on this call, he's probably damning me right now because I'm not describing it very well. But just be aware of this change. And I think it probably would be good for us to cover this in more depth in a future webinar around the broader intent of the legislation. Next. (DESCRIPTION) Title, M I S Reporting Issues - WIOA Title II Colleges (SPEECH) Well, this is really just-- I threw this in, again, only because I do continue to get the question, if we're reporting into MIS, do we still have to report using TE? Or if we're reporting in TE, do I have to report everything into MIS? And basically, you have to report into both systems fully if you're a WIOA Title II college. We do use both data sets to populate the data on the LaunchBoard. And there are specific data elements where, if you're WIOA Title II as a college, we may pick up EFL games that we might not pick up through CB21 or SA07, in particular because there are some ways in which colleges are coding their non-credit math and English colleges which means we can't capture a level gain, because those courses do not have CB21 codes assigned to them. And that's because colleges are using the top codes for basic education and secondary education. And then they CB21 code those at equals Y, which actually means that we miss all of those potential level gains using the CB21 methodology. (DESCRIPTION) Title, Adult Education Pipeline. (SPEECH) So just a couple things about the LaunchBoard. If you are new to CAEP, I would encourage you to google Cal-PASS LaunchBoard. And it will take you to a link. So this is a publicly accessible dashboard that rolls up all of the CAEP data from K12 adult education, community college MIS system and then matches that against the EDD wage file that allows us to capture-- it's probably the first fully functional integrated intersegmental dashboard in the country that I'm aware of, that I've seen. I presented this at COABE a short while ago, and no one's shown anything to me that looks exactly like this. But it captures-- it follows the student journey. So it captures enrollment-- who's in your programs. It captures progress, captures transition, captures success. And it captures employment and wages. Next. Yeah, go ahead. Forward. No, go forward. (DESCRIPTION) CAEP Student Metric Buckets (SPEECH) So this shows the metric buckets. It closely models what you saw in terms of the structure of the dashboard itself. But under Participation you can see adults served, participants. You can see program enrollments. You can see demographics for your student population and progress measures. You can see EFL attainment. You can see workforce preparation milestone attainment. You can see occupational skills gains. We have transition from adult basic or ESL to adult secondary education and transition to post-secondary education. We show completion of diploma or high school equivalency and attainment of post-secondary credentials, whether that's at an adult education CTE certificate, whether that's a low-unit or high-unit college credit certificate, or even an associate's degree. Now, obviously some of these are fairly low numbers as the transition work ratchets up across the consortia. But these are important things that people aren't able to capture in some of the other states in the country. Then employment-- we look at employment. We look at wage gains pre and post-participation in adult education. And we also look at living wage attainment. And then on the disaggs-- go to next. (DESCRIPTION) CAEP Metric Disaggregation (SPEECH) So these are all the ways you can slice and dice the data. So you can disaggregate any metric by race and ethnicity. So if you want to see how many African-American students are getting EFL attainment, you can actually look at that. You can look at it by gender, by age. You can just aggregate the outcomes by program area. So if you want to see how many of my adult secondary education students are transitioning to post-secondary, you can look at that. And then we've created two new categories or visualizations for first-time and continuing students. Next. (DESCRIPTION) Title, Disaggregations and Drilldowns. Note, Drilldowns work together, Therefore, a user can see how many first time A S E students or returning A B E female students met the metric outcome. (SPEECH) This just shows what the disaggregations and drill downs look. This is the same information that you can see-- you can disaggregate all the metrics by age, race, ethnicity, and gender, and some of the metrics by program, and some by first-time and continuing. Next. (DESCRIPTION) Title, New Metrics and Views in Build 3 (SPEECH) These are just some of the new metrics. I'm not going to necessarily go through all of them. We are-- as Carolyn was discussing, we now can report adults served with one to 11 instructional contact hours. It allows you to say, how come we're not helping more of these students get beyond 12? Service-only students, participants in workforce prep, first-time or continuing participants. We're looking at participants taking courses in more than one program area. So we have a metric for that. Taking courses at more than one adult school. We're now showing, in the progress section, students who subsequently make it to transfer-level math or English at the college. And we're looking at year-to-year persistence. And there's a community college district view just pertaining to the structure of the data for some of the metrics. (DESCRIPTION) Title, Data and Metric Alignment. Number 1, Alignment with CASAS T E Calculations. A, Aligned program, population, and outcome definitions. b, lowered age to include 16 and over for T E and COMIS data. c, CASAS West Ed review of T E calculations to validate A E 3.0 construction. d, Decision made not align to CAEP definition for Reportable Individuals since students with 0 less than 1 program or contact hours are not included unless the student was flagged as receiving services. (SPEECH) And this is just identifying that we have been very careful about trying to-- for over two, almost three years now, reviewing calculations with CASAS. When we ran into some bumps in terms of data that wasn't matching last year, we spent a lot of time pulling additional data and looking at the reasons why, then making coding changes to be sure that, as much as possible, the data that we're getting from CASAS and TOPSpro Enterprise-- that we're counting those things up the same way that CASAS does. We have the difference around students with less than one instructional pro-- contact area. But as long as you know why those numbers look different, then we knew-- we're comfortable with that. (DESCRIPTION) Number 2, Alignment with Student Success Metrics. A, Alignment if adult ed ESL and short term CTE journey metrics with existing CAEP definitions. (SPEECH) And the student success metrics, which is sort of the one dashboard to rule them all in the LaunchBoard-- there's an adult education and ESL journey. We try to align as much as possible with existing CAEP metric definitions. However, the TE reported data for colleges does not get reported in that journey, which is why the EFL or skills gain metrics are so lowly reported there. And we haven't been able to get the Chancellor's Office to make that shift yet. Next. (DESCRIPTION) Text, A E Pipeline 2020 through 21. 1, User interface and resource improvements. User interface support enhancement to improve understanding of metrics and calculations. Better dedicated A E resources to improve reporting and understanding of data processes and usage. (SPEECH) So for 2021, just some quick things. We're going to be making some user interface upgrades this summer and into the fall. So we're trying to improve the resources and how they're structured. We would like the dashboard to really be self-teaching in a way, that you should be learning about data and data types. We should be explaining better to you what you're looking at when you're looking at an infographic, and how it's defined, and why it's defined that way. You will still have access to the underlying metric dictionaries and all the technical information. But we're trying to do a better job at just being sure that you are able to understand better what you're looking at, particularly if you're new to this kind of data work and looking at educational outcome data, particularly in this format. Build 4 we are shooting, once again, for the March 1, 2021, which is the date when the final report is supposed to be due to the legislature for CAEP. So that allows us to meet that deadline in a couple different ways. We're going to be doing a lot of professional development this year. All of the work of our LaunchBoard dashboard teams are focusing more on professional development, so using LaunchBoard data for program improvement and planning. We'll probably do more work on non-credit code alignment. We would actually like to pull non-credit coding data for colleges and go sit with them and look at that data and explain why it's affecting their outcomes in certain ways. We want to do some training around CB21 and AB705 to create stronger core sequences to college-level courses and programs using the alignment work that we did last year. And then we have a CTE mapping project that's been going on for about a year now. And we're going to be releasing a beta version of an adult education to workforce data tool that we'll be doing some work with afield around this year. And we're going to be looking at some, maybe, cohort-based metrics this year. And then, yet again, we're going to be having our annual conversation with the Chancellor's Office about things we would like to see change in MIS that will make reporting better and more accurate for you. (DESCRIPTION) Text, Adult Ed Education to Workforce Dashboard. August 6, 2020 11 A M to noon. Register at cal adult ed dot org. (SPEECH) This is just a shameless plug for the fact that on Thursday, at 11:00 AM, we're going to have a webinar, basically, that's going to look at the adult education to workforce dashboard. So it's going to be at 11:00 AM on Thursday. We basically collected-- the short version is that we collected course data for every K12 adult education school and community college non-credit program in the state and then organized those by what we're calling occupational metaclusters, and looked at the relationship between those offerings with each other with credit programs that are offered in the same cluster area and/or the regional labor market data. And you can cut the labor market data lots of different ways. It's a flexible tool. It allows you to look at the data by region, by occupational cluster, by educational level required for employment. So it's a really interesting tool. We're really just trying this out for the first time. So we'll be showing that on Thursday. And we encourage you to hop on and hear more about it. Next. (DESCRIPTION) Title, Resources. 1, COMIS Data Element Dictionary, back on the C O website, HTTPS colon slash slash web data dot c c c c o dot e d u slash d e d slash d e d dot htm. 2, California Adult Education, https colon slash slash cal adult ed dot org. 3, Chancellor's Office Datamart, https colon slash slash data mart dot c c c c o dot e d u slash outcomes slash default dot a s p x. 4, CASAS, https colon slash slash casas dot org. 5, LaunchBoard Resources, https colon slash slash launchboard dash resources dot west ed dot org slash resources question mark t underscore id equals sign all. 6, LaunchBoard and Noncredit FAQ, Available on LaunchBoard Resources, cal adult ed dot org, and RP listservs, (SPEECH) And I believe this is my last slide. So this is-- you can come back to this PowerPoint. And these are links to various resources-- the COMIS data element dictionary, well, the CalAdultEd.org site, Datamart, CASAS, LaunchBoard resources. And then we also built a LaunchBoard FAQ document based on the work we did in the non-credit coding sessions this year. That's a pretty large document with a lot of useful information. And we encourage you to go there. And I think that's all I've got. Thank you, Randy. We will continue the presentation on with Jay from CASAS. However, Randy, there are quite a few questions in the chat. So if you'd like to address those, that would be helpful. If not, we can save them till the end of the webinar. Can I do them in the chat, some of them? Yes. Absolutely. Jay, are you there? Yes, I am. I was just waiting for my cue. So is this it? Yes, take it away. Is this the cue? Yes. Can everybody hear me OK? Just making sure. Yes. OK. Sorry. (DESCRIPTION) Title, WIOA Alignment to A B 101. The WIOA Performance Indicators, along with 5 types of MSG, comprise the framework for the six AB 101 outcomes. Indicators - 1, Employment. 2, Wages. MSGs - 1, Literacy gain. 2, secondary. 3, post-secondary. 4, training milestone. 5, skills progression. A B 104 Outcomes - 1, improved literacy skills. 2, completion of high school diplomas or their recognized equivalents. 3, completion of postsecondary. 4, placement into jobs. 5, improved wages. 6, post secondary transition. (SPEECH) So anyway, moving on here to slide 39, we can go to the next slide. I'll just-- before I jump into the specifics I'll just say, these are a little bit all over the map. There's some intro-level slides and some advanced-level slides. Definitely a mixed level batch here. So this is what we were talking about at the very beginning, when we looked at the outcomes. We aligned it to the federal system so that the feds have their measurable skill gains and other indicators. We basically used that to come up with the six areas of CAEP outcomes. Next slide. Detailed here in this next slide, 40. (DESCRIPTION) Title CAEP Outcomes. Literacy gains - pro/post level completion, Carnegie units/ HS credits, CDCP certificate, occupational skills gain, workforce preparation. HSE/HS Diploma - high school diploma, passed G E D, passed H I SET, passed TASC. Post-Secondary- College Degree- A A, A s, B A, B S, graduate studies, training credential, occupational licensure/ certificate, apprenticeship. Enter Employment - get a job, retain a job, enter military. Increase wages - increase wages, get a better job. Transition - Transition to A S E, transition to post-secondary/CTE, transition to post-secondary/college. (SPEECH) This is one that most of you have seen many, many, many, many, many, many, times. This just lists the six categories of CAEP outcomes and itemizes out exactly which outcomes fall into each of those six categories. Definitely a Pete and repeat. Next one, I think another Pete and repeat. (DESCRIPTION) Title, Literacy Gains. A two column chart with AEBG Outcome on the left and Recording Method on the right. (SPEECH) This is detailing the literacy gains area specifically. Of the six areas, this is definitely the one that generates the most questions. Part of that is because it's the one that includes pre and post-testing. Part of that is that that's where we've added some retro fit areas to mark measurable skill gains for programs such as CTE and workforce preparation. I won't go into all of them here on this slide, but this just shows you what the literacy gains are and how we account to it. (DESCRIPTION) Outcome, pre/post test gains. Method - enter pre/post test results. Outcome, carnegie units. Method, no bubble but via self reported level. Outcome, CDCP certificate. Method, mastered course competencies and skills progression. Outcome, occupational skills gain. Method, met work based project and training milestone. Outcome, workforce preparation. Method, acquired workforce readiness. (SPEECH) Obviously, for pre, post we use the actual test results. For some of those other categories we've specified particular bubbles or checkboxes to serve as the way you mark it. Next slide. (DESCRIPTION) Title, Literacy Gains - HS Credits. (SPEECH) What we're going to do is not go over every single outcome in the interest of time. I did note a lot of new folks interested in, maybe, a lower-level session. So I will take that to heart, that we might want to schedule something here later in the month or in September that gets into the CAEP basics. But for now we're just going to select specific areas that tend to generate the most questions. So one of them is the high school-- area of high school credits that is in CAEP. We use the high school credits outcome that's been identified in the WIOA federal system. That is, instead of showing gains through standardized tests, you're showing gains through credit attainment in a high school diploma program. So what the feds say if a learner advances from ASE low, which is the ninth or 10th grade level, to ASE high, which is the 11th or 12th grade level, within a year-- that basically constitutes a measurable skills gain. (DESCRIPTION) Text, In T E, go to Records, Students, Records and refer to instructional levels. Select A S E Low upon enrollment. Select A S E High later in the year once student progresses to the 11th or 12th grade level. (SPEECH) There are ways to mark high school credits in TE. However, every district manages it a little bit differently. So we've decided all along that that just won't work successfully. So we use the self-reported level field as you can see in the screenshot, where you can mark ASE low at the beginning of the year. And then, sometime later in the year, you mark ASE high to show that that student made progress through high school credits. Next slide. (DESCRIPTION) Title, Literacy Gains, CTE Related Outcomes. (SPEECH) The next big highlight is-- this has probably the biggest one. This is a major Pete and repeat, is we have two literacy gains, quote unquote "outcomes," that we've identified for CTE and workforce prep programs. That is, for ABE and ESL we do testing. For CTE and workforce prep, probably not. But we want to have other ways to show that those students are making gains, just like students in ABE, ASE, and ESL are. So we have what's called occupational skills gain and workforce prep milestone. These are-- there's no real specific error you make if you confuse the two. But I would say, in general, use occupational skills gain to record a partial accomplishment of a longer-term program, whereas workforce prep milestone represents full achievement of a short-term program. So the example for occupational skills gain is, you've got a welding student. You've got a welding student who is in a five-semester long program. The student completed the first semester. They passed a quick little exam to show that they're on track. So they now move in to module 2, or the second semester. That would be an example where you could record occupational skills gain. The example for workforce prep milestone would be a 15-hour class on job search strategies. So they finish all 15 hours. They're still around. So that would be an example of workforce prep outcome. You can see in bold, we put the skills check or the written test in there. A lot of you remember from last year, that is how we modified this a little bit, is at the federal level they changed this from skills progression to passage of an exam, meaning that it's basically the same federal outcome. But the feds were saying, yeah, there should be some sort of written documentation that shows that the student really made this progress. So we basically passed on that love to CAEP outcomes last year and modified these definitions slightly. That's what's in bold in the slide. Next slide, please. (DESCRIPTION) Title, Occupational Outcomes: Post-Secondary versus literacy gains. (SPEECH) This is just another way to look at it. We also have those outcomes in post-secondary. That is completed post-secondary, getting an occupational licensure, occupational certificate. So we're comparing that to these literacy gains we've been discussing. So for literacy gains, that's either partial completion of a longer-term program or full completion of a shorter-term program. We would say this outcome rises to the level of post-secondary when the student achieves full completion of the longer-term program. So using that other example, you might say the student moves from module 1 to module 2. And that five-module welding example, that would be an example of occupational skills gain-- that is, the student progressed within the program. You would have that rise to the level of post-secondary when, in my example, the student completed the fifth module and achieved that certificate that officially certified that person as a welder in State of California. That's when you would mark the licensure certificate-- that is, you'd mark a post-secondary outcome, not just a literacy gain. So next slide, please. (DESCRIPTION) Title, Passing Knowledge-Based Exam. The Skills Progression MSG has now been replaced by passage of an exam. Learner passes an exam during the year that is required for a job or that demonstrates progress in attaining technical or occupational skills. Exam can be hands on occupational skills demonstration, written test, standardized pre/post test, or other method of assessment that clearly demonstrates skill progression or attainment. (SPEECH) So this is looking at it from the federal level. Again, we had that called skills progression. Just added that slide from last year to remind everybody of that change-- that is, we've had these outcomes for a few years now. Last year we modified the outcome to require that exam. It doesn't have to be anything formal. But somehow, you should have the student demonstrate that they really did obtain that skill in order to mark it. Next slide, please. (DESCRIPTION) Title, MSG's for CTE. The Passage of Exam Measurable Skills Gain for WIOA aligns with the CAEP Occupational Skills Gain. When a student achieves an Occupational Skills Gain, that now entails that the student passes an exam such as work skills demonstration. The bubbles met work-based project goal and training milestone are shaded in under the title work. (SPEECH) This is just showing you a little bit more in the weeds where to mark it. So for occupational skills gain we have those two bubbles you can mark that equate to occupational skills gain. Next slide. And then the one bubble acquired readiness skills for workforce preparation. A lot of you have noted, they're not always the perfect bubbles. They're bubbles we selected in 2016. We realize there were choices that don't make complete sense. But we've erred on the side of caution and just kept it the same way, knowing that we've disseminated this documentation all over the state now. So making changes just doesn't make a lot of sense. So those are the bubbles we aligned. (DESCRIPTION) Title, Literacy Gains Outcomes - COVID 19 Update. (SPEECH) Next-- this is the next slide. So what we're moving toward is sticking with literacy gains. But we're taking a page from a workshop we did in mid-July as well as in mid-June. This is kind of relating it to COVID-19. When we did those workshops in June and July we were mostly presenting it as a way to look at your annual data from '19-'20 and try to look at ways to show that your students are making great progress, even though a lot of you are closed or exclusively remote during COVID-19 shut down. Obviously, a lot of you were going to continue to be in that mode here in 2021. So we left a lot of this information in. So this is some suggestions we made a month ago. We're continuing them here. So we know that we're doing distance learning. Penny and Neda presented a lot about that earlier. So we're looking at achievements that we're getting as a result of those distance learning classes. So you can still do remote testing, as Carolyn suggested. Lots of creative ways to do site testing, the parking lot testing and so on. So a lot of you have come up with really good ideas to do testing. But all that said, we know that the way it is now, it's much more difficult to test large volumes of students than it normally is. So we're looking at other ways to show that students are still making outcomes and still making good progress. So we're using that passage of exam outcome as one example. So one example would be if they're passing an informal exam such as a virtual assessment, maybe a written assignment, oral interview-- some of the activities you might be working on in your virtual class. Another example might be a skills demonstration and workforce preparation activities. That's obviously a real generic, neutral way to describe what a lot of you are doing in programs such as the EL civics. Next slide. (DESCRIPTION) Title, Literacy Gains Outcomes - COVID 19 Update. (SPEECH) So one way we were talking about are virtual assessments or written assignments. So Penny talked a little bit about learner mastery earlier. That is, you're defining what activities the students are doing and you're crediting the hours once the students complete those activities. I think Penny referenced one way you could do it is provide a simple exam that the students pass at the end of completing the video, or educational software, or web-based instruction, or whatever it is that the student was doing-- is you might have a short class-- or, a short quiz the student passes in order to record hours. That would be one example of the informal virtual assessment where you could record that the student made some sort of workforce preparation milestone. Another example would be educational software. A lot of you have moved into that a lot with distance learning and so on. A lot of those higher-end software products have quizzes and checks for understanding built into the software. So when students complete those quizzes on the software and they're moving from one unit to the next, that would be another good example. And then a lot of you have local program milestones that you're already doing. That would be another way to record it. Next slide, please. (DESCRIPTION) Title, Literacy Gains Outcomes - COVID 19 Update. (SPEECH) Another way is skills demonstrations. That's been something you've been doing more in virtual classes, without the-- having the ability to test and do other activities. So EL civics is what I'm referring-- is not exclusively what I'm referring to here, but that's obviously a lot of it, where you're doing things like additional assessments or COAAPs. So most of those are related to workforce preparation. When students pass those, that would be another good way to show workforce preparation milestone. If they're making a transition to workforce, that would be another good example. And then a lot of you have students that are co-enrolled with Title I. So if they're making transitions to Title I or completing some of those activities in Title I, that would be another good opportunity to show student progress. Next slide, please. (DESCRIPTION) A two column chart with AEBG title on the left and update record on the right. (SPEECH) Here's another blanket slide for another one of those six categories. We showed literacy gains. Now we're showing another one, transition. We are just being selective and choosing the ones where most questions have come up historically. So this transition area is another one where we get a lot of questions. So we have transitioned to ASE, CTE, and college. ASE is all under the hood. That's just basically, the student starts in either ABE or ESL and ends up in either HSE or high school diploma. You don't need to mark anything. That's automatic. There are some bubbles for showing transition to CTE or college. You can see we've got those three to show that they moved to CTE and those two below to show that they moved to college. Next slide. (DESCRIPTION) A chart showing a transition (SPEECH) And this is the graphic we've had for quite a while to try to map out exactly how those transitions take place-- that is, the starting point is either in K12, adult ed, or non-credit community college, specifically ABE, ASE, or ESL. And then, in order for it to be a transition to CTE they're moving either to K12, adult ed, CTE, or community college CTE. If it's a transition to college they're transitioning to for-credit community college. Next slide. (DESCRIPTION) Title, CAEP Short Term Services (SPEECH) Transitioning to services. So again, we're continuing to record services. We have supportive training and transition services. Those are just the three categories the feds have. So those are the three categories we have. You can see them in the screenshot. Next slide. (DESCRIPTION) Title, Supportive Services, A supportive service received in program form. Text, Services that better enable an individual to participate in adult education activities or related activities such as WIOA Title 1 such as transportation, child care, dependent care, housing, and personal needs. (SPEECH) Supportive services are directed mostly toward the individual. I like to say they're services that address personal needs, that help make the person whole so they can perform better with their instruction and focus more on their instructional issues rather than their personal issues. Examples would be child care, transportation, and so on. Next slide. (DESCRIPTION) Title, Training Services. A training services received in program form. Services that help individuals: select programs that relate to economic priorities in local planning region, enroll/meet minimum qualifications for longer term employment and/or employment training programs. Services administered to individuals who have been determined to: Be unlikely to obtain/retain employment, be in need of additional services in order to attain economic self-sufficiency/permanent employment, have skills sufficient to enroll in appropriate training that provides skills necessary for self-sufficiency. (SPEECH) Training services are more specific to their instructional program or what they're doing at your school, working on getting a specific occupation, meeting qualifications for a job, for workforce training, for college, and so on. Next slide. (DESCRIPTION) Title, Transition Services. A transition services received in program form. Text, Service that help individuals: Facilitate successful transition from school to postsecondary life, such as attaining employment, enrolling in college, or accessing designated pre-employment transition services. Provide opportunities to receive training and other services necessary to achieve competitive employment or postsecondary enrollment. (SPEECH) Transitions are similar to training services, but they're more specifically geared toward transitioning to employment, workforce training, or college. Next slide. (DESCRIPTION) Title, CAEP Short Term Services - COVID19 Update. (SPEECH) So we have some updates related to services, just like we did for outcomes. A lot of you have been reporting a lot of services that you've been providing a lot now that we're in COVID-19. Lots of types of services have been increasing. (DESCRIPTION) Current examples - Additional health care needs and requirements, financial services, service to assist with new software applications, skill upgrades to work remotely, job seeking skills. (SPEECH) Here are some examples that we've been hearing a lot about-- health care services, obviously. Financial services. Lots of digital literacy upgrades going on as we move to distance learning. So a lot of those things that we're all doing naturally with our students to help them recover from COVID-19 and/or to keep them on track with instruction in a virtual environment-- those are a lot of short-term services. So what we're saying is, there may not be a match in every situation, but it's definitely true that a lot of those-- we'll just say that crash course in distance learning that we're all taking includes a lot of additional short-term services. So we're just saying there's a lot of things you can mark in the services section that really do apply with what a lot of you are doing. Next slide. (DESCRIPTION) Title, CAEP Short Term Services - COVID19 Update. Current examples - prerequisite training, assessment/testing, job readiness training, emergency financial services, health care services. (SPEECH) So the first one just gave practical examples in real world. Now we're basically giving you, really, the same examples. But we're just giving it in services speak. These are the specific checkboxes you might use that equate to some of those areas we were talking about that have happened a lot here, in the area of COVID-19. Next slide. (DESCRIPTION) Title, CAEP Short Term Services. Supportive Services - transportation, child care, personal counseling, and financial assistance. Transition Services - assessment, other than required pre/post, academic/career counseling, and job development. Training Services - Student Orientation, community support training, and prerequisite training. (SPEECH) Here's just a little chart for services, just like we have for outcomes-- the three types of services and some examples of common services that fall in each of the three categories. Next slide. (DESCRIPTION) Title, Barriers to Employment (SPEECH) Moving on to barriers to employment, that was one of the big changes from last year. Not really that big of a deal, because everybody was doing a really good job of this anyway. But just to reiterate, everybody should be recording barriers for all students. We've been aligning these barriers to federal WIOA II barriers. So part of that means that ABE, ASE, and ESL will automatically tie into specific barriers. So you can see cultural barriers apply to both. English liter-- English language learner applies to 100% of our ESL population. Low levels of literacy applies to 100% of our ABE/ASE population. And then old news, but that bottom bullet was another one of those changes that we talked about last year, where prior to last year we had barriers tied to-- or, we had specific categories for workforce prep and reentry tied to certain barriers categories. So we're no longer doing that. Before we tied certain barriers and basically conditioned it on the student being 55 years or older for a few years. Lots of you said, well, hey, it should apply to everybody, not just that specific population. So we're not tying barriers to workforce preparation or workforce reentry anymore. If you have students with specific barriers and/or you have students that are 55 years or older, by all means, you can still use programs that way if that's how you choose to do it. It's just no longer a requirement. Next slide. (DESCRIPTION) Title, Immigrant Integration A B 20 98. A map of the United States filled with people of all different ethnicities. (SPEECH) A little bit on immigrant integration. Next slide. (DESCRIPTION) Title, A B 20 98 - Immigrant Integration. (SPEECH) This is more repeat from last year. This is just word-for-word what we said here in the first one. Again, it's optional. It's not required. But we had the legislation on immigrant integration. We talked a lot about it last year. Much less about it now that we've been in COVID-19 quarantine. But it is still out there. It still will be part of CAEP reporting this year. Making a really, really long story short, we will be adding reports in TE. That's why we moved it down in this section. So we will have a new category on the CAEP summary. It's not in the CAEP summary yet. It will be available in the next couple weeks. But there will be a separate column now in the CAEP summary where you can report immigrant integration indicators outcomes, or we're calling them I3 outcomes and I3 reports for short. But that is, when we talked about immigrant integration last year, we talked about how we had done a lot of work at CASAS. There were a couple consortia that also did the same legwork. And so we matched together and compared notes and merged our efforts together so it was a strong report. But bottom line-- the long story short is, we strongly agreed that we had a lot of these immigrant integration indicators already built into our California EL civics system. So we made a series of crosswalks that basically connected those immigrant integration indicators to EL civics outcomes we've been recording for years and years and years. So as part of that process we will now have some additional reports at TE. Again, we're calling them the I3 reports that relate to EL civics COAAPs to these goal areas. So there's eight goal areas, I believe-- things like economics and civic participation and so on. I'm sure I'm not giving the right names. But it's all those obvious areas that we've been talking about the last few years. Very similar to the areas that we've been addressing in EL civics. So it basically looks at those EL civics outcomes, relates them to the I3 goal area. As a framework for reference, I'll say they're similar to the reports we already have for things like CASAS competencies and CASAS content standards-- that is, when a student passes one of those EL civics COAAPs, there's some very specific learning activities that suggests the student has now mastered as a result of passing that COAAP. So it's relating that passed COAAP to those specific learning activities, and in turn relating it to those specific immigrant integration areas so at the end of the day, by agency, by class, by student, just like you do with test results reports, you can try to match and see what specific areas your students are improving in based on their progress with EL civics COAAPs. (DESCRIPTION) Text, Key points about A B 20 98 (SPEECH) Next slide, please. So these are just the selling points on why you might want to consider it. Again, we're serving these students already. We're already doing EL civics. So our feeling is, it's not that much extra work given that a lot of you are doing EL civics anyway. It's just reporting a lot of the things you're already showing for WIOA II learners and just getting credit for it in CAEP, now, in addition to the credit a lot of you are already getting through this reporting at WIOA Title II. (DESCRIPTION) Text, This is not a mandate. (SPEECH) Next slide. (DESCRIPTION) CAEP Summary. Displays outcomes in three separate sections: 1, Literacy gains using NRS Table 4 guidelines. 2, Other A B 104 outcomes using WIOA II reporting requirements but not pre/post. 3, Services received that do not impose WIOA II reporting requirements. (SPEECH) Here's the CAEP summary. Talk about an oldie but goodie. We've talked about this, obviously. Again, there will be one big change to this in that we'll add those I3 reports. Another one I'll bring up here, while we're on the slide, is I think we're still a little bit unsure how we want to show it on the report. It's not that complicated. But you can just see there's a big time real estate issue with this report. But I think it was-- there was a little bit in what Carolyn presented and a little bit more in what Randy presented-- that is, that distinction between zero hours of instruction or being a services-only student versus being a one to 11 hours and having some activity in CAEP programs but not necessarily achieving that 12 hours. So we will have a way to make that distinction between 12 or more hours versus one to 11 hours versus zero hours. Again, we're still debating a little bit how we want to show it, because there's no more room on the report. But just an FYI-- those will be the two big updates to the CAEP summary for 2020-- for 2020-2021. And they will both be available here, I'll just say, by the end of August. Next slide, please. (DESCRIPTION) Text, A E P Data Integrity displays 27 different data elements related to the A E P instructional programs and outcomes. (SPEECH) We also have the CAEP data integrity. I think most of you are familiar with this-- again, the 27 data elements. We've been using this as the report everybody includes with their quarterly and end-of-year data submission. It includes a lot of information important for our reporting. It includes information about demographics, hours of instruction, pre and post-testing, barriers to employment, hours between tests, and so on-- that is, all kinds of different issues that are relevant to the different reporting areas of CAEP. (DESCRIPTION) Item Description lists 27 data elements that may prevent or contribute to official A E P outcomes. The D I R displays the item count and percentage for each listed item. Item percent equals item count divided by the number of students enrolled in 7 A E P programs. (SPEECH) Next slide, please. This is what we've called the bubble boy slide, another Pete and repeat. This is mostly for reference. But this is just showing all of those CAEP outcomes and exactly which bubble or checkbox it is you need to select in order to show that outcome correctly for students when they achieve the outcomes. So we've got it color coded into those six different areas. So just to use a simple example, we're coding all the employment outcomes blue. So that very first one got a job. You can see it's bubbled in blue. That equates to employment, as you can see from the legend below, simply saying, obviously, that whenever you mark "get a job" that equates to an outcome in the area of employment for CAEP reporting and so on. So this shows you exactly what it is you need to mark to make sure your outcomes correspond to the appropriate area. Next slide. Here is the same information, just in a different way. Here's a screenshot straight from TE. It's letter coded instead of color coded. (DESCRIPTION) L for literacy gains, H for HSE/HSD, P for post-secondary, E for enter employment, I for increase wages, and T for transition post-sec. (SPEECH) Same six areas, just showing what it is you need to check instead of showing you, on the previous slide, what it is you need to bubble. That's a record-- student record is where you get all this detailed information, as you see on the slide. Next slide, please. (DESCRIPTION) Title, Update to Resources and Materials (SPEECH) Updates. So beginning of year letter coming soon. Should be out very shortly. But it will be the introductory letter covering a lot of what I'm talking about right now as well as a lot of what Randy and Carolyn and others have been talking about. Next slide. (DESCRIPTION) Title, 2020 - 21 CAEP Date Dictionary. (SPEECH) This one seems bigger-- this one's bigger than it seems, or bigger to me, anyway. Probably bigger to me than anybody else. But there is a new CAEP data dictionary. I'll make this one as clear as possible. I'll just say, many suggestions that a lot of you have made for the past few years but have fallen on deaf ears-- and I've been given all kinds of excuses for why it never seems to quite make it. I'll just say, long story short, is quarantine has let it settle in a little bit better or whatever. But a lot of changes people have suggested are finally here. And the CAEP dictionary is on that CASAS California accountability page, right there with the WIOA II dictionary. But I'll just state in particular, what everybody's wanted forever and ever-- and I've just conveniently ignored it forever and ever and ever for whatever reason. It was not that hard, but I have ignored it. I'm not going to lie-- is relating all of that bubble boy information and checkbox information to the data dictionary. So if you're not that big on the bubble boy slides and you wish it was in a more formal data dictionary, it now shows all of those outcomes and indicates exactly which ones relate to CAEP outcomes and which ones are just there for your reading pleasure only. So that's what I think a lot of people have wanted. There's been a lot of revisions to, maybe, down ballot things that I think will also be helpful. On the WIOA side I really think it will be-- with that, there's also a summary of changes. So that's more on the WIOA side. But of course, if it applies there, by definition it applies on the CAEP side as well. So if you look at that summary of changes-- again, another thing. Not rocket science. Probably triple-digit numbers of suggestions over the years where I've just conveniently ignored it. But that's another one a lot if you have wanted. And that's another one where, now that we're sweat-- "sweatling" into quarantine mode, it's finally gotten done. So you may want to take a look at that summary of changes that just itemize out which one of those-- which items on the table are actual changes between '19-'20 and 2021. Next slide, please. Sorry, way too big of a paid program announcement with that one. But that's one a lot of you have been wondering about for a very long time. (DESCRIPTION) Title, Data Submission Calendar. First Quarter, July 1 through September 30, the reporting deadline is October 31, 2020. Second quarter July 1 through December 31, the reporting deadline is January 31, 2021. Third quarter, July 1 through March 31, the reporting deadline is April 30, 2021. Fourth Quarter, E OY, July 1 through June 20, the reporting deadline is August 1, 2021. (SPEECH) Data submission-- same due dates. The reminder is always is, it all starts on July 1. Again, I think everybody has it now. But one of those you never realize how confusing it is until you get a million questions about it is, again, first quarter is July 1 to September 30. Nobody's ever confused about that. But again, each quarter gets a little bit larger-- that is, the start date is always July 1. And it's the end date that changes-- that is, each quarter gets a little bit larger. Next slide. Next slide. Next slide. (DESCRIPTION) Title, T E Quarterly Data Submission Wizard (SPEECH) So here's a little bit of a change in how we're doing it. A lot of you have heard this, probably, on the WIOA side. Long story short is, WIOA and CAEP will be the same, where you can do your quarterly data submission. And there's a new wizard that allows you to-- a new wizard that allows you to submit your data and the DIR together. There's been a little niggling issue for CAEP and WIOA where your data's been automatic for quite a while, but you still need to send that little PDF to submit your data integrity report. It's been noted by many that that's a little annoying. So we've had this quarterly data submission wizard in progress for a while. It's been ready for a little bit. But we decided not to use it for end of the year. We decided it would be better to do clean break. And we will start using it now for the first quarter submission of 2021. So that is, we'll use it for the first time for the data that's due October 31. But there will be a wizard in the TE tools menu. Again, it's a little bit like that core performance wizard. Not the same issue at all, but just the same type of wizard, where you're going screen to screen to screen and then clicking Finish. So again, you use that quarterly data submission to go through the process. And once you click that final click, that will send both your data and your quarterly DIR to CASAS so some of the mechanical aspects of each quarterly data submission should be a little bit easier. Next slide. (DESCRIPTION) Title, CASAS Assessments Authorized for NRS for 2020 - 21. (SPEECH) Here's a little bit on assessments. Long story short is it's exactly the same as last year. If we were in normal times there most likely would have been some big changes with authorized tests for this year. But because we had COVID-19 and all that, obviously a lot of those federal deadlines have been delayed. Way TMI, but there was a big federal deadline of February 2021 that we were a little bit worried about. Now that we've hit COVID-19 the feds have basically postponed that due date until further notice. So we're not really sure what the timeline will be. But we are sure that the list of authorized tests for 2021 will be exactly the same as the list of authorized tests for '19-'20, meaning for ABE/ASE we'll use reading goals and math goals. For ESL we'll use life and work listening, life and work reading. (DESCRIPTION) Text, Power, A A dash A A A A A not authorized for NRS, but are authorized for C A reporting. (SPEECH) Next slide. (DESCRIPTION) Title, Use of Assessment Modalities. (SPEECH) We'll continue to use reading or math goals for ABE/ASE, reading or listening for ESL. That, too, was unchanged. And again, you've got to have it be the same modality. You can't have a goals pre and a life and work post. (DESCRIPTION) Text, If using CASAS GOALS, both the pretest and post-test must use the same test series. Cannot match GOALS with any other CASAS test series. (SPEECH) You've got to have the modality and the series be the same for both pre and post. Next. (DESCRIPTION) Title, Remote Testing, (SPEECH) Remote testing. Carolyn talked a little bit about that. There were some big webinars that she did with CASAS a couple months ago. For-- OK. Well, let me finish this one up. So we-- here's the link to our remote testing page. (DESCRIPTION) https colon slash slash WWW dot casas dot org slash training dash and dash support slash cases dash peer dash communities slash california dash adult dash education dash accountability dash and dash assessment slash california dash remote dash testing. (SPEECH) Lots and lots and lots of resources there. Some videos. Some guidelines. Some PowerPoints that should be helpful. So all sorts of information on remote testing. We're working on bigger ratios moving forward. We're trying to get it to where you can test more at one time. Lots of you have been testing and working on more options. But for the latest and greatest, there's the link to our CASAS remote testing page. Next. (DESCRIPTION) Title, CASAS Reading Level Indicator, RLI (SPEECH) Part of this is we have a new reading level indicator. I'm not sure if this was covered before or not, but this is what we've been working on a lot here in the last month. It should be available here in mid-August. That is, you-- what-- I definitely did hear from Carolyn at the beginning, is she mentioned OCTAE memo 20-5, they basically say it's OK to use informal means for placement. But it still requires a pre-test and a post-test to record a measurable skill gain. So it's basically staying on both sides of the fence, you might say, for now. But it's allowing the informal placement. So you're not required to use this. But a lot of you have said that, gee, we'd really like to be able to do this, but we don't necessarily have a good instrument at our agency that will do this. So we're developing the reading level indicator. It's no cost if you're already on eTest and TE. It will give you an estimated placement level when you complete it. It's not an approved test, but it will-- so it will not allow you to make a gain. But it will allow you to place the student into the appropriate level. The beauty of this is, number one, it will be something that the student can complete on her or his cell phone. And number two, you don't need to have a proctor present when you do this. (DESCRIPTION) Text, The Reading Level Indicator is Form 601 R. (SPEECH) Can you go next slide? I think there's one more slide on this. (DESCRIPTION) Title, About the CASAS RLI. Initate the RLI by selecting students in TE's Student Demographics lister. Click the Send Test Invitations button. (SPEECH) That was-- took a while. But there it is. So this gets a little bit more into the weeds, where the way it works is, again, the student would be doing it on his or her cell phone. But again, you don't need to administer it through eTest. So you don't need to have a proctor present. So if the issue is with staffing or the length of time it takes to go through all those security steps or whatever, the good news is you don't have to worry about either one of those with the RLI because the student can access it on their cell phone and do it independently. Again, because of that, it definitely will not give you a skill score. But it will at least give you that level placement. So if you're concerned, here at the beginning, about just getting the student in the right level or in the right class, it should work very well for this. And then, again in the weeds, the way you work it mechanically is you send out invitations. You can send out an invitation by text, by email, or both. That is, it's using a very similar functionality to what we've been using for that employment and earnings survey where with our student portal you can send out those invitations. So it's using that same mechanism to reach students either by email or by cell phone. Next. (DESCRIPTION) Title, I 3 Reports in T E. T E now includes Immigrant Indicator, I 3, reports that track EL Civics COAAP's outcomes and relate them to Immigrant Integration goals. (SPEECH) Here are those I3 reports. This is just a screenshot. This is an example, a higher-level example. Again, it shows you those I3 goal areas, shows you how many have been attempted in past, and gives you a percentage. That is, it's not the same thing, but it's modeled very similar to what we've done with things like content standards and content-- or, content standard and competency reports already, where it's giving you percentages based on instructional area and giving you information on all those different areas based on class, agency, student level, et cetera. Next slide, please. (DESCRIPTION) CASAS website - whats new, online registration, California accountability, AEBG web page, CASAS forums, download centers, WWW dot CASAS dot org. (SPEECH) CASAS website-- here's all the wonderful things on our website. Next slide. (DESCRIPTION) Title, Online Training Registration. Register for all face to face and web-based trainings on the CAEP TAP website, https colon slash slash WWW dot C A adult ed training dot org. (SPEECH) And I think this is where TAP takes over. Or did you want me to just do these? Sorry. I'm having the most difficult time finding the Unmute button. But thank you, Jay, very much. Yes. So this is the California Professional Development training page. And on this page you will find training opportunities for not only CAEP, but also CASAS, OTAN, as well as CALPRO. So if you're in need of any professional development opportunities or if you like to know what's going on or what's happening next, definitely feel free to log onto this page to explore professional development opportunities. And again, if you have any-- if you need support, contact CAEP TAP and we will be sure to help you. And I was attempting to post our other URL, where all of our trainings are also located. And then, if you go on to CalAdultEd.org you'll see several announcements that we have. Right now, one thing that we are promoting-- and hopefully you all will be able to join us-- starting next week, on Monday, August 10, will be our second iteration of the regional network meetings. And so that's an opportunity for us to come together. And the roles will be reversed there. So it will be less of us presenting, but more of a conversation about what things are going on and that are important to you. So one thing that we did this time around is we sent out a Google form where we wanted to identify all of the topics that are important to you in this moment or in this space and allow you to prioritize your topics of importance as well as subtopics that are important to you. So we want to make sure that we have resources available to you as well as engage in a conversation in figuring things out and helping you all support one another for things that are going on within your respective agencies, within your region, et cetera. So definitely be sure to register for the upcoming regional network meetings. They will take place beginning August 10. And they'll go August 10 and 12, and then 17, 19, and 24, and 26. So you have six opportunities to be able to engage with CAEP across the region. Of course, you would attend your regional network meeting. But you are more than welcome to attend other regional network meetings, as what's going on in one region may not be going on in the other. So please take that opportunity. The next thing is our CAEP Summit 2020 will take place on October 26 through 29 of this year. Registration is not quite open yet. The pivot from the in-person event to the virtual conference is taking us a little bit longer. But registration will be open soon. However, we still are collecting proposals. So if you have a strategy or a practice that you would like to share with others throughout the state, especially as it relates to the impact of COVID-19 and other things that are going on in our society at this time, definitely feel free to submit a proposal. We want to be able to learn from one another and share and collaborate with one another. So I will post the URL for that, as well. I don't know if you had any additional questions in the chat or if anyone had any questions at all. This will be the time for us to ask questions. We have about another 20 minutes before the end of this webinar, so plenty of time to ask questions. Feel free to unmute yourself if you have a question or type it in the chat. And either Randy or Jay-- I see that OTAN is still present. So if you have any questions for any of those agencies, definitely feel free to ask them. And if you have any questions for CAEP leadership, I see that Javier is still on the line, as well. Any questions at this time? I will say, we've been having very vigorous discussions in the chat-- [LAUGHTER] --with colleges during Jay's presentation. Great. Veronica? Yes? This is Wendy Miller in San Francisco. Hi, Wendy. Hi, everybody. This morning at the Chancellor's Office biweekly COVID-19 presentation they were talking about the need for all of the community colleges to develop administrative procedures and board policies for credit for prior learning. And I wanted-- by December. And so I wanted to just throw it out there to the group-- I put it twice in the chat box but didn't get a response-- if anyone in the field has taken steps for credit for prior learning. And how is that something we could report up through any sort of data collection instrument? And I will mute myself to hear an answer. (DESCRIPTION) Slide Title, Wrap Up and Questions (SPEECH) Thank you for that question, Wendy. I'm not seeing any answers or anyone unmuting themselves. Yeah, Wendy. It seems like there's not-- I think everybody's probably in the same place as you, trying to figure that one out. But that's something we'll keep in-- start thinking about, how we could identify anybody out there that's leading the way that can inform us all. So that's something we'll take back with us to track. Wendy, the complicatedness of this-- and I'm thinking, we had these discussions at Contra Costa when I was still there-- is that the-- this becomes an issue very quickly between the faculty and-- at every individual college. So basically, it's [INTERPOSING VOICES]. Correct. Yeah. It's 10 plus 1 academic senate. Yeah. It's a shared governance issue, which basically means-- and asking the colleges to please do this as quickly as possible does not actually alleviate, then, the fact that there's no way to apply a solution that can't-- across the system. And I'm almost wondering if there needs to be-- that almost feels to me that there should be some legislative support for a shared solution for this if it were to happen. I'm just thinking about the way I'd think of it as a dean at my old district, because you don't really have strong leverage, necessarily, depending on where your faculty are with this conversation. Correct? Correct. I'm just acknowledging the complicatedness of the problem. I'm not helping you in any tangible way. Let's just be clear. I think I'm just suggesting that it be a topic that we, as a group that can think creatively together, should spend some time with, because I think it's always good, if you have to go through a shared governance process, that you present something that they can respond to as opposed to asking them to develop it. Well, and if you look at the way the language around implement is shifting, there's much more emphasis on trying to help people looking to improve their employment situation to understand the skills they have that are translatable. And one of the ways to help honor or recognize that is actually providing some form of credit for that within the educational system. Correct? So I think the question is, how do we form a stance that captures that shift in conversation towards skills and competencies in ways that are persuasive to faculty-- and that probably, in some ways, can benefit the institution at a higher level. But you'd have-- we have to quantify and think about how to parse that out. Great. Thank you. Wendy, this is Trina. One out of the box thinking, potentially, is to looking at a secondary model, meaning from high school levels. There's partial credits for homeless and foster students that's mandated. I don't know if that's something that would be applicable to higher ed. Hm. It would be interesting to hear more about that sometime. I mean, this could be a working discussion that we could talk about having with the broader community. And there were a couple more questions in the chat. So Ilsa-- she wants to know, "great news about the addition of a 13-column to CAEP summary report. Which data, in addition to EL civics COAAPs, will go into this column? Short-term services, for example?" I was just typing that one. I'll just say, for now it's just COAAPs. Services might potentially work. I guess the short answer is, we do have a lot of information that relates COAAPs to those indicators. But we don't really have the same information related to services. So it would just be some information we would need to collect and figure out, I think. Thank you, Jay. And Karen wants to know-- a while back there was an indication that there would be some change to attendance reporting this year. Any updates? I don't know what that-- I'm not sure what that means. Is this a college issue? Is this-- is this for the [INTERPOSING VOICES] Oh, maybe it's-- sorry. Yeah. Sorry. There are changes in how colleges are coding student attendance. So college non-credit programs can now use census rather than instructional contact hours, which is going to create challenges related to how we identify whether someone's a participant, because once you choose-- basically, census, as the funding methodology for the course-- you can no longer report instructional contact hours into SX05. That's getting too technical. But so we have that in front of the Chancellor's Office. The Chancellor's Office is supposed to be convening policy groups. And I have that on the policy list for them to discuss the best way to think about how to handle that issue in terms of an approximation of what's-- if someone shows up in census, do we just presume they're a participant? But then we would never see anyone who has 1 to 11 instructional contact hours, probably. So there's some things we're trying to hammer out there for the next build. Thank you, Randy. And Lorilee asks, "can student hours only be recorded based on instruction, not on work completed by students?" I guess the question is, what do they mean by work? Do they mean work that students do outside of the classroom? Or do they mean work work, like work-based learning? Lorilee, if you are able to unmute yourself, could you ask-- answer that clarifying question, please? Yeah. I'm talking about project-based learning. If we were to do activities that-- the way that I interpret what it-- what you said was that we can't do assessments. And Will provided a little bit more clarification. I was just wondering on the hours of instruction when we count the hours, if there are activities that the teacher finds-- if they are project-based learning activities, do those-- can we count those as student support instruction hours or contact hours? I'm having a little bit difficult-- problem hearing you. So maybe if there's something you can write out in the chat, and we could either address it here or address it offline. And the next question comes from Elena. "Would a placement assessment or testing out work for assigning credit for prior learning?" The forms of assessment that you could use for that are going to be based on what the faculty agree to at your college, if I'm-- at least in the college environment. So it would depend on the nature of the assessment and what the discussions were about what you would use for that. Thank you. Kevin, I see a note here saying that you echo most of Monica's question. However, I didn't see that. So if either you can please post it again, that would be great. And then Trina wants to know, "is there a possibility to look at secondary model for non- credits for homeless and foster students?" I just indicated that I think if we have a broader discussion about this, these are all things we would probably want to look at in that discussion. But if anyone else has a take on that, it'd be great to hear. And-- Sorry. Hey, Veronica, I'll just belatedly say, the one related to what Kevin was asking and what Monica was asking was addressed by Margaret in the Q&A. It was a CASAS RLI question. Great. Thank you. And I am not seeing any additional questions. Hello. Well, I have, actually, one question. It might not be the space. But regarding the CASAS score, when we stop receiving funding for a student, is it 245 or 247? I'm sorry. What do you-- what is-- I'm sorry, what-- when you stop getting funding-- I'm not quite sure what you mean by that question. So you know how there's a maximum-- when a student takes the CASAS test, there's a maximum amount of points that they can take, right, in order for-- and then it says exit the program, or something like that, on the top for eTesting. Oh, I-- so you're talking about if they score really high, where they're too high for ABE. Yes. So, sorry, there's no magic number anymore. The short answer is, it depends on which test series and test modality you're giving. That used to be 236 for ESL and 246 for ASE. But now that we have different series for different programs, it differs a little bit from series to series where that cutoff is. And would we-- If you want, you can email me. And it will be a longer presentation, but there's a series of charts, now, you can use. Because again, the different tests have different levels of difficulty. So there's no one size fits all number for all forms anymore. OK. Got it. Thank you. Yeah. Great. And one more question came in related to CASAS, from Elsa. She says, "I take it the CASAS RLI I is not paper based." Right. It's something you do either on the computer or on the cell phone. Thank you, Jay. Other discussions going on, but no more additional questions. So if anyone has any additional questions, now is your time. We have about seven minutes before the webinar is technically over. So we do have a little bit more time if others have questions. You can unmute yourself or you can type in the chat. I'm not seeing any questions come in, just thank yous. You all are welcome. Well, I think it may be safe to close the webinar. So again, thank you all very much for your time and for hanging out with us for almost two hours this afternoon as we talked about data and accountability. And I do see a question from Ryan regarding the video posting. So Ryan, please give us until tomorrow morning to post the video. We have to go through a new process now, so it takes us a little longer. But we'll definitely have it posted by tomorrow morning, along with the PowerPoint presentation. And since the chat was a little rich today, with a lot of questions and answers, we'll-- I think we'll go ahead and post that, as well, so that you all have access to that information. Definitely join us on Mon-- oh, tomorrow, actually, for the CTE pathway mapping webinar, tomorrow at 11 o'clock AM. Register and attend that. And then join us again next week on Monday for the regional network meetings. And again, if you have not, let us know the topics that are important to you to discuss. Definitely feel free to go ahead and let us know about that. Please complete the evaluation. That is our way of assessing what your needs are at this time. So if there's something that wasn't covered here or something that you would like us to continue the conversation on, we can definitely look at that data and be able to respond with future webinars and/or training opportunities. So thank you all very much for your time and your participation this afternoon. And we look forward to seeing you tomorrow or Monday. Have a great afternoon.