[audio logo]

Speaker 1: OTAN, Outreach and Technical Assistance Network.

Carolyn Zachry: Well, thanks, everyone, for being here. We appreciate you taking the time on your Friday to ask questions about the upcoming Continuous Improvement Plan and everything that that entails.

And so this is really an opportunity for you to ask questions. And you can put them in the chat, you can come off mute if you need to. We have with us today several people that can help you, that are here to really answer those questions.

We have Carol Hirota, Marian Thacher, Penny Pearson, and we also have Jong from OTAN, who can answer any of your questions related to putting data or putting information into the system. And so I'm going to kick this off with just a reminder that the Continuous Improvement Plan is something that should be written not in a vacuum, right?

Not by one person, but you should be working with a team similar to what you do with WASC. And really starting with your data, looking at your data, looking at last year's CIP, right? How have you been successful with your goals from the last year? And then again, moving into this next year, where do you want to go with that, right? How is it going?

So that's really that. And then, of course, you're writing them to be SMART goals because that will really help you in the long run to achieve your goals if you're very specific, you make sure they're measurable, they're achievable, and they're time bound, right? That you can do all of those things with those goals.

So I know that is one of the areas that we really look at when we're evaluating your goals to make sure that they are SMART and have all those components, because that really will help you as an agency to move forward with those goals and to hopefully be successful in reaching them or almost achieving them.

So I'm going to turn this over to, I guess, Marian perhaps and Penny to chat a little bit more about maybe the reviewer process and what you do and what an agency might see in the notes pieces in the OAR system. And I'm actually going to jump off the Zoom, just so you all know for a moment, and then I'll come back in. So all right, there we go. And, Marian, you're muted.

Marian Thacher: Thank you. And thank you. So we're here today really to just answer your questions. The CIP applic-- submission form is open. You can start working on it now. You can actually submit it now. And I think we have, Jong, like one submission so far. Somebody already submitted their plan, which is great.

And if you do-- it's due April 30, but if you do submit it early, it will be reviewed in a timely way. And you will be able to get feedback if you need it or to just get it done if that works for you. So we do encourage you to go ahead with the planning process, and go ahead and submit whenever you're ready, because we're watching, and we're getting notified whenever the plans are submitted.

Your plan will be reviewed by a couple of people. And you will see feedback. Last year, you only-- and previously, you only saw feedback if your plan was not approved. But now you'll see feedback no matter what. So even if your plan is approved, you'll have notes.

You can see those right away. And you'll also see them, not next year, because there won't be a CIP next year. But the following year when you go back to do your next one, you'll see the notes from the previous one, which should be helpful, hopefully.

So really, we want to just know what your questions are and try to help you with those if we can. So you can come off mic if you want, or you can use the chat. It's not so many of us, so go ahead and talk if you want.

Philip Villamor: I'll go ahead and talk. I think everything's-- I mean, it hasn't changed much, but I just want to make sure, because I did read the notes on what was presented before. And the examples are still examples that go from like June to June.

And I just wanted to make sure because we haven't been doing that because it doesn't make sense. If we're reviewing how we did for the year before and it's due in April, we've been just writing it up in there like more specifically to March to March.

Making sure that that's OK and that it makes sense to you as well. Because if we're reviewing data and saying whether or not we made it or not, we don't really know for this year what happened in June yet. Does that make sense?

Marian Thacher: Yeah. So you're talking about the goals that you have now whether they're completed or not?

Phillip Villamor: That's right.

Marian Thacher: Yeah. So yes, that does make sense. And you don't know until June. And you're writing this plan that your work you're working on now is for '26-'27. So it ends in June '27.

Phillip Villamor: Right. But we'll review it before then.

Marian Thacher: Right. Right. You'll be checking in on it periodically. And if you're carrying over, so maybe you're asking if you carry over a goal, you can't really say if it's done. But you can just say where you are with it now, and you're carrying it over because whatever the reason is.

And please change the dates because that's one of the biggest things that we fail people for, is like, they're still back in 2024. And I lost my thought. Change the dates and update, update what it is you're doing. Because probably some thing's happened this year that about already. So just to make it.

Phillip Villamor: Yeah, [ INAUDIBLE ]. I guess we're just-- everything you're saying makes sense. I just want to make sure it makes sense when I write it that way. I'm never writing a goal that goes to June. If it has to do with looking at the gains and stuff, I'm writing that and saying I'm going to look at it again in March.

And I put it in March even though we can start now just because we make it a little bit more through the year and say whether or not we actually did improve on those gains and stuff.

Marian Thacher: Mm-hmm.

Phillip Villamor: OK.

Marian Thacher: Well, just so you know, you're not really being scored on whether you succeeded or not. I mean, that's your internal process. And however you want to determine that. So we're just looking for what you're planning to do. And does it make sense, and is it achievable and that kind of stuff.

We're not going, oh my gosh, you only-- you were going to improve 50%, and you only made it to 25. There's nothing like that. Nobody even-- that's for you, not for us.

Phillip Villamor: I get it. I do. But I would be thinking you would be sending ones back that say they're going to make a goal, and they're going to base it on the gains that end in June when they can't really look at that. So I would be thinking that's specifically what plans would be sent back for and not other things.

Marian Thacher: Well, you will be able to look at that in June, and the plan is for next year. So I think that would be OK.

Phillip Villamor: OK.

Marian Thacher: If that makes sense. Are we talking the same language, Philip? Yeah.

Alisa Takeuchi: Hi. This is Alisa.

Marian Thacher: Alisa, hi.

Alisa Takeuchi: Hi. I just wanted to make a small comment. So, like, Phil, for your CIP plan, instead of saying by June of 2027 or whatever, just say by March. I mean, just make your--

Philip Villamor: That is exactly what I'm saying we do. And we will continue to do. I just want to make sure that folks are looking at that and understanding that, and that really that's what it should be.

Alisa Takeuchi: Yeah. Because like Marian said, the plan is for you guys. I mean, that's not-- [interposing voices]

Philip Villamor: Right. If we're going to look at it again this time, that's the way we should.

Marian Thacher: OK. Carol, are you raising your hand?

Carol Hirota: Yeah. My comment about, definitely use March. But in program year, our state uses program year from July 1 to end of June. And so when we write our goals and we have our faculty do their data sets, we always know that we will get all that information in the summer.

And so when we start up school again in August, they set up their new data system that aligns with the goals. But it's never impacted us that we've done July 1 to June 30, because you don't consolidate the end of the data until the end of the year along with all the fiscal stuff.

So it's worked. And they have to review their stuff monthly anyway, so we would have a present level of status in every month of the program year, going from July to June.

Philip Villamor: What are you saying you have a present level of status? What does that mean?

Carol Hirota: Every month, teachers know where they are in relationship to the goals. If it's increasing, let's say student learning gains, and they have their data system set up, I could just check in with their data system because most of them have it on a Google Doc in the Google Drive. And so every month you can check and see where they are.

Are they matching and aligning with the learning gains that would contribute to our goal for the school or the agency?

Philip Villamor: Yeah, absolutely. I mean, that's what--

Carol Hirota: So it's every month. They review it every month. It's not at the end of the year. So it's continuous. It's a 12-year process. I mean, a 12-month process.

Philip Villamor: Right. But if, again, we're being charged with looking at we made these last year, let's look at it again, and let's compare it to the year before, it would make sense to compare it to the same month in the year before and not the end of the year.

Marian Thacher: OK, thank you. What else? Who else has a question?

Lisa Gonzalves: Question.

Marian Thacher: Yes. Hi, Lisa.

Lisa Gonzalves: Hi. So I'm looking at my CIP right now on the Cali Adult Ed reporting. And for some of our goals, we have, like, eight supporting strategies. And by April 30, I don't think I'm going to have completed all eight. But maybe pretty close. But still just the relationship with the goal really has to do with the June 30 deadline.

So my understanding is if I click no, it's going to get rolled over to the next CIP. But if I also plan on finishing that goal before the end of the school year, and I don't want it to roll over because I know I'll be done-- and again, I know this goes back--

Marian Thacher: Yeah.

Lisa Gonzalves: So I'm just wondering how--

Penny Pearson: Just don't click no.

Lisa Gonzalves: Yes. Even though I have not yet completed but plan to?

Penny Pearson: You're on track for it though, correct?

Lisa Gonzalves: Yeah. I mean, I guess there's always a possibility that we won't meet the mark, but.

Marian Thacher: Yeah. The way the system works is if you want to see that goal again for next year and update it for next year, you have to say no. Even if you think you're going to hit it this year but you want to add-- make it a little higher goal next year, whatever, you say no, and you adapt it for next year.

Lisa Gonzalves: Oh, so when I get to the next page, I'll still be able to edit, tweak it?

Marian Thacher: Yes.

Lisa Gonzalves: Great.

Marian Thacher: Yes. In fact, you have to tweak it. Because it's going to say '25-'26 or whatever. And things probably something-- at least something has changed. So yeah, please update them. But if you want to carry it over, say no. And if you don't want to carry it over, say yes. That's just how it works.

Lisa Gonzalves: Thank you so much.

Marian Thacher: Good question. Tatiana.

Tatiana Roganova: Yes. And this is continuation of the conversation that we just had with Philip. Because like the number of graduates, if we want to increase the number of graduates and to a certain percentage or a certain number, again, we won't know till the end of June because our graduation is the end of June and our HSC students really start working like the last two weeks. They start passing the test. So I can't estimate it in March.

And also I just watched the video that was suggested in the email where Caroline was presenting about an example and gave an example of the goal. And that goal was to be done by June, the end of June '26-'27. And it was a data goal on increasing, I think, persistence or increasing the number of students that complete 12 hours.

Again, we won't be able to estimate that til we get the final data at the end of June. There is a certain conflict here with CIP plan and the way you structured it with the deadlines that we have. And I don't know, I don't want to put the March as a benchmark because the data is not complete, as Carol just pointed out to us.

And I don't want to carry all the goals. Because if they are all measurable, the only way we measure it is through the data that we collect. And we won't have it by the end of the school year. So there must be some kind of a way to resolve this conflict. But at this point, I don't see how we can do it.

Marian Thacher: Well, I guess I'm going to say the same thing. Which this is your plan and your process. And you can-- of course, you'll be looking at the results at the end of June or August or whenever your data is complete in terms of how that affects what you're doing next year.

So at this point, you're just planning for next year though. Nobody outside of you guys is looking at your goals for this year and saying, oh, they made it or they didn't make it. You're making that determination. And you can make that determination in June or August or whenever makes sense to you.

But you can still-- at this point, but you're still doing a plan for next year. Does that make sense, Tatiana, or no? No.

Penny Pearson: Tatiana, would you have any trending data that might help to inform how your goals are being met this year that would help you to literally speculate or forecast what you anticipate those final numbers to be in June?

But again, as Marian said, this is your goal. This is your-- you have your levels of measurement of how that goes. And you're going to be able to see like, I really think we're on target here. We're really-- this goal that we set for the '25-'26 school year is on the mark. I'm going to mark it as complete.

Because again, as Marian said, you're the only one that's going to see this final goal as being done, complete, successful, or not. Because we're interested in seeing what you're doing for next program year. That's what you're planning for now. And that technically doesn't start until July 1 of 2026 for the next academic year.

So like Marian said, we really want to stress that we're not coming back in and peeking through your data going, well, did they really get 15 more graduates? No. We're looking at the planning process, the SMART goal, that you're actually going to have something that is achievable. You have a measurement. It is specific. That's what we look for.

You're going to make that decision of whether it was successful or not as we go through the next planning process. So I mean, I would encourage you to give yourself a little more grace that you know what's happening in your organization.

Tatiana Roganova: So am I understanding it correctly, we can make our decision whether the goal was completed or not based on the estimate on that data that we have at that point by the end of April? Is that how I understand it?

Penny Pearson: Yeah.

Tatiana Roganova: OK. Thank you.

Penny Pearson: Elsa.

Elsa Magana: Hi. So the three goals that I had for this year, right? I believe I met the goal. So that's a good clarification that you said it's for us to believe whether or not we met the goal. OK. So if I put that I met the goal, do I have to explain it, or it just says met the goal and that's it? And then we start off a whole new template?

And then my question is it's always stressed that one of them has to be technology based. Which is great, but this year I put I needed to buy more Chromebooks. I mean, I can always buy more Chromebooks because that's something that the students are always needing.

Last year, it was to increase basic computer offerings. I met that one, because now I'm offering like four. Chromebooks, I bought new Chromebooks this year. So I met that goal. And then the students are asking for-- on the survey, because I did look at the survey, they're asking for classes, and they're asking for devices to lend them out. I already have that. I don't know what else I can look for that is technology based.

Marian Thacher: Well, one thing, Elsa, I'll say about that is that it doesn't have to be things. It can be teacher training, professional development. It can be increasing the way you use an online curriculum like Burlington English or something like that. It's just related to technology. It doesn't have to be buying tech. Does that help?

Elsa Magana: Yes, that does. Because I was thinking whether it was to buy technology or to offer-- do more tech-- digital literacy classes, right?

Marian Thacher: Yeah, digital literacy would count. Increasing online or HyFlex classes would be related, things like that.

Elsa Magana: OK. OK.

Marian Thacher: Tatiana, just a second. Is there any of you other answerers have an answer to that? We're good?

Carol Hirota: Isidro had his name up too.

Elsa Magana: If I need the old plan reopened or-- I don't know if it's reopened or not. But because on some of them, I kind of clicked ahead of time, and I was like, wait, I need to go back, and it just now it's not there. It's kind of weird. I'll look into it today.

Renee Collins: Jong, are you able to assist with that?

Elsa Magana: What was that?

Jong Choi: Yeah, I'd actually need a description of the problem. I'm not sure what's happening.

Renee Collins: Yeah. Maybe Elsa and Jong you can-- you can connect.

Elsa Magana: OK.

Marian Thacher: So Elsa and everybody, if you have a technical problem or a question and it's not today between 12:00 and 1:00, you can email support@otan.us and somebody will get back to you and help you. Isidro. Sorry, Tatiana.

Isidro Rodriguez: No. No worries. Good afternoon to all of you. It's nice to see all of you again. I have a question on the number of completed teacher self-assessments. The view of the teacher self-assessment data and the student technology intake.

Again, based on the information you just spoke about, the CIP and how it's self review plan, is the assessments the same way? Viewed the same way? So is it for our purposes and data collection just for-- or is that required to complete?

Because I noticed that on the previous years, I guess I'm assuming it was not completed. So I'm just starting this program. I'm just new to getting this done. So I'm just kind of learning and asking questions as well.

Renee Collins: They are-- they are required to complete. The student intake survey is something that you can be doing all year long if you wanted to. You could target a particular population within your school, or you could do all of your students within your school.

And generally speaking, the teacher survey, you need to have-- I forget exactly the percentage, but about 25%-- 25% of your students-- I'm sorry, teachers take the teacher survey. And then use the results from both to be able to design your goals for the year.

Penny Pearson: And, Isidro, you can make a choice, say, on your teacher survey, if you're looking to grow one particular program, like your ABE or ASE or high school completion program, you can have those teachers take that technology survey. And that can help to inform you maybe what you might want to do for professional development for your teachers for that program because you have a specific goal to grow in that area.

Or you can have your entire staff, ask them. And we'd really like to see at least 25% of staff taking that. And I know that can be tough. I don't know how many full-time teachers you have versus part-time. So it's a really nice way to gather some data on your teachers' technology skills.

Isidro Rodriguez: Yeah. Just looking at the survey, it's pretty general questions. I need a, I have enough of, I need, I receive, I need, I receive, I need. So it doesn't say specifically for specific course. So I'm just going to make it as a general survey.

And the other question is, can I just copy the survey and then send it out, or is it allowable editable, or do I just send what you have here to all the staff?

Penny Pearson: It's actually accessible through the CIP reporting site. You can get a URL for your agency, and then you can distribute that. And we've seen teachers do it, or agencies do it by a QR code posted in the Teacher Lounge or send out via email saying, hey, go and fill out this teacher survey. And then it's within your agency's bucket of data, so to speak. So it's your teachers taking it from your agency.

Isidro Rodriguez: And then I'm not sure why only two students did ours last year's. And so I'm just trying to figure out. I don't see any question-- well, yes I do. Never mind. I see the questions. OK. And then the same thing for the students, they just get a QR code and have them fill that out?

Penny Pearson: Yeah, well, students might be a little bit different just because you might have more challenges there. We see some agencies that have the students fill this out at an orientation session, for example. And again, that is linked to your particular agency. So you get the data on how your students filled it out.

Some of them do it when students register. So you can decide how you might want to gather that data over the next year. And honestly, you've got something to target there in terms of making sure that you get that information from your students, because it can inform you on, wow, I have a lot of students that have no internet connectivity at home.

So maybe I can provide them with either hotspots, or I can gather resources in my area that would help them find library services or something that would actually meet those goals and get your students with more digital literacy skills, too.

And you have to go into the reporting site and into the CIP area, and you'll see where the teacher and student surveys are.

Isidro Rodriguez: Yeah, I'm there now. That's where I picked this up. So I'm looking at the student survey. I see only nine questions, which is great. But it's been completed, but I don't have a clean survey with zero marked in it. So need a new one. And then secondly, can I get it-- can I get permission to edit it and add Spanish into it? Or unless you have one in Spanish already.

Penny Pearson: That's a good point. I think we had it. And, Renee, you might be able to help. Didn't CAEP have a student technology survey, and you could change the language on it?

Renee Collins: Exactly. I was just going to say the same thing. So I will find the URL, and I'll pop it into the chat.

Penny Pearson: And that one is outside of the CIP reporting site, right, Renee? Because--

Renee Collins: Yes, it is.

Penny Pearson: Yeah. That's--

Renee Collins: And you'll be able to see your old-- you can see your older results if you're a WIOA funded agency. You should be able to see those within your WIOA OAR.

Isidro Rodriguez: That makes sense. I see the results, but I don't see a new survey questions or a clean slate. That's all.

Renee Collins: OK, let me look for it.

Alisa Takeuchi: Penny, this is Elisa. Sorry I'm just going to interrupt real quick. So, Isidro, when the student goes to the link to do the student survey, on the top right-hand corner, it'll say translate. And they can click on it, and it'll bring down a whole bunch of languages. And the student can choose which language, and then it immediately translates it for them.

Isidro Rodriguez: Perfect. Thank you for that. Yeah, as soon as I get that, I'll take a look at that. Thank you.

Penny Pearson: So I think it's Elsa and then Niccole had their hand up.

Marian Thacher: And Tatiana.

Penny Pearson: Tatiana, if you-- yeah, I saw you physically put your hand up. Yeah.

Elsa Magana: Hi. OK, so I'm looking at the CIP right now. And it says even though I met-- I put that the goals were met, it tells me, you have carried over goals from last year with no changes. Please update the schools with action steps. You must enter three SMART goals. Currently you have two.

And I was like, I haven't even done any because I haven't-- I was waiting to actually meet with-- and that's what I was going to ask Renee. When I saw you at the [ INAUDIBLE ], I was like, I need help with the CIP, because I'm stuck there. I can probably do the new SMART goals, but this red thing, red, it's not good.

Renee Collins: So, Elsa, I would suggest that you reach out to-- I'm sorry. I would suggest that you reach out to the support@otan.us you have a very specific technical item to address. And then sorry, Marian.

Elsa Magana: OK. I'll email you guys. And I'll take a snippet of what it's telling me so that I can--

Marian Thacher: Yeah, Elsa, somebody can meet with you online if you need that. But it sounds like you said no to two goals, so they're carried over. And then you have to add one more goal. So you would be editing those carry-over goals. And then--

Elsa Magana: No. So when I click on Review Previous Zip Code, the preview I review, and it's like, has this goal been completed? Yes. Have this goal been completed? Yes. Have this goal been completed? Yes. Because I actually did. And I save it.

Marian Thacher: Oh, I see. And then you still have two goals in there when you go in for this year?

Elsa Magana: And it still says that I have carried over two goals from last year.

Marian Thacher: OK. OK. Yeah. Renee is right. Yeah. Email OTAN.

Elsa Magana: Just OTAN--

Marian Thacher: Support@otan.us.

Elsa Magana: OK. Thank you.

Marian Thacher: Somebody put that in the chat.

Renee Collins: Yeah, I can drop it back in there.

Marian Thacher: Thank you. It was Niccole, right? And then Tatiana.

Penny Pearson: Yes.

Niccole Scrogins: Hi.

Marian Thacher: Niccole. What's up? Hi.

Niccole Scrogins: I'm kind of digging for gold with this question. But it's just basic. And I'm wondering when you guys are reviewing CIPs statewide, what distinguishes a plan that's truly driving improvement from one that just meets expectations?

Marian Thacher: Well, that's an intense question.

Penny Pearson: Yeah. I mean, I can-- I'll take a stab at this because I am a reviewer, and I've done it for many years. And I'm sorry to say this, but it really depends. Because it depends on the agency. We really look at-- we really try as reviewers, I mean, we're looking at how are you looking at your plan?

How are you telling us that it's going to make an improvements for your student learning, for your program outputs? And it all comes together. And there's always two people that review your plans. And they have to be in agreement. And believe me, there are often very interesting conversations around some of those questions, because we see completely different plans.

If it's from a very small rural agency, it can be so tiny. But to them, it's extremely significant because it's a huge leap forward for them to do something like this. But a larger agency would go, oh, that's no big deal. We can do that in a week.

So that's a very difficult question to answer with a finite answer. I mean, just from my perspective, there isn't any one holy grail of a perfect whatever. So I don't mean to pop your bottle-- bubble.

Marian Thacher: Niccole, I will say that we look for goals that will improve student outcomes and that are student focused. So even if you're buying computers, you're buying computers in order to add this aspect to instruction or improve digital literacy skills for our students or whatever it is. So that's one aspect.

Niccole Scrogins: OK. I appreciate it. A follow-up question. And, Penny, you gave me more than you realize. So I appreciate that. I appreciate the honest answer. OK, so when we're drafting the action steps, how granular should we be? Is it more important to articulate strategic direction or to outline operational specifics?

Marian Thacher: The strategies are for you. So it's really whatever works for you. That's not being scored. Well, we will read it to see if it's-- like it's even related to the goal, but we're not saying you have the wrong strategy or you don't have enough or you have too much. So just make it be something that's useful to you, basically.

Niccole Scrogins: OK. OK. I have-- sorry, I don't want to monopolize time. I'm going to give it to Tatiana, and then I'll raise my hand again.

Marian Thacher: OK.

Niccole Scrogins: OK, cool. You bet.

Marian Thacher: Tatiana.

Tatiana Roganova: I appreciate it. I used technology to make sure that you can't ignore me.

Marian Thacher: You raised your hand.

Tatiana Roganova: Right. So anyway, I think, Marian, you kind of answered my question already. Answering Niccole about the goal should be more student centered. In the past, I think we were taught that the best way to have it formulated would be the students will be able to, or the students will do this and that by the end of the school year.

So I was a little bit surprised to hear people are using the number of computers they're going to buy. But also, I think you mentioned that professional development for the teachers might be one of those goals. Or do-- still we need to focus on students first, saying the students will benefit from this professional development in this and that way. I don't know how I'm going to measure it though, but any suggestions here?

Marian Thacher: Yes. Well, hopefully professional development of the teachers does benefit the students. That's pretty much the idea of professional development. So you can frame it like that. Just be specific. Like, don't say we're going to have four professional development sessions for our teachers this year.

It should be based on your data, like our students are asking for more, I don't know what, help with getting into careers. And we're going to do some professional development with our teachers on developing our IET program or something like that.

So that it's specific, and then the outcome is measurable that you did the professional development, and it was on these topics. And how are you going to measure how that affected students? That's tough. I don't think any of us has the perfect answer to that. But you hope for more learning gains because of whatever you do. But it's hard to measure because there's so many variables.

Carol Hirota: Tatiana, I like the idea of always focusing on learning gains. And then in your implementation plan would be things like supporting professional learning for faculty. Supporting integrated technology curriculum for students. But putting those in the plan helps you get to that goal, right?

And like learner persistence could be the plan, not necessarily something you're measuring. But if you have improved learner persistence, you will probably get better improved learning gains. That's what you hope. So if you're always focusing on something about student learning, you could look at your plan and be very creative in your plans because there are certain things that will provide support for your students to do better. And that's really plan implementation.

So if you didn't meet your goal, it's not like it's a bad goal, we just need to look at our plan and say, OK, what do I need to do differently to reach that goal? And it's usually in the plan. That's maybe what needs to be changed. I don't know, it's just another thought.

Tatiana Roganova: Thank you. Thank you.

Marian Thacher: OK, Niccole, do you still have your hand up? I mean, do you have a new hand up? OK.

Niccole Scrogins: I have a new hand up. OK. So I want to preface this next question just with my driving intention might help the way you answer it. But I'm trying to understand what the state's goal is-- like the floor and the ceiling. I'm trying to find the space between the floor and the ceiling around the 12-hour participation conversion and wondering obviously CDE sees equal emphasis on persistence beyond that 12 hours and the post-testing rates.

But I guess I'm just trying to find, yeah, what's the floor? What's the ceiling? What is the overall goal? Is it just simply progress beyond that 12 hours?

Marian Thacher: Dr. Zachry, is that-- can you speak to that one?

Carolyn Zachry: Can you repeat that again? Progress past just 12 hours of instruction?

Niccole Scrogins: Yeah. So simply put, just beyond increasing the 12-hour participation conversion, does CDE see equal emphasis on persistence beyond the 12 hours and the post-testing rates?

Carolyn Zachry: Yeah. So we look-- so we're looking for multiple things, right, when we're talking about closing the gap. So certainly, those students that we move from just reportable, right? Less than 11 hours or less to a student that has 12-plus hours, which would include a pretest.

And then we're also looking at the other gap, which is that persistence from pretest and post-test. So I would look at them as two separate-- I mean, we look at the data as two separate pieces, right? Those students that we get to that 12-hour mark and they stay in the program, and then those students that you've pretested and post-tested.

So those are actually two different data points that you'd see in the admin portal on TE.

Niccole Scrogins: Yeah. I think I'm overcomplicating it.

Carolyn Zachry: Yeah. Yes, you are, Niccole. You're making it a little more complicated for yourself, right?

Niccole Scrogins: OK.

Carolyn Zachry: I mean, so if you really want to look at student persistence, and what you see is you have students that are your participants. They're there at least 12 hours, but they're not staying past 20 or 30 hours, and you're not getting them to that post-test, then why is that? What's the barrier to those students?

Or you're getting them to the 40, 50, 60 hours, but then you can't seem to get them in for the post-test, right? Again, what's happening there? So it's looking at-- it's using your data to look at those pieces. On the other side, on the other side of that is if you have, say, an orientation, and at that orientation, students do their pretest, right?

They get all their information, enrollment and everything filled out, and then they don't come back, why is that happening? Because maybe that was one hour or two hours. So they're reportable, but they're not a participant, because they haven't gotten to those 12-plus hours. So what is that barrier?

So those have been the really-- that's really been our focus for the last, like this whole grant cycle, that's really been our focus. Kind of coming back from COVID and everything is really looking at those two areas of our students.

Niccole Scrogins: Got it. OK.

Carolyn Zachry: Does that help?

Niccole Scrogins: That makes me feel better. OK. One more macro question for you guys. Again, digging for gold. Just curious what common pitfalls agencies fall into when drafting their SMART goals, and if there's anything we should be proactively avoiding.

Carolyn Zachry: Well, I can jump in a bit on this, and then I'm going to really let the readers jump in. I think part of it is you're making your goals too big, right? They're too big, too grandiose to really be achievable within a year.

So if you think about this as knowing that you have to do a CIP every year, right? You could look at this as some long-term planning with smaller goals during that time, right? Which I think a lot of agencies are looking at. We're trying to move the needle with persistence over a multiyear process.

And so this year we're going to do this piece. And then, oh, yay, we achieved that. So then next year-- but you don't go from persistence of, OK, 20% to 80% in a year, right? So what you're measuring might be too grandiose or too big to be achievable.

So really, I think that's one of the pitfalls or the things that people fall into, is wanting to do a really big goal that really isn't achievable. And so I am all for those long-term-- that long-term planning of where do you want to move the needle over a longer period of time and using the CIP as your, OK, we're going to get there. This year we're going to move it 10%, and then--

Niccole Scrogins: Incrementally. Yeah.

Carolyn Zachry: Incrementally. Yeah. So that's just my-- just things that I've heard from our readers, and then I'm going to let our readers chime in.

Niccole Scrogins: Thank you.

Marian Thacher: So Niccole. OK, Penny. I was probably going to say what you're going to say. So go ahead.

Penny Pearson: That's OK. You go first then.

[laughter]

Marian Thacher: I was going to say one thing that happens that's difficult-- that's challenging for the readers is you say we're going to increase our persistence by 10%. And there's no way for somebody who's not there with you to know what's 10%? 10% of what?

So if you give the baseline, like this year, it was 38. Next year we want to be at 42 or whatever percent, that is helpful. So just be aware that who's reading your plan doesn't know you. And we try to be very careful that we're-- readers aren't scoring people that they're-- where they work or where they're next door to or whatever.

Niccole Scrogins: Right. OK. That's helpful.

Marian Thacher: So who's reading it, if you can just give them a little even a clause in a sentence of context for your like, we serve 550 students or something like that. It gives us an idea of what's the context for your goals. That's very helpful.

Niccole Scrogins: Perfect. OK.

Penny Pearson: And, Niccole, what I was going to say as well is referencing what Carolyn said, keeping it simple. And second, check your dates, especially on your carryover goals. For whatever reason, that seems to fall through the cracks. And we're like, you might have even revised that carryover goal, you just didn't change the dates as to when they were going to be implemented.

Simple things, but that's one of the things we catch a lot. And it's very heartbreaking to me because I'm like, you're almost there, but I'm seeing it as for last year. Show me as for this next year.

Niccole Scrogins: Yeah. No, these are great tips. I appreciate you guys so much. Thank you.

Marian Thacher: Thank you.

Penny Pearson: Tatiana.

Tatiana Roganova: Yeah. My question is about the readers. How do you select the readers? And do they have a professional training to become a reader?

Marian Thacher: Yes, they have training, and they have norming so that they look at a bunch of goals together and score them together and see if they agree. And every year, we talk about what were the challenges that we-- how can we make this better next year?

They're all people-- they're all adult educators from the field. So they know adult education, but they might not know you. They're not going, oh, yeah, Tatiana, I know she was going to do this.

Tatiana Roganova: Thank you.

Marian Thacher: Mm-hmm.

Penny Pearson: Have we exhausted all of your questions?

Marian Thacher: And that's OK if we have. But now is your chance. You got 10 more minutes.

Penny Pearson: So, Marian, we should do a poll. How many of you have-- of in this group, you can just do it with a little raise of your hand, have started your CIP? I know Niccole has. I got Kate.

Elsa Magana: Does it count if we tried?

Marian Thacher: Yes. Yes.

Penny Pearson: Absolutely. Absolutely.

Elsa Magana: I looked at my data. I looked at the student survey, and I looked at the teacher survey, and I tried, but we'll see.

Penny Pearson: Great.

Marian Thacher: Thank you. Yeah, that makes sense what you're doing now, is looking at what your data is and where you're going to get it, and pulling your team together and meeting and talking about what you want your goals to be. That totally makes sense.

Penny Pearson: Yeah. And we really do encourage you to submit your plans as early as you can, because then it gives the reviewers the opportunity to send it back. And there's like no perceived penalty because-- there is no penalty at all, but we can then get it back to you, and you're going to be able to revise it and resubmit it.

And it's all on time. It's not past the deadline. And it's very clean for you and your organization to have your CIP plan approved and on the books by April 30.

Marian Thacher: And one thing I want you guys to know, if you don't know this already, is that last year, if your plan did not pass, that was sent to your superintendent and everyone. And so that's not happening this year. If your plan does not pass, it's sent back to the person who submitted it.

And passing, I think, Jong, I think goes to everyone. But not passing just goes back to you. So that should make life a little easier. Walter.

Walter Kirklin: Oh, sorry. I'm not used to Zoom. I'm a Team guy. OK, so I deal with incarcerated students. And I haven't asked my boss yet how we did it last year, if we did it last year. But the technology survey for students, it's kinda hard.

Our guys, the only computers they have access to are in class, and then every link has to be vetted and approved by our technology people. And so that is a laborious process. Can we get--

Marian Thacher: That-- yeah. That's a really good question. And I did see Oscar on here, but I don't see him on here now. But I don't know how you guys did it last year.

Renee Collins: I think-- I'm not positive about this, but I think they maybe took it on paper in some cases, and then there was somebody who inputted the answers into the URL, into the actual survey. Which is labor.

Walter Kirklin: Yeah, 13,000 [ INAUDIBLE ] students, so.

Carolyn Zachry: Yeah. Walter, thanks for that question. I think I'm going to take that back, and we'll have a discussion about that because I know there are some pieces that we-- corrections doesn't have to do because of some of those limitations that you have. So I'll bring this back to our team, and we'll talk about that.

Walter Kirklin: [ INAUDIBLE ] Senior Center Institute

Renee Collins: Tatiana.

Marian Thacher: Tatiana, one more.

Tatiana Roganova: I just want to thank you, Marian, that you mentioned that if the plan needs adjustments, it's not going to go to the school district, where that's what happened to us last year.

Marian Thacher: Oh, sorry.

Tatiana Roganova: We needed a minor adjustment. And then the whole school district was panicking. What is the matter? And I hope it's not going to happen this year. And if there is an adjustment that needs to be made, we are happy to do that. But we don't need to alert the whole school district about it.

Marian Thacher: Yeah, that was pointed out to us very intensely last year. So we got that.

Tatiana Roganova: Thank you.

Marian Thacher: OK. Well, thank you, everyone. Great to see you here. Glad you're thinking about this. Carolyn, closing words.

Carolyn Zachry: Yeah. Thank you. Thanks, everyone. Great questions. Really thoughtful questions. And I know I actually want to hand this off to Renee to talk briefly about our digital learning guidance. And she has an invitation for all of you.

Renee Collins: Yes.

Carolyn Zachry: Renee.

Renee Collins: Thank you so much, Carolyn. I'm sharing this slide. So last week, we published our updated California Adult Education Digital Learning Guidance. And I've been sitting here thinking that this is perfect timing with your CIP planning, because if some of you are still working on or trying to discern what your technology goal might be, you could look to the Digital Learning Guidance and see what is recommended.

There's a lot of practical, relevant examples for how to integrate technology into your programs. And this year, we added a full chapter on artificial intelligence. We have information in there about accessibility and UDL in practice.

We have something called voices from the field, where we're hearing from several of our practitioners throughout the state on how they're using the guidance to make impactful changes for their students and their programs.