[audio logo]

Announcer: OTAN, Outreach and Technical Assistance Network.

Penny Pearson: Well, first, let me just do a quick welcome and introduce my co-presenters here. Let them introduce themselves. But first, my name is Penny Pearson. I am currently retired, but as educators, we can never fully retire. So I came back to work with OTAN on a couple of special projects. So that's the fun part. I get to pick my projects. And I'll turn it over to Jaemi so she can introduce herself and then pass it along and we'll get started.

Jaemi Naish: All right. I'm Jaemi Naish. I am a OTAN subject-matter expert, and I'm also Director at Tamalpais Adult School in Marin County. And a little bit about me, I've been in adult Ed now for about 17 years. And before that, a total of about 25 years in education. And came into adult Ed by accident, like many of us. Not really knowing what adult Ed is. And just the minute I got into adult Ed, I really never looked back.

And I feel so privileged and honored and love my job. And so when I get a chance to be around adult educators like you all today, it makes a Friday even better. So welcome and happy that you're here today.

Penny Pearson: Thoibi?

Thoibi Rublaitus: Good morning, everybody. Oh. It's afternoon. Sorry, I am still feeling like morning right now. So yeah. My name is Thoibi Rublaitus. And I serve as the principal at Corona Norco Adult Education. I have been in adult education in California for 17 years. And the school that I serve has about 3,000 students on average that we serve total. But actively, we all know adult students come and go. So we have about 1,600 right now.

I have been a part time ESL teacher for 10 years before I became an admin. And I also serve on CCAE, CAEAA, and I am the current president of the adult education council at ACSA. Thank you for being here and the opportunity to share a little bit of what we do and learn a lot from you.

Jaemi Naish: And Penny, you got to go already. Anything else, anybody wants to say? I'm going to just kick us off, I think.

Penny Pearson: I think that's a good start.

Jaemi Naish: OK. We are recording this, and we will share our PowerPoint. So if you'd like it, you can email one of us. I'll just start off with that. Usually that's one of the questions of the day. What would be really helpful for the three of us is if you could put your name and your position or how you're associated with CIP, that is really helpful as we go through our presentation today.

We have a good hour of content to provide to you, but we also have time at the end where we're going to play with AI, and we're going to look at how-- we're going to really deconstruct CIP goals. And that's for people that want to do that work in the last half hour. So the other thing I'd like to know is how many of you, this is your first CIP goal experience. And you can just either do a thumbs up or put a thumbs up or you can put it in the chat.

It's really helpful for us to know how many people. Yeah. OK. So a little bit of a combo, I think, of people. I'm seeing some thumbs up and a lot of nots. So that's great. So one of the things that we're hoping from our presentation today is that it's interactive. It's really fun when we can get together and share our goals, also our best practices, of course. And so those are some of the things that we're going to do today. So I'm going to share my screen. I'm going to get us started. Let me move that out of the way.

OK. So as I mentioned, I love adult Ed. Any time I get to be in a room with adult educators, it feels like a good day to me. There are many things that we're required to do as a WIOA-funded agency. And I imagine many of you are that. And that's why you're on this call today to learn about the Continuous Improvement Plan. It's one of the things that we have to do in WIOA.

And I think one of the nice things about this, this goal or this task is that we have an opportunity at each of our adult schools to make it really meaningful and really region-specific or school-specific on what that plan looks like. So what we're hoping to do today is to make sure that everybody understands what needs to get done. That there are three SMART or SMARTIE goals that are required, one being tech-focused.

We're going to make sure that you learn how to assemble a team, an effective team. We're going to talk about using data to create these goals and to evaluate the goals. And hopefully, we will all get to be able to share out those of us that want to share out your goals and have feedback on them, or you want to listen to what others are doing. We hope there'll be a few in the group that will want to do that today.

That makes it fun. So I am going to share [muffled voice] goals, and one of the things that I just want to say up front about these goals is they're very region-specific. So the goal that I choose might be really interesting to you, but it might not fit with your demographic and your school's needs. So that's just something to think about as we go through these goals.

Let's see. Who can, in the chat-- and I can't see the chat well, so I'm going to ask Thoibi and Penny to let me know. What's the IE in SMART goal? So we have SMART goals and then we have SMART with an IE. Who can tell me what the IE stands for? I'll just put it in the chat. And Thoibi you'll have to let me know if you can see it in the chat.

Thoibi Rublaitus: Yeah. We have one response from Aaron Anton, and he says-- he or she, I'm sorry. Inclusion, Equity. We got another one from Carrie.

Jaemi Naish: OK.

Thoibi Rublaitus: And Another one for Meredith.

Jaemi Naish: OK. Good. So you know what the I and E is. Now, while that's not a requirement of the SMART goal process, we would highly encourage you to consider doing that, especially those that are not new to CIP. Maybe if you're new to CIP, you can consider what the I and E means to the SMART goal process.

But we will challenge those that are not new to CIP to consider adding the I and the E onto your SMART goals. So we're going to go through a couple myth busters. And again, I'll rely on Thoibi and Penny to help me out here. And you can just do a thumbs up or you can put a yes or no in the chat. Are the goals that you create, are these CIP goals evaluated by at least two evaluators? Yes or no? Thumbs up, thumbs down. Yes/no in the chat.

Thoibi Rublaitus: Nothing yet. At the bottom of the chat box, you'll see a little smiley face. And that's a place where you can also put your thumbs up or other type of answer. We don't have the full gallery up on the screen, so it's a little more difficult to watch all of you do a thumbs up on the screens. So if you do it in the chat.

Jaemi Naish: All right. I see Compass says yes.

Thoibi Rublaitus: Oh. I see a few coming up.

Jaemi Naish: OK. So yes. Your three goals are evaluated by two experienced reviewers. And that is the case every year. They are trained adult educators that do look at them and evaluate them carefully using the rubric. And I might have just given you a clue as to what number five is, but we'll go through them.

You heard me say three goals are required, right? Yes. Hopefully everybody heard that. That's a little bit different than in past years. It was up to three goals, no less than 1. So going forward, it's three goals. Does one of those goals have to be-- do all three of those goals need to be tech-related? Yes/no. Thumbs down. OK. Good. Yay. Everybody knows that one.

They can be. You certainly can write three tech-related SMART goals. That would be OK. But the requirement is just for one. And we're going to do a little look in the portal later in a couple of slides from now, and I'll show you how you indicate that you're looking to add one of your SMART goals and make it tech-focused.

What about number 4? Does your goal need to meet all five components of SMART? Yes or no. Thumbs up or thumbs down. Yay. Seeing thumbs up. Yes. So your goal will probably not pass through the evaluation process if it doesn't meet all the S-M-A-R-T. So that's really important that it does have to go through and meet each of those requirements. And what's going to be really fun at the end of our presentation is Penny is going to walk through a little bit of AI and have AI be your assistant to see if you have a question.

Now, most of you are going to be able to get through it, because you're going to be doing it as a team. And so you can help each other when it doesn't look SMART or SMARTIE enough. And there's always ways to make the goals smarter. But as long as they're in the SMART category and they meet those components, you're good to go. We do have a rubric. I gave you the answer to that one. So yes. There is a rubric, and it is shared with you. It's in the portal. And then what about number 6? Is the CIP supposed to be written by one person? Yes/no, thumbs down. Yeah.

Thoibi Rublaitus: Lots of nos, that's good.

Penny Pearson: Yeah. So yes. Please don't just do it in isolation. It is definitely a group experience. It's a team approach. And Thoibi going to be talking more about that. All right. So moving on. Oops. Here we go. So I'm not going to cover this extensively because I feel like this is something we talk about a lot in adult education.

But I will just make sure that everybody understands what the acronym is. SMART-- Specific and Measurable, A-- Actionable, R-- Realistic, and T-- Time-bound. So when you're creating your goals, you need to make sure that these five areas are covered. So you don't have to be perfect, but they do need to fall in these areas. And when they're being reviewed, the reviewer's job is just to make sure that they're meeting these minimum requirements of SMART. And again, I will challenge some of you to add on the IE as well. One of the things that we wanted to put out there, sometimes the SMART goal process can feel intimidating, especially if it's new.

Usually, we see people running into problems if they're just doing it themselves and they're not doing it using a team approach. But OTAN and CALPRO-- Penny and Thoibi and I, we're here to help you create or craft a SMART goal if you need that or to look at it with a reviewer eye, if that's something you're interested in. One of the things that I wanted to share also, and Penny, I'm hoping-- or Thoibi I'm hoping at the top of the page, you can cut and paste the SMART goal worksheet that I would love to just make sure that everybody has access to.

It's a document that you can use just to make sure that your goal is SMART. If you're feeling more confident, you can just verbally take it through the S-M-A-R-T. And I'll do that with you a little bit later in our goal. Any questions about SMART goals or anything like that? Hopefully Penny or Thoibi is able to copy the worksheets that I shared there. Let's see what else.

One of the things that I want to also say about the CIP process, not so much to do with SMART, but this is a really unique opportunity for you to come up with what your agency or your school or your students need. Being very school site-specific. It doesn't necessarily have to match or be aligned to WIOA. Most likely, it will be matched to or aligned to CAEP. But it's just a nice opportunity to get a group of people together and think about what do you need. What student voice needs to be brought forward or heard or elevated?

What need same, so that you can make your program stronger, you can bring better support services to students, that kind of thing. All right. And moving on. This is just another even easier without needing a worksheet way to go through your goal as you create it. So if you create a goal, you can just take it through this acronym quickly.

And then we have talked about the I and the E. And that it's important to start crafting your goals eventually, and including the I and the E work. And in adult education, we do that a lot. We do it without even often knowing that that's what we're doing. I mean, that is our work in adult education. All right. So next, I want to talk about the portal. How many of you have access or have been into the California Adult Education online application and reporting portal? I'm hoping that you have access. Good. OK.

Thoibi Rublaitus: I See a couple of yeses coming through.

Jaemi Naish: OK. And has anyone started working on their goals out of curiosity? You can just put a yes and no. OK. Thanks. Yeah. Good. Some people are-- OK. It's probably a really good time to get started if you haven't. And hopefully, this will give you some motivation to get started.

There are some components to it. I think the first piece is that you need to make sure you have access to this portal because that's where the CIP, I guess, program is held. The surveys, the teacher surveys are held, and also where you put your information. So a lot of times, your administrator has that information and can add you to the portal or you'll have to speak with your CTE consultant also. Those are different ways to get into the portal.

So what I'm going to do is take us through-- oops. A little bit about the portal. So I'm going to quickly take us through a couple slides, but then I'm going to stop sharing for a sec and get into the portal and show you a little bit about how we've gone through and worked on our goals. So these are the components that we're going to talk about. So just looking at your plan last year and evaluating it. They do ask you to look at what was your goal last year or goals last year. And are you going to carry them forward?

And a little bit more about SMART, which I don't think we need to talk about anymore. And here we go. So let me stop sharing here for a second. And I'm going to get us to my portal here. OK. All right. So hopefully, you're seeing the portal here. I've already signed in. And you're going to start with WIOA. All right. So you're going to click on WIOA up here. It's going to take you to this navigation pane here. And you have a couple of different options.

You're going to press continuous improvement plan, and we'll do that in a moment. But I want to just draw your attention to surveys for a second. Surveys are the student intake surveys that were done. And I'm going to click here now. And what's nice about this page is that you have data for the last three years.

So if you click on any of these years, not only will it show you how many students you that have taken your intake survey. And I have somewhere around 200 right now. But it'll show you last year and the year before, and you can look and see how data has changed. The survey is a tiny bit different this year than it was last year. And then down below, you'll see that there is the teacher assessment surveys as well. And how many of your staff have taken them.

We're not going to spend a lot of time talking about student surveys in this webcast because we've already covered that, but we're happy to answer any general questions about that. So before we go there, I'm going to bring us back to where I really want to show you today, which is the Continuous Improvement Plan.

So we talked about the guidance that is here. So if you click here, it will bring up the guidance document. And I would highly recommend that everybody look at that document, if you haven't, and use it as you're crafting your goals. It also contains the rubric. So it's important that as you craft your goals and when you're done with them, that you look at the rubric. Because that's what people need to-- the reviewers are going to be looking at when they review your goals.

And then these are some webinars that you can watch as well if you need extra support. And I will say that there's lots of support available to you. So not only can you schedule like a 15-minute consultation with any of the OTAN subject-matter experts in some with CALPRO, but you can also email any of us for extra support. We want to make this as painless as possible and make sure that you get your goals created and passed.

So I'm going to click on Edit Plan. And it will go through just the steps you need to take. I haven't done a lot. You can see that I've created some goals, but I haven't created the action steps. So I'm going to click on my goals here. And I'm going to show you a little bit about what we've done here.

So we've created three goals, and I will let you know how we've done that. In fact, let me just get-- let me click here for a second. So with my CIP team, we have-- myself, our assistant director, we have our ESL, our CTE, and our GED diploma leads and our office staff looking at our CIP goals.

I really want them to be meaningful. So I don't want to do them in isolation. So I do actually make a big deal of it because I do want to use this data. A lot of the time, the goals that I'm creating are really doing what I'm hoping they'll do, which is highlight our schools and our programs and our student needs.

So one of the things that we've created locally is we look at this data table, and I'll show this to you quickly. Don't want to really focus too much on this, but this is some of the data that we've created in a way that is meaningful for us. So one of our goals is just to do better than we did the last year. OK. So that's one thing that we use and look at. And then we looked at-- this is another document that we use in terms of data that we look at that helps craft these goals and make sure that we're in the right direction.

Again, a little bit of our data table, but one of the things that we created locally that really brings the data in a little bit tighter for us to use for our goals is our demographics and where we see the trends in terms of cities or programs and the changes. So that's a little bit in a nutshell what we use. And then again, I think I mentioned that this is a document that you can certainly use to create a SMART goal and just make sure you're on track with making sure that your goal is SMART.

So that's a little bit about the process before we get here. Now, I've already had a couple meetings with my team. And so we have identified, we've brainstormed, we've looked at where we wanted our goals to be, what we thought was going to be most impactful, what the need was right now, which really is actually different than last year.

So that's been an interesting discussion in our staff. We're seeing some trends in ESL that we didn't know or that were different from last year. And I've actually heard a lot from the field that waitlists for ESL are everywhere across the state. So that's been interesting. So the way that I personally like to write my SMART goal or that we look at is I like to give a little bit of an overview. Three sentences about the SMART goal itself or why I'm leaning in that direction.

And then my SMART goal is here. That's it. And what I've done is I've taken it through the S-M-A-R-T. I know what's SMART. Could it be amazing and perfect and better even than this? Yes. But right now, this is good. This actually fits. And you'll note up here that it indicates in green that it's tech-focused or tech-related. OK.

Thoibi Rublaitus: Jaemi?

Jaemi Naish: Yes.

Thoibi Rublaitus: There is a question from Lori saying, can you tell us a little more about your first data table?

Jaemi Naish: Oh. Sure. Is there anything in specific?

Thoibi Rublaitus: And I'm sure which one that was.

Jaemi Naish: It's probably this one. You can let me know if that's not the case.

Thoibi Rublaitus: Lori, is this the one you meant that she's showing? Yes. This is a correct one.

Jaemi Naish: OK. We get so much data in adult Ed. We're looking at TOP Center Price, we're looking at ASAP or other SIS, there's so much data coming. We're looking at courses, data portal, and Thoibi going to do a lot of data discussion later. This is something that our agency put together. And it's really like an h year view. We don't need eight years. We probably need more three or four. But it's just we've kept up the dock.

And it just gives us some ideas of where we are. We use this data. It comes from the payment point summary report that we look at a lot. And so this is just our generalized data. And if we go down and look at, of course, it's by program year. Again, we just try to do a little bit better or as good as we did the year prior.

These are our different levels. We're a small program. We do a little bit-- we serve a little bit under 600 students every year. And we just like to-- I just am drilling down in certain areas, and I'm sharing this data out with not only my CIP team, but really my whole school-wide team. So data is a big part of our discussion in our meetings.

Thoibi Rublaitus: Jaemi?

Jaemi Naish: Yes.

Thoibi Rublaitus: Lori is also asking, where does the percentage come from? Are you calculating that based on prior year data?

Jaemi Naish: This comes right off the payment point summary report.

Thoibi Rublaitus: OK.

Jaemi Naish: Yeah. So I'd be happy to do a side discussion at some point, if she wanted to just have someone walk through that report. But it comes directly from the payment point summary report.

Thoibi Rublaitus: Yeah.

Penny Pearson: OK. And yeah. Please let me know if there's any other questions because it is really hard for me to-- I don't have that whole view. So again, these are some of our goals here. The next step, let's say we're ready now actually to move on and start adding our tasks. So if I press the Edit button, I can change anything here. You'll notice how many characters are left, how many marks I have.

And this is the next plan. So each of my teammates is going to be taking one of the goals and starting to work on the action steps. So what are key supporting strategies? How are you measuring the success? There's that M. Who's going to be doing it? When are the due dates? And who can tell me what this SLP support is? CDE is for the Department of Ed, but what does SLP stand for? Can you put that in the chat, if you know?

Thoibi Rublaitus: And Jaemi, I'm going to add one more thing. And this is only because experiences have shown me in the past. When you're on this page and if you make any changes, I mean, even if you find a little typo, you hit that Save button, because you can get easily distracted and you can end up-- you lose whatever you've added. That's why that previous word practice document can be very valuable, because you can do all of your writing there and just copy and paste the plain text into the SMART goal area.

And if you needed more space, in the lower right-hand corner of that screen, there's those little dashed lines on the corner, on the right side, Jaemi, to the right of the Save button up. And you can expand that window. Oops. A little farther. Oop. On the actual goal, the box where the goal is, OK. Where your cursor is right now, go to the right.

Jaemi Naish: Oh. This.

Thoibi Rublaitus: Right there. That, you can drag out and make a little bit bigger, because sometimes, you might have your screen zoomed in or whatever because you want a larger text. And if you miss some of it, you might miss that there needed to be a correction made or whatever. And this just comes from my accessibility side. Sometimes I personally need to zoom in on my screen so I can see everything. And this is a way to it allows me to see everything, even in a text box like this. So that's it. I just wanted to--

Jaemi Naish: Thank you for that--

Thoibi Rublaitus: --save all the time.

Jaemi Naish: Thank you. I actually never knew that existed. I was like, geez, man. How long have I been doing this and I didn't know that I could pull that out, because I was looking in a small little window. Yeah. So I think, like I said, the next step here is going to be for us to go through and start adding our action steps.

And this looks a little different than it did last year and in prior years. But you're putting in your action steps here. How are you measuring your success? How do you know your goal was successful? Who's going to be doing-- who has responsibility for parts of the goal? And then when do you hope it's done by? When's your plan? And then back to this question. SLP. And I'll say it's-- I'm sure somebody put it in the chat and knows, but I'll say, it's our state leadership projects. And who are those people?

You're on one of them right now. A combination. So it's OTAN, it's CALPRO, and it's CASAS. Those are the state leadership projects that interact with adult education in California. In addition to CDE. So if you have OTAN coming in and doing work for you around one of your goals, for instance, one of our goals probably saw is to implement Canvas in our ELL Pathways class that we're going to be offering in building this semester and working on next year, I am most definitely going to be relying on OTAN to come in and help. I also am going to use instructor.

So that's just one example of how we're going to be partnering with one of our SLPs. And I have to imagine that everyone on here attends J's webinars regularly, or perhaps you're going to be looking to get some help with maybe the teacher portal, and that's going to be part of one of your goals. There are so many different opportunities and options that you could go with.

So that in a nutshell is-- let me just go back to our goals. Is what the portal looks like. And what's nice about this portal is that it's pretty straightforward. You definitely have the option to get going and working on it now and to submit early if you're ready to go. So my goal is to get it done in the next couple of weeks, because I don't like to have all these deliverables hanging over my head.

So I'm going to stop-- I'm actually going to go back to our presentation here and come out of that portal, and see if there are any questions for me about any of what I've presented so far. OK. Let me see if there's any questions. Now I can look-- OK. Thanks. Thank you, Maria. One of the questions I have for all of you before Thoibi we get started is-- and you could just put in the chat, do you have goals ready to go? Do some of you know know what your draft goals are? You can put yes/no. You just saw mine-- that's what we're presenting.

Good. Oh. That's so great. Good. OK. Good. Great. And what I'm hoping today is that we'll get to share some of those. And that might really help some of the people who haven't quite gotten there yet get there, because it's a little intimidating. And encompasses roughly. And that's great too. It's OK to have some draft ideas. So I am going to continue sharing, but I'm going to move ahead and be toybiz. Oops. There we go. All right. Thoibi, you're up. And I'll just move the screen ahead for you.

Thoibi Rublaitus: Thank you, Jaemi. Thank you. That was really helpful. Even for me, I've done this before, but just going through all of that step again just reminds me of all the different things that we can still familiarize ourselves before we get on board with this. So for my part, I will be sharing a little bit more about deeper diving into data and looking at data little bit. But before I start, let's do a quick check. When is the CIP due? If you know it, put it in the chat box.

Wonderful. There you go. We are all aware, it's April 30. Then do you have a timeline to complete this? How many say yes? Timeline. All right. That's good. We've seen quite a lot. A few not yet, but hopefully today, we'll get the timeline started. Who do you include in your CIP team? For those of you who've already got a timeline, do you already have a list of people on your CIP team?

I agree. This is the first step today. All right. That is what I would start with. So going on to my team, how we do things or how we've been doing things is taking stock. Yeah. Thank you. So taking stock. What do we do? We set a timeline, we create a data team, we start analyzing data. And then for my team, we have set a timeline. I will be sharing a copy of my timeline here with all of you, which you can use so that you don't have to recreate the wheel. That's why we have these meetings, learning from each other in community professionals-- community COP, it's called, right?

Community of professionals. We are now looking at the last year's set of goals. And we learned from Jaemi about the SMARTIE goals. So we want to go and check if our goals that we have set or we are drafting right now are SMARTIE. And do you meet the goals? Did you meet the last year's goals? Because before you even start editing and planning the next step, we have to first go back and report on last year's CIP goals.

And then the last thing that my team was thinking we want to do is, do we want to keep the same goals with a slight change, or do we have to recreate it? So to do that, we have to go back and see, did we meet all of our goals? If we met last year's goals, then we don't need to redo that. But if we haven't met the goals, maybe we can make a slight change and continue the last year's goal.

So last year, we were required to do only one, but this year, last time. But last year, of course, we didn't have to do the CIP because we had the WIOA reauthorization. So moving on to the next slide, so here is a timeline that I set for my team. So I'm going to share some of the practices that we do, hoping that you will learn from it and it will help you save time.

So for my team, I put this little Google Sheet together. Today is the 8th. So starting from week of March 11, we're going to take stock. Week of March 18, we are going to look at data analysis. Week of March 15, we will go and do a little bit more of the detail into it. Then we have a spring break and then we're going to make sure we have a draft.

And once we do that, for those of you who've done it the last time, first step is writing the SMARTIE goal, which Jaemi has shown, and her team has been ahead of the game by getting the SMARTIE goals ready, the three goals. But then what really matters is those action items, right? What action plan do we have to meet those goals.

So it'll take time. Just mind you, it doesn't happen easily. You have to have really thoughtful goals and thoughtful action items. So if we set a timeline like this, it helps us a lot. So I'm keeping ourselves-- giving ourselves one week extra so that if we need some more time, we're ready for it. We have that margin. Going to the next slide, I'm going to share this as a resource at the end. So hold on. And then the same Google Sheet, I also have another tab that talks about who will be in the team.

And in my school, like Jaemi's, we have the principal, the assistant principal who's going to do what, and the responsibilities. Who's going to lead the analysis of which data set and how we're going to do it. The data technician is an important part of being part of this team because that's the person who is going to go and get you all the data, from TE, from any other sources that you have.

We also make sure that we have one from each of our department. And so we have our ESL team, the ABC team. A CTE is not part of this team, but our IT is part of ESL. So we have that. And then because this year we need a technology goal also, we're going to include a tech team member in our team. So this is our winning team. I'm sure you will have a very good winning team on your team as well using this data. This tool that I will be sharing later.

We're ready for the next slide. So let's do another quick sanity check, because when we are on Zoom, we're also worried about listening to the person and we forget sometimes it's nice to engage, right? So let's do this. Again, think about it. We'll do a waterfall chat. So when I say waterfall, then you can put it in the chat. But what are some data sources you use to create your goals? Think about them. Take a couple secs and right, waterfall.

I see a lot coming up here. Was action plan, surveys, which we have seen. There are threee-year data surveys that we can get. Last CIP, TOPs, classes data portal, TE data, hours of attendance, student voice surveys, last data, last year's goals. Thank you, everybody, for sharing all the different data sets that we can use for this. Let's go to the next slide, Jaemi. Thank you.

So here is a set of data sets, the whole bunch of data sets that we can use that I've put together. So for our school, we've been looking at the WASC action plan. We're also due for a WASC mid-cycle. So it's perfect timing. We also use our district strategic plan and the consortium's three-year plan. Let's not forget that. We all have our part of a consortium. And when we are making our plans right now, our goals, we have to also make sure that they all align.

Then there are all kinds of surveys. And then often, when we talk about data, we think only about our numbers. No. Numbers are not the only data we have to look at. We have to think of qualitative data as well. So there's a tech survey. In the tech survey, there are portions where students write in. Don't miss those part as well. Teacher survey, then all the-- here's another very important part. Earlier, we talked about the State Leadership Project.

And if you didn't know, OTAN, CALPRO, and CASAS, they all offer numerous professional development. If we don't know what they're offering, we don't know what to add in our action plan. So please make sure you go and get all the offerings from all these state leadership projects before we start writing those plans.

The SMARTIE goal is easy to write. No. I'm saying. Sorry. It's not easy. It's difficult. But it's the first step. Once you have that, those action items, action plan is what makes those SMARTIE goals happen. So if we don't have the right tools to make those SMARTIE goals a success, we will be doing a disservice to ourselves. So look out for all those PD offerings and then all those office hour visits that are available so that you can add them in and make use of all those free resources that the state is providing us, including the links. And COP is community of practice.

So links have a huge resource on their website. That is the national project of not just state of California, but also at the state level, where communities of different sections have practice and their information is all there. That is part of AIR. Also, some of you have already mentioned TOPSpro, CASAS data portal, and we've heard Jaemi talk about and share with us the performance report, the payment point report.

And then on CASAS, you have the NRS table 4 and table 4B and the persistent report. This section is what I'm going to do a little bit walk through in the next few slides. Let's go to the next slide. So that's the CASAS data portal. So this is not a link. So what I'm going to do is, Jaemi, if I can take over the screen share now, if you can stop and I can share, I don't know if I can share. So I'm just going to go to a Google page. And you're going to see is that simple. Can you see my Google page?

Jaemi Naish: Yes. So that's on Google page. What I'm going to do right in front of all of you is just do CASAS data portal. And I do it quite often. So it's right on the top 2. So all you do is this. And voila. It's right there. We go to WIOA title 2 database. That's California. So then you see here that you can have the state data, you can have table 4, table 4B, and persistent report.

These are some of the most handy tool that you can see, or you can find for WIOA data. For CAEP, of course, you'll have to go to TE and get it from your TE. But for VIOA, if you're not looking for this year's data, but the past year's data, it's very simple to go here. So then you also have employment follow-up outcomes measures. So that's another one you can look at. Let me just take you through at least one of them. What's the difference between table A and table 4B, some of you might ask.

Is there anybody who knows the answer? Feel free to write it on the chat. But what I know is that if table 4 is the one that includes everything, all of your educational functioning levels, whereas table 4B is for only the people who have got pre and post-tests. So it's good to look at both because that will help you give a bigger picture.

But for now, let's just only look at table 4A. So if I were to go to 4A, this is what you see. You see your state, the state goal, and the state average. This is very helpful in making your plan. Remember how the state's biggest priority right now is the gap. Mind the gap. You want to mind the gap.

So how are we going to mind the gap? We're going to mind the gap by looking at, hey. What's the state average? What is the state goal? Am I meeting the goal? So it's so easy to find your first goal. Just go there and see, OK. Am I meeting my goal in all of these different levels that we serve, or am I not?

If I'm not, then I know where to focus. So this is the one that J shows us all the time as well. So to go to find this, all you do is when you do the dropdown menu, you can see the county, the provider, geographical region, you can see more. But what I usually do for my school is just go to agency. Then I'm going to do call, go for my school name, which is coronanorco.school. So Corona-Norco Unified School District.

And this here, program here, the latest is at the bottom. So if I want to go to the last three years, usually, I personally, when I go to look at data, one year's data doesn't tell a lot. When you look at least three or four-year data and then do an average, it really helps. That's why Jaemi had six-year or seven years list of data.

So at least three years is best, because that way, you see the year before and the past year, and you average it out. That's better than just going to one year, because one year doesn't do a lot. So for example, I can just go to 2023 and submit. And then I get this here very easily. I can get to see, where is my school and where is the state goal, and where is the state average?

So I'm below state average here. So this is something I can focus on. And how many students did I serve in this level of ABE beginning lit? 21 students I serve. So the bottom one in parentheses are the number of students you served in that program year. So then I also want to look at 2021-22. I click that and submit it. And then I get to see the next year.

Then I'm going to get at least three years so that I can get an average. I'll do that and add that. So I get three years worth of data. And then now I can really look at how my school is performing as compared to the state goal. And if I'm not meeting the state goal, that's the goal we have to try to meet. That's how we're going to bridge that gap.

So usually, this thing is really handy because you can download as an Excel sheet. And then you can plan it for your-- and then you can play with the Excel as you need. So this is one excellent tool. This one is only federal table data 4, right, table 4. You can also look at table 4B if that's what you want to look at more, especially for the ones who want to mind the gap between pre and post is better to look at 4B.

I would look at 4B if I were to look at how my pre and post are doing because this is everything, whereas the other one includes pre and post only. And then if you want to focus more on the persistent report, then I can go back. And OK. We'll go back to that.

Jaemi Naish: Thank you for sharing this all. It's so helpful to just see someone else's agency data. And from a pro like you who is so data-focused, it's really, really helpful for all of us or at least--

Thoibi Rublaitus: You're welcome.

Penny Pearson: --I think a lot of us. Thank you.

Thoibi Rublaitus: And here is your persistent report. What is the persistent report, you may ask? The persistent report is to see that how many students in your school takes the pre and the post-test and make the gain. So, for example, if I'm looking only at the state average, it's not enough. So we'll just look at least one agency, which will be my agency right now because I'm right here doing this. Please remind me of the time too, Penny and Jaemi if I'm going over too long.

Jaemi Naish: You're doing well. We're at 1:50, so we have a little bit more time before we play around with AI.

Thoibi Rublaitus: So here is what you can see. Like the state average is 65 and our school is at 64. Now this one is even more-- this is the one which is like if you have-- the one that we keep talking about 12 hours or more. So a student's number will not be part of this data if we did not have a pre and post. On this table, only students who had pre and post and made a gain are the ones that are going to come up here.

That is what they call persistent. If they're a student, persistent means they have done their 12 hours. They've done a pre and post. And then you cannot do pre and post if the student hasn't had 40 hours or more. So this tells a lot whether the student has persisted or not. So overall, you can see our school served 1,650. Like I said before, we serve about 3,000 students on average on Cape.

But on WIOA, only 1,658 are showing up. Why? Because this takes only the people who did pre and post and got a gain, right? This will not show up all of them. And so this doesn't even show CTE, which is also another program that we serve. So this can really tell you what your persistence is like at your school. And then you can compare it with your neighbors or with your consortium members. So this is what I wanted to show everybody. Feel free to play with it. It's really interesting to know how you were doing. And that helps you to make your goals.

Jaemi Naish: Can I just say one thing too, that I really appreciated you talking about what's on this report and what's not on it, because sometimes you think that everything at your school is included. And we all know that even if you have an amazing one day workshop that's only eight hours like we do, it will never go on your WIOA report because it needs to at least be 12 hours or more with a pretest. So I'm hoping Lynn's question was answered. I think Thoibi got to it. But in case she didn't, do you want to elaborate a little bit more on that, Lynn?

Lynn Tanner: Thank you. I thought the persister report showed students who had a pre and post, but not necessarily that they had a gain. Just that they have a pre and a post-test. Have I been wrong that those are all-- that percentage shows pre-post with a gain. If they have pre-post and no gain, they're not on that persister report. Is that right?

Thoibi Rublaitus: Thank you for the correction, Lynn. When I talk about as if I am the expert on data, but I'm not truly an expert. I'm still learning. So correction like that is helpful. So persistent is only pre and post, whereas the 4B is a pre, post, and EFL gains. And if I'm wrong, somebody should correct me as well. But we can also go and confirm with Jay. You got that right, I think. Persister is pre-post.

Lynn Tanner: Thank you, Thoibi. I appreciate that.

Thoibi Rublaitus: No, thank you very much. That's why this is a discussion. It's a good discussion. So think can move on with my slides now, which is back to you, Jaemi.

Jaemi Naish: All right, here we go. OK.

Thoibi Rublaitus: All right. After that, I think what I was going to share a little bit about was the quick tour of the survey data. So this one, what I was thinking-- and I think Jaemi's already covered this though, right? So I also give you-- this is a different set of data. The one that we saw on CASAS data portal is completely different from this data. This is the data that we've been talking about that we've been collecting through OTAN OAR, which is Online Application and Reporting, as you can see right at the top there.

So for this, Jaime's already covered, so we can go to the next slide. So, like Jaime, we, Jaime and I both agreed that because we are practitioners and Penny is the one who is going to share, share, share, it will help us more to make our goals smart or smart by using later, which you're going to do. But Jaime and I, because we are practitioners, we thought we will share with you our goals and analyze it together with you so that that could be the practitioner point of view.

So this was our goal in the last CIP cycle. So this is what we wrote. And then we're going to practice to see whether we met our goal or not so that going forward, are we going to continue with the same goal or are we going to make a completely new goal? If we already met this, we don't need to do the same goal. But if we didn't, we will have to continue or tweak this.

So this is our goal. Take a moment to read it. Based on the partial gains we made in the goals set in 21-22 Corona-Norco Adult School will continue to focus on the ASE program. By the end of the academic year 2022-23, the goal is to improve the ASE low educational functioning level completion rate from 62% to 66%. And the persister rates will be maintained at 40%, if not higher, because we didn't want to set unreasonable goals.

Because when I did the same session the last time, I remembered sharing with everybody, we had very unrealistic goals. We went to say 5%, 10% without really thinking. But this time we said we will maintain it even if we don't want to go higher. So every time we set a goal, it doesn't have to be like, we're going to make that much gain. If you can maintain it, that's good enough as long as you're not going lower, right? So trying to make realistic goals.

Through timely-- this part is the part, which a lot of us forget. We can just leave with what I had before. But then we also added a little bit to include what we had in our action items. We're going to do this, but how are you going to do this? Through timely recruitment, registration, onboarding, placement of learners in the right classes, and implementation of learner-centered curriculum.

Using both in-person and online learning systems, the ASE team aims to increase the number of students who graduate to more than 100. So this is a good example to see what worked and what did not work and what we learned from it. So every time there's a saying that if you did not win, you learned something new. So here, you can try to practice this on your own later.

Go to the OAR website, log in. Look at the last submitted goal. This was my last submitted goal. Focus on the key action items to assess completion or not. Plan new goals with your team. So let's go to the next slide, and I'll show you what. So that same CASAS data portal, I just put a clip from there, the last three years' goal. Four years, in fact.

And then that kind of gives me whether our goal was to improve the ASE low educational functioning level completion rate from 62 to 63. So I'm looking at, if you look at the call ASE low, that is-- Jaime, can you point that thing to the ASE low. So there. The state goal is 40%, state average is 41%. At Corona-Norco in 1920-- that was before the pandemic or the year of the pandemic-- it was 48%.

And then the year after the pandemic, it was 28%. And then the year after 21-22, it was 58%. And then it now, it is at 73%. So this 62 is what we got by doing the average of the past three years. And then we wanted to get 66%. So now we are at 73% So did we meet our goal? Yes, we're happy. So we're going to move on.

Jaemi Naish: I'm going to say kudos to you on that. That is amazing. That is awesome. That's really impressive.

Thoibi Rublaitus: Thank you. That was setting goals. And sometimes, we don't meet. Let's go to the next slide. And you will notice, same thing, the persister rate will be maintained at 40% is what we said. So if you're looking at the same level-- we focused only on one level, ASE low. And we said we will maintain that 40%. We are at 48% now if you look at the bottom row there. 48%.

So that means, do we meet our goal? Yes, we did. So we're very happy. But then let's go to the next slide. Remember, in our goal at the very last sentence we had, the ASE team aims to increase the number of students who graduate to more than 100. 100 number this time, not percent. Now, looking at our payment point summary, you will notice that the previous year we had 89 and 42 was 131.

But the following year, we were overshooting, thinking that the previous year we had 131 graduates. Now we can make 100. But where did we land? Only 78 total, 46 and 32. So in some sense, if we looked at our smart goal, we met our goal. But by adding that one extra sentence, now we missed our point, right? But it doesn't matter.

If you were in my school's place, will you consider moving on from this goal and working on something or would you repeat this goal? Just take a moment to think of it. But in our case, our team has decided we basically met our goal. Being short in the graduates or HSE completion is not a big deal because as the economy improves, the number of students that come back to finish school with us are lower.

But we can still keep this as a side goal for this year to say, did we set a realistic goal to get to 100 this next cycle or not? So that's what our team is discussing. So with that, I think we can go to my next slide. I think my time is up too, so I will just say, let's all work smarter. Our team learned better by looking at the rubrics first.

So earlier, Jaime shared with us that if you downloaded the worksheet that comes on the page-- and it comes with very good worksheets. If you use those worksheets, look at the rubric. And as you are making your action items, follow the rubric. If you meet those rubric, you're good to go. So that's what I wanted to give my last word of advice before I move to my next page.

And so this is just some-- avoid this and remember this, which we have already talked about. Feel free to look at it when we send the slides over to you. And then the next slide is the end of my section. Yes, this is-- over to Penny.

Penny Pearson: So I have a really quick poll. I'd like everybody to just take a moment and vote because we're going to have a little short discussion because we don't have too terribly much time yet of talking about AI and what impact it might have on this process. And there's not a right answer per se. It's just, we're kind of getting a feel for, well, what would this be like in this process? So if you take--

Looks like we got almost half of us. If the other half would vote as well. This is anonymous. We don't know who's making the answers, so please vote with your heart. That's what we need. So we've got just 29 out of the 46 of you. So if you can just take a second look at the screen and read the question, and we'll see how we all kind of stack up against each other. And we'll see how that looks. And then I've got some other questions and things for you.

Jaemi Naish: Penny, you can let me know if you want me to continue just sharing or advancing the slides or do you want to take over and share your screen? Either way is fine with me.

Penny Pearson: Yeah, I will in just a minute. Once the poll is done, then I'll go ahead and take the share to share my screen. OK. It looks like everybody stopped voting. So I'm going to end the poll, and I'm going to share the results. So let's see what we got here. So it looks like we've got positives for most of the folks here. And there's a few that are uncertain or just they're not there yet, which is perfectly reasonable.

Actually, I was expecting a few more of the uncertains and nope, no way. Because it takes us a while sometimes to really get our heads wrapped around some of this new technology. So I'm going to go ahead and stop sharing that result and close this off. So I get an idea of what that reaction is. And let's go ahead and move to the next slide, please, because I want to show you-- we already just did that.

And we're going to go ahead and I'm going to stop the share. Jaime. We jumped one ahead because I want to share with you what I've done. And that is to go out and see about using ChatGPT or some type of generative AI to help you determine how effective or how well is your SMART goal written now. This is not the end all, be all. And I know that I have participants within this group that use generative AI for a lot of different things.

And they might have some great ideas they're willing to share about how it might be useful. But this is the prompt. And with ChatGPT down at the bottom of the screen here, I would write my prompt or my question. And in this case, this was my prompt here. And you can read it for yourself. It's pretty basic, not much. But I'm asking in the prompt to analyze that statement against the criteria of a SMART goal.

Now, what ChatGPT came back with it said, well, it's not a fully SMART goal as it lacks these things. Remember, we're trying to hit all of those letters in the acronym to make it SMART. And then it gives us the SMART recommendations of, it should be measurable, it should have be realistic and attainable. So it gives you some ideas. And then it will actually give you a suggested revision.

Now, as with all things AI, I would definitely be working with the team here. This is not a one person deciding that this is the all-in type of working with AI because we've all heard the stories. It hallucinates and it gives you bad info. So it really is just meant to provide you some assistance. So I did another one.

So this was the same kind of prompt. If it meets a criteria-- and this was my SMART goal. And you can read it and you may have a reaction initially, it's like, oh, no, yes, no, maybe. And then let ChatGPT come back and say, well, it's partially smart. Now, remember, before, it said, well, it didn't meet these criteria. So this can be very interesting. You're literally-- I see it in the chat-- you're having a conversation with this AI.

And for some people, that's really cool. And other people, it's a little creepy. And I get it. But what I did like about what ChatGPT did is it aligned itself with our acronym. It's making sure that what you're putting into that statement is actually meeting those acronym letters. Now, it doesn't differentiate between this is really great versus this is not so great.

So when I looked at this revision, now, this revision is what it wrote of that first statement. I like that, Maria. We called ChatGPT our unpaid intern. I love it. Yeah, that's nice to have an unpaid intern, too, right? So in this case, again, working with your team. This is not about just one person doing this. It's working with the team. So there were other ones that I had done here.

So I had something that I thought, OK, this is a SMART goal. And sure enough, yeah, it says it's a SMART goal. And it broke it down for me and I go, OK, OK, this is a SMART goal. But could it be strengthened? Could it be better? So I fiddled around with a little bit and I created more of a longer SMART goal statement. And what I liked is that what ChatGPT told me is that it provided additional clarity and specificity, making it even more effective. OK, that's a good thing for my program area.

So this is something that as a team, I encourage you to go in and play with it a little bit. Look at it. But don't do it individually because I think, honestly, the team can make all of the difference in ensuring that the SMART goal is really reflecting your program goals for what you're trying to do in this next round of the Continuous Improvement Plan.

Now, ChatGPT is not the only one out there. For those of you that have used ChatGPT, maybe you've used something else that you like even better. I did the same experiment with Microsoft's Copilot and-- OK, now it got rid of my stuff here. Where did it go? I had it on here and it-- maybe it's in my notebook.

Thoibi Rublaitus: I saw your Copilot just now.

Penny Pearson: Yeah, but-- OK. Wait. OK, it's over here. Let me see if I can get this to come up on this side without all of the-- OK. I don't want to copy it. I just want it to go full screen. And I'm just started using this one just a few days ago, so I'm not quite there yet. But I did the same thing. And I think that because of the way our district is, it didn't show the original prompt. And that's just because of what SCOE does.

But you can see that it's doing similar things. It's going through the acronyms. But it doesn't give me that potential of a better SMART goal. Let me see what happens when I hit Let's Chat. OK. Let's see what we've got here. And yep, I have lost my-- I didn't copy and paste it. So it looks like it's only got-- wait, let me scroll. Sorry for the scroll, guys.

Yeah, it's not showing actual prompt that I did because it's protecting me, which is very nice because I'm broadcasting this to the planet. But it does give you, you know, some good reference points, and it does give you additional prompts at the end. So again, it's something that you can experiment with. I tend to lean toward ChatGPT because that's what I've been working with the longest.

But as I've been seeing more and more of these generative AI tools come out, there may be other ones out there that you and your team would find even more effective for what you're trying to do. So I'm going to go back to the screen here. So hold on just a second. I'm going to stop my share. And I'm going to bring up-- because we've only had a few minutes because I want to leave at least 15 minutes for us to have a chat.

But given what you just saw, I'm going to pull up another poll, I hope. Hold on. Wait not that one. I want this one. And let's see if you think now that you could or you'd be willing to use AI in the development and refining of your team's SMART goals. And then I asked an open-ended question that saying, well, what helped you to reach that answer? Whether it's yes or no that you might try using AI.

And this is important. It's a personal thing for me as well because I'm kind of leaning on the pessimistic side of AI, iRobot, that kind of thing. And I'm just real curious of how people look at this to make decisions on what they're going to use or not use in their work, as well as in their personal life. I mean, I tend to use AI more personally because I'm researching or I'm looking for answers. I haven't really implemented a whole lot in my work life.

So if it's something you're interested in learning more about, I know that-- I know York is on this webinar today. He's done some wonderful sessions for OTAN about using AI. And if it's something that you would want more of, then let OTAN know because we can definitely find the folks that are really proactive in using it and getting you more information and more practice, so to speak.

Maria, what did you say? I missed the chat here. Yeah. Thank you, Thoibi. Yeah. And Maria, you can run it through another AI system. Yes, if you have--

Maria: I was asking Lori Anne which one you could upload Excel because I didn't realize you could upload Excel.

Lori Anne: Yeah. So from the ChatGPT, depending on which version you have. But the WIOA Title II tables, if you download them as an Excel spreadsheet, it will actually analyze the data for you and give you your strengths and all that and tell you which areas to look at.

Maria: That is really smart.

Lori Anne: Yeah, we just played around with it with my team this past week.

Maria: Yeah. Thank you for sharing. I knew it could do PDFs and photos, but I didn't know about that. That's really smart. Thank you for sharing.

Lori Anne: You're welcome.

Jaemi Naish: Wow, that is--

Penny Pearson: Thank you, Lori Anne. That's awesome.

Jaemi Naish: That is really good to hear. And you are some of the brave people getting it going and seeing what's possible and then sharing it out with us. That's amazing.

Penny Pearson: That is awesome. All right. I'm going to end this poll. Thank you very much for all of you that answered. I appreciate it. And again, this is all anonymous. So I can share the results, but I'm not going to because I asked for some specific stuff unless you all really want to see it. I mean, basically, we got 27 yeses and one no and two I don't nos. So I think we got some good results of sharing and showing how ChatGPT can be helpful. So I'll go ahead and close it for now.