[audio logo]

Announcer: OTAN, Outreach and Technical Assistance Network.

Amukela Gwebu: What we're going to talk about right now is what I call the brass tacks of the program. This is where the rubber meets the road. Because it has to do with your deliverables, it has to do with your continued funding.

And my goal in this presentation is to make it-- Oh, I mean, to make this process regardless of what you had thus far as easy as possible so that by next week you know exactly what you're on hold for and there is no confusion about that. So at the end of this presentation, you should be very clear about certain things.

So what we're going to talk about. We're going to talk about data and accountability. It entails both academic performance data and it also entails some things that are related to the fiscal part of our program.

So every year, at the beginning of the year, we come out with the Beginning-of-the-Year letter. It stipulates what you're supposed to do during that fiscal year. I'll share that with you. And then we also provide you with a calendar of deliverables. And then within our program, we have a program-- I mean, we have agencies that have grouped together and applied as coalitions. And so that's what 1.3 refers to.

And then as an administrator, there are some things that you have to address in 1.4 that relate to data collection, training, and reporting. I mean, you have to know these things and make sure that your staff does them. And on the monitoring part, it'll be your work. But yeah, these are the sort of things that relate to 1.4 and you should be clear when we're done here.

1.5 is about your local assessment policy. Jay talked a lot about it. The only thing I'd add to Jay's conversation is that he forgot to mention that we added a new line as the CDE to the local assessment policy. And what we want you to do is tell us when you reviewed it with your staff.

And that should happen every year, because that's a working document or living document. It kind of guides your work as it pertains to your assessment, and if your assessments are wrong it's going to affect your future funding and so on. So you need to stay on top of that. And then we'll talk about the End-of-Year data submission and how that all works.

Item 2 in our objectives is data privacy and personally identifiable information. We're going to talk about Social Security numbers. We're going to talk about individual tax identification number. We're going to talk about consent forms, both the paper-based and the electronic version. I'm sorry.

And then finally on 3, we just talk about data monitoring. Like I said, you know what you do in the interim as an administrator in those intervals between your calendar of due dates, which are due usually quarterly.

OK, so let's begin. This is a web page. I mean, this is a slide showing you the Beginning-of-the-Year letter. On our website it's titled program and accountability requirements. I'm hoping in the future we'll change that and make it the Beginning-of-the-Year letter, because I think that makes a straight line of where all this begins.

And then as a housekeeping type of thing, if you see on the slide any of these items underlined, it means that you can click there and go to the web page, and then you can visit this. So this is obviously a sample that-- I mean, the picture. And you can see where it begins. It talks about how to manage your local administrative cost. Arturo discussed that in depth. And then it talks about monthly attendance requirement in the next paragraph, and so on, all right?

And if you want to look at it, I mean, I'm not going to keep clicking these links, but-- Oh it's taking its time. This is what it looks like. And it just tells you all the changes we've made like that are pertinent to that particular fiscal year.

Because this thing changes from year to year, right? Like, I just talked about the assessment policy. There's a new guideline for this fiscal year regarding the administrator formulating the local assessment policy with their staff. We want that to happen. So this part has changed. And I've made you aware of it.

There's memorandums that have been discussed by Neo and Catherine Peacock. So it covers all the things, all the changes that you're on the hook for. And these things are legal requirements, fiscal requirements, and yeah. I mean, just follow them. That's all I can say. I'm going back to my slide.

Back in my slide, I've shown you how the slides are going to go moving forward. There's illustrations of what the web page looks like. And this one covers the beginning-of-the-year letter or program and accountability requirements. And then we're going to move fast now.

And then we also provide you with a calendar of grant deliverable due dates. Like I said, I'm going to make this simple. When I first came to the CDE and was working with my 13 or 14-- I mean, no, 23 esteemed colleagues, this is what they told me I must do. They told me, Amukela, you need to get the calendar of deliverable due dates. That's what you as a consultant and a CDE are on the hook for. And you need to help agencies do this.

And then if you want to go deeper and deeper into the WIOA Title II program and learn more, just like this venue or this conference, what it is providing, you can go deeper. But by next week, you should know that this is what I'm supposed to address. So there are some things that happened that are required by September the 1st, by September the 15th regardless of--

I know I'm addressing a crowd of experienced and some people that are not experienced, these are the things that you do for, and these things keep you in good standing with the CDE. Like David said, how do you identify programs that are performing well. This is one of the criteria. I mean, if these things are not coming on time, it makes-- I mean, it just peaks our ears, and we'll see where it goes.

OK, so keep that post it on some board somewhere. And as you're running the program, learning more about these intimate details about the program, keep that calendar there. Within that calendar, however, now I'm on the next slide, we have agencies that group together and applied for the grant as a coalition. And so some of those deliverables have to be submitted by the individual agencies, and some of them have to be submitted by the coalition. And we provide this table that you see in front of you to demarcate between those lines.

So the physical agent, they can see their responsibility in the second column of that table. And the individual agency, you can see their responsibilities demarcated in that table. Next. Well, later on since I'm still in the overview, we're going to go over data collection, training, and reporting. And this is stuff that is separate from the calendar.

And some of the things that are pertinent there is that you need to have your agency complete two trainings, OK? Two trainings. Somebody during this next fiscal year from July 1, 2023-- No, no, yeah, June 1 all the way to July-- I mean, to-- No, to June-- I'm sorry, July 1 to June 30, you should have somebody's always present that has taken the accountability training and the implementation training all the time, and you should be able to provide CDE, especially when the people go on FPM records.

If we call you because the data is not coming in correctly, I mean, this is one of the things we look at to see if we have someone that's competent to handle TE and understands what we're talking about right now in terms of what are the deliverables. And the CDE through CASAS offers you all these manuals, technical assistance, and please take advantage of it.

Next thing. Oh I'm sorry I skipped. I'm sorry I skipped the-- OK, the next topic I'm going to talk about is the California Assessment Policy. If you go to this link, you will find a document that mentions all the tests that have been approved by the feds for us to test our students for the next fiscal year.

As Jay mentioned earlier, some of you may be continuing with the old test, but there's a grace period regarding that. And so after a while, those tests are going to expire. And then some of you have jumped on to the new test. And so this document is a guide for you as an administrator to decide what sort of test you want to do, what sort of protocol, and so on.

And this is a pass through item, I need to stress that. When I say it's a pass through item, what I'm trying to say is that the CDE cannot decide which tests we should use for this particular program. They have to be approved for reliability and validity through the NRS, and then they can be administered throughout the country. And California is working with CASAS to do that. And so look at this document very carefully. And it will inform you on what is valid for this fiscal year.

And then so we pushed that down to you. That's why I said it's a pass through type of deal. We ask you to make a local assessment policy, to guide your agency, and to guide your staff. And everyone who's in your data team should be aware of what the local assessment policy, the content. We provide a template and you need to fill out the content.

Things like, how does your orientation work, who does it, what are your testing intervals, when are you going to pre-test, when are you going to post-test. Those things need to be expressed in your local assessment policy. And you need to share it with your staff. That's why I said it's a working document earlier in the presentation. You need to share it with your staff.

And then if you do remote testing, there's documents that you have that help you authorize-- I mean, get authorized with that, with CASAS. Because there are some rules you know about being a proctor and all that you need to educate your assessment people on how to do the virtual testing, OK? Our next slide.

And then since a lot of you new administrators maybe you haven't participated in this exercise of submitting end-of-year data, four [audio out] five months, usually four months before the end of the fiscal year, before June 30 of next year, we're going to send you some documents on how to submit your end-of-year data.

And as you can see, like Jay mentioned earlier, end-of-year data is due before July 15. And if you submit your data before July 15, submit what you have, OK? Submit what you have. I mean, do your best to clean it up, but submit what you have.

Because sometimes we get questions from agencies that sort of postponed the whole thing because they're trying to be perfect. But CASAS can look at that data for you and tell you where the holes are much quicker and help you fix your data. So submit what you have on July 15. You have a grace period where you can correct whatever data you've submitted to us. It's about 15 days. So take advantage of this.

And within that end-of-year data submission notification, you will find documents such as the submission instructions. And these provide you with a step-by-step guide of how to submit your data in TE. I mean, as you can see, all those areas that are in bold within this diagram, I mean, it just tells you the clicks to follow.

And if you have trouble with this, we have a troubleshooting document. And it allows you to look at the data integrity detail, payment point summary audit, data payment point flags, and so on. So if your data or the payment points are not looking like you have seen your agency's performance in the past, these are some of the things that will-- these are the low hanging fruit to find out where the problems could be. Because you want to maintain your payment points and make sure that they're the same level as they always have been because that's what's going to stabilize your funding. And so that's what this relates to, OK?

Next item. We're going to talk about data privacy, securing personally identifiable information. We have a management bulletin that describes how you're supposed to treat Social Security numbers. We are updating it right now to include the individual tax identification number. But what I can say ahead of that even before we approve is that the same guidelines that are working for the Social Security number are exactly the same as those for the item.

There's three requirements basically with this stuff. One, you have to tell the student that it's voluntary. And not that [ INAUDIBLE ] only-- Yeah, so, one, you have to tell the students is voluntary and it's not going to stop you or it's not a barrier for you participating in this program. That's one thing that us as CDE consultants will look at, right? So it is not a barrier. And then the fact that it's voluntary. And then you also tell them what it is being used for, right? Just to track their employment status. And basically that's it.

And so after reading the bulletin and learning more about the data privacy guidelines as indicated in number one, you can begin to formulate your own. And we have a template. And it relates to item number two in this slide, and where it begins voluntary authorization to share personally identifiable records form.

So that's a template that contains all the three things that I mentioned. And you can create your own. And it can be online. You know, when you give the student and they can read it on their own, just like we do with our bank records and all those kind of things. And yeah, I'll finish that-- I mean, with that on that part.

And then we also have translations in the 16 most popular languages. I'm actually proud of this page. So I'm going to share it. Here it is. So I've witten Arabic, Armenian, Cantonese, Farsi, Hmong, Japanese, all those kind of things. And if you guys can make more translations depending on your location, we'd like to see more of them, OK? I'm going to go back to the slide.

And then we have an FAQ. You can dive into that. And yeah, that's all I have to say on data privacy. There are the three requirements. We provide you a template. See item number two here. We have translations to deal with some of the-- I mean, the most common languages that you will encounter, the 16 top. And then we have some questions that have been asked by previous administrators.

And this is management bulletin 1701, just a sample. This is the voluntary authorization to share identifiable. This is a template. And it has a lot of words. Yours does not need to have all these words. As long as somehow you find a way to put the three requirements that I mentioned, you're good, whether in electronic or paper form.

Here's the place where we keep the translations. And if you click that underlined area, as I mentioned in housekeeping, it will get you to that site. And then here are the FAQs. And these FAQs also relate to the item. So we are going to update our documents because the item is a new development. And essentially what we're going to be doing really is just putting item next to SSN, because the rules are the same.

And then in terms of action-- I mean, next, we're moving on to data monitoring, reporting. And I'm going to be talking about a lot of action items here. So you as the administrator right now you should have dealt with the program entry and update record. Because you have some students that you've enrolled, I imagine that you have tested them to determine what level they are at and you've inputted that data. That is the best practice and that should be an action item for you.

And then as I'm discussing this, I also wanted to let you know that for this program, you have to enter attendance monthly in that entry and update record. Because that's how we track exit. And we mentioned this attendance monthly as a requirement in our assurances.

And so everyone signed up and said, Yeah, we can do this. And we really need this to happen, because we're trying to increase our employment-- I mean, improve our tracking of employment outcomes. And if we don't do this, it's very hard to be successful on the other end.

And we provide a wizard, like Jay described in the morning, that allows you to send texts and emails to conduct the surveys. I really recommend that you take advantage of that and you make sure that you do that as prescribed in our calendar and deliverables.

OK, so that's one. These are action items, things that you have to do. You can't get around them. If you don't do them, you're going to be getting a call from the CASAS program specialist or just CASAS Technical Support. And then if it doesn't happen, you're going to be getting an email from the CDE consultant. And if some of these things are not completed, it's very hard for-- I mean, well, it will delay your payment. Not that it's very hard, it will delay your payment. We need these done before we process the payment.

And on that a survey, like Jay said, it's not necessary that-- I mean, it would be fantastic to get the responses. That's what we're shooting for. But when I say full responses, I'm saying the student entering all that information as we require. All we're looking for here is response rate, did they say something in return. So don't be afraid to send the emails and the text actually to improve your numbers sometimes. Another--

Jim Shields: Amuketa, I'm sorry to interrupt. This is Jim. We have some questions. Would you like to take those all at the end, or what's your preference?

Amukela Gwebu: We can go right now.

Jim Shields: OK. Maybe when you move to a new section each time, just check in with me and see if there's any questions. But we'll go over the ones we have. Just a moment, please. OK, our first one, if we have all of our students enrolled in ESL and HSE classes fill out the CC apply admissions application, does that count as the Social Security release form? The application included the consent to release information statement.

Amukela Gwebu: Yeah.

Jim Shields: OK.

Amukela Gwebu: Because I mean-- OK, so I'm saying yeah because I imagine that the community college when they design that CCC Apply electronically, they have a consent form, right?

Speaker 1: Yeah, that's part of the application. I asked that question. So I work for a community college. And the students already fill out a lot of forms. So I don't want to duplicate adding an addition to the Social Security number release form if the CCC Apply already counts, because it has the release of information, the consent to release information.

So most of the students mark that they're willing to share their information. But there's some that do not want to share, and we pull that information from our student information system. So I just want to make sure. Because right now we're going to be monitored. So I'm submitting all the documents, and I want to make sure that we comply.

Amukela Gwebu: Yeah, so all you need to do if someone was conducting the FPM and monitoring your agency, or even if the feds came and asked you about your data security policies, you just show them the portal where the students enter. And this is the language that's there. And then that's it, you're done.

Speaker 1: OK, thank you.

Amukela Gwebu: That's perfect.

Jim Shields: Great, and we have just a couple more. Actually Janice just provided a link from CASAS about student data security if you want some additional information. You can find that link in the Q&A. And then our final question, do we need to keep the Social Security number release form for record keeping, and if yes, for how long?

Amukela Gwebu: I mean, it's-- That's a tough one. I think you should destroy it after five years. Because we already have that SSN in our database. And we also have to destroy it too. Because after we've done our match with the EDD, it's very confusing what's going on in that space, because the feds want us to do longitudinal studies, and the longitudinal studies take more than five years. They're confused, we're confused.

But I'll say, like if you listen to I don't know if Arturo mentioned like in the fiscal part like how long you're supposed to keep certain documents before you destroy them. I think it-- Arturo, can you help me on the documents that we're supposed-- I mean, when we're supposed to delete them if it's present.

Arturo: Yeah, I am here. For the federal regulation it's three years. But the CDE recommends five because the CDE or the feds have authority to go back five years for the federal part. I'm sorry, I didn't hear the specifics, but like for example, for personal information, transcripts and all those, those are permanent. You keep those as long as you-- forever. And now you can keep those electronically.

But it will depend whether the documents are federal or just the state that are under the education code. So it will depend on the specific, but again, in general, federal it's three years, says the regulation, but we recommend five because the GEPA requires or I guess authorizes federal agencies to go back five years and look for if there was any misspelling [ INAUDIBLE ] or something like that.

Amukela Gwebu: Thank you, Arturo. So I think we should go with five years, right? Because we're going to be reviewed by the feds, right?

Arturo: Yes.

Amukela Gwebu: It's going to come to CDE and ask about agency documents. And that's what they do. Sometimes they take a sample of the agencies to see if us as CDE are doing our job properly. So in order for us to answer those questions properly, we need--

Arturo: Yeah, I agree. I agree. I just mentioned the extra step because sometimes we get questions about specific transcripts or records, the personal records of students, personal records. So those are different than these ones for data accountability and those that are required for the federal-- for the AEFLA grant, which I agree five years is the recommended time.

Abby Medina: Amukela and Arturo, I posted a link on our records retention bulletin in the chat just for everyone's reference.

Arturo: Perfect. Thank you, Abby.

Abby Medina: We have a management bulletin on it.

Amukela Gwebu: OK, thank you.

Jim Shields: And we had one more question that came in. Do we need to complete the SIP every year.

Amukela Gwebu: So yes. I mean, well, in the years that we are doing an RFA, the CDE recently decided that in those years we don't do the SIP. Because we don't know who the successful candidates are going to be. And so it's a continuous improvement plan. It goes over one year. So in the years that we do the RFA, your application, you may not have to do the SIP.

And that's more of a Carolyn type question. But that's how I'm thinking about it, right? Is that, yeah. So the next three years, you have to do a SIP, right? And then that fourth year when we go to the RFA application, you may not-- Yeah, it's very unlikely that you'll have to do a SIP, based on our logic right now. Does that help? I hope I answered the question.

Jim Shields: Yeah. So it looks like we-- OK, we just had another one roll in. For students that re-enrolled in the current year, do they need a new signed form, especially for students enrolling towards the end of the previous year?

Amukela Gwebu: Help me understand what's a signed form?

Jim Shields: They perhaps are-- Laura, could you clarify, are you talking about the Social Security release form, or? If you feel free, go off mute.

Laura: Hi there.

Jim Shields: Hi.

Laura: Yes, I am referring to that form. And what I'm talking about is for those students that are enrolled, let's say, towards around May. They finish the year in June. They come back and enroll in the program in July. So do we need to have one form for each program year?

Amukela Gwebu: No, no. No, you don't--

Laura: What is--

Amukela Gwebu: You don't need to do that.

[interposing voices]

Amukela Gwebu: It's not like you can avoid the fall. You remember when we were doing the community college example, if the student took off and then they come back and CCC Apply has that default page where you have to read it, you know what I mean?

Laura: So part of the guidelines

Amukela Gwebu: Yeah. Because well, yeah, you do have to do it again, sorry. If you're doing paper-based, you do have to do the form again.

Laura: So every year?

Amukela Gwebu: Yeah. If they take off. And then Jay talked about-- well, Jay and then Carolyn have been harping to you guys about periods of participation, right? You've heard that [ INAUDIBLE ], right?

Laura: Yes, right. But what I'm saying, if there's a student [audio out] the fiscal year is ending 30. So if they enroll in May, a fiscal year ends in 30, so they're going to come back as of July 1 for the new year.

Amukela Gwebu: We have to do it again, Laura.

Laura: OK.

Amukela Gwebu: Yeah.

Laura: Thank you.

Jim Shields: OK.

Amukela Gwebu: Thank you.

Jim Shields: Yeah. Then we need to have one other one. A person's asking, does that mean no SIP for '23/'24, but a SIP will be required for this program, yeah, right, Amukela?

Amukela Gwebu: Yeah, so they said no SIP for '23/'24?

Jim Shields: Yeah, but that just applied to last year because we were doing the RFA. But for this year, it is going to be required, right? That's my understanding.

Amukela Gwebu: Yeah, so the last fiscal year it was not required. And then the current fiscal year. I'm a little bit confused by the question because they're saying '23/'24?

Jim Shields: Yes.

Amukela Gwebu: And it's not required?

Jim Shields: Well, she's asking does that mean is no SIP required for '23/'24.

Amukela Gwebu: In April?

Jim Shields: No, it will be due. Yeah, in 2024, right?

Amukela Gwebu: Yeah, in April.

Jim Shields: OK.

Amukela Gwebu: In this coming April--

Jim Shields: OK.

Amukela Gwebu: --we have a SIP due.

Jim Shields: OK. OK, she wants to clarify her question. So Lynn, go ahead and go off mute, please.

Lynn: Thank you. That's exactly right. I know that a SIP is due on in April 2024. I meant we don't have one to continue on because we finished ours for '22/'23 that we had written the previous year. So we don't have one that we're working on right now, is that correct? What we're working on now is creating a new SIP that will be due on April 20, in 2024. Am I saying that right?

Amukela Gwebu: Yeah.

Lynn: Is that correct?

Amukela Gwebu: That's correct. And I don't think you have to formulate something like a radical change. I mean, I don't know your situation, but if you are a continuing agency, I mean, your stuff has been continuous even though we said, we don't want you to submit something. Does that make sense?

Lynn: I think so. We are a continuing agency. And so we're going to be working on creating a new-ish SIP, right? We'll re-examine our goals and go forward.

Amukela Gwebu: Yeah.

Lynn: I think I got it.

Amukela Gwebu: Yeah, we were trying to deal with agencies that are not continuing.

Lynn: OK.

Amukela Gwebu: I mean, that's why this policy is there. That's what I'm trying to say. It's like why have them submit when they're not going to continue.

Lynn: Got it. Thank you.

Amukela Gwebu: Thank you. So it should inform your future, that's all I'm trying to contribute. OK, and then I talked about the two trainings that are required. This is another action item as an administrator. It's just one California Accountability Training Course, and then one implementation course, which is item number two, how to use TE basically, that's what that is. And Jay gave you a little bit of a taste of that this morning.

Next is the DIR. So I gave you a calendar of deliverables. I gave you some action items. I shared with you all this stuff. But as an administrator, you also need to think about the things that you do in the interim period. I mean, the simple things, right? Because these simple things, like looking at the DIR, the data integrity report in the interim, can tell you that you're heading into problems before the quarter is over.

If you look at the DIR and there is no pre-test after three weeks when the students have been there, that's a problem for you as the administrator. Because that's going to affect your performance. Because we are all educators in this conference, and we know that the most learning happens at the very beginning.

Like, at the very beginning, the learning graph is like this. And then later on when the student becomes more proficient, it becomes harder and harder to extract performance out of them. And so if you miss that first part, it means that maybe you've hurt your agency in terms of getting payment points.

Because that's when the student could have grown the most and your agency would have been rewarded for your effort teaching that student and intervening with the ABE, ESL and getting them to the next level. I mean, this is a very, very important point for all your administrators to understand.

And if you delay in testing your students, sometimes you can delay too much that is too late to do a post-test. We've had instances where sometimes you find a program hasn't done a pre-test in January, in the spring. Sometimes even late spring. And we at CDE and CASAS are kind of scratching our head. Because once it gets to that point, there's very little that we can do, and it's going to hurt your data, and all the work that you put into that student is not going to show.

So in the interim, that's why the administrator needs to look at the data integrity report. And so I'm going to share some things with you on the data integrity report.

OK, so if you go to the CASAS site-- Yeah, this is the DIR. If you go to the CASAS website, you click on their Documents that pertain to data collection, there are some PDFs there that have the data dictionary, those-- The data dictionary is above on the table. It's more in the overview of the section. And then CASAS is the table. And they have all these documents that you can look at that are PDFs. And one of them is called the DIR. And this is our latest one.

And so with the DIR, if you look items 1 through 12, all these things related to gender, no ethnicity in your data, missing birth date, you can do a pre-test and post-test with that student, and if you're missing any of those things, your work will not count. That's what I'm trying to stress with this, OK?

And then just to show you what-- I mean, this is almost like a template of the DIR. What your agency DIR will most likely look, something like this. OK, so I create-- I mean, I played around with the CASAS thing, I called it No Administrator School, because I cannot share the agency where I got this from, so I was just hiding the data.

But you can see they have 128 missing-- I mean, less than 12 hours of instruction. You as the administrator in the interim need to discuss that with your staff, hey, what's going on with our orientation, right? Zero or? These are the types of things you need to address in the interim so that you have something to submit that represents the way you want your agency to be working and compensated after the program here, right?

So this is a working document. Like I said, I want to keep this simple. We gave you the beginning-of-the-year letter. Gives you a calendar of the things that you need to do throughout the year in your first year. Very simple. Right, you talk to your data coordinator, you create a program for that.

But then in the spaces in between to save your energy as an administrator, because all these things are happening, this is the report from CASAS, the DIR that you should be looking at. I mean, it'll make your life easier. And then later on, as you become more proficient in the program, you can go deep in the weeds and start going in depth, or if you have questions about some of those things in the calendar, some of these things in the data integrity report, OK? Jim, do we have questions?

Jim Shields: There are currently no questions.

Amukela Gwebu: OK. I'll keep moving then. And I'm at the end of my presentation. And I mean, initially I wanted to have questions at the end, but it looked like it was working better during the presentation. So in terms of Technical Assistance resources, I mean, Jay shared with you the site. And here are a few links that may help you.

Anything to do with California Accountability, if you have questions, here is the email address in item number 1. Shipping and payment questions. That's where you get your orders for electronic and paper assessments addressed with item labeled number 2.

And then on a short term basis, if you and your data staff are confused, item number 3 on the slide tells you where to get this, the tech support. And sometimes CASAS will come to your site to resolve these things.

In fact, most of the time if you guys are struggling and it's just not happening, do not hesitate to contact CASAS and ask them to come to your site. Because our program specialists are located in different geographical region. Unlike us, the consultants. We're here in Sacramento. And then it's their software. So they know how to navigate it better than us.

And then if you cannot resolve your problem within that space, then you consult with your CDE Regional consultant.