Will Neddersen: Hello, everyone. We're glad you're here with us today. Very quickly we're going to introduce ourselves. And I would like to start with my colleague, Karima, if you could introduce yourself.

Karima Feldhus: Good afternoon, everyone. Thanks for joining us. I'm Karima Feldhus, and I'm the Executive Dean of Extended Learning at Saddleback College and I'm also the Co-Chair of the SOCRC.

Will Neddersen: Nancy, would you please introduce yourself?

Nancy Miller: Hi, I'm Nancy Miller. I am the consultant with the South Orange County Regional Consortium. And prior to retirement, I was the Director of Sonoma County Adult Education and SRJC Adult Education.

Will Neddersen: And then I am Will Neddersen. I am the coordinator for Tustin Adult School and Tustin Unified School District and also a Co-Chair with Karima for the South Orange County Regional Consortium District, or if we say it, SOCRC.

So we are, again, glad you're here. Forgive me, I'm just reminding housekeeping one more time. We will be checking in with the chat. So if you'll use that for questions and hopefully getting to hear from you or ask you some questions towards the end. So please, again, be ready for that.

And with that, our focus for today-- again, this is part 2, so our focus for this part 2 is on SOCRC's member effectiveness journey. We're going to be honest with you. It's been a journey, not something quick and easy, but good work to be put in there. So you'll hear more of that.

Then we're going to share with you some tools to monitor member effectiveness that we've created for our consortium. And then last but certainly not least, we're going to talk about incorporating member effectiveness into bylaws, which really is a key and important piece to understand is you need that to be a part of your bylaws.

And the other part of this is I don't know about you, but when I come to presentations, I love receiving things to be able to use as tools. And that's our goal today is to share with you some tools.

So you will be seeing links shared in the chat as well as a QR code at the end if you don't want to click on links because it gets a little bit confusing. But we're going to share with you in the sequential order. You'll see it 1 through 6. The folder that's being shared will have documents numbered 1 through 7, 7 being this presentation.

But we will share these items with you as we get to them, discuss them. And know that they're in a Word form or a Google Doc form that you can then take, alter if you would like to use. Again, totally optional, but we'll give you that insight. With that, I am going to hand this over to Karima.

Karima Feldhus: Well, thank you, Will. Those of you who joined us last week, we covered this and we thought we'll repeat this information in case we have new members joining us today.

So quickly, the South Orange Regional Consortium consists of 10 voting members, six of whom are funded members and 4 are non-funded members. As part of AB86-- and you have the list. Just for the sake of time, I'm not going to read the list of all 10 members.

But as part of AB86 or AB86 planning process, Saddleback Valley Unified and Capistrano Valley Unified agreed to turn over their adult schools to Saddleback College.

So as a result, Saddleback College has a little over 50% of the funding for the consortium and that's because it took over the two K through 12 feeder schools funding that came with this adult schools. With that, I'll turn it over to Will to discuss the specifics of the funding.

Will Neddersen: So funding and connecting member effectiveness, the two important pieces, right? We need the funds to be able to implement programs, but we also have to understand if the funds are being used effectively. And so that's where our consortium really started in trying to discuss member effectiveness was looking at a vote about funding in those pieces.

So we originally were a consortium that was a fiscal agency where our group agreed to put the money into one area with our community college district and then have the members funded from them.

In our journey of trying to receive funds and be effective with our funds when we receive them, we went through a journey of changing from a fiscal agency to direct funded agencies.

So know that that change in us brought out a component of the money that was set aside for our fiscal agency, the community college district, to manage and operate what we were going to do with those funds.

We as a consortium do not have a director position. That's why you hear us talk about co-chairs. So how do we manage those funds? And the conversation in that funding was how do we redistribute those funds or our one primary agency that has a little bit of extra funds added to them to help with some of the management components, as well as working with the co-chairs.

With that, also brought the understanding of, hey, there's a pot of funds that are a little extra. And what are we going to do with those funds, and how will we collaboratively share those funds?

And that was something that we've created in process that we'll talk a little bit more about. And then advocating for our funds is really an important piece of dialogues that come into our general sessions and work together. So with that, I'm going to hand this over to Karima to talk about member effectiveness.

Karima Feldhus: Well, thank you, Will. We thought it would be good for us to review quickly parts of our 2019 SOCRC bylaws member effectiveness before we share with you our newest bylaws from 2024.

So we have a screenshot of this language here because with that to point out some of the even negative language, right? I'm not going to read this for you. You have the language in front of you, but I'd like to draw your attention to the word "ineffectiveness" at the bottom, the last two lines.

So it was a little vague and it was negative. So that's what we had in 2019, and you see the big difference where we are. Is it five years or 4 and 1/2 years later? With that, I would like to invite Nancy to dive deep into what we're doing now and what we've done the last several years.

Nancy Miller: Thank you, Karima. Because we knew that we wanted to develop the tool that was outlined in the 2019 bylaws, again, we went to the use of subcommittee so that any member who wanted to participate, whether they were a voting member or a member that regularly attends the general membership meetings, we wanted to have their input on, what does member effectiveness look like? Were the four criteria enough? Did we need more? And then, how are we going to measure each one of those criteria?

We did not do this in general meetings. Subcommittee work probably took 20 hours at least to develop the shells for the tools, and then we went back to polish them up. But this really needed more attention than we could give it in the general meeting. So that's why we used a subcommittee, similarly to the process to develop the bylaws.

We developed three specific tools. We're going to go in order of how we started using the tools, but the three tools that you'll get today along with directions for the quarterly fiscal reporting are the member self-assessment. That was the first one that we actually developed, but the first one we started using was our quarterly fiscal reporting.

And then as part of member self-assessment, we were utilizing the facilities classroom equipment and technology assessment that will show up in a couple of different ways. And we'll discuss all of these tools later.

Again, in linking the tools to the annual and three-year plan, before we even started the development of the member effectiveness tools, we actually came up with a list of metrics that we would be using in each of these tools.

The metrics for evaluation are included in your folder, and I think that they are document 5 or 6, number 5, I think. So you can find and look in detail about the metrics. They consisted of the bylaws, the annual plan, and the three-year plan metrics that were set out by the state but selected by the consortium. So let's see. Karima, it is back to you.

Karima Feldhus: Thank you, Nancy. So the first tool-- this is actually a three-part fiscal reporting template that is filled out quarterly. Funded consortium member present this information at our monthly consortium meetings prior to the consortium certification.

As you know, there is a 30-day window between the member quarterly report due date and the NOVA-- in NOVA, rather, and the consortium certification. And we hold our meetings sometime in between and we make sure that all agencies, funded agencies, present their information before we as a consortium certify the quarterly report.

And we've had actually to schedule a special meeting to accommodate agencies that miss the presentation meeting. It didn't happen overnight. Of course, it happened very gradually. We were more forgiving at the beginning. And now we're at the point where if an agency does not present their data at a meeting, then we as a consortium will not certify it, certify the quarterly report.

So part 1 is self explanatory. Funding members enter their amounts as they submit in NOVA. Part 2 asks about the staffing in place to support adult education programs.

This is work in progress and we continue to refine it. Some agencies actually only reported permanent CAEP positions, for example, and didn't enter all the staffing that support the program. So again, we're refining it.

Part 3 asks about classes offered by programs as well as the enrollments. So the challenge for some of the agencies in filling this out-- and these are six funded agencies. So the challenge in filling this out is that our classes are based on a semester, for example, at a college and not a quarter.

Another challenge is that some agencies use TOPSpro to report the data and others use MIS. At Saddleback College, we use both because we're a WIOA funded school.

And of course, student enrollment numbers and sections allow us as a consortium to have a higher picture of all students served by the consortium, culminating into a discussion about outcomes as we work on the annual and three-year plans.

This information also brings a discussion of how much it may cost per student to run the agency's program. CTE programs tend to be more costly, for example, than language classes, ESL classes.

I'd like also to note that not everyone participated at the beginning. So it took us years to get where we are now. And the change, as I said earlier, didn't happen overnight and certainly AB 1491 helped us in this journey. It took us several years, again, to get where we are. With that, I'd like to invite Will.

Will Neddersen: Karima, I just want to add here as well that as you talk about our journey, some of it was a change in administration in some of our agencies in understanding how we shared out that information, not just uploading into NOVA where we vote on it to say, OK, yes, we approve it, but really being able to process what we approve.

So that really is where part of this journey came. And I think for all of these documents, and this being one of them that started up first for us, was because we were trying to process that sense of effectiveness and finding numbers when we couldn't get numbers from everybody at one point.

So I think that is an important piece to remind ourselves. As you go in that journey, what is it that you want to understand and process together as a consortium? Especially for us, we've put a big emphasis on completing this and sharing it out before we approve our quarterly components in NOVA as well. So it's really an expectation that we've set. So thank you, Karima.

In moving into another component for us, when we looked at the fiscal side of it, you heard me share about changes to the way we were funded from being a fiscal agent to a directly funded agencies.

It brought a conversation about an excess of funds or members who might have had some carryover that had been carryover for a few years. What are we going to do and discuss about looking at those resources, and can we support the consortium as a whole and linking those resources?

And one thing that came up was the understanding of what are we using as a consortium in the sense of facilities, classrooms, the technology and equipment that's being used in classrooms to really truly understand the needs as well as what is being used so that we as agencies could share together some of the technology updates that other agencies had done.

But this also brought out the idea of being able to look at, when talking about needs, we could represent it in an assessment of what we have. So we as a consortium is set up where we complete this every February so that way it gives us a sense as we're looking at funding, hey, if we're reporting, there's needs in classroom or facilities or the need to upgrade technology. This report is already done to be able to look at and reflect together.

Also, we look at it as an opportunity to really say what areas need a component. One of these areas was-- one of our agencies needed classrooms. And so funding was shifted through looking at this report, funding was shifted to them to be able to purchase some portables to place on that and have that conversation.

And then it really helps because again, the funding isn't something that we always have carryover on, but it's that idea of a one time. So that gives us that quick view that we can look at the six agencies. And then we tie it in as part of member effectiveness to hold it with the next tool that we share.

So if you look at it, it's a pretty simple yes/no kind of question/answer. And then it gets into the sense, do you have needs from the above places? So it's just that sense for a self-reflection of what is owned by an agency and possible needs.

So that is the tool that we use for really evaluating needs through facility and equipment that maybe others can support or just allows us if we're needing to advocate as a consortium and looking for future things that's that dialogue as well. So with that, I'm going to pass this over to Nancy.

Nancy Miller: OK, so the self-evaluation tool, that one is overarching. So we use both the quarterly fiscal reporting and the classrooms facilities technology, as Will mentioned, as part of the self-evaluation tool. This one is completed once a year after the submission of quarter 4.

So once you have all your information, everything's in NOVA, all of your fiscal information, everything is in TOPSpro if you're a TOPSpro user or submitted through MIS, so MIS, then this particular tool is filled out and we'll go through the specifics on it.

This year was the first year that we did this self-evaluation tool. So it was a learning experience. We did compile all of the results from each agency to do an overview document that was presented at the meeting, but definitely had feedback from the members.

So what you're seeing is page 1. There are three parts to the self-assessment, again, what is listed in the bylaws. The bylaws is part 1. So it may be a little bit hard to see the description of the metric. I'll read the first one to you.

It's regular attendance and participation at monthly consortium general membership meetings. So was this fully executed, partially executed? Have we had some progress or have we not started this?

And while that might not specifically apply to attendance at a meeting, we did this as a rubrics, something that we're all used to using for assessment. And then each agency would rate themselves on how well they did on this particular metric.

So again, this first section goes through the bylaws. So anything that was in the bylaws as a member effectiveness metric was used here and the agencies rate themselves on that. So we can go on to the next slide, Will.

Part 2 is the annual plan. So again, this information is pulled directly out of the annual plan. This is a document that's going to get updated every year because it does have to match the annual plan, and then the three-year plan metrics when they come out.

We did just update bylaws and pass them in January. So there were some new member effectiveness criteria that were adopted with that particular set of bylaws. So we'll be updating the bylaws section as well.

So this isn't we're going to do it once and then we never have to look at it again until we pull it out once a year to fill it out. This is a living, growing, changing document, specifically related to our three main member effectiveness areas.

Will Neddersen: Nancy, may interrupt you before we move to the next slide?

Nancy Miller: Absolutely.

Will Neddersen: Can I ask you why did we make this an important component as a self-evaluation tool and not somebody ranking that's not a part of that agency?

Nancy Miller: Number 1, when we were looking-- and again, we may hit on this a little bit earlier, but we didn't want this to have a negative connotation. Really, we wanted members to be involved in the process of evaluation.

This is not a punitive document. This is to assess how we are doing as providers of adult education. This also gives members the opportunity, again, comments, explanation, provide feedback.

So as we're doing the annual plan, you can see on the third line there was nothing highlighted. This is how the agency will select whether it was fully executed, partially executed. So they select or highlight their rating of themselves.

The other thing that we were very specific on not wanting is we didn't want to attach a number to it. So you had a numerical grade at the end of this. Like we see here on the third one, if no one filled out that third one, is that a necessary part of our evaluation tool? Should we even have it in our annual plan if no one's doing this?

So those are some of the things that we took into consideration as we were, excuse me, as we were developing and putting in the different levels of participation or completion of the various metrics in this.

Again, for annual plan, as we're going through and writing it, we have a whole section on recruiting and retaining students. Here we are evaluating our classrooms, facilities, and equipment. That was written into our annual plan because we had two agencies that we knew either needed to upgrade facilities or get new facilities.

And so this was an important part. You can't spend money on it if it isn't in your annual plan, in three-year plan. So as we were going through the development of the tools and as we were going through the launch of the tools for self-assessment, we went through this with our agency members. Not well enough, not well enough.

Will Neddersen: Nancy, may I interrupt? I see in the chat the question is, did the members complete the assessment, the self-evaluation assessment during a consortium meeting or in preparation for the meeting? And Karima, thanks for responding, but want to make sure we respond that.

This was done ahead of time, which allowed it to be turned into somebody could compile the rankings. We didn't show everybody's plan itself. We just took the highlight of they said, this member said they were fully executed or partially executed to be able to give that sense of general overview so we could be reflective.

Especially if there's a conversation about a member, I'm going to speak as a chair now, that we are wondering, hey, how do you think you're performing? And if they think fully executed, it allows for conversation.

Just like the fiscal tool, the quarterly fiscal tool that's reported, it starts allowing a conversation more at the quarterly instead of just saving it for the end. But we knew we needed that total summary, as Nancy shared. I'm sorry that I keep interrupting you, Nancy, but I want to make sure we address that question--

Nancy Miller: Oh, that's OK. This is a teamwork thing.

Karima Feldhus: I'd like to interrupt, too. Thank you, thank you, Nancy and Will. Just to piggyback on what Tim in the chat asked, actually he made me think that we could consider doing a retreat where we work together on these tools instead of just submitting them.

And our meetings are just an hour and a half once a month. That doesn't really allow for a lot of discussions. I just wanted to acknowledge that we would consider that. Thank you, Tim. OK.

Nancy Miller: And Tim, we actually gave each member two months to fill this out. There were questions once teams got together to fill it out, and it was interesting because I was the one that actually did the compilation of all the responses from the agencies.

So you could tell when the agency had a team that was working on the responses or when they had a single person that was filling this out to turn it back in. And that's because of the responses that were said or the way it was filled out, any questions that came back into the co-chairs or myself regarding this.

And definitely Will and Karima can speak more to this because we were back and face-to-face and didn't get to do a Zoom meeting. So I did not participate when these were presented to the general membership meeting.

But we are planning to do two to three-hour professional development session regarding the member self-assessment to go through it to make sure everybody understands, to have a document that would be completed so they have a template similarly to the quarterly fiscal reporting.

Definitely update information in this and have more things that would be filled out or an information sheet to give each member more information to be able to fully participate.

One of the areas that people did get held up is in the annual plan. We definitely have some consortium related information for marketing or for a consortium transition specialist type work. And agencies didn't necessarily understand how to fill out that information if it wasn't specific to their agency and it was generalized to the consortium.

So those are some of the areas that we have in the evaluation tool. Again, you have this tool to look at in full. But those were just some of the things that when you're creating it, you think, oh, we're being really thorough and this will be great.

And then it comes to putting pencil to paper or highlighter to computer screen and it doesn't really go the way you think it might have gone. So again, it's a labor of love and definitely something that will continue to be updated so that members can get the most use out of it.

Will Neddersen: As being one of the members that was part of the subcommittee for this work, it was interesting because we thought we had hit all these areas and talked with people, but it was bringing the group together.

So there is the pros and cons for both. Do we fill this out together collectively as a group in that professional development or retreat situation, or having the time to let teams process together and then bringing everybody together to then share it?

It was rather interesting this third page that you're seeing looking at the numbers. We really wanted to be able to share numbers, but that even brought confusion within our agency. So really needing to articulate that together.

But it shows for us where we need to process. How are we gaining that data of enrollment from systems that are TOPSpro and systems that are MIS, or MIS? Forgive me, I'm not sure how you describe it. I come from the K-12 side of the world.

But really, how do we share that information so we can get that collective number? Waiting two years for LaunchBoard to be able to show and represent it together, that's too long, right?

And not to look negative on that, it's great to be reflective with that. But moving us forward, how do we take those numbers and celebrate our successes as the annual plan and three-year plan are asking us to look at and do as well?

So this third page we'll be honest is something that we're going to work on and reflect together on still, knowing that we'll bring some subcommittee work back together on it. And then we shared it before, but a reminder in our subcommittee-- and I believe Nancy said it too, I think. And I just want to make sure clarity.

We would always bring subcommittee work back into the general session so that way it was heard what was being done. So if an agency couldn't send a member to represent, they still heard the information of the progress that was going forward.

Nancy Miller: And this last one, one of the things that we discussed after the presentation was these are the metrics that are out of the three-year plan. What we'll do differently this year is to actually fill these out for each agency so they don't have to look. The one difficulty is that if you aren't uploading your information into TOPSpro, it is harder to get it back out to fill out this information.

So again, our quarterly fiscal reporting tool hopefully will help us to get that information quicker and have that for agencies during that fourth quarter submission, and then we can get this tool out and they can have two months to fill it out and work with their teams on this.

Again, it's an activity that the agencies use to determine how well they are completing each of their metrics in adult education. OK, I think I'm handing it over to you, Will.

Will Neddersen: Yes, I just realized I talk a lot. I don't want to talk over you, Nancy. And I do apologize for that.

Nancy Miller: Not at all.

Will Neddersen: The interesting piece I'm seeing in the comments right now-- somebody is sharing. I think it's Molly saying, I'm thinking through what to adopt and personalize to help your consortium understand your work better together.

That's our hope is the tools that we're sharing it was helping us understand each other and be able to move us forward, especially in the idea of being effective as a consortium for our students.

And that's the hope in these tools that we're sharing here is that you can look at them and say, hey, I don't want to address it or I do want to look at it, but AB 1491 is really bringing that dialogue of member effectiveness forward. So we do hope that that's why we're sharing is that we hope that you make it your own.

With that, we start getting into the conversation of bylaws now. So if you just give a tool without a connection to why we need it, sometimes we don't do it. I don't know about you as administrators. We get a lot of different things. But here we want to talk about the tools supporting member effectiveness bylaws.

And the bylaws really are drivers of the work that we do. And with that, we had to look at, how do we insert what we expect of our policies and practice to be transparent and make sure that we're collectively sharing information for the betterment of our consortium and our students?

And so we wanted that dialogue to be transparent that way and really bring it out in by laws to give us guidance. And to that, looking at adding these tools that were created into the bylaws so that way it kept a constant dialogue forward that we didn't have to feel like we were in our own silos, but collectively working together.

It also helps us in defining certain things as well. I go back to why we shared the 2019 bylaws. There was punitive, as Karima shared, and a general vagueness to it. And when we're vague, we're unclear where we want to be able to celebrate and then say, how do we support if support is needed as well?

And then that's why that clear definition of technical assistance, really reading AB 1491 and understanding technical assistance, I had always understood it that's CAEP TAP will help support that.

But AB 1491 really brought it back to the consortium and saying, how are you going to provide technical assistance to your members if they're not meeting those expectations? And then really trying to address if our funding, again, if we have that excessive carryover, what are we going to do about that to keep us as an effective group working together?

So that's where we go to start looking at creating a complete article. And I know Nancy will go more into this, but really looking at member effectiveness. So Nancy, I believe I'm handing this over to you. Nancy, I apologize. I think you were kind to me to be on mute, but I'm not hearing you now.

Nancy Miller: I was. And I was trying to be really kind to you being on mute because my dog has a different idea about inside voice. For member effectiveness, last week if you were with us, we are comparing our approved 2019 pre-AB 1491 or the consideration of AB 1491 and approved 2024, which happened in January.

So the little glimpse you got earlier it was-- member effectiveness was two paragraphs or two paragraphs and the second one was really a two-sentence paragraph about member effectiveness. And now member effectiveness is its own article. I think it's two or three pages now.

There were four measures of member effectiveness. There was no distinction between measures for funded members and non-funded. Now, there's a definition of consortium effectiveness that comes directly from the state, but we thought it was important to define the difference between consortium and member.

And then we defined measures for funded members and non-funded members. The reason we did that is because non-funded members did not think that they had a role in member effectiveness. They didn't think that part of the bylaws applied to them.

So we were saying, oh, no, we want you at the table, we want you to be there. There is definitely a piece that you have as a member of this consortium. And so effectiveness applies in that instance.

And prior in the 2019, we indicated that there was a process to determine member effectiveness and interventions would be developed if they were needed. And now there's a clearly defined process. It's not ambiguous. There's member technical assistance and we'll go into later member non-performance.

The other thing about this was prior to the 2024 member effectiveness was really related to whether or not a member could vote at the meetings, and you had to be a member in good standing. But we didn't really give a tool to say, how do we know you're in good standing?

And now member effectiveness relates specifically to AB 1491 and we do write that language into the bylaws. And it also still links back to the retention of votes since that is a measurement. OK, I think we're on to Karima.

Will Neddersen: Karima, my kind, fellow colleague, you are awesome. You still are.

Karima Feldhus: I am so sorry, so sorry. We have office noise in the background. So thank you. I didn't think this would happen to me after so many years. Anyway, so this takes us to the current bylaws consortium effectiveness criteria in Article 5.

So we have Section 1, and you see a screenshot now. And that deals with Consortium Effective Criteria, what makes the consortium effective. Of course, we-- and again, I'm not going to read everything, we took the language from the code and AB 1491.

So consortium carryover reflects the carryover of all funded members of the consortium. And any carryover funding that exceeds the 20% threshold in one or more prior fiscal years shall be determined as excessive carryover.

Consortium that has been deemed by the chancellor or superintendent to have excessive carryover is required to submit a written plan to the state. And then consortium with excessive carryover shall be subject to technical assistance from the state as outlined in Section 3 Technical Assistance.

My colleague, Will, be covering the details of Technical Assistance in Section 3, and then Nancy will follow-- or rather Consortium Section 3 and Member Technical Assistance is Section 4. Next slide, please.

Section 2 deals with Member Effectiveness Criteria. And of course, we have criteria applying to all members and criteria applying to funded members. So regular attendance for all members and participation at monthly consortium general membership is the first criteria.

We have to be honest with you. We haven't started applying this yet. However, we're in the process of checking the last 12 months attendance. If we have members who are voting members but who missed many voting meetings, which we do have currently, then we'll start the conversation, right?

And number 2 here, participation in and contribution to the development of the annual and three-year plan. So we also require them to be active or actively participate. As we heard before, we do have committees. So like all agencies, we have 10 voting agencies to participate.

Timely, of course, submission of approval and approval of all CAEP and consortium deliverables in NOVA in particular. I have to admit, we've had agencies who did not meet the deadlines. So now we have some teeth. It's in the bylaws, right?

If the first of the month is the deadline, we have a deadline tomorrow Friday coming up, we sent a reminder. And tomorrow we'll go in NOVA prior to 5 o'clock and send more nudges to those agencies.

So the message really here is by having this in the bylaws, it is not an option to be late submitting your reports, it is not an option to not participate, it is not an option to attend meetings, especially when we need quorums or we hold votes.

Funded members, we have additional criteria. Of course, it has to do with student success goals as defined in our annual and three-year plans, financial expenditures aligned with the annual three-year plan.

And you saw the tool earlier. That's why we have part 2 with staffing and part 3 in that fiscal quarterly report dealing with classes and enrollments. So everything lines up. And here number 6-- funds expended in a timely manner in accordance 1491. So now we look at that.

And last but not least, we defined as a consortium that member carryover is going not to exceed 19.9. We didn't keep it at 20. As you know, we were asked to define that percentage excessive carryover and our consortium decided that 19.9% per member is our carryover excessive. Anything beyond 19.9% will be considered excessive carryover.

Now, next slide, please. So what happens if an agency is not being effective? So we're not even saying ineffective. So it's more positive. What if they're not being effective?

So you don't have in front of you. You just have summary bullets. But I'm going to go ahead and read you the paragraph from the bylaws. And you have that in your folder. I believe it's number 6 bylaws 2024.

So if members are unable to meet at least one of the criteria of member effectiveness, they may need the support and mentoring of the executive committee and/or other members of the consortium to come back into and remain in compliance with consortium and state requirements for California Adult Education Program funding.

Members who are out of compliance with the member effectiveness standards may be subject to technical assistance through SOCRC and placed into monitoring for a period of at least 12 months or until areas of deficiency have been remediated.

And my colleagues, as I said a moment ago, will be covering the technical assistance part, both for consortium and members. Next slide, please. I think we're covered. Let's see.

Will Neddersen: I think that goes to me, Karima, if that's OK.

[interposing voices]

Karima Feldhus: Yes, we're covered

Will Neddersen: So when we start talking about technical assistance-- and you heard me say, oh, I thought technical assistance was just at the state level. We need to clearly identify as AB 1491 has.

There's a consortium technical assistance and there's a member technical assistance. And we felt that it was important in the bylaws to include the definition of the consortium technical assistance so that way all agencies understood this is at the consortium level, this will take place. And then as Nancy will go into, there's that member technical assistance that's also needing to be embedded in what we do.

With that, I want to bring it up, as a new member to the adult Ed world at one point, the bylaws were a little vague for me. So reading these bylaws, we just thought if there is an admin change at any of our agencies, the bylaws might give clarity as well. That's why we were focused on adding definitions where definitions needed to be added or to give the assembly bill or the law to give guidance as well.

So I don't want to read this here, but if a consortium has excessive carryover for at least one prior fiscal year, they shall be assigned to technical assistance by the chancellor or the superintendent of California Public Schools.

We took that directly from AB 1491. I will be honest I'm not going to read the rest of it, but it gives us clarity on that. And Nancy, I know member effectiveness is where we had to work and move forward to. So I'm going to advance it on so you can start talking about member effectiveness.

Nancy Miller: OK, for member technical assistance, we really needed to lay out a process for members to go through. Again, we wanted this to not be a punitive process. We value all of our members and we want our members to be successful. So this is the process that we felt would best address how members could get technical assistance.

And one of the really nice things is that this technical assistance process came out of the fact that we had members who weren't performing and these are some of the things that the co-chairs did to work with members to help them get into-- I wouldn't say compliance.

Effectiveness wasn't an out of compliance. It was more non-transparency issue. And so in having conversations regarding operations or sharing of information, this is part of the process.

So the co-chairs if after the 12 months technical assistance is needed by a member, then the co-chairs would send a notice to the agency and their superintendent that specifically outlines the areas of non-compliance.

So those are going to be identified in the bylaws. Agencies already have a copy of the bylaws. So we would just note the section or the area that they're out of compliance. What are the measures that are necessary to regain compliance if it's an SOCRC rule or if it's a CAEP regulation? What do they have to do to come back into compliance?

Regular monitoring of member effectiveness of the areas that are out of compliance. I'll give you an example of why or how this was developed. If a member is consistently going over in their carryover, then we would want to have them report back to the co-chairs on a monthly basis with their financials so that we can make sure they're keeping on track.

Maybe every three months is too long to go before having a discussion about fiscal responsibility or how an agency is spending down their funds. It also helps the consortium better say, hey, is this something that you can share this year? And then next year, if you need money, we'll hopefully have money to share out next year.

To develop and implement an excessive carryover reduction plan so that member may be required to do that, where if it's a consortium, the consortium would decide together what that excessive carryover reduction plan would be.

So if a member who is in technical assistance, and technical assistance is going to go at least 12 months, if they haven't remediated their areas that are out of compliance, then they may have an extension of technical assistance monitoring for an additional 12 months. Or again, if it only takes three more months to get back into compliance, then they're only going to go three more months.

A reduction in funding, again, this is right out of AB 1491, not to exceed the amount of excessive carryover from the prior fiscal year. So if carryover is your issue, then we may have to reduce so that you aren't in that excessive carryover whirlwind that you can't get out of.

And then the final one-- again, this is language that is supported by education code, it would be removal of being a service provider. Because now you're two years into this, if you are really unsuccessful in meeting CAEP and consortium regulations, specifically around being a service provider, if you can't spend the money, if you aren't spending it in accordance with plans, then we're going to have a discussion about potentially being a service provider isn't the best thing for your agency.

So it's kind of like, when you get a job evaluation, it always says that clause "up to and including termination." Well, this is you're up to and including termination clause in member technical assistance. So Will, next slide.

So again, this is member non-performance. The agency is gone through at least 18 months. Most likely, they've gone through 24 months of technical assistance without remediation.

The decision to remove a member for non-performance, again, number 3 in Section 3 above, its 2/3 vote. Normally, we have a majority vote rule. This is one of the areas where it's a 2/3 vote.

The executive committee is going to send-- executive committee being the co-chairs, so Will and Karima would send a notification of potential action to the agency representative their governing board or the superintendent, if it's the county, outlining actions taken by SOCRC technical assistance for non-compliance areas and the date of the consortium general membership meeting at which the member being removed from service provision will be addressed.

So again, when we were going through this and had a retreat with the member agencies, we had a long discussion about this. And once we really looked at this, by the time it would come to a member non-performance action, we're about four years into the state technical assistance, the consortium technical assistance, additional remediation and follow-up with the agencies.

And 36 to 48 months later, it's a non-performance issue. So we did write this in case it ever came to that. We don't think that we'll need it, but it's there just in case. OK.

Will Neddersen: And Nancy, I think if we make that link you just shared when we get into technical assistance. But if you think about the other tools in their design that we shared with you today, they're there to bring that dialogue, that transparency and understanding of each agency together annually or quarterly so we have that process of understanding.

And just like, if you go back to the fiscal tool, you'll see just like it says in NOVA as well. What's your plan if you have a carryover and really making sure that we dialogue about that together?

I know for myself in Tustin, I've had to share. We didn't meet this mark, but I see spending going this way as I'm adding two more sections here of an advanced ESL class or those elements.

So that was the other in the subcommittee work as well as with the general session. Like you said, Nancy, understanding it's three to four years out. But even before that, that's why all the other tools make an impact into the work that we do to be effective together as a consortium.

Nancy Miller: Yes. And again, your technical assistance is provided just with the voting member and whoever from their team they want to bring and the co-chairs. It's done at an executive committee. It is not done in front of everybody at a general session meeting.

We really want this to be personalized approach to ensuring that the agencies are doing what they're supposed to do according to education code and SOCRC bylaws. We tried to make it a constructive process and not a punitive process. OK, Karima.

Karima Feldhus: All right, thank you. We thought we'll take a brief moment to ask you to share your practices or about your consortium or consortia. So we'd like to know, who has established member effectiveness criteria already?

Who has established a member technical assistance plan? Would like to also learn from you as well. As you see, this is in progress for us. And prior to AB 1491 guidelines, how did you address non-performing members? If you would please share with us in the chat would appreciate it.

And at the same time, we'll ask you, do you have any questions? We do have a few minutes. We're at 1:54. CAEP TAP will need a couple of minutes to--

Nancy Miller: Karima, we do have a question and it is, will you be posting these slides on the CAEP website? I'm going to send that one over to Mandilee or Holly. But in the meantime, you do have access to this presentation. It's number 7 in your folder.