Help them, their analysis of the data that everyone submitted. For hours of instruction and expenditures for the PY 2018 - 2019 period, which I've done and I'm going to talk about that. Also as a subcontractor to class, which many of you may be familiar with. I did the analysis of those data for PY 2017 - 2018. And so basically, I've done that the analysis of these data now for the last couple years.

And kind of in a position to talk a little bit about what's changed from year to year, et cetera. So I'd like to do four things this morning. It's afternoon for me by the way, because I'm in Illinois. But any case, I'd like to do four things. Give a little bit of background about this work. Then spend most the time walking you through the results. What we found when we looked at the data that you adults submitted. I have some recommendations that's the third thing I wanted to review with you.

These are recommendations that are my recommendations to the state CAEP office. They may or may not be implementing these. But we can may be talk about that. But I just wanted you to be aware of what those recommendations are. And then most of the time or half the time I think I'd like to reserve if we can for your questions and discussion. So we'll let when we get to that point will on people et cetera. So you can do that. In the meantime, as Veronica said, feel free to use your chat to put in your questions as they occur to you.

Then I'll be looking at that chat dialogue. And try and answer as many questions as I possibly can given the time available. So background. As you know for 2017 - 2018, the state office implemented a new day catch process through NOVA to collect information on hours and expenditure. So program expenditures and hours of instruction reports that we require to be submitted to the state CAEP office, for that first year. And we analyze those data and made a number of recommendations to the state office.

As a result of that first round of data reporting, which was the first time we tried doing all this. So there's a lot of rough edges to it as you probably recall. And so we made a number of recommendations. And those, the ones I've shown here on this background slide, are the recommendations that the state office implemented between that first round of reporting. And the most recent round of reporting that we're going to talk about today. So first just to tick those off for those of you who may be new to this, or who may be forgotten what it is that changed.

The reporting requirements for our early instruction were clarified, to avoid any duplication of our reporting between CTE and workforce reentry, which is now called Workforce prep. We clarified that. Secondly, we defined instructional hour and contact hour, in a way that was meant to conform better with federal requirements. Third the requests for reporting of services hours was dropped. So that's not required any longer just instructional hours. Fourth for K-12 and COE, County Office of Education guidance was provided regarding the tracking of hours of instruction for integrated courses.

So courses that combine features of patient instruction and academic instruction across the digits and program categories. Instruction was provided on improving the way in which those hours should be allocated to those two types of things. Fifth The funds source expenditure allocation reporting requirement was simplified. It was separated into required funds sources and optional found sources. So the required funds sources are Kate, we owe title to, non-credit, a portion that CalWORKS, Perkins, LCFF and jail ed funds, fees and in kind. Those are the required ones.

And all the other ones are optional, which means contracted services, community college supported services, donations, WIOA Title I, ITAs, other federal grants, other state grants and strong workforce. So all those were made optional, and the other ones are the ones that you see at the top that list of required ones. And we'll get into that in more details. As we talk about who reported what kinds of funds and in the actual reporting of expenditures. And then finally, definitions of in kind and fees or program income were aligned with applicable we are entitled to definitions and reporting requirements.

And all of these changes in command and clarifications are communicated with consortium members back a year ago. And also covered in webinars and presentations at the CAEP summit back in October 2019. And just as a reminder. Those of you who are at the CAEP summit. I was privileged to be in attendance at that CAEP summit on it and make presentation. Unfortunate webinar will do that this year. So this kind of webinar sort of substitutes for that. We're going to go over what we found out. What we think it means. And then, so then, we launched into our second round of data collection, of for hours and expenditures for PY 2018 - 2019.

And that was done the first iteration. That was done in two phases. And estimated hours and expenditure data, was submitted on September 1st. And then a final or certified hour and expenditure data was requested by December 1st of last year. And then on the 10th of December, a state sort of drove a stake in the grounds that were done. Everyone's had enough time now to report those data. And we had 66 of the 71 consortia submitting certified reports. And that represented 392 CAEP member organizations, which is in everybody, but it's a pretty good that's almost everybody.

So given all that. Then, the state office asked me to help them with these five things that you see on this slide now. They wanted me to characterize hours of instruction by program area. They wanted me to characterize expenditures by program area and fund type, because for expenditures we're collecting two types of information, especially by program and expenditures by fund type. So both those things they wanted me to look at.

And then, number three they wanted me to compare those first two things across provider types. Particularly K-12, Community Collage, and we created an other category which frankly hardly has anything in it. So it's basically, K-12 and Community Collages. And then fourth they asked me to compare all of that all of one two and three across the two years. PY 2017 - 18 and PY 2018 - 19. And finally to look at some data quality issues. In terms of how did we think we did. Did their quality get better or worse, in or sideways. What happened with the data quality.

So, that's what I want to get into now. So we spend mostly slides we get may be a dozen or so slides would go through those results. And please take notes. And if you have questions, we'll come back and we'll get into the details the particular questions that you might have. I'm just going to take these things one at a time to one to five. So first thing. Hours of instruction. So, for hours of instruction we looked at three things. We looked at what programs have the greatest number of hours.

How is that different across the different types of providers, K-12, schools and Community Collages. And then how does PY 2019 compared to PY 2018. Right. So figure why in this first pie chart shows the percentage distribution of hours of instruction by program area. So taken together as you probably are not surprised to hear. E S L, El Civics short term CTE and ABE, ASE account for the vast, vast majority 90 in fact '91 .4% of all instructional hours provided in 2019 were in one of those three categories.

The green the blue orange if you will. And the rest of it is very small by comparison. This next slide shows table 1, which drills down into that a little bit more detail. This shows the totals and percentages of hours of instruction by program area and provider type. So if you look at the right hand side of that table where you see the highlighted column of row percentage. That corresponds to the pie chart that you just looked at figure 1. And across the bottom the highlighted area shows that percentage distribution across the two main or the three provider types.

But as they said, there's really only two as far as any appreciable amount of expenditures or hours of instruction either one frankly. But this does show the breakout between the two primary types of providers. So just to remind folks. The K-12 category, I grouped this together to simplify things into these three categories. K-12 school districts include Elementary Schools District, High School District, Unified School District, County, Office of Education, Charter Schools, Joint Power Associations and Regional Education Programs.

And that group of providers together account for about 2/3 or 16.5% of the hours of instruction. The other major group course is Community Collages and that's Community Collages and County College districts account for about a third or 32.4% of all the hours of instruction. And with a tiny little rounding error amount that goes to other providers which includes, workforce development boards, libraries, community based organizations and the like is that small out there.

All right. Table 2, we're still on hours of instruction now. But here table 2, compares the hours of instruction by program area across the two program years, 18 and 19. So there's a substantial increase in the number of instructional hours for ABE, ASE and substantial decreases, in hours for K-12 success and workforce prep. But overall the hours are about the same across the two years. So very little difference in the total hours, but 65 million hours of instruction, which is a lot of hours.

But the distribution changed a lot and we think or I think that, that has something to do with changes in definitions of some of the program categories perhaps. Or some clarification about how our hours get allocated between one thing and another, that resulted in a reduction saying workforce prep. And some increases perhaps in some of the other categories. So, but I think it's interesting to see that even though total hours didn't really change that much from one year to the next. There was a lot of shifting in the program categories between one year and the next.

All right. So that was hours now we're going to turn our attention to expenditures. And so for expenditures we looked at essentially the same, basic questions. What programs have the highest expenditures? What fund sources have the highest expenditures? Which fund source that the highest expenditures associated with them? How do those expenditures vary across the different types of providers again same question as we asked before. And then how does 2019 compare 2018.

So a very similar set of questions. In this first figure shows the expansion or distribution by program area. So it's now goes to that in the first pie chart that you saw for hours. And it's very similar. And its results to about almost all the expansion fall into one of those three categories. ESL EL Civics, short-term CTE and ABE, ASE. And the identical expenditures are may be a little bit higher for ESL E L civics relative to hours. I'm sorry they're a little bit lower relative to hours. The spending for short term CTE is about the same as hours were.

And the spending for ABE, ASE is a little bit higher than the hour utilization was. And there's some other minor changes. But they're fairly similar distance overall terms of between the two. Which you would expect, because things cost what they cost. And if you're spending money. If you're writing hours instruction you're spending money in that category. So they ought to match up reasonably closely, at least at this aggregate level for the whole state. So now, this next chart shows the second thing we looked at, which is in addition to reporting expenditures by program area. This state office requires consortia to allocate expenditures for each program across the various funds that contributed to those program expenditures.

And there's a long list of funds as you can see there. So the reason for this is that the Kate box can have a more complete picture of how locally available fund sources are being leveraged to support the operation of Kate programs. So this figures shows the percentage distribution of expenditures by funds source type. And what jumps out here, of course, is that nearly 84% of all expenditures fall into one of three, the first three fund types. California adult education program or formerly AEBG, non-credit apportionment and we are entitled to AEFLA funds. All right.

Yeah reporting expenditures associated with each of these fund types is required. As is reporting for fees, in kind contributions, CalWORKS, LCFF K-12 adult Ed jail funds and Perkins funds. And all the rest of the funds that are on that chart, they are not required. So not surprisingly with the exception may be contracted services hardly anything in any of those. So again that wasn't terribly surprising but may be a little bit that there were so few funds associate with some of those fund categories.

I might have expected to see may be a little bit more on some of those other ones. But anyway, that is the result from at the state level for fund type. So then we drill down into that a little bit well you look into those expenditures by program type. OK. So we had we have our fair program, type in that and we have expenditures by program type. So this table shows that broken down by provider type. So down the left hand side to get your programs in the columns are the types of providers. And the highlighted row on the right hand side of that table corresponds to that pie chart I showed you a minute ago.

And this across the bottom is the distribution of the expenditures across basically the two provider types. So for 2018- 2019, over 812 million dollars were expended by k providers. Nearly 3/4 of that was done by K-12 schools. And about a fourth by Community Collage and Community Collage District. So the distribution of total spending by program areas between the two provider types, is similar. But not identical to the expenditures or the distribution of the hours, if you will, across the two provider type. So for instance, I'm just checking back here in my other slide.

Yeah as hours Community Collages were 30 to about a third of that hours 32.12%. And they're about 26% of the expenditures. K-12 districts were about 2/3 of hours, but they're about 3/4 of spending. So those are some interesting differences there in terms of distribution of those two things between the two provider types, whether you're looking at hours or whether you're looking at expenditures. So then the next thing we looked at was how did this change from year to year from 2018 - 2019 by major program category.

And you can see there this is table 4. That compares the expenses across to program year by program type. So there were some substantial increases in expenditures for K-12 success ABE, ASE and AWD that were postponed. There were substantial decreases in expenditures for workforce preparation, which again may be related to some definitional changes. There pre- friendships and short term CTE. So all those had reductions. But overall expenditures are almost identical were in a prior year down slightly, which could be due to the fact that we had a couple fewer consortia report. I think is in this latest round than we had in the first round.

So that alone could probably explain the reduction in expenditure total. But it's pretty close. But there are some interesting differences and types of program categories in terms of changes from year to year. And some of them are pretty significant. All right. So that is hours, and expenditures, and time one time two and provider types, all the basic things that they want to me to have a look at are shown there. So then one last thing I give is I kind of looked at data quality questions.

And so for data quality, I was really looking at a handful of things. And I'll go over these it's like is an important issue with respect to know what do we do going forward with this requirement. So I looked at compliance with the reporting requirement. I looked at consistency within reports by consistency between hours and expenditures. Those things relate to each other, which would be good if they were right. And I looked at extreme values, because I mean you have lots of extreme values that can make it difficult to make use of the data.

And then I looked at year to year consistency for individual members. How the individual member level did things change a lot from year one to year two, because that could be a little red flag as well, right. So those are things I've looked at for data quality. And they've got some tables and charts and stuff for each of those things. And want to go over with you if I could here. So first table 5 is the re answers the question to what extent did members and consortia comply with the reporting requirement.

So table 5 shows a number and percent of CAEP members who either did not submit a report or submitted a report with zero total expenditures or zero total hours of instruction. Now you might say how can you do that? Well, you can and many did probably shouldn't be until we get to that second. But there we had just for just to pick some numbers that we had in 2019 we had 38 members. These are all members right. We had 38 members that submitted certified report zero expenditures. We had 46 that submit a certified reported zero hours of instruction. Right.

So and we had a couple have few and a couple of dozen that submitted zero hours of instruction in both years, both 18 and 19. So I would say that we've got some compliance questions here around at the member level at least, around the utilization report now. Some of it yes one year and not the other year could be members coming and going from existence. I was a consortium member 19 but not an 18 or vise versa. But I try to control over that by only applying this to places where I had to identify the member as being present in both years.

In other words, at least present as far as the report was submitted by them. Right. So these weren't failures to submit or any other big. They were there but they weren't submitting report basically. As they existed but we didn't have report form. So how exactly interpret that probably depends upon your knowledge of what's going on actually on the ground particular consortia. But I think it's something you'd want to follow up on and have a better understanding of what's going on. Right.

The next thing and I apologize for the size of this the slide here. But I think we'll be getting these slides to everyone so that you have them at your disposal can look at them at your leisure. But so beyond with the overall compliance with the reporting requirement did or did I not submit a report with something in it. To what extent do members actually provide information for each of the major report data elements. So that's what table six shows. The persent of members providing a non-zero response to each of the major report elements. Now these are not all the report elements because as I'm sure you recall there's hundreds of report elements.

But these are just the major ones. The top group are the major totals for hours of instruction, for the whole member. And the second is the major expenditure categories by program, for at the member level. And then the last group or all the fun source categories. And the ones above the line are the one I'm sorry the ones that have the asterisk are the required ones. So as you can see there's really only like I think eight highlighted those eight cells there those are the eight categories for which a majority of members provided some non-zero response. Right.

So it's most categories particularly when you look at expenses by fun category. Most of fun categories really just aren't used by most members. So that's I think that raised some question about whether the effort that goes into actually frankly gathering all that fun category stuff. And whether you're really getting much for that. Because really there's not a lot of members reporting basically on any of those categories. All right so beyond compliance did people summit data, et cetera. Then I wanted to look at the sort more of the tactical level at that kind of consistency within the report it's themselves.

And so the first thing I looked at was the relationship between expenditures and hours of instruction. So that's what this was that's a Figure Four does it's a scatter plot. And it has colored dots on it blue for Community Collages, red for K-12 and green for other. And again not what others but they're all on there together in this slide. So each on this graph represents a member. And it's positioned on the graph, in accordance with the relationship between their total 2019 expenditures and their total 2019 hours of instruction.

And if they were one of those members that did had zeros, in either one of those categories they're not on this slide. Is it just people with non-zero results. And it's also a log chart which means that as you go to the right or as you go from low to high each of those lines is 10 times greater than the one below it or to the left of it. So that's just done. So we can get all these dots on the slide. And you can sort of see what's going on without them all being crowded down in one corner, which is what you would have if it wasn't for the fact that use a log chart to display the data.

So the real question that this kid said is that, are the underlying values that would be used to create a cost per result, which are hours in instruction and expenditures, right. How are those related to each other in some kind of predictable manner. In other words, do members with high report hours of instruction also have high reported expenditures. And with those with low report hours instruction of low report expenditure that's really what we're getting at here. And so this plot suggests that this is similar to what we saw in the original data. That the relationship between expenditures and hours of instruction is relatively close for K-12 providers and unless so for Community Collages.

That again it's a part we saw before. And so look at the North to see that a little bit more clearly. I created the next two scatter plots which separate those two types of providers apart. So you can see on the left hand side Community Collages are obviously a lot fewer of them and they're much more randomly organized there. And then the large number of K-12 providers on the right hand slide much a little bit more disciplined to grouping. If you will, around that so in a higher are our squares anymore.

We can't see those because they're tiny, but our squares their show that they're reasonably decent in both cases really. But so there's a decent relationship for both sets of providers. But it's stronger for the K-12 providers in terms of relationship between hours and spending. Which is what you would expect to see if people were reporting real stuff. All right. So then the next thing I looked at was. What does this look like when we plot expenditures against the cost per hour. So if we computed what he did that computed a cost per hour total all expenditures all hours. it's not broken down by program it's just everything.

Computed a total cost per hour that's 1A the X axis is a law is the expenditure between 19 and this is all 2019. And again but the two the three provider types. So overall there's not great correlation between that. So then I this is a little easier to see when you break it down. Which the next guy does between two different types providers Community Collages and K-12. So I mean I was just make it personal. I would expect that if you're doing a really good job of measuring hours and expenditures. You might expect that there would be economies of scale and you'd be able to see that in the data.

In other words big, big providers would have some how lower unit costs than little bitty providers. Because they have economies of scale that smaller providers don't appreciate. So you would expect to see some correlation in other words, those costs per is being lower for the bigger guys. And don't really see that so much and so I'm not sure why that is. And it's OK if it's not it's just it's just one check I looked at it does. There is a little bit tighter relationship for the K-12 than there is for CME colleges which look to be like pretty much completely random. But there's a little bit of relationship that goes the opposite direction of what I might have expected for the I'm sorry Yeah for the K-12 providers, .

But in either neither one of those hasn't much been our square associated with it. So I didn't finalize looking for there. All right. So the last couple of things we looked at on data quality was extreme values and year to year member consistency. And that's what's on these two tables here table 7 in table 8. Table 7 looks at members that reported low values in 2019 and I did these two low values $50,000 of total expenditures and $1,000, 1,000 hours of instruction are arbitrary. I just picked those because they seemed to me that they were that that's a pretty low amount to have as a member, what could be members I should mention it's possible that there are members that have different functions that are not necessarily instructional related.

For instance and they might have very low hours of instruction. And so that could possibly explain some of the 31 there with less than 1,000 hours of instruction. But as a general rule of thumb it seemed to me that we would be interested in somebody who's expenditures were or hours were that low. And about nearly 8% of total members have report hours of instructional less than 1,000. And about 6% or 5 and 1/2% had less than $50,000 of spending. And 3% had both of those things. So that group there that 13 is I think of particular interest as to whether this is a relevant requirement for them given the size or what's actually going on or whether that's accurate.

So, and then the table 8 is looks at year to year consistency at the member level. So and again this is a little arbitrary on my part but I just think that if members are actually reporting their hours and expenditures. Then we wouldn't necessarily expect to see huge differences in total expenditures or hours of instruction from 18 and 19. Since there have been large changes in available funding. Now that's a caveat there we might see large differences in particular programs or particular fund categories of course because a lot of variation reporting instructions et cetera.

And it's possible that we might see large variation it provide at the member level if we had new members, or retiring members, or some other reason why at the member level there had been major changes in funding particular member. But it kind of jumped out at me that we had so many members with relatively large changes in total expenditures or hours of instruction from one year to the next. So for instance 81 of the 360 members who reported expenditures in both years. 81 or 22.5% had a variation of greater than 25% from for year one year two.

And 120 of 348 who reported hours in both years are over a third had greater and 25% variation in the hours of instruction. Which seem to me and I could be dead wrong in this. But it does seem to me that was a pretty high as a just total variation going on from one year to next year. And implies at least there might be something going on with reporting compliance say or understanding of the reporting requirement. And I would want to take a closer look at that if I was the state CAEP office.

OK. Finally I came up with some of these recommendations which probably aren't news to you may be. The first is just some technical stuff I think are important not sure they can all be done. But they are there's seem to me to be things that would be sort of relatively straightforward things that if they could be done can dramatically improve reporting quality and completeness going forward. The first is like standardizing member names across reports. I had to find people, in the two different years had the names that changed. The spelling in the way the name was presented or described it altered from one year to the next.

It will be super it is like standardize that. So a member is called what a member's called from one year to the next. And makes it a lot easier to find them in the data and make sure that things are matching up. Secondly, we probably ought to prohibit report certification to zero entries. I'll say 21 members that submitted certified reports with zero entries for hours instruction and 14 that did so for expenditures. So really strictly speaking, if you don't have anything to say, you probably should be able to certify your report.

Third, we ought to require consistency. These last two are a little harder I recognize. But I think it would be great if we could do it. Require consistency between hours and expenditure entries at the program level by the main not the funds but the program categories. It shouldn't be possible to certify a report with hours of instruction in a program category where you don't have any expenditures. Or expenditure or no hours of instruction. So if you're doing ABE, ASE, then you need to have if you've spent ABE, ASE money then you probably ought to have hours of instruction for ABE, ASE and vise versa.

So that's just I could consistently check at the program level between the different types of things. If you don't workforce development then you need to have if you're doing one they should have to have both things spend hours and hours. And then finally, if we ought to try to link report requirements to records of fund availability at the member level. So that if we know that a member has been funded to do X program ESL EL Civics. We know they've been funded to do that. Then there should be something in the report. They should have some expenditures and hours associate with that.

Now if you have members out there that are not instructionally. Then their mission is not instructional. They don't have hours of instruction. They shouldn't have hours instruction. They probably should be exempted from report requirement because it's not relevant to them. This is about people that are doing instruction. The whole point of this is to figure out what's going on between spending and hours right. So if you don't do hours then you shouldn't be in this report with cluttering it up. Sorry I have a tendency to be more candid and proficient but anyway.

Then these last set of recommendations are more policy oriented. And these are all things that are kind of just continuations of work that's already being done. And I just wanted to encourage the office to continue to do that. Continued support consortia and members with training and TA on reporting policy and procedure. Putting best practices for allocating costs. Continue to work with consortia and members to gather input to make further improvements in a policy and procedure going forward and how that make best use of the data. And I know those things have been going on.

And then this fourth one is I think a new idea and that is that the state office should provide feedback to members on their reporting results, including unit cost estimates prior to member submission of revised reports. And what I mean by that is that would be, I think, really helpful for members to be able to see what the implications of their report are with respect to things like unit cost at the program level.

So in other words, according to the data you sent in last time, you said your unit cost per hour for ESL EL Civics was $1,500. And then people can look at that and say, that's not right. I probably ought to look at what I said there and see if I can get that a little closer to what's actually going on, right?

So that would be helpful. That kind of feedback loop in any kind of data collection system. If you don't have a good feedback loop you're not going to get good data. That's just the bottom line. I think creating that kind of feedback loop would be super helpful to improving the quality of day over time. If you're going to have this requirement you must take it seriously. And get that feedback in place they can get the data where you want it to be. And then finally I felt that the office should consider options in addition to everything else I've said. To improve compliance with the reporting requirement.

Once all the other recommendations have been done. Then this compliance reporting requirement and how utilization data all that stuff should go into the regular monitoring process for members. So that might include are people taking this requirement seriously. Are the run kind of problems they run into. What can we do to help make this better at the member level. So we have so we can just work on making that better over time. OK. There I took a slightly more time than I wanted to but I wanted to make sure I got through these things.

So I'm going to, with Veronica's help I'm going to open this up. And answer questions that you may have. Let me look at my chat log. Well, there's a lot of stuff in the chat log. OK. I'm going to take these not necessarily. I'm just going to take these in the order I see them that's OK. Let's see here. Scrolling down through the greetings can Veronica can people see my Zoom chat on my screen or not. Or are they just relying on me to get it right.

No, we're all looking at the same Zoom chat. And so

OK.

The first question related to the content I believe. There was a question from Frank regarding service hours. And then.

This one? From Frank?

Yes. This was more of a general question to the audience. about seven hours.

Right. And that's a good question. Frank I think that was a significant change that was made to the policy from year one to year two. And I'm not if there's state office folks on the call I would encourage you to weigh in here on this. But my impression of that is that there were a lot of issues that came up around reporting service tracking service hours. What it's a service? What the service hour? How do you know what's what? And how to keep track. A lot of people really weren't doing that. They didn't really have a mechanism for doing it, et cetera.

So I think all of those things together kind of led the state to say may be we should just, that was a bridge too far, should back off of that. So if the feeling is that. There's still a requirement or an itch that needs to be scratched we probably ought to talk. That they need to be some sort of dialogue go on around how may be to approach that I guess. OK. This is great all right. Yeah. OK well we kind of thought that we covered that right. And then there's the PowerPoint excellent.

Is their definition description list of what each of those fund source types are? I believe there is and the reporting instructions now don't ask me to quote do the issuance no. But I'm sure that someone else could provide that went out back prior to this most recent round of reporting. I know is part of that clarification or part of that reporting issue. And I think there was some definitional stuff put out on the funds source types. What they were what was required and what their definitions were.

And Tim I posted the document into the chart.

Good for you. Thank you so much. OK. Be wary putting too much faith in expenditures they fund source Yeah I would too. Andrew be wary of putting too much faith in spending by founders. That because I don't know that we have cracked the code on how really at the member level. And again this is Tim's opinion only so don't you know don't take this to the bank. A state might feel differently about this. But my feeling is that I don't know that we've cracked the code on. How to properly recognize these costs at the member level? And how to properly associate them with the appropriate funds source?

So I think there's a lot of variation there. I agree with you that's what you're suggesting. Around how those costs get recognized and allocated. And it's not like it's just I can only use my kind of Midwestern experience here. In a lot of places what a state agency would do would say here's fund categories. Here's what's required. Here is the allocation formula the allocation model that you must use. And here's how we're going to enforce that. And that's just not model that CAEP is taking with this.

And so it means that if we're going to have policies around. And procedures that they're going to arise from the interaction between each of you as members and consortia. And the state office kind of working together to come up with best practices. That's my sense of it. So is it possible that until that happens, we don't know how much will really be able to get out of the funds source level data frankly. Yeah. There it is. OK. Thank you. All right. What type of designation would student support services fall under?

Do you mean what type of designation in terms of what program category? Because you could have student support services that are relevant for any of the program categories. Because it's a generic service that applies to all of your students. So it's part of the cost of running a particular program. So I think the program hours of instruction and expenditures follow the student and the program that they're in or programs that they're in. And that would include costs associated with your support services. You're not tracking support hours but you are tracking expenditures of all kinds and that would be one type of expenditure that needs to follow the program that student is in.

And if there are multiple programs or cost had the outcome across the program categories, that's my understanding. In other others we have something else I'd like to say please at the state level please weigh in. I'm wondering how schools are planning to report in kind funding width distance learning. We always calculate in kind by the rate charged to the public to use district facilities. The number of hours that are in classrooms would have their own facility. But obviously we aren't using facilities right now. So you had a mechanism, a very clever mechanism, for keeping track of that stuff it just got blown up by COVID 19. Because everything's happening distance wise.

That strikes me as exactly the kind of question I think that you need to be working on as a group in your consortia. At the end through this dialogue. About finding and sharing best practices. And if there are consortia or members out there that have got have given thought to this. It would be great to hear what you have to say about it. I don't necessarily have a suggestion about how to allocate those costs, or how to identify in kind costs when you're not using facility itself.

Because it's mean you're going to be again. Though so you might skip over this process and use prior allocations as a way of practicing for what's going on I suppose, but that's a temporary measure. But if we have long term changes in the way you do business then that could affect that too. Right. Yeah I get your point, Andrew there about the allocation assumptions. I do think if it was me and it isn't. But I mean if it was me one thing I would be doing that I'm not sure is happening is I would be asking each consortia if consortium to document how it's doing. How its members are doing what they're doing.

Frankly I would encourage them to have their members follow some standard procedure. And which I know it might not be a popular thing to say. But I would do that. And I would encourage consortia to document what they're up to. In a way that could be tracked back to what you're actually reporting. Because this is a moving target and things do change over time. And we don't have standardized allocation requirements. The procedures for allocation if you will allocation rules on.

So in the absence of that, the only way to get to good practice, is to gather data about what people are doing and start comparing notes. And frankly you know it's possible that one of the reasons we are seeing some really big shifts from year to year even at the member level. Is because allocation processes altered from year to year. And so you get different students being allocated different programs differently. So that can make things look a lot different than they actually are in the real world. OK.

Need to be careful on how salaries hours and expenses used. Because K-12 does go to cover. Facilities costs I don't believe are counted into the college expenditures. Yes so in other words, when you're saying that look we have costs greater than may be hours, because we have certain kinds of costs that aren't getting picked up. And other types of entities don't have. So that I mean may be the larger point is that these two types of providers aren't directly comparable to each other. I think we recognize that. So I think it's an important point though. Oh that was your answer to her question good. Thank you.

OK. That's more on the issue of cost. How get cost get recognized? Yeah. OK. Well, Crystal, that's good. There are zero hours of instruction. But there still are services. So here's me though this is the question that raises. If the policy is that you're not tracking service hours right. But you are tracking instructional hours and expenditures then we just need resolution of that it seems to me because if the mission is not instructional fundamentally then I'm not sure why you're in the reporting domain for this.

Because the only way to be in it is to have some hours. In it just to me I don't know that's is my point. It's just more straight forward, if you say this is what this is. This is about instructional hours and about expenditures for those instructional hours. And that includes everything else I'm doing for the student. But if all I'm providing is services then what? And I think if there's not we have to have clarity on that I think there is going to be bedeviled by this question of people with expenditures been no hours of instruction. Because we've restricted one and not the other. I think that's my problem with they we think we need be consistent in how we treat that.

OK. This is a debt Grace's response to Diana on custodial services et cetera. OK. This is good there is dialogue. I like it. Something that causes that facility costs and of course some programs that are offered at offsite location. Yes. Right. I mean there's no cost or cost whether the cost is on campus or off campus it still costs. And should be recognized as such. But Yeah not obviously now we're in the middle of COVID all this stuff is stone off the rails. So but we're assuming and hoping that there'll be some kind of return to normal operating procedures here in a year.

So. OK. How is the instruction by program area. The term hours of instruction is defined as any report able individual that is at least one hour program related services or instruction that was a definition given in the budget bill requirements. I'm reading that related services also service hours. Am I understanding this correctly? I think so. But I would rather have someone from CAEP office give you a definitive answer on that. But Yeah I think so. Veronica, do you have an opinion on that or not. No I do not. But I'm capturing the conversation as well as the outstanding questions that the CAEP office will be weighing in on. I'm going to be reporting back to them after this webinar. And so if there are any additional questions that we are unable to address here, I'll be relaying that information. So type them in the chat or send them to tap@caladeled.org.

And I'll include it in a package of questions that they'll be addressing. And I'm also going to propose if there are additional questions or additional discussion that needs to take place I'm going to ask help proposals that we set up another CAEP office hours. So that everyone can come on and ask those questions with members from the CAEP office.

Yeah that's excellent. . That's great. So Andrew I appreciate this comment here. And you're right they could have something to do with what's going on with this. The fact that we're not seeing that strong enough a relation that we thought we might see between of those things. Is just that there's different allocation rules at work and I agree that getting exchanging better information about how that's happening may be is a good way to something productive to do as we move forward. Yeah thank you.