Veronica Parker: Randy and Jessica, go ahead with the start of the webinar.

Neil Kelly: All right. Thank you, Veronica and welcome everyone. I think we had about 160 people signed up. Hopefully, they're all going to-- well, we don't want all of them to show, that might be a little too much. But we do have a good crowd right now, so we welcome those who arrived on time, and looking forward to today's launch. It's a big deal for-- if you're a data person like me, I really like to dig into the launch board and look at the Adult Ed data.

As you know, the Adult Ed Pipeline is one of the few dashboards out there that combine data from many systems. So on this dashboard, we include the TOPSpro data, which is managed by classes and that's the K12 keep data, as well as the WE02 literacy data. And then, of course, on the college side, we have the community college, MIS data coming in, as well as the EDD base wage file.

So this is a real comprehensive dashboard and they've been working the WestEd team has been working closely with classes to align the data, to make the data better quality. And every year, we improve the quality of this data as you can tell and we add ways to slice it and dice it. And just digging into some of the things that WestEd will show you today is really exciting related to transition, employment, student progress, ethnicity, all the breakdowns of the demographics, student barriers.

So I'm really excited, and this is really going to be helpful in your planning, your long range planning if you're doing a three-year plan for the Adult Ed program or just long range planning in general. So I'll be manning the chat in case questions come up. If they're technical, I'll defer to WestEd. But I'm going to turn it over to Randy to take us from there. So thank you for coming and look forward to today's webinar.

Randy Tillery: Thank you, Neil. Thanks for being here today. My name is Randy Tillery. I am the area director of workforce development of post-secondary education at WestEd, and I'll be just kind of leading up to the main presentation, which will be Jessica. I'm just sort of the window addressing today, which is why I wore my fancy red shirt as Neil pointed out when he jumped on earlier.

So today's webinar is going to be focused on really new features and updates. It's not going to be a comprehensive deep dive into the launch board, but this is really the first of two webinars this week that we're doing on the launch board. So we invite you clearly to come back on Friday, although, we much kind of deeper exploration of all the bells and whistles and how to use it for planning. So a little bit about us, Jessica, next slide.

So like I said, my name is Randy Tillery, also with me today is Jessica Chittaphong. She's a Product Manager, Data Systems and Tools. She is the build manager for the Adult Ed Pipeline. Every one on the dashboards and the launch board has a build manager assigned to it. And she does an incredible job with this build, so I'm very excited for her to get to present today.

So just a little bit about our group of WestEd. So WestEd is an enormous organization, 700 people work there. The post-secondary education and workforce group is just one component of a very large entity. So oftentimes, you see other people from WestEd and other contexts. It doesn't necessarily mean that they're connected to us or that we talk to them. But our real focus is really trying to strengthen the role of higher education workforce and economic development programs to improve student access and outcomes of higher education, and increase economic mobility for low income families and communities.

One of our pet projects is we're trying to convince WestEd right now to allow us to create an actual center for economic mobility a little tie together, all of our work in secondary, post-secondary transition, adult education, workforce development and higher education. Next.

So today it's going to be pretty simple. So we're going to be reviewing changes on the Adult Education Pipeline built 4.0. This is the fourth year we've built this. There will be a 4.1 version that will have some minor changes in it downstream. We do sometimes do kind of iterate updates.

But Jessica is going to be doing most of the presenting. She's going to talk about updates to the calculations places where we've refined the calculations like what you're going to see is the account of reportable individuals for colleges in particular is going to be much more closely aligned to your actual student numbers because we were picking up some other non-credit courses that we didn't know colleges were offering in that metric.

She will be talking about new metrics and she will also be talking about new features and tools. Some of them are new features that are not new like when we did the field testing with folks from Adult Ed, they love the new drill down, which actually weren't new, but it was great to see them actually finally discover them and get to work with them.

If you're in the wrong-- if you're thinking you want the deep dive, the exploration of the launch board and all the features and all the metrics and kind of the underlying data on how it all works, this is not that webinar. So this is going to be focused on new features and tools. So we are presuming you have a little bit of understanding of what the launch board is and maybe have spent some time on it.

If you haven't and you want to parallel, you could always open up your browser and Google cow pass launch board and go to the Adult Ed tab and kind of be looking at it for the Adult Ed Pipeline kind of while we're going through it. That could be fun to do in parallel. But on this Friday, we're having a deeper dive webinar and that's what I had up on the screen.

So the Adult Ed Pipeline just a couple of things about it and our approach to it. We feel like we need to say this all the time because people always look at it and say like, this telling me if I'm doing good or bad. That is not the primary purpose of it. So it's really meant to be a resource for adult educators, colleges, and consortia to improve student outcomes.

So there are dozens of metrics-- actually hundreds of metrics if you include all the drill downs to allow you identify, hey, how is this group of students doing in terms of-- are they achieving EFL at the same rate as my other students? How many of my students are transitioning to post-secondary? Is that a number I want to improve? And then it helps you begin to identify places where you really want to be doing better in relationship to your student outcomes, and to really focus on that kind of improvement.

The metrics are aligned to the student journey if you look at the buckets you have a kind of students and programs bucket. You have a progress bucket, you have a transition bucket, a completion or success bucket, and then employment. So it allows you to kind of think about whether students connect how they progress. Do they transition to post-secondary education? Do they complete credentials, and then what happens in terms of job outcomes? And we do that to kind of make it seem a little more logical in terms of where the metrics are related to specific points in the student journey.

As Neil said, it includes college MIS data, that's the enrollment management systems for all the community colleges that data is uploaded to the Chancellor's Office every year. And includes all of the K12 Adult Education TOPSpro data, and we match those that enrollment data to each other. So we can identify for students started to K12 and adult school that shows up in MIS and a credit program. And then we also match it to the EDD employment development department wage data to identify students who've achieved employment or earnings outcomes.

It's really the only complete source of college noncredit in K12 Adult Education student data and outcomes. This in no way implies that the data you get from CASAS on a quarterly basis is not important or is not good data. It just means we package that data all together so that you get a summative view and after the year ends of really what your consortia data looks like complete. All the college data, all the TOPSpro data and then the EDD wage file matching.

And I think as Neil indicated, I think he talked about the three-year planning. We are going to be loading some of the pipeline data into NOVA for you to use in relationship to three-year plans for you to kind of set goals related to specific data points. Next.

This is just kind of illustrates what I was saying about kind of student journeys. You can see, again, the ones for students and programs, progress, transition, success, and employment and earnings. I will say just really quickly all of the metrics on these tiles when you go into the pipeline, these are live metrics. And when you go in, you will see a summary metric. And then underneath that are detailed metrics where you can look at all kinds of different slices of data and then you can use drill down to disaggregation to look at those specific student populations, or you could look at outcomes say just for your ABE students or your ASE students or your CTE students as well.

Why make changes? Well, I like to say data is a journey. It's not a destination. We make changes because the data environment is changing all the time and there are other dashboards that you say noncredit data. And so we're always working to be sure that the information you see in the Student Success metrics or the college pipeline as aligned as possible as it is to the AE Pipeline, particularly on the college side.

Also, we really are aggressive in taking feedback from the field about what they want to see. In fact, we had a team of 13 or 15 people who actually field tested the dashboard for us prior to it going live on Friday and we really did take their feedback very seriously in identifying things that we'd like to improve for the future. Also, feedback from these webinars really helps us understand what you would like to see in the dashboard and oftentimes people will reach out for-- reach out to us and say, how come we're not seeing this or how come we're not seeing this view? Or this doesn't look right.

And we actually totally invite you to break it. We invite you to go on and find things that don't make sense and ask us questions about them. Oftentimes, it's just a clarification. Just saying like, well, this is how, this is what this metric how it's really constructed and what it shows you. But other times there's something going on underneath the data by either about you're entering data and coding in ways we didn't anticipate. Or sometimes there were just errors in the calculation, like, there's a coding error or problem with the calculation. We have to go in and fix it.

Also, we're continuously refining the definitions and the metrics themselves. And in particular, I will say the noncredit world and how noncredit institutions were coding their data in MIS was kind of the Wild West when we started this journey four years ago. And we're getting more and more precise about understanding how people are actually coding their data in ways that allow us to kind of define the metrics in ways. So the numbers look like they make more sense to you when you're actually using it.

Jessica Chittaphong: Thanks, Randy. And before we move on, I just want to point out for folks who are new to the AEP Adult Education Pipeline is actually a part of a suite of dashboards of the Chancellor's Office support, which we call the launch board. And a lot of work happens to think about the definitions and the data elements. And sometimes metrics are shared between dashboards and they kind of have to work together so that we're capturing the same data concept throughout the dashboard. So I just wanted to add that as well.

Randy Tillery: Another thing just to build on what Jessica said, if you're interested in particularly knowing more about what the noncredit programs are, there's much more detailed say in the college pipeline about what specific noncredit CTE programs that pathways exist. And so you can use the dashboards in ways that complement each other. So don't feel like because you're working in adult education, you're part of a consortia. You can't look at those other dashboards. It's really helpful to see what your college is teaching, and go explore those pathways and to have that feed into the conversations within your consortium.

Jessica Chittaphong: Thanks. And the community college pipeline will be updated sometime this summer just for fun.

Randy Tillery: Stay tuned.

Jessica Chittaphong: Stay tuned. All right. So I think we're going to jump into a quick review of the changes and updates. I'm keeping at a pretty high level, I'm not going to go like, for example, I'm not going to talk about specific data elements. This is just to help people understand the big picture what we change and what we update it. I'll talk about a couple of metrics in detail. But if you do have any questions, feel free to use chat and we'll address them there. And we'll also leave some for the Q&A at the end.

So for the first two metrics I wanted to go over in terms of how we've updated the calculations. These first two metrics are what we refer to as some foundational metrics. The changes made here will affect a lot of other metrics. So I wanted to make sure that folks understood what happened here. So for the first change we did was, how we counted reportable individuals or adults served. We heard from the field, as well as a noncredit working group that we should exclude specific top codes from the count. So that's what we've done. We've excluded supervise tutoring, as well as study skills from the COMIS courses.

And that has definitely cleaned up the counts, but it also you'll see that some of the college counts will look very different from last year's bill. So that's the big reason why. Secondly, we ensure that students who are receiving any sort of services regardless of enrollment are being counted. So that's another big change. So the reason behind these two changes are really to correct the inflation that we were seeing in the noncredit program side as well as ensure that anyone receiving services are getting counted both at the Adult Ed programs, as well as the noncredit programs.

Randy Tillery: So for the AE folks, what we're trying to correct here is a lot of colleges use noncredit core sections for their tutoring centers or for other services they're providing to students who are primarily in credit programs. And they're coded very similar to courses that we use for Kate, and so we needed to exclude those top code to be sure we weren't counting say, all of the students in Chaffee's tutoring and study skills programs, which is inflating their count by like 8,000 students, in terms of reportable individuals.

Jessica Chittaphong: Yeah. Thanks, Randy. Yeah, and just to reiterate, these changes mainly affecting college counts and not the accounts from the K12 adult school. So the next change is how we change, how we counted the participants? i.e. the reportable individuals who made it to the 12-hour threshold and the participants is really the metric we use as the main denominator for our outcome metric. So we really wanted to get this more defined and definitely more accurate.

So the big change here was along with excluding those two topics that I talked about before. We also made sure that we had some logic in place to only count enrollments in the identified six major CAEP program areas. This just kind of fine tune the counts for these participants a lot more. It aligns more with the CAEP priorities. And again, it'll influence a lot of our outcomes metrics because this discount is a big foundational metric, and the main student university begins to count outcomes like transition, as well as the program areas-- enrollment in the program areas themselves. So I'm going to pause here and make sure that there's no question. OK.

So we also included some small demographic changes of how we're counting students. The non-binary option was made available to the community college of folks a couple of years ago that has been added as an option in the Adult Education Pipeline. So you'll see that coming on board. There won't be a lot of data populating that category for previous years, but you'll see it in 2019 for sure.

Additionally, we've updated the age bucket so that in the dashboard, you can look at students by age. And these age buckets have been updated to align more with WIOA reporting. So you'll see a slight change there. Additionally, we've noted some new data elements that will be used to flag students with disabilities. We've identified a new CASAS data, as well as a student flag in COMIS to be able to capture more students that way.

The last thing we did was really-- and we implemented additional matching efforts on the coding side for being able to find students social security number. Now for those of you who are not familiar with the Adult Education Pipeline, we have a whole employment and earnings page mainly based off of our data match with the employment department wage files. And to be included in those metrics, you actually have to have a Social Security number. And so being able to match more students with Social Security numbers have definitely increased our accounts in that regard and that will help us see a lot more outputs in the employment and earnings page. So please look forward to that when you don't want to renew dashboards.

OK. So a new metric this year is the immigrant integration milestone. We've actually replaced the old metric, which is that we had the EFL milestone. And it's not really a change in metric for say, but I categorize this as like a total revamp. It's a totally new metric, and it's come out of the work that's been done regarding the immigrant integration field. So we have a new data element from CASAS. It's called the Immigrant Integration Indicators Report. That's a new data element that just started in 1920 and that'll be available on the dashboard. And you'll see kind of a big jump in that regard because it's a pretty new data element.

The COMIS side of things, we're still mainly counting successful completion of the EL Civics course. And so with this change in the metric, we're really hope it aligns more with the Immigrant Integration initiatives that are happening in the state. And the other change we make here is that we need to track all participants who are completing this milestone, not just ESL students. Because we realize that Immigrant Integration may be more expensive than ESL.

OK. So a brief overview of some more changes. So here I've highlighted some of our what we call completion metrics or these are the metrics we use in terms of capturing community college awards. The big changes here is that we've brought in the universe to include all participants. That was really decision to match a little bit more with how CASAS was doing the reporting for post-secondary outcomes, as well as refining where we're counting students in terms of when they achieve the outcome.

And we've included either new award categories within each award or we've refined how we count. We refined how the courses are counted here. So in broad terms, we've made some small tweaks to actually a lot of metrics on the dashboard. I'm not going to go over them all here on the webinar. But in general, it'll be something along the lines of that right-hand column there. It's either we expanded the universe. We've added some additional courses or data elements that we've identified that might refine the calculations a little bit more stuff like that.

If you're very interested on all of the metrics and what changed, we have a link here so that you can access something called the changes and definitions document. That's where we've tried to capture for each metric what tweaks did we make and what were the high level impacts from the counts from last year to this year. So that's live on the dashboard. But you can also access it using this link here. So I'll show you where to find it on the dashboard in a minute. But that's where you'll find very detailed notes about what changed and why.

So how do these metric and data definition changes look on the dashboard? So you'll know that-- just know that the any calculations changes made, we actually apply that retroactively. So that the trend lines and the trend data will still be-- we'll still make sense. And so if I make a change this year, it'll affect all counts for all years shown on the dashboard. As of this year, we're only showing the academic year 2016, 2017 up to 2019, 2020. And that's because we wanted to align with when we started getting the K12 Adult Ed's students data from CASAS TOPSpro enterprise.

We started getting it in 2016, 2017 and hopefully the trend data will make a lot more sense going forward. Then some numbers will be updated in the metric, so keep that in mind. We have made improvements to how we display the drill down, so demonstrate that as well in a few minutes. And they'll be new metrics, as well as additional information to help you with your planning and learning about how your students are doing.

So before we do the demo, I'd like to talk a little bit about some of the new metrics, as well as some of the fun stuff that you see before, too, as you jump into the dashboard. So the first thing is the transition to post-secondary metrics. So for this metric, we've traditionally defined it as transitions to post-secondary and post-secondary in terms of the metric definition includes transition into CTE, as well as to credit college courses.

For this field, we broke up those two concepts into two distinct metrics. So hopefully that will help programs who are maybe more focused on one over the other. So now we have transitioned career technical education, as well as transition to credit college coursework, which are what I call submetrics to the transition to post-secondary. So hopefully, this will get you a little bit more refined information on transitions.

Additionally, we've included the top five institution charts. This is something that we've pulled over from the community college pipeline. On the left-hand side, you'll see a list of metrics. Those are the metrics that have currently a top five chart. The way the top five chart works is that if you want to look at-- if you want to see who the top five institutions are in one of these metrics on the left-hand side. The top five chart will display the institutions as individual bars.

And this is really based on the proportion of students attaining that specific outcome because we do know that smaller institutions do make an impact and we wanted to show how their impact is compared to the larger institution. So this is going to be a proportional difference. And you can see the top five at both the statewide level, as well as the regional level. It won't work at the consortia level. But hopefully, these two options will be helpful for you. Lastly--

Randy Tillery: We had a quick question in the chat. I want to confirm that I answered it the correct way. So someone is asking about the difference between the transition to CTE, as opposed to the transition to credit. The transition to CTE in this case is it just for noncredit or Adult Ed CTE because that counts as a post-secondary transition.

Jessica Chittaphong: No.

Randy Tillery: Obviously they could transition to credit CTE.

Jessica Chittaphong: It's CTE regardless of credit status.

Randy Tillery: OK. So I answered that wrong badly. OK.

Neil Kelly: Then currently, we were able to get to Jason's question about immigrant data.

Randy Tillery: That may have got by me. OK, Neil, do you want to read that out, Neil, so I can address it.

Neil Kelly: OK, let me find it. This is Jason from North Orange, is there a way to identify the number of non-immigrants among all participants if all participants will be part of the denominator?

Randy Tillery: We actually don't have a way to track immigrants separately from their general student population. Am I correct, Jessica?

Jessica Chittaphong: Yeah, not that I'm aware of. There's no like MIS flag for or like immigrant versus non-immigrants, I guess.

Neil Kelly: Yeah, we've done in the past-- sometimes cause us does this, I mean, they don't have a flag either, but they could look at, I guess language [audio out] at home or if they have out of country education credits or something like that from another country. But it's all speculative, there's no immigration button.

Randy Tillery: Yeah, it's really like you're trying to--

Neil Kelly: Tough to use it in that context.

Randy Tillery: In that product categories.

Jessica Chittaphong: Yeah. Oh, Karen did list a MIS element that does affect citizenship, but I think it would need to work with the causes data side as well so that they're kind of tracking the same thing. Thank you, Jason, for your question. All right. Any other questions? I think you're pretty good at answering these questions, Randy?

Randy Tillery: I'm going as fast as I can.

Jessica Chittaphong: All right. So the next thing I wanted to point out is our new tool tips feature. So we do recognize that sometimes these metrics can be complicated or can be a little bit difficult to understand. So we've tried to include an additional to the tip where you can click the question mark next to the chart description and get a little bit more information on that particular metric, and a little bit more about the background.

We're still prototyping it, so we would love your feedback about, which of these text box were actually helpful. What additional information would you have liked to have? This is very new is the first thing we've tried on the lock more suite in this type of effort to support users. So yeah, we'd love your feedback on how you find it, was it helpful? Next we're going to talk a little bit about drill down. As Randy mentioned, this is not new.

We do have three drill down active on the Adult Education Pipeline. For all metrics, you'll be able to drill down by demographics, race ethnicity, age, and gender. For some metrics, not all, you'll be able to see the program type breakdown. It's usually just the big four about basic education, your secondary education, career technical education, as well as English as a second language.

Also, some metrics we are able to separate out the first time versus continuing students. So first time really is the first time in any system both systems and returning is students who may have had taken an absence and came back. All of these three still don't work together. So if you select-- if you're on a metric that has all three and you selected a drill down for all three, they'll work together and display the information we're finding that way. All right, so we're going to jump to the dashboard now. And Randy, are there any questions?

Randy Tillery: Most of there is a whole series of questions about whether you can use citizenship status as a proxy for immigrant status. And there are immigrants, who are citizens like there's no precise way to say, yes, these are absolutely that all of your immigrants. So there's been some back and forth about that, but things are I think we're caught up right now.

Jessica Chittaphong: OK. All right, so we're navigating to the dashboard. Again, I'm just going to show folks how to access it on Google. I just Google launch board. Launch board also up here. If I click the homepage, it will take me to the launch board homepage where you can access any of the other launch board dashboards.

You can find the Adult Education Pipeline right here. We're going to explore. And it will take you here, and let me put in the link into chat again for folks who are-- oh, thanks, Neil. I don't have to do that. Perfect. So this is where we ended up. This is the adult education homepage. You'll see here that the tiles that Randy was mentioning. We have some lagging indicators on this page. So just to see if you want to see those metrics, you want to go back a year.

Here you can make your selections so we want to look at statewide data. If you want to look at your specific institution, what you can do here is go to the institution box and just start typing your institution name and it'll slowly populate with the list as you continue to type. So that's a quick way to access your institution. You can also look at the data by region, by consortia, as well as by Community College District.

So I'm going to stick with statewide right now. I'm going to go back to '18, '19 data just to show you that the homepage will repopulate once they click View with the new data. Now the homepage is just what-- just a sampler is what I like to say. Some of the metrics were pulled to give you a very high level view of the program areas. And if you hover over any of these charts, you'll see some numbers.

Now I'll go into a little bit more detail on the 30th about how to navigate and how to use each of these charts. But I want to save enough time for us to do Q&A. So I'm going to jump right into showing you what the new features look like on here. So if I jump into one of these tiles, I'm going to click View Progress. I'm going to be taken to the summary page. Now for those of you new to the Adult Education Pipeline, the summary page is what we like to call the infographic version of the dashboard.

It's just one or two of the most important metrics displayed in a very graphic way. And it's for folks who don't really want to dig into each metric individually. For folks who want to look at each metric, we go to the detailed data page. So this year has the metrics on the left-hand side. We're looking now at completed one or more educational functioning levels. You'll see here-- here's the chart. The default view is going to be the time frame view. So you'll see 2016, 2017-- sorry, 2017, 2018, '19, '20, and then '19, '20 at the end.

Underneath this chart, we'll also have a data table displaying all the years, as well as all the values for that particular metric. Here is that tooltip I mentioned. You can click here and you can see a longer explanation or a little bit more information about how we calculate this and what goes into it and stuff like that, as well as a reminder as to what a participant is.

So definitely utilize this chart if you're feeling a little bit lost or confused about what's this metric means or what it's meant to be. So there's that a quick demonstration of the drill down. So if we look at gender, you'll see the chart will update and show you a bar graph of the gender breakdown for the selected here, which was '18, '19. And so you'll see here the counts of females and males who completed one or more educational functioning levels at statewide level.

If you hover over here, you'll see the hover texts. And the way to read this drill down is that it's not-- the percentages are not going to add up to 100. The percentages are actually a reflection of the number of total female participants in the state. So the way to read this first line is that out of 239,645 female participants, 97,919 had completed one or more educational functioning levels in the year 2018, 2019. So that's 41% of the student population of participants were able to complete the EFL level.

So that's kind of how you read that percentage point that hover type. If I am interested in just the gender breakdown of my ASE students, I click here. And the chart will update and it'll tell me a new number and now I'm looking at the gender breakdown of ASE students who completed one or more educational functioning levels. And that'll keep going if I click first year or continuing. First time we'll continuing.

So there's that. And then, let's look at a top five chart. So let's look at top five institutions for a transition to post-secondary. So at the statewide level, you'll see here a list of five colleges, and hovering over we'll show you the counts and the percentage breakdown of that. Underneath you'll see as well the data table. And yeah, we can also change this to look at what are the top five in the region. So if I want to look at Sacramento, you can see here the institutions have changed and now it reflects the top high performing institutions in that particular county or in that particular region.

I'm going to go back to statewide view. Yeah, so going back to the technical documentation work where you can find the changes and definitions document is if you go all the way at the bottom, you'll get a link to general resources. But underneath the find out more, there's actually a little bit more curated resources as well. We tell you exactly when we incorporated, which data file into the dashboard. So if you had updated your data after these dates, then they may not be reflected in the data.

We also provide you with a link to the metric definition dictionary, as well as the changes in definitions document, which as I mentioned earlier will outline the exact changes and impacts made to each of the metrics from last year's bill versus this year's bill. So that resource for you. And for those of folks who are not interested in the data visualizations at all, I just want to download the data. You can actually do that using both the export data to CSV button here.

So clicking this export data link will download all of the metrics, as well as all of the desegregation into a CSV file for your selected fields here. So be for the statewide data at in the '18, '19 academic year. Last year we introduced the expert consortium data team where if you click here, it'll take you to a new page. And you can actually download data for all of the institutions in your consortia.

So by choosing your consortia here clicking all of the institutions. You can download all the metrics and all the bill down for this particular year into one CSV file so that you can look at all institutions in your consortium together. And then navigate back to the dashboard now. And let's see. Yeah.

And just for folks who are very interested in how we've calculated the data, but merge the TOPSpro enterprise together with the COMIS data, please refer to the metric definition dictionary. It has a lot of information. It has a list of the crosswalks we use how we identify institutions, which institutions are part of which consortium, as well as how we deal with desegregation, how we deal with suppression. As well as the calculations themselves.

So I'm going to stop my demonstration here and see if there's any questions on board. OK, looks like there's a question from Sherry, but you guys seem to be on it. If you're not an official member of the consortium, we may not be capturing. They have to be part of the data download that we get from CASAS, and we need to now to pull them from the COMIS side of things if that's the case.

Randy Tillery: There are we are entitled two agencies that are not CAEP members of consortia, so that we could be getting it that way as well.

Jessica Chittaphong: Is it feasible to update the data quarterly? I think there's a couple issues we come across with that, Crystal. I think part of the issue is that the COMIS data is updated in a schedule that may not fit with the quarterly update. So I know a lot of the student files are updated after each term, but the award files are updated annually and we do need both to fully build out the dashboard.

Randy Tillery: It would affect the integrity of the data you see, because it wouldn't really be summative for the quarter. Because of the way the data sort of comes in, I know that we are entitled two agencies and the adult schools report into CASAS quarterly until you get quarterly updates. But the way the college files come in would make the consortium data look very wonky. So we haven't really tried to do that.

At one point, we were discussing maybe doing a mid-year update or something like during the year, but we've never really gotten that far with it. And to be honest, to give you the best possible data some of the end year data is probably the best if we're actually planning purposes.

Jessica Chittaphong: OK. So there's no other questions, Randy. I'll send it back to you.

Randy Tillery: So we highly advise that you use this data for your consortium planning. One of the things to think of is the data isn't-- it is like I said, it's not a destination, it's a journey. And really data what it should do to make you ask more questions about what's going on with your students and how you're providing kind of service is also looking at where you're at from year-to-year can be really informative. Are we getting better in this area? Are we not getting better in this area?

Looking at populations can help you identify where you may be having gaps in services where you could actually think about, hey, often we are targeting these students to increase our impact. I was talking to adult school to remain unnamed about the fact that they're ESL students. Were mostly above 50 and some as old as 70 and their CTE students were all over 40. And so they were struggling with the fact that suddenly we're not really serving younger adults who could really benefit from say, CTE programs or good really benefit from transitioning to the community college when they saw how capital began to target more of those students.

And then when you're digging in with your three-year plans, it's going to be really, really important just to think very creatively about where you want to be around certain data points. So really sort of think carefully about this. These will be topics that will come up again on Friday about how to think about using the data for planning as sort of continuous improvement.

The other thing is maybe bring in your will of partner, your will title on partner, to work for sports and other partners to kind of sit down with your data and actually talk to you about it. And then finally, we do come out a lot into the field working with several consortia around their data right now and helping them understand it. Or if there's something that's really you're struggling with in terms of thinking about your data. We're more than happy to come in and actually help you with that.

We'll just sort of hang on here and sort of see if people have questions or more things you want to discuss. Like I said, we will be back on Friday. You could even go off mute in that question. Be there.

Doreen: I'll go up, Randy. It's Doreen.

Randy Tillery: Hi, Doreen.

Doreen: So I still don't understand the answer about CASAS. We won't have any CASAS data in the system until next year. Just so what does that mean about looking at the dashboards today?

Randy Tillery: Today. Yeah. So if you just started using TOPSpro Enterprise this year. So that will be part of next year's data set because basically the dashboard-- so the dashboard there right now is basically showing you 2021 data when you weren't using CASAS, right? So if you're using TOPSpro Enterprise, you're getting quarterly reports from CASAS. You could look that in comparison to your own MIS reports.

Now next year, the TOPSpro Enterprise data will be a part of what we look at in relationship to your consortium with two things related to the impact. So that one of which is we did duplicate between the TOPSpro Enterprise and the MIS data. So we don't count students more than once. There are some places where I think your accounts could shift. So I think EFL attainment if a student's not showing course progression or you're not using SD07 to actually report those EFL gains, we may see more EFL gains based on pre and post-test data.

Not a lot else in your data should change. I would say overall, though, in terms of impacts. Is that makes sense, Doreen?

Doreen: Yes, thank you. That help. I get it.

Randy Tillery: All right.

Neil Kelly: Randy, there's a question from Paul about you made a suggestion it would be helpful to have-- let's see suggests. My screen just went-- TA about how to make data displays on Patricks download.

Paul: And Neil, I can clarify. I hadn't seen that feature where you could-- that Jessica showed us where you can do the download for all members, all metrics. And the files kind of intimidating.

Randy Tillery: Yeah, it's not for the faint of heart.

Paul: And I know some data folks would probably do this in a snap. But I just know that some of the adult schools some of the consortium don't have institutional research capacity. But if there were just some helpful hints, so we could like put it in our three-year plan or whatever, you know.

Jessica Chittaphong: Yeah, I would recommend for sure, Paul, to come on the 30th. I know my colleague Blaire who is going to really lead the presentation is going to talk a lot more about how to use the data in a very student centered way. We also have a panel discussion plan for May 5th where some great volunteers from different consortia are going to talk about and share about how they use the data in the past. So that should give you some idea. So we can also develop definitely a research, resource on how to navigate those exports because it is intimidating.

Randy Tillery: Paul, you may want to-- I'm trying to by the best way to help you other than just doing that kind of webinar on how to build kind of infographics how to Excel or something like that. It might be worth partnering with like the researcher at Delta, which I know is your college there to kind of figure out what you'd like to pull out of the data see if they can and maybe kind of-- maybe they can help you with that.

Paul: That's a great idea. Thanks.

Neil Kelly: Randy, just to add. I don't know if it was WestEd or SCOE, but someone was going to take the info charts in the last three-year plan three years ago and update them. I think Greg Hill did those and we were planning to update them. But I wasn't sure if that was WestEd or SCOE was going to take that on.

Randy Tillery: Are you talking about the factsheets?

Neil Kelly: Yeah.

Randy Tillery: Yeah, so we will be updating those. We're actually working on the plan for that now. I would say, Paul, and other suggestions of what I do with those large crazy spreadsheets for other things is I just go through it carefully start kind of hiding the field you don't want to see like all the disaggregation to get it down to a manageable set and look much easier to build a chart out of a smaller kind of subset of the data. It takes some time, but that's the way I usually find myself having to grapple with that.

Paul: Yeah, cool. Thanks.

Randy Tillery: We have seven minutes left.

Jessica Chittaphong: OK. Well, if anyone wants me to walk back to the dashboard and show anything, I can definitely do that as well. But I can move on to the next slide.

Randy Tillery: So these are upcoming webinars. I'm hearing a whole series this year. So like I said this Friday is the understanding and using the Adult Education Pipeline for continuous improvement. We're going to be doing something on the fifth related to specifically how to use the dashboard for your three-year consortium planning. And we're doing a kind of a wonky thing on the 12th that we get this question all the time, why do my quarterly CASAS reports look different than the AEP data?

And it's not a fault with either data set says they're structured differently the quarterly TOPSpro data sets don't include all of the college data for your consortium, and so that's part of the challenge. We'll be going into that in more detail on May 12th. And then we're going to start, we're going to do something on the 19th, which we've been trying to do for a couple of years is thinking about how you align curriculum and the basic skills. So what is CB 21? What are the NRS educational functioning levels, and how to think about curriculum alignment?

And I was having a conversation with Kathy Booth this morning. We're talking about doing an entire series this next year, which basically is going to focus on, what does it mean for adult schools to potentially be a gateway to college? So it'll be looking at things like the relationship between what adult schools do and guided pathways looking at this kind of CB 21 EFL issue and how you can align curriculum to help students kind of make that leap to college.