Dulce Delgadillo: All right, welcome, everyone. You are in the Professional Learning Forum for the Understanding My Data. Happy Friday, everyone. That is the classroom you're in.

And we're going to get started in just about a minute or two. Just letting people in. I see still several coming in. OK. All right, good morning, everyone. Welcome. We're going to go ahead and get started. And I'm going to go ahead and hand it over just really quick to happy Friday and we're going to go ahead and get started, so I'll hand it over to Lisa.

Lisa Mednick Takami: Good morning, everyone, and welcome to our inaugural professional learning forum entitled, understanding my data. I'm Dr. Lisa Mednick Takami. I have the privilege to serve as a special project director for CC TAP powered by the Chancellor's Office and housed at NOCE, the North Orange Community College District.

We'll go ahead and start with our agenda for this morning. If we could move ahead. We'll start with some introductions of our team. We've had the pleasure to begin to get to know many of you either through these online convenings or in person at the CAEP Summit and other meetings. But we'll do some re introductions of our team for those who may not have met us previous or been able to join us.

Our agenda this morning is really understanding data. We're going to provide a contextualized, brief background as to what our scope is for today and then we will have some time for questions and discussion. And then we'll wrap up today with providing the field with some resources that we hope will be helpful to you.

We always conclude our webinars or anything actually that CC TAP does as part of the integrated CAEP TAP office with an exit survey. We really encourage you to read it, reread every single response. We take feedback from the field very seriously and we encourage you to do that. At this time, I will-- once we change our slide, then turn it over to my boss, Dulce Delgadillo, who will introduce herself and then our colleagues.

Dulce Delgadillo: Hi, everyone. Good morning. I'm excited to be here. My name is Dulce Delgadillo. I'm the Director of Institutional Research and Planning at North Orange Continuing Education, where CC TAP is housed. We are a standalone noncredit institution and we are lucky enough to have noncredit researchers on our campus. We are a research department that focuses solely on noncredit. And I'm going to hand it over to my colleague Harpreet.

Harpreet Uppal: Thank you, Dulce. Good morning, everyone. My name is Harpreet Uppal. I'm senior research and planning analyst at NOCE and for our North Orange County Regional Consortium for Adult Education. Nice to see you all. And Jason?

Jason Makabali: Morning, you all. I'm Jason Makabali. I'm senior research analyst over here at NOCE. I do MIS type things and numbery type things and database type things. Good to see you all here. Happy Friday.

Lisa Mednick Takami: Chandni would you like to introduce yourself and Pragyee?

Chandni Ajanel: Yes.

Lisa Mednick Takami: and Andy.

Chandni Ajanel: Sorry! My name Chandni Ajanel. I'm going to really quickly go over some housekeeping details. So first, as I've seen a lot of people in the chat do, we invite you all to put your name, role, and institution into the chat. Secondly, this meeting is being recorded and the PowerPoint and the recording will be shared on the CAEP website once they have been remediated.

Third, at the moment all the participants are muted. And once we reach the discussion section, if you have a question, you can raise your hand and I'll unmute you or if you prefer to type your question, you can use the Q&A function for that. And third, oh, no, sorry, fourth, we have the survey at the end, which Lisa mentioned. So again, if you could fill that out, I will be sending that at the end of the meeting. And I'll pass it over to Pragyee.

Pragyee Mool: Hello, everyone. My name is Pragyee Mool. I'm the administrative assistant of the Office of Institutional Research and Planning at NOCE. I provide administrative support to OIRP and I'm here to support the team with the webinar today. Andy?

Lisa Mednick Takami: Andy?

Andy Pham: Hi, everyone. My name is Andy Pham. I'm a project coordinator for the CC TAP team. And welcome. I'm glad that everyone is here today. And happy Friday.

Lisa Mednick Takami: Thank you. I'd also like to recognize a leadership from both the Chancellor's Office Mayra Diaz. I'm not sure if Cora is on with us. I'd also like to recognize our colleagues at the Sacramento County of Education, Mandilee Gonzales and Holly Clark. I'm not sure if Renee Collins is here and also our leadership from California Department of Education, Neil Kelly, and potentially Dr. Carolyn Zachry. Again, not sure if she's with us.

We are all part of the integrated CAEP TAP office with arms at Sacramento County of education, as well as at NOCE really working in an integrated manner to enhance services to the educational systems that serve over 480,000 students in the state of California. And with that, we will go ahead and transition into, what are these professional learning forums?

So I want to spend just a couple of minutes on talking about our purpose, our intent, our format, our expectations, what we are planning to do, and also what we're not doing. So the purpose and intent, as you see here on the slide, is to enable dialogue, discussion, and response to questions posed by participants in the field on data and accountability, programming, or other topics related to adult education.

Our intent is to create a positive, productive, and informative environment for practitioner practice and learning. Our professional learning forums will be monthly 60 minutes. They will always have a specified topic-- today it's understanding my data-- and there will always be essential questions linked to the topic. And we'll be covering those essential questions prompt discussion in the field a little bit later.

The PLFs will open with an engagement poll, which we'll be getting to next, as I mentioned, include a brief contextualized presentation and provide time for participants to ask topic-related questions. Each PLF will conclude with helpful resources or resources that we hope will be helpful to you and an exit survey to gather feedback for future sessions.

The PLFs are intended to inform attendees on a particular topic and the opportunity to pose questions to the CC Tap team. They are not intended to provide training per se or a deep dive on a topic. So if at the end of this webinar you may have additional questions about the LaunchBoard, you would reach out to the WestEd team for the adult education board.

You may have additional questions on TOPS Enterprise, you would reach out to the CASAS team. Or you may have additional MIS or other more generalized data and accountability questions and we strongly encourage you to reach out to our team. All of our contact information will be provided at the end. Thank you. Next slide.

OK, here is our opening poll. As Chandni asked you before, if you've not already, put your name and your affiliation in the chat, that would be great. We have just two questions on our poll. We'd like to get a sense of how long you have been in your role and what you're hoping to gain from this session today.

So we have a variety of ways in which we are addressing data accountability and training questions. And this just gives us an immediate snapshot of who is in the room today so that we can go ahead and contextualize our remarks to you in the field today based on who is actually here.

So today, we have a lot of folks who have been with CAEP for more than seven years, that's fabulous. They're veterans. We also see that we have some folks, 17%, who are newer to their roles, which means-- and we have 25% of folks who've been with CAEP for less than a year. So what that tells us is that some of the information today, someone like Neil Kelly, is going to be redundant.

But we also have folks who are brand new and we hope that this information will be helpful to you. I know when I joined CC TAP in May, there were a lot of things about adult education that were new to me and some that were known from previous roles that I've held in the community colleges.

So as I said, our goal today is to provide an overview on the topics that are presented. And then if you or your colleagues have additional questions, more detailed questions, we encourage you to reach out to us, to our colleagues at SCOE or to our colleagues at CASAS, or at WestEd.

I think we can go to our next poll question which is, what do you hope to get from the session today? You will definitely be receiving more information. We hope that information will be timely and relevant. We will be covering data basics. That is part of our intent for today.

We will be discussing college data. While we will not be providing formal guidance because that really comes down from the Chancellor's Office and CDE, we do hope to guide you as practitioners and better understanding your data, as well as best practices for data use.

We actually had considered telling the data story for NOCE, but we realized we were going to go over our time if we did that. You may see that as a subsequent session from us and we also are very curious to learn about your data stories because CAEP has a wonderful tradition of highlighting model programs. And I think model programming around data use, training, and accountability would be a wonderful topic for a subsequent PLF.

OK, understanding that is relevant and important to progress, how to collect and discuss data with consortium members, excellent. Further training in the value of noncredit, yes. Adult ed data, good. So I hope based on what I'm seeing in these responses that the way in which we have planned today's session will be helpful to you.

And as I said, if you need more follow-up, I've provided a couple of different ways to do that. And by all means, please provide us with your feedback. We keep marching out on the exit survey because we tend to have many folks who attend our sessions and only about 10% who are providing us feedback.

So the more feedback we get, the more that we're able to respond to the field. We're now going to transition into a background of the data story or the data context and I'm going to hand it back to Dulce. Thank you, everyone.

Dulce Delgadillo: Great, thank you, Lisa. Thank you so much, everyone, for sharing those two polls. It really helps us to really understand who is our audience. And we know that there are some people in the room who know the ins and outs of what is TOPSpro and what is MIS, but really, we're trying to provide a broad overview of just, why is it important? Why is data important in the CAEP land, in the CAEP universe for all of the consortium members?

And so really what we wanted to start with is the data and accountability requirements that come from the legislation itself. So we know that the California Adult Education program is very unique and special in the way in which it was constructed and envisioned where it was really a partnership between two systems that are already in place and already have data systems in place for themselves and really serving these students across both systems in an effective manner.

And so when legislation was written in for our veterans out there as AEBG, the Adult Education Block Grant before it was shifted over to the California Adult Education Program. Legislators were very intentional and saying we are going to measure certain things out of the results, out of the implementation of this program and the money that's been invested in this program with specific metrics.

And so that's what you see in front of you right here. You see that no later than July 1, 2019, this was assigned as part of the consortia of how our legislators and leadership at the state level was going to measure and assess the effectiveness of the programs that were being implemented and executed through the consortium, through the CAEP programs at this point, known AEBG.

And so really it comes down to these two bullet points. You can see that two is much longer than one. But first of all, how many adults are being served by the members of the consortia? So just in general, how many individuals are being served through the consortium?

And then how many are served that have demonstrated the following skills, improved literacy skills, immigrant integration in areas consistent with, but not limited to the following completion of high school diplomas, completion of post-secondary certificates, placement into jobs, improved wages, and transition to post-secondary? This was added by the state later.

But what it comes down to is that these were very intentional right from the beginning of the inception of the adult education program. And so really when we're looking at our data and we're looking at, what are we? Seven years down now, five years down, and looking at what is being displayed in the LaunchBoard and given to our legislators, it really ties back from the inception of tying it back to this legislation specifically.

In addition, we wanted to provide that in the legislation itself and we provide two very helpful resources, which is the compendium-- a tongue twister-- of allocations and resources. So this is a great resource here that we're going to be sharing that is in the slides. That is a resource that says all of the categoricals.

So in the California Community College system specifically, we have our general fund. But in addition, we have our student equity and achievement program, which is categorical. But we also for adult education, we have our strong workforce, which is categorical. We have other data sources, such as-- I'm sorry, other funding sources, such as WIOA or Perkins.

And so what you can see through this resource is what those allocation amounts are specifically for your institutions. And so you can really start diving into, what is it? And where are the resources that we could potentially tap into within our community college system to be able to leverage.

So I'm going to go ahead. This is a very, big document. So I'm just warning you. If you choose to print it out, I believe it's around 290 pages. But this is released out every year. And this is really an outlay of what is being allocated out to each of the community colleges around.

And you can even go specifically into the CAEP area. You can see how much was that allocation and you can even see, what is the funding formula? What are the spending guidelines? And we have found this to be a very helpful resource as practitioners ourselves, as part of a consortium.

You can also see the allocation and who is the direct funding or the fiscal agent and the allocation for that actual year as well. So again, this is released out on an annual basis. And I encourage you to share it out because it has been a useful resource as a community college system participant to be able to leverage some of the resources that are also being allocated to serve the student population.

The next one that I'm also going to share here is going to be our CAEP welcome letter. So in this welcome letter there is conversation or there is narrative around specifically the data accountability requirements. And so you can see here what those requirements are for our K through 12 agencies, specifically around TOPSpro enterprise and then specifically how that data flow comes in for community colleges.

Again, CAEP is unique in the sense that we are working in partnership with these two systems that already have data infrastructures for those two systems. And so really today we want to talk about, how do those data systems intermingle to eventually feed into the LaunchBoard?

I don't know what I did. OK, all right. And so that's where we get here, right. So this is a visual of really what we're trying to communicate and big picture. So we know that there are some CAEP partners, community colleges, and K through 12 partners that are using TOPSpro Enterprise.

We know that community colleges are using the COMIS data, the Chancellor's Office Management Information System data, that is our MIS system. And we know that there is a process called matching where you're going to be looking at both of these systems and you're going to even be bringing in some third party data systems such as the ADD wage file, because we know that going back to the legislature, we need to measure those wages and placements into jobs.

We're looking at National Student Clearinghouse for transitions across and student pathways and again, tying it back to the legislation because we know that was the last one as one of the legislated outcomes that should be assessed. And we know that all of this intermingling of this data from both systems, in addition to third parties, are feeding into the California Adult Education dashboard. And we'll cover that a little bit.

But ultimately, on the right-hand side, what you can see is that the AEP dashboard is made up of multiple data sources, not just one. And so when we look at our data at a consortium level, it's all of our data that's intermingling and for us to understand that there are these different data sources that we really need to dive into in order to understand why is it that our data looks the way it does in the LaunchBoard. All right, I'm going to go ahead and hand it off to my colleague, Dr. Harpreet Uppal.

Harpreet Uppal: Thank you, Dulce. Good morning, everyone. So before we go into this slide, we do have a Zoom poll and I'm going to ask Pragyee if she can launch it, please.

All right, thank you for participating. I know we have a few other that are still answering the question. But as you can see, we do have several members have indicated that they have not utilized the dashboard and/or have not heard about it. So our goal today at least is to get you a little bit familiar with the Adult Education Pipeline Dashboard.

And as Dulce and Lisa had mentioned earlier, we won't be able to go deeper into it, but WestEd produces a lot of training around Adult Education Pipeline Dashboard and I know there would be new ones coming forth. So we really encourage you to attend those when they are out into the field.

But we will briefly just cover it. So at least you when you leave out from today's presentation, you get some sense of what is in Adult Education Pipeline Dashboard.

Lisa Mednick Takami: Harpreet, I'm sorry to interrupt. I just saw someone in the chat say that they were not able to see the poll results. Would you like to go ahead and read the results just so folks know?

Harpreet Uppal: My apologies. I didn't even click the show results button. But you should be able to see now. Could you see now?

Lisa Mednick Takami: OK, excellent. Thank you.

Harpreet Uppal: All right, yes. So 40% said yes, 39% said no, and then 21% indicated never heard about it. So our goal is that as we're talking about data, we encourage you to start looking at your data, your consortium data, your institutional data on these dashboards.

And we can put a link in the chat as well where you can go in and actually access the dashboard to see your local data or your data that's reported to the state and is reflected on the dashboard. Thank you. Sorry, Dulce, could you go back to the slide, please?

I just wanted to mention-- so this is again relating back to what Dulce had covered, the data that needs to be collected and reported per legislative guidelines. So as you know that this is a statewide data dashboard, so those that are familiar with it and that it houses data around the metrics that are really called out in the legislation. So about our successes, our progress like EFO gains on transition outcomes, employment outcomes.

And this is also the data that gets back reported to the legislature. So it's very important to look at your data to see what's being reflected for your institution or for your consortium. Next slide, please. And this is just another visual. We have used this visual in our local presentations with our consortium members and it's been helpful to just lay out.

So again, we just broke it down into the same tiles that are on the AEP dashboard to just indicate, what are the specific metrics that align with the big buckets like students and programs, progress, transition, success outcomes and employment and earning outcomes?

And again, we won't go deeper into each of these just because I know there have been a lot of trainings done on this and there will be future trainings most likely coming when the new AEP dashboard, the data is available for '22-23. And next slide, please.

And lastly, I just wanted to point out about what actually happens in the dashboard in terms of matching. So as Dulce had mentioned, data is collected from multiple data sources. And CASAS TOPSpro and MIS are two data sources where most of our data is being collected from and reported from.

So several colleges, especially they submit their data through both TOPSpro if they're funded and MIS. So in the LaunchBoard, the programmers behind the LaunchBoard. They basically use what's called "a drive key" to match student records. And the reason we wanted to point out this is that it's really important for our institutions to collect and report data so that when students are matched, if there's anything missing, then that record gets excluded from the denominator.

And we want to make sure all the students that are being served are being reflected on these dashboards. So it's very important that we collect all of these key pieces of information and report them. So thank you. And next slide, please. And Jason is going to go further into some of the specifics around LaunchBoard.

Jason Makabali: So considering that our topic of today is understanding our data, one way of understanding the data that is populated in the LaunchBoard is by using the metric definition dictionary abbreviated as MDD. It can be accessed by clicking on that link that is shown in the top right corner of the dashboard. It's on every page of the dashboard. And the direct link itself, I just dropped it in chat for anyone who is interested.

But it basically gives a descriptions of how all these different metrics are calculated for the Adult Education Pipeline using both TOPSpro and MIS data. So next we'll go into what it looks like so that you can understand how it looks like for your TOPSpro data.

Basically, WestEd has done a lot of work on this to provide a detailed descriptions regarding how exactly they look at the cost of data that is provided and how it translates into the Adult Education Pipeline LaunchBoard. In this case, for example, to identify participants in a program, they outright tell you that you just select the bubble in which the student is participating in the update record or you fill it in the update record that is in CASAS.

However, for the MIS definition-- next slide, please-- it's a little bit more complicated. It takes many variables through MIS, across multiple MIS files to define this. So it does take more of a little bit of a research perspective possibly or a deeper understanding of MIS data to, in a sense, disentangle these calculations to identify them.

So just a cursory look at how you can actually identify things that it is possible to identify, how these numbers are being calculated, and where exactly they're coming from. Next slide, please. And I think it's back to Dulce for the discussion section.

Dulce Delgadillo: Yeah, great. Thank you so much, Jason. So that's just a quick overview of just very basics of-- we can definitely dive into each of those topics that we each covered into rabbit holes, but really giving you just some basic concrete tools right there with the DERKEY making sure that your data is complete with your first name, last name, date of birth and gender.

Those are four variables right there, four data fields that you can start with to making sure that there is matching happening across your consortium, across these two data systems that we're really trying to merge data with and have it be reflective of the work that's happening on the ground right through the CAEP program.

So we're going to go ahead and shift gears in terms of the agenda. And we're going to open it up for some discussion. So I'm going to go ahead and hand it over to Lisa.

Lisa Mednick Takami: Yes, so as we move into our discussion, I'm going to ask Andy from our team to share some Community Agreements. These are agreements that were put together at Sacramento State, but also reflect a community agreements that are used in a number of DEI spaces. So Andy, if I could ask you to share our community agreements, please.

Andy Pham: Yes, hi, everyone. We just want to make a safe environment for everyone to share their ideas and thoughts. So please practice the art of calling people in, not calling them out and understand the difference between intent and impact. And be open to learning and unlearning.

Practice using the words and instead of buts. And stay engaged, but be open to share your own truth. And in this webinar, we'll be focusing on two essential questions. So if you have any additional specific questions, please feel free to type your question in the Q&A chat or reach out to us by email.

Lisa Mednick Takami: Perfect, Andy. Thank you. Andy is very modest and I just want to give a shout out to him because when you contact CC TAP for a specific technical assistance request, Andy is the person that is both logging it and disseminating it either to our team or to other experts in the field.

For those of you who were in our, I believe it was a November webinar, for the first time we shared a form because we are gathering field expertise and really trying to put together a pool of field experts. We will share that, again, in the next PLF, but we want to make sure that Andy is right on the front lines receiving your technical assistance requests when they come in. So next slide, please.

Our first essential question for today is, how is data for your adult education and noncredit programs being captured? So you can go ahead and put that in the Q&A. And if you have questions that we can address, we're-- Dulce has another essential question, but we first want to get some ideas of how your data is being captured.

We in the seven months that CC TAP as an arm of the integrated TAP office has been in operations, that folks have access to a variety of different data sources and folks on the ground who are doing the data collection analysis varies quite a bit across the consortia. So do we have some responses?

Dulce Delgadillo: So we have some at the agencies by staff there. OK, thank you. And you can also unmute yourself. This is a learning community. We encourage you to please share your data story as to where you are. We know that everybody has different infrastructures and different support systems and we're really trying to take a pulse of what is the status of this infrastructure out in the field.

Lisa Mednick Takami: Yeah, so if it's easier to unmute and just say a little bit about your situation in terms of data capturing or who is responsible for that, feel free to do that. This is the time.

Molly Stimpel: my name is Molly Stimpel. I'm in the far north with Shasta-Tehama and Trinity. We happen to have had a Q2 practitioner meeting yesterday and this was part of the meeting agenda. And one of the things that was fleshed out is that while every agency is responsible for reporting as a member.

Not everybody is reporting the same way, not everybody is defining and understanding what should or should not be included or how to include that. So that would be part of my work is helping to guide the facilitation of coming together with that shared understanding so that it's accurate information.

Lisa Mednick Takami: Thank you, Molly. That's helpful. I see that we have a raised hand and we also had a comment from Ventura County. Let's see. We have a shared consortium data manager who works with all sites in obtaining information to submit to the state. A piece of that is working closely with the college's institutional research to capture data from MIS.

This is also from Shannon Eller. We also have quarterly surveys that all students complete to make sure we are not missing any outcomes, excellent. Vincent Chambers through Google Sheets then into TOPSpro, Kathleen Carbone ASAP TOPSpro done by teachers. We are a small program to teachers. Also, our consortium data specialist gives great assistance. This is terrific to know.

Nancy Miller, TOPSpro for adult schools and one community college MIS for the other CC. Some agencies have coordinators and others use teachers or administrators to collect data. This is Tonya Bibbins, "In Waterford, we use TE, which is shared used at consortium level regularly. Data is shared biannually with the school board."

And let's see here. TOPSpro for adult school. Sorry, I already read that one. Let's see. This is a question. TOPSpro for adult schools. TOPSpro data for instruction is captured. TOPSpro MIS and CASAS. But employment is first captured in CalJOBS by our workforce board, this is MaryAnn Pranke, and then given to us to update our TOPSpro MIS. I want to be able to honor Beth, who has her hand raised. Beth, go ahead.

Dan Cutler: I'm going to actually speak. My name is Dan Cutler. I also work with the Monterey Adult School. Some of the things that's been a little tricky for us is the inconsistency between the data with ASAP TOPS Enterprise and the LaunchBoard. So at times we're not exactly sure which data to focus on.

Also I know that the LaunchBoard at times is not completely-- like it might be a year behind or something to that effect. So it's just been a challenge for us is trying to find which data to specifically focus on. Obviously, we've been-- how would you say? Looking at all the data and trying to come up with what we feel is most appropriate, but just wanted to put that out there.

Dulce Delgadillo: Yeah, I think that's a really important point because we have similar issues at the community college because we're entering it in our banner system and we have TOPSpro Enterprise as well. And so we have to do somewhat of a reconciliation between the two systems to make sure.

One of the things that I would take into consideration is how often your data is being updated in each of the systems. So if one is being updated every once-- just once they entered the school and maybe that was five years ago versus you're collecting TOPSpro on an annual basis, then I would take the TOPSpro data because that's going to be most up to date.

So timing is going to be one of those pieces that you want to take into consideration, but also processes. And this was one of the cases, I believe, about four or five years ago for our consortium where we saw a lot of missing data in TOPSpro Enterprise. So we had to proxy it.

So it really took some data trainings to tell and communicate to those who were entering the data, again, why is this important? What are the fields that you are priority? First name, last name, date of birth, and gender, that was our big message to them. Focus on these because these are going to make-- these are low hanging fruit that we can focus on.

And then we can start prioritizing. For TOPSpro you have those reports that show how much data is missing. So that could be one tool and then-- but also seeing what that looks like at a local level. And so definitely, if anybody else has those experiences as an adult, this is the space to share those best practices as to, how are you prioritizing which data system to use?

We've also grappled with that. But typically, we look at timing, how often that data is being updated, and then what are some of the processes that are in place locally to enter that data?

Dan Cutler: Thank you.

Lisa Mednick Takami: Of course, I think in the interest of time we'll go ahead and move on to our second essential question. Just to let you know since we had a question someone asked, "How can we see all the questions and the answers?"

One of the members of our team is going to capture all of that. And we will send that out I'm guessing probably through the Listserv as early as today. So just rest assured, we're gathering information and we will be sharing it with you. So our second question, Dulce.

Dulce Delgadillo: Yeah, here we go. So the next question is, what area of data reporting does your campus find most challenging? So we know we're capturing data in various aspects of the student journey. Application data when they're coming in, it could be when they're doing the assessment and we're gathering that data on the onboarding piece.

We know that TOPSpro Enterprise is also done at the end at the post. So there's different timings and then there's also different types of data that you're gathering, student services versus academic if they're receiving a specific student service, exactly, yeah.

There's also the employment and earnings survey, exactly. So we know that there's all-- the data is being intermingling with all of these data sources. But we wanted to make sure and bring it up here to see, what is the most challenging? I see, yes. So the employment and earnings survey, Connie, agrees, employment data.

Yeah, we see, OK, from Kathy Walker, yeah. I see, yes, with lots of exclamation points. Yes, and we know for employment data it is being captured in different ways on both end and it's bringing in that third party. One of the pieces that we have also heard is, how do we-- again, we're pulling numbers here and it's not appearing in LaunchBoard.

And so we know that WestEd is doing those calculations and is leading those efforts. So we know that there's a resource there where you can match up and say, hey, what's not coded correctly and why isn't it being pulled in. Or what is the issue? And maybe, is it a timing situation? So WestEd is definitely a resource. And we also have other resources with CASAS. We hope to share those at the end and actually in a couple of minutes. So we are looking at employment and earnings survey, again, data for instruction.

OK, data for instruction is captured in TOPSpro, MIS, and CASAS. But employment is first captured in CalJOBS by our workforce and then given to us to update our TOPSpro and our MIS. Mary Anne, can I ask a question? Where are you updating the employment data on the MIS?

Mary Anne: I'm sorry, I don't enter the data into MIS. So I'm not sure. But yeah, I entered data into TOPSpro, but I'm not sure exactly how they do it over at MIS. But I do send the information over.

Dulce Delgadillo: The employment data, right?

Mary Anne: Yes, I do send the information over. So our workforce board, I mean, obviously some students will get a job on their own when they complete a program and so if I have that information, then I enter it.

But our workforce board is our primary source for job placement and they do the paid internships and paid externships that they fund with their own dollars. So they're tracking those students in case managing those students because they're providing job placement assistance for them. And also because they're providing that, it's a lot easier for them to get the Social Security numbers than it is for us.

And since I'm housed at their site and I also work under the workforce board, so I am 65% adult education and 35% workforce, so it's easy for them to just walk the information over and give me the reports. And then I enter the Social Security numbers and the placement information in TOPSpro.

Dulce Delgadillo: Yeah, very manual it sounds like.

Mary Anne: Yeah, and then I sent it on to MIS so that they can update any other records, yeah.

Dulce Delgadillo: Yeah, thank you for sharing that, Mary Anne, sharing that small little tidbit of your data story. And that is common that we have heard very manual processes and what does that look like? So what I'm seeing in terms of trends in report here in terms of the most challenging, definitely employment data. That seems to be right at the top, employment data.

And taking into consideration the Social Security number and how is that being matched up with that third party, the EDD data that's being matched up for the LaunchBoard. So the other piece I'm finding here or I'm seeing here is student updates with students who don't provide information.

So we definitely have some tools, LARS if you want to reach out to us, because we had that issue as well where we had a proxy. We had a proxy and say, OK, well, that information is updated in this system, OK, but we need to put it into here. So how are we going to proxy and how are we going to measure those things and make sure and again, knowing, is this updated data?

And then I see here also some student and employment. And then the other one that I have also seen to be at the top is transition results. What is it? And how are those being calculated? And so definitely this is definitely some guidance for our team at CC TAP that we need to focus and really provide some clear resources around how consortium members can really dive into their own data across these main themes.

So I'm going to go ahead. Thank you so much, everyone. We are at the 10:45 mark. And we're going to go ahead and transition over to sharing some resources, some tangible tools that you can use to start diving into your data and start having that narrative and that dialogue within your consortium as to how you want to use data and how you want to have a conversation around your consortium data. I'm going to go ahead and hand it over to Lisa.

Lisa Mednick Takami: OK, great. Thank you, everyone. We appreciate this robust discussion. We hope it is as valuable to you all as we found it. I'm now pleased to turn it over to my colleague, Mandilee Gonzales from the SCOE arm of the CAEP integrated TAP office, who is going to give a tour of what we hope will be navigational tools for finding resources on the CAEP TAP website. Mandilee.

Mandilee Gonzales: Hi, everyone. Thank you, Lisa. So I'm happy to be here today. I'll go ahead and see if I can share screen. If the host can enable the screen sharing?

Dulce Delgadillo: Yes.

Mandilee Gonzales: OK, I didn't know who it was.

Dulce Delgadillo: Are you good to go, Mandilee?

Mandilee Gonzales: I am OK.

Dulce Delgadillo: OK, great.

Mandilee Gonzales: So, Mandilee Gonzales, for those of you who may not have met me yet, I am the newer coordinator for CAEP TAP here at the SCOE arm and I appreciate that my colleagues over at NOCE invited me to be here, walk through some of the resources that we have housed on our cal.ed website.

So I know I'm sharing this time with our partners so I'll be really quick. For those of you who haven't visited the website, it's here cal.ed.org. And I'll also go ahead and pop that into the chat so you guys can also follow along if you'd like or refer back to it later on.

So while you're here, we do try to keep all of the most updated information trending this month. I think for a lot of your data pieces our administrators will find most useful our administrators tab. Here on our website, here along the left hand side is the navigation bar. And all of these various buckets. That's where you're going to find a lot of your different tools, as well as recordings of past webinars.

So some of the areas that I did want to go over with you are going to be the planning section. So here in the planning section where you're going to use all of the data sets that the NOCE team has been talking about, you'll find your annual plan. You can pop over to your CAEP 2021 fact sheets.

And as just a quick update, I'm sure it's been mentioned, but you also-- the fact sheets are being worked on by our WestEd counterpart and once that's updated, you'll also see the newer fact sheets here and we'll also share that out with you. So as you can see, there's a lot of resources here for you to come and dive into.

Other areas that are helpful are going to be our guidance section and our governance section. So if you have any governance questions, you can go here, as well as our webinars. Annual plan in three-year plan, which is now on the top of everyone's mind, which is right around the corner.

And one of the other areas I really wanted to pop out to you is the student data collection. And that is here under reporting and your student data collection. I think it was an area that our colleagues popped up. So here you can find PowerPoints, you can find recordings, as well as the beginning of the year letter.

So if you ever need to reference it, you can look on the most current as well as some of the historical beginning of your letters. All right. so then I'm going to go ahead. And I also want to highlight a couple other areas here. So up along here on our menu bar, for those that dive into NOVA, you have a quick button here that'll launch you right into NOVA and then LaunchBoard.

Here in LaunchBoard, this is going to take you to your pipeline, your community college pipeline, your Adult Education Pipeline. This is really that Cal-PASS where you can find a lot of those data sets that people have been sharing and talking about.

And I believe Lisa mentioned, if you ever have any questions or if you need additional support, you can always direct email us, or you can also hit the request support button here and then you can submit a support request. Either one of our teams will get back to you if it is specific to NOCE arm and all of their experts over there, we definitely forward those requests on to them and they reach out to you directly. And that is, I think, my time. That's all I really have to share with you guys.

Lisa Mednick Takami: Thank you, Mandilee. Awesome job. I did just see a question in the chat about, where will our webinar recordings be housed? And they will be housed on the CAEP website. We may have one other way of getting them out to you. There will be under thank you, Mandilee under that webinars tab specific other menu choice for CC TAP webinars. We're in process of updating that. So that will be on its way.

So thank you, Mandilee. I'm now pleased to introduce everyone to Nicole Jordan, one of our colleagues from CASAS who will do a similar tour of the CASAS of the TOPS Enterprise. Thank you.

Nicole Jordan Clark: Hi, my name is Nicole Jordan Clark.

Lisa Mednick Takami: Sorry.

Nicole Jordan Clark: No worries. I'm a senior data analyst here at CASAS. I'm going to share the multitude of resources that we have available for you. So for starters, we have our dedicated web page for California resources and then one specifically for our CAEP resources.

This information includes information on how to order TOPSpro Enterprise if that's not something that you have already. For those that are required to submit the data through TE or TOPSpro Enterprise, we have information on data submission and using the Quarterly Data Submission Wizard.

We also have guidelines on how to complete the employment and earnings survey. And then we also have a couple more resources on that employment and earnings survey that we'll be posting as we get them throughout the year. If your agency is administering EL Civics or participating EL Civics, we have that resources posted on this page.

And then we also have a link to the WIOA and CAEP Data Dictionary. There's also a link to that will link back to the CAEP website for that beginning of the year letter too. And it shows how we're working together with the CAEP team. Now, for those that are administering eClasses, eTests, or any CASAS assessments or are interested in administering those assessments, we have our CASAS eTest page.

Information here can be found on how to get started with CASAS eTests or CASAS assessments, whether you're going to be administering paper-based assessments or all through computer online information on remote testing because I know that's helpful for some of the agencies that are in more rural areas.

Information on system, information that may be your IT group needs to get a hold of so that they can implement tests or TOPSpro Enterprise. And then we have a list of various assessments and details on those assessments, including their NRS levels, the new CCRS and all that stuff associated with the assessments posted on here.