[relaxing music]

Speaker 1: OTAN, Outreach and Technical Assistance Network.

Anthony Burik: All right. So officially, hello, everyone. Good morning. Welcome to our session, California adult ed technology and distance learning, a look at the data. My name is Anthony Burik. I'm the coordinator for distance learning projects with OTAN. And I want to introduce my co-presenter Matthias.

Matthias Sturm: Oh, hi. [chuckles] Thanks, Anthony. I'm an adult education researcher. I'm doing my PhD at Simon Fraser University that's in Vancouver, Canada. And I work also for other projects with the Canadian immigration authorities and language programs. And I'm sorry, I should say, and Anthony and I are working on the annual distance learning report that we're sharing some preliminary data with you today.

Anthony Burik: Yes.

Matthias Sturm: Thanks for having me.

Anthony Burik: Yeah, thank you, Matthias. And welcome from Canada, our neighbor to the North.

Matthias Sturm: Thank you.

Anthony Burik: OK. So here's our agenda for the next hour, just a little bit about OTAN. And then we'll talk about the distance-- technology and distance learning report. We also want to talk about the California Adult Education Digital Learning Guidance. So, hopefully by now, you're familiar with it. You've heard about it. Maybe you've looked at it. Maybe you read sections of it.

But we want-- we want to talk about the guidance today as well. We're going to take a look at the data. It's a slice of the data that we're going to look at today. We're still actually putting the report together. But we want it to give you just a sense of some of the data that's going to be included in the report.

And then we'd like to do an activity both with folks in the room and for those of you online, give you a chance to get into smaller groups talk about some of the data little bit more intensively, see what questions you have, what things come to mind when you look at this data. We're interested in getting some feedback from you all on your reaction to the data that we're going to share today.

As we go along here we'll take questions. But also if there are questions towards the end, we'll address them at that point as well. And so I do have a bit.ly link here. So in the folder there's a copy of the slides that we're going to look at today. And then also the-- when we get to the activity, our activity documents are going to be there as well. So these resources will be available after today. So feel free to take a look at them, come back to them later at some point.

Matthias Sturm: Great, Anthony. I put the bit.ly link in the chat for the online folks.

Anthony Burik: Great. Thank you, Matthias. Appreciate it.

Matthias Sturm: Thank you for setting it up.

Anthony Burik: Yeah, sure. OK. Just a reminder about OTAN, so we are one of three state leadership projects in California that support adult education. We focus on technology application and integration.

And for this particular report, we actually do a lot of work with our partner Casas. So Casas says collects data. They're the data folks, the assessment folks, the data accountability folks. And so a lot of the data that we get comes from Casas. So that's one of our data sources, and we'll talk about that a little bit later. But we just want to give a shout out to Casas for the work that they do that benefits us when we put the data-- the distance learning report together. OK, Matthias, over to you on the report itself.

Matthias Sturm: Absolutely. I'm just also setting up the transcript here, so just-- so everybody knows online, there is also the live transcript available. Fantastic. Thanks, Anthony. So starting in 2001, adult education agencies submitted what became the annual Technology and Distance Learning Plan, or TDLP for short. It was meant to capture an agency's ongoing and proposed technology integration goals.

There was also a self-assessment of teacher technology skills, and a learner survey on technology access and usage. The aggregated information from the TDLP was-- has been included in OTANs annual reports since then.

In the 2016-17 OTAN annual report, there was a comprehensive distance learning report for the first time. It took a deeper dive into both the TDLP and the National Reporting System, NRS for short, the data from there. In the annual report since then, TDLP and NRC data has been included and reviewed with more in-depth analysis.

In the last few years, recommendations for further research were also added. And recently, the TDLP was incorporated into a new reporting deliverable known as the Continuous Improvement Plan, or CIP for short. Agency technology goals and the teacher and learner survey data are gathered for the CIP, and they are still key elements of distance learning reports. For the last report, Penny-- and shout out to Penny. Is Penny in the room?

Anthony Burik: She is.

Speaker 3: Yes, I'm here.

Matthias Sturm: All right. Fantastic. Good to see you. So, yes, for the last year's report Penny and I made some changes because we wanted to move beyond the quantitative analysis. And so we added some qualitative findings from focus groups with selected agencies about what and how they were making the shift to online and back to in-person or back to blended.

For the last two years, the report has also referenced data from prior years, provided comparisons, and offered insights into some of the new delivery models such as HyFlex options. There were great presentations yesterday. We included data from the Program Implementation Survey and added findings from the student technology intake survey.

This year instead of the focus group, Anthony, Mia and I decided to do a survey with all agencies, so that we can cast the net wider and get a better sense of where things are heading. You will see some trends in the data that we're sharing with you today.

This year's reports with findings from the last program year will be available in a little bit later in the spring. We are sharing some preliminary findings with you today, and we will use these for our discussion today. We may also mention some of the discussion points from today in the report, that's why we will be asking you to take notes in a shared Google Doc during the discussion. And that goes both for online and for the in-person participants of the session.

After we have a look at some selective findings, you can join one of those three groups in person or in the breakout rooms. And Anthony and Penny and I will drop in to see how things are going. But first Anthony will tell you all about the guidance. Please, Anthony.

Anthony Burik: Thank you, Matthias. So we just wanted to make a reference to this new document that has appeared in the fall of last year, the California Adult Education Digital Learning Guidance. And so-- again, hopefully by now you've had a chance to take a look at it. You've heard about it. You know how it's organized. You know what the guidance is about.

Basically, it's really meant for the adult education field to have a sense of all of the different topics that are related to digital learning, learning online in blended environments, in HyFlex environments, and a lot of different things that go into digital learning, the state of digital learning. So we actually-- on the OTAN website you can get a copy of the guidance from there, we're actually building out our digital learning guidance website, and we hope to add a lot more information there as well.

So if you are from-- well, if you are, but if you're not familiar with the guidance this is basically how the guidance is organized. So there is an introduction. And then there are six chapters, which form the core of the guidance. And you can see the topics that are listed here.

And so Matthias and I when we started looking at this year's report, distance learning report, it's hard not to notice the fact that there's now this guidance document that's been created that's very extensive. A lot of folks in the field actually contributed to the guidance document. And so one of the things that we're trying to do starting with this year's report, is think about the data, but in a different framework. And that framework would come from the Digital Learning Guidance document itself.

So for example, you may notice that I bolded chapters 2, 3, and 5 here. So some of the data that Matthias is going to talk about these topics are mentioned in the guidance document directly, right? So for example, a lot of the student data is interesting when you look-- take a look at chapter 2 which talks about ensuring digital and digital equity and access.

So this is really like what is the state of students? What kind of devices do they have? What is their ability to connect with online learning, and blended learning, and HyFlex learning? What digital skills do they possess? All of you face that on a daily basis. What are the skills that students either have or don't have? And we're finding that the data directly corresponds with the topics that are covered in the guidance.

So starting with this year, we'll see how far we can get into it. But we really like to create a tighter connection between the data that we collect for this report and the topics that are mentioned in the guidance. So that for all of you, it will give a lot more hard data on these topics that are covered in the guidance.

So that's the new framework that we're operating under. And as we get more into our reporting, we find that probably we'll be able to cover all of the chapters, maybe not the last chapter. So the last chapter of fostering healthy, equitable and inclusive digital communities talks about topics like for example, like social-emotional learning is mentioned in that chapter.

I'm not sure if any of us are collecting data on adult education in SEL at this point. But maybe in the future, that might be a recommendation that we come up with. That we may find that we really aren't collecting data on chapter 7 topics. So maybe that's a new direction that we need to go in terms of data collection.

But again, we're really trying to put these two things together in a much tighter format. So more to come on that when the report comes out, as Matthias mentioned later this year. OK. So, Matthias, I'm going to turn it back over to you. And Mathias is going to talk about some of the data that we want to share today in the presentation.

Matthias Sturm: Great. Thanks, Anthony. And what I'm sharing now you will also be able to see in those Google Docs that we set up for the discussion groups, and of course in the slides that are in the shared folder that we shared a bit.ly link to before. So what you see here, the charts, and this slide, and the table on the next slide show the combined adult student enrollment for regular classroom and distance learning students for the program year from July 2017 to June 2022.

Last program year, the number of distance learning students declined by 24% while the number of students in regular classrooms increased by 28% compared to the previous year. You can see that in the chart nicely here with the blue line tilting upward and the red one tilting downward again, and that's not surprising of course with the return to in-person instruction.

So on the next slide, you see the same data in the table. So in the table on top you can see the student number corresponding to the chart we just looked at. In the last program year 2021 to 2022 there were 187,371 students in the regular classrooms and 67,588 in distance learning classrooms. This is the increase of students in regular classroom and a decrease of students in distance learning over the previous year that is noticeable in the chart, as I just said before.

And that's not surprising given the end of some of the limitations and challenges with respect to in-person program delivery due to the pandemic during the 2020-21 program year. And when in that year when there was a decline in regular classroom enrollment and an increase in distance learning enrollments.

Before the start of the pandemic in 2018-2019 there were 299,720 students in regular classrooms and 10,574 distance learning students reported. Notable is that regular classroom enrolments were only reduced by more than half, and that demonstrates a need for in-person instruction which increased again of course in '21-22. At the same time, distance learning enrollment had increased by more than eight times in 2020 to '21 and that because of the pandemic.

On the chart below, when students were asked-- on the table below-- excuse me. When the students were asked during the last program year if they wanted to continue learning online. 63.1% said they would like to continue and that compared to the 93.9% in the previous year.

Yeah, next slide, please. Thank you. When students were asked how they connected to the internet. Less students, 76.8% said that they used a connection at home than last year when it was 86.9%. In program year 2021-22, more than a third 35.8% use their phone to get online compared to only a quarter 22.6% in the program year 2020-2021.

In the same year 2021-20-- no, in the last year-- sorry. --21-'22 laptops or computers were still the most common choice of devices used for online learning at 64.3%, and that was compared to 76% in the previous year. Alongside cell phones at 64.1% and tablets 19.7%. But there are more than four times more respondents in this last year and that's 4.3%, that you see in the table below there on the right, who said that they did not have a device at all. And the year before there was only 1%.

Next slide, please. Thank you, Anthony. Over one third of students said that they had to share the device with others at home. There were also data limits that kept 19.4% of respondents from learning and 15.7% did not have a quiet place to study this last year. That was compared to 12.6% in the previous year.

Given the limited access to in-person services at adult schools and places with public internet connections during the pandemic, of course, not being able to use the device and connect to the internet at home meant that limited access meant also that was limited access to educational opportunities.

When students were asked if they had ever taken an online class before. 54.4% said that they had, and that's compared to 71% in the previous year. And on the one below, when students were asked about what would help them to study online. 14.1% said that they needed a mobile hotspot to get on the internet. 41.2% said flexible study times. 27.3% said a device to study online.

And 24.8% said that they needed assistance with getting into online textbooks or classes. And 15.1% said that technical troubleshooting would be helpful. These needs increased by between 1 and 3 percentage points compared to the previous program here. So that's going up that need.

Anthony Burik: Matthias?...

Matthias Sturm: Yes.

Anthony Burik: Sorry. Ryan, raised his hand. So, Ryan, are you able to come on?

Matthias Sturm: Please.

Speaker 4: I'm sorry. I could have put it in the chat, if you'd prefer.

Matthias Sturm: No, no, no. That's all right.

Speaker 4: OK. Yeah, I'm just curious what the end is on this survey.

Matthias Sturm: I'm sorry, what's the?

Speaker 4: How many respondents we have.

Matthias Sturm: How many respondents? Oh, this is data. This is-- that's a good question.

Speaker 4: Do you mind?

Matthias Sturm: I can check for that. This is data that we get from Casas.

Speaker 4: Yes.

Matthias Sturm: So this is a pretty large sample.

Speaker 4: I thought so. That's incredible. A quiet place to study is concerning and has me thinking.

Matthias Sturm: Mm-hmm. Absolutely. Absolutely. And that's-- and that's very good points, Ryan. And that's in the discussion groups I think you can together discuss that a little bit more and then also document your discussion that will be very helpful for us to get some input there. I appreciate that. Yes, Anthony, so let's move on to professional development. It's the next slide. Great, thanks.

So teachers were also asked to rate their skills and the importance they placed on various tasks. In the program year '21-22 teachers rated themselves equally strong as in the previous program year with respect to integrating technology into daily instruction at 20%. Acting as a guide for learners when researching on the internet at 19%. And using technology to manage organize their work at 18.9%.

When they were asked about the importance placed on tasks, they rated integrating technology into daily instruction at 70.8%. Using technology to manage and organize my work at 17.3%. And troubleshooting problems that occur when using technology during and for instruction at 17.1%, the highest. So that was the last column that you see here in the slide.

And on the last slide about the areas of technical needs and improvements, what teachers said that they needed. More than half agreed or strongly agreed that they needed more time to learn to use applications that's 50.8-- 58.8% in this last program year and that's compared to 62.4% in the year before. A third indicated that they needed more time to integrate technology into the curriculum, that's 33.2% compared to 36% the year before.

Less than in the previous year said that they needed more options for professional development in the areas of technology. This year at 28.4% compared to 36.2% the previous year. And that they needed more technical support to keep computers and applications running through assigned technical support. This year there was 29.4% compared to 33.1%, and that's combined the strongly agree or the agree.

OK. So, yes, on the next slide, you'll see our discussion groups, and so we would like to-- if possible-- I don't know how many-- how many people are in the room, Anthony?

Anthony Burik: So we have about 10 in the room here. And I think we have about 10, 12 online or so I think.

Matthias Sturm: Yeah, I see 13 that of course counts myself and you probably, so it's about even.

Anthony Burik: OK. Yeah, so what we want to do here is-- we have three Google Docs set up. And if you're in the folder, you should see them there. So we've organized-- again, what we're sharing today is just a very small slice of the data, right? But Matthias and I think that these are pretty-- the topics themselves are pretty heavy, they're pretty weighty. These are probably topics that you're talking about at your agency already.

So the first topic is about blended and distance learning, and so those were the first slides that we looked at. The second topic is digital access and equity, those were the second two slides that we looked at about devices and connecting and things like that. And then the last group-- the last topic is professional development, those are the slides that we just saw that Mathias just showed to us.

So what we wanted to do-- so maybe for about the next 10 or so minutes 10, 15 minutes is we'd like folks to get into smaller groups depending on the topic that you're most interested in. And let me switch to the folder that might be a little easier.

Matthias Sturm: I'm going to create three breakout rooms, OK?

Anthony Burik: Yeah, actually I think we-- Matthias, I already have the room set up.

Matthias Sturm: Oh, I'm sorry.

Anthony Burik: Yeah.

Matthias Sturm: I couldn't-- I couldn't see them, so I guess you have to activate them for me to see them, right?

Anthony Burik: The-- Yes, we need to turn it on.

Matthias Sturm: Yeah. No problem. Thanks.

Anthony Burik: Here we go. This is what we're looking at. So we have-- so again, we have a copy of our sites in there. But then we have the three Google Docs. So let me just open up the first one for you.

Matthias Sturm: And I also put the bit.ly links for the Google Docs in the chat.

Anthony Burik: Thank you, Matthias. So again, you'll see the slides-- you'll see the slides that reference this particular topic on blended and distance learning. And then at the-- so this is our-- sorry. The bottom here, below the slides, this is our prompt for the groups.

So when you take a look at the data, what are your initial observations, anything that stands out to you, any questions that you might have. Any and all of those comments, questions, ideas go ahead and put them in the Google Doc at the very bottom.

So let's come back together as a whole group. We have our online friends back in the room. So what we can do is why don't we go through each of the three groups. We'll start with the first group on the blended and distance and maybe just get some of your comments, things that stood out.

Why don't we start with the-- let's start with the online folks. What were some of the topics that came up for you in your discussion. Nothing? Yeah, that's OK. So online folks who was in-- who was in the blended and distance discussion group?

Speaker 5: Is it a specific worksheet?

Speaker 6: No, I did type it in.

Speaker 4: There were three of us in there, Anthony. Florence was there. And I'm sorry, I missed the name of the gentleman.

Speaker 5: That was me, Corey.

Speaker 4: Corey, yes. OK, great. And we talked about a variety of things, but we just discussed our different options. And I brought back up the concern about having a place to study and work in a quiet environment, a good learning environment.

And we were kicking around. I learned at Pittsburgh, they have 6 hours a day-- Pittsburgh Unified adult-- correct me, if I'm wrong. --6 hours a day where they have open lab time for students available, but they don't offer it at nighttime. They don't have that at nighttime. In Sweetwater we don't have it at all. We don't have any true lab time set up like that. And so yeah, even in our own two institutions there is a lack of access and opportunity.

And we talked about how there's no time at night. And in California, as many of you know, the Californians know, there's typically-- not typically, but often multiple families in one household, which means a lot of kids and a lot of learning happening, and that makes it harder for the adults to learn. So maybe we need to think about providing opportunities at night while night classes are going on for daytime students to come and work in a nice place

Anthony Burik: Thank you, Ryan. Any other topics that came up in that group one, besides the study the quiet place to study?

Speaker 5: Yeah, I'll just say quickly that we talked about that second question there about the what is your feeling about online learning, learning online. And just how grateful I am that we have that data because it's a way for us to actually use data to make our decisions and not intuition or our own biases or our own preferences. So I'm really excited about that data, and I'm looking forward to seeing how that evolves over time.

Anthony Burik: Great. Thank you, Corey. OK. What about our group-- how about our in-person group what things did you talk about in your group on the blended and distance?

Speaker 7: So we talked about-- so some of our schools are different, so before COVID in terms of regular classroom, participation and then distance learning. Some of our schools already had a distance learning program as an option before COVID and some of our schools didn't. So that played into a factor whether the new mandate after COVID to come back to face to face. And that's what people transitioned from Zoom to now mandatory face to face or whether students transition to lesson taught.

People seem to like online distance learning. How can we accommodate our students a little bit more for those who need it? And some of our programs actually did adopt more distance learning and in terms of the signature as before or some of us decided no face to face. So that might be one reason why it seems that in-person classrooms seem to decline from the board.

And also it might be that the population that we're serving and this economy in California, a lot of people are moving out of the state may have to do with economics. It might have to be that more people need to work in jobs to survive. The economy might be why more in-person members have declined also. It could be declining went up a little bit, but it seems to be on the decline from 17 to 279 members. There-- and then anything else you all want to say?

Speaker 8: Yes, we were speculating about different factors that might contribute to that, that chart we were looking at the spikes in declines. And then we were questioning the survey about the feelings about learning online because it looks like it's from an intake survey.

So these are returning students that are maybe filling this out for not the first time because to say that they're continuing to learn online means they've already been doing that versus I don't think I can. So, yeah, we-- I was-- myself anyway, I was wondering how students were thinking about that question and those options and answer choices.

Anthony Burik: But that is a good question too, is who exactly are these students, right? Are they first timers to the adult ed program? Or have they been with us for the last couple of years and are just continuing with their interest in staying online? They had much support in that. Of course, we're looking to improve our programs and supports we provide, so might we change their minds?

Speaker 7: Yeah.

Speaker 9: This is what I'm wondering for the students that said they-- I had to put my glasses on. That they can't learn online right now. I don't think I can learn online. How many of those responded to the, I don't have a quiet place?

Anthony Burik: Right. So, yeah, what are the-- so the question-- I think ultimately the question is where do some of these data points connect, right? So are the people who say-- do the people who say, I don't think I can learn online right now, is it because they don't have the quiet place, they don't have the flexibility, they don't have enough-- they don't have a dedicated device for themselves?

So, right. And Matthias is there-- I don't know. You're the data analyst here. Can we make-- can we make some of those connections? Have we made those in previous reports?

Matthias Sturm: No, no, we haven't. But the questions are they're very good questions. That would be something if we wanted to investigate that in the next report for example, we would have to talk I think to Angela and see.

Right, Angela she's the data analyst wizard at OTAN. And then ask specifically for to get a data filtered for example by those two questions and to make the-- to see where the intersections are. I'm sure she can do that, but we definitely would have to ask specifically for those things.

Anthony Burik: Yeah, so those are good things that we should include, Matthias, in the recommendations.

Matthias Sturm: Absolutely. Absolutely. Yeah.

Anthony Burik: OK. So let's move on to the group two, so are the topic two. So, Penny, and your group I think you-- we didn't have an online group, but why don't you go ahead and let us know what you were talking about in group two.

Speaker 3: Well, we were looking at the second question, and talk about that for a little bit, which device-- oh, on the first slide.

Anthony Burik: OK.

Speaker 3: Which devices do you or can you use for online learning? And we thought it might be helpful to have some more clarifying questions, so we can really understand when students might choose to use one device over another, if they have more than one.

And then also maybe asking the question in a way where we can see which students have only a cell phone. How many students can only use a cell phone? How many students might have a cell phone and a laptop that they can choose from? So changing that question and maybe adding a few more to get some deeper information about that.

Also, we are at same for the fourth option, I don't have a device. For that question, that is probably also going to-- that percentage is probably higher than it actually shows here because students who don't have a device might not be taking the survey and leading it. And I don't know if you want to add to that.

Speaker 10: Yeah, we were talking about the quiet place to study too, and just trying to understand that more on the following slide because it's hard to understand. And I was sure like on our program we tried it pre and post pandemic, and we just had really bad luck with it.

So I was thinking it'd be nice to know what programs these students connected with because like our ESL has zero interest in our school but our high school programs really love it. They love having a study hall. When they come in, even though they can do everything at home. And so we talked a little bit about that. And then we were encouraged that there's a 55% retention rate basically, so they taken our online class before and are returning. So at least that there is some type of consistency with students that you're getting.

Matthias Sturm: That's great. Anthony, the question came up how do-- and this relates to this I think. --how do agencies get their hands on data that is collected through the service that OTAN conduct? So the student technology survey and the teacher-- and the self-assessment, right?

Speaker 10: Right.

Matthias Sturm: And is it possible-- and then my follow up question would be, is it possible for agencies to get the raw data for their agencies, so they can run those queries themselves?

Anthony Burik: Yeah. So in the room here we have Renee Collins who's the director of OTAN who can speak to both of those points. So, Renee.

Speaker 11: Right. So the student technology intake survey is available to our VOA agencies via the Online Application and Reporting site. It's actually I believe it's under Resource or maybe it's on a tab. But that should be available and up to date at adult points for VOA agencies.

If you are a non VOA agency who has been doing the intake technology intake survey, and we do want you to do that. So if your tape agency and not a we OTAN agency, that data can still be shared with you, but you would have to specifically contact OTAN via our support box that we'd be able to probably email that information to you.

As far as the teacher self-assessment, that is going to only come out and be released with every continuous improvement plan which is generally done on an annual basis. But this year, because we were doing the request for application for VOA there is not a CIP, and so the teacher survey will not be released this year. So it will be coming out again next year.

If you do-- if your agency does want to have access to it, you can make a special request for it. But generally speaking it's not going to be made available to everybody. But the student technology intake survey has recently been updated, and as of July 1st, there will be-- you'll notice some differences on it.

The advisory team that originally built the survey came back together within the last month and made recommendations for improvement. So those will go into effect as of July 1st. And so anything July 1st or later should be that new year agency should be utilizing that new survey.

Anthony Burik: OK.

Matthias Sturm: Thanks. Thanks, Renee.

Anthony Burik: Yeah, OK. Why don't we move on to the group three. Group three talked about professional development. And why don't we start with-- I know there was a group online that talked about that. So do any of the folks in the group three want to speak to the topics that you talked about?

Speaker 12: In person or online?

Anthony Burik: Let's start with the online folks. Anybody in group three that was-- talked about professional development online?

Speaker 13: Hi, Anthony, can you hear me OK?

Anthony Burik: Hey, Tanya. Yes, yeah we can.

Speaker 13: Hey. So we talked about in particular the concept of time. So in that professional development slide, that second one on the document. And the concept of time and how-- what does time mean? Does it mean paid time? Does it mean literal time?

Because perhaps they are connected. If people have more paid time, maybe they'll then have more actual time to participate in professional development. So we discussed that a bit. And I think someone else in the group had a comment. And then Matthias joined us and helped us out with trying to figure out that first slide because there was a little bit of confusion there for us.

Anthony Burik: OK. Thank you, Tanya.

Speaker 13: Mm-hmm.

Anthony Burik: How about the-- our... All right, how about-- how about our in-person group, what were you talking about with the PD?

Speaker 14: We had a great discussion about our challenges with PD. And we talked about how the transition can be overwhelming for some people transitioning from different technologies. But also one of the things that we really delved into was the consistent PD offerings like the consistently give those offerings, but also that you have to follow through with the professional development. And it's an agency effort and OTAN and CALPRO can help out with that.

But the important thing is the follow through with those professional development. It has to be admin, teachers, and other staff that participate in it so that it can be successful. And it's not just the tools. You can't just be given the tools, you have to teach how to use those tools. And then follow through with teaching how to use those tools. That was in a nutshell.

Anthony Burik: OK. Great. Great. Any other talk? Oh, no you gave us a nutshell. OK. Super. So this is great stuff. Matthias, do you have any-- I know that you spent more time with the online group, so you want to add any-- add anything?

Matthias Sturm: No, it all came up and the access to the data was one of the main points. So thank you for clarifying that.

Anthony Burik: OK, great.

Matthias Sturm: Yeah, thanks, Anthony, and for doing the in-person groups as well. That works fantastic.

Anthony Burik: Yeah. So if you can-- if you whether you were in-person or online. If you can get your comments onto the Google Docs, that'd be great. As we said Matthias and I are starting to work on the final version of the report, and so we want to make sure to be able to refer to some of these comments and feedback when we put the-- when we put the report together.

Especially the-- so in the report, towards the end of the report, Matthias does a very good job of organizing like so what are some things that we should be looking at that we're not looking at now, right? Or what are further questions about the data that we actually have collected that maybe we need some more information about? So like what Monica was saying, well, it would be good if actually we had a little bit more depth to some of these questions that we're asking, right?

Because on the surface, it seems good information, but actually it's we could use a little bit more clarity and fine tuning of some of those questions. So that's great feedback. That's what we can include in the report, so that we're all aware maybe future directions for data collection when it comes to upcoming reports.

And then again as well, like I said earlier, we're trying to we are trying to make this connection between the data that we're taking a look at and this guidance document. This guidance document is really-- is providing a great framework for these big topics that we're trying to tackle-- trying to tackle at our agencies.

But it's really helpful to connect data with those topics, right? So we have a sense of the where the issues lie, or are things going OK, or where do we need improvement. And then you can also use that when you're looking at your own individual agencies as well. Like how are you comparing to what's going on, on a state level or a larger level? I think we're almost done. Matthias, you want to-- you want to-- any closing thoughts here before we finish up?

Matthias Sturm: Well said, Anthony. Thank you everybody for participating, for your thoughts. I think that's very valuable. Like Anthony said, feel free to still add to those stocks, if you have time. We also have the live transcripts, so I will save that. And enjoy the conference. I'm sorry I can't be there, maybe next year, hopefully.

Anthony Burik: Thank you, Matthias. And as Matthias mentioned, we are working on the final version of the distance learning report. We hope to have that out relatively soon or maybe the next month or so. And then once that comes out, we'll make sure to share it with the field and look for other opportunities to connect with the field on the findings in the report.

All right, I think with that, we will take a break for lunch. And thanks so much, Matthias. And thanks to everyone online for your participation. Thanks to everybody in the room.